IJB-2295;
No. of Pages 9
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 0 6 ) xxx–xxx
journal homepage: www.intl.elsevierhealth.com/journals/ijmi
What factors are associated with the integration of evidence retrieval technology into routine general practice settings? Farah Magrabi ∗ , Johanna I. Westbrook 1 , Enrico W. Coiera 2 Centre for Health Informatics, University of New South Wales, Sydney 2052, Australia
a r t i c l e
i n f o
a b s t r a c t
Article history:
Background: Information retrieval systems have the potential to improve patient care but
Received 28 November 2005
little is known about the variables which influence clinicians’ uptake and use of systems in
Received in revised form
routine work.
24 March 2006
Aim: To determine which factors influenced use of an online evidence retrieval system.
Accepted 28 June 2006
Design of study: Computer logs and pre- and post-system survey analysis of a 4-week clinical trial of the Quick Clinical online evidence system involving 227 general practitioners across Australia.
Keywords:
Results: Online evidence use was not linked to general practice training or clinical expe-
Clinical informatics
rience but female clinicians conducted more searches than their male counterparts
Information retrieval
(mean use = 14.38 searches, S.D. = 11.68 versus mean use = 8.50 searches, S.D. = 9.99; t = 2.67,
Evidence-based medicine
d.f. = 157, P = 0.008). Practice characteristics such as hours worked, type and geographic loca-
Family practice
tion of clinic were not associated with search activity. Information seeking was also not
Evaluation studies
related to participants’ perceived information needs, computer skills, training nor Internet
Internet
connection speed. Clinicians who reported direct improvements in patient care as a result of system use had significantly higher rates of system use than other users (mean use = 12.55 searches, S.D. = 13.18 versus mean use = 8.15 searches, S.D. = 9.18; t = 2.322, d.f. = 154 P = 0.022). Comparison of participants’ views pre- and post- the trial, showed that post-trial clinicians expressed more positive views about searching for information during a consultation (2 = 27.40, d.f. = 4, P ≤ 0.001) and a significantly greater number reported seeking information between consultations as a result of having access to an online evidence system in their consulting rooms (2 = 9.818, d.f. = 2, P = 0.010). Conclusion: Clinicians’ use of an online evidence system was directly related to their reported experiences of improvements in patient care. Post-trial clinicians positively changed their views about having time to search for information and pursued more questions during clinic hours. © 2006 Elsevier Ireland Ltd. All rights reserved.
1.
Introduction
Clinicians are thought to pursue around 30–55% of their questions, resulting in a considerable proportion of unanswered
questions related to patient care [1–4]. In a recent investigation of clinicians’ information needs, not having sufficient time was reported as the third most common reason for not following up a question after uncertainty about the question being
∗
Corresponding author. Tel.: +61 2 9385 9009; fax: +61 2 9385 9006. E-mail addresses:
[email protected] (F. Magrabi),
[email protected] (J.I. Westbrook),
[email protected] (E.W. Coiera). 1 Tel.: +61 2 9385 9015; fax: +61 2 9385 9006. 2 Tel.: +61 2 9385 9026; fax: +61 2 9385 9006. 1386-5056/$ – see front matter © 2006 Elsevier Ireland Ltd. All rights reserved. doi:10.1016/j.ijmedinf.2006.06.009 Please cite this article as: Farah Magrabi et al., What factors are associated with the integration of evidence retrieval technology into routine general practice settings?, International Journal of Medical Informatics (2006), doi:10.1016/j.ijmedinf.2006.06.009.
IJB-2295;
2
No. of Pages 9 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 0 6 ) xxx–xxx
answerable and the availability of a referral [4]. This finding is consistent with previous studies of information needs in clinical settings [1,2,5]. Young and Ward [6] examined the views of general practitioners (GPs) and found that lack of time to search, appraise and discuss the implications of evidence with patients was a significant barrier to practicing evidence-based medicine. Electronic access to clinical evidence at the point of care has the potential to improve information seeking within the time constraints of a busy clinical setting. Having ready access to information means that clinicians can pursue answers to questions as they arise. The potential of online evidence to improve the quality of answers provided by clinicians has been demonstrated in controlled laboratory settings. In a previous study, we found a 14% improvement in GPs’ answers to typical clinical scenarios following use of online evidence [7]. However, there is still a poor understanding of how information retrieval systems are used in clinical settings [8]. Few studies report individual patterns of use and variables which influence clinicians’ uptake of systems and their integration in routine settings have not been extensively examined. We sought to investigate factors which influenced the use of an online evidence retrieval system, called Quick Clinical (QC) in routine general practice settings. QC is based on the generic use of search filters explicitly designed to meet the information needs of specific user groups, and the filters are customisable to meet the varying needs of different groups [9]. Users first select a search filter that matches their question type (e.g. diagnosis, treatment, etc.) and then enter keywords that more specifically describe their query. Five types of search filters or “profiles” specifically designed for GPs were available in this study (disease aetiology, diagnosis, treatment, prescribing and patient education). Up to four different types of keywords could be used in association with a given profile (e.g. disease, drug name). For example, a clinician who encounters a 32-year old woman with a fourth presentation of pelvic pain in the last 6 months with ultrasound and swabs for infection all negative, may have a question regarding the social, psychological as well as biological causes of pelvic pain. The clinician could select the ‘etiology’ profile and enter ‘pelvic pain’, ‘pathology’ and ‘psychosocial’ as keywords (Fig. 1). The search filters retrieved evidence from resources including PubMed, MIMS (a pharmaceutical database), Therapeutic Guidelines, Merck manual and Health Insite (a government funded health database at http://www.healthinsite.gov.au). Users could also search each of these resources individually. Detailed patterns of QC use are reported in a separate paper [10]. During a national trial of the system 193 GPs used the online evidence system to conduct on average 8.7 searches per month. The majority of these searches were conducted from consulting rooms (81.1%) during office hours. The most frequent searches conducted related to diagnosis (37.3%) and treatment (32.1%). Search subjects included a broad spectrum of diseases, including common conditions such as asthma, diabetes and hypertension. Some clinicians (N = 30, 15.5%) used QC for only one search and there was marked individual variation in use of the system (1–74 searches). Based upon evidence from the literature [11] and informed by a program of research undertaken in the use of online evidence retrieval systems [7,10,12–20] we
Fig. 1 – Description of a search: keywords and search filter selection for a clinical question in the Quick Clinical user interface.
hypothesised that the variation in the use of QC could be due to a number of factors such as patient load and casemix; frequency of clinical questions; access to colleagues and other sources of clinical information; lack of time to use the system; lack of experience with online systems and expertise in searching; and difficulty in accessing and using the online evidence system during the trial. For example, it was hypothesised that practice type would influence use of QC, with clinicians working in group practices and medical centres expected to have lower levels of QC use compared to those clinicians working in a solo practice with no access to information from colleagues to answer clinical questions. In this paper we report an investigation of factors which may have influenced clinicians’ use of the online evidence system. Factors examined include clinicians’ genders, hours worked weekly, country of training, general practice experience, practice type and geographic distribution, clinical information needs, computer skills, training, Internet connection speed, success in answering questions and attitudes to information seeking. Changes in participants’ behaviour and attitudes to information seeking at the end of the study were also examined.
2.
Methods
2.1.
Procedures
Two hundred and twenty seven GPs from across Australia were recruited to participate in a trial of QC via a call for volunteers advertised in journals, newsletters and a clinician list-server. Participants were asked to use the online evidence system in their practice over a 4-week period and complete online pre- and post-trial surveys. QC was available from a standard web browser interface (e.g. Microsoft© Internet Explorer, Netscape® ). Each participant obtained a personal username and password to access QC and completed an online tutorial about how to use the system (Appendix A). Participants were randomly allocated to receive advanced online training
Please cite this article as: Farah Magrabi et al., What factors are associated with the integration of evidence retrieval technology into routine general practice settings?, International Journal of Medical Informatics (2006), doi:10.1016/j.ijmedinf.2006.06.009.
IJB-2295;
No. of Pages 9
3
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s x x x ( 2 0 0 6 ) xxx–xxx
which provided additional guidance on how to select appropriate keywords and resources to answer clinical questions. To minimise the possibility of contamination in group practices and medical centres, participants working in the same practice were grouped and the sample was stratified by number of clinicians. Within each stratum practices were then randomly allocated to either the advanced or standard tutorial groups. An online help manual was available and help desk support was provided to all participants during the trial. A pre-trial survey was used to gather demographic data and participants’ information seeking habits. In a post-trial survey clinicians were asked to rate their success in finding information. Respondents were also asked to indicate whether they had direct experience of QC use resulting in improved patient care. Participants’ attitudes to information seeking, computer use during consultations and patient information requirements were assessed in the pre- and post-trial surveys by measuring agreement to 10 statements using a five point scale from strongly disagree to strongly agree. Face validity of the surveys was assessed by asking general practitioners to review a draft version. We subsequently incorporated their comments in the final version. Frequency of system use was determined from automatically generated computer logs used to record details of each search including the user IP (Internet Protocol) address, date, time and duration of searches. Clinicians were asked to identify if the IP address was their home or consulting room. The study was recognised by the Royal Australian College of General Practitioners (RACGP) for its continuing medical education (CME) program. Education points were not directly linked to the number of searches performed, but to trial completion. Ethics approval for the trial was received from the
University of New South Wales’s Human Research Ethics Committee.
2.2.
Analysis
The geographic distribution of participants was determined by relating the rural and remote area classification (RRMA) [21] to the post-code of each GP practice. The seven RRMA categories were grouped into two strata, metropolitan and rural and remote. GPs’ pre-disposition to online evidence use was determined by averaging responses to pre-trial attitudinal questions related to information seeking and computer use during consultations. Statistical analysis of the computer log and survey data was undertaken using SPSS v11.5.0 (SPSS Inc.). Chi-square analyses, Student’s t-test and one-way analysis of variance (ANOVA) were used to examine individual variation in QC use. Duncan’s range test was applied to determine which groups within a significant ANOVA differed significantly from each other. The P values from comparison of participants’ responses to the 10 pre- and post-trial attitudinal statements were adjusted using Bonferroni correction. Factors associated with the use of online evidence were confirmed by using multiple linear regression, with the number of searches as the dependent variable, forward entry of the independent variables, and a probability of entry of 0.05.
3.
Results
A total of 227 GPs registered to participate in the trial. Eighteen percent were female (N = 41) and 73.5% (N = 167) were aged
Table 1 – Demographics of participants in QC trial QC (N = 227) (%)
GPs (N = 2479)a (%)
Gender (2 = 15.12, d.f. = 1, P ≤ 0.001) Male Female
81.9 18.1
69.7 30.3
Age (2 = 8.71, d.f. = 3, P ≤ 0.05)
