support nurses. Patients. To assess the reproducibilities of the computer and paper versions of the SF-36, over a 6-month period. 51 patients w31 vasculitis and ...
Rheumatology 2002;41:268–273
Computerized information-gathering in specialist rheumatology clinics: an initial evaluation of an electronic version of the Short Form 36 A. S. Wilson, G. D. Kitas2, D. M. Carruthers, C. Reay, J. Skan, S. Harris1, G. J. Treharne, S. P. Young and P. A. Bacon Department of Rheumatology, Division of Immunity and Infection and 1 CRC Clinical Trials Unit, University of Birmingham, Edgbaston, Birmingham B15 2TT and 2Department of Rheumatology, Dudley Group of Hospitals NHS Trust, The Guest Hospital, Tipton Road, Dudley, West Midlands DY1 4SE, UK Abstract Objectives. Longitudinal outcome data are important for research and are becoming part of routine clinical practice. We assessed an initial version of an electronic Short Form 36 (SF-36), a well-established health assessment questionnaire, in comparison with standard paper forms, in two specialist rheumatology clinics. Methods. Out-patients (20 with systemic lupus erythematosus and 31 with vasculitis) were randomly selected to complete either paper (n = 29) or electronic and paper SF-36 versions (n = 51) before and after consultation (paper vs paper comparison). Data were evaluated as the response correlation, internal consistency, missing data, patient satisfaction and preference. Results. There were very good correlations in SF-36 responses (P < 0.001) between the paper and electronic forms and the paper and paper forms. Internal reliability coefficients (Cronbach’s a) showed good internal consistency for all reported responses in either computer or paper forms. There were no missing data in the computerized version but 24% of patients failed to answer all of the paper form questions. Ease of use of the computer version was rated highly by 71% of all the respondents, and 69% would prefer to use the computer version in future. Discussion. Computerized data collection is acceptable to patients and feasible in clinical settings. It provides responses that are at least comparable to those to the paper form, improves data capture and is available immediately. KEY WORDS: Rheumatology, Systemic lupus erythematosus, Vasculitis, SF-36, Electronic data capture, Quality of life.
Chronic autoimmune diseases such as vasculitis and systemic lupus erythematosus (SLE), though uncommon, can affect multiple organ systems and lead to increased mortality w1–3x. However, with more effective treatment w3, 4x chronic long-term morbidity and relapse are now the most significant aspects of such diseases w3, 5, 6x. Improved survival with continuing morbidity highlights the need for accurate assessment of patients suffering from these diseases and their response to therapy w7x. We have routinely collected outcome data on patients visiting our clinics. This has entailed using indices such as the Vasculitis Integrated Total Assessment Log (VITAL) w7x and the British Isles Lupus Assessment Group (BILAG) index w8x and the Systemic Lupus
International Collaborating ClinicsuAmerican College of Rheumatology (SLICCuACR) damage index w9x. An important component of these indices is the evaluation of the patient’s own perception of their health. This has been accomplished using instruments such as the Medical Outcome Studies Short Form (SF-36) w10x, a self-assessed multilevel utility for the evaluation of both the physical and mental well-being of a patient w10x. It comprises eight major facets (Physical Function, Role Physical, Bodily Pain, General Health, Vitality, Social Function, Role Emotional and Mental Health) w10x. Assessments such as these are complex and timeconsuming and require data manipulation before they can be used for clinical decision-making. A recent audit of SF-36 data collection on paper forms within our clinics over a 6-yr period showed that 154 patients who were habituated to this had attempted 436 consecutive SF-36s but only 336 (77%) had actually been completed w11x. This was despite repeated explanation of how to fill
Submitted 8 May 2001; revised version accepted 31 August 2001. Correspondence to: A. S. Wilson.
268
ß 2002 British Society for Rheumatology
Computerized SF-36
in the form, why it was important to complete all the questions and why the data were being collected. Successful evaluation of disease status within a clinic relies upon efficient information collection and management and the immediate availability of results. With continual improvement in technology, the development and implementation of electronic information systems aimed at addressing clinical problems is desirable and is becoming more common w12–14x. Over the last 2 yr we have been developing clinical electronic databases designed to aid data collection for use in various specialist rheumatology clinics. Within these systems, we have incorporated an electronic version of the SF-36. This was designed in order to reduce the time taken for the information to pass from the patient to the consulting physician and, hopefully, to increase the compliance in answering the questions posed in the questionnaire. Several studies have already shown that other health assessment questionnaires, used to collect quality of life information from cancer patients, have benefited from computerization w15, 16x. In the present study we evaluated, comparatively, the original paper methods and a newly developed, simple electronic system for SF-36 data capture. We assessed (i) reproducibility by comparing responses to the electronic questionnaire with those given in the paper format; (ii) effectiveness and efficiency by assessing the number of unanswered questions to the individual SF-36 questions, and (iii) acceptability of and preferences for the two systems by asking the patients to complete a preference questionnaire after completion of the two styles of SF-36.
Patients and methods The clinic Two specialist interest rheumatology clinics—vasculitis and systemic lupus erythematosus (SLE)—in the Department of Rheumatology, University of Birmingham, UK, were chosen for this study. These operate once a week and are staffed by up to two consultant rheumatologists, two rheumatology trainees, one research nurse, one research associate and two support nurses. Patients To assess the reproducibilities of the computer and paper versions of the SF-36, over a 6-month period 51 patients w31 vasculitis and 20 SLE patients of mean (S.D.) age 50 (14.7) and 43 (12.0) yr respectivelyx were selected from alternating positions of each clinic list. This allowed sufficient time for all the necessary explanations about the study and the system, and for an auditor to record impartially all the measures described, after the patients’ arrival and before their departure from the clinic. Patients were informed of our intention to conduct the study and what it entailed, including the need to use a computer. Participants were not selected on the basis
269
of their ability to use such technology. All patients invited to participate agreed to do so. Each patient was asked either to complete the computerized version before and the paper version after their clinical consultation or vice versa. Neither medium was promoted over the other. To control whether the beforeuafter approach affected the patients’ responses, 29 patients (all with vasculitis, seven of whom had previously participated in the computer vs paper evaluation) were asked to complete paper SF-36 forms both before and after their clinical consultation. The system The electronic interface was designed using a simple computer interface of an Access 97 form (Fig. 1) programmed with Visual Basic for Applications (Microsoft), the data being collected in underlying tables. The questions posed on the computer screen were phrased exactly the same as those in the paper questionnaire and were presented in the same style (with the same words underlined and the same punctuation). Patients were asked to use a standard mouse as a means of navigating through the menus and questions. The design of the interface (Fig. 1) was such that the text was presented in large, easily read type and the buttons were large and easily navigable. After each question, the next question was automatically tabulated but with the option to return to the previous question, the answer being highlighted. Although the application was designed to encourage the user to complete all the questions, the programme could be closed down by the user at any point. On completion, the programme gave the user the option to change any of their answers should they wish to do so. As this study involved a change to the normal routine of SF-36 data collection in our clinics, we used unique identifiers for the patient. Each patient was assessed individually in a private room off the main patient reception area, thus maintaining patient confidentiality throughout the study and also reducing disruption to the normal functions of the clinic. Analysis Reproducibility. We compared the answers reported on the paper form vs the electronic version using Pearson’s correlation coefficient. The internal reliability of the SF-36 scale scores was measured using Cronbach’s a. Where this was not appropriate (i.e. for facets comprising the two items Bodily Pain and Social Function), Spearman’s r correlation coefficient was calculated. Compliance. As unanswered questions are one problem encountered when using the SF-36, we recorded the total number of missing answers on both the paper and the electronic form. SF-36 composite scores were not corrected for missing data; if data were not present, the score for that facet was not calculated and was omitted from the analyses.
270
A. S. Wilson et al.
FIG. 1. Screen showing graphical user interface for an electronic version of the SF-36.
Patient’s preference. Patient’s preferences for the two versions of SF-36 were recorded. This included their opinions on the ease of use and clarity of the layout, both of which were determined using a five-point scale score (1 = excellent, 5 = very poor). The results were analysed with the Mann–Whitney U-test. We also asked the patients which system they liked using and which they would like to see adopted for future use.
Results Reproducibility There was an excellent correlation between before and after (consultation) responses for all SF-36 facets, ranging from 0.67 (Bodily Pain) to 0.87 (Role Physical) (P < 0.001). This suggests that the beforeuafter design did not affect the patients’ responses. There was very little difference between the responses given to any of the facets of the SF-36 by all patients when they used either the electronic system or the standard paper questionnaire (Table 1). Correlations ranged from r = 0.80 (Social Function) to r = 0.96 (Physical Function) for the whole patient group. These were all highly significant (P < 0.001). Responses of patients from the individual clinics were very similar. In the SLE clinic, correlations ranged from r = 0.75 (Role Emotional) to r = 0.99 (Physical Function), whereas in the vasculitis clinic they ranged
from r = 0.77 (Social Function) to 0.95 (Physical Function) (P < 0.001). Internal consistency coefficients (Cronbach’s a and Spearman’s r) showed that all reported responses had good to excellent internal consistency for both the computer and the paper form, which were very similar to those previously reported for each SF-36 dimension. Cronbach’s a ranged from 0.83 to 0.94 (paper version) and from 0.84 to 0.95 (computer version). Spearman’s r correlation coefficients ranged from 0.68 to 0.97. All were highly significant (P < 0.01). There was no difference in the degree of concordance between the reproducibility of responses given to the electronic and paper versions of the SF-36 when we analysed the responses as two groups (younger than the mean age of 47 yr and older than this age). Compliance A review of the completeness of the SF-36 questionnaires showed that there was 100% compliance using the electronic version of the SF-36. This was not the case with the paper questionnaire: 24% of responders w10% in SLE clinic, 36% in the vasculitis clinic; x2(1) = 4.16, P < 0.05x failed to answer all of the SF-36 questions. Patient preference Of the 51 patients tested on the computer, all were very keen to complete the task; no-one refused to finish the computer test. Sixty-seven per cent of patients preferred
Computerized SF-36
the computerized version to the paper version (SLE group 71%, vasculitis group 64%) (significantly more than the paper version; P < 0.01). When asked their preferences for adoption of these systems in future, 69% of patients requested the computerized version (SLE group 78%, vasculitis group 63%) (again significantly more than the paper version; P < 0.01). The 51 patients who completed the preference questionnaire were classed into two age groups about the mean (47 yr). Twenty one of these (42%) were younger and 29 (58%) were older. When we asked users their preference of medium, we found that significantly more of the younger group preferred the computer (n = 17, 81%) when compared with the older age group (n = 12, 41%; x2(1) = 7.83, P < 0.01). There was a similar response when the patients were asked their preference for future use of these systems. When patients were asked to give a general opinion of why they preferred a particular system, ease of use and speed (71%) were given as reasons why the computer version was favoured. The patients felt that the on-line SF-36 took less time to complete; this was a subjective response as they did not time the exercise themselves. However, a review did indicate that average time to complete the computer form was 5 min, ranging from 4 to 10 min, whereas the patients could take a lot longer to answer the paper forms as they were habituated to answering the paper forms at their own pace whilst they waited for their consultation. Lack of computer literacy (54%) was the main reason given why people did not like using the computerized version. When all patients were asked to rate the individual systems in terms of their ease of use and clarity on a scale score, the computerized system was rated better than the paper version (Table 2).
271
Discussion In order to have a better understanding of disease progression, the effects of therapy and disease management, we and others have developed assessment protocols devised specifically for this purpose w7–9, 17–20x. Until recently this has been hindered by the lack of systems for effective collection and direct recording of information in our specialist rheumatology clinics. For many years, paper questionnaires have been the norm. Where information is simple and requires little posttranslation, this may be adequate. However, in the case of assessments used to monitor rheumatic disease, which would benefit from immediate feedback and the ability to monitor changes against previous responses, this is not feasible solely with paper forms. Gathering data from patients within busy ‘real-life working’ specialist rheumatology clinics is hindered by the practicalities of their day-to-day running. Patients have been habituated to completing the SF-36 health assessment questionnaire whilst they attend the clinics, but they have generally been left to complete the SF-36 in their own time and at their own pace. Collecting the data on paper requires further transcription to computer before the results are available for the clinician; this precludes review of data completion, leading to the loss of data. With the burgeoning developments in information technology, simple, effective yet highly specialized electronic data capture systems can be easily developed and implemented with the intention of improving practices w21, 22x. We have developed a system which allows effective collection of SF-36 data from patients with little or no intervention by the clinical or nursing staff. This circumvents the need to transpose the
TABLE 1. Pearson’s correlation coefficients between paper and electronic versions of SF-36 questionnaire SF-36 facet
Study group
Physical Function
Role Physical
Bodily Pain
General Health
Vitality
Social Function
Role Emotional
Mental Health
0.96
0.95
0.93
0.92
0.89
0.80
0.83
0.86
27 < n < 51. Data are Pearson’s correlation coefficients; P < 0.001 for all correlation coefficients. TABLE 2. Patient opinion about electronic and paper versions of the SF-36 Ease of use Patient group All Vasculitis SLE
Clarity of layout
Computer
Paper
Computer
Paper
1.0 (1.0–3.0)* 1.0 (1.0 – 2.0) 1.5 (1.0 – 3.0)
2.5 (1.0 – 3.0) 2.5 (1.0 – 2.0) 2.5 (1.5 – 3.0)
1.0 (1.0–2.0)** 1.0 (1.0 – 2.5) 1.0 (1.0–2.0)*
2.0 (1.0 – 3.0) 2.0 (1.0 – 3.0) 2.0 (1.5 – 2.0)
Data are median and interquartile range of patient opinion about electronic and paper forms (n = 48).Opinion is scored from 1 = very good to 5 = poor. Mann–Whitney U-test: *P < 0.05; **P < 0.01.
272
A. S. Wilson et al.
information from the paper questionnaires which the patient has previously completed. This facilitates the information-gathering process and reduces the delay in monitoring the changes in patient responses, and the information is available to the clinician immediately. Improvements in the mechanisms for data collection need to be assessed prior to adoption. We have developed a simple electronic graphical user interface for the SF-36, prepared in line with guidelines for such design w23, 24x. Although this was an initial version tested for usability, we tried to incorporate into the design aspects which we thought would be helpful to patients hindered by the limitations of conditions such as SLE and vasculitis. This included large, easy-to-read text, easy navigable interfaces and large, simple-to-use buttons for them to record their responses. To test this system, we invited attendees of two of our specialist rheumatology clinics to try the computer programme. These clinics were chosen in order to evaluate how this system would operate in two distinctly different styles of clinic. Although it might be expected that a change in the style of presentation of the questionnaire (electronic as opposed to paper) could influence the response, this did not seem to be the case. We did not find that there was any significant difference between the responses given by the patient to any of the major components of the SF-36, whichever medium they were asked to complete. Evaluation of the internal reproducibility of each method (Cronbach’s a) showed that, irrespective of the medium used, patients were fairly consistent in their responses to each question and the medium used did not affect their reported scale scores for the SF-36. The cross-over study design attempted to control for any effects that completing the forms before and after consultation or the stay in the clinic may have had upon the responses given. We did not want to expose the patient to either style of SF-36 immediately after the other in order to prevent them from duplicating their answers from one medium to the other. Therefore, we imposed an intervening period during which the patient would visit the clinician. This is not an ideal situation, as consultation may have influenced the SF-36 results. We tried to minimize this effect by alternating the medium that was attempted first and by adopting as the third arm of the study the paper vs paper evaluation: before and after consultation. This latter arm assessed if length of stay in the clinicucontact with the physician affected the patients’ responses to the SF-36 questions. We did not find this to be the case and therefore deemed that it was not necessary to pursue this aspect any further. The introduction of the computer format into our clinics resulted in 100% compliance with completion of all the questions of the SF-36 when compared with the paper collection aspect of the study. These findings are similar to those of Velikova et al. w16x, who reported that their patient cohort achieved full compliance in their answers to their electronic quality-of-life questionnaire. Comparisons of paper (mailshot) and telephone
administration have shown that mailed SF-36s are more likely to be incomplete when compared with a telephone survey, with about 3 : 1 unanswered questions w25x. This could indicate that some degree of personal contact may aid compliance to answering the questionnaires and that self-assessment paper forms are not the most advantageous method of collecting data. The electronic system was well accepted by the patients, a majority favouring the computerized form over paper. The main reason for this was that they believed the former could be completed more quickly. The novelty of the computer system and the presence of the auditor conducting the study may have made it appear to be quicker to fill in the computerized form; however, patients had normally been used to filling in the paper forms at their convenience. This could take anywhere between 5 min and half an hour, depending upon the patient. The distinct advantage of the computer system in terms of speed is the calculation and presentation of the data. Paper forms have to be transposed into relevant computer applications in order to calculate the SF-36 components, and this adds considerably to the time taken to produce a result. The lack of computer literacy was the main reason why people were initially hesitant about an electronic style of questionnaire. This is consistent with findings reported from other studies which had evaluated the acceptability of computerized versions of other healthrelated quality-of-life questionnaires w15x. We did find an age difference in preference for the type of system. Splitting the patients into nearly equal groups around the mean age (47 yr) showed that the younger patients preferred the computerized system whereas the older ones still preferred to use paper forms. The slightly lower mean age of the SLE patients could explain their greater preference for using a computerized system. Despite this, the patient’s age had no bearing on the reproducibility of the responses given to either the paper form or the electronic version. With better design of the computerized SF-36, it may be possible to introduce a system which is acceptable to all patients. Reports have already shown that qualityof-life questionnaires using touch-screen technology are an acceptable approach for presenting information w15, 26, 27x. In the present study, patients used a standard mouse for navigation through our interface. However, as rheumatic diseases can manifest themselves in debilitating joint mobility problems it may be possible to enhance the acceptance of this method of capturing data by using touch-screens. Disease activity scores and patient global assessments are now being accepted as viable tools in clinical research w28x. The development of effective methods of recording this information, as we have with our electronic system, will improve the quality and quantity of data that we collect. Use of computers in a clinical setting is now being seen to be an acceptable activity, patients regarding this as normal and indicative of doctors being up to date w29x. There is little evidence to suggest that this can lead to loss of the personal
Computerized SF-36
touch as long as verbal skills and eye contact are maintained w29x. In order to evaluate whether a computerized SF-36 could be used in larger, more generalized rheumatology clinics, it would be necessary to perform a more in-depth assessment. We have not attempted to do that here but have described one method that has been effective in achieving our goals of collecting quality research data in a busy working clinic. This system has been acceptable to our patients and has allowed the clinician to have a complete patient record, with immediate feedback about the patient’s progress over time.
Acknowledgements The Short Form 36 was used by kind permission of ‘SF-361 Health Survey (Medical Outcomes Trust)ß Medical Outcomes Trust and John E. Ware Jr. All rights reserved’. ASW and CR are funded by an Arthritis Research Campaign (Derbyshire, UK)uARC (UK)uIntegrated Clinical Arthritis Centre Award. PAB is an ARC (UK) Professor of Rheumatology.
References 1. Goodman GC, Churgh J. Wegener’s granulomatosis: pathology and review of literature. Arch Pathol 1954; 58:533–53. 2. Rose GA, Spencer H. Polyarteritis nodosa. Q J Med 1957;26:26–43. 3. Emery P. Systemic lupus erythematosus. In: Butler RC, Jayson MIV, eds. Collected reports on the rheumatic diseases. Arthritis Research Campaign, 1998:87–92. 4. Adu D, Luqmani RA, Bacon PA. Polyarteritis, Wegener’s granulomatosis and Churg–Strauss syndrome. In: Maddison PJ, Isenberg D, Woo P, Glass DN, eds. Oxford textbook of rheumatology. Oxford: Oxford University Press, 1993:846–59. 5. Gordon M, Luqmani RA, Adu D et al. Relapses in patients with a systemic vasculitis. Q J Med 1993; 86:770–89. 6. Hoffman GS, Kerr GS, Leavitt RY et al. Wegener’s granulomatosis: an analysis of 158 patients. Ann Intern Med 1992;116:488–98. 7. Bacon PA, Moots RJ, Exley A, Luqmani R, Rasmussen N. VITAL (Vasculitis Integrated Assessment Log) assessment of vasculitis. Clin Exp Rheumatol 1995;13:275–8. 8. Symmons DP, Coppock JS, Bacon PA et al. Development and assessment of a computerised index of clinical disease activity in systemic lupus erythematosus. Members of the British Isles Lupus Assessment Group (BILAG). Q J Med 1988;69:927–37. 9. Stoll T, Seifert B, Isenberg DA. SLICCuACR Damage Index is valid and renal and pulmonary scores are predictors of severe outcome in patients with systemic lupus erythematosus. Br J Rheumatol 1996;35:248–54.
273
10. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). 1. Conceptual framework and item selection. Med Care 1992;30:473–83. 11. Carruthers DM, Wilson AS, Raza K et al. Computerised data collection for disease assessment in a vasculits clinic. Arthritis Rheum 2000;43:S108. 12. Cesnik B. The future of health informatics. Int J Med Inf 1999;55:83–5. 13. Hendee WR. Informatics at the National Institute of Health: a call to action. J Am Med Inform Assoc 1999;6:267–71. 14. Jastremski CA. Nursing informatics: issues for critical care medicine. Crit Care Clin 1999;15:563–76. 15. Buxton J, White M, Osoba D. Patients’ experiences using a computerized program with a touch-sensitive video monitor for the assessment of health-related quality of life. Qual Life Res 1998;7:513–9. 16. Velikova G, Wright EP, Smith AB et al. Automated collection of quality-of-life data: a comparison of paper and computer touch-screen questionnaires. J Clin Oncol 1999;17:998–1007. 17. Luqmani RA, Bacon PA, Moots RJ et al. Birmingham Vasculitis Activity Score (BVAS) in systemic necrotizing vasculitis. Q J Med 1994;87:671–8. 18. Hay EM, Bacon PA, Gordon C et al. Relapses in patients with systemic vasculitis. Q J Med 1993;86:770–89. 19. Exley AR, Bacon PA, Luqmani RA et al. Development and initial validation of the Vasculitis Damage Index for the standardized clinical assessment of damage in the systemic vasculitides. Arthritis Rheum 1997;40:371–80. 20. Gladman DD, Urowitz MB. The SLICCuACR damage index: progress report and experience in the field. Lupus 1999;8:632–7. 21. Kenney N, MacFarlane A. Identifying problems with data collection at a local level: survey of NHS maternity units in England. Br Med J 1999;319:619–22. 22. Law M, King G, Russell D, Mackinnon E, Hurley P, Murphy C. Measuring outcomes in children’s rehabilitation: a decision protocol. Arch Phys Med Rehabil 1999;80:629–36. 23. Fowler SL, Stanwick VR. The GUI style guide. AP Professional, 1995. 24. Sneiderman B. Designing the user interface: strategies for effective human–computer interaction, 2nd edn. Reading (MA): Addison-Wesley Publishing Company, 1992. 25. McHorney CA, Kosinski M, Ware JE. Comparisons of the costs and quality of norms for the SF36 health survey collected by mail versus telephone interview: results from a national survey. Med Care 1994;32:551–67. 26. Lau LM, Wright SO, Garlick-Longhurst TJ, Graybill CS, Warner HR. Quality assessment and patient participation in care by means of touch-screen computer. Clin Perf Qual Health Care 1996;4:10–3. 27. Carr RL, Snow CD, Youngson RA. Design and implementation of an automated questionnaire. Stud Health Technol Information 1997;46:88–93. 28. Houssein DA, Stucki G, Scott DL. A patient derived disease activity score can substitute for a physician-derived disease activity score in clinical research. Rheumatology 1999;38:48–52. 29. Ridsdale L, Hudd S. Computers in the consultation: the patients view. Br J Gen Pract 1994;44:367–9.
