use of pen-based electronic diaries in an international ... - PenScreen

Drug Information Journal; Vol. 34, pp. 129-136,2000 Printed in the USA. All rights reserved.

0092-8615/2000-02-29 Copyright © 2000 Drug Information Association Inc.

USE OF PEN-BASED ELECTRONIC DIARIES IN AN INTERNATIONAL CLINICAL TRIAL OF ASTHMA BRIAN TIPLADY AND AUDREY H. JAMIESON AstraZeneca Clinical Research Edinburgh, Edinburgh, United Kingdom

GRAHAM K. CROMPTON Western General Hospital, Edinburgh, United Kingdom The use of pen-based electronic diaries was evaluated in an international clinical trial of asthma. The study compared bambuterol with salmeterol in nocturnal asthma, and was carried out in Italy, Norway and UK. Two hundred and sixty-five patients were enrolled, of whom 135 were randomised and 118 completed the eight week study period. Patients completed the electronic diary at home each morning and evening throughout the 8 week study period. Data could be entered into the diary only within specified time “windows”. No retrospective entry was permitted. Data collection was very satisfactory. Ninety-four percent of patients enrolled completed their diaries during the run-in period to the required standard for inclusion in the study (at least 5 out of the last 7 days of the run-in complete). After randomisation, entries were completed on 86% of scheduled occasions. At the randomisation stage, a review facility was provided for the investigator which gave a summary of the run-in data to indicate whether the patient met the inclusion criteria. This saved work for the investigators, and helped to reduce the rate of incorrect randomisation by comparison with a previous similar study. Data handling was substantially faster than in similar paper-based studies, contributing to locking the database well within schedule. Thus the potential gains seen with this method in earlier evaluation studies have been realised in a full-scale clinical trial. Key Words: Pen-based electronic diaries; Asthma; Clinical trial; Electronic data capture.

procedures through the elimination of manual data editing and entry (1-4). A number of different types of device have been used for this purpose, including handheld organisers with keyboards (e.g. Psion®); touch screen devices using a pen for input (e.g. Apple MessagePad®); and custom-built electronic diaries (e.g. MiniDoc®). We have been particularly interested in pen-based systems because of the natural interface they provide, particularly for patients who may not be comfortable with computer keyboards.

INTRODUCTION HANDHELD ELECTRONIC DEVICES are increasingly being used to collect diary data from patients in clinical trials. Such electronic data capture (EDC) offers the benefits of improved data reliability and quality by the time-stamping of entries and the prevention of retrospective entries; and can simplify and speed up data handling

Reprint address: Dr Brian Tiplady, AstraZeneca Clinical Research Edinburgh, 10 Logie Mill, Edinburgh, EH7 4HG, UK. EMail: [email protected]

129

130

We have carried out several pilot studies on the use of such devices in clinical trials, which have shown that the method works, that patients like it, and that there can be a real improvement in data quality (4). These evaluations were on a small scale, and designed only to assess the methodology. The next step was clearly to use the method in a full-scale clinical trial, in order to assess whether the gains identified by these initial evaluations are realised in normal practice. The present report deals with the use of a pen-based electronic diary in a comparative study of bambuterol (Bambec®) with salmeterol (Serevent®) in patients suffering from night-time asthma. This study was international, involving centres in the UK, Italy and Norway. The electronic diary was basically similar to that used in the previous evaluation studies in asthma (4), collecting symptoms of asthma, as well as peak flow, on awakening and at bedtime. Peak flow was measured on a Vitalograph® peak flow meter, and then recorded manually in the diary. There were also a number of differences from the previous diaries. Since this was a study focusing on nocturnal asthma, peak flow was measured if the patient woke early due to asthma, and the number of night time awakenings and use of rescue medication were recorded. The peak flow values recorded in the diary were the primary outcome measure in the study. An important aspect of the study was that the inclusion criteria used data recorded in the diaries during the two week run-in period. In particular, the change in peak flow from the night time reading to that for the following morning had to be calculated for each night of the run-in. Clearly, automating this task would be beneficial both in terms of reducing the workload for the clinic staff and in reducing the potential for error. We therefore provided a summary screen presenting the relevant data from the runin, for use at the randomisation visit. We expected that it would be possible to show improvements in protocol compliance as a result of this facility.

Brian Tiplady, Audrey H. Jamieson, and Graham K. Crompton

The clinical results of the study have been presented elsewhere (5), so the present report will focus on the data collection aspects of the study.

STUDY DESIGN This was a parallel group design with a two week run-in period, and a six week active treatment period. The patients visited the clinic on four occasion: 1. At the beginning of the run-in period, 2. At the end of the run-in (randomisation), 3. After 3 weeks of treatment, and 4. After 6 weeks of treatment. They filled in the diary at home every day during both the run-in and treatment periods. To be eligible for randomisation, patients had to meet the following criteria during the run-in period: 1. At least one awakening (either waking during the night or waking earlier than usual) due to asthma. 2. Have a 15% or more fall in overnight peak flow on at least three of the last seven recorded nights (i.e. nights for which both the evening and the following morning readings were both recorded). 3. Have completed the diary on at least five out of the last seven days of the run-in.

THE DIARY APPLICATION The diary was set up on an Apple Newton MessagePad model MP110. The screens used to enter diary data are shown in Fig 1. Patients entered peak flow values by “dragging” the diamond with the pen along the screen to the appropriate point on the scale. They could also tap on the grey scale to move the diamond. The peak flow value appeared in the box above the scale, and could be adjusted by tapping the up and down arrows. To answer Yes/No and severity questions, the patient tapped on the appropriate choice, which would then highlight. The number of times the relief

Electronic Diaries in Asthma

FIGURE 1. Data entry screens from the patient diary. Patients used the pen to select the appropriate choice on the touch-sensitive screen of the diary. Entries could be changed until “OK” was tapped after which the entry was fixed.

131

132


inhaler was used was entered by tapping the up and down arrows to the right of the number box. Definitions of the scale points and instructions for numeric entry were attached to the lid of the device. All entry fields were initially blank, with no default choice selection. Patients had to specifically enter a zero or “None” value. Choices could be changed by the patient in any of the questions until the OK button was tapped, when the choices were recorded. It was not then possible to go back to a previously completed page. If a patient tapped OK without answering all the questions on the screen, a warning message appeared. Data could be entered into the diaries during specific “time windows” as follows: Morning entries (normal waking time): Evening entries (normal bedtime): Night-time awakening:

3 am - 3 pm

3 pm - 3 am Midnight - 10 am (but not after a morning entry)

Once an entry had been made during a particular window, no further entry of that type could be made until the next appropriate window. The diary software recorded the times at which all entries were made When entries were not being made, an information screen was displayed to the patient which indicated whether an entry could be made or not. Examples of such displays are shown in Figure 2. All entries were patient-initiated - no alarm signals were used. As well as making entries, patients could see a graph of their peak flow at any time during the study. When the patient returned to the clinic, the investigator tapped on the “Exit” button, which led to the Review level of the diary. As this level included screens for entry of patient data, password protection was used to prevent the patient from gaining access to this level. The screen used for review at the randomisation visit is shown in Figure 3

A third level of the software, also password protected, was used by the monitor, and included functions such as data transfer, and the checking and setting of the clock/calendar. In addition to the electronic diaries, each study centre was equipped with a second MessagePad which functioned as a Data Collection Utility. This utility application was set up on a MessagePad MP2000, a slightly larger device with more storage capacity than the MP110 used for the diary application. Data were transferred from the individual diaries to the utility each time the patient visited the clinic. Data were then collected by the monitor from the utility at each monitoring visit, and finally transferred to PC at the sponsoring office. Data transfer from MessagePad to MessagePad initially used infrared beaming. However, this proved problematic, and was replaced during the study with a procedure using PCMCIA storage cards. Transfer from MessagePad to PC used a cable serial link. Data were transferred into an ASCII (text) file, which was read into an Oracle database using SQL*Loader. The following procedures were used to assess the performance of the diaries and the associated procedures: • Problem Logs • Hardware accounting • Completeness of data • Times of entries • Correctness of randomisation • Investigator questionnaire • Time to file locking

OUTCOMES Two hundred and sixty-five patients were enrolled, 141 females and 124 males, aged 17-85 years (mean 43.3). Of these, 135 were randomised, and 118 completed the study. The reasons for the nonrandomisation of patients are shown in Table 1. Of the 119 patients who were not randomised because they did not meet the criteria, only 17 (6%) did not meet the completeness criterion (five out of the last seven days completed). For the patients


FIGURE 2. Information screens from the patient diary, When no entry was due, the information panel was surrounded by a dotted line. When an entry could be made, this changed to a solid, rounded form.

FIGURE 3. An example of the display provided for the investigator at the randomisation visit.

133

134


patient withdrew because she “did not wish to continue”, and the investigator reported that she did not like the electronic diary Of the 135 randomised patients, 10 (7%) were incorrectly entered into the study. The diary design included a “Randomise Anyway” function, which allowed the investigator to override the exclusion of a patient. This was included for two reasons: 1. The possibility of a protocol change during the study, which might have necessitated recall of the MessagePads to make changes, and 2. The possibility of the patient giving the investigator information in addition to that in the diary. In retrospect, this provision was probably inappropriate, but even so the incorrect randomisation which occurred here was much less than that found in a previous similar study (6) , so improved protocol compliance has indeed been achieved. The times at which diary entries were made are shown in Figure 5. The time windows used here were deliberately set very wide, and while the great majority of entries were made at sensible times a small number occurred at times such as 2 pm for a morning entry or 4pm for an evening entry. If considered a problem, this could be dealt

TABLE 1 Reasons for exclusion of patients from the study at the randomisation visit Reason for non-randomisation

Number of patients

Did not fulfil diary randomisation criteria Asthma exacerbation Did not wish to continue Adverse event Stopped inhaled glucocorticosteroid (IGCS) in error Upper respiratory track infection at Visit 1

119 7 1 1 1

1

who were randomised, data completeness over the subsequent six weeks of the study averaged 86% (Figure 4). It can be seen from these results that patients used the electronic diaries as intended. The level of data completeness after randomisation was highly acceptable, and comparable with that obtained in previous studies using EDC (4). The number of patients excluded because of inadequate diary completeness during the run-in (6%) was not excessive. Excluding such patients at an early stage is important, particularly where diary data are the primary outcome measure. In addition, one 100

80

N of Patients

Overall Mean = 86%

60

40

20

0 0-10

11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100

% Complete

FIGURE 4. Completeness of data in the patient diaries after the randomisation visit.


135

1200 1000 800 N of Entries 600 400 200 0 Evening Morning Awakening 0

3

6

9

12

15

18

21

24

Entry Time (h)

FIGURE 5. Times at which data entries were made into the patient diaries.

with in several ways. The first is to use narrower time windows. This is straightforward to implement, but may cause frustration to patients if an entry is only just outwith the specified interval, particularly if intervals are set to be rather narrow. A second approach is to use wide windows, but to have narrower target windows within them. Any entry within the wide window but not the target window would generate a warning and a reminder of the desired entry times. A third approach would be to use wide windows as here, but to exclude from the data analysis entries that are too far from target times. While the additional control over entry times given by the first option may seem attractive, it should be borne in mind that even the least restrictive approach is a great improvement over paper, where there is no information at all concerning entry times, nor any control over them. A number of problems were reported during the study, and the commonest of these are shown in Table 2. The most important of these related to data transfer. Initially infrared beaming was used to move data from one MessagePad to another. This was not as reliable in the field as we expected, and excessive battery drain

occurred. We changed the method to that used previously in the evaluation studies, using a data storage card, and this solved the problem. No problems occurred due to the repeated insertion and removal of the PCMCIA cards used in the study. TABLE 2 Problems with electronic diaries reported during the study Problem

N

Batteries failed/low Investigator unable/forgot to copy data: transferred later or from archive: Data were OK Incorrect ID entered Technical problem, Newton replaced or program re-installed Diary application closed unexpectedly Failure of data transfer (infra-red beaming) No data available at end of run-in Data copied to card but diary says not

15 13

7 9 6 4 3 3

Two devices failed (out of 150 used in the study) and were replaced. Although the types of devices used for EDC are generally reliable, it is always necessary to have a procedure for getting replacements to patients within 2-3 days.

136

Nineteen investigators completed the Investigators’ Survey. Of these 17 said they would like to used EDC again, 18 found the randomisation summary helpful, and 16 found the peak flow graph helpful. The randomisation summary was the feature of the diary most liked in general by the investigators. A few patients found the screen a bit difficult to see, and this may have been a problem for a few elderly patients in particular. However, as far as we are aware, no data were lost because of this, and substantial numbers of elderly patients have successfully used these diaries in this and other studies. Since the completion of this study, the Apple MessagePad has been withdrawn from sale, and other devices must be considered for future work with EDC. Most of the likely replacements for pen-based data capture are somewhat smaller devices, such as Palm-Size PCs using the Windows CE® operating system, and the Palm Pilot®. The smaller size of these devices would be an advantage if diaries need to be carried around by patients during the day (not the case in the present study), but clearly attention must be paid to screen layout and design to ensure adequate readability if devices with smaller screens are to be used as electronic diaries. It may be appropriate, for example, to display a single question on the screen at a time, rather than two or three, as used here (figure 2). This would allow a larger display font to be used. Another possibility would be to have single or multiple question layouts as options selected by the patient or the investigator. The price of these devices is currently around $300 each. Assuming a useful life of four years, and 50% utilisation gives a cost of $12.50 per patient month. To this must be added additional devices necessary for replacements and for use by the centres, batteries and shipping, so an overall equipment cost in the region of $20-25 per patient month may be expected. The electronic diaries succeeded in greatly simplifying the data handling


process. Data editing was virtually eliminated - the only task still required was to check that the correspondence between diary entries and visit dates was correct., particularly in the few cases where a replacement diary had been issued to a patient. Keying in of data and proofreading, of course, are eliminated completely. Naturally, it took some time for the new processes to become familiar to research staff, since this was the first study using EDC that they had been involved in. Nevertheless the study procedures were completed rapidly, and the time from last patient out to file-locking was much shorter than in other similar studies using paper diaries.

CONCLUSION Pen-based electronic diaries proved to be an effective method of data capture on a production scale, giving the key benefits of improved data reliability, better protocol compliance, and shorter time to filelocking.

REFERENCES 1. Hyland, M.E., Kenyon, C.A.P., Allen, R. and Howarth, P. (1993) Diary keeping in asthma: comparison of written and electronic records. British Medical Journal 306, 487-489. 2. Chowienczyk, P.J., Parkin, D.H., Lawson, C.P. and Cochrane, G.M. (1994) Do asthma patients correctly record home spirometry measurements? British Medical Journal 308, 1618 3. Tiplady, B., Crompton, G.K. and Brackenridge, D. (1995) Electronic diaries for asthma. (Letter) British Medical Journal 310, 1469 4. Tiplady, B., Crompton, G.K., Dewar, M.H., Böllert, F.G.E., Matusiewicz, S.P., Campbell, L.M. and Brackenridge, D. (1997) The use of electronic diaries in respiratory studies. Drug Information Journal 31, 759-764. 5. Crompton, G.K., Ayres, J.G., Basran, G., Schiraldi, G., Brusaco, V., Eivindson, A., Jamieson, A.H. and Olsson H (1999) Comparison of oral bambuterol and inhaled salmeterol in patients with symptomatic asthma and using inhaled corticosteroids. American Journal of Respiratory and Critical Care Medicine 159, 824-828. 6 Wallaert, B., Ostinelli, J. and Arnould, B. (1999) Long-acting 2-agonists: a comparison of oral bambuterol and inhaled salmeterol in asthmatic patients with nocturnal symptoms (abstract). European Respiratory Journal 8 (suppl 19), 1S