Ó The Author 2008. Published by Oxford University Press. All rights reserved. For permissions, please e-mail:
[email protected].
doi:10.1093/fampra/cmn085
Family Practice Advance Access published on 14 November 2008
UK research staff perspectives on improving recruitment and retention to primary care research; nominal group exercise Jonathan Graffya, Julie Granta, Sue Boasea, Elaine Wardb, Paul Wallaceb, c, Julia Millerd and Ann Louise Kinmontha Graffy J, Grant J, Boase S, Ward E, Wallace P, Miller J and Kinmonth AL. UK research staff perspectives on improving recruitment and retention to primary care research; nominal group exercise. Family Practice 2009; 26: 48–55. Background. Primary care studies often encounter recruitment difficulties, but there is little evidence to inform solutions. As part of a National Institute for Health Research School for Primary Care Research and UK Clinical Research Network programme, we elicited research staff perspectives on factors facilitating or obstructing recruitment. Objective. To identify factors that experienced research staff consider important in successful recruitment and retention and their confidence in achieving them. Methods. An iterative series of three workshops was held. The third used a modified nominal group technique to categorize whether factors related to the ‘context’ in which the research took place, the ‘content’ of the study or the recruitment ‘process’ and to prioritize them by their importance to success. Results. Eighteen research staff participated in the prioritization workshop. They prioritized positive attitudes of primary care staff towards research and trust of researchers by potential participants as major contextual factors affecting recruitment. Studies needed to be considered safe and relevant by staff and fit with practice systems. They proposed that researchers strengthen relationships with staff and participants and minimize workload for primary care teams. Although confident in many recruitment processes, respondents remained uncertain how to achieve cultural change so that research became part of normal practice activity and how best to motivate patients to participate. Conclusions. Research workers taking part identified factors which might be important in recruitment, several of which they expressed little confidence in addressing. Understanding how to improve recruitment is crucial if current efforts to strengthen primary care research are to bear fruit. Keywords. Attitude of health personnel, health services research, patient selection, primary health care, randomized-controlled trials. concern about disparities between the age, gender, ethnicity and co-morbidity profile of trial participants and that of the wider population affected by a condition, who are often seen in primary care.2 As a result, successive UK health policy initiatives have emphasized the importance of conducting more research in
Introduction In most health systems, the majority of patient contacts take place in primary care, but much of the evidence used in these consultations is derived from studies conducted in hospital settings.1 This has led to
Received 30 April 2008; Revised 01 October 2008; Accepted 8 October 2008. a General Practice and Primary Care Research Unit, University of Cambridge, bUnited Kingdom Clinical Research Network, Stephenson House, 158-160 North Gower Street, London NW1 2ND, UK, cDepartment of Primary Care and Population Sciences, University College London, Holborn Union Building, Witthington Campus, Highgate Hill, London N19 5LW, UK and dNIHR School for Primary Care Research, University of Manchester, Williamson Building, Oxford Road, Manchester M13 9PL, UK. Correspondence to Jonathan Graffy, General Practice and Primary Care Research Unit, Institute of Public Health, University of Cambridge, Forvie Site, Robinson Way, Cambridge CB2 0SR, UK; Email:
[email protected].
48
Researcher perspectives on improving recruitment
primary care settings. However, there are challenges to achieving this and a number of trials have encountered difficulties recruiting the numbers they set out to include.3 Problems with recruitment and retention of both primary care sites and study participants can delay studies, increase research costs and threaten the validity of the findings.4 Recruitment difficulties have sometimes led to trials being abandoned, both in primary care and more widely.5–7 A review of 114 UK trials in all health care contexts found that only 31% met their recruitment target and 45% recruited fewer than 80% of their target. Fifty four per cent of trials required an extension.4 In primary care, a smaller survey found similar problems, with approximately one-third of trials recruited to time table.3 Specifically, recruitment strategies which required GPs to gain patient consent often encountered difficulties. Responses to recruitment problems included extending the recruitment period (56%); increasing the number of sites (44%); seeking additional funds (31%); recalculating power (21%); introducing other recruitment methods (18%) and finishing with insufficient patients (18%). While some of the problems encountered reflect inappropriate optimism on the part of research teams, it is clear that current recruitment rates fall short of expectations. Despite the wealth of experience among the academic and pharmaceutical industry research communities, the evidence on factors promoting success is patchy and most published reports are based on case studies from individual trials, rather than prospective comparisons of different recruitment methods.8,9 Two approaches which have, however, been investigated are the use of financial incentives, which appeared to help in some situations,10 and adopting ‘opt out’, rather than ‘opt in’ recruitment strategies. When participants to an observational study of angina prognosis were invited to opt in, 38% did so, but when contacted repeatedly unless they signalled unwillingness to participate, 50% agreed to take part.11 While this may be acceptable in situations where participation carries minimal risk, ethics committees normally require an opt in strategy. As part of the UK Clinical Research Network (UKCRN), research infrastructure has been developed to engage those providing health services in recruiting to studies. In England, eight local networks form the Primary Care Research Network (PCRN), while parallel initiatives operate in Scotland, Wales and Northern Ireland.12 A National Institute for Health Research (NIHR) School for Primary Care Research was established in 2006, bringing together five highly rated English academic departments of general practice and primary care to conduct research at each stage in the patient pathway (from prevention to the management of long-term conditions).13 In view
49
of the difficulties that have been documented in primary care recruitment and the limited evidence on how to address these, the NIHR School and PCRN have established a research collaboration to examine recruitment and retention strategies in UK primary care. Preliminary work includes a literature review, the development of web-based study platforms14 and a series of workshops to draw on the expertise of those in the field. This report is based on the findings from the workshops.
Methods As part of this joint programme, three workshops were held during 2007, following an iterative approach to initially elicit and then prioritize the factors that experts believed were important for effective recruitment and retention. The programme group met regularly to plan and consider the outcomes from each workshop and findings were shared with participants through an electronic mailing list. Participants included clinical research staff, clinicians, academics and individuals with research leadership roles. The first was a nationally promoted UKCRN training day, which included presentations based on reviews of the literature and a range of approaches to recruitment and retention. (UKCRN Training Course: Recruiting and sustaining trials in a primary care setting, March 14, 2007, Cambridge.). A second workshop was held at the Society for Academic Primary Care (SAPC) conference and considered areas of uncertainty and what further methodological research might be needed on recruitment and retention. (Recruitment and retention of patients in clinical studies in primary care: developing the research agenda, July 6, 2007, SAPC Annual Conference, London.) Participants included conference delegates and others who attended particularly for this topic. After introductory presentations, they were divided into three focus groups to consider the elements that contributed to successful recruitment and retention and to identify topics for further exploration. A third workshop, reported in detail here, was held at the joint UK Federation of Primary Care Research Organisations and PCRN conference. Participants were conference attendees who opted to join in and most were actively involved in recruitment. During the workshop, they were asked to prioritize the factors identified in terms of their importance to success in recruitment and retention and to rate their confidence in addressing them. A modified Nominal Group Technique was employed, where the structured format enables several people to contribute on a particular problem and the output is a set of issues to address, ranked by their perceived importance.2,15,16 This addresses some of the problems that can arise in more informal groups and prevents the discussion being dominated by
50
Family Practice—an international journal
particular members or ideas. Instead, it encourages more passive group members to participate (Fig. 1). In preparing for the third workshop, reports from the focus groups conducted at the previous workshop were reviewed and a list of the key topics in recruitment and retention was collated. To do this, we used a framework for studying organizational development that had been developed by Pettigrew and Whipp.17 This involved categorizing the topics according to whether they related to the ‘context’ in which research was conducted, the ‘content’ of the research or the ‘process’ by which participants were recruited. Adopting this framework made it easier to consider research recruitment as an activity which needs to be managed in the context of the complex system that is clinical care (Box 1). Groups of five or six participants sat at three separate tables. After explaining the rationale and process for using the modified nominal group technique, we first asked participants to work individually or in pairs to identify factors that they considered to ‘contribute to successful recruitment and/or retention of participants’. They were asked to write each factor on a separate ‘Post-itÒ’ note and to generate as many factors as possible rather than to evaluate the importance of those factors. The next stage involved collating the factors identified. Participants were first asked to individually decide whether the factors they had written down related to the context, the content of the research or the process of recruitment and retention. We then asked them to add their topics to three flip charts, which had been prepared in advance with topics from the focus groups. As they did this, facilitators discussed their suggested factors with participants to establish whether the item was new or belonged to an already established category and also whether the item was correctly assigned to context, content or process. To clarify and summarize at the end of this stage, one Ideas from workshops 1 & 2
of the facilitators then briefly described each factor, checking for consensus on meaning, and that it was correctly assigned to one of the three categories. Factors were regrouped as appropriate. Next, each participant was given a scoring sheet, which listed the factors identified in the previous workshops and included space for them to write in the newly identified ones, using standardized wording. Individuals then evaluated each item without discussion, scoring the statement ‘In successful recruitment and retention, how important is each of the following factors?’ A four-point Likert scale ‘(not important, slightly important, fairly important, very important)’ was used. In addition, they were also asked ‘How sure are you about how to achieve (or address) each of the following factors?’ ‘(not at all sure, slightly sure, fairly sure, very sure)’. During the workshop, participants voted by show of hands for the factors they considered most important, and from these, the ones that they were least confident about addressing. This provided the basis for further discussion, but the data presented here are based on the anonymous score sheets. Following the workshop, the data from these were entered into the statistical package SPSS for further analysis.
Results Eighty-four people attended at least one workshop and 18 took part in the third of these (Table 1). Participants included 4 GPs with research leadership roles, 4 research managers and 10 research coordinators, facilitators and clinical studies officers. The workshop lasted 75 minutes. In the first phase, they identified 13 new topics, in addition to the 30 brought forward from the focus groups at the second workshop. These are shown in Table 2, ranked by participants’ assessments of their importance (scored from 1 to 4), with the new topics marked with ab.
Ideas from delegates
BOX 1
Ideas collated on flip charts
Clarification of meaning
Prioritisation (importance & confidence in addressing topic)
FIGURE 1 Nominal group process employed in prioritization workshop
Key factors identified prior to the nominal group workshop
A. Context: The attitudes towards research of practice staff, patients and the wider public. The way that these three groups make decisions on participation in research projects. The way that practices organize their services and other calls on staff time. B. Content: The nature of the research being proposed. The relevance of the research question. What practice staff are asked to do. C. Process: The way that participants are approached for involvement (techniques include using technology, incentives, building relationships and database screening).
51
Researcher perspectives on improving recruitment TABLE 1 Discipline
Clinical studies officera Research manager Clinical academic Non-clinical academic Total
Workshop 1 (UKCRN training day)
Participants at the three workshops Workshop 2 (SAPC conference)
Workshop 3 (prioritization workshop, UKFed/PCRN conference)
Attended any workshop
10 (31%) 8 (20%) 12 (34%) 5 (14%) 35
10 (56%) 4 (22%) 4c (22%) 0 18
34 (40%) 26 (31%) 18 (21%) 6 (7%) 84
20 (43%) 18 (38%) 7b (4%) 1 (2%) 46
a
Includes research nurses, facilitators and coordinators. Includes two service GPs. c Includes three service GPs. b
The participating research staff rated building and maintaining relationships as the most important feature of successful recruitment and retention (78% rated this as very important), alongside primary care teams having positive attitudes towards research (72% rated this as very important). Most were confident about addressing these issues; 71% (12/17) were very or fairly sure about how to address the former and 62% (12/18) the latter. They were least sure about how to achieve an organizational culture where research is part of normal activity, which only 22% (4/ 18) were very or fairly sure about. Some factors, such as the need to minimize workload and ensure that projects fit with practice systems, are primarily relevant to the decisions that primary care teams make about whether to assist with a study, while others, such as the need to win the trust of potential participants, relate to the decisions that patients make about participation. Following the workshop, the research team classified the factors rated as important (mean score of 3 or more) as relevant at the practice level, the patient level or both (Table 3). Implications of randomization have been added because it was highlighted as particularly important in the literature review and earlier workshops, although participants in the third workshop rated its mean importance below the threshold (2.83). Confidence in addressing the topics identified Workshop participants were asked to record how sure they were about how to address the factors identified. Their responses for the 29 factors that they had rated as most important (mean importance score of 3 or more) are shown in Table 4. Most of these experienced research staff were confident in their ability to address the majority of factors that they saw as important [60% were ‘fairly sure’ (36%) or ‘very sure’ (24%) about this]. However, at the practice level, they were unsure how to achieve an organizational culture in which research was part of normal activity, with practice staff having protected time and research fitting well into practice systems. At
the participant level, they were unsure how to motivate people to participate, whether by developing trust, promoting altruism or finding ways that taking part in research could ethically offer patients better clinical care. Conversations with participants after the workshop and subsequent email responses suggested that the majority found the workshop helpful, stimulating and productive, with only one dissenter. Several would have liked more time for discussion.
Discussion We report here on the factors that a range of people who are experienced in recruiting to primary care research studies believe are important for successful recruitment and retention, and also the things about which they were unsure. Strengths and limitations of the study Over the three workshops, we explored different approaches to recruitment and retention, identified areas of uncertainty and in the third workshop used a modified nominal group technique to better understand what participants saw as most important, or most problematic. This was a small-scale study, and the findings reflect the perspectives and expertise of a limited number of research staff who chose to attend, although across the three workshops we sampled and confirmed opinions of 84 participants. Had we conducted similar exercises with other groups, for example pharmaceutical industry researchers, clinicians who were less active in research or members of the public, different factors might have been identified or emphasized as more important. Nominal group techniques, such as the approach we adopted, have been advocated as a powerful tool to analyse health care problems2,16 and have been used to investigate topics such as the research agenda for intensive care.18 Advantages of this approach are that it promotes equal participation of group members and that distractions inherent in other group methods can be
52
Family Practice—an international journal TABLE 2
Research staff assessments of the importance of the factors identified
In successful recruitment and retention of participants, how important is each of the following?
Mean importance scorea
% of participants scoring item as Not Slightly Fairly Very important important important important
A. Context (ranked by importance): Positive attitudes of primary care teams towards research Trustb Appropriate for participants’ ethnicity Positive attitudes of participants towards research Staff have appropriate training Staff have protected time for research activity An organizational culture where research is part of normal activity Participants’ altruismb Participants’ health statusb Reputation of researchers/institutionb Being part of a group taking part in study Previous experience of taking part in research B. Content: Relevance of research question to practice teams Relevance of research question to potential participants Safety of researchb How well the research fits with practice systems? Underpinned by sound pilot/feasibility work Receive better care/get something backb Research content helps clinicians achieve performance measures Patient and public involvement having an influence on contentb Implications of randomizationb Pattern of presentation of ‘cases’ for research C. Process: Building and maintaining relationships Minimizing workload for primary care teams Providing clear and concise information to primary care teams and potential participants Maintaining contact with participants over duration of study Regular feedback to participants during study Simple protocols Staff given appropriate, study-specific training Facilitators acting as interface between researchers and primary care teams Minimizing workload for participantsb Prompts to keep the study in mind Financial incentives Participants having choice to participateb Continuity of research staff Effective information technology systems Matching study information to the individualb Non-financial incentives Participant recruitment by research staff Recruitment by primary care teams Evaluation by ‘exit interviews’ Recruitment outside primary care settingsb Involving all members of team at primary care site Distribution of scores across all factors (mean %)
6 6 17
6 17 6 11 11 6 22 17 28 39 61 56
22 6 33 39 39 50 33 67 39 44 33 22
72 78 61 50 50 44 44 17 28 17 6
3.67 3.61 3.56 3.39 3.39 3.39 3.22 3.00 2.89 2.78 2.28 2.17
39 44 28 61 22 61 39 39 44 53
61 56 61 39 50 28 33 22 22 6
3.61 3.56 3.50 3.39 3.22 3.17 3.00 2.83 2.83 2.47
6
17 28 22
78 72 72
3.72 3.72 3.67
6 6 6 11 22
28 39 44 39 33
67 56 50 50 44
3.61 3.50 3.44 3.39 3.22
6 12 22 17 22 28 35 33 17 22 28 50 33 19.2
50 59 39 11 56 44 41 50 56 61 33 33 50 39.3
39 29 39 56 22 28 24 17 11 6 22 11
3.22 3.18 3.17 3.06 3.00 3.00 2.88 2.83 2.61 2.61 2.61 2.50 2.33
11
6 6 18
28 11 22 39 28 24 6
6
17
17 11 17 6 17 3.4
38.1
a
Importance scale: ‘(not important = 1, slightly important = 2, fairly important = 3, very important = 4)’. Items newly identified during the prioritization workshop.
b
minimized. For example, by using score sheets, we ensured that voting was anonymous. Disadvantages with nominal groups are that opinions may not converge in the voting process, cross-fertilization of ideas may be constrained and the process may become mechanistic. In our exercise time constraints, and the number of factors to consider, limited discussion of individual
items. The initial phase, when participants added new factors to the three flip charts, was quite pressured but the convenors were satisfied that the group did reach consensus in the subsequent discussion to clarify their meaning (Fig. 1). There was, however, little time for discussion about how research might address the topics which people felt least sure about. In a similar
53
Researcher perspectives on improving recruitment TABLE 3
a
Factors influencing engagement of professionals and recruitment of individual participants Engaging professionals at primary care sites
Recruiting participants
Context in which study is conducted:
Positive attitudes of primary care teams Organizational culture where research is ‘normal’ Staff have protected time
Positive attitudes of primary care teams Participant’s attitudes Trust in practitioners and researchers Altruism Appropriate study for participant’s ethnicity
Content of study:
Safety of the study Underpinned by pilot/feasibility work Implications of randomization Relevance to team Research helps clinicians achieve performance measures ‘Fit’ with practice systems
Safety of the study Underpinned by pilot/feasibility work Implications of randomization Relevance to potential participants Receive better care or other benefit
Process of recruitment:
Building and maintaining relationships Minimize workload for primary care team Financial incentives Provide clear information for professionals Effective information technology systems Facilitator to link with primary care teams Continuity of research staff Simple protocol Prompts to keep the study in mind Training
Building and maintaining relationships Minimize workload for participants Financial incentives Provide clear information for participants Effective information technology systems Participants having choice to participate
a Factors listed are those with mean importance score > 3 in Table 2 (including implications of randomization as discussed in text). Following the third workshop, the research team categorized each as operating at the practitioner or patient level. Two factors which were deemed to relate solely to retention of participants (maintaining contact with participants over duration of study and regular feedback to participants during study) were excluded.
study on responding to unmet need in the elderly, Drennan et al.19 also encountered difficulties in prioritizing potential solutions in the time available. Some factors identified in case reports of studies which encountered difficulty recruiting were not highlighted by this exercise. Perceptions of clinical equipoise and the preferences that both practitioners and potential participants have for particular treatment options can be important influences on whether they agree to recruit to, or take part in randomized trials.5–7 Concerns that trial participants may receive inferior treatments (or treatments that they would not opt for) have been highlighted in trials of localized prostate cancer (surgery versus radiotherapy versus monitoring),20 psychological problems (counselling versus medication)5,21 and in systematic reviews.8 Although this clearly relates to the content of the research, several aspects of which were prioritized in our workshop, the specific factor ‘implications of randomization’ was identified but not seen as particularly important. This may reflect the role of workshop participants in recruiting to rather than designing studies. Also, this is a specific concern for randomized-controlled trials, but the workshop was concerned with recruitment across the full range of study designs. Implications for policy and future research The framework of factors which operate at different stages in the recruitment process (engaging primary care teams and then recruiting individual participants) may be useful to investigators designing projects and
to funding agencies in assessing whether the recruitment strategies proposed in grant applications are realistic. It could guide the content of training programmes for new research staff and the development of a recruitment and retention resource or toolkit. Although derived from work in primary care, much of the content is just as relevant for those conducting hospital-based studies. It is worth noting that although on the face of it the proposed framework appears valid, its utility has not been tested in the design and conduct of actual research studies. This could, however, be assessed if trial management committees were to use this to structure discussions about recruitment and retention strategies. Capturing the relevance of the different components would allow the framework to be evaluated and further refined. When participants rated their confidence in achieving recruitment goals, they may have considered whether there was evidence for how to proceed; however, their responses suggest that they were influenced by their perceptions of whether they had any control over these factors. For example, they expressed more confidence about undertaking recruitment processes than addressing cultural factors which might require a societal or organizational response. A first step in designing a research agenda might explore first the evidence base for the topics that workshop participants viewed as most important and were least sure about. Given that many of the highlighted contextual factors relate to people’s attitudes, there is a case for social marketing and a public engagement initiative to
54
Family Practice—an international journal TABLE 4
How sure research staff felt about how to address the factors rated as important % of participants scoring item asa
In successful recruitment and retention of participants, how sure are you about how to achieve (or address) each of the following factors?
Not at all sure A. Context: An organizational culture where research is part of normal activity Staff have protected time for research activity Participants’ altruismc Trustc Positive attitudes of primary care teams towards research Appropriate for participants’ ethnicity Positive attitudes of participants towards research Staff have appropriate training B. Content: Receive better care/get something backc Implications of randomization How well the research fits with practice systems Research content helps clinicians achieve performance measures Relevance of research question to practice teams Relevance of research question to potential participants Safety of researchc Underpinned by sound pilot/feasibility work C. Process: Continuity of research staff Effective information technology systems Financial incentives Participants having choice to participate Minimizing workload for primary care teams Building and maintaining relationships Staff given appropriate, study-specific training Minimizing workload for participantsc Prompts to keep the study in mind Simple protocols Maintaining contact with participants over duration of study Facilitators acting as interface between researchers and primary care teams Providing clear and concise information to primary care teams and potential participants Regular feedback to participants during study Distribution of scores across all factors (mean %)
Slightly sure
Fairly sure
Mean uncertainty scoreb
Very sure
28
50
11
11
2.94
35 29 24 22 6
35 41 35 17 47 50 28
18 18 18 56 35 33 28
12 12 24 6 12 17 44
2.94 2.88 2.59 2.56 2.47 2.33 1.83
18 18 12 12 12 6
29 29 53 29 18 35 12 12
47 35 12 47 53 24 59 35
6 18 24 12 18 35 29 47
2.59 2.47 2.53 2.41 2.24 2.12 1.82 1.76
24 41 24 24 35 29 35 24 13 24 24 24
24 18 41 35 41 53 35 41 44 53 47 29
18 18 12 24 18 18 29 29 31 24 29 47
2.76 2.71 2.59 2.35 2.29 2.12 2.06 2.06 2.06 2.00 1.94 1.76
6
53
41
1.65
41 36.1
47 23.8
1.65
6 35 24 24 18 6
6 13
11.5
12 28.5
a
Based on 17 respondents because one participant did not complete the uncertainty ratings. Uncertainty scale: ‘(very sure = 1, fairly sure = 2, slightly sure = 3, not at all sure = 4)’. c Items newly identified during the prioritization workshop. b
promote research participation (as is done for blood and organ donation).22 Those developing new studies may also find the marketing approach advocated by the authors of a recent report for the Health Technology Assessment programme of interest.9 Participating research staff identified a need to facilitate cultural change within primary care, including funding so that practice staff have protected time to work on studies. Whereas participation in commercially funded research may be directly reimbursed, in the UK, funding for time spent on recruitment to publicly endorsed studies within the UKCRN research portfolio will come via the Comprehensive Clinical Research Networks. The way that this new support infrastructure operates will have a key role in
determining whether research participation becomes routine in UK primary care. This new infrastructure also provides an opportunity to prospectively investigate which approaches to recruitment appear most effective. While observational studies to compare recruitment rates to studies adopting different approaches are unlikely to be problematic, further feasibility work would be needed before alternative approaches were tested in a randomized design. This could compare initiatives to enhance recruitment at some centres within a trial, or between trials. Despite high levels of investment in health research, many studies fail to recruit their target numbers. The views of these research staff, distilled through the workshops, identify a number of avenues that research
Researcher perspectives on improving recruitment
funders, principal investigators and service providers can follow so that research participation is increasingly seen as routine and a positive contribution to health care.
7
8
Acknowledgements 9
We are grateful to workshop participants, to Helen Morris and Liz Young for work on the earlier workshops, to Toby Prevost for statistical advice and to Peter Bower for comment on the manuscript.
Declaration Funding: East of England Research and Development Support Unit. Members of the NIHR Primary Care Research Recruitment Methodology Group also received resources to carry out this study through Primary Care Research Network and the NIHR School for Primary Care Research. Ethical approval: None. Conflicts of interest: None.
10
11
12
13
14
15
16
References 1
2
3
4
5
6
Medical Research Council. MRC Topic Review; Primary Care Research. London: MRC, 1997. Van de Ven AH, Delbecq AL. The nominal group as a research instrument for exploratory health studies. Am J Public Health 1972; 62: 337–342. Bower P, Wilson S, Mathers N. Short report: how often do UK primary care trials face recruitment delays? Fam Pract 2007; 24: 601–603. McDonald AM, Knight RC, Campbell MK et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials 2006; 7: 9. Fairhurst K, Dowrick C. Problems with recruitment in a randomized controlled trial of counselling in general practice: causes and implications. J Health Serv Res Policy 1996; 1: 77–80. Rendell JM, Licht RW. Under-recruitment of patients for clinical trials: an illustrative example of a failed study. Acta Psychiatr Scand 2007; 115: 337–339.
17
18
19
20
21
22
55
Hetherton J, Matheson A, Robson M. Recruitment by GPs during consultations in a primary care randomized controlled trial comparing computerized psychological therapy with clinical psychology and routine GP care: problems and possible solutions. Prim Health Care Res Dev 2006; 5: 5–10. Ross S, Grant A, Counsell C, Gillespie W, Russell I, Prescott R. Barriers to participation in randomised controlled trials: a systematic review. J Clin Epidemiol 1999; 52: 1143–1156. Campbell MK, Snowdon C, Francis D et al. Recruitment to randomised trials: strategies for trial enrolment and participation study. The STEPS study. Health Technol Assess 2007; 11: 48, 1–126. Bryant J, Powell J. Payment to healthcare professionals for patient recruitment to trials: a systematic review. Br Med J 2005; 331: 1377–1378. Junghans C, Feder G, Hemingway H, Timmis A, Jones M. Recruiting patients to medical research: double blind randomised trial of ‘‘opt-in’’ versus ‘‘opt-out’’ strategies. Br Med J 2005; 331: 940. Sullivan F, Butler C, Cupples M, Kinmonth AL. Primary care research networks in the United Kingdom. Br Med J 2007; 334: 1093–1094. National Institute for Health Research. Transforming Health Research: The First Two Years. London: Department of Health, 2008. Peterson K. Practice-based primary care research—translating research into practice through advanced technology. Fam Pract 2006; 23: 149–150. Delbecq AL, VandeVen AH. A group process for problem identification and programme planning. J Appl Behav Sci 1971; 7: 466–491. Gallagher M, Hares T, Spencer J, Bradshaw C, Webb I. The nominal group technique: a research tool for general practice? Fam Pract 1993; 10: 76–81. Pettigrew A, Whipp R. Managing Change for Competitive Success (ESRC Competitiveness Surveys). Oxford: Blackwell, 1991. Vella K, Goldfrad C, Rowan K, Bion J, Black N. Use of consensus development to establish national research priorities in critical care. Br Med J 2000; 320: 976–980. Drennan V, Walters K, Lenihan P et al. Priorities in identifying unmet need in older people attending general practice: a nominal group technique study. Fam Pract 2007; 24: 454–460. Mills N, Donovan JL, Smith M, Jacoby A, Neal DE, Hamdy FC. Perceptions of equipoise are crucial to trial participation: a qualitative study of men in the ProtecT study. Control Clin Trials 2003; 24: 272–282. Hunt CJ, Shepherd LM, Andrews G. Do doctors know best? Comments on a failed trial. Med J Aust 2001; 174: 144–146. Buckley B. The need for wider public understanding of health care research. Prim Health Care Res Dev 2008; 9: 3–6.
