A secondary analysis of existing data sets and a cross-sectional survey were carried out in Lothian ... quality is not a new endeavour.5â11 During the last decade, the Department of Health ... SciencesâGeneral Practice, 20 West Richmond Street,. Edinburgh EH8 9DX, aImperial College School of Medicine,. Department of ...
Family Practice © Oxford University Press 2002
Vol. 19, No. 1 Printed in Great Britain
The development of a routine NHS data-based index of performance in general practice (NHSPPI) David J Heaney, Jeremy J Walker, John GR Howie, Margaret Maxwell, George K Freemana, Peter NE Berreyb, Tom G Jonesc, Morag C Sternd and Stephen M Campbelle Heaney DJ, Walker JJ, Howie JGR, Maxwell M, Freeman GK, Berrey PNE, Jones TG, Stern MC and Campbell SM. The development of a routine NHS data-based index of performance in general practice (NHSPPI). Family Practice 2002; 19: 77–84. Objectives. The aim of this study was to compare two different approaches to the measurement of quality in general practice: data derived from routine NHS data sets and results from an index derived from patient-collected data. Methods. A secondary analysis of existing data sets and a cross-sectional survey were carried out in Lothian, Coventry, Oxfordshire and west London. The subjects comprised randomly selected and consenting practices, and a sample of patients within these practices. A National Health Service Practice Performance Index (NHSPPI) was constructed from 16 routinely available NHS performance indicators. The Consultation Quality Index (CQI) combines the Patient Enablement Instrument (PEI) with a measure of how well the patient knew the doctor, and with observed consultation length. Results. Scores for 12 of the 16 indicators varied significantly across the four regions. Mean practice NHSPPI score overall was 21.6 (SD 4.3), which varied significantly across regions. NHSPPI was predicted by practice list size, weighted deprivation index and proportion of other language patients in the practice, although their effects could not be separated. Overall there was no correlation between NHSPPI and CQI, although the prescribing component of the index was positively correlated to mean consultation length and negatively correlated with how well patients knew their doctors. Conclusions. Good quality care as assessed by patients on completion of their consultation is independent of good quality care as assessed by best available measures of practice performance. We suggest that the CQI and the NHSPPI are at least as ready for use as other measures of performance in general practice. Keywords. Consultation Quality Index, general practice, performance, Practice Performance Index.
continual improvement in quality of service.3,4 As such, clinical governance sets a considerable challenge to general practice in the UK, although the measurement of quality is not a new endeavour.5–11 During the last decade, the Department of Health made several attempts to utilize routinely available NHS data in a systematic form at district level. The measures of performance have included prescribing rates, referral rates, preventive care targets and measures of administrative efficiency. These initiatives have laid the foundations for the indicators published in the national framework for assessing performance.12 Recent work has assessed the face validity of quality indicators for use in general practice.13 Although a total of 240 separate quality indicators were identified as proposed for use in general practice by health authorities,
Introduction Quality has been placed ‘at the heart’ of the NHS.1,2 The introduction of clinical governance means performance has become a corporate responsibility: the aim is
Received 8 December 2000; Revised 29 May 2001; Accepted 3 September 2001. University of Edinburgh, Department of Community Health Sciences—General Practice, 20 West Richmond Street, Edinburgh EH8 9DX, aImperial College School of Medicine, Department of Primary Health Care and General Practice, Chelsea and Westminster Hospital, 369 Fulham Road, London SW10 9NH, bLothian Primary Care Trust, cOxfordshire Health Authority, dCoventry Health Authority and eNational Primary Care Research and Development Centre, UK.
77
78
Family Practice—an international journal
none related to effective communication, care of acute illnesses, health outcomes or patient evaluation. Available and valid individual routine measures may be important but they cannot be used in isolation to represent the complexity of general practice. The gap between available data and what might be seen as comprehensive assessment was recognized in the development of a framework for assessing quality in general practice.14 This framework divided measures into three categories (common acute problems, continuing chronic problems and asymptomatic problems/health promotion) and also classified by time (before, immediately after, 14 days after and 6–12 months after the consultation). This approach emphasized the patient, the doctor and the consultation rather than the population as the denominators in the construction of measures of outcome. This present paper describes work attempting to compare two different approaches to the measurement of quality. Randomly selected and consenting practices from four different parts of the UK participated in a study investigating quality of care in general practice. Sixteen performance indicators derived from routine NHS data sets were collated for each practice in the study and aggregated into a National Health Service Practice Performance Index (NHSPPI). This paper compares results from this routine data collection with results from an index derived from patient-collected data.
Methods In 1997, our research team approached a set of 140 randomly selected practices in four participating regions (Lothian, Coventry, Oxfordshire and West London) to gain permission to access routinely held NHS data sets. The research team included three health authority medical advisers and they identified a list of routinely available NHS data which were already in use by them or had been suggested as appropriate measures of quality performance in general practice. From their peer group’s experience and publications, a short list of relatively robust prescribing performance indicators was identified. Some prescribing data that are routinely available in England and Wales are unavailable in Scotland, and vice versa: this added a constraint to the range of acceptable markers. To this list was added a short set of routinely available data comprising performance towards remunerated targets, eligibility for postgraduate education allowance, remunerated non-general medical services and levels of remuneration claimed for ancillary staff. The main constraint imposed in the production of ‘routine management data’ was to produce results without investing large amounts of health authority resource in their computation. The exact definition for each of the selected indicators is included in Appendix 1. The medical advisers subsequently worked closely
together to ensure that data collection was consistent across regions. Once the results for each region were available, the research team met to define thresholds of acceptable practice performance for each indicator. The team devised four categories: ‘fully acceptable’ performance, ‘possibly acceptable’ performance, ‘probably unacceptable’ performance and ‘unacceptable’ performance. The thresholds selected for each indicator are presented in Appendix 1. The team assigned values of 2, 1, 0 and –1 to these categories. Two of the original indicators were dropped at this stage: the cost of inhaled steroids (as it was felt this was not a measure of quality) and the number of contraceptive fee claims (although this was a measure of quality, it was felt that achieving it was out with the control of the practice as other agencies provide contraceptive services). For two further indicators (quinolones and appetite suppressants), it was decided that it was too difficult to differentiate between ‘probably unacceptable’ and ‘unacceptable’, and the lowest (–1) category was not used. Data for registration for child health surveillance, minor surgery and maternity care were aggregated into one indicator. The practice scored 2 if any combination of doctors in the practice were registered for all three, scored 1 if registered for any two out of the three, scored 0 if only registered for one of the three and scored –1 if registered for none of these. Responses were aggregated across the basket of 16 indicators to devise a ‘National Health Service Practice Performance Index’ (NHSPPI) for each practice. Thus a practice could score a maximum of 32 if for each measure their performance fell within the defined accepted range. The theoretical minimum (–14) would be scored if the practice failed on each indicator. The component parts of the NHSPPI were defined as follows: a practice prescribing subindex (PPSI items 1–10), a practice preventive subindex (PRSI items 11–13) and a practice external quality subindex (PQSI items 14–16). The above exercise was carried out in parallel with a large consultation-based survey in which 53 of the 140 practices (38%) participated (a condition being the agreement of all partners in small practices and of all except one or two in practices of five or more partners). A total of 26 000 patients attending 221 doctors completed the Patient Enablement Instrument (PEI) 9 after consultations during a 2-week period in March/April 1998. The PEI score was combined with a measure of how well the patient knew the doctor, and with observed consultation length to create a single proxy measure of quality of care at the generality of general practice consultations (Consultation Quality Index; CQI). All doctors with at least 50 qualifying consultations were ranked in descending order separately for their mean enablement score, mean consultation length and the percentage of patients consulting who knew them well or
79
A routine NHS data-based index of performance in general practice
very well using all data available. The ranks were divided into sextiles. Doctors in the top sextile for any variable were allocated a score of 6 points, down to a score of 1 point for a place in the sixth (lowest) sextile. A total CQI score was then calculated for each doctor by adding their scores for all three variables. The scores thus ranged from 3 to 18.15 For the purposes of this paper, the CQI, which we have presented as an index of quality of care at doctor level, was aggregated to practice level and compared with practice score on the NHSPPI to test for correlation between these different approaches to the measurement of quality in general practice. The methods and results from the consultation-based study,16 and the development of the CQI15 have been described more fully elsewhere. The hypothesis that practice demography would predict NHSPPI was tested. A multiple regression model was constructed with overall NHSPPI score as the independent variable. Practice list size, a deprivation rank (calculated using enumeration district level UPA scores), proportion of other language patients in the practice (obtained from the patient questionnaire) and region (i.e. Lothian, Coventry, Oxfordshire and West London) were selected as predictors.
TABLE 1
Results Over the four regions of study, 120/140 of the randomly selected practices gave their permission to be included in the analysis of routine data. Table 1 summarizes the results for each individual performance indicator for all practices, and for those participating and not participating in the consultation study. Only two (item 3 at the 5% level, and item 15 at the 1% level) of the 16 indicators were scored significantly differently between practices which did or did not participate in the consultation study. Table 2 compares the mean results for each indicator in the four regions in the study. Scores for 12 of the 16 indicators varied significantly across the four regions, but relative performance did not always favour the same regions. Some of the differences appeared to be of marginal clinical significance, but four indicators were of particular interest. The number of items per 1000 patients for cough suppressants and nasal decongestants was 159.8 in Coventry and 115.3 in West London, compared with 32.4 in Lothian and 17.4 in Oxford. Average prescribing for cough suppressants and nasal
Minimum, maximum and mean practice values (SD) for each indicator included in the NHSPPI, and mean values (SD) for those practices participating and not participating in the consultation-based survey
Indicator
n
Minimum Maximum
Mean (SD)
Participants Mean n
SD Non-participants Mean SD n
Pa
1. Quinolones
120
0.0
21.0
3.4 (2.6)
52
3.2
1.9
68
3.5
3.0
0.46
2. Bendrofluazide
120
0.0
78.3
26.7 (18.3)
52
24.6
16.1
68
28.4
19.7
0.27
3. Diuretics
120
3.0
80.6
27.7 (14.3)
52
24.6
13.1
68
30.1
14.7
0.04
4. Cough suppressants
120
2.0
875.0
90.6 (117.4)
52
74.1
101.6
68
103.3
127.4
0.18
5. Appetite suppressants
120
0.0
9.0
0.4 (1.2)
52
0.3
0.9
68
0.5
1.4
0.37
6. NSAIDs
120
57.1
99.2
83.8 (7.2)
52
84.7
6.3
68
83.1
7.7
0.24
7. Antibiotics
120
63.0
100.0
89.3 (5.1)
52
89.1
3.6
68
89.4
6.0
0.74
8. Steroid/bronchodilator
120
24.1
70.2
47.3 (9.1)
52
47.7
8.7
68
47.0
9.5
0.66
9. Statins
120
0.0
489.0
87.2 (59.7)
52
92.3
72.3
68
83.3
48.1
0.44
10. HRT
120
0.0
1790.0
898.9 (286.1)
52
903.0
272.3
68
895.7
298.2
0.89
11. Primary immunization
120
75.0
100.0
95.9 (4.2)
52
95.4
4.3
68
96.2
4.1
0.31
12. Pre-school immunization
120
54.2
100.0
91.4 (7.9)
52
90.8
8.0
68
91.8
7.9
0.50
13. Cervical cytology
120
40.0
98.7
81.5 (9.5)
52
82.9
8.0
68
80.5
10.5
0.15
14. PGEAb
118
0.0
100.0
95.4 (12.5)
51
97.0
7.4
67
94.2
15.3
0.21
15. Doctor(s) registered for Child health surveillance/ Minor surgery/ Maternity care
119
–1
2
1.8 (0.6)
51
1.9
0.3
68
1.7
0.7
0.13
16. Staff reimbursement
118
3133
31 982
12 962 (3517)
51
13 554
4185
67
12 511
2862
a For all indicators except number 15, the column headed ‘P’ gives the result from a t-test of the null hypothesis of equal mean values between participating and non-participating practices. For indicator 15, the value shown is the result of a Mann–Whitney test. b Three values in excess of 100% were restricted to 100% in all analyses.
0.13
80
Family Practice—an international journal TABLE 2
Mean practice values (SD) by region for each indicator in the NHSPPI
Indicator
Summary values for area Oxford Mean
1. 4-Quinolones
Lothian SD
Mean
West London SD
Mean
SD
Pa
Coventry Mean
SD
3.7
2.6
4.1
3.9
3.3
2.0
2.9
1.9
0.26
2. Bendrofluazide
22.5
13.8
25.6
14.8
24.9
17.5
32.4
22.8
0.44
3. Diuretics
18.9
10.4
18.2
7.9
33.6
15.4
35.0
11.9
,0.01
4. Cough suppressants/nasal decongestants
17.4
9.5
32.4
40.1
115.3
100.6
159.8
157.8
,0.01
5. Appetite suppressants
0.8
1.9
0.1
0.2
0.4
1.1
0.4
1.1
0.37
6. NSAIDs
87.4
3.4
78.7
4.1
90.0
4.5
78.6
6.6
,0.01
7. Antibiotics
88.2
2.7
90.9
3.7
91.0
4.5
87.3
6.7
,0.01
8. Steroids 9. Statins 10. HRT 11. Primary immunization
49.9
5.3
44.6
8.4
44.9
9.6
49.8
10.4
,0.01
127.7
100.8
89.6
31.9
85.3
43.5
58.6
24.9
,0.01
1049.9
226.6
723.0
135.0
880.0
347.6
935.5
274.4
,0.01
96.5
2.8
97.9
1.3
93.7
6.3
96.2
2.8
,0.01
12. Pre-school immunization
94.1
4.7
93.4
4.8
85.4
11.0
93.9
4.0
,0.01
13. Cervical screening
80.0
11.2
87.0
6.6
75.9
10.5
84.3
4.9
,0.01
95.5
8.4
98.4
4.3
95.0
18.1
93.7
12.1
,0.01
2.0
0.0
1.9
0.4
1.6
0.7
1.7
0.7
,0.01
14.
PGEAb
15. Doctor(s) registered for Child health surveillance/ Minor surgery/ Maternity care 16. Staff costs (£ per 1000 patients) a b
12 765
2214
12 469
5197
13 821
4050
12 568
2032
0.25
The column headed ‘P’ gives the result from a Kruskal–Wallis test (data classified by area). Three values in excess of 100% were restricted to 100% in all analyses.
decongestants in Coventry and West London therefore fell into the ‘probably unacceptable’ range as defined for individual practices by the research team (100 items per 1000 patients per annum or above). Similarly, average prescribing in Coventry for the ratio of bendrofluazide 5 mg tablets to all bendrofluazide items and for the ratio of compound diuretics plus diuretics with potassium to all diuretic items fell into the ‘probably unacceptable’ category. Finally, Oxford practices prescribed double the number of statins per 1000 patients compared with Coventry practices (the Oxford average falling into our ‘acceptable’ category whereas the Coventry average was only ‘possibly acceptable’). Table 3 shows mean practice NHSPPI scores across the four regions in the study. The mean score overall was 21.6 (SD 4.3), the median score was 22 and the modal score was 24. The ‘best’ performing practice (score 31) had acceptable scores for all individual indicators except one. The ‘worst’ performing practice (score 11) had five ‘acceptable’ scores, three ‘possibly acceptable’ scores, six ‘probably unacceptable’ scores and two ‘unacceptable’ scores. There was a statistically significant difference in mean NHSPPI across regions. The mean score in Oxford
practices was 24.8 compared with 19.9 in Coventry practices (P , 0.001). When two indicators were removed (cough suppressants and diuretics) from the total scores, this reduced the differences although they remained significant (P = 0.03). The rank correlations between list size, deprivation and proportion of other language patients in the practice were strong, as shown in Table 4. The overall regression was highly significant (P = 0.003), and the proportion of variance explained (adjusted R2) was 0.27. Yet, reference to the parameter estimates showed that none of the predictors were significant at the conventional 5% level (except region = Oxford). These variables and their effects cannot be separated reliably. Table 5 shows the correlation between the NHSPPI (and its components) and the CQI (and its components) for practices who participated in the consultation-based survey in each region. Overall, there was no correlation between practices’ rank ordering for their scores on the NHSPPI and their scores on the CQI. However, rank order for mean consultation length was correlated with rank order for NHSPPI and to the prescribing subindex (PPSI). Also, practices’ rank ordering for how well
81
A routine NHS data-based index of performance in general practice TABLE 3
Minimum, maximum and mean NHSPPI scores by region Pa
All
12/27 19.9 (4.4)
,0.001
11/31 21.6 (4.3)
11/24 19.2 (3.6)
12/25 19.3 (3.7)
,0.001
11/29 20.4 (3.8)
13/27 21.3 (3.5)
11/25 19.3 (3.8)
11/26 19.3 (4.1)
,0.001
11/29 20.5 (3.9)
11/25 19.5 (3.4)
11/25 18.5 (3.4)
12/24 18.7 (3.4)
0.03
11/27 19.3 (3.4)
Oxford (n = 25)
Lothian (n = 23)
West London (n = 35)
Coventry (n = 34)
Performance index summary value (all indicators included) Minimum/maximum Mean (SD)
17/31 24.8 (2.9)
15/29 23.0 (3.7)
11/26 20.1 (4.0)
Performance index summary value (cough suppressants excluded) Minimum/maximum Mean (SD)
15/29 22.8 (2.9)
13/27 21.2 (3.6)
Performance index summary value (diuretics excluded) Minimum/maximum Mean (SD)
18/29 23.1 (2.7)
Performance index summary value (cough suppressants and diuretics excluded) Minimum/maximum Mean (SD)
16/27 21.1 (2.7)
a Gives the result of a Kruskal–Wallis test, with the data classified by area. The null hypothesis is one of equal median values of NHSPPI across all areas.
TABLE 4 Spearman correlation coefficients of NHS Practice Performance Index (NHSPPI), list size, deprivation and the percentage of patients who speak a language other than English in participating practices
NHSPPI List size Weighted deprivation index
List size
Weighted deprivation index
Other language patients
0.25 (0.07)
–0.57 (0.001)
–0.44 (0.001)
–0.36 (0.01)
–0.41 (0.002) –0.62 (0.001)
patients knew the doctor was negatively correlated with rank ordering for the PPSI.
Discussion Strengths and weaknesses of the method The indicators selected for inclusion in the NHSPPI are intended to reflect both good clinical practice and costeffectiveness. They were chosen within the constraint that the data had to be reasonably easy to collate in a consistent manner across the four regions under study. Quality is a multifactorial concept, and individual markers derived from existing routine sources do not always place performance in context. We recognize that in some cases there may be legitimate reasons unknown
to the research team which explain why a practice has fallen outside the bounds of acceptability for one particular indicator. The research team agreed the ranges of acceptability for each indicator after examination of the results but before attempting comparisons between regions or practices. This made our decisions considerably easier, although even then some indicators were not easily subjected to the four-level categorization we devised. It may be that some commentators will disagree with the values we have chosen or that some indicators should be excluded or weighted. Our decisions in defining ‘acceptable’ practice were arbitrary; it would be easy to re-adjust the NHSPPI accordingly. In a recently published Delphi study17 looking at the validity of prescribing indicators, all but two of the prescribing indicators in the NHSPPI were rated valid. The exceptions were the ratio of steroid to bronchodilator items which was not assessed because data in terms of defined daily doses (DDD) and specific therapeutic groups age sex-related prescribing units (STAR-PU) are not routinely available in Scotland. Secondly, the number of items for statins per 1000 patients was rated neither valid nor invalid. All preventive components of the NHSPPI are already elements of GPs’ remuneration. An equally important question is whether aggregating individual performance indicators is an appropriate and correct way of assessing quality. Does the performance index presented in this paper reflect a cohesive measure of one aspect of quality in general practice? All the routine indicators used in this study have necessarily been aggregated to practice level. This means that the NHSPPI cannot identify difference in performance between
82
Family Practice—an international journal TABLE 5
Rank correlations between NHS Practice Performance Index (NHSPPI) and its components and Consultation Quality Index and its components by region na
Measure
Rank correlation with practice level b Mean Mean PEI score consultation length
Mean ‘know the doctor’ score
Mean CQI scorec
Aggregated NHSPPI: participating practices in Oxford
12
NS
0.58 (P = 0.05)
NS
0.66 (P = 0.02)
Practice prescribing subindex (PPSI): participating practices in Oxford
12
NS
0.64 (P = 0.03)
NS
NS
Practice preventive subindex (PRSI): participating practices in Oxford
12
NS
NS
NS
NS
Practice external quality subindex (PQSI): participating practices in Oxford
12
NS
NS
NS
NS
Aggregated NHSPPI: participating practices in Lothian
13
NS
NS
NS
NS
Practice prescribing subindex (PPSI): participating practices in Lothian
14
NS
NS
NS
NS
Practice preventive subindex (PRSI): participating practices in Lothian
14
NS
NS
NS
NS
Practice external quality subindex (PQSI): participating practices in Lothian
13
NS
NS
NS
NS
Aggregated NHSPPI: participating practices in West London
14
NS
NS
NS
NS
Practice prescribing subindex (PPSI): participating practices in West London
14
NS
NS
NS
NS
Practice preventive subindex (PRSI): participating practices in West London
14
NS
NS
NS
NS
Practice external quality subindex (PQSI): participating practices in West London 14
NS
NS
NS
NS
Aggregated NHSPPI: participating practices in Coventry
9
NS
0.66 (P = 0.05)
NS
NS
Practice prescribing subindex (PPSI): participating practices in Coventry
9
NS
0.66 (P = 0.05)
NS
NS
Practice preventive subindex (PRSI): participating practices in Coventry
9
NS
NS
NS
NS
Practice external quality subindex (PQSI): participating practices in Coventry
9
NS
NS
NS
NS
Aggregated NHSPPI: all participating practices
48
NS
0.31 (P = 0.03)
NS
NS
Practice prescribing subindex (PPSI): all participating practices
49
NS
0.36 (P = 0.01) –0.33 (P = 0.02)
Practice preventive subindex (PRSI): all participating practices
49
NS
NS
NS
NS
Practice external quality subindex (PQSI): all participating practices
48
NS
NS
NS
NS
NS
a Data
are restricted to practices which returned at least 50 valid enablement scores for English-speaking adult patients. practice-level values for mean PEI score, mean consultation length and mean ‘know the doctor’ score are derived from consultations involving English-speaking adult patients only. c Computed at practice level as the mean of the CQI scores of the individual doctors working in the practice. b The
individual GPs. In terms of representativeness, it was important to note little difference in the NHSPPI between practices agreeing and declining to participate in the consultation study. Implications of results Variation within general practice is a well-established phenomenon. The results presented for each of the performance indicators are worthy of examination in their own right. Some of the variation appears to be statistically rather than clinically significant, but particularly striking were the 5-fold differences in prescribing for cough suppressants and nasal decongestants across regions and 2-fold differences in prescribing of statins across regions. It is possible that the context (which
includes expectations and values of patients) predicts the need for different standards and that some of these variations are due to cultural differences between regions rather than differences in competence between practices. Equally it is possible that there are more practices in Oxford than in Coventry which are ‘early adaptors’ of change, and that the differences (both to lower use of discredited therapies and to higher use of new ones) do reflect better as opposed to poorer care. Higher practice-level deprivation is associated with lower scores on the NHSPPI, but a causal relationship cannot be inferred. The effect of deprivation is confounded with smaller practice list sizes and with higher proportions of patients who speak languages other than English.
A routine NHS data-based index of performance in general practice
Clinical governance Little evidence of correlation was found between the CQI and the NHSPPI or between components of either. However, practices where doctors had longer mean consultation lengths performed better on the prescribing indicators (in two of four regions). In contrast, practices in which the patients knew the doctors better performed less well on prescribing indicators. The first of these results has good face validity, the second suggests that familiarity may compromise prescribing quality standards. This finding may be due to confounding factors, and requires further examination. The lack of an overall correlation between the CQI and NHSPPI may be due to the problem of extrapolating between measures which use different denominators. Otherwise the measures capture aspects of quality in general practice which are not related: good quality care as assessed by patients on completion of their consultation is independent of good quality care as assessed by best available measures of practice performance. Nonetheless, it seems reasonable to ask whether practices with a poor performance on the NHSPPI added to a poor performance on the CQI warrant concern. Would a significant absolute improvement in performance as judged by either the CQI or the NHSPPI over time be enough to demonstrate that a practice has improved the quality of care it provides? The complexity of trying to answer such questions demonstrates that there are likely to be few short cuts in the introduction of clinical governance and that the measurement of quality in general practice is only now at the starting line. We would suggest that the CQI and NHSPPI are at least as ready for use as other measures of performance in general practice, and that continued effort is required to develop and promote instruments which will allow a meaningful assessment of whether continual quality improvement is being achieved.
work was supported by grants from: the Chief Scientist’s Office at the Scottish Office Home and Health Department; Anglia and Oxford NHS R & D Directorate; West Midlands NHS R & D Directorate; and North Thames NHS R & D Directorate.
References 1
2
3
4
5
6
7
8
9
10
11
12
13
14
Acknowledgements We thank Dr R Elton (University of Edinburgh), Ms H Rai and Dr M Pierce (Imperial College School of Medicine). We also wish to acknowledge our indebtedness to the 221 doctors and 53 practices that participated, together with their managers and reception staff. The
83
15
16
17
Secretary of State for Health. The New NHS: Modern and Dependable. London: HMSO, 1997. Secretary of State for Health for Scotland. Designed to Care: Renewing the National Health Service in Scotland. Edinburgh: Stationery Office, 1997. Donaldson LJ, Muir Gray JA. Clinical governance: a quality duty for health organisations. Qual Health Care 1998; 7 (Suppl): S37–S44. Roland M, Holden J, Campbell S. Quality Assessment for General Practice: Supporting Clinical Governance in Primary Care Groups. Manchester: University of Manchester, National Primary Care Research and Development Centre, 1998. Wilkin D, Hallam L, Doggett M. Measures of Need and Outcome for Primary Health Care. Oxford: Oxford Medical Publications, 1992. Toon PD. What is Good General Practice? Occasional Paper 65. London: Royal College of General Practitioners, 1994. Baker R, Streatfield J. What types of general practice do patients prefer? Exploration of practice characteristics influencing satisfaction. Br J Gen Pract 1995; 45: 654–659. Majeed F, Voss S. Performance indicators for general practice. Br Med J 1995; 311: 209–210. Howie JGR, Heaney DJ, Maxwell M. Measuring Quality in General Practice. Occasional Paper 75. London: Royal College of General Practitioners, 1997. Lawrence M, Olesen F. Indicators of quality in health care. Eur J Gen Pract 1997; 3: 103–108. Safran DG, Tiara DA, Rogers WH. Linking primary care performance to outcomes of care. J Fam Pract 1998; 47: 13–220. NHS Executive. The New NHS; Modern and Dependable: A National Framework for Assessing Performance. Consultation Document. London: NHS Executive, 1998. Campbell SM, Roland MO, Quale JA, Buetow SA. Quality indicators for general practice: which ones can general practitioners and health authority managers agree are important and how useful are they? J Publ Health Med 1998; 20: 414–421. Stott N, Kinnersley P, Jones Elwyn G. Measuring general practicebased primary care: generic outcomes. Fam Pract 1997; 14: 486–491. Howie JGR, Heaney DJ, Maxwell M, Walker JJ, Freeman GK. Developing a ‘consultation quality index’ (CQI) for use in general practice. Fam Pract 2000; 17: 455–461. Howie JGR, Heaney DJ, Maxwell M, Walker JJ, Freeman GK, Rai H. Quality at general practice consultations: cross sectional survey. Br Med J 1999; 319: 738–743. Campbell S, Cantril. Assessing the validity of prescribing indicators for cost and quality. Br Med J 2000; 321: 425–429.
84
Family Practice—an international journal
Appendix 1 Definitions of performance indicators included in the study and thresholds for ‘fully acceptable’ performance, ‘possibly acceptable’ performance, ‘probably unacceptable’ performance and ‘unacceptable’ performance for each indicator Indicatora
Range of valuesb
Direction Fully acceptable
‘Near miss’
Unacceptable
1. Percentage of items for 4-quinolones to all antibiotic items in past year
Lower is better
,3.0
3.0–4.9
2. Percentage of bendrofluazide 5 mg tablets to all bendrofluazide items in past year
Lower is better
,20.0
20.0–29.9
30.0–39.9
>40.0
3. Percentage of (compound diuretics+diuretics with potassium) to all diuretic items in past year
Lower is better
,25.0
25.0–34.9
35.0–49.9
>50.0
4. Nunber of items per 1000 patients for cough suppressants and nasal decongestants in past year
Lower is better
,50.0
50.0–99.9
100.0–299.9
>300.0
5. Number of items for appetite suppressants per 1000 patients in past year
Lower is better
,2.0
2.0–2.9
6. Percentage of NSAID items comprised by the practice’s top four NSAIDs in past year
Higher is better
>80.0
70.0–79.9
50.0–69.9
,50.0
7. Percentage of antibiotic items comprised by the practice’s top ten antibiotic items in past year
Higher is better
>95.0
85.0–94.9
70.0–84.9
,70.0
8. Ratio of steroid to bronchodilator items in past year
Higher is better
>50.0
40.0–49.9
30.0–39.9
,30.0
9. Number of items for statins per 1000 patients in past year
Higher is better
>100.0
50.0–99.9
20.0–49.9
,20.0
10. Number of items for HRT (BNF 4.4.1.1) per 1000 female patients aged 45–65 in past year
Higher is better
>750.0
600.0–749.9
300.0–599.9
,300.0
11. Uptake rate (percentage): primary immunization course for infants
Higher is better
>95.0
90.0–94.9
70.0–89.9
,70.0
12. Uptake rate (percentage): pre-school immunization
Higher is better
>95.0
90.0–94.9
70.0–89.9
,70.0
13. Uptake rate (percentage): cervical cytology screening rate for the target population
Higher is better
>80.0
50.0–79.9
40.0–49.9
,40.0
14. Sum of PGEA (all partners) as percentage of maximum that could have been obtained
Higher is better
>95.0
90.0–94.9
0.1–89.9
0
Higher is better
Registered for all three
Registered for two
Registered for one
Registered for none
Higher is better
>£14 000
£12 000– £13 999
£5000– £11 999
,£5000
15. Are there doctors working regularly in the practice who are registered for:d (a) Child health surveillance (b) Minor surgery (c) Maternity care 16. Level of reimbursement for staff costs (£ per 1000 patients) a
>5.0
Cause for concernc
>3.0
N/A
N/A
Two indicators featured in the original ‘basket’—cost of inhaled steroids and number of contraceptive fee claims—have been dropped. In computing the PPI, each ‘fully acceptable’ result is assigned a score of 2; each ‘near miss’ result a score of 1; each ‘unacceptable’ result a score of 0; and each ‘cause for concern’ result a score of –1. c An entry ‘N/A’ indicates that the full four-band grading system was not considered applicable to the indicator in question, and that the indicator was, in consequence, reduced to only three bands. d Indicator 15 consists of three subitems, each of which attracts a score of 1 when the condition is met (e.g. doctor registered for child health surveillance); otherwise zero. An overall score for item 15 is arrived at by summing scores on the three subitems then subtracting 1, the resulting score ranging from –1 (no conditions met) to 2 (all conditions met). This score is interpreted in the same way as scores on the other (non-binary) indicators, i.e. 2 is considered fully acceptable, –1 represents a ‘cause for concern’ level of performance. b
