Emergency Medical Services Outcomes Research

EMERGENCYMEDICAL SERVICES OUTCOMES RESEARCH:

Prehosp Emerg Care Downloaded from informahealthcare.com by University of North Carolina on 04/11/13 For personal use only.

EVALUATING THE EFFECTIVENESS O F PREHOSPITAL CARE Samuel A. McLean, MD, MPH, Ronald F. Maio, DO, MS, Daniel W. Spaite, MD, Herbert G. Garrison, MD, MPH Highway Traffic Safety Administration (NHTSA) has organized several initiatives to promote EMS evaluation and improvement. In 1993, NHTSA convened the Uniform Pre-hospital Emergency Medical Services Data Conference. The conference report1' provides detailed guidelines for the collection of standardized data in the prehospital setting. In 1994, NHTSA convened a workshop on methodologies for "measuring morbidity outcomes in EMS." The experts in this workshop concluded that implementation of EMS outcomes research was essential. However, it was noted that the methods applicable to prehospital outcomes, especially those using non-mortality measures, had never been developed.12In response to these conclusions, NHTSA funded a five-year cooperative project, the Emergency Medical Services Outcomes Project (EMSOP), to facilitate EMS outcomes research. The ongoing results of this project provide much of the information included in this review.

Outcomes research, a type of clinical research, focuses on determining whether interventions performed in clinical practice actually work. The techniques applied in clinical trials determine whether an intervention works in a controlled research setting (treatment efficacy). However, the goal of outcomes research is to determine whether an efficacious intervention can work in the less-controlled setting of "real-world" practice (treatment effectiveness). Outcomes research is important to health care because many of our current clinical practices have not been proven to actually improve patient care. The discovery of wide variations in medical practice between different health care providers and different geographic regions has indicated that many interventions are not evidence-based.l At the same time, health care costs have increased dramatically, and managed care organizations and others are increasingly demanding evidence that what they are paying for works.' As a result, assessment of medical interventions using outcomes research methodologies has become a necessity. Prehospital emergency medical services (EMS) systems are particularly in need of evaluations that identify effectiveness. Most prehospital interventions are not supported by good evidence of improved patient outcome.2-4A number of experts have questioned the value of the breadth of out-of-hospital care services currently p r o ~ i d e d . ~The , ~ -Department ~ of Health and Human Services has recommended that Medicare end reimbursement for out-of-hospital services for which there is no evidence of medical necessity, initially targeting ambulance transports between doctors' offices and nursing homes.'" In response to these concerns, the National

THESCOPEOF EMS CARE:How CAN

WE MAKE A DIFFERENCE?

Prehospital care involves the treatment of a vast array of maladies. Panel experts at the 1994 NHTSA conference recognized the need to identify specific prehospiTABLE 1. Prioritv Conditions for Children and Adults* Children 1. Minor trauma 2. Major trauma 3. Respiratory distress 4. Airway obstruction 5. Respiratory arrest 6. Cardiac arrest 7. Seizure

From the Department of Emergency Medicine, University of Michigan Medical Center (SAM, RFM), Ann Arbor, MI; the Section of Emergency Medicine, Department of Surgery, University of Arizona (DWS), Tucson, AZ; and the Department of Emergency Medicine, The Brody School of Medicine at East Carolina University (HGG), Greenville, NC.

Adults 1. Minor trauma 2. Respiratory distress 3. Chest pain 4. Major trauma 5. Cardiac arrest 6. Airway obstruction 7. Respiratory arrest

Supported in part by the US. Department of Transportation , National Highway Traffic Safety Administration (contract no. DTNH22-96-H-054245). Address for correspondence and reprints: Samuel A. McLean, MD, University of Michigan, Department of Emergency Medicine, TCB 1382 Taubman Center, Box 0305, 1500 East Medical Center Drive, Ann Arbor, MI 48109. e-mail .

~

*Used with permission from: Maio RF, Garrison HG, Spaite DW, et al. Emergency Medical Services Outcomes Project I (EMSOP I): prioritizing conditions for outcomes research. Ann Emerg Med. 1999;33:423-32.

S52

McLean et al.

s53

EMS OUTCOMES RESEARCH

Model for identifying the effect of treatment from each unit of service in the episode of care

k OUT-OF-HOSPITAL

RA

EMERGENCY SUBSPECIALTY CARE

ED CARE

T

OUT

RA

T

OUT RA

I Prehosp Emerg Care Downloaded from informahealthcare.com by University of North Carolina on 04/11/13 For personal use only.

INPATIENT CARE

I

t

Precipitating Event

T

OUT

7 FOLLOW-UP

RA

T

OUT]

t

Long-Term outcomes

RA = Risk Adjustment Measures T = Therapeutic intervention(s) OUT = OutcomeMeasures FIGURE1. The Episode of Care Model. ED = emergency department. “Used with permission from: Spaite DW, Maio RF, Garrison HG, et al. Emergency Medical Services Outcomes Project I1 (EMSOP 11): developing the foundation and conceptual models for out-of-hospital outcomes research. Ann Emerg Med. 2001;37:657-63

tal conditions that should serve as a focus for EMS outcomes research (“Priority Conditions”). It was decided that all relevant patient outcomes should be considered for each condition.12These outcomes have been referred to as the 6 ”Ds”: death, disease (physiologic derangement), disability, discomfort, dissatisfaction, and destitution (cost).12The selection of priority conditions using these outcome criteria was the initial task of the EMSOP.I3 The EMSOP study group developed a list of common EMS conditions using data element items from the NHTSA Uniform Data Conference.” The relative frequency of each condition was also determined, using information from a variety of geographic regions. Data were obtained for all age groups. After gathering these frequency data, 42 EMS experts rated the potential impact of EMS intervention for each condition. The experts also rated the relevance of each potential prehospital care outcome (the 6 ”Ds” above) for each condition. The study group then used this information to rank-order EMS conditions, based upon the predicted importance and relevance of performing effectiveness research for each condition (Table 1).13 The conditions listed represent 65% of all adult transports, and 85% of all pediatric transports. Non-mortality outcomes, including discomfort, patient satisfaction, and disability, were identified as being a high priority for many of these conditions. For example, discomfort was identified as the most relevant outcome for future EMS outcomes research for the top three adult conditions and the first and third children’s conditions. The EMSOP study

group concluded that while almost no work has evaluated the effect of EMS care on non-mortality outcome measures, these measures should be a major focus of future research efforts.l3

A MODEL FOR EVALUATING RESEARCH E M S OUTCOMES In order to evaluate prehospital interventions, a sound conceptual and methodological framework for conducting outcomes research in this setting is necessary. The EMSOP project has suggested two models: the Episode of Care Model and the Prehospital Unit of Service M0de1.l~ A patient who enters the medical system through EMS may receive a long series of interventions, commencing with treatments or interventions that occur in the field, and continuing with care rendered in the emergency department, as an inpatient, or following discharge. Therapeutic interventions delivered along the entire process of care have the potential to impact outcomes that are measured near the end of the clinical course. The Episode of Care Model is intended to help identify the influence of prehospital interventions on overall patient outcome. The Episode of Care Model separates the care delivered in different settings during a patient’s clinical course into different “units of ~ e r v i c e ’ ’ ~ (Fig. ~ - ~1). ~ Numerous units of service typically occur in an episode of care. For patients who enter the medical system through EMS, prehospita1 care is the first unit of service. Obviously, not all patients require all of the potential units of service.

s54

.. . .. . 0

0

0


0

PREHOSPITAL EMERGENCYCARE SUPPLFMENTTO APRIL/JUNE 2002

TABLE 2. Dimensions of Risk* Age Sex Race and ethnicity Acute clinical stability Principal diagnosis (”case mix”) Severity of principal diagnosis Extent and severity of comorbidities Physical functional status Psychological, cognitive, and psychosocial functioning Cultural and socioeconomic attributes and behaviors

0

Health status and quality of life

0

Patient attitudes and preferences for outcomes

‘Used with permission from: Iezzoni LI (ed). Risk Adjustment of Measuring Health Care Outcomes. Chicago: Health Administration Press, 1997 (Table 2.1, p 45).

Using the Episode of Care Model, it would be optimal to identify the impact of care rendered during each unit of service on short-, intermediate-, and longterm outcomes. This is accomplished using riskadjustment and outcome measures. Early in each unit of service, risk-adjustment measurements are collected. For example, for a study of respiratory distress, risk-adjustment measures would include patient age, initial vital signs, initial level of consciousness, and initial oxygen saturation. Then therapeutic interventions are performed, such as administering oxygen or a nebulizer treatment. Finally, outcome measurements are obtained, such as vital signs and oxygen saturation after the intervention. Utilizing this methodology in a stepwise fashion allows the impact of therapeutic interventions rendered in each unit of service to be identified and measured. As noted above, for patients who enter the medical system through EMS, the first unit of service in the Episode of Care Model is the prehospital unit of service. At times, the only outcome of interest occurs in the prehospital setting, such as in studies of patient satisfaction or relief of pain in the prehospital setting, or in studies of patients who refuse transport. By focusing on this unit of service, a more detailed model can be developed to help identify the impact of care rendered before arrival at the hospital. This Prehospital Unit of Service Model was developed from a previously validated time interval model.20After the arrival of EMS personnel at the patient’s side, the Assessment Interval begins. During this time, appropriate riskadjustment measurements are taken. For example, for a study of relief of pain in the prehospital setting, such measures might include vital signs and the use of a visual analog scale to measure pain severity. This is followed by interventions that occur during the Scene Treatment and/or Transport Intervals, such as the administration of intravenous morphine. Outcome measurements, such as repeat measurement of vital

VOLUME6 / NUMBER 2

signs and pain severity, would typically be collected during the Transport Interval after the pertinent interventions have occurred. Multiple interventions may also be evaluated in series.

RISKADJUSTERS AND OUTCOME MEASURES Within any unit of service, three elements determine patient outcomes: random events, patient factors, and treatment effectiveness.21Patient factors are patient characteristics that may contribute to a treatment outcome. Risk-adjustment measures are used to account for patient differences, so that the effect of treatment on outcome can be most accurately determined.21For example, in a study examining the effect of nebulizer treatments on patients with reactive airways disease, patient smoking status would be an important patient characteristic to include as a risk-adjustment measure in data analysis. Patient factors that may influence outcomes are also known as ”dimensions of risk” (Table 2). In any given episode of care, there are many dimensions of risk. For the above nebulizer study, in addition to smoking status, important dimensions of risk might include patient age, past medical history, previous medication use, initial vital signs, initial oxygen saturation, initial peak flow, and a measure of initial dyspnea severity. Many other patient factors may also aid in outcome evaluation, but collection of information on risk-adjustment measures is limited by the unique context of the prehospital unit of service, which is usually brief and often rushed. The prehospital care researcher has a special challenge: to identify, from all the possible dimensions of risk, those riskadjustment measures that allow a determination of the effectiveness of EMS interventions. Most of the prehospital risk adjustment measures will be measured or collected by EMS professionals. Some will have to be obtained by linkage to other medical databases.22 Both core and specific risk adjustment measures are necessary for quality prehospital research. Core measures are necessary for prehospital outcomes research involving any condition. Examples of core risk-adjustment measures include patient age, sex, ethnicity, vital signs, level of responsiveness, and event to treatment interval. If EMS databases are to be used for future retrospective assessments of prehospital interventions, core risk adjustment measures should be obtained systemically for every prehospital patient. Specific riskadjustment measures are those measures that are needed to evaluate a particular condition. Examples of specific risk-adjustment measures for the conditions of respiratory distress are peak expiratory flow rate, pulse oximetry, and the visual analog scale for dyspnea. In addition to selecting risk-adjustment measures to adjust for patient differences, outcomes researchers must also identify and measure appropriate core and specific outcomes measures. Two considerations are


McLean et al.

EMS OUTCOMES

s55

RESEARCH

important in choosing these measures. First, investigators must consider which types of outcome (the 6 "Ds") will be important to study. Next, they need to consider the time period over which they wish to examine outcomes. That is, whether short-, intermediate-, or long-term outcomes will be measured. These time periods lack uniform definition in the prehospital literature. A logical immediate outcome time period would begin at the time the event occurred and end at the time of disposition from the emergency department. Intermediate- and long-term outcome time periods are more difficult to define, and definitions may vary depending on the condition being examined. Evaluating the impact of prehospital care on these longer-term outcomes is a major goal of outcomes research. Thus, regardless of how intermediateand long-term time intervals are defined, it is critical that immediate outcome measures are identified that can be linked to these subsequent periods. Many useful outcomes measures are also risk-adjustment measures, because the spectrum of outcomes defined by the 6 "Ds"are also important dimensions of risk. For example, a measure of dyspnea severity can be used as a severity measure as well as an outcome measure for discomfort. Due to the great variety of potential outcomes of interest, and the limited research available on these measures, only a small number of measures can currently be recommended as core outcomes. These are vital signs before and after an intervention, level of consciousness before and after an intervention, final vital signs, and mortality. The core outcomes measures described above relate primarily to disease progression and the degree of disability present. A number of other "specific" outcome types are also of great importance in prehospital outcomes research, but cannot yet be universally recommended because the science of measurement is incomplete. For example, patient satisfaction is an important outcomes measure, and a measure of satisfaction should ideally be included in any prehospital outcomes study. However, no validated EMS patient satisfaction instruments currently exist. Thus, measurement of patient satisfaction cannot be recommended as a core outcome measurement at this time. Similarly, an examination of cost should ideally be included in any prehospital outcome study. However, much more research is necessary on the evaluation of cost in the prehospital setting before it can be recommended as a core outcome. Disposition from the emergency department, a marker of extent of disease progression, may be an important outcome for many studies. Patient discomfort is another outcome that will be important for many outcomes studies. As noted above, discomfort was identified as the most relevant endpoint for future EMS outcomes research for several of the highest-priority conditions (Table 1)in both children (minor trau-

ma, major trauma, and respiratory distress) and adults (minor trauma, respiratory distress, and chest pain).

IMPLICATIONS FOR THE FUTURE The development of a robust discipline of prehospital outcomes research is long overdue; many critical questions need to be answered. The effectiveness of prehospital care for most EMS interventions remains unknown. The feasibility of developing high-quality, validated risk-adjustment and outcome measures as described above is also unknown. For most conditions, little is known about the effect of prehospital interventions on immediate outcome, or whether changes in immediate outcomes influence intermediate- or long-term results. The need to use societal resources responsibly demands that this situation be changed in the future. Because of this, the development of a discipline of prehospital outcomes evaluation that produces large volumes of high-quality research is mandatory.

References 1. Kane RL. Approaching the outcomes question. In: Kane RL (ed). Understanding Health Care Outcomes Research. Gaithersburg, M D Aspen Publishers, 1997. 2. Spaite DW. Outcome analysis in EMS systems. Ann Emerg Med. 1993;22:1310-1. 3. Callaham M. Quantifying the scanty science of prehospital emergency care. Ann Emerg Med. 1997;30:785-90. 4. Brice JH, Garrison HG, Evans AT. Study design and outcomes in out-of-hospital emergency medicine research. Prehosp Emerg Care. 2000;4:144-50. 5. Koenig KL: Quo vadis. "Scoop and run," "stay and plan," or "treat and street?" Acad Emerg Med. 1995;2:477-9. 6. Koenig KL. Unscheduled access to health care: reengineering the 911 system. Acad Emerg Med. 1996;3:986-91. 7. Reines HD, Bartlett RL, Chudy NE, Kiragu KR, McKnew MA. Is advanced life support appropriate for victims of motor vehicle accidents: The South Carolina Highway Trauma Project. J Trauma. 1988;28:563-9. 8. Smith JP, Bodai BI, Hill AS, Frey CF. Prehospital stabilization of critically injured patients: a failed concept. J Trauma. 1985;25:6570. 9. Spaite DW, Criss EA, Valenzuela TD, Guisto J. Emergency medical services systems research: problems of the past, challenges of the future. Ann Emerg Med. 1995;26:146-52. 10. Office of the Inspector General, Department of Health and Human Services. Medical Necessity of Medicare Ambulance Services. Bethesda, MD: Department of Health and Human Services, December 1998. Available at: http://oig.hhs.gov/oei/ reports / a319.pdf 11. National Highway Traffic Safety Administration. Uniform Prehospital Emergency Medical Services Data conference. Washington, DC: U.S. Department of Transportation, August 16-18,1993. 12. National Highway Traffic Safety Administration. EMS Outcomes Evaluation: Key Issues and Future Directions. Proceedings from the NHTSA workshop on methodologies for measuring morbidity outcomes in EMS. Washington, DC: U.S. Department of Transportation, April 11-12,1994. 13. Maio RF, Garrison HG, Spaite DW, et al: Emergency Medical Services Outcomes Project I (EMSOP I): prioritizing conditions for outcomes research. Ann Emerg Med. 1999;33:423-32. 14 Spaite DW, Maio RG, Garrison HG, et al. Emergency Medical

S5 6

TO APRIL / J U N E 2002 PREHOSPITAL EMERGENCYCARESUPPLEMENT


Services Outcomes Project I1 (EMSOP 11):developing the foundation and conceptual models for prehospital outcomes research. Ann Emerg Med. 2001;37657-63. 15. Hornbrook ML, Hurtado AV, Johnson RE. Health care episodes: definition, measurement, and use. Med Care. 1985;42:163-218. 16. Solon JA, Feeney JJ, Jones SH, Rigg RD, Sheps CG. Delineating episodes of medical care. Am J Public Health. 1967;57:401-8. 17. Young KM, Fisher CR. Medicare episodes of illness: a study of hospital, skilled nursing facility and home health agency care. Health Care Financ Rev. 1980;2:1-23. 18. Grady ML, Schwartz HA (eds). Medical Effectiveness Research Data Methods. Rockville, MD: Agency for Health Care Policy and Research, 1992.

VOLUME 6 / NUMBER 2

19. Steinwachs DM. Episode of Care Framework: Utility for

Medical Effectiveness Research. Agency for Health Care Policy and Research, Publication N. 92-0056, Rockville ,MD, June 1992. 20. Spaite DW, Valenzuela TD, Meislin HW, Criss EA, Hinsberg P. Prospective validation of a new model for evaluating EMS systems by in-field observation of specific time intervals in prehospita1 care. Ann Emerg Med. 1993;22638-45. 21. Iezzoni LI. Using risk-adjusted outcomes to assess clinical practice: an overview of issues pertaining to risk adjustment. Ann Thorac Surg. 1994;58:1822-6. 22. Waien SA. Linking large administrative databases: a method for conducting emergency medical services cohort studies using existing data. Acad Emerg Med. 1997;4:1087-95.