A guide to economic evaluation: Methods for ... - Wiley Online Library

3 downloads 3252 Views 151KB Size Report
To illustrate how to conduct and evaluate a cost-effectiveness analysis, a hypothetical study ... provide a hypothetical example to illustrate key steps in economic ...
C 2006) Journal of Traumatic Stress, Vol. 19, No. 6, December 2006, pp. 787–797 (

A Guide to Economic Evaluation: Methods for Cost-Effectiveness Analysis of Person-Level Data Jeffrey S. Hoch Centre for Research on Inner City Health and St. Michael’s Hospital, Toronto, and the Department of Health Policy, Management and Evaluation, University of Toronto, Ontario, Canada

Mark W. Smith Health Economics Resource Center, U.S. Department of Veterans Affairs, Menlo Park, CA, and the Center for Primary Care and Outcomes Research, Stanford University, Stanford, CA The authors introduce economic evaluation with particular attention to cost-effectiveness analysis. They begin by establishing why health care decisions should be guided by economics. They then explore different types of economic evaluations. To illustrate how to conduct and evaluate a cost-effectiveness analysis, a hypothetical study about the treatment of posttraumatic stress disorder with psychotherapy versus pharmacotherapy is considered. The authors conclude with recommendations for increasing the strength and relevance of economic evaluations.

Economics is the study of decision-making in the face of scarcity. All resources in health care are scarce: money, time, people, space, and technology. Not everything that is costeffective is affordable. Economic evaluation addresses the challenges associated with scarcity in health care by considering both the costs and consequences of treatment alternatives. With this dual focus on both treatment costs and patient outcomes, economic analyses can provide much more useful information for policy decisions than analyses based solely on costs or outcomes. Health systems or health care payers that treat large numbers of persons with posttraumatic stress disorder (PTSD), such as the U.S. Department of Veterans Affairs (VA), have the potential to find

economics analyses useful because the results clarify the relative benefits and costs of competing treatment modalities. In addition, funding agencies may require researchers who conduct clinical trials to include plans for economic analyses in their grant proposals as well as illustrations of the degree of statistical uncertainty in the results. Should costs, monetary or otherwise, even be considered in health care decisions? A natural way to think of cost is accounting cost, the price of the resources consumed. Economists also think about opportunity cost, the foregone value that would have been gained from using those resources for the next best alternative. They argue that “anyone who says that no account should be paid to

Dr. Hoch gratefully acknowledges funding from a Career Scientist Award from the Ontario Ministry of Health and Long Term Care. The Centre for Research on Inner City Health is sponsored by the Ontario Ministry of Health and Long-Term Care. Dr. Smith gratefully acknowledges funding from the VA Cooperative Studies Program (CSP 519). Staff members of the VA Health Economics Resource Center, the editors, and two anonymous reviewers provided helpful comments. The opinions, results, and conclusions are those of the authors and no endorsement by the Ontario Ministry of Health and Long Term Care or the U.S. Department of Veterans Affairs is intended or should be inferred. Correspondence concerning this article should be addressed to Jeffrey Hoch, Centre for Research on Inner City Health, 30 Bond Street, Toronto, Ontario M5B 1W8, Canada. E-mail: [email protected].  C 2006 International Society for Traumatic Stress Studies. Published online in Wiley InterScience (www.interscience.wiley.com) DOI: 10.1002/jts.20190

787

788

Hoch and Smith

[opportunity] costs is really saying that no account should be paid to the sacrifices imposed on others” (Williams, 1992, p. 7). To make health care decisions without reference to costs as well as outcomes is to assume that there were no alternative uses for the resources consumed. When considering how to treat a patient suffering from PTSD, perhaps the only consideration is patient outcome; however, when considering how to treat an entire population of patients with PTSD, one must be mindful of both patient outcome and the resources available to treat other patients. For caregivers interested in participating in resource allocations decisions, an understanding of economic evaluation is critical because such studies make explicit the tradeoffs between patient outcomes gained and resources spent. Although costing studies exist of patients at high-risk for PTSD (Chan, Medicine, Air, & McFarlane, 2003; Hoff & Rosenheck, 1998), economic evaluations are not common in trauma research. Only 3 years ago, researchers found no economic evaluations for combat-related PTSD (McCrone, Knapp, & Cawkill, 2003) and noted that others found no economic component in the studies they reviewed for their meta-analysis of different interventions for PTSD published between 1984 and 1996 (van Etten & Taylor, 1998). Our purpose here is to introduce common economic evaluations, emphasizing cost-effectiveness analysis. We provide a hypothetical example to illustrate key steps in economic evaluation. We conclude with a series of questions to guide the use and design of economic analyses of health outcomes trials.

TYPES OF ECONOMIC EVALUATION There are many different types of economic evaluations, including cost-benefit analysis, cost-effectiveness analysis, cost-utility analysis, and cost-minimization analysis. What distinguishes one from another is the treatment of patient outcomes (see Table 1). The key questions are “How many outcomes?” and “How are they measured?” In costbenefit analysis there are typically many outcomes and all outcomes are valued in dollars, a task difficult and unpalatable to many. For this reason, cost-benefit analysis is rarely

Table 1. Common Types of Economic Analysis Type

Abbreviation

Number of Outcome outcomes unit

Cost-benefit CBA Many Dollars Cost-effectiveness Cost-utility CEA, CUA One QALYs Cost-consequences CEA, CCA One or more Clinical Cost-minimization CMA None N/A Note. A cost-consequences analysis is also called a cost-outcomes analysis.

used in health care. For a notable exception, see Weisbrod, Test, and Stein (1980). In cost-effectiveness analysis, a single outcome is analyzed. Commonly, the outcome is measured in clinical units such as symptom-free days or outpatient visits. A second form of cost-effectiveness analysis is cost-utility analysis, which values outcomes in quality-adjusted life years (QALYs), equal to the number of life years remaining multiplied by a factor reflecting quality of life. In cost-minimization analysis, only costs are compared; patient outcomes are assumed to be equivalent. For example, suppose that a new psychotherapy method (psych) is being compared to a medication regimen (med). A cost-benefit analysis will identify all of the consequences from the treatment options (e.g., reduced severity and duration of PTSD symptoms) and convert these into monetary benefit figures represented by Benefitpsych for psychotherapy and Benefitmed for medication. These are then compared to costs, represented by Costpsych and Costmed . The net benefit for each treatment modality equals the difference between its benefits and its costs. What is of primary interest in an economic evaluation is the extra net benefit of using one treatment regimen versus the other. The difference in net benefits for the two treatments is called the incremental net benefit and is calculated as follows: Incremental net benefit (INB) = Net Benefitpsych − Net Benefitmed INB = (Benefitpsych − Costpsych ) − (Benefitmed − Costmed ) INB = (Benefitpsych − Benefitmed ) − (Costpsych − Costmed ) INB = extra Benefit − extra cost INB = B − C.

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

Methods for Cost-Effectiveness Analysis of Person-Level Data

When the incremental net benefit > 0, the extra benefits outweigh the extra costs (i.e., B > C). If the benefits from the two regimens are equivalent then the decision maker can focus on minimizing cost. If benefits are not equivalent, however, then focusing solely on costs is inappropriate. In practice benefits are rarely equal, and thus an important element of economic analysis is deciding which outcomes to measure and how.

Valuing Outcomes in Quality-Adjusted Life Years In a cost-utility analysis, the outcome is measured in quality-adjusted life years (QALYs). Quality-adjusted life years are obtained by multiplying a weight representing quality of life in a health state by the length of time spent in that health state. The weight ranges from 0 (death) to 1.0 (perfect health; Bala & Zarkin, 2000). Thus, 10 years in a health state valued at 0.60 is equivalent to 6.0 QALYs. The Clinician-Administered PTSD Scale (CAPS; Blake et al., 1995; Weathers, Keane, & Davidson, 2001) combines the intensity and frequency of individual PTSD symptoms through addition; the QALY combines quality and length of life through multiplication. The incremental gain in QALYs with psychotherapy relative to medication is calculated as: Incremental QALYs gained = (QApsych × LYpsych ) − (QAmed × LYmed ),

where QA is a quality-of-life adjustment, and LY is life years remaining. To illustrate how QALYs balance QA with LY, we now consider a completely hypothetical and somewhat extreme example. Suppose that elderly patients with severe PTSD can be treated with either psychotherapy or medication, and that on average, those receiving psychotherapy live 5 years at an average quality of life of 0.9, whereas patients receiving medication live 7.5 years with an average quality of life of 0.6. Which treatment modality yields more QALYs? QALYspscyh = (QApsych × LYpsych ) = 0.9 × 5 years = 4.5 QALYs QALYsmed = (QAmed × LYmed ) = 0.6 × 7.5 years = 4.5 QALYs

789

QALYs for psychotherapy: 0.9 × 5 years = 4.5 QALYs

Utility 1.0

QALYs for medication: 0.6 × 7.5 years = 4.5 QALYs

0.8 0.6 0.4 0.2

Years 1

2

3

4

5

6

7

8

Figure 1. Quality adjusted life years (QALYs) for competing therapies: Hypothetical example. Although patients taking medication live longer (7.5 years instead of 5), patients receiving psychotherapy enjoy a higher quality of life (0.9 vs. 0.6). Consequently, there is no gain in QALYs in this hypothetical example.

Figure 1 illustrates the calculation. Although patients taking medication live longer, patients receiving psychotherapy enjoy a higher quality of life. Consequently, there is an incremental gain of 2.5 years of life with medication; however, because of the poor quality of life, there is no incremental gain in QALYs. In this hypothetical scenario, there is no overall gain in patient outcome by switching treatments. A more typical example would have patients pass through several health states over time, each with a different quality-of-life weight. Because the true number of possible health states numbers in the thousands—if not tens of thousands—models must be greatly simplified to be tractable. There are two benefits to choosing QALYs as the primary outcome measure. First, treatment alternatives with different quality-of-life and length-of-life profiles can be compared, as in the example above. Second, using the QALY as a common currency allows for the comparison of treatment breakthroughs for different diseases. For example, new treatments for PTSD and for migraine headaches can be compared once the outcomes are phrased in QALYs. Unfortunately, there is no universally accepted way of measuring the quality-of-life weights needed to make QALYs, and different methods of estimating QALYs can produce considerably different results (Arnesen & Trommald, 2004; Krabbe, Essink-Bot, & Bonsel, 1997; Shumway, 2003).

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

790

Hoch and Smith

Currently available measures for QALYs “may not be sensitive enough to the kinds of changes usually found in mental health care to provide the sole indicator of impact” (McCrane, Knapp, and Cawhill, 2003, p. 521). Thus, although it is considered a best practice to perform costutility analysis, clinicians and managers alike may appreciate additional analyses using clinical or health services outcomes.

What Is a Patient Outcome Worth? How much should a decision maker pay for an extra QALY? There is an oft-stated threshold of $50,000 per QALY for health care interventions in the United States. According to some (Hirth, Chernew, Miller, Fendrick, & Weissert, 2000), the $50,000 QALY threshold goes back more than two decades, although this round number is open to criticism on several grounds. Nevertheless, it remains a commonly cited—and most likely flawed—standard (Ubel, Hirth, Chernew, & Fendrick, 2003). International examples of the varying nature of decision makers’ willingness to pay for QALYs also provide insight (Devlin & Parkin, 2004; George, Harris, & Mitchell, 2001; Hoch, 2004). One can also define a cost-effectiveness threshold for clinical and health services outcomes. Using estimates of extra cost (C) and extra effect (E), a new treatment is deemed cost-effective relative to usual care if the increC ) is less than the amount mental cost-effectiveness ratio ( E the payer is willing to pay for an extra unit of outcome. Organizations that pay for health care decide, on largely unobserved grounds, their willingness to pay. In short, willingness to pay is in the wallet of the beholder. Suppose that a new treatment produces an extra C = $1,000). The symptom-reduced day for $1,000 ( E treatment is considered cost-effective if the payer values a symptom-reduced day at $1,000 or more. Advances in economic evaluation methodology have been possible because of the recently noted equivalence of the questions, “Is the incremental cost-effectiveness ratio less than the willingness to pay threshold?” and “Is the incremental net benefit greater than zero?” (Stinnett & Mullahy, 1998; Tambour, Zethraeus, & Johannesson, 1998). Next, we use regression

methods to estimate the incremental net benefit in a hypothetical study of whether psychotherapy is cost-effective compared to pharmacotherapy.

A HYPOTHETICAL STUDY To illustrate how to perform and evaluate an economic evaluation, we present data from a hypothetical one-year clinical trial comparing psychotherapy (psych) to pharmacotherapy (med). For 105 people with PTSD (Npsych = 60 and Nmed = 45), we generated baseline CAPS severity data (Blake et al., 1995; Weathers et al., 2001) as well as annual cost and annual effect data during the course of follow-up. For this example, we assume that the effect is symptomreduced days obtained over a year (e.g., number of days with 50% reduction of PTSD severity over baseline). The hypothetical data were generated to share a passing resemblance to real data while illustrating the importance of: 1. Considering patient-level costs and effects simultaneously. 2. Exploring statistical uncertainty in the data. 3. Applying regression methods in economic evaluation.

Data Collection and Study Sample Table 2 lists sources that could be consulted for an economic study involving health care costs. The primary intervention is typically micro-costed, meaning that the individual elements of the intervention are elucidated and their costs are summed. Secondary components of the intervention are often assigned an average cost. An average from the system under study is typically used. Researchers often employ out-of-system average costs for sensitivity analyses or when in-system costs are unavailable (e.g., using a low, a medium, and a high estimate for the cost of a hospital day). In health care systems with internal pharmacies, an average cost for particular prescriptions is usually available. An alternative source is the Average Wholesale Price, a commercial figure that forms the basis of Medicaid payments to pharmacies. The Average Wholesale Price can deviate significantly from prices paid by consumers at

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

Methods for Cost-Effectiveness Analysis of Person-Level Data

791

Table 2. Health Care Cost Sources Type

Source

Explanation

Example: Cost of 50 minutes of psychotherapy

Primary

In-system micro-cost

Secondary

In-system average cost External average cost

Determine elements of an encounter, find cost of each Average cost of all similar encounters Average cost of all similar encounters

Average 50-minute cost of therapist (salary + benefits) + department overhead + system overhead Average cost of all 50-minute psychotherapy encounters over a set period Average cost of all 50-minute psychotherapy encounters in another system (e.g., VA, Medicaid, private provider)

Tertiary/sensitivity analysis

Note. Department overhead may include administrative support, supplies, and equipment. System overhead may include supplies and equipment, security, utilities and maintenance, food service, mortgage/rental costs, and capital costs (interest on loans). VA = U.S. Department of Veterans Affairs.

retail pharmacies and from prices paid by federal purchasers such as the U.S. Department of Veterans Affairs (Smith & Joseph, 2003); for details of the Average Wholesale Price calculation, see the Red Book (2004). For additional information on cost data sources in general, see Gold, Siegel, Russell, and Weinstein (1996). Table 3 presents descriptive statistics. On average, patients receiving psychotherapy experienced 40 more symptom-reduced days and had $200 more in expenditures. Although these figures are not statistically significant by conventional standards, they may still be used in figuring a cost-effectiveness estimate. The variation in the costs and outcomes will be reflected in the uncertainty around the cost-effectiveness estimate. The incremental cost-effectiveness ratio and the incremental net benefit statistics can be calculated from C and E reported in Table 3. The incremental cost-effectiveness raC = $5 per day, meaning that the novel psychothertio = E

apy method costs $5 for each extra symptom-reduced day it provides. To illustrate how to calculate the incremental net benefit, we assumed the willingness to pay for an additional symptom-reduced day was $10. In this scenario, the incremental net benefit equals: [($10) (40 extra symptom-reduced days)] − ($200 extra costs) = $200. Both the incremental cost-effectiveness ratio and the incremental net benefit indicate that psychotherapy appears to be cost-effective by conventional standards. Table 3 also provides weak evidence that patients randomized to psychotherapy may have had more severe cases of PTSD at baseline on average. Regression analysis can be used to adjust for baseline severity (or other factors) and to explore whether psychotherapy is more cost-effective for more severe patients. We next illustrate how to accomplish these tasks using net benefit regression (Hoch, Briggs, & Willan, 2002).

Table 3. Simple Tabulation of Treatment Effect and Cost-Effectiveness Based on Hypothetical Data Effect size in symptom-reduced days Treatment group Psychotherapy (n = 45) Medication (n = 60) Difference

Cost in US$

Severity

M

SD

M

SD

M

SD

205 165 E = 40

117 101

$2,200 $2,000 C = $200

1,207 1,357

61 58 −3

9 8

Note. Figures have been rounded to the nearest whole number. Severity represents hypothetical ClinicianAdministered PTSD Scale (CAPS) scores. The incremental net benefit (INB) when willingness to pay is $10 is ($10 × E) − C = $200. The incremental cost-effectiveness ratio (ICER) is C/E = $5 per extra symptom-reduced day. In the Difference row, E and the Severity variable were significant at the 90% confidence level but C was not.

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

792

Hoch and Smith

r The simple regression estimates for α psych and β psych

Regression Analysis With patient-level data we can estimate the extra effect (E) and the extra cost (C) using ordinary least squares regression analysis. The α psych coefficient in Regression 1a is an estimate of the extra effect; β psych in Regression 2a is an estimate of the extra cost. The indicator variable psych = 1 if a participant received the novel psychotherapy method and 0 otherwise. The error terms are ε and ν and the subscript i (denoting individuals) have been dropped for ease of exposition. effect = α0 + αpsych psych + ε

(1a)

cost = β0 + βpsych psych + ν

(2a)

We can likewise use regression to estimate the incremental net benefit by assuming a value for willingness to pay. The γ ps y c h coefficient in Regression 3a is the estimate of the incremental net benefit (Hoch et al., 2002): net benefit = γ0 + γpsych psych + ω.

(3a)

The variable net benefit = [(willingness to pay) × effect] − cost, and ω is the error term. Each study participant’s net benefit value is calculated by the analyst prior to the regression analysis. Because willingness to pay is unknown to the analyst, several net benefit regressions are performed each with a different value for willingness to pay. See Hoch et al. (2002) for a mental health example using a variety of willingness-to-pay values. For simplicity we have calculated net benefit assuming willingness to pay = $10. In the real world, a decision maker’s willingness to pay is likely unknown to the analyst (but probably more than $10). One can estimate the incremental cost-effectiveness ratio and the incremental net benefit with a calculator using the sample average values reported in Table 3. An alternative is to use regression analysis as explained above. Table 4 shows results for the simple linear regressions 1a, 2a, and 3a. Note how the two methods produce similar outcomes:

r The simple regression estimates for α0 and β0 in Table 4 (165 and 2000) equal medication’s average effect and average cost (Table 3, Medication row).

(40 and 200) equal E and C (Table 3, Difference row). It is simple to show that adding the incremental difference (α psych or β psych ) to medication’s averages (α0 or β0 ) yields the average net benefit values for the novel psychotherapy method. r The simple regression estimate for γ psych (200) is the incremental net benefit from Table 3. In net benefit regression, the coefficient on the treatment indicator variable estimates the treatment’s cost effectiveness (i.e., γ psych is an estimate of the incremental net benefit). Net benefit regression analysis provides important advantages. By placing the economic evaluation in a regression framework, it is possible to adjust for disease severity and other variables, run regression diagnostic procedures, and explore whether treatment has different effects for different patient groups. The multiple regression columns in Table 4 highlight the value of the net benefit regression approach for conducting subgroup analysis in costeffectiveness analysis. The three multiple regressions correspond to the simple regressions 1a–3a: effect = α0 + αpsych psych + αSeverity SeverityC +α p × s psych × SeverityC + ε

(1b)

cost = β0 + βpsych psych + βSeverity SeverityC + β p × s psych × SeverityC + ν 

(2b)

net benefit = γ0 + γpsych psych + γSeverity SeverityC + γ p × s psych × SeverityC + ω

(3b)

where ε , ν  , and ω are the error terms. To facilitate the interpretation of the psychotherapy treatment coefficient, we centered the baseline severity variable around its mean and called the new variable SeverityC . Likewise, we multiplied psych by the mean-centered value of Severity to create the interaction term psych × SeverityC . Centering Severity around its mean is useful because for patients with an average severity score, both SeverityC and psych × SeverityC are then 0, and the coefficient on the psych variable is the only one of direct relevance.

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

Methods for Cost-Effectiveness Analysis of Person-Level Data

793

Table 4. Simple and Multiple Linear Regression Results with Hypothetical Data (n = 105) Effect regressions Variables Constant Term Psych SeverityC Psych x SeverityC Model fit statistics Adjusted R 2 F (df)

Cost regressions

Net benefit regressions with willingness to pay (λ) = $10

Simple regression 165∗∗∗ 40 — —

Multiple regression 170∗∗∗ 37 −4∗∗ 5∗

Simple regression 2000∗∗∗ 200 — —

Multiple regression 1947∗∗∗ 206 42∗ −69∗

Simple regression −350 200 — —

Multiple regression −244 164 −83∗∗ 124∗∗

.024 3.51 (1, 103)

.077 3.89∗ (3, 101)

−.004 .61 (1, 103)

.039 2.42 (3, 101)

−.007 .27 (1, 103)

.074 3.79∗ (3, 101)

Psychotherapy (Psych) overall statistical significance

F(2,101) = 3.74∗

F(2,101) = 3.20∗

F(2,101) = 4.08∗

Note. Figures have been rounded to the nearest whole number. The CAPS severity variable has been centered at its mean of 60 and renamed SeverityC . For a patient with a CAPS severity score of 60, both the SeverityC and the psych x SeverityC variables are equal to zero. An overall test of statistical significance for psychotherapy (Psych) was done by testing the joint hypothesis that both Psych and Psych x SeverityC = 0. ∗ p < .05. ∗∗ p < .01. ∗∗∗ p < .001.

In Table 4, the simple incremental net benefit estimate of $200 falls to $164 in the more complex model. When one conducts a multiple regression analysis with real data— in contrast to the hypothetical data in this case—changes in coefficient values between the simple and the multiple regressions (as presented in Table 4) are expected. This is possible because of the addition of other independent variables in the regression equation, and because missing data may lead to the exclusion of a few subjects’ data. In this hypothetical example there are no missing data, so the reduced estimate of cost-effectiveness (from $200 to $164) is a result of adjusting for severity. The positive coefficient on the interaction term psych × SeverityC indicates that as disease severity increases, psychotherapy is associated with greater net benefit. Based on the interaction term’s coefficients in the effect and cost regression equations (see Table 4, multiple regression columns), the increase in cost-effectiveness stems from psychotherapy’s increasing effectiveness and decreasing overall cost for patients with more severe cases of PTSD. The statistical significance of the novel psychotherapy method was tested in the net benefit regression using a joint F -test of γ psych and γ p × s (the coefficients on the

variables psych and psych × SeverityC ). The overall effect of psychotherapy was statistically significant at the 98% confidence level, F (2,101) = 4.08, p < 0.05. These results were confirmed with net benefit regressions with other values of willingness to pay (results not shown).

Recommendations Regardless of the type of economic analysis, two key considerations are the time horizon and analysis perspective. In a standard cost-effectiveness analysis, the time horizon for valuing costs and benefits is the patient’s remaining lifetime. In cases such as smoking cessation where nearly all benefits accrue many years after the intervention takes place, counting costs and benefits over the patient’s remaining lifetime is necessary to appreciate the full outcome of the intervention. Analysts typically use a discount rate of 3% or 5% to make costs and benefits occurring in different time periods commensurate. With a 3% discount rate, for instance, a $1,000 cost savings one year from today is worth $1,000/1.03$970 today.

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

794

Hoch and Smith

The perspective for the analysis is another critical choice. Comprehensive analyses use a societal perspective showing benefits and costs accruing to payers, providers, patients, and other members of society. Adopting a societal perspective is critical for assessing the full impact of an intervention. It is commonly required for publication of costeffectiveness analyses in major journals. Additional analyses from other perspectives, such as that of the provider, may also be of interest and can be performed using a subset of the data collected from the societal perspective. Many checklists exist for evaluating the quality of economic evaluations (e.g., (Drummond & Jefferson, 1996; Drummond, Richardson, O’Brien, Levine, & Heyland, 1997; Evers, Goossens, de Vet, van Tulder, & Ament, 2005; O’Brien, Heyland, Richardson, Levine, & Drummond, 1997). The criteria are based on a commonsense approach that would be appropriate for any study on patient outcomes. From these guidelines, we suggest five key questions. We end by touching briefly on each question. Key questions. Would the trade-off between costs and outcomes help you make a decision? This question addresses the need for an economic analysis. If the answer is “yes” then an economic analysis is indicated. In some cases, however, cost is not a significant factor in deciding whether to offer a new treatment. For example, any treatment modality that cured PTSD would be accepted by every provider unless its cost were truly astronomical. When available treatments have limited effect and quality of life is severely diminished, cost will not likely play a major role in determining whether an effective new treatment will be provided. Lastly, when the time horizon or the perspective for the analysis is not congruent with the decision maker’s needs, the estimated trade-off between costs and outcomes will be of limited value. If the outcomes studied, the time horizon, and the perspective are relevant for the decision maker, then the estimated cost-effectiveness ratio will be useful. Were appropriate alternatives examined? Once it has been determined that an economic analysis is justified, selecting the comparator(s) is arguably the most critical aspect of study design. If a different type of psychother-

apy is more relevant or a different type of medication is standard care, then the cost-effectiveness results from our hypothetical study would be of little use. Are costs and outcomes appropriately selected and valued? The credibility of a study will be diminished if important cost elements or patient outcomes are omitted. There are several legitimate approaches to micro-costing and average costing, some of which appear in Table 2. For a fuller description of which costs to include, see Drummond (2005) and Gold et al. (1996). Outcomes should also be selected with care; QALYs are most appropriate for some audiences, clinical measures for others. The example analyzed patient-level data from a trial whose primary endpoint was measured at 12 months. In some cases a longer study period may be needed to develop clinical meaningful results. Likewise, when patient-level data are unavailable, models can be constructed from results reported in the scientific literature. The analyst must be careful to note differences in population, time horizon, and outcome measures across studies, as they can make results difficult to compare. Did the analysis adequately account for uncertainty? Statistical uncertainty should be reflected in a number of ways. The first is reporting confidence intervals for regression coefficients and goodness-of-fit statistics for regressions. Sensitivity analyses represent a second important method. By varying the values of key variables, one provides important evidence on the generalizability of the results to other populations as well as testing how sensitive the results are to assumptions (e.g., would medication be much more economically attractive if its price were cut by 10%?). More advanced methods for representing uncertainty include cost-effectiveness acceptability curves (Fenwick & Byford, 2005; Fenwick, O’Brien, & Briggs, 2004; Lothgreny & Zethraeus, 2000; van Hout, Al, Gordon, & Rutten, 1994) and two-dimensional cost-effectiveness planes (Fenton, Hoch, Herrell, Mosher, & Dixon, 2002). The methods to create them are beyond the scope of this introductory article. A technical report illustrating them for this hypothetical example is available from the authors. Nevertheless, they are becoming more common in prominent medical and economics journals (Fenwick,

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

Methods for Cost-Effectiveness Analysis of Person-Level Data

Marshall, Levy, & Nichol, 2006; Hoch, Rockx, & Krahn, 2006). Are additional types of analyses indicated? Different analyses suit different audiences. Publication in a health economics or health services journal may require a costutility analysis performed according to standard methods. Clinicians may be more swayed by analysis that uses a few clinical outcomes they find familiar. Clinical managers are consumers of both of these types of analysis. They will also appreciate an accurate assessment of the net budgetary impact of a new intervention (Trueman, Drummond, & Hutton, 2001), sometimes called a “business case analysis.” These value only the payer’s costs and benefits and use a short time horizon (Nicholson et al., 2005). Finally, when reading an economic evaluation, it is important to consider the possibility of bias introduced through the publication process (Baker, Johnsrud, Crismon, Rosenheck, & Woods, 2003; Bell et al., 2006). Additional reference information related to mental health economic evaluation can be found in textbooks on the subject (Hargreaves, Shumway, Hu, & Cuffel, 1998; Magruder & Miller, 1999).

CONCLUSIONS Scarcity in health care forces choice about what resources will be used to treat whom. By examining both the extra cost and extra effects of a novel treatment, economic evaluations estimate the “value for money.” It may seem harsh to deny treatments that are more effective based on the results of a cost-effectiveness analysis, and this could easily occur with an incremental cost-effectiveness ratio that is very large or an incremental net benefit that is only positive for very high willingness to pay. Is it worse, however, than wasting precious health care resources on inferior investments in health care? The costs to consider are not only the money that is spent, but also the potential patient outcomes forgone. The analyses described above do not exhaust the potential contribution of economics to trauma studies. Labor economic outcomes such as disability status, hours of employment, and earnings constituted are often considered

795

important secondary outcomes of treatment for persons with psychiatric conditions. Economists also study health care financing, a topic of particular interest in mental health services due to longstanding caps on payments for mental health treatment in many private U.S. insurance plans. A broader topic is the political economy of mental health treatment. The American experience in Vietnam and other conflicts shows that many combat veterans will suffer from PTSD for at least a short period. Using economic analysis to evaluate treatments for PTSD and other types of trauma, we will be able to make better treatment decisions. Each health care dollar can only be spent once; economic evaluation can help inform and improve our choices.

REFERENCES Arnesen, T., & Trommald, M. (2004). Roughly right or precisely wrong? Systematic review of quality-of-life weights elicited with the time trade-off method. Journal of Health Services Research and Policy, 9, 43–50. Baker, C. B., Johnsrud, M. T., Crismon, M. L., Rosenheck, R. A., & Woods, S. W. (2003). Quantitative analysis of sponsorship bias in economic studies of antidepressants. British Journal of Psychiatry, 183, 498–506. Bala, M. V., & Zarkin, G. A. (2000). Are QALYs an appropriate measure for valuing morbidity in acute diseases? Health Economics, 9, 177–180. Bell, C. M., Urbach, D. R., Ray, J. G., Bayoumi, A., Rosen, A. B., Greenberg, D., et al. (2006). Bias in published cost effectiveness studies: Systematic review. British Medical Journal, 332(7543), 699–703. Blake, D. D., Weathers, F. W., Nagy, L. M., Kaloupek, D. G., Gusman, F. D., Charney, D. S., et al. (1995). The development of a Clinician-Administered PTSD Scale. Journal of Traumatic Stress, 8, 75–90. Chan, A. O., Medicine, M., Air, T. M., & McFarlane, A. C. (2003). Posttraumatic stress disorder and its impact on the economic and health costs of motor vehicle accidents in South Australia. Journal of Clinical Psychiatry, 64, 175–181. Chisholm, D., Healey, A., & Knapp, M. (1997). QALYs and mental health care. Social Psychiatry and Psychiatric Epidemiology, 32(2), 68–75. Cohen, H. E. (2004). Foreward. Red Book 2004.

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

796

Hoch and Smith

Devlin, N., & Parkin, D. (2004). Does NICE have a costeffectiveness threshold and what other factors influence its decisions? A binary choice analysis. Health Economics, 13, 437–452. Drummond, M. F. (2005). Methods for the economic evaluation of health care programmes (3rd ed.). New York: Oxford University Press. Drummond, M. F., & Jefferson, T. O. (1996). Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. British Medical Journal, 313(7052), 275–283. Drummond, M. F., Richardson, W. S., O’Brien, B. J., Levine, M., & Heyland, D. (1997). Users’ guides to the medical literature. XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid? Evidence-Based Medicine Working Group. Journal of the American Medical Association, 277, 1552–1557. Evers, S., Goossens, M., de Vet, H., van Tulder, M., & Ament, A. (2005). Criteria list for assessment of methodological quality of economic evaluations: Consensus on health economic criteria. International Journal of Technology Assessment and Health Care, 21, 240–245. Fenton, W. S., Hoch, J. S., Herrell, J. M., Mosher, L., & Dixon, L. (2002). Cost and cost-effectiveness of hospital vs residential crisis care for patients who have serious mental illness. Archives of General Psychiatry, 59, 357–364. Fenwick, E., & Byford, S. (2005). A guide to cost-effectiveness acceptability curves. British Journal of Psychiatry, 187, 106–108. Fenwick, E., Marshall, D. A., Levy, A. R., & Nichol, G. (2006). Using and interpreting cost-effectiveness acceptability curves: An example using data from a trial of management strategies for atrial fibrillation. BMC Health Services Research, 6, 52. Fenwick, E., O’Brien, B. J., & Briggs, A. (2004). Cost-effectiveness acceptability curves—Facts, fallacies and frequently asked questions. Health Economics, 13, 405–415. George, B., Harris, A., & Mitchell, A. (2001). Cost-effectiveness analysis and the consistency of decision making: Evidence from pharmaceutical reimbursement in Australia (1991 to 1996). Pharmacoeconomics, 19, 1103–1109. Gold, M. R., Siegel, J. E., Russell, L. B., & Weinstein, M. C. (1996). Cost-effectiveness in health and medicine. New York: Oxford University Press. Hargreaves, W. A., Shumway, M., Hu, T. W., & Cuffel, B. (1998). Cost-outcome methods for mental health. San Diego: Academic Press. Hirth, R. A., Chernew, M. E., Miller, E., Fendrick, A. M., & Weissert, W. G. (2000). Willingness to pay for a quality-adjusted

life year: In search of a standard. Medical Decision Making, 20, 332–342. Hoch, J. S. (2004). Cost-effectiveness lessons from diseasemodifying drugs in the treatment of multiple sclerosis. Expert Review of Pharmacoeconomics & Outcomes Research, 4, 537– 547. Hoch, J. S., Briggs, A. H., & Willan, A. R. (2002). Something old, something new, something borrowed, something blue: A framework for the marriage of health econometrics and cost-effectiveness analysis. Health Economics, 11, 415– 430. Hoch, J. S., Rockx, M. A., & Krahn, A. (2006). Using the net benefit regression framework to construct cost-effectiveness acceptability curves: An example using data from a trial of external loop recorders versus Holter monitoring for ambulatory monitoring of "community acquired" syncope. BMC Health Services Research, 6, 68. Hoff, R. A., & Rosenheck, R. A. (1998). Long-term patterns of service use and cost among patients with both psychiatric and substance abuse disorders. Medical Care, 36, 835–843. Krabbe, P. F., Essink-Bot, M. L., & Bonsel, G. J. (1997). The comparability and reliability of five health-state valuation methods. Social Science and Medicine, 45, 1641–1652. Lothgren, M., & Zethraeus, N. (2000). Definition, interpretation and calculation of cost-effectiveness acceptability curves. Health Economics, 9, 623–630. Magruder, K. M., & Miller, N. E. (1999). Cost-effectiveness of psychotherapy: A guide for practitioners, researchers, and policymakers. New York: Oxford University Press. McCrone, P., Knapp, M., & Cawkill, P. (2003). Posttraumatic stress disorder (PTSD) in the Armed Forces: Health economic considerations. Journal of Traumatic Stress, 16, 519–522. Nicholson, S., Pauly, M. V., Polsky, D., Baase, C. M., Billotti, G. M., Ozminkowski, R. J., et al. (2005). How to present the business case for healthcare quality to employers. Applied Health Economics and Health Policy, 4, 209–218. O’Brien, B. J., Heyland, D., Richardson, W. S., Levine, M., & Drummond, M. F. (1997). Users’ guides to the medical literature: XIII. How to use an article on economic analysis of clinical practice: B. What are the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group. Journal of the American Medical Association, 277, 1802– 1806. Shumway, M. (2003). Preference weights for cost-outcome analyses of schizophrenia treatments: Comparison of four stakeholder groups. Schizophrenia Bulletin, 29, 257–266.

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.

Methods for Cost-Effectiveness Analysis of Person-Level Data

797

Smith, M. W., & Joseph, G. J. (2003). Pharmacy data in the VA health care system. Medical Care Research and Review, 60(Suppl 3), 92S–123S.

van Etten, M. L., & Taylor, S. (1998). Comparative efficacy of treatments for posttraumatic stress disorder: A meta-analysis. Clinical Psychology & Psychotherapy, 5(3), 126–144.

Stinnett, A. A., & Mullahy, J. (1998). Net health benefits: A new framework for the analysis of uncertainty in cost-effectiveness analysis. Medical Decision Making, 18(Suppl 2), S68–S80.

van Hout, B. A., Al, M. J., Gordon, G. S., & Rutten, F. F. (1994). Costs, effects and C/E-ratios alongside a clinical trial. Health Economics, 3, 309–319.

Tambour, M., Zethraeus, N., & Johannesson, M. (1998). A note on confidence intervals in cost-effectiveness analysis. International Journal of Technology Assessment in Health Care, 14, 467–471.

Weathers, F. W., Keane, T. M., & Davidson, J. R. (2001). Clinician-Administered PTSD Scale: A review of the first ten years of research. Depression and Anxiety, 13, 132– 156.

Trueman, P., Drummond, M., & Hutton, J. (2001). Developing guidance for budget impact analysis. Pharmacoeconomics, 19, 609–621. Ubel, P. A., Hirth, R. A., Chernew, M. E., & Fendrick, A. M. (2003). What is the price of life and why doesn’t it increase at the rate of inflation? Archives of Internal Medicine, 163, 1637–1641.

Weisbrod, B. A., Test, M. A., & Stein, L. I. (1980). Alternative to mental hospital treatment: II. Economic benefit-cost analysis. Archives of General Psychiatry, 37, 400–405. Williams, A. (1992). Cost-effectiveness analysis: Is it ethical? Journal of Medical Ethics, 18, 7–11.

Journal of Traumatic Stress DOI 10.1002/jts. Published on behalf of the International Society for Traumatic Stress Studies.