Disclosure of Information about Health Care Quality: Review of ...

10 downloads 1837 Views 122KB Size Report
Feb 10, 2008 ... Edmonston, author of the popular Lemon-Aid guides to new and used cars), and a diverse panel that included a practising family physician, the ...
Disclosure of Information about Health Care Quality: Review of Research and Expert Opinion

Tanya Verrall, Ph.D., P.Dt., Ben Chan, MD MPH MPA, Jack Wallace, MA, Gary Teare, Ph.D., Bonnie Brossart, MA Health Quality Council February 2008

1 Health Quality Council Principles on Disclosure of Information about Health Care Quality



The Health Quality Council (HQC) is a steward of the public interest. An integral part of this role is our legislated mandate to report publicly on the quality of health care in Saskatchewan.



We advocate disseminating information about health care quality in a manner that helps providers and decision-makers better manage and deliver care, and equips members of the public to fulfill their roles as citizens and health care users.



As an independent organization with expertise in quality measurement and reporting, HQC produces reports that stimulate action to improve the quality of health care in Saskatchewan. We do this by working with health system stakeholders and members of the public to: o determine which audience(s) can make the greatest impact on quality in response to reports; o select and develop appropriate quality indicators; o ensure the highest standards for data quality; o understand the capacity and limitations of the data; o apply the most suitable methodology for analysis; and, o design reporting products that convey clear, accurate, actionable messages about health care quality.

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

2 Executive Summary The Health Quality Council (HQC) is a steward of the public interest. Established in 2002, we are an independent organization with a legislated mandate to report publicly on the quality of care in Saskatchewan.

Since our inception, we have reported on many aspects of the health care system, and worked with our partners on various quality improvement initiatives. It became evident through our early reporting experiences that stakeholders varied in their expectations around public reporting. We also recognized that, given our mandate to do ongoing measurement and reporting, we had a responsibility to be clear and forthcoming with our health system partners about our principles on disclosure of information about health care quality.

In late 2005, we did a thorough review of the literature on public reporting; and then updated this review in 2007. We looked at the experiences of health care systems around the world, including: •

fundamental reasons for public reporting;



merits of and challenges to public reporting; and,



key issues that must be considered when publicly reporting on quality of health care.

Public reporting on goods and services has become increasingly common in our society, in both private industry (through such mechanisms as Consumer Reports) and the public sector (e.g., Maclean’s magazine annual report on Canadian universities1). While public reporting on health care quality is still in its infancy in Canada, health care systems in the United Kingdom (UK) and United States (US) have been disclosing increasingly detailed information for more than two decades. Both jurisdictions have reported publicly on quality of health care – by country, region, organization, and even individual provider – to increase accountability of the health care system and encourage improvement in health care quality.

Although it is widely recognized that research on and evaluation of the effects of public reporting have not kept pace with its growth, there is a consensus among researchers and commentators that the practice is necessary and inevitable. As a result, the focus in the literature on public

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

3 reporting has shifted from articles of opinion about whether and what to report on health care quality, to an analytical exploration of how to most effectively use reporting to improve quality.

In 2006 we shared an earlier draft of this background paper with our health system stakeholders and the public through two events: a public meeting in March 2006, and a day and a half-long discussion forum in October 2006. The intent of these sessions was to raise awareness and understanding about public reporting on health care quality, discuss the opportunities and challenges facing Saskatchewan, and inform development of principles to guide our reporting.

Based on these discussions and evidence from our review of both opinion and research in the literature, we developed the following principles: •

The Health Quality Council (HQC) is a steward of the public interest. An integral part of this role is our legislated mandate to report publicly on the quality of health care in Saskatchewan.



We advocate disseminating information about health care quality in a manner that helps providers and decision-makers better manage and deliver care, and equips members of the public to fulfill their roles as citizens and health care users.



As an independent organization with expertise in quality measurement and reporting, HQC produces reports that stimulate action to improve the quality of health care in Saskatchewan. We do this by working with health system stakeholders and members of the public to: o determine which audience(s) can make the greatest impact on quality in response to reports; o select and develop appropriate quality indicators; o ensure the highest standards for data quality; o understand the capacity and limitations of the data; o apply the most suitable methodology for analysis; and, o design reporting products that convey clear, accurate, actionable messages about health care quality.

We stand firm in our commitment to the principle that constructive information on the quality of health care must be available and accessible to all who have a role in improving health care quality. Moreover, we expect that the opinions of stakeholders in Saskatchewan will continue to evolve, as they have in other jurisdictions.

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

4 1.

INFORMED PUBLIC REPORTING

As part of its legislated mandate, the Health Quality Council (HQC) is required to report publicly on the quality of health care in Saskatchewan. Because we recognized from the outset the many challenges inherent in this activity, we made consultation with stakeholders an integral part of our early reporting initiatives. Through these initial experiences we learned that our partners vary considerably in their opinions about and attitudes toward public reporting. It was clear that we needed to engage stakeholders in further discussion on the topic.

The question facing HQC is not whether we should report publicly, but how best to carry out this important activity. Therefore, the aim of this paper is to provide a basis for informed decision making for HQC, its health system partners, and the public regarding this issue. Our approach to reporting will likely evolve over time, in response to research and experience in Saskatchewan and other jurisdictions.

In 2005, an earlier draft of this paper stimulated province-wide discussion about the challenges and opportunities associated with measuring and reporting on quality of health care. What follows is an updated review of the research literature on public reporting in Canada, the United States, and the United Kingdom, and a description of the ways we have engaged our audiences in a dialogue about this important aspect of our work.

1.1

Introduction to the literature

Public reporting has become increasingly common in today’s society. Rapid advances in communication technology have made information readily available to consumers on a variety of topics. Increasingly, the general public can access from a variety of sources reports about the quality of the cars they drive, the best dishwasher to buy, and the nutrient value of food.2 This trend toward greater public reporting has been driven by initiatives of both the business and financial world—to protect consumer interests as well as to report financial information for publicly traded companies. Public reporting has also expanded into other areas such as education, where performance ‘report cards’ are published at the school, district, and regional levels.3 In Canada, comparative information about the performance of university institutions is published annually in the popular press.1 _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

5

Despite increased reporting in a variety of disciplines, there is relatively little information available publicly about a basic element of our society: the health care system.2 There may be several reasons for this, among them the fact that the ‘institution’ of health care has been “underpinned by an implicit trust on the part of the general public.”4 There is overwhelming evidence, however, that the public’s level of trust in the health care system is diminishing. This phenomenon is not limited to Canada, and has been seen in other developed countries such as the United States (US) and the United Kingdom (UK).

Recent initiatives in Canadian health care reform have included public consultations through various federal and provincial task forces and commissions. A key recommendation from these consultations is to increase public reporting on health system performance as a means of instilling confidence and strengthening accountability.5 In Saskatchewan, citizens appear to be demanding more engagement with and information about the quality of the health care system. The Patient’s Rights Association of Saskatchewan (PRAS) announced late in 2005 that a draft Patient’s Bill of Rights has been submitted to the legislature.6 The group maintains that there is a growing discontent among Saskatchewan citizens who want to be better informed and engaged in improving the system.6 The message from the public seems clear: the health care system’s paternalistic approach to the people it serves is no longer acceptable.7 As taxpayers, people have a right to know how their money is being spent and how the health care system is performing. As citizens in a democratic society, the public has a right to be informed so they can engage in health policy reform.

The Health Quality Council shares in the responsibility to strengthen public accountability. As an independent agency formed in response to recommendations for health care reform, we are mandated to report to the public about the quality of Saskatchewan’s health care system. However, we also are guided by our mission, “to improve the quality of care…through our close working relationships with our health system partners. We support, nurture, inspire, and act as a catalyst for quality improvement activities throughout the system.”8 Thus, our reasons for public reporting move beyond simply strengthening accountability to include reporting as a key tool for quality improvement. The literature suggests that effective accountability is best achieved _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

6 through a reporting system that encompasses both performance reporting and an associated “corrective process.”9

1.2

Scope of literature review

To maintain brevity, this literature review focuses on experiences in public reporting about health care performance over the past two decades. Most of the research is American, though some comes from the United Kingdom and Canada.

2.

OVERVIEW OF PUBLIC REPORTING OF HEALTH CARE PERFORMANCE

2.1

Brief history of public reporting

The literature cites two main examples of early public reporting. In the 1860s, Florence Nightingale published standardized mortality data that showed higher mortality rates among hospitalized British soldiers than those fighting in battle. Response from the Royal College of Surgeons was not favourable, however, and the work was suspended in 1866.5 There was also resistance in the United States to early efforts to report publicly on health outcomes. In the early 1900s, a physician in Massachusetts, Ernest Codman, began to report on health outcomes. Codman’s medical colleagues did not share his commitment to reporting, so he opened his own hospital; Codman reported on health outcomes for this new facility.5,10,11

The impetus for today’s public reporting of information about quality of health care services has its roots in the United States, where the practice began as part of efforts to contain health care costs.12 However, a shift has occurred over the past two decades, from collecting and reporting for financial purposes, toward improving the availability and utility of information related to health care quality.12,13

2.2

Fundamental reasons for public reporting

It is difficult to distinguish between a fundamental reason for and an objective of public reporting—the literature refers to these two terms interchangeably. However, one of the most commonly cited rationales for reporting is to motivate change that will improve the quality of the system13 In broader terms, public reporting of health care performance occurs for two main

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

7 reasons: to increase accountability of the health care system and to encourage improvements in health care quality.

2.2.1

Increase public accountability

Increasing public accountability of the health care system can be viewed as a “static” rationale for reporting. That is, the public’s right to know may be considered sufficient justification for publishing information about the health care system, regardless of the consequences.11 However, if one considers motivating improvement to be an underlying rationale, then a more dynamic viewpoint of accountability applies. By increasing accountability, the public becomes better informed about the quality and performance of the health care system. This facilitates the public to act directly through individual choice of provider (i.e., consumers of health care) to optimize their quality of care, or indirectly through political channels (i.e., exercising their rights as a citizen and taxpayer) to create improvement by participating in health care policy reform.13

2.2.2

Encourage improvements in quality of care

This rationale is central to public reporting of health care performance. It is theorized that public reporting of performance serves as a catalyst for improvement in the health care system.11 Providing comparative data on performance to providers (i.e., clinicians, institutions, and health regions) facilitates identification of best practices and focuses attention on areas for improvement. Providing this same information to the public may increase the responsiveness and motivation of providers to engage in quality improvement projects.11,14

3.

CURRENT PRACTICES IN PUBLIC REPORTING

Reporting on health care performance has evolved over the past two decades in North America and the UK and is now recognized as an established practice. This evolution is the result of an increased commitment to quality improvement, an increase in public expectations for accountability, and improvements in the data systems that have been used to measure performance.5

A number of reporting systems exist in the US, and to a lesser extent in the UK and Canada. These initiatives vary in their objectives, target audiences, types of data produced, and reporting _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

8 formats. In the US, a large number of public reports aim to influence consumer choice of physician provider, hospital, and health insurance plans. However, in the UK and Canada reporting efforts tend to focus more on accountability to citizens and quality improvement.5 In some cases, the government has directed the reporting process, but there are also a significant number of reports that are produced by independent agencies—either groups that are arm’slength from government or, particularly in the US, groups that are privately funded.5 The following sections describe key public reporting examples from each country—highlighting initiatives from both governmental and non-governmental groups as well as the diversity in reporting objectives, target audiences, and reporting formats.

3.1

United States

Public reporting on the quality of health care in the United States is done for a variety of reasons, among them controlling costs, ensuring accountability, regulating health care providers, and promoting competition between health providers based on the quality of care that they provide.2,15 Over the past two decades, the breadth and depth of information about the quality of health care in the United States has grown considerably and as a result this country has played a dominant role in shaping current public reporting efforts. A recent review provides numerous examples of public reporting initiatives focused on quality of health care delivered within different health plans (i.e., insurers and integrated delivery systems), hospitals, medical groups (general and specialty), nursing homes, dialysis centres, and by individual physicians.15

The most comprehensive and continuous information on the quality of health care is generated by managed care plans. Since the early 1990s, the National Committee for Quality Assurance (NCQA) has collected and reported publicly on selected processes and outcomes of care, patient experience, as well as results from accreditation activities for managed care plans across the United States.15 Information on the quality of care provided by hospitals began being reported publicly in 2004 by the Centers for Medicare and Medicaid Services. While this effort was initially voluntary, recent legislation and financial incentives spurred virtually all eligible hospitals to participate by 2005.15

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

9 Perhaps the most notable public reporting in the US over the past decade is of mortality rates for coronary artery bypass graft operations (CABG), first implemented by New York State’s Department of Health in the early 1990s.16 New York State’s Cardiac Reporting System reports risk-adjusted, in-hospital and physician-level rates for CABG surgery for all hospitals and providers in the state. Two other states, Pennsylvania and California, have also reported on this procedure.17,18

In New York State, the Department of Health collected data from hospitals on patients who had undergone CABG surgery. Mortality rates, risk adjusted by a multivariable statistical model, were used to compare outcomes for hospitals and surgeons performing the procedure.16 The data were made available publicly in the form of a comparative report card, which did not initially disclose surgeon-specific results. Surgeon-specific data were made available a year and a half later, after Newsday sued the Department under the Freedom of Information Act.11 Many clinicians refused to submit identifiable data, thereby jeopardizing the initiative. However, an agreement was reached whereby only high-volume surgeons (defined as performing more than 200 operations in the previous 3-year period) would be included in the report.11 Better risk-adjustment procedures and data quality have made this annual reporting system an important quality improvement tool that informs the public and enables hospitals and surgeons to compare levels of performance.18

3.2

United Kingdom

There are fewer publications about reporting in the UK than in the United States. Although reporting practices in the UK have historically had the goal of improving health quality, there has been a shift more recently toward reporting to improve accountability and consumer choice. The National Health Service (NHS) has initiated most public reporting initiatives through various arm’s-length organizations, such as the Commission for Health Care Audit and Inspection (i.e., Health Care Commission), which is mandated to focus on both quality and accountability.5

In 1991 the NHS introduced a Patient’s Charter, which included the first set of performance indicators intended to help consumers assess the performance of their health services. This information focused primarily on access to care (i.e., waiting times) rather than clinical _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

10 quality.13,19 The first hospital outcome data were published in the early 1990s in Scotland in the form of a hospital report card. This information was not disseminated widely, however, because of concerns that the media and clinicians would respond unfavourably.13

The NHS launched the Performance Assessment Framework in 1998, to coordinate a more purposeful approach to public reporting.13 This framework has produced a number of reporting initiatives, including the development of the NHS performance ratings system in 2001. For this reporting system, all hospitals are placed into one of four categories (ranging from three to zero stars) based on their performance. Comparisons are made to key national targets using a balanced scorecard of measures covering patient care (e.g., percentage of patients waiting less than six months for treatment), clinical care (e.g., mortality rates within 30 days of surgery for patients admitted on an unplanned basis), and staff issues (e.g., rates of staff vacancies).19 The aim of this comparative reporting system is to highlight strong and weak performers in order to motivate quality improvement. It also functions to improve public accountability.19 Report production is managed by the Health Care Commission, an independent organization with a mandate to promote and drive improvement in the quality of health care and public health.20 The reports, targeted at both providers and the public, are available mainly through the Commission’s website.

More recently, a significant reporting system has been initiated through a private, independent source. This source, called Dr. Foster, was established in 2000 to report comparative performance information on all public and most private health care providers in the UK.13 It is managed by physicians, who analyze the data, and by journalists, who are responsible for communicating the results.5 Information includes comparisons to best practice guidelines, number of staff, waiting times, number of complaints, and available services.13,21 The reports are targeted at both health care providers and the general public, and are primarily disseminated through a web site as well as through newspaper supplements.13 Provider level data are also available through the Society of Cardiothoracic Surgeons in the UK. This group has published mortality rates for individual operating units on their website;13 while they plan to eventually report mortality rates for individual surgeons, they currently do not have the data required to do so.22

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

11 3.3

Canada

Canada’s experiences in public reporting of health care performance are more limited than either the UK or US. As in the UK, the majority of public reporting has focused on ensuring accountability, improving the quality of health care delivery, or both.5 To date, federal and provincial/territorial governments and related organizations have played a primary role in directing public reporting activities.

At a national level, public reporting of health care performance has been undertaken by a few groups. One of the key organizations is the Canadian Institute of Health Information (CIHI), which was formed in 1994. CIHI is an independent, non-profit organization that acts as a “focal point for collaboration among provincial governments, health professional associations, researchers, regional health authorities, and individual hospitals.”23 It maintains national health care data sets and reports to the public on the performance of the health care system. In November 2007, CIHI released its first annual nation-wide public report on hospital standardized mortality ratios.23

There is a great deal of variation in public reporting initiatives across the different provinces and territories. This is likely because health care in Canada is a provincial responsibility.5 Public reports have been produced annually on specific conditions, procedures, or waiting times. It is most common to find information regarding waiting times for specific procedures, classified by regional health authority, hospital, and surgeon (i.e., Alberta’s waiting list registry; Saskatchewan Surgical Care Network, SSCN). This information is mostly accessible to the public through provincial government websites.5 Although there is no empirical evidence on the impact of these initiatives (e.g., SSCN), anecdotal observations suggest that awareness and access of these websites is growing.

Canada’s best known example of hospital report cards is produced in Ontario. Since 1998, the government of Ontario and the Ontario Hospital Association, recently in collaboration with CIHI, have produced a series of reports that use balanced scorecards to assess outcomes and activities in the areas of financial performance, clinical utilization and outcomes, system integration and change, and patient satisfaction.24,25 The information published in these reports is derived from _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

12 administrative data and surveys of hospital management. Participation by hospitals is voluntary.5 The reports, which compare performance across hospitals and identify benchmarks, are intended to assist hospital managers with quality improvement.24,25

In addition, Saskatchewan’s Health Quality Council represents one of three provincial organizations that are mandated to evaluate and report on health care performance, with the overall aim of quality improvement. The other organizations are located in Alberta and Ontario. The Health Quality Council of Alberta focuses its activities in five key areas: patient safety, collaboration, primary care, complaint handling, and annual reporting. It produces an annual report on the health of Albertans, and to date has also produced a provincial report on patient satisfaction, a ‘best practices’ report on morphine handling practices in Alberta’s emergency departments, and a report on medication safety that was widely distributed across the province.26 The Ontario Health Quality Council is mandated to produce a report each year – QMonitor – which focuses on accessibility, safety, effectiveness, patient-centeredness, equity, efficiency, resources, and the integration of prevention and primary care. To date OHQC has produced two such reports.27

4.

CURRENT ISSUES REGARDING PUBLIC REPORTING OF HEALTH CARE PERFORMANCE

4.1

Impact of public reporting on quality improvement

Public reporting has grown more rapidly than research and evaluation to support it.11,15,28,29 A systematic review of the literature by Marshall et al.11 in 2000 found only seven reporting systems that had been subject to formal evaluations which were subsequently published in peerreviewed journals. Furthermore, many of these evaluations had weak study designs.11 In 2008, Fung et al.29 conducted another systematic review, to build on the review done by Marshall and colleagues in 2000. They found that there was still limited evidence about the impacts of public reporting, and that differences in study designs made comparing the research difficult. Fung also concluded that the research focussed heavily on public reporting programs that have already been evaluated. There is evidence that publishing performance data can generate discussion and debate that facilitates improvements in analytical procedures and dissemination efforts.12 However, more _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

13 evaluative research is necessary to further understand how to make public reporting more beneficial.5,10,11,28 In particular, more work needs to be done to design public reports to be relevant and useful for their target audiences, and evaluate their effectiveness in doing so.15,29

4.1.1

Overview

Evidence from research suggests that public reporting of health care performance can bring about improvement through changes in public, professional, and institutional behaviour.30,31 However, the relative impact among these key audiences differs, with minimal impact on individual (i.e., public/consumer) behaviour and more promising results associated with the response of the provider (i.e., institutional).11,13,28 The following framework, adapted from Marshall et al.,11 illustrates the potential impact that public reporting may have on provider/public behaviour and quality improvement (figure 1). A more detailed discussion of the evidence follows. Fig. 1 Potential impact of public reporting (Adapted)11 Information Needs

Attitudes

Resources

Performance Data for Public Release Content Rigour Presentation Format Mode of Release

Impact on Behaviour of Physicians/Clinicians

Impact on Behaviour of Provider Organizations

Impact on Behaviour of General Public (Consumers)

Impact on Quality of Care Outcomes

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

14

4.1.2

General public

Evidence suggests that increased access to comparative performance information has not changed consumer behaviour (i.e., choice of provider).5,15,28 A review by Marshall et al.11 found only one study that suggested public reporting had an impact on decision-making by consumers.11 An evaluation of the impact of the Pennsylvania CABG Guide found that, of the 12 per cent of cardiac patients surveyed who were aware of the guide, a handful stated that it influenced their selection of provider.30 Hibbard et al.14 found that two years after the release of a public report on quality of hospital care in Wisconsin, only a small percentage of the general public were able to recall the report and any information that it contained. In a systematic review of the literature, Fung et al.29 found the evidence is mixed on whether the public uses quality reports to choose providers.

Various reasons have been proposed to explain the limited effect of public reporting on the public’s behaviour, including that consumers do not understand the indicators, how to interpret them, or how they were developed or selected.11,14,15,29 As well, it has been suggested that reported indicators are of little relevance to the public and that there has not been enough effort made to assess the information needs of the public.32,33 Researchers have also found that the public tend to rely on the advice of family and friends or their own personal experiences in selecting providers, rather than formal reports.10,12 In the UK, it has been suggested that the public may not want to view their health care in terms of a market commodity.34 Within certain health care systems, such as those in Canada and the UK, the public may not use comparative data for selection because citizens either do not have – or do not believe they have – choice over which providers they receive care from.5,10

Evidence from a more recent American study suggests that the public may indirectly influence quality improvement; this research found providers may address issues flagged in reports in order to improve their public image or reputation. Hibbard et al.14 observed a shift in public opinion regarding ‘better’ or ‘worse’ performing hospitals during a four-month period after the release of comparative performance data. Moreover, there was evidence that those who stated they had seen the report, had already talked to others, or intended to talk to others, about what they read.14 _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

15 Similarly, there is evidence that hospitals may improve quality of care in order to protect their public image.35

4.1.3

Individual providers (i.e., physicians/clinicians)

A great deal has been written about the medical profession’s resistance to public reporting of performance data. Earlier reporting attempts were met with distrust by physicians, who viewed the data as unreliable and the analyses as methodologically flawed.13 There were, in fact, problems with data quality and analysis methods in some early reporting initiatives. As well, many reports were based on stale or outdated data.5 Subsequent reporting efforts, however, have addressed many of these concerns.5,16

Public reporting has also been shown to have a limited impact on physician behaviour. Physicians appear to be aware of reports containing comparative data, but only a small number use these reports in their practice, either for sharing with their patients or in deciding where to refer patients.11,29,30 However, there is evidence that this impact may be increasing. Morris and Zelmer5 cite a Canadian study that reported half of clinicians surveyed responded to a published cardiac report card by increasing quality improvement activities in their hospitals. In the UK, there is some indication that physicians involved in quality improvement collaboratives and networks are supportive of benchmarking initiatives based on publicly reported performance data.11

It is unclear why so few physicians use performance data. Some researchers have suggested that public reports challenge the traditional autonomy of the medical profession.11 Others suggest that physicians continue to distrust the reports produced by external organizations because of concerns about data quality and the relevance of measured indicators to their practice.5 Nevertheless, improving physicians’ acceptance and implementation of publicly reported performance data is regarded as a key factor in an effective reporting system.11

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

16 4.1.4

Hospitals and other provider organizations

Provider organizations, especially hospitals, appear to respond more actively to publicly reported performance data than either the public or the medical professions.11,29,35 A review by Marshall and Romano34 highlights two key studies to support this. A quasi-experimental study conducted in Missouri found hospitals that received low rankings for obstetrical care in public reports increased their delivery of related process of care programs (e.g., breastfeeding education, child car seat program, follow-up care). As well, a more recent non-randomized trial from Wisconsin found that public reporting of information on hospital performance led to significantly more quality improvement activity than reporting only to hospitals or no reporting.35 In Canada, 92 per cent of hospitals stated that they made changes to their organizational practises one year after the publication of a patient satisfaction survey.5

Evidence suggests that provider organizations will pursue quality improvement in response to public reporting, even if they disagree with the information. For example, the Wisconsin study found that the hospital that initiated the most quality improvement efforts in response to a public report also complained more than other institutions about the report’s validity and appropriateness of the information for public use. Negative views of public reporting among hospitals have been described in other studies as well.13 Better-performing hospitals are more likely to express a positive view of the report and poor-performing hospitals are more likely to respond negatively and discredit the report.11 Researchers have suggested that greater collaboration with hospitals in developing the report and providing facilities with their own version of the document prior to public release may lessen negative responses.35

4.1.5

Outcomes

While improving outcomes of care is the ultimate goal of many reporting efforts, little evaluative research has been conducted in this area.5 Attempts to design evaluative studies that can clarify the relationship between published performance data and quality of care have been difficult.34 Evidence from a few studies (quasi-experimental) suggests that public reports may improve outcomes, or at least contribute to increases in quality improvement activities.11,29 Many of these studies focused on the evaluation of the New York State CABG reporting system. For example, risk-adjusted mortality rates decreased by 41 per cent among patients undergoing CABG surgery _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

17 in New York State between 1989 and 1992. Some researchers disputed these results and claimed that the ‘improvement’ was due to clinicians refusing service to high-risk patients in order to improve their performance. However, further study did not find evidence of this.5,16

In addition, several other reporting programs have shown some improvement associated with public reports. Evaluation of an obstetrical care study from Missouri reported improvements in patient satisfaction and caesarean delivery rates that were associated with the quality improvement activities undertaken by poorly performing hospitals.34 In Wisconsin, Hibbard et al.35 found more quality improvement initiatives undertaken by hospitals that were publicly reported on than those that were not.

While publicly reporting quality information may not always result in significant improvement, it does help to identify issues that could be addressed. Goldman et al.36 found that among hospitals in the US with publicly reported quality information on the Hospital Compare website, those serving a higher number of Medicaid patients had a lower adherence to best practices. This was true whether or not the hospitals were teaching institutions. While the authors are cautious about drawing conclusions from their findings, the identified quality gaps are relevant for decision makers who are responsible for, and those patients who use, hospitals with a high Medicaid population.

4.2 Adverse consequences of public reporting There is little evidence in the literature about adverse consequences of publicly reporting on health care performance. Most research has instead focused on assessing potential benefits rather than on whether public reports negatively impact behaviour of providers.11 This lack of evidence does not mean that reporting initiatives are not without risk. Several researchers have conjectured about the potential adverse consequences of public reporting (summarized in Table 1). The table also notes the relevance of each consequence to the design of HQC’s reporting program.

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

18 Table 1. Potential adverse consequences of public reporting on health care performance (Adapted)28,37 Unintended Consequence

Definition

Relevance to HQC

– = + = ++ = Tunnel Vision Sub-optimization Myopia Convergence Misrepresentation

Ossification

Gaming Negative effect on morale of providers Negative effect on public confidence/trust in health professionals and health care system

Organizations focus on areas that are measured, to the exclusion of other important issues. Narrow focus on local objectives at the expense of broad organizational goals. Focus on short-term issues at the expense of long-term strategies. An emphasis on not being exposed as an outlier rather than a desire to be outstanding. Deliberate manipulation by a provider organization or implicit “massaging” of data. A disinclination to experiment with new and innovative approaches, for fear of appearing to perform poorly. Altered behaviour of providers to obtain strategic advantage. Loss of reputation and lowering of morale among health care providers and institutions. Explicit information about under-performance may have particularly adverse consequences in a public system where consumers have relatively little opportunity to seek care elsewhere.

Not relevant Relevant Very relevant

++ ++ + + – (potential if collecting primary data)

+ + ++ ++

What little evidence there is supports only a few of these conjectures about potential adverse effects. In this review of evaluations of earlier reporting efforts in the US (New York State CABG reporting system), Werner and Asch28 did find evidence of gaming. However, subsequent studies failed to reach the same conclusion.11 More recent evidence comes from a UK study, in which staff and managers from low- and high-rated hospitals were interviewed to explore the impact of the ‘star rating’ reporting system. Similar to previous studies, hospital staff reacted negatively toward the reports, particularly staff from low-rated hospitals.37 Evidence of misrepresentation of the data and gaming was also found, including cancelling surgeries the night _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

19 before the procedure was scheduled to take place, so the cancellation was not recorded for the scheduled day of the surgery.37 In addition, physicians reported that their clinical priorities were altered in order to meet short-term waiting list targets (tunnel vision). In their 2008 systematic review of the literature, Fung et al.29 found that while there is some evidence that reporting on individual providers causes gaming, overall, the research findings are inconclusive.

Interpreting this evidence is problematic. For example, evidence suggested that the “star” rating system in the UK (which awarded hospitals a score of zero to three stars) had unintentionally damaged staff morale and diminished public trust in providers.37 However, because this system is relatively new, the reactions reported in the literature represent only an initial response from providers.37 Other researchers have suggested that providers’ negative reactions diminish as they become more accustomed to frequent reporting.4,38 Similarly, there is evidence that organizations use public reports to motivate and engage in quality improvement activities within the areas measured, despite expressing negative reactions.35,38

Nevertheless, the potential for unintended behaviour changes in provider organizations is a critical consideration in designing a reporting system.4 Unintended outcomes are often associated with poorly designed reports that contain unreliable data or that have been released without consultation with providers.4 As a result, some proponents of reporting have advised that a balance be established between designing reports that attract sufficient attention from providers and the public, and reporting in a manner that minimizes inaccurate or unfair interpretations of data.4

4.3 The Media Some health care providers and organizations are concerned about the potentially negative impact of media coverage of public reporting.13 Marshall et al.13 claim that journalism can be sensationalist and alarmist, with the potential to misrepresent data presented in quality report cards. On the other hand, news media routinely report information from biased and sometimes anonymous sources. Such information can have worse consequences than media distortion of analytical reports from accountable organizations like HQC which could request the correction of any errors. _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

20

In 2007 the Canadian Health Services Research Foundation commissioned HQC to describe the best practices associated with reporting to the public on the quality of health care in Canada.39 As part of this project, interviews were conducted with several organizations across Canada currently reporting to the public on the quality of health care. When asked about the media, each organization responded that they saw the media as a partner in getting important information to the public. All agencies indicated that, in their experience, the media presented the information with little or no distortion, largely because they engage the media when reporting, reducing the potential for negative consequences.

4.4 Type of data and number of measures to be reported Different reporting objectives require different data to be measured and reported.12,19 In the United States, public reporting has focused primarily on changing health care consumers’ behaviour. Therefore, the information needs of the consumer (i.e., the public) have been highlighted. Critics have stated that comparative performance data often lack public relevance and focus primarily on technical quality of care rather than more direct and personal aspects of care (e.g., communication, timeliness, ease of access).4,40

Reporting initiatives aimed primarily at changing provider behaviour have also been criticized. Concerns have been raised in editorials and opinion articles in journals that performance indicators are often irrelevant to the local health care situation of providers and that there have been too few measures related to processes of care.12 Critics charge that reporting has focused too much on what is ‘easy to measure,’ with little attention paid to the relevance of data to various audiences (i.e., providers and the public).9,13 Edmond and Bearman41 believe one limitation of public reporting in the US is that states which have reporting programs have each developed their own unique approach. As a result, the quality indicators are difficult to compare, limiting the usefulness of the data for improvement or accountability. McGlynn and Adams15 offer evidence that, at times, reports from different agencies end up with different results, despite deriving their information from the same data sets.

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

21 Another challenge in creating reports that are meaningful to their target audiences is determining the appropriate number and variety of quality indicators to communicate essential information about quality without oversimplifying or overwhelming the intended user. The issue of too much versus too little information is debated among those producing reports on quality.15 It is important to summarize information to make it accessible. Thus, there is increasing attention being paid in quality indicator research to the development of summary scales or indices of quality, such as those created from composites of multiple quality indicators.42 The challenge in summarizing information on health care quality is that quality is multi-dimensional and the correlation among measures of the various dimensions (or even among specific aspects within dimensions) are low in most real-life settings.43

4.5

Level of data to be reported

There is considerable debate in the literature regarding the appropriate level or ‘granularity’ of reporting.44 Comparative data have been reported at a geographical (e.g., province/territory, regional health authority), institutional (e.g., hospital, care group, unit), and individual provider (i.e., individual clinician) level.5 Experts make two recommendations on this issue: first, that publication objectives should dictate the granularity of reporting, and second, that data should be reported at the level most likely to simulate improvement.2,11 No strong evidence has been reported in the literature about what level of reporting is most ‘effective’.11 This is largely because researchers have used different methods and reached different conclusions about the impact of public reporting on different audiences.15 Adding to this lack of clarity, appropriate measures are different depending on whether the subjects of reports are individual providers, hospitals, or health organizations. Also, the level of data included in a report may only be relevant to a limited audience. What the public expects, in terms of accountability, may be different from information clinicians find useful for quality improvement.15

Experts in the area of quality improvement have suggested that reporting should not focus on ‘weeding out’ poor-performing individuals, but rather on moving systems and providers as groups toward greater improvement.45 This has been expressed as the need to ‘move and improve’ rather than ‘shame and blame.’45 Moreover, many components of quality are not the sole responsibility of the clinician and may be beyond their control.44 Marshall et al.44 explain _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

22 that, “…even mortality following cardiac surgery is dependent on an anaesthetist, surgical team, general hospital care, and the patient.” Most experts concur that health care delivery is viewed as a team activity, rather than an individual responsibility, and quality improvement focuses on changes at the system level rather than changes in the individual.30,44

Some supporters of reporting at the level of individual providers have argued that, in certain cases (i.e., specialty surgeons), presenting information on individual professionals may increase the impact of public reporting.44 The rationale is that a focus on individual performance may force health professionals to, “take notice and responsibility for improving care.”44 The argument has also been made that because poorly performing clinicians can cause public harm and erode confidence in the health care system, it would be unethical to withhold this information from the public.45

The available evidence suggests that the most effective way to motivate quality improvement is to publish performance information at the level of provider organizations (e.g., hospital, primary care group of group general practices).4,11 Reporting at the regional level (i.e., RHA) is less likely to have an impact on quality of care, because RHAs may have less direct control than providers over the quality of care issues identified in the reports.44 Although the public may be demanding more information on individual providers, limitations of measurement methods, data quality issues, and potential negative consequences make this level of reporting controversial and, some argue, unwarranted in most places at the present time.44,46 Furthermore, reporting on the performance of individual providers may generate resistance that impedes achieving the goals of quality improvement.30

4.6

Risk adjustment

Regardless of the level of aggregation at which quality is reported on, analysis of the data must be adjusted to account for systematic differences in the populations receiving services. There is consensus that only those characteristics of populations that demonstrably cause differences in the outcomes or processes being measured should be subject to adjustment. For example, while age and comorbidities have a strong biological or clinical causal link to certain outcomes – such as death following heart attack – other factors like ethnicity, which may also be associated with _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

23 variation in outcomes but do not biologically cause the differences, should not be adjusted for.15 Risk adjustment should include only those factors where the health care provider has little or no chance to influence the relationship between the factor and the indicator of quality. Furthermore, because risk adjustment generally requires use of advanced statistical methods, which add to the complexity of communicating the results, these methods should be applied when it can be demonstrated that doing so significantly changes the conclusions that can be drawn from the data.15

4.7

Comparisons and the evaluation of differences

One of the functions of measurement and reporting on health care quality is to enable comparisons among places where similar services are delivered. Such comparisons help identify organizations whose excellence should be emulated elsewhere and can help health care providers, managers, and the public identify places or issues where there are gaps between current quality and some preferred level of performance. There is considerable debate about how to select reference points against which comparisons are made; the most common ones are national or local averages of all providers or within peer groups or some benchmark of best performance.15 Those who favour reaching for an ideal goal tend to focus on comparison to benchmarks or targets. Others, who emphasize the importance of trying to ensure that all places provide quality of service at least in keeping with the norm of what is currently widely available, favour comparison to averages.

There are potential problems both with comparison to an absolute standard and with relative comparison among the performance of providers. The former could influence providers to deliver care that may be clinically inappropriate or unacceptable for some patients, for example if a patient has a contraindication to a medication that is part of a standard recommendation, or objects to the use of a particular therapy that is part of the standard. The latter (i.e., relative comparisons) may fail to provide sufficient incentive to improve if providers feel comfortable performing at an “average” level – particularly when that level falls short of what the research shows is optimal.15 One option for dealing with these difficulties is to compare to the best performer (or top 10% of performers) in an area. This would serve to provide a better incentive

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

24 for improvement for most providers, while grounding the comparison in current realities within a given health system or area.

4.7

Timing of release of reports

The issue of when public reports are released warrants consideration. Some critics have complained that the information in public reports is not sufficiently timely or current.31,44,47 Conversely, there are those who believe that public reports should only be released if they contain highly valid and reliable data that have been subjected to rigorous methodological analysis: This ‘waiting for the best data’ approach may, however, impede the process and limit the potential benefits of more timely release.44

The timing of a report’s release may impact its effectiveness in bringing about quality improvement. A leading expert has argued that provider ‘buy-in’ of public reporting is critical in motivating quality improvement.44 However, the acceptance of public reporting will require a change in the culture of those provider organizations involved. This cannot be expected to occur quickly and experts have argued that implementation of a public reporting system must be managed carefully. Strong resistance to quality improvement may occur if the agenda is “pushed too far or too fast.”44 To facilitate acceptance, some public reporting programs have allowed health professionals to review and digest performance data for a period of time before it is released publicly.

Examples of this are common in Canada; CIHI reported the Hospital

Standardized Mortality Ratio (HSMR) results privately to hospitals for nearly two years before publicly releasing these data48 and Ontario’s Hospital Report project reported results privately to hospitals for a year before taking the reports public. As well, some providers have been given the opportunity to provide comments that are subsequently appended to the public report.44

4.8

Mandatory versus voluntary reporting

Many advocates of mandatory public reporting argue that public accountability validates the need for obligatory participation. They argue there is no justification for ‘protecting’ health care providers and that it is a provider’s duty to “tell those they serve how well they are doing their jobs.”46 As well, public scrutiny is part of the responsibility of health care organizations and their

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

25 leaders.46 In Canada, the Commission on the Future of Health Care (Romanow Report) states that transparency in provision of care is an important expectation of health care organizations.5,49

Mandatory public reporting has also been advocated because of its potential to have an impact on quality improvement. Although voluntary participation allows providers to show their commitment to quality improvement and accountability, there is evidence that voluntary reporting has compromised quality improvement efforts by encouraging ‘poor performers’ to withdraw from the reporting system.5 A review by Marshall et al.13 suggests the initial development stages of a reporting system may need to rely on voluntary reporting, but that ultimately a system should move to mandatory reporting in order to be most useful and effective. Other methods, such as the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 in the US, tie reporting of quality information directly to funding increases (or decreases, where quality information has not been submitted).15

4.9

Privacy Considerations

Federal and provincial governments have created policies to safeguard the privacy and confidentiality of personal information, particularly health information. Saskatchewan’s legislation, the Health Information Protection Act,50 stipulates that an individual has a right to consent to the use or disclosure of personal health information about himself or herself. The Personal Information Protection and Electronic Documents Act51 is federal legislation that places similar stipulations around use of personal health information by commercial enterprises in health and other industries.

The right of individuals to privacy and the confidentiality of their personal health information places some limits on public reporting of information. Information reported on a very small number of patients with similar characteristics may inadvertently identify that a specific individual received a particular health care service, and by implication, may have had a particular type of medical condition. Under its data access agreement with the Ministry of Health, the Health Quality Council may not report information based on fewer than six observations or individuals (providers or patients). Statistics Canada has developed a set of comprehensive guidelines for more complex situations.52 For example, the p-percent rule suppresses information _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

26 in certain instances where the data are highly concentrated within a single cell. Strategies are available – such as collapsing categories or deleting sensitive cells – to address these concerns.

Privacy legislation applies only to users of health services. It does not offer providers the right to privacy regarding their performance in the delivery of health services. Thus, policy created by the data stewards, not privacy law, currently precludes reporting at the level of individual providers in Saskatchewan. The Ministry of Health is the steward of administrative databases such as the Discharge Abstract Database for hospitals, the Medical Services Plan, and the Prescription Drug Plan Historical Claims Database. In other instances, the steward may be Regional Health Authorities, professional regulatory bodies or associations, or other groups.

5.

DEFINING THE HEALTH QUALITY COUNCIL’S PRINCIPLES ON DISCLOSURE OF INFORMATION ABOUT HEALTH CARE QUALITY: ENGAGING WITH HEALTH CARE SYSTEM STAKEHOLDERS AND THE PUBLIC

After we finished our literature review in late 2005, we distributed a discussion paper to health care system stakeholders a discussion paper that highlighted the key themes around public reporting. Stakeholders were invited to provide their responses to this document and to attend a public meeting entitled “What’s behind the curtain?” The meeting, which was broadcast using Telehealth technology in March 2006, connected 314 participants in 19 sites around the province; more than half of the attendees were health care professionals.

The three-hour event featured presentations by a well-known consumer advocate (Phil Edmonston, author of the popular Lemon-Aid guides to new and used cars), and a diverse panel that included a practising family physician, the Saskatchewan Information and Privacy Commissioner, and the Globe and Mail’s public health reporter. The presentations focused on the key themes found in the literature review: •

the public’s right to know about the quality of health care services;



the challenges associated with providing information on individual practitioners; and, _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

27 •

the need to avoid reporting practices that create an environment of blame rather than support for improving quality of care.

Participants appreciated the opportunity to hear a frank, open debate on the topic. For many of them, the concept of publicly reporting on health care quality – and all of its attending issues – was new.

In the months after we distributed our discussion paper, we received responses from stakeholders throughout the province. While most respondents generally supported the concept of public reporting and our proposed principles, some expressed reservations about reporting on health care quality in Saskatchewan. Their concerns included: o clarification of HQC’s role in reporting relative to that of regulatory organizations; o the level of aggregation to be included in HQC reports; o the rationale for producing reports (accountability? quality improvement? consumer choice?); and, o clarification about what topics HQC would be reporting on.

Based on feedback to the discussion paper and comments raised at the public meeting, we concluded that further consultation with stakeholders and interested members of the public was in order before we finalized our principles on reporting. In October 2006, we hosted a one and a half-day discussion forum at which we presented our proposed principles, then facilitated breakout discussions on three key issues: 1. Reporting on individual health care facilities, individual units within facilities, or both. 2. Reporting on individual providers. 3. The level of aggregation to be included in HQC reports.

Participants included senior executives and board members from health regions, unions and regulatory bodies, representatives from Saskatchewan Health, journalists, and members of the public; each brought an important perspective to the discussion. Generally speaking, most participants: o supported public reporting at the facility level, even for small facilities; _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

28 o felt the risks of reporting publicly on individual health care providers outweigh the benefits; o indicated that anonymous reporting on providers or facilities was not beneficial and lacked transparency; and, o supported reporting on individual providers to health regions and other health care organizations (in particular the regulatory bodies), but not publicly.

Although participants could not agree on any one best method for reporting, they felt the forum was a worthwhile exercise and suggested that continued support for HQC’s public reporting program would only be achieved through ongoing engagement with our partners. This feedback strengthened our resolve to develop reports in close consultation with those organizations we are reporting on, and those audiences we are reporting to.

6.

CONCLUSION

Based on evidence and informed opinion from the literature reported in this paper, and ongoing discussions with health system stakeholders, HQC is committed to the principles stated at the beginning of this document to guide its disclosure of information about health care quality.

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

29 Reference List (1) Maclean's Magazine. Maclean's University Ranking. 2008. Available from: http://oncampus.macleans.ca/education/rankings/; accessed Feb 10, 2008. (2) Marshall M, Davies H. Public release of information on quality of care: How are health services and the public expected to respond? Journal of Health Services & Research Policy. 2001;6(3):158-62. (3) Stecher B, Nataraj Kirby S. Introduction. In: Stecher B, Nataraj Kirby S, editors. organizational improvement and accountability: Lessons for education from other sectors. Santa Monica, CA: Rand Corporation; 2004. 1-10. (4) Marshall MN, Romano PS, Davies HT. How do we maximize the impact of the public reporting of quality of care? International Journal for Quality in Health Care. 2004;16(Suppl 1):i57-63. (5) Morris K, Zelmer J. Public reporting of performance measures in health care. Ottawa, ON: Canadian Policy Research Networks Inc.; 2005. Report No.: 4. (6) Patient's Bill of Rights. Patient's Rights Association of Saskatchewan. 2005. Available from: http://www.patientsrights.ca; accessed Feb 10, 2008. (7) Coulter A. Paternalism or partnership? Patients have grown up-and there's no going back. BMJ. 1999;319(7212):719-20. (8) Health Quality Council. 2007. Available from: http://www.hqc.sk.ca/; accessed Feb 10, 2008. (9) Fooks C, Maslove L. Rhetoric, Fallacy or Dream? Examining the Accountability of Canadian Health Care to Citizens. Ottawa,ON: Canadian Policy Research Networks Inc. 2004. Report No.: 1. (10) Mannion R, Goddard M. Public disclosure of comparative clinical performance data: Lessons from the Scottish experience. Journal of Evaluation in Clinical Practice. 2003;9(2):277-86. (11) Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: What do we expect to gain? A review of the evidence. JAMA. 2000;283(14):186674. (12) Mannion R, Davies HT. Reporting health care performance: Learning from the past, prospects for the future. Journal of Evaluation in Clinical Practice. 2002;8(2):215-28. (13) Marshall MN, Shekelle PG, Davies HT, Smith PC. Public reporting on quality in the United States and the United Kingdom. Health Affairs. 2003;22(3):134-48.

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

30 (14) Hibbard JH, Stockard J, Tusler M. It isn't just about choice: the potential of a public performance report to affect the public image of hospitals. Medical Care Research & Review. 2005;62(3):358-71. (15) McGlynn EA, Adams JL. Public release of information on quality. In: Anderson R.M., Rice TH, Kominski GF, editors. Changing the US health care system: Key issues in health services, policy, and management. 3 ed. San Fransisco: Jossey-Bass; 2007. (16) Peterson ED, DeLong ER, Jollis JG, Muhlbaier LH, Mark DB. The effects of New York's bypass surgery provider profiling on access to care and patient outcomes in the elderly. Journal of the American College of Cardiology. 1998;32(4):993-9. (17) Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA. 1998;279(20):1638-42. (18) Romano PS, Zhou H. Do well-publicized risk-adjusted outcomes reports affect hospital volume? Medical Care. 2004;42(4):367-77. (19) Smee CH. Improving value for money in the United Kingdom National Health Service: Performance measurement and improvement in a centralised system. Paris, France: Organisation for Economic Co-operation and Development; 2002. 57-85. (20) Commission for Healthcare Audit and Inspection. 2008. Available from: http://www.chai.org.uk; accessed Feb 10, 2008. (21) Dr Foster. Dr Foster Intelligence. 2008. Available from: http://www.drfoster.co.uk/; accessed Feb 10, 2008. (22) The Society for Cardiothoracic Surgery in Great Britain and Ireland. UK Thoracic Surgical Audit Project. 2008. Available from: http://www.scts.org/; accessed Feb 10, 2008. (23) Canadian Institute for Health Information. 2008. Available from: http://secure.cihi.ca/cihiweb/splash.html; accessed Feb 10, 2008. (24) Ontario Ministry of Health and Long-term Care. Ontario Wait Times. 2008. Available from: http://www.health.gov.on.ca/transformation/wait_times/wait_mn.html; accessed Feb 10, 2008. (25) Ontario Hospital Report. Hospital Report Research Collaborative. 2008. Available from: http://www.hospitalreport.ca; accessed Feb 10, 2008. (26) Health Quality Council of Alberta. 2008. Available from: http://www.hqca.ca; accessed Feb 10, 2008. (27) Ontario Health Quality Council. QMonitor. 2007. Available from: http://ohqc.ca/pdfs/final_ohqc_report_english.pdf; accessed Feb 10, 2008. _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

31 (28) Werner RM, Asch DA. The unintended consequences of publicly reporting quality information. JAMA. 2005;293(10):1239-44. (29) Fung CH, Lim Y, Mattke S, Damberg C, Shekelle P. Systematic review: The evidence that publishing patient care performance data improves quality of care. Annals of Internal Medicine. 2008;148:111-23. (30) Leatherman S, McCarthy D. Public disclosure of health care performance reports: Experience, evidence and issues for policy. International Journal for Quality in Health Care. 1999;11(2):93-8. (31) Felt-Lisk S, Barrett A, Nyman R. Public reporting of quality information on Medicaid health plans. Health Care Financing Review. 2007;28(3):5-16. (32) Shaller D, Sofaer S, Findlay SD, Hibbard JH, Lansky D, Delbanco S. Consumers and quality-driven health care: A call to action. Health Affairs. 2003;22(2):95-101. (33) Retchin SM. Overcoming information asymmetry in consumer-directed health plans. American Journal of Managed Care. 2007;13(4):173-6. (34) Marshall MN, Romano PS. Impact of reporting hospital performance. Quality & Safety in Health Care. 2005;14(2):77-8. (35) Hibbard JH, Stockard J, Tusler M. Does publicizing hospital performance stimulate quality improvement efforts? Health Affairs. 2003;22(2):84-94. (36) Goldman LE, Vittinghoff E, Dudley RA. Quality of care in hospitals with a high percent of Medicaid patients. Medical Care. 2007;45(6):579-83. (37) Mannion R, Davies H, Marshall M. Impact of star performance ratings in English acute hospital trusts. Journal of Health Services & Research Policy. 2005;10(1):18-24. (38) Davies HTO. Public release of performance data and quality improvement: Internal responses to external data by US health care providers. Quality in Health Care. 2001;10(2):104-10. (39) Wallace J, Teare G, Verrall T, Chan B. Public reporting on the quality of healthcare: Emerging evidence on promising practices for effective reporting. Canadian Health Services Research Foundation. 2008. Available from: http://www.chsrf.ca/pdf/Public_Reporting_Summary_E.pdf; accessed Feb 10, 2008. (40) Reilly T, Meyer G, Zema C, Crofton C, Larson D, Darby C, et al. Providing performance information for consumers: Experience from the United States. Paris, France: Organisation for Economic Co-operation and Development; 2002. 97-116. (41) Edmond MB, Bearman GML. Mandatory public reporting in the USA: An example to follow? Journal of Hospital Infection. 2007;65(S2):182-8. _________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008

32 (42) Agency for Healthcare Research and Quality. Talking Quality. 2008. Available from: http://www.talkingquality.gov/; accessed Feb 10, 2008. (43) Wilson IB, Landon BE, Marsden PV, Hirschhorn LR, McInnes K, Ding L, et al. Correlations among measures of quality in HIV care in the United States: Cross sectional study. British Medical Journal. 2007;335(7629):1085-8. (44) Marshall M, Shekelle PG, Brook RH, Leatherman S. Dying to know: Public release of information about quality of health care. Santa Monica: RAND Corporation; 2000. (45) Naylor CD, Iron K, Handa K. Measuring health system performance: Problems and Opportunities in the era of assessment and accountability. Paris, France: Organisation for Economic Co-operation and Development; 2002. 13-34. (46) Berwick DMM. Public performance reports and the will for change. JAMA. 2002;288(12):1523-4. (47) Castle NG, Lowe TJ. Report cards and nursing homes. Gerontologist. 2005;45(1):48-67. (48) Canadian Institute of Health Information. 2008. Available from: http://secure.cihi.ca; accessed Feb 10, 2008. (49) Romanow R. Building on values: The future of health care in Canada. The Commission on the Future of Health Care in Canada: Ottawa; 2002. (50) Health Information Protection Act, Bill H-021, Government of Saskatchewan, (1999). (51) Personal Information Protection and Electronic Documents Act, Bill C-6, Government of Canada, (2000). (52) Statistics Canada. Statistics Canada Quality Guidelines. 2003. Ottawa, Government of Canada.

_________________________________________________________________________ Disclosure of Information about Health Care Quality: Background Paper February 2008