Quality Improvement, Patient Safety, and Continuing Education: A ...

Research Report

Quality Improvement, Patient Safety, and Continuing Education: A Qualitative Study of the Current Boundaries and Opportunities for Collaboration Between These Domains Simon Kitto, PhD, Joanne Goldman, PhD, Edward Etchells, MD, MSc, FRCPC, Ivan Silver, MD, MEd, FRCP, Jennifer Peller, MA, Joan Sargeant, PhD, Scott Reeves, PhD, and Mary Bell, MD, MSc, FRCPC

Abstract Purpose Quality improvement/patient safety (QI/PS) and continuing education (CE) efforts have a common aim to improve health care outcomes. Yet, minimal collaboration occurs between them. This lack of integration can be problematic given the finite resources available and the potential value of approaching health care challenges from different perspectives. The authors conducted an exploratory study to understand Canadian leaders’ perceptions and experiences with both their own and the other domain, with the aim of increasing their understanding of the boundaries

T

he health care sciences include the domains of continuing education (CE), patient safety, quality improvement, and knowledge translation. Initial efforts to link these domains try to bridge CE, quality improvement, and patient safety, as demonstrated by recent organizational programs, scholarly publications, conference agendas, and regulatory requirements.1–5 Leaders in CE are promoting a shift from the traditional focus on medical knowledge acquisition to a focus on ongoing selfand performance assessment and practice improvement to improve health care processes and outcomes.4 Leaders in quality improvement and patient safety are working to expand understanding

Please see the end of this article for information about the authors. Correspondence should be addressed to Dr. Kitto, Office of Continuing Professional Development, Faculty of Medicine, University of Ottawa, RGN 2211, 451 Smyth Rd., Ottawa, Ontario, Canada K1H 8M5; e-mail: [email protected]. Acad Med. 2015;90:00–00. First published online doi: 10.1097/ACM.0000000000000596

Academic Medicine, Vol. 90, No. 2 / February 2015

and opportunities for collaborative approaches to improving health care. Method The authors conducted this study in 2011–2012 using a qualitative interpretivist framework to guide the collection and analysis of data from semistructured interviews. They used criterion-based, maximum variation, and snowball sampling to select 15 leaders from the domains of QI/PS and CE to interview. They transcribed verbatim the interviews and coded the transcripts using a directed content analysis approach.

Results Participants described the relationship between QI/PS and CE in four ways: (1) the separation of QI/PS and CE as distinct interventions, (2) (re)positioning CE in QI/ PS activities, (3) (re)positioning QI/PS in CE activities, and (4) further integrating QI/PS and CE. Conclusions These findings have important implications for how leaders in QI/PS and CE should mindfully and strategically negotiate their relationship to ensure the relevance and effectiveness of their domain’s activities.

of effective educational approaches to support medical trainees’ and faculty members’ quality improvement knowledge and behaviors.6 Although these leaders recognize that the further development of their own domain is contingent on the expertise offered by the other domains, they have not made a concerted effort to deconstruct and understand the complex relationships, interactions, and disintegration across the domains. This exploration is important because such insights can promote greater networking, information sharing, collaboration, and ultimately more effective health care improvement strategies. To address this gap, we completed a multistage research project.

goal of achieving a trustworthy system of health care delivery. Patient safety is also an attribute of health care systems; it minimizes the incidence and impact of, and maximizes the recovery from, adverse events.”8 Finally, quality improvement encompasses the “actions for improving the processes and outcomes of health care, including increasing value; improving responsiveness to customers and consumers; improving outcomes in the areas of safety, effectiveness, timeliness, patient centeredness, equity, and efficiency; reducing variation in outcomes; and increasing organizational adoption and implementation of continuous improvement methods in ongoing operations.”9

In our research, we used the following definitions. CE includes the “educational activities that serve to maintain, develop, or increase the knowledge, skills, and professional performance and relationships a physician uses to provide services for patients, the public, or the profession.”7 Patient safety is “a discipline in the health care sector that applies safety science methods toward the

The first stage of our research consisted of a scoping review of the literature to provisionally map the interactions and interplay amongst CE, patient safety, quality improvement, and knowledge translation.10 This review resulted in a description of the mission, stakeholders, methods, and limitations of each domain and a provisional assessment by the authors of their similarities and

1

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Research Report

differences and of their integration. Our findings showed that the domains are conceptualized in the literature as relatively disintegrated from each other, yet similarities suggest that they are integrated at some level. We concluded that the ambivalent relationships between the domains must be clarified for a more integrated, holistic approach to health care improvement to be possible. During the second stage of our research, reported here, we interviewed leaders in CE, patient safety, quality improvement, and knowledge translation to gather their perspectives and experiences with their own domain, the other domains, and opportunities for collaboration. This aim of gaining insight into leaders’ perspectives was meant to illuminate their “vocabularies of interest,” a concept originated by Mills11 to explain that individuals’ motives are not an intrinsic part of them but are shaped by the situation, including the existing social rules and power relations, in which they find themselves. An exploration of individuals’ motives or interests is a way of understanding their “attempts to define and enforce contingent forms of social order.”12 Thus, our study was a way to trace the interest maps generated by the leaders themselves, interest maps that subsequently shape their behavior.12 In short, how a leader understands the purpose, objective, and effectiveness of an intervention in another domain affects the likelihood that she or he will partner with the leaders of that domain. Leaders have begun to explore the integration of these domains conceptually by writing commentaries and descriptive articles about integration initiatives,5,13,14 but little research-based empirical data analysis that explores the nature of relationships amongst the domains has been conducted. Addressing this gap is an important step in determining the barriers to greater collaboration among leaders and to further the integration of health care intervention designs. Here, we focus on the perspectives and experiences of leaders in quality improvement, patient safety, and CE. We initially conceptualized quality improvement and patient safety as separate domains, but the findings from the second stage of our research, reported here, demonstrated a conflation between them. Safety was frequently positioned as one element of quality improvement, and many leaders described their work

2

as both quality improvement and patient safety. In response, we combined the sample and data from these domains and refer to it here as quality improvement/ patient safety (QI/PS).

associations, hospitals, and academic institutions. Some worked as clinicians in addition to their academic and/or administrative responsibilities.

The aim of this study was to examine the perceptions and experiences of leaders in QI/PS and CE in Canada to increase our understanding of the current boundaries and opportunities for collaborative approaches to improving health care.

We collected data through individual in-depth, semistructured interviews between July 2011 and May 2012.19 We used an interview schedule to guide the interviews (see List 1). In keeping with standards for conducting in-depth, semistructured interviews, each interview was flexible enough to allow participants to reflect on the historical development of the domain, compare and contrast domains, and discuss contextual issues affecting the domains. We conducted six interviews in person and nine interviews over the telephone. Two research associates conducted the interviews, and each lasted between 30 and 90 minutes. After informed consent was secured, interviews were audiorecorded and transcribed verbatim. Leaders did not receive incentives for participating.

Method

Study design and participants We used an interpretivist framework15 to guide our collection of data through semistructured interviews and the subsequent analysis of those data. This exploratory framework sensitizes the study towards participants’ understandings, perceptions, and experiences. The University of Toronto research ethics board approved this study. The aim of our recruitment strategy was to interview individuals identified as leaders in the domains of QI/PS and CE. We employed three sampling strategies. First, we used a criterion-based sampling strategy, where a leader was defined as either a person who has made a significant contribution to the scientific literature in one of the domains through publication and grant capture or a person engaged in a domain through active and sustained participation in organizations and activities.16 The two other sampling strategies we used were maximum variation sampling and snowball sampling.17 We used a maximum variation sampling frame to recruit leaders working in different organizational contexts. We supplemented this strategy with a snowball sampling approach, soliciting participants’ suggestions of others to invite. We determined the number of participants to recruit using theoretical saturation, defined as the point at which additional interviews cease to provide new insights into the questions being explored.18 A total of 15 leaders participated in our study—10 from QI/PS and 5 from CE. The sample consisted of 11 males and 4 females, with representation from 5 Canadian provinces: Ontario, Quebec, British Columbia, Alberta, and Saskatchewan. Participants worked in a range of organizations, including professional

Data collection

Data analysis We completed data analysis iteratively throughout the data collection period. We coded the interview transcripts using a directed content analysis approach whereby previous research guides the initial coding of themes and categories.20 On the basis of our initial scoping review,10 we structured the directed content analysis to explore participants’ understanding of the QI/PS and CE domains and the relationships between them. We analyzed the data paying specific attention to the language and contextual meaning of the text. In addition, we used researcher triangulation throughout data collection and analysis to enhance research rigor.21,22 Two research associates (J.G., J.P.) initially coded the same four transcripts. They then reviewed the codes and definitions with the two principal investigators (S.K., M.B.). Subsequently, one research associate (J.G.) coded the interviews and met with the other research associate as well as the principal investigators to review the coded transcripts and discuss the coding process, codes, and scope of analysis. Results

Participants described the relationship between QI/PS and CE in four ways: (1)



Research Report

perpetuate this siloed approach, as one participant commented:

List 1 Interview Guide for a Study of the Boundaries and Opportunities for Collaboration Between Quality Improvement/Patient Safety and Continuing Education, According to Leaders in Each Domain, 2011–2012 1. History of your domain Prompts: • Please describe your understanding of the historical development of your domain in Canada? 2. Definition of your domain Prompts: • In your own words, can you describe the definition of your domain? • Can you describe the theories it uses? • Can you describe the methods it uses? • Can you describe the outcomes it tries to achieve? • Can you describe the stakeholders it involves? • Can you describe its target population? • Can you describe its drivers? • Can you describe its repertoire of interventions? • Can you describe its settings and the scale of action it is directed towards? 3. Current state of your domain Prompts: • What are the current limitations of your domain? • What are the current areas of contention within your domain? • What areas of future research and/or development are being called for in your domain? 4. Relationship to other domains Prompts: • How do you define the other domains? • What are their limitations? • How is your domain related to the other domains (refer to above units of analysis)? 5. Integration with other domains Prompts: • What has been your experience in integrating your domain with the other domains? • What are the challenges to an integrated approach? Theoretical? Methodological? Institutional? Financial?

the separation of QI/PS and CE as distinct interventions, (2) (re)positioning CE in QI/PS activities, (3) (re)positioning QI/PS in CE activities, and (4) further integrating QI/PS and CE. We organized our findings according to these themes. A summary of our key findings can be found in List 2.

or as part of the “delivery of education.” For example:

The separation of QI/PS and CE as distinct interventions

In terms of work practices, participants perceived their colleagues to be working in distinct locations and not communicating with each other. They attributed this practice to individuals being too busy with activities in their own domain to attend to, or value information from, other domains.

Participants from both domains indicated that the interventions, work practices, and research activities of each domain occurred separately from those of the other domain. Participants described QI/PS interventions as occurring within institutions or as part of the “delivery of care,” whereas CE interventions took place in the university


Oftentimes, the continuing medical education activities happen outside of the context of the care delivery system and, to a certain extent, are not always solving the problem of the day. (QI/PS 8)

Participants noted that research also occurs separately, which is problematic. They felt that funding structures

I think that unfortunately the research has also been in silos and I think that we need collaborative research in order to find out how to do these things in the best way.… I think that we all need to understand each other’s disciplines to a certain extent, do collaborative research together to determine where each group, bringing their set of skills, fits in. (CE 4)

(Re)positioning CE in QI/PS activities Participants from the CE domain discussed the (re)positioning of CE activities as components of QI/ PS initiatives. They recommended three strategies to do so: (1) CE is one component amongst others in a QI/PS approach, (2) there is a lack of explicit recognition for the role of CE in QI/PS activities, and (3) there is a need for more rigorous CE activities in QI/PS initiatives. Participants from the CE domain recognized that their domain is one component of QI/PS. They commented that QI/PS issues extend beyond just education issues and that some QI/PS issues do not require education initiatives at all: Quality improvement is a well-defined cycle, and I have a slide that I show all the time showing that CE is only a very small component. And, in fact, some changes and improvement in quality don’t involve education at all. I think in order to improve health care outcomes, continuing professional development has to address the system and the improvement cycle in order to effect change. (CE 4)

Participants from the CE domain noted that insufficient explicit attention has been paid to the role of education as an intervention in itself in QI/PS. They also expressed concerns about the quality of CE in QI/PS, including questions about its development and assessment. For example, one participant felt that QI/PS education was not based on educational research and theory: Now it isn’t true to say that education doesn’t exist in that quality improvement menu of things to do, but it’s dealt with like another logistical element.… We’re going to throw some education at you, as generic as that sounds and then we’re going to study the outcome without any thoughtfulness about motivation to learn, how to learn, is it a small group or large group, is it done over a two-week session or is it done interprofessionally? (CE 2)

3


Research Report

following quote illustrates this focus on quality improvement content and skills:

List 2 Key Findings in a Study of the Boundaries and Opportunities for Collaboration Between Quality Improvement/Patient Safety and Continuing Education, According to Leaders in Each Domain, 2011–2012 1. The separation of quality improvement/patient safety and continuing education as distinct interventions in relation to: a. The location of the programs b. The workers in each domain c. The research activities of each domain 2. (Re)positioning continuing education in relation to the quality improvement/patient safety domain in three ways: a. Continuing education as one component of quality improvement/patient safety, amongst others b. Explicit attention paid to the role of continuing education in quality improvement/ patient safety c. More rigorous use of continuing education in quality improvement/patient safety 3. (Re)positioning quality improvement/patient safety in relation to the continuing education domain in three ways: a. Use of quality improvement/patient safety data to inform continuing education activities b. Use of continuing education to change physicians’ attitudes towards their role in quality improvement/patient safety c. Use of continuing education to teach about quality improvement/patient safety strategies 4. Further integrating quality improvement/patient safety and continuing education by: a. Movement into community, administrative, and primary care settings b. Create space to explore intersections

(Re)positioning QI/PS in CE activities Participants from the QI/PS domain felt that CE programs did not sufficiently integrate a QI/PS perspective. They indicated three ways in which QI/PS could figure more prominently in CE: (1) use of QI/PS data to inform the selection of CE activities, (2) use of CE to change physicians’ attitudes towards their role in QI/PS, and (3) use of CE to teach QI/PS strategies. Participants from both domains described the value of using QI/PS research data to inform the planning of CE initiatives and physicians’ selections of CE programs. They felt that QI/PS data highlighted the needs of practitioners and could address the limitations of self-assessments in CE: I think for sure the increased pressure on continuing quality improvement and patient safety is bringing a new dimension too into the way we teach so that it becomes a more important factor in giving us the unperceived need for sure, that hadn’t come across people’s thought processes until fairly recently. (CE 3)

Participants also spoke about the importance of changing physicians’ mind-sets concerning QI/PS, which, they observed, requires physicians to consider their roles in terms of those delineated in national physician competency frameworks.23,24 They noted that CE currently emphasizes the medical expert role yet is moving towards fostering the

4

development of the collaboration and communication roles. Still, competencies also must emphasize the accountability and quality improvement roles. For example: So, I think there’s a huge overlap between using continuing education to actually make people aware of, so what are the theories if I actually make a system safer and what’s your role in making that system safer, as opposed to, what’s your role in becoming the best provider you can be? And then, what’s your role in actually taking best evidence and turning it into best practice, and then actually doing it consistently and reliably? And again we’re never taught, I don’t think, in any of our schools how you actually do that. (QI/PS 7)

Participants from the QI/PS domain offered suggestions about how to teach QI/PS skills in CE activities. One approach was to integrate QI/ PS knowledge and skills in relation to a particular clinical topic area. One participant gave the example of diabetes, whereby those taking part in CE programs would be exposed to the relevant QI/PS tools, such as a self-audit, in addition to learning about advances in diabetes management. A second approach was to teach program participants about the QI/PS tools related to their particular discipline, such as checklists for surgery. A third approach was to provide a course on quality improvement knowledge and methods in which participants would work together on a group project. The

I think that the first steps would be broadening the content so that it’s not just about the latest drugs and technologies. The next step though would be to equip [participants] with basic skills in auditing their own practices and engaging them in the idea that quality improvement is their responsibility. (QI/PS 6)

Further integrating QI/PS and CE Participants provided some general suggestions for how to further integrate the domains of QI/PS and CE, and they broadly endorsed such efforts, as they viewed these domains as being “overlapped and interdependent” (QI/PS 7) with “broad scopes that are very well interrelated” (QI/PS 3). Participants from the QI/PS domain described the importance of extending QI/PS and CE activities in both the hospital setting, where QI/PS has traditionally occurred, and in community and administrative settings. They recommended pursuing an integrated QI/PS and CE approach in interprofessional primary care settings and at the health care administration level whereby leadership roles could involve both QI/PS and CE components. Finally, participants described the challenges to creating the space to have the opportunity to explore such intersections and how different perspectives can enrich each other. Still, they expressed interest in further integrating the domains: That is one of the areas where I think there’s a strong synergy between the quality improvement thinking and the work that many people have done in continuing professional development. Because I think the most effective means to develop a practice come from an understanding of the current ways in which that practice happens and then a thoughtful integration of new ideas, new methods, new approaches, into that practice. (QI/PS 4)

Discussion

Our findings demonstrate that collaboration across the domains of QI/ PS and CE is an important issue today. Leaders from both domains perceived that QI/PS and CE interventions, work practices, and research activities were operating separately, and they put forward suggestions for how to overcome



Research Report

these divisions. Their suggestions included (re)positioning CE in QI/PS activities and (re)positioning QI/PS in CE activities. They also acknowledged the importance of continuing to explore how to further integrate the two domains. Our findings have important implications for future integration efforts. First, they provide a needs assessment of how to align these domains. Second, they enable us to reflect on current conceptualizations of the relationship across domains and critically explore opportunities for integration. Participants noted that these two domains are similar and have interrelated objectives yet are operating separately from each other, which reinforces the findings from our scoping review.10 Their suggestions that future efforts explicitly focus on the role of each domain in the context of the other provide valuable strategic direction. Promoting a theoretically and empirically informed approach to the use of CE in QI/PS and the more explicit use of QI/ PS in the planning and implementation of CE programming would strengthen both domains and appeal to the interests and motivations of the domain leaders. These approaches build on the emerging conceptual and empirical literature concerning current and future approaches to working across the domains.4,5,13,25 For example, Shojania and colleagues5 discussed how quality improvement activities could be embedded in CE programs focused on clinical content areas. Such programs could highlight clinical areas with quality problems in traditional continuing medical education or supplement continuing medical education with postevent deliverables, such as submitting the results of quality improvement projects after a course. Researchers also have begun to study the role of education in supporting QI/PS knowledge.13,26 This research has shown the need for faculty development to promote QI/PS expertise in concert with further study on the education principles and details underlying teaching in QI/PS. In addition, participants identified the need to further explore how the domains can inform each other and can allow for collaboration, although our findings provided less insight into how integration across the domains can overcome the siloed thinking within each. Although our


findings focused on an intervention-based approach to further integration of the domains, we suggest that further attention should be paid to thinking more broadly and thoughtfully about the nature of the relationships across domains. Social theory on boundary work provides insight into the social organization of scientific knowledge, illuminating how disciplines, specialties, and theoretical orientations are formed and institutionalized through the use of relational and often political processes operating across institutions and contexts.27 For example, boundary work describes the social process by which scientists attribute selected qualities to scientists, scientific methods, and scientific claims to differentiate between their science and a “lesser” science and to assert epistemic authority (the preeminence of their knowledge base over others). These boundaries can contribute to separation and exclusion or to communication and exchange.27 The theory of boundary work helps to contextualize current assumptions about the boundaries of CE and QI/PS and, in turn, to inform how these boundaries can be redefined or bridged. The next stage of our research will extend this qualitative work to include an in situ study of the performance of (dis)integrated QI/PS and CE activities. We hope this research will provide an understanding of the relationships between the domains and how they are manifested in a particular organizational context, while providing insight into local barriers and facilitators to these domains operating effectively together in making targeted changes to the health care system. Our study has a number of limitations. First, our findings are based on a sample that may not represent all the domain leaders in Canada. In addition, it focuses only on domain leaders in Canada. As an interview-based study, the data provide insights into attitudes, experiences, understandings, and what participants say about their attitudes and behaviors, not their actual actions.28 Our study also has a number of strengths. It provides insight into leaders’ perceptions of activities in relation to the two domains and serves as a valuable foundation in identifying problems and solutions to their integration. Although our sample of 15 leaders might appear small, the findings are important from a conceptual

generalizability standpoint, rather than being empirically generalizable. Conceptual generalizability refers to how well a study’s findings inform health care contexts that differ from those in which the original study was undertaken.29 The conceptual generalizability of the key themes relating to the relationship between QI/PS and CE could inform future conceptual, empirical, and theoretical work. Conclusion

Our study adds to the early work being done to overcome the isolated research and practices of the domains of QI/PS and CE. On the basis of our findings, further work to link QI/PS and CE requires a deeper understanding of the processes of collaboration amongst the domains. Further research should improve our understanding of the varied factors that inform and influence such efforts and their outcomes. Such research should address how to overcome the isolated thinking underpinning some, but not all, QI/PS and CE approaches, and to support an integrated approach that is responsive to emerging and ongoing problems within a complex health care system. Acknowledgments: The authors would like to acknowledge the quality improvement/patient safety and continuing education leaders who volunteered to be interviewed for this study and Paula Rowland for her input on an early draft of this article. Funding/Support: This study was funded by a grant from the Association of Faculties of Medicine of Canada, Standing Committee on Continuing Medical Education/Professional Development National CPD Research Fund. Other disclosures: None reported. Ethical approval: The University of Toronto research ethics board approved this study (protocol reference #26461). Previous presentations: Kitto S, Bell M, Goldman J, et al. The perceived intersections and boundaries between the domains of continuing education and quality improvement: Preliminary findings. Presented at: Canadian Conference on Medical Education; April 22, 2013; Quebec City, Quebec, Canada. Kitto S, Bell M, Goldman J, et al. The perceived intersections and boundaries between the domains of continuing education and quality improvement. Presented at: Association of American Medical Colleges Annual Meeting; November 3–6, 2012; San Francisco, California. Dr. Kitto is associate professor, Department of Innovation in Medical Education, and director of research, Office of Continuing Professional Development, Faculty of Medicine, University of Ottawa, Ottawa, Ontario, Canada.

5


Research Report Dr. Goldman is research associate, Continuing Professional Development, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada. Dr. Etchells is associate professor, Department of Medicine, Faculty of Medicine, University of Toronto, and medical director of information services, Sunnybrook Health Sciences Centre, Toronto, Ontario, Canada. Dr. Silver is vice president of education, Centre for Addiction and Mental Health, Toronto, Ontario, Canada.

5

6

7

Ms. Peller was research associate, Continuing Professional Development, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada, at the time this study was done. Dr. Sargeant is professor and head, Division of Medical Education, Faculty of Medicine, Dalhousie University, Halifax, Nova Scotia, Canada. Dr. Reeves is professor in interprofessional research, Faculty of Health, Social Care, and Education, Kingston University/St. George’s, University of London, London, England. Dr. Bell is associate professor, Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada.

References 1 Kahn N, Bagley B, Tyler S. Performance Improvement CME: Core of the New CME. CPPD Report. Chicago, Ill: American Medical Association Continuing Medical Education; 2007. 2 American College of Emergency Physicians. Assessment of practice performance. http:// www.acep.org/Continuing-Educationtop-banner/Assessment-of-PracticePerformance/. Accessed October 20, 2014. 3 Society for Academic Continuing Medical Education. 2013 Spring Meeting: Partnering to Improve Evidence, Quality and Patient Safety in Medical Education. Madison, Wisconsin. April 10–13, 2013. http://www. sacme.org/event-444885. Accessed October 20, 2014. 4 Davis NL, Davis DA, Johnson NM, et al. Aligning academic continuing medical education with quality improvement:

6

8

9

10

11 12 13

14

A model for the 21st century. Acad Med. 2013;88:1437–1441. Shojania KG, Silver I, Levinson W. Continuing medical education and quality improvement: A match made in heaven? Ann Intern Med. 2012;156:305–308. Wong BM, Levinson W, Shojania KG. Quality improvement in medical education: Current state and future directions. Med Educ. 2012;46:107–119. Davis D, Bordage G, Moores LK, et al. The science of continuing medical education: Terms, tools, and gaps: Effectiveness of continuing medical education: American College of Chest Physicians evidencebased educational guidelines. Chest. 2009;135(suppl):8S–16S. Emanuel L, Berwick D, Conway J, et al. What exactly is patient safety? In: Henriksen K, Battles JB, Keyes MA, Grady ML, eds. Advances in Patient Safety: New Directions and Alternative Approaches (Vol. 1: Assessment). Rockville, Md: Agency for Healthcare Research and Quality; 2008. Alexander JA, Hearld LR. What can we learn from quality improvement research? A critical review of research methods. Med Care Res Rev. 2009;66:235–271. Kitto S, Bell M, Peller J, et al. Positioning continuing education: Boundaries and intersections between the domains continuing education, knowledge translation, patient safety and quality improvement. Adv Health Sci Educ Theory Pract. 2013;18:141–156. Mills CW. Situated actions and vocabularies of motive. Am Sociol Rev. 1940;5:904–913. Callon M, Law J. On interests and their transformation: Enrolment and counterenrolment. Soc Stud Sci. 1982;12:615–625. Boonyasai RT, Windish DM, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: A systematic review. JAMA. 2007;298:1023–1037. Davis DA, Baron RB, Grichnik K, Topulos GP, Agus ZS, Dorman T. Commentary: CME and its role in the academic medical center: Increasing integration, adding value. Acad Med. 2010;85:12–15.

15 Berger PL, Luckmann T. The Social Construction of Reality: A Treatise in the Sociology of Knowledge. New York, NY: Anchor; 1967. 16 Miles MB, Huberman AM. Qualitative Data Analysis. 2nd ed. Thousand Oaks, Calif: Sage Publications, Inc.; 1994. 17 Liamputtong P, Ezzy D. Qualitative Research Methods. Oxford, England: Oxford University Press; 2005. 18 Glaser BG, Straus AL. The Discovery of Grounded Theory: Strategies for Qualitative Research. Chicago, Ill: Aldine; 1967. 19 Kvale S. Interviews: An Introduction to Qualitative Research Interviewing. Thousand Oaks, Calif: Sage Publications; 1996. 20 Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15:1277–1288. 21 Thurmond VA. The point of triangulation. J Nurs Scholarsh. 2001;33:253–258. 22 Flick U. Triangulation revisited: Strategy of validation or alternative? J Theory Soc Behav. 1992;22:175–197. 23 Royal College of Physicians and Surgeons of Canada. The CanMEDS Framework. http:// www.royalcollege.ca/portal/page/portal/rc/ canmeds/framework. Accessed October 20, 2014. 24 Canadian Patient Safety Institute. The Safety Competencies. http://www. patientsafetyinstitute.ca/English/ toolsResources/safetyCompetencies/Pages/ default.aspx. Accessed October 20, 2014. 25 Davis DA, Prescott J, Fordis CM Jr, et al. Rethinking CME: An imperative for academic medicine and faculty development. Acad Med. 2011;86:468–473. 26 Allaire BT, Trogdon JG, Egan BM, Lackland DT, Masters D. Measuring the impact of a continuing medical education program on patient blood pressure. J Clin Hypertens (Greenwich). 2011;13:517–522. 27 Lamont M, Molnár V. The study of boundaries in the social sciences. Annu Rev Sociol. 2002;28:167–195. 28 Reeves S, Kuper A, Hodges BD. Qualitative research methodologies: Ethnography. BMJ. 2008;337:a1020. 29 Kitto SC, Chesters J, Grbich C. Quality in qualitative research. Med J Aust. 2008;188:243–246.

