Lessons Learned From the Application of Mixed ...

Article

Lessons Learned From the Application of Mixed Methods to an International Study of Prehospital Language Barriers

Journal of Mixed Methods Research 1–18 Ó The Author(s) 2016 Reprints and permissions: sagepub.com/journalsPermissions.nav DOI: 10.1177/1558689815627712 mmr.sagepub.com

Ramsey C. Tate1, Peter W. Hodkinson2, and Andrew L. Sussman1

Abstract Mixed methods research is increasingly common in emergency medical services, but methodological expertise among prehospital researchers has been found lacking. The purpose of this article is to describe unique challenges that the authors encountered in the application of mixed methods to a multisite, international study of prehospital language barriers. Lessons learned include the role of formative research in identifying cultural and organizational norms that affect researcher engagement with emergency medical service agencies, the necessity of developing approaches for member checking and assessing respondent validity, and the importance of promoting mixed methods as a rigorous methodology in international settings. Keywords prehospital emergency care, emergency medical services, mixed methods, communication barriers, language barriers Prehospital care, also referred to as emergency medical services (EMS) care, occurs across a complex sequence of interactions that are difficult to assess with strictly quantitative or qualitative research methods. The continuum of EMS care begins when a patient or bystander contacts an EMS telecommunicator who elicits critical information from the caller, such as address and complaint. Based on the information elicited from the caller and the EMS telecommunicator’s local protocols, EMS field providers are dispatched to the patient’s location. Once the field providers arrive on scene, they continue to gather information from the patient and bystanders, perform a medical assessment, and determine the most appropriate level of care. The continuum of EMS care concludes when the patient is either discharged from care at the scene or is transferred to a receiving health care facility. While prehospital research has traditionally focused on studies of clinical interventions and relied on quantitative research methodologies, we have conducted research investigating the impact of language barriers on prehospital care using a mixed methods approach. Language 1

University of New Mexico, Albuquerque, NM, USA University of Cape Town, Cape Town, South Africa

2

Corresponding Author: Ramsey C. Tate, Section of Emergency Medicine, Department of Pediatrics, Baylor College of Medicine, 6621 Fannin St, Suite A2210, Houston, Texas 77030, USA. Email: [email protected]

Downloaded from mmr.sagepub.com at University of Cape Town on January 25, 2016

2

Journal of Mixed Methods Research

barriers have been demonstrated to have deleterious effects on patient outcomes in a variety of health care settings (Cohen, 2005; Flores, 2005; Hampers & McNulty, 2002; Karliner, Jacobs, Chen, & Mutha, 2007; Meischke, Chavez, Bradley, Rea, & Eisenberg, 2010), but the impact of language barriers on care delivered in the prehospital setting has not been well-described. In part, this is due to the inherent challenges of measuring health outcomes related to prehospital care (Callaham, 1997; Institute of Medicine, 2007; Moore, 1999; Moscati, 2002). Prehospital care faces a unique challenge in that care is delivered across time and geographic space for a single patient. Additionally, it is unclear how and whether language barriers have impacts distinct from communication barriers related to cultural discordance in the prehospital setting. Most EMS systems operate in multilingual and multicultural communities, and the discordance in language and culture for any single patient encounter may vary by provider and by patient’s ethnicity. EMS providers are required to make complex decisions in time-sensitive situations fraught with uncertainty and neither the impact of language barriers on decision making nor the techniques and tools that EMS providers use to attempt to overcome language barriers have been described. In order to address the challenges of describing the impact of language barriers on prehospital care, we developed a mixed methods research design to collect and analyze data from EMS providers along the continuum of EMS care as well as from EMS providers in comparable EMS systems in the United States (New Mexico) and South Africa (Western Cape province). Mixed methods is a robust methodology to describe and study complex systems with dynamic characteristics, such as EMS systems, and is uniquely well-suited to the study of nuanced constructs like language barriers (Creswell, Klassen, Plano Clark, & Smith, 2011; Johnson, Onwuegbuzie, & Turner, 2007; McManamny, Sheen, Boyd, & Jennings, 2014). In a recent article in this journal, McManamny et al. (2014) reported the results of a systematic review of the application of mixed methods to prehospital research. They concluded that expertise among prehospital researchers lags behind the increasing popularity of using mixed methods in the prehospital arena. Many of the reviewed studies neglected to address critical aspects of mixed methods research design, such as the rationale for choosing mixed methods or the integration of quantitative and qualitative data. The authors concluded with a call to action encouraging prehospital researchers to develop high-quality, rigorous mixed methods designs. We encountered unique challenges in the design and implementation of our international mixed methods study that prompted adaptation of the study methodology both to the pragmatic requirements of setting up and conducting research in EMS systems and to the cultural nuances of international collaboration. This article presents the key methodological challenges and lessons learned in the design and conduct of a multisite, international mixed methods study of prehospital language barriers. By sharing our experiences, we hope to further contribute to the development of mixed methods expertise by prehospital researchers.

Method Study Objectives The objective of the present article is to describe the methodological challenges and lessons learned in the application of mixed methods to the design and conduct of an international, multisite study of prehospital language barriers.

Theoretical Perspective and Rationale Our approach to the study of prehospital language barriers was rooted in a pragmatist epistemology (Creswell, 2014; Feilzer, 2010; Tashakkori & Teddlie, 2010). We conceptualized the


Tate et al.

3

Figure 1. Overview of the study design.

impact of language barriers in the prehospital setting as the multiple points of engagement along the continuum of EMS care at which language discordance could potentially influence the interaction between patients and EMS providers, as well as influence the interactions of the EMS system with minority language–speaking communities and with other institutions of the health care system. Accordingly, we developed a study that integrated different approaches to study points of care along the EMS continuum and across EMS systems with varied regulatory and sociopolitical environments to better triangulate the impact of language barriers. With this pragmatist approach, we focused on identifying commonalities in the EMS providers’ experience of working with language barriers and strategies for overcoming these language barriers.

Overview of Study Design The research study design is described in detail to provide context for the specific methodological challenges and lessons learned from the application of mixed methods to this prehospital research study. We selected a sequential explanatory mixed methods study design consisting first of a survey of EMS telecommunicators followed by semistructured interviews of EMS field providers (Creswell, 2014; Figure 1). Given the limited prior research on this topic, we selected this design to first gain a general understanding of the range of barriers and strategies identified by EMS telecommunicators and then examine those in greater depth during the qualitative phase. The study design was interactive with data from the survey informing revisions of the interview guide and qualitative data were weighted as a higher priority. The survey data were analyzed first followed by the qualitative data and then findings were merged to develop a


4


final analytic template. This research study was approved as exempt from review by the Human Subjects Research Committee at the University of New Mexico, received a waiver of review from the institutional review board of Presbyterian Hospital in Albuquerque, New Mexico, and was approved by the Human Research Ethics Committee at the University of Cape Town.

Setting EMS systems display tremendous heterogeneity in resources, training, and local protocols. This heterogeneity in systems constrains the development of multisite research designs; this variability also limits the generalizability of results obtained in single-site studies. Within this context, EMS systems in the Western Cape province of South Africa and the state of New Mexico in the United States offer a compelling opportunity to investigate EMS approaches for overcoming language barriers across multiple sites. Both regions have historically marginalized and segregated populations and both regions have an urban metropole that is attracting new waves of minority language–speaking migrants seeking employment opportunities. Nearly half (47%) of New Mexicans are Hispanic or Latino, 10% are Native American, and 36% speak a language other than English at home with the most common non-English languages spoken being Spanish and Navajo (U.S. Census Bureau, 2014). The Western Cape is predominantly trilingual, with 50% of residents reporting Afrikaans as a home language, 25% reporting Xhosa as a home language, and 20% reporting English as a home language (Statistics South Africa, 2012). These regions also have similarly structured emergency medical response infrastructures composed of an urban tertiary trauma and pediatric center, well-developed and professional urban/suburban EMS systems, and large rural areas requiring long-distance transports. Additionally, one of the coinvestigators (RT) has prior experience collaborating with EMS systems in both locations, reducing the logistical challenges of conducting research in multiple international locales. All EMS agencies that were approached participated in the study. New Mexico. Providers from four EMS systems in the state of New Mexico participated in the study. Within the urban/suburban metropole, EMS telecommunicators for Albuquerque Fire Department (AFD) and Bernalillo County Fire and Rescue were recruited to participate in the survey of telecommunicators. These two agencies manage the majority of emergency response calls for the Albuquerque metropolitan region. EMS field providers from Albuquerque Ambulance Services (AAS) respond to calls from both of these dispatch centers and were recruited for the interview component of the study. A rural EMS system in northern New Mexico that borders Navajo Nation, San Juan County Fire Department (SJCFD), served as a site for both the survey of telecommunicators and interviews of field providers. Western Cape. The Western Cape province of South Africa has centralized operations for its publicly funded EMS system, Medical Emergency Transport and Rescue (METRO). EMS telecommunicators were recruited from all four dispatch centers in the province, METRO’s urban metropole dispatch center in Cape Town as well as three suburban and rural dispatch centers (METRO Karoo, METRO Eden, and METRO Winelands). EMS field providers from the METRO Pinelands base who respond to calls dispatched by the Cape Town dispatch center were recruited for the interview component of the study.

Phase 1: Survey of EMS Telecommunicators Survey development occurred in multiple stages. Key domains to be addressed in the survey arose from discussion with EMS providers as well as prior published research based on


Tate et al.

5

telecommunicator surveys (Bauer, & Alegrıá, 2010; Meischke, Calhoun, Yip, Tu, & Painter, 2013; Ong et al., 2011). Construct validity for the concept of ‘‘language barrier’’ was assessed through discussion with EMS providers in multiple systems and both countries. Pilot surveys were distributed to a convenience sample of EMS dispatch supervisors in South Africa and New Mexico and subsequently revised to clarify language based on feedback. The final survey instrument consisted of 20 items (Appendix A). In addition to demographic data, the survey collected information on both fluent and nonfluent language capacity. Eight five-response Likert-type items assessed the perceived frequency of language barriers, frequency of impaired assistance experienced by callers with language barriers, frequency of overall impaired care delivered by the EMS system to callers with language barriers, stress or difficulty of calls with language barriers, frequency of use of informal strategies to overcome language barriers, and frequency of use of a telephonic interpreter system. Respondents were also asked about the adequacy of their training for delivering assistance when faced with language barriers. Informal strategies for overcoming language barriers were documented by selection from a prepopulated list of possible strategies as well as by free-text responses. The survey included qualitative, free-text response items inquiring about the biggest challenges that telecommunicators face in handling calls with language barriers, the advantages and disadvantages of telephonic interpreter systems, and the factors that influence use of a telephonic interpreter system. The surveys were distributed over a 2-week period in September 2013 to the seven participating dispatch centers, three in New Mexico and four in Western Cape. The paper survey sheets were distributed to every active telecommunicator over all shifts and collected at the end of 2 weeks in a closed envelope by managers. Expected sample size of 150 was a convenience sample thought to be representative of all actively employed telecommunicators at all sites during the survey period. Responses were anonymous. All nontraining telecommunicators were eligible to complete the survey. Results were double-entered into a spreadsheet to minimize transcription errors. No surveys were discarded. Responses that were unclear were coded according to uniform data entry rules applied by the investigators. Respondent validity was tested at multiple points, through feedback of summary reports of responses to dispatch supervisors and through presentations of preliminary data to EMS audiences.

Phase 2: Semistructured Interviews of EMS Field Providers A pilot interview guide was developed based on literature review, discussion with EMS providers in both locations, and preliminary review of the survey data. The interview guide was piloted with EMS field providers in two services in a total of three interviews and revised to clarify language and improve content capture prior to initiation of the study. The interview guide was further refined through iterative review of small batches of interview transcripts as the study enrolled subjects. The final interview guide (Appendix B) elicits information in the following domains: training and language proficiency, language barriers and distinctions between language and cultural barriers, strategies to overcome language barriers and desired resources, the impact of language barriers on decision making, and unique challenges in the care of pediatric patients whose caregivers are language-discordant. Interview prompts were developed to preferentially elicit narratives from participants rather than opinions or perspectives to generate richer source data. Based on reviews of small batches of interview transcripts, the interview guide was adapted partway through interview recruitment to clarify the wording of an interview prompt about pediatric patients and include additional questions on treatment decisions and the impact of uncertainty on treatment decisions.


6


Interview participants were enrolled from three EMS agencies (METRO, AAS, and SJCFD), all of whom respond to calls originating from participating study dispatch centers. All nontraining, active-duty EMS field providers who provide care for both pediatric and adult patients were eligible to participate. Subjects were identified through a combination of purposive sampling and snowball sampling to ensure that subjects with differing views and experiences were enrolled. Interviews were conducted between July 2013 and April 2014. Interviews were digitally recorded and subsequently transcribed. Transcriptions were spot-checked for accuracy. All interviews were conducted in English, the working language for all participating EMS services, and any non-English interview content was translated to English as well as retained in the original language. The average interview duration was between 30 and 60 minutes. Participants in New Mexico received a $25 gift card, while the participating agency in Western Cape received a donation of equipment for participation, in accordance with human subject guidelines in each country. Respondent validity was tested through presentation of preliminary data findings to EMS audiences at both sites.

Integration and Data Analysis The research design utilized a qualitative priority and quantitative methods were used in a secondary role. We chose to emphasize the qualitative data as we expected that the survey findings would help identify the range and frequency of use of language barriers and strategies to overcome these challenges. However, based on our prior research experience, we anticipated that the qualitative interviews would provide rich insights into the circumstances and decisionmaking processes that guided field providers. Quantitative data were generated primarily by the survey of EMS telecommunicators, although limited quantitative demographic information including fluent and nonfluent language capacity and information about training and work experience was collected during the interviews. Quantitative data from the two phases were mixed at the point of interpretation. All quantitative data were coded as categorical and analyzed in Stata version 13.1 (StataCorp, College Station, Texas). Outcome measures include descriptive statistics and nonparametric analysis of variance. Additionally, the quantitative data were also entered into Dedoose version 5.0.11 (SocioCultural Research Associates, California) for integrated data analysis. The qualitative data from free-text responses in the survey of EMS telecommunicators and the qualitative data from the interviews of EMS field providers were mixed at the point of data analysis to identify themes from EMS providers at different points in the continuum of care as well as in different locations and at different levels of training. The research team collaboratively developed an analytic framework for thematic coding through iterative review of the freetext survey items and small batches of transcripts. A priori anticipated themes were integrated with emerging themes to revise the analytic framework as data collection progressed. The final version of the coding template is available as Appendix C. All transcripts were coded independently by two team members. A third team member then integrated the coding and reconciled any coding discrepancies either through reference to the coding template definitions or through collaborative discussion with the primary coders. Ongoing data analysis continued until the research team concluded that thematic exhaustion had been reached. Qualitative data were coded and analyzed in Dedoose version 5.0.11 (SocioCultural Research Associates, California). Following preliminary analysis of the quantitative and qualitative data, integrated analysis of the quantitative and qualitative data was performed in Dedoose version 5.0.11 (SocioCultural Research Associates, California). Outcomes measures included measures of association for themes by quantitative descriptor.


Tate et al.

7

Key Lessons Learned We identified five unique challenges that arose during our development and implementation of a mixed methods study of prehospital language barriers: establishing credibility with EMS agencies, integrating redundancy into recruitment, adapting interviewers to local culture and context, assessing validity through stakeholder feedback, and supporting conduct of mixed methods research internationally.

Establishing Credibility With EMS Agencies One of the principal challenges of conducting research on prehospital care is the fierce independence that characterizes many EMS systems (Bledsoe, 2003) and which requires establishing credibility with participating EMS systems for research participation. EMS providers are required to rapidly make complex decisions with limited information and resources. Providers pride themselves on their ingenuity and ‘‘can do’’ attitude, a prominent theme that we termed ‘‘MacGyver’’ in our coding template in reference to the American television series from the 1980s that featured a particularly resourceful hero. At the same time, many EMS providers have experienced dismissiveness on the part of receiving health care providers for their treatment decisions due to the receiving health care providers’ lack of appreciation for the difficult and chaotic environments in which EMS works or the constraints of EMS protocols. As a consequence, EMS providers may be reluctant to cooperate with outside researchers who they do not feel understand or will accurately portray the stressful and dynamic circumstances in which they work. Recognizing the critical importance of establishing credibility with potential participant EMS agencies, our research team began building relationships with agencies in both Western Cape and New Mexico up to 2 years before initiating the research study and the research team was recruited specifically to build on this groundwork. The research in the Western Cape was coordinated by coinvestigator Dr. Hodkinson, an Emergency Medicine physician at the University of Cape Town who works closely with METRO medical directors. A quality assurance manager for METRO actively assisted in interview recruitment. Additionally, the other coinvestigator (RT) has a history of EMS clinical work with METRO and is known to the paramedics, medical directors, and supervisors who participated in the research. Dr. Tate is a member of an Albuquerquearea EMS research consortium that vetted the research study and that includes the medical directors of AFD, Bernalillo County Fire and Rescue, and AAS. She also participates in EMS education and training exercises for EMS agencies and training programs. Key brokers for the relationship between the research team and SJCFD were the New Mexico state EMS medical director, a research mentor for Dr. Tate, and an SJCFD paramedic who served as a research assistant on the study team. A second research assistant, a former critical-care paramedic with AAS, was pivotal in recruitment of Albuquerque-area interview participants.

Adapting Interviewers to Local Culture and Context Establishing credibility with participating EMS agencies was a time-consuming and laborintensive process that preceded study development and implementation and honoring these relationships occasionally required adapting research elements to the preferences of participating agencies, particularly in conducting interviews. Furthermore, interviewing field providers about language barriers, which inherently bring up issues of race and ethnicity, required sensitivity to local culture to optimize meaningful participation. In the Western Cape, sensitivity to issues of race and ethnicity required careful selection of an interviewer that would be considered neutral and unintimidating. Accordingly, we recruited a young anthropologist with a similar


8


socioeconomic background to many of the participants to conduct the qualitative interviews in the Western Cape. In addition, we felt that it would be advantageous to have an interviewer that was not associated with EMS in the Western Cape to reassure respondents that the content of their interviews would not be shared with their employer in an identifiable fashion. In contrast, EMS informants in New Mexico felt that participants would be more comfortable speaking to an interviewer that had an EMS background and was known to participants. We recruited two interviewers with prior experience as paramedics with two of the participating agencies. We believe that we gathered a greater depth of perspective from interview participants by adapting the choice of interviewers to local culture and context. However, there was a consequent loss of standardization of the study protocol as a result. Our interviewer in the Western Cape was less integrated into the research team than our interviewers in New Mexico and she completed a larger number of interviews than anticipated before providing interview recordings. The New Mexico interviews were conducted in a more iterative fashion with interviews reviewed after small batches of interviews were completed. Additionally, our interviewer in the Western Cape had prior experience with qualitative interviewing, whereas our interviewers in New Mexico underwent training in qualitative interviewing specifically within the context of this study. These choices in different types of interviewers were important adaptations to the local context and culture to ensure participation and trust.

Integrating Redundancy Into Recruitment There is an adage in the EMS community that ‘‘Once you’ve seen one EMS system, you’ve seen one EMS system.’’ A key challenge of conducting research on prehospital care is the tremendous variation in organizational and bureaucratic structures that characterize EMS systems and the lack of unified management. EMS systems range from rural volunteer organizations to large branches of local government and obtaining research engagement can be challenging. Research involving EMS providers should not occur without the appropriate authorizations from supervisors. However, research is not an equal priority for all EMS supervisors and obtaining authorizations at different levels of management can be a time-consuming process that is often reliant on personal recommendations by trusted brokers. We discovered that the establishment of credibility with the leadership of participating agencies was a necessary but not always sufficient component to obtain the level of engagement required to conduct research. We encountered a number of challenges in engaging individual managers at EMS agencies despite extensive prior relationship development with agency leadership. One challenge arose from the variability in EMS agency structures at participating sites. The authors had existing relationships with both the local supervisors and the agency directors for METRO that facilitated research participation in the Western Cape. However, the participating EMS agencies in New Mexico are distinct operational entities and do not share central medical direction or operational management. Each agency required a unique approach to obtaining permissions to conduct research and, unexpected to our research team, permission for each study component had to be obtained separately and sometimes from different levels of management. As an example, the permission to conduct the surveys of dispatchers was obtained from a single dispatch supervisor at each agency and these supervisors actively facilitated the survey study. However, permission from separate supervisors to conduct interviews was required for several agencies. We had anticipated that our existing cooperation from leadership would be sufficient to obtain permissions and were surprised by the difficulty in navigating the bureaucratic structures of these agencies. Even with the support and brokering of leadership within the agencies, obtaining the appropriate permissions from the nonmedical management required


Tate et al.

9

multiple e-mail exchanges, phone calls, and in-person visits, resulting in an unanticipated delay of several months in data collection. Furthermore, we were unable to successfully engage the mid-level supervisors of two groups of field providers prior to termination of the study (AFD and one location of METRO), which appeared to be more due to difficulty in obtaining permission from supervisors at multiple levels of management concurrently than due to reluctance of the agency to participate. Because providers from AFD and AAS respond to the same dispatched calls, we did not feel that excluding AFD providers altered the generalizability of our findings. Similarly, we felt that providers dispatched from the participating METRO base had significant operational overlap with the providers at the METRO base from which we were unable to enroll participants. Because our study design had integrated redundancy in recruitment opportunities with overlapping groups of providers, we believed that modifying recruitment did not influence our results and we ceased attempting to obtain permissions at the point in which we determined that we had reached thematic exhaustion for the interviews. An important lesson learned from this experience is to anticipate variability in EMS engagement even with strong support from EMS leadership and develop a recruitment plan that can tolerate nonparticipation by some recruitment targets without sacrificing the integrity of the study.

Assessing Validity Through Stakeholder Feedback Because of the variation in EMS systems and the communities that they serve, we developed several strategies for triangulation to increase the external validity of our results. We obtained data not only from different types of providers along the EMS continuum of care but also data from different EMS agencies within both similar and highly disparate geographic regions. We hypothesized a priori that there would be certain common elements of the impact of prehospital language barriers on prehospital provider decision making despite the anticipated stochasticity of data arising from such varied sources. However, we were surprised early in the collection of interview data by the marked convergence in our initial results. We developed intentional methods to assess respondent validity to address our concerns that we were not gathering all representative viewpoints. We assessed respondent validity at a number of points during the study. Our initial interview sampling strategy was purposive, but evolved to snowball sampling in an intentional effort to identify potential participants who may hold differing viewpoints. As an example, snowball sampling in New Mexico suggested a small group of providers referred to as ‘‘Gadsden flag,’’ a reference to a popular American image of a coiled snake and the words ‘‘Don’t tread on me’’ that is often displayed as a libertarian or Tea Party symbol (Wikipedia Contributors, 2014). These providers were anticipated by their peers to hold English-language-only beliefs or strong anti-immigration beliefs. Interestingly, and unexpectedly, we discovered that the few individuals suggested by interview participants as holding more extreme viewpoints were no longer actively working in EMS within the participating agencies, suggesting that there are cultural or managerial value norms affecting these agencies that have overlap with the specific area of providing services to minority language speakers. We ceased interviewing when we determined that we were not missing key populations of EMS providers in our recruitment. We also assessed respondent validity by creating a brief report of the preliminary results of the survey data for each agency, highlighting commonalities with the other participating agencies as well as unique findings. These reports were submitted to the dispatch supervisors and medical directors who had facilitated the research at each dispatch center for member checking. Preliminary study results were also presented at regional conferences attended by EMS providers and medical directors not only from participating agencies but also nonparticipating


10


agencies. Through this extensive process of member checking, we determined to our satisfaction that the early convergence in our data was not due to selection or respondent biases.

Supporting Conduct of Mixed Methods Research Internationally Mixed methods research is a relatively novel research study design in many countries and the conduct of a mixed methods study encountered unanticipated concerns in research reviews in South Africa. The ethical review of research containing human subjects is similar in the United States and South Africa and we submitted applications to all relevant human subjects review committees prior to initiating the study. In New Mexico where mixed methods research is frequently pursued, the research study was declared exempt by the primary review committee (the Human Research Review Committee at the University of New Mexico) and received a waiver of review at the secondary review committee. However, mixed methods research is not well-established in South Africa (Creswell & Garrett, 2008; Ngulube, Mokwatlo, & Ndwandwe, 2009) and review by the relevant ethics committee for the Western Cape arm of the study, the Human Research Ethics Committee at the University of Cape Town, required a full committee review. The principal concerns of the review committee were around discomfort with the sample size and the application of mixed methods that were outside of the usual expertise of researchers on the committee. The concerns of the committee were adequately addressed after providing references to support the methodology, but did highlight possible barriers to conducting such research in mixed methods–naı¨ve settings.

Conclusions Mixed methods research is a valuable tool for prehospital researchers to explore the dynamic and unstable contexts that frame EMS provider interactions with patients. As the field of prehospital research matures and broadens its scope, we anticipate that mixed methods methodologies will be increasingly called on to investigate questions that are not well-answered by strictly quantitative or qualitative methods. However, there is a dearth of expertise among prehospital researchers in the conduct of rigorous mixed methods research and the EMS context requires thoughtful adaptations in methodology. Acknowledging the difficulty in conducting prehospital mixed methods research and identifying key challenges is an important step in strengthening our collective expertise. In this article, we detailed the development of an international, multisite mixed methods study of the impact of language barriers on prehospital care and identified lessons learned from five key challenges that arose in the implementation of this study. Several of the challenges relate to consideration of organizational and cultural norms that affect entreé into research with EMS agencies. We discovered that establishing credibility with participating EMS agencies is a pivotal factor in the success of EMS research. We also realized that honoring these relationships, as well as local culture and context, may require adaptations in study design or implementation that are not optimal from a methodological perspective, such as variation in types of interviewers, but which are necessary to solicit meaningful participation and high-quality data. We also experienced recruitment challenges arising from inconsistent participation from mid-level EMS supervisors. Supervisors expressed willingness to participate and enthusiasm for the research study, but it was difficult to obtain permissions at multiple levels of management with variable interest in the research concurrently to allow interviews to proceed in two of the recruitment settings. We learned that establishing credibility and personal relationships with agency supervisors was not adequate groundwork for successful recruitment. We overcame this challenge by integrating redundancy in our recruitment strategy such that failure to recruit from some locations did not affect the generalizability of our results. An ideal approach based on these lessons learned would include conducting formative research as an initial step of study design to identify these norms and build adaptive strategies into the research process.


Tate et al.

11

A challenge more internal to the methodological integrity of the study was reflected in the central issue of assessing the validity of the findings. We were concerned by the convergence of themes early in data collection, having expected greater variability given the variety of data sources. Although we had developed the study with specific strategies to enhance triangulation, the early emergence of common themes suggested that we might be missing representative groups. As a consequence, we developed several approaches to member checking and assessing respondent validity that we implemented throughout data collection. Our recommendation based on this experience would be to closely monitor emerging findings in small, iterative loops of data, as well as building in feedback mechanisms to the study design. Last, we encountered an unanticipated challenge in pursuing human subject’s research review in South Africa due to the novelty of mixed methods research. Addressing the concerns of the review committee in Western Cape required establishing the credibility of mixed methods research designs. Mixed methods researchers should anticipate that submissions to institutional review boards and other relevant reviewers in countries with little mixed methods research experience will require more detailed methodological justification of study design than strictly quantitative or qualitative research. The challenges that we faced in conducting mixed methods research in the prehospital context internationally encouraged us to develop strategies for pursuing rigorous research in a highly variable participant population. Although there is no single correct approach to addressing these types of challenges when conducting mixed methods research, we hope that sharing our experiences and lessons learned will provide other researchers with a critical lens through which to evaluate their relationship as partners with participating EMS agencies and the internal and external validity of their findings.

Implications We appreciate the critique of the rigor of existing mixed methods studies of prehospital care by McManamny et al. (2014). We have highlighted challenges encountered in our own application of mixed methods to international prehospital research to further the development of expertise in mixed methods among prehospital researchers. We hope this serves as a call to prehospital researchers to continue the discourse of designing high-quality mixed methods studies as well as contribute to the broader body of scholarship in mixed methods methodology.

Appendix A Survey of Telecommunicators 1. Do you know (before you talk with a caller) if the incoming call is medical in nature? (In other words, are you a secondary dispatcher that receives calls from a primary dispatcher?) u Yes, all calls I get have been triaged and identified as medical calls prior to transfer. u No, I am the first to triage calls as medical or other (fire, police, etc.) u Unsure 2. [New Mexico] Which languages do you speak fluently (check all that apply)? u English u Navajo u Spanish u Other__________ [Western Cape] Which languages do you speak fluently (check all that apply)? u English u Afrikaans u Xhosa u Other__________ Note: Some questions adapted from Meischke, Chavez, Bradley, Rea, Eisenberg 2010.


12


3. [New Mexico] Which other languages do you sometimes speak with callers, but do not speak fluently (check all that apply)? u English u Navajo u Spanish u Other [Western Cape] Which other languages do you sometimes speak with callers, but do not speak fluently (check all that apply)? u English u Afrikaans u Xhosa u Other__________ 4. About how often do you encounter callers who speak a different language than you? u Almost never u About once a week u Almost daily or daily u About once a month u More than once a week u More than once a day 5. Please describe the biggest challenge you face in handling such calls:

6. How often does a language barrier affect your ability to provide assistance to these callers? u Almost never u About once a week u Almost daily or daily u About once a month u More than once a week u More than once a day 7. How often do you believe that language barriers affect the medical care that these callers receive? u Never u Rarely u Sometimes u Often u Always 8. In what way do you believe that language barriers affect the medical care that these callers receive?

9. In general, how difficult or stressful are emergency calls with callers who speak a different language than you (compared with calls with fluent, same-language speakers)? u Much more difficult or stressful u Slightly more difficult or stressful u More difficult or stressful u No difference in stress or difficulty u Somewhat more difficult or stressful 10. How prepared are you to effectively work with callers who speak a different language than you? u Very prepared u Prepared u Somewhat prepared u Not very prepared u Not at all 11. What strategies do you use to communicate with callers who speak a different language than you? (Please check all strategies that you have used) u I speak slower u I repeat words or phrases u I speak louder u I use the telephonic interpreter u I ask for a bystander who can translate the call for me u I rephrase what I’m saying using different or simpler words (e.g., ‘‘Help is coming’’ vs. ‘‘The medics are on their way’’) u Other:

12. Please check the ONE communication strategy you have found to be most effective: u Speaking slower u Repeating words or phrases u Speaking louder u Using the telephonic interpreter u Asking for a bystander who can translate the call for me


Tate et al.

13

u Rephrasing what I’m saying using different or simpler worse (e.g., ‘‘Help is coming’’ vs. ‘‘The medics are on their way’’) u Other:

13. How often do you use a telephonic interpreter when speaking with callers who speak a different language than you? u Almost never u About once a week u Almost daily or daily u About once a month u More than once a week u More than once a day 14. If you have used a telephonic interpreter, what were the ADVANTAGES of using this service?

15. If you have used a telephonic interpreter, what were the DISADVANTAGES of using this service?

16. What factors influence whether or not you choose to use a telephonic interpreter during a call with language barriers?

17. How would you describe the amount of training you have received for strategies for callers with a language barrier? u More than enough training u Not enough training u Enough training u No training 18. How long have you worked as a telecommunicator or emergency dispatcher? u Less than 1 year u Between 6 and 9 years u More than 20 years u Between 1 and 5 years u Between 10 and 20 years 19. What is your gender? u Male u Female 20. What is your age?_______ The final questions asked for a little more information about you. Again, all your answers are anonymous and will not be linked to you personally. Your surveys will only be seen by research staff and will not be shared with your supervisors. Thank you for participating in this survey!

Appendix B Final Interview Guide Do you consent to participate in this study? ************************************************** Domain: Training and Language Proficiency First, I would like to learn more about your career as a medic. How did you receive your training? What year did you first start working as a medic?


14


I am interested in learning more about how medics work with patients who speak different languages. What languages do you speak? How did you learn (languages mentioned by subject)? ************************************************** Domain: Language Barriers Could you give me an example of a time when you encountered a language barrier with a patient? Did you feel there were differences in language, culture, or both in that example? Could you talk a little more about those differences and how you think they affected your interaction? What strategies did you use to overcome the language barrier? About how often would you say you run into situations with language barriers? ************************************************** Domain: Strategies to Overcome Language Barriers Have you ever used an interpreter service to communicate with patients? What do you think prevents you from using a telephonic interpreter service? How do you make transport decisions for patients when you have difficulty communicating at the scene? How does care change when you have difficulty communicating at the scene? (If respondent mentions uncertainty) How does being uncertain (about the diagnosis/complaint) change care? Can you think of any suggestions for how the EMS system could better serve patients with language barriers? Any tools or training that might be useful for you? In your opinion, what responsibility does the EMS system have to provide emergency services in languages other than English? ************************************************** Domain: Pediatric Patients Now, I’d like to talk more specifically about pediatric patients. About what percentage of your runs involve pediatric patients? Have you received special training in pediatric care? Can you think of a time when you experienced a language barrier with a child’s parent or caregiver while taking care of a pediatric patient? Are there different strategies that you use to overcome language barriers for the parents or caregivers of pediatric patients than for adult patients? Can you give me an example? How do you make transport decisions for children when you have difficulty communicating with parents or caregivers at the scene? Do you have suggestions for ways that the care of these pediatric patients could be improved? Those are all the questions that I have. Before we end the interview, is there anything else that you’d like to add? Thanks for taking the time to talk with me today.

Appendix C Coding Template With Definitions Quotables: Excerpts that exemplify themes and could be used in paper


Tate et al.

15

Attitudes or Beliefs: Triggered by encountering language barrier Providers about patients/callers Threatening: fear of violence, thefts, assaults Limited knowledge: about medical treatment Empathy: Putting self in patient/caller’s shoes Impatient/frustrated/anxious: Negative attributes Negative prejudgment: Negative preconceived bias or prejudice toward minority language speakers Providers about self Inadequate: Either in ability to provide service or as judged by others MacGyver: positive view that self always achieves good outcome using what’s available Negative emotional response: Stress/anxiety/frustration/anger Cultural Barriers: Identified as such by respondent to discriminate from language or gender barrier Gender Barriers: Identified as such by respondent to discriminate from language or cultural barrier Values: Core values expressed by respondent Professional responsibility: Obligations due to profession Should serve everyone: Obligation regardless of language Should serve majority minority: Obligation only to most common minority languages Balancing personal priorities: Recognizing limitations of available time, expresses priorities either work or personal Emotional support of patient/caller/family: Reassurance of patient/caller expressed as core part of job Decision Making: Information synthesized to decide on course of action and outcomes of that process No effect on decision making: As stated by respondent Uncertainty: Expressing confusion, indecision, doubt, lack of clarity Uncertainty of information obtained: such as address, chief complaint, history Uncertainty of diagnosis: uncertainty after synthesis of available information Uncertainty of appropriate care/action: specific to uncertainty in care due to language barrier, excludes uncertainty due to limitations in skill or knowledge Impact on transport: Change in likelihood of transporting pt Increased likelihood of transport: Specific to language barrier Inability to obtain refusal: Specific to inability to obtain transport refusal due to language barrier Decreased likelihood of obtaining refusal: Due to language barrier Refusal liability: Concern about provider liability Patient best interest: Concern about patient’s best interest No change in transport: Same likelihood as if no language barriers Change in destination: Transports to a different receiving facility Impact on care: Change in type of care delivered Decreased time with patient on scene: Including scoop ’n run No change in care but different approach: Same care/treatments/prioritization delivered as if no language barrier, but manner of treating patient different Reliance on next providers in chain: Anticipating next providers (e.g., hospital staff, field providers) have better ability to obtain information than current provider, specific to language barrier Objective indicators: Decision making based on vitals/exam/SATS/mechanism


16

Journal of Mixed Methods Research Strategies to Overcome Language Barriers: Tools/techniques/resources to improve communication between patient/caller and provider/dispatcher when language barrier Current strategies: Strategies described by respondents Bystander (adult): Use of a lay adult to translate informally, includes family members, excludes other professionals Bystander (child): Use of a lay child, including family or patient, to informally translate Telephonic interpreter: Use of a professional interpreter, service provided by employer Coworker: Use of another professional to informally interpret, includes other EMS, other health staff, other public services Nonverbal communication: Communication other than speech, includes gestures, body language, tone of voice Digital tool: Use of available commercial tool for translation in digital format Nondigital tool: Use of dictionary, card, or other paper-based tool for translation Nonprofessional language acquisition: Any language acquisition not provided by employer, includes personal courses, informal teaching by others, classes prior to employment Simplified language: Includes repeating phrases, slower speech, basic words in any language Limitations of strategies: Barriers identified by respondents to success of strategies to improve communication Accuracy of interpretation: Uncertainty of faithfulness of translation, includes professional and lay interpreters and translation tools Unable to identify language: Cannot utilize strategy because unknown language Concern about delay: concern on the part of the provider/dispatcher Delay in obtaining information: Increased time in obtaining the information required to make treatment or transport/dispatch decisions compared with no language barrier Delay in dispatch/transport: specific to delay in time from initiation of call to dispatch or initiation of transport Expectation of on-scene/transport time: Concern about potential delay due to employer scene/transport time regulations Unable to explain process to patient/caller: Language barrier prevents provider/dispatcher from explaining either the strategy to address barrier to caller/patient (e.g., waiting for telephonic interpreter or coworker) or the means to access services (e.g., caller from out-ofservice area) Unaware of resources: Provider unaware of existing resources Lack of time for additional trainings: Identified as a potential concern by employer, excludes personal concerns about available time Expense: specific to costs prohibiting use of strategy To service: identified as a potential concern by employer Personal: identified as potential costs borne by the provider and not reimbursed by employer, includes theft of personal resources (e.g., phone, dictionary) Lack of equipment: specific to equipment that would be provided by an employer to utilize strategy Liability: Concern for provider legal liability Unnecessary: Perception that strategies are not needed specifically because no need for additional communication Lack of specific fluency: Expressed as a specific limitation in provider’s competency in patient’s language (e.g., medical terminology, slang) Nonlanguage communication barrier: Nonlanguage barriers to communication that would not be overcome by strategy (e.g., mentally ill, intoxicated, deaf)


Tate et al.

17

EMS has bigger issues: Perception by either provider/dispatcher or employer that resources should be prioritized for other issues Confidentiality: Concern that patient info not kept confidential Suggested strategies: Potential tools/techniques/resources mentioned by respondents to improve communication between patient/caller and provider/dispatcher when language barrier Telephonic interpreter: Use of a professional interpreter, service provided by employer Work cell phone: Specific to employer provision of a cell phone to access interpreter services or translation tools Recorded line: Specific to employer recording of all communications on a line utilized to access interpreter services for liability concerns Multilingual staff: Increased hiring of EMS based on proficiency in minority languages, excludes hiring based on race or ethnicity Language classes: Specific to classes provided, reimbursed, or incentivized by employer Phrase books/dictionaries: Physical resources provided by employer Pictograms: Visual communication tools provided by employer Pediatric Patients: Responses specific to interactions with pediatric patients Increased anxiety or stress: As compared with interactions with adult patients By provider: Experienced by the provider By caregiver/family: Provider perceives the caregiver/family to have increased anxiety or stress as compared with caregiver/family of an adult patient Concern for maltreatment: Concern by provider for abuse or neglect of a child More impact from language barriers: Includes impact on any aspect of service delivery, as compared with adult patients Less impact from language barriers: Includes impact on any aspect of service delivery, as compared with adult patients Caregiver refusal of care: Refusal of care for a minor against EMS medical advice Language barrier with caregiver only: Specific to encounters in which providers can communicate with the patient but not the caregiver due to language barrier Acknowledgments The authors would like to thank Kelly Meehan-Coussee and Noah Cooperstein for their dedication and assistance with this project; Michael Lee and Natasha Hendricks for their input; as well as Brian Moore and Shiraz Mishra for their advisement.

Declaration of Conflicting Interests The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a grant from the Valente Family Fund.

References Bauer, A., & Alegrıá, M. (2010). Impact of patient language proficiency and interpreter service use on the quality of psychiatric care: A systematic review. Psychiatric Services, 61, 765-773. doi: 10.1176/appi.ps.61.8.765 Bledsoe, B. E. (2003). EMS needs a few more cowboys. Journal of Emergency Medical Services, 28(12), 112-113.


18


Callaham, M. (1997). Quantifying the scanty science of prehospital emergency care. Annals of Emergency Medicine, 30, 785-790. doi:10.1016/S0196-0644(97)70049-0 Cohen, A. L. (2005). Are language barriers associated with serious medical events in hospitalized pediatric patients? Pediatrics, 116, 575-579. doi:10.1542/peds.2005-0521 Creswell, J. W. (2014). Research design: Qualitative, quantitative, and mixed methods approaches (4th ed.). Thousand Oaks, CA: Sage. Creswell, J. W., & Garrett, A. L. (2008). The ‘‘movement’’ of mixed methods research and the role of educators. South African Journal of Education, 28, 321-333. Creswell, J. W., Klassen, A. C., Plano Clark, V. L., & Smith, K. C. (2011). Best practices for mixed methods research in the health sciences (Office of Behavioral and Social Sciences Research). Retrieved from https://obssr-archive.od.nih.gov/mixed_methods_research/index.aspx Feilzer, M. (2010). Doing mixed methods research pragmatically: Implications for the rediscovery of pragmatism as a research paradigm. Journal of Mixed Methods Research, 4(1), 6-16. doi: 10.1177/1558689809349691 Flores, G. (2005). The impact of medical interpreter services on the quality of health care: A systematic review. Medical Care Research and Review, 62, 255-299. doi:10.1177/1077558705275416 Hampers, L. C., & McNulty, J. E. (2002). Professional interpreters and bilingual physicians in a pediatric emergency department: Effect on resource utilization. Archives of Pediatrics & Adolescent Medicine, 156, 1108-1113. Institute of Medicine. (2007). Emergency medical services at the crossroads. Washington, DC: National Academies Press. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133. doi:10.1177/1558689806298224 Karliner, L. S., Jacobs, E. A., Chen, A. H., & Mutha, S. (2007). Do professional interpreters improve clinical care for patients with limited English proficiency? A systematic review of the literature. Health Services Research, 42, 727-754. doi:10.1111/j.1475-6773.2006.00629.x McManamny, T., Sheen, J., Boyd, L., & Jennings, P. A. (2014). Mixed methods and its application in prehospital research: A systematic review. Journal of Mixed Methods Research, 9(3), 214-231. doi: 10.1177/1558689813520408 Meischke, H., Chavez, D., Bradley, S., Rea, T., & Eisenberg, M. (2010). Emergency communications with limited-English-proficiency populations. Prehospital Emergency Care, 14, 265-271. doi: 10.3109/10903120903524948 Meischke, H. W., Calhoun, R. E., Yip, M.-P., Tu, S.-P., & Painter, I. S. (2013). The effect of language barriers on dispatching EMS response. Prehospital Emergency Care, 17, 475-480. doi: 10.3109/10903127.2013.811565 Moore, L. (1999). Measuring quality and effectiveness of prehospital EMS. Prehospital Emergency Care, 3, 325-331. doi:10.1080/10903129908958963 Moscati, R. (2002). Protection of human subjects in prehospital research. Prehospital Emergency Care, 6(2 Suppl.), S18-S23. Ngulube, P., Mokwatlo, K., & Ndwandwe, S. (2009). Utilisation and prevalence of mixed methods research in library and information research in South Africa 2002-2008. South African Journal of Libraries and Information Science, 75(2), 105-116. doi:10.7553/75-2-91 Ong, B. N., Yip, M. P., Feng, S., Calhoun, R., Meischke, H. W., & Tu, S. P. (2011). Barriers and facilitators to using 9-1-1 and emergency medical services in a limited English proficiency Chinese community. Journal of Immigrant and Minority Health, 14, 307-313. doi:10.1007/s10903-011-9449-6 Statistics South Africa. (2012). 2011 Census in brief. Retrieved from http://www.statssa.gov.za/census/ census_2011/census_products/Census_2011_Census_in_brief.pdf Tashakkori, A., & Teddlie, C. (Eds.). (2010). Sage handbook of mixed methods in social & behavioral research (2nd ed.). Thousand Oaks, CA: Sage. U.S. Census Bureau. (2014, July 8). New Mexico QuickFacts from the US Census Bureau. Retrieved from http://quickfacts.census.gov/qfd/states/35000.html Wikipedia Contributors. (2014, December 7). Gadsden flag. Retrieved from http://en.wikipedia.org/wiki/ Gadsden_flag
