Qualitative Social Work

Qualitative Social Work http://qsw.sagepub.com/

Establishing Trustworthiness in Qualitative Research in Social Work : Implications from a Study Regarding Spirituality Cynthia A. Lietz, Carol L. Langer and Rich Furman Qualitative Social Work 2006 5: 441 DOI: 10.1177/1473325006070288 The online version of this article can be found at: http://qsw.sagepub.com/content/5/4/441

Published by: http://www.sagepublications.com

Additional services and information for Qualitative Social Work can be found at: Email Alerts: http://qsw.sagepub.com/cgi/alerts Subscriptions: http://qsw.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://qsw.sagepub.com/content/5/4/441.refs.html

>> Version of Record - Nov 13, 2006 What is This?

Downloaded from qsw.sagepub.com by guest on February 19, 2013

03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 441

Qualitative Social Work Copyright ©2006 Sage Publications London, Thousand Oaks, CA and New Delhi, Vol. 5(4): 441–458 www.sagepublications.com DOI:10.1177/1473325006070288

ARTICLE

Establishing Trustworthiness in Qualitative Research in Social Work Implications from a Study Regarding Spirituality Cynthia A. Lietz Arizona State University, USA Carol L. Langer University of Wisconsin, USA Rich Furman University of North Carolina Charlotte, USA ABSTRACT

KEY WORDS: qualitative methodology rigor spirituality

As the use of qualitative inquiry increases within the field of social work, researchers must consider the issue of establishing rigor in qualitative research. This article presents research procedures used in a study of autoethnographies that were written regarding the experience of being Jewish. In this project, the researchers utilized reflexivity, audit trail, triangulation by observer, peer debriefing, member check and prolonged engagement in order to manage the threats to trustworthiness as discussed by Padgett (1998). Implications of the project suggest that research procedures utilized by qualitative researchers to establish rigor are an important way to increase our confidence that the voice of the participants is heard, therefore fitting the mission of the social work profession.

441

trustworthiness


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 442

442 ■ Qualitative Social Work 5(4)

INTRODUCTION Researchers within the field of social work are using qualitative inquiry with increasing rates (Padgett, 2004). The percentage of doctoral dissertations in social work that utilized qualitative methods tripled between 1982 and 1992 (Brun, 1997). Dozens of texts are now available discussing qualitative methodology in the social sciences (Padgett, 2004) and the social work programs have increased the number of courses being offered that describe qualitative research (Borochowitz, 2005). Many social science researchers acknowledge that methods for conducting qualitative research have advanced over the past two decades (Lowery and Evans, 2004). In addition, debates regarding the fit between qualitative inquiry and social work have led to a dialog about the ways in which this methodology can fit the field of social work’s mission to give voice to underprivileged populations (Bein and Allen, 1999; Gilgun, 1994; Padgett, 1998). The social work profession is acknowledging the role that qualitative inquiry has in our field as more and more social workers are turning to qualitative methodology in their research (Bein and Allen, 1999; Gilgun, 1994; Padgett, 1998, 2004). Yet, while qualitative research has become more influential, qualitative methods have not been fully accepted in some circles. While this may be due to conflicting views regarding the nature and role of science in social research, this may also be a function of the lack of clarification regarding the standards in qualitative methodology. Typically, traditional researchers aligned with logical positivism and other quantitative schools of thought attempt to view qualitative research through the lenses of their own training and philosophy (Stein, 2004). Quantitative methodology has clearly established concepts and procedures that allow the researcher to deal with the issue of objectivity (Creswell, 1998, 2003; Rubin and Babbie, 2005). Concepts such as reliability and validity and strategies that are outlined in quantitative design provide researchers with guidelines by which to conduct their research. As such, quantitative researchers view validity and reliability as the main means of establishing rigor in social science research. However, for qualitative studies these two measures are not applicable standards for establishing rigor (Aguinaldo, 2003; Morgan and Drury, 2003). The purpose of this article is to explore other notions of rigor available to qualitative researchers. To do so, the authors discuss how they grappled with issues of rigor in a qualitative study that analyzed autoethnographies written regarding spiritual identity. The authors demonstrate various methods used in increasing trustworthiness and explore implications for future qualitative research.

RIGOR AND QUALITATIVE RESEARCH As the use of qualitative inquiry within the field of social work grows, so does the ongoing debate about the appropriateness of the concept of rigor for


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 443

Lietz et al. Trustworthiness in Qualitative Research ■ 443

qualitative research (Creswell, 1998; Langer and Furman, 2005; Sandelowski, 1993; Seale, 2002; Szto et al., 2005). Sandelowski (1993: 1) asserts that ‘there is an inflexibility and an uncompromising harshness and rigidity implied in the term “rigor” that threaten to take us too far from the artfulness, versatility, and sensitivity to meaning and context that mark qualitative works of distinction’. Qualitative inquiry that stems from a postmodern paradigm denies the existence of an objective reality and instead focuses on the co-construction of meaning between the researcher and the participant (Guillemin and Gillam, 2004). Criteria that require postmodern researchers to evaluate the validity of their research do not fit the underpinnings of this paradigm. Defining ‘rigor’ in research as that which is ‘valid’ contradicts philosophical positions regarding social construction and the acknowledgment of multiple realities. Considering these differences, one could argue that the very construct of rigor and postmodern qualitative traditions are not aligned. Despite the differences between the postmodern qualitative traditions and positivist thinking, many argue that qualitative inquiry needs to have established procedures that can assess the quality of the work (Creswell, 1998, 2003; Horsburgh, 2003; Johnson and Waterfield, 2004; Li, 2004; Padgett, 1998). Although qualitative inquiry, as framed within a postmodern paradigm, acknowledges multiple realities and the role of social construction in establishing meaning, it is still important to establish some level of confidence that qualitative research represents the meanings of its participants (Creswell, 1998; Creswell and Miller, 2000; Horsburgh, 2003; Johnson and Waterfield, 2004; Li, 2004; Lincoln and Guba, 1985; Padgett, 1998; Seale, 2002). For example, Guillemin and Gillam (2004: 274) suggest that the process of co-constructing meanings is ‘an active process that requires scrutiny, reflection, and interrogation of the data, the researcher, the participants, and the context that they inhabit’. Accepting that qualitative inquiry involves the co-construction of meanings does not mean that no efforts should be made to minimize the effects of reactivity and bias therefore giving priority to the meanings of the participants over the perspective of the researcher. As Seale (2002: 108) states, ‘If there is one thing that produces poor studies, it is a researcher who is blind to the methodological consequences of research decisions’. Rigor in qualitative inquiry does not have to be an inflexible set of standards and procedures as is imposed in quantitative inquiry (Gambrill, 1995), but instead involves engaging in efforts that increase our confidence that our findings represent the meanings presented by our participants. Researchers such as Creswell (1998, 2003), Gambrill (1995) and Padgett (1998) support the usefulness of the concept of rigor in qualitative research in the field of social work. Padgett (1998) suggests that establishing a set of strategies to increase rigor in qualitative research will help qualitative researchers to manage reactivity and bias, legitimizing qualitative findings. In addition, some


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 444


would suggest that social work has an ethical responsibility to conduct qualitative research that is rigorous (Gambrill, 1995; Guillemin and Gillam, 2004; Padgett, 1998). Considering social work’s mission to raise awareness of the needs of underprivileged populations, our qualitative research should as closely as possible reflect the thoughts, feelings and experiences of the people who participate in our research. In this way, social work researchers prioritize the voice of the participant over that of our own. Desiring to give priority to the meanings of participants does not mean that qualitative researchers must deny the process of co-constructing meanings. The postmodern tradition acknowledges the role of the researcher in creating meaning with its participants (Guillemin and Gillam, 2004). However, rigorous qualitative work engages in activities that would assist us in giving priority to the meanings of participants over those which are our own. Therefore, efforts taken to manage issues of reactivity and bias can help social work researchers to describe qualitative data in a way that is credible. For example, Fawcett and Hearn (2004: 205) state that postmodern approaches to qualitative research ‘necessitates researchers’ values, prejudices, beliefs and attitudes being stated and interrogated, and their likely influence on the research being appraised’. Failing to engage in such efforts could allow qualitative research to hinder the voices of its participants. This would also contradict the underpinnings of postmodern thinking. Therefore, although there are debates about rigor and the fit between this concept and the postmodern philosophy (Creswell and Miller, 2000; Seale, 2002), qualitative researchers in social work are in need of strategies that can allow them to engage in the co-construction of meanings while also being able to hear and give priority to the meanings of their participants. Qualitative researchers have conceptualized the idea of rigor in multiple ways. For the purposes of this article, the work by Lincoln and Guba (1985), which discusses the concept of trustworthiness, will be used. Trustworthiness is established when findings as closely as possible reflect the meanings as described by the participants (Lincoln and Guba, 1985). Padgett (1998: 92) explains that trustworthiness is not something that just naturally occurs, but instead is the result of ‘rigorous scholarship’ that includes the use of defined procedures. Threats to trustworthiness can include problems such as reactivity and biases on the part of the researcher and the participant (Padgett, 1998). In order to manage these threats to trustworthiness, qualitative researchers must engage in a variety of strategies in order to describe research findings in a way that authentically represents the meanings as described by the participants (Creswell, 1998, 2003; Horsburgh, 2003; Lincoln and Guba, 1985; Padgett, 1998). The strategies include prolonged engagement, triangulation, peer debriefing, member checking, negative case analysis, audit trail and reflexivity (Creswell, 1998, 2003; Horsburgh, 2003; Johnson and Waterfield, 2004; Li, 2004; Lincoln and Guba, 1985; Mauther and Doucet, 2003; Padgett, 1998).


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 445


ESTABLISHING RIGOR IN A RESEARCH STUDY ABOUT SPIRITUALITY This article will describe an autoethnographic project that explored the spiritual identity of two researchers who completed extensive journal writing describing their daily experiences as a Jewish man and a Jewish woman. This article will present the strategies employed by these two researchers as they created a research team in order to establish rigor in their autoethnographic research project. Specifically, trustworthiness of the project was increased through the use of reflexivity, audit trail, triangulation, peer debriefing, member checking and prolonged engagement. Description of the Research Project

The research project described in this article is an autoethnography. The project involved two researchers who wrote about their spiritual identity. These autoethnographies reflect one woman’s and one man’s experience with their Jewish identity. The project stemmed from a research question that sought to explore and understand the daily experiences of spirituality for these researchers. Due to the exploratory nature of the project as well as the desire to understand a complex phenomenon such as spirituality, qualitative methodology was a good fit for the purposes of this research question. Qualitative inquiry allowed these researchers to explore spirituality within the context of the daily living of the two participants. The qualitative methodology offered the ability to seek depth in the understanding of a complex experience rather than seeking the breadth of findings that quantitative design offers. The exploration, depth and complexity of this project fit qualitative inquiry. Once the usefulness of qualitative inquiry was established, the first stage of the research project involved data collection. In order to collect data regarding daily experiences with spirituality, the two researchers completed their own personal journals with their daily reflections about spiritual identity. In order to develop their thoughts regarding this topic, the researchers completed periodic daily journals for one month describing how they experienced their spiritual identity throughout each day. During that one-month period, one researcher wrote in his journal 13 times and the other had 12 journal entries. Specifically, the two researchers wrote about their daily experiences regarding their Jewish identity. As described by Froggett and Chamberlayne (2004: 62), ‘biographical methods are well adapted to considering human attachments, belonging and suffering. They can also explore spaces and silences between people’. In these ways, biographical methods can help to explore the lived experience including a complex phenomenon such as spirituality. For the purposes of this project, autoethnography involved ‘a self-narrative that critiques the situatedness of self with other in social contexts’ (Spry, 2001: 710). Autoethnography is a method


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 446


that seeks to give voice to marginalized populations whose stories might otherwise not be told. As the researchers in this project sought to share the stories of their Jewish identity as well as the way that this identity is oppressed in the context of the dominant culture, the method of autoethnography fits the purposes of this project. The first participant is a Jewish man, raised in a Jewish home. He has identified himself as being Jewish throughout his life, yet he discussed currently feeling disconnected from the Jewish community. He stated, ‘It is amazing how a month turns into a year, and a year can turn into a decade. I keep intending to plug into the “Jewish Community” here, but find myself not doing so’. In addition to discussing his desire to reconnect with others of a similar belief system, he also shares stories about the ways in which he experiences his spiritual identify throughout his day-to-day life. For example, he wrote: Today, at our faculty retreat, a student overheard that I was Jewish. She asked me if I was Jewish by ethnicity or by religion. I found myself feeling defensive, and just answered ‘yes and yes’. I wondered how much I should have dialogued with her about the question, but found myself remaining uncharacteristically silent . . . this is how anti-Semitism has impacted me. I have internalized a defensive posture.

These and other stories highlight the value he finds in his Jewish heritage as well as the daily struggles he faces in regards to this identity. The other participant is a woman who is currently considering converting to Judaism. This participant studied Judaism for many years, and she identifies strongly with its belief system. In regards to this experience, she wrote: I’ve really thought about what it means to be Jewish. I tried to examine whether or not I felt ‘different’ than the driver in the car next to me or the gas station attendant where I stopped for a fill-up. It occurred to me that I didn’t feel any different because I can hide being Jewish. Nobody would ever have to know unless I told them. That thought made me realize that I rarely say ‘I am Jewish’ or ‘I am converting to Judaism’. I usually say ‘I’ve studied Judaism for the past several years’. It makes me wonder if I’m purposely hiding a part of myself.

Although the experiences of these two participants are not the same in that the history of their spiritual identity is quite different, the participants also identify many similar experiences with their Jewish identity. Both discuss a desire to pass this identity on to their children, their need to connect with the Jewish community and the value that they find in spiritual activities such as prayer and song. The Use of Reflexivity

Once the autoethnographies were completed, the two researchers involved in this project engaged in the first strategy used to increase rigor in this qualitative


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 447


study by engaging in ‘reflexivity’. Reflexivity is defined by Horsburgh (2003: 308) as ‘active acknowledgement by the researcher that her/his own actions and decisions will inevitably impact upon the meaning and context of the experience under investigation’. Reflexivity involves deconstructing who we are and the ways in which our beliefs, experiences and identity intersect with that of the participant (MacBeth, 2001). This reflection occurs both in individual thought and through dialog with others that acknowledges the researcher’s own experience and perspectives ( Johnson and Waterfield, 2004). Instead of trying to hide behind the false sense of objectivity, the researcher makes his or her own sociocultural position explicit. Finally, reflexivity is not a point in time event as the term ‘strategy’ might suggest. Rather, reflexivity is a process that occurs throughout the research (Guillemin and Gillam, 2004). Reflexivity is an important part of qualitative inquiry because it is through this reflection that qualitative researchers can ponder the ways in which who they are may both assist and hinder the process of co-constructing meanings (Guillemin and Gillam, 2004; Reich, 2003). This concept of standpoint and looking at the ways in which the researcher’s position or social location can interfere with the research process is an important concept when studying ‘others’ (Fawcett and Hearn, 2004). Mauther and Doucet (2003) suggest that one important way to engage in reflexivity is through meetings with a research group. In this project, the researchers engaged in extensive dialog regarding the project via email communication. For example, in this study the researchers engaged in reflexivity as they discussed with one another a plan for data analysis of their autoethnographies. Certainly, there are advantages to these researchers analyzing their own work as one of the strengths of autoethnography is the opportunity for the researcher to give voice to his or her own experiences (Reich, 2003). However, since the research project included the intersection of two autoethnographies, the researchers dialogued about the advantage of having a third person engage in the data analysis process. Specifically, the researchers felt that a third person might be able to uncover hidden meanings in each narrative. In addition, they also felt that the process of looking at connections and potential contradiction between the two documents might also work better by a third person. Finally, these researchers also talked about problems that can arise when analyzing one’s own narrative along with that of another. Russell and Kelly (2002) suggest that working with a team of researchers can help postmodern researchers to deal with subjectivity by having multiple perspectives come together in the decision-making process in qualitative research. It was through this reflexive communication that the researchers determined that the data analysis could be enhanced if another researcher was added to the team to conduct the data analysis. Once the researchers decided to seek an additional source to complete the data analysis, a third researcher became involved in the project. At this point,


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 448


reflexivity was again an important part of the research process. When this researcher was approached about analyzing journals that were written regarding Jewish identity, the researcher considered her own spiritual identity. Specifically, she wondered whether or not a Christian could analyze the spiritual experiences of a Jew in a way that would be authentic. As Mauther and Doucet (2003: 419) state, ‘Situating ourselves socially and emotionally in relation to respondents is an important element of reflexivity’. This reflexivity was a critical part of establishing rigor in this project, because acknowledging how one’s identity could both help and hinder interpretation of the narrative data is important (Reich, 2003). Reflexivity involved considering the multiple identities and perspectives of the researchers in consideration of ways in which these factors could both support the process of data analysis while also acknowledging the potential for reactivity and bias. If we do not consider the ways in which who we are may get in the way of portraying the voice of the participant, we may miss important meanings that are being presented by our participants. We are neither saying that in order to conduct rigorous qualitative data analysis a researcher must completely identify with the participant, nor are we saying that qualities of the researcher must match that of the participant. What we are saying is that asking questions, engaging in reflection on both differences and commonalities, will sensitize the researcher to ways in which varying perspectives can both support and potentially hinder this process. The reflexivity that occurred by this third researcher early on in the process occurred in two specific ways. First, extensive thought occurred prior to even agreeing to do the project. The researcher considered her own spiritual identity and questioned whether or not this identity might interfere with her ability to hear and understand the experience of being Jewish. She had attended a few Jewish ceremonies in the past and remembered feeling very out of place. She did not know how to read Hebrew and was not clear about spiritual traditions and what meaning they held for a person identifying with this belief system. Finally, she did not have extensive knowledge of the Jewish history affecting her ability to understand some of the context of the autoethnographies. Her willingness to acknowledge this lack of knowledge and to consider how this might affect the ability to understand context was important. In addition, reflecting on the feelings of being an outsider was also important as emotional reactions that stem from our own experiences may taint our ability to hear what the participants are trying to share. In addition to considering ways in which the standpoint of the researcher might hinder the process, the researcher also reflected on potential ways that her sociocultural position could support the process of data analysis. For instance, the researcher reflected on the differences in the identity attached to the spiritual experiences of a person identifying herself as Christian versus a person


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 449


identifying herself as Jewish. However, she also acknowledged that as a spiritual being she personally maintains a strong value in spiritual belief systems. Themes such as having a strong commitment to faith and an interest in meaning making related to our connection with a creator resonated with this analyst. So, just as differences were considered, commonalities between the researcher’s experiences and that of the participants were also considered. As she questioned her ability to analyze this data in a way that would be rigorous, she began to identify activities that could be useful to manage the potential threats of reactivity and bias and therefore agreed to participate in the project. After making the decision to engage in the data analysis of these autoethnographies, the researcher started by reading both narratives through completely. Before engaging in further analysis, she stopped and wrote her own narrative in response to the two autoethnographies she had read. This narrative identified her reactions to the journals. Much of the narrative records her reflexivity as feelings are described along with questions leading to further thought. For example, her narrative states: I was asked by a colleague to participate in a research project she is working on by providing the narrative analysis of two narratives written by herself and another faculty member regarding their experiences with being Jewish . . . I had mixed feelings. I remember thinking that I would enjoy this topic as I myself am a spiritual person and value spirituality . . . on the other hand, my other reaction was one of doubt, I wondered if I as a Christian could really understand the experiences of someone who is Jewish.

Two important things happened in this narrative. First, the author was transparent about her own Christian beliefs and the ways in which she identified and did not identify with the Jewish narratives. Identifying who we are and being explicit about our own perspectives is an essential part of being reflexive (MacBeth, 2001). In addition to this acknowledgement of self, the narrative reflection also includes many questions. Asking questions leads to more reflection and is therefore an important part of the process of reflexivity. These questions do not eliminate threats to trustworthiness. Instead, asking the questions sensitized the researcher to these threats while leading to an awareness of the researcher’s sociocultural position in relation to these autoethnographies. This awareness of standpoint then compelled her to be more diligent in the data analysis process. Once the researcher completed her own narrative, she continued by completing the data analysis of the two autoethnographies. The Use of Audit Trail

During the data analysis process, the researcher engaged in another strategy used to manage the threats to trustworthiness. This researcher kept an audit trail throughout the data analysis process that clearly described the steps she took


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 450


including a description of the reflexivity within each step. An audit trail is an important part of establishing rigor in qualitative work as it describes the research procedures ( Johnson and Waterfield, 2004; Padgett, 1998). As Sandelowski (1993) suggests, qualitative inquiry can be considered an art that requires versatility in order to create research procedures that can fit unique circumstances that exist when studying complex phenomenon. Although flexibility is an important component of qualitative work, this does not mean that research procedures are not a part of qualitative inquiry. Instead, qualitative researchers must be more diligent in describing the research decisions that are made along the way since there may be a greater range of versatility present in qualitative work. An audit trail allows a researcher the freedom to make unique research decisions not previously prescribed while still requiring that each decision and the justification for that decision be recorded along the way. An audit trail allows the researcher to follow his or her own research procedures consistently. It also helps a qualitative project to be open for critique by the research community as the research procedures are fully described. The ability to critique research is an essential part of the research process. Whether it is quantitative or qualitative inquiry, critical analysis is a part of the research tradition. Audit trails allow for critical thinking to occur in qualitative inquiry. The Use of Triangulation and Peer Debriefing

During the initial analysis of this data, the researcher used reflexivity and audit trail to increase the trustworthiness of this project. Once the initial analysis was completed, this researcher sent her analysis along with her personal narrative to the two other researchers to engage in further dialog regarding the trustworthiness of this analysis. The reflexivity at this point in the process included extensive dialog, both in person and through electronic communication that allowed the three members of this research team to consider whether or not this data analysis was a trustworthy representation of the themes identified in the journals. Working in a research team, as suggested by Russell and Kelly (2002), allowed multiple perspectives to be considered at each stage in the research process. During this dialog, the researchers considered many questions that relate to standpoint and the complex nature of researching others. As Fawcett and Hearn (2004: 201) discuss, attention must be given to ‘historical context and to the maintenance of a critical relation to the research topic’. In this way, the team reflected on issues of spiritual identity that may keep hidden meanings from being uncovered by researchers who could not understand the history of oppression of Jewish people as well as the context of some of what was shared. For example, some of the Jewish traditions described in the autoethnographies were outside the scope of experience for the third researcher. Since this was not an interview in which clarification could occur through the data collection process, the analyst was left with some gaps of understanding. By considering these issues


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 451


of standpoint the research team determined that they needed further efforts in order to increase the trustworthiness in this project. It was at this point that the research team decided to contact a fourth outside source to analyze the autoethnographies one more time. At this point, the research team decided to use ‘peer debriefing’ and ‘triangulation by observation’ (Creswell, 1998, 2003; Horsburgh, 2003; Johnson and Waterfield, 2004; Li, 2004; Padgett, 1998). Triangulation by observation uses two observers to look at qualitative data. Having two observers analyze qualitative data allows commonalities and differences between the two observers to be analyzed further. Triangulation can be an important strategy for establishing rigor in qualitative work as opposing perspectives can bring an increased understanding of the data (Creswell, 1998, 2003; Johnson and Waterfield, 2004; Li, 2004). The research team engaged in triangulation by observation when they asked another person who had not been involved in this project to conduct a simple data analysis of the autoethnograpies as a check against the initial analysis. Considering the personal nature of the topic of spiritual identity, the research team decided to seek a colleague who is both capable in the area of narrative analysis as well as being someone who identified themselves as being Jewish. This matching of analyst and participants was important to the researcher who engaged in the original analysis of this data. Since one of the critical questions she identified about her own analysis was whether or not a Christian could accurately analyze the spiritual experiences of a Jew, this initial analyst wanted to compare her findings with that of someone who identified with the spiritual tradition similar to that of the participants. In addition, the desire to match the spiritual identity of the analyst and participants also stemmed from the original two researchers. They similarly wondered whether a Christian analyst could establish trustworthiness of the findings in a study about being Jewish. The research team was able to identify a colleague who was willing to do a less extensive analysis of the data in order to compare the two interpretations. It is important to note that the time constraints of this outside resource led the second analysis to utilize significantly less reflexivity than the first.A narrative response was not completed. In addition, the second analyst did not keep an audit trail. Once the second data analysis was complete, the two researchers who completed the analyses met for ‘peer debriefing’. Peer debriefing involves the process of engaging in dialog with colleagues outside of a research project who have experience with the topic, population or methods being utilized (Creswell, 1998, 2003; Li, 2004; Lincoln and Guba, 1985; Padgett, 1998). Peer debriefing was an important part of this project, as it allowed a member of the research team to consult with a colleague outside of the research in order to minimize the effects of reactivity and bias. During this peer debriefing, conversation between these two analysts showed some consistencies in analysis as well as some clear differences. Both researchers identified themes related to spiritual identity


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 452


and identified both struggles and strengths associated with maintaining this spiritual identity. For example, both analysts discussed seeing a struggle in both autoethnographies regarding living as a Jew when this belief system is not the dominant paradigm of our culture. Both analysts identified anti-Semitism and the oppression that results. Both analysts also identified strengths in both narratives regarding the benefits of this spiritual identity. For example, both analysts saw the role of religious practices such as ceremonies, prayer and song as connecting both participants to something greater than themselves. In addition, both analysts identified the importance of connecting with the Jewish community as well as a connection to the history of the Jewish culture. Despite identifying similar themes in the autoethnographies, differences in interpretations also existed. One of the more striking differences was the way in which the process of analysis occurred. The primary analyst looked at both autoethnographies to identify common themes among the two. However, the second analyzed the autoethnographies separately. She felt that the experience of being Jewish as a ‘birthright’ versus being Jewish by ‘choice’ through the conversion process made the two narratives so different that she analyzed them separately. She also stated during the peer debriefing that she felt like she could identify very strongly with the first narrative, the one who was raised Jewish, compared with the experience of the second participant. This led the findings of the Jewish analyst to see more differences between the two journals than did the first. The Jewish analyst saw the Jewish identity in one journal as being ‘embedded’. She stated, ‘Jewish identity is so deep it transcends actual physical involvement in a community or organization’. With regard to the journal of the participant who was converting to Judaism, she stated ‘as a convert, the identity seems mixed in the “how to do it”, yet there is a desire to be part of a community’. The first analyst did not see Jewish identity as that different between the two. Instead, she presented this identity as being both deeply personal and including a component of connection with community by both authors. She identified the struggle as a reaction to the larger society asking ‘How do I develop/express my Jewish identity when the society in which I live does not acknowledge this identity?’. These differences are important in that they illustrate the benefit of the strategy of triangulation by observer. Having two analysts look at this data allowed consistencies and differences in interpretation to surface. These differences led to further reflexivity and questioning about threats to trustworthiness of the findings as the two analysts met for a peer debriefing meeting to discuss their findings. The Jewish researcher questioned whether or not the Christian researcher could truly understand Jewish narratives as she was lacking some knowledge related to the use of language within the narratives as well as missing an understanding and therefore compassion for the history of the Jewish people. On the other hand, the Christian researcher felt that although the Jewish


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 453


researcher could understand the language and history to a greater degree, she also felt that at times the interpretation by the Jewish analyst transcended the journals of the participants and may have stemmed from the Jewish researcher’s own experiences. Fascinating questions arose during the dialog including whether or not identifying with research participants may actually make interpretation even more difficult. On the one hand, the personal knowledge of being Jewish was of benefit, because the Jewish researcher had a better understanding of the context. On the other hand, not identifying with this identity led the first analyst to work harder at seeing the meanings of the participants as she had less of her own context and knowledge to assist in this process. The peer debriefing led both analysts to question as well as recognize ways in which their analysis could have been both hindered and strengthened based on who they are. The Use of Member Checking and Prolonged Engagement

Once the second analysis was complete and the peer debriefing meeting occurred between the two analysts, the findings from both were shared with the original research team, the authors of the autoethnographies. At this point, the research team engaged in ‘member checking’ when the findings from both analyses were shared with the participants in order to check with the source the trustworthiness of what was found. Member checking, also known as respondent validation, allows participants to review findings from the data analysis in order to confirm or challenge the accuracy of the work (Creswell, 1998, 2003; Horsburgh, 2003; Johnson and Waterfield, 2004; Lincoln and Guba, 1985). This is an important strategy used to establish trustworthiness as it gives authority to the participants’ perspectives therefore managing the threat of bias (Padgett, 1998). The member checking in this project involved sending both researchers/participants the findings from both analysts in order for them to discuss which aspects of the data analysis best fit their perspectives. The participants were also able to identify areas that may have been missed or misinterpreted. In response to this task, the two researcher/participants decided to complete their member check by writing a final journal entry that would discuss their reactions to the findings. To some degree, the research team also engaged in a final strategy known as ‘prolonged engagement’ at this step. The idea of prolonged engagement stems from anthropological fieldwork in which researchers spent extensive time with their participants in order to increase rapport leading participants to be more open in their interactions with the researcher (Creswell, 1998, 2003; Li, 2004; Lincoln and Guba, 1985; Padgett, 1998). Furthermore, collecting additional data and spending more time with the participants increases the ability for a qualitative researcher to reach saturation of the data. Having the participants engage in the process of writing one


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 454


additional journal entry allowed the researchers to collect additional qualitative data increasing the trustworthiness of the findings. The feedback provided from the participants in their member check identified aspects of both analyses that seemed to accurately reflect their experiences. In addition, there were aspects of both analyses that the members reported did not resonate with experiences. For example, during this member check, one participant stated, ‘The second analysis leaves me feeling a bit misunderstood. Two statements seem to be misinterpretations of what I was saying’. Similarly, the other participant wrote: The reason that she was asked to do the analysis was to see if there would be a difference between her analysis since she is Jewish. Yes, there is a difference, and yes, she identifies some things that the other analyst didn’t understand. She comes down on the issue of identity with both journal authors, I believe, in a way that comes from her own heritage, not from her training as a researcher. This does speak to what Richardson maintains about all data analysis being socioculturally situated. It does not make her work more or less accurate. It does, on a personal level, feel less accepting.

During the reflexivity that occurred during the peer debriefing meeting, the Jewish analyst recognized that she identified with the first participant who was raised in a Jewish home more than she did with the participant who was converting. Both participants wondered in their member check about the ways in which the history of this analyst was both helping and hindering the way she interpreted the data. In response to feeling that there was some misinterpretation by the second analysis, one participant stated: I would like to talk to her. I think what is missing from using narratives only is the face to face interaction that happens during interviews.

In this comment, again, we can see reflexivity that occurs through dialog, or the lack of it, is highly important to achieving the ultimate goal of understanding. Although dialog did occur between the two analysts during the peer debriefing, all four researchers involved in this project never met. Looking back, we wonder if dialog could have led to better understanding. In addition, as mentioned earlier, the second analyst did not engage in extensive reflexivity and did not complete an audit trail. Conducting her analysis without these critical strategies may have also affected the trustworthiness of her findings. However, having two researchers conduct the analysis and then engage in peer debriefing helped to minimize these effects as the dialog between the two analysts along with the member check ultimately managed these threats of reactivity and bias.


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 455


Interestingly, the participants seemed to feel that the primary analyst ultimately presented findings that were a better representation of their stories. For example, one participant stated ‘I believe she captures what I was saying in my journals’, while the other stated ‘the themes that she uncovered appear congruent with the data’. The original question was, can a Christian researcher analyze data about Jewish identity in a way that is trustworthy? What we discovered was, yes, this can happen. However, this can only happen when extensive efforts occur that support that analyst in achieving rigor in the process. By engaging in several strategies to increase trustworthiness including reflexivity, audit trail, member check, peer debriefing, and prolonged engagement, this researcher was able to present findings in this project in a way that was found to be trustworthy by its participants. As one of the participants stated, the ‘self reflective narrative inspired trust in me, from the perspective of the participant. She clearly explores her own bias and is transparent’. Efforts such as the reflexivity seemed to allow this researcher to minimize the threats of bias and reactivity by being transparent and open about the problem of standpoint. At the end of this project, it appeared that the standpoint of the researchers conducting the data analysis had the potential to both help and hinder the process. In this project, it was not the standpoint that created or interfered with trustworthiness of the findings, but instead it was the research procedures that were used that seemed to make the difference. Final Use of Reflexivity

As the project was completed, the researchers again engaged in reflexivity regarding this project. Through this discussion, we agreed that the process of engaging in multiple strategies in order to increase trustworthiness was one of the most worthwhile aspects of this experience. As discussed earlier, the term rigor has commonly been associated with the positivist paradigm and can suggest a rigidity that would not fit well with qualitative research (Sandelowski, 1993). However, after engaging in multiple strategies that seek to establish rigor in a qualitative project, we found these strategies not only to meet the needs of versatility inherent in qualitative work, but we also felt that these strategies greatly strengthened the findings of the project. Specifically, we felt that the ongoing efforts of reflexivity were a critical way that we managed reactivity and bias in this project. In addition to the value of reflexivity, the member check was also a critical way that trustworthiness was strengthened in this project. Even after engaging in triangulation by observer and peer debriefing, there remained some contradictions between the findings of the two analysts. The only way to assess which analysis better fitted the experiences of the participants was to ask the participants themselves. It is our opinion that reflexivity and member checking were


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 456


the most critical strategies employed in this project. We also feel that the other strategies, the audit trail, the triangulation and the peer debriefing greatly added to the rigor of the project as well.

IMPLICATIONS This project offers implications regarding developing rigor in qualitative research. The results show that engaging in strategies such as reflexivity, audit trail, triangulation, peer debriefing, member checking and prolonged engagement can greatly add to the trustworthiness of a qualitative project. Finally, the project found that engaging in such strategies that seek to build rigor in qualitative work did not hinder the flexibility or artfulness of this project. Instead, these strategies placed the participant’s voice at the highest priority. Presenting findings that reflected the feelings of the participants as closely as possible was the ultimate goal of the project. As Padgett (1998) states, there is an ethical responsibility within social work to uncover voices that have been hindered or to bring awareness to perspectives that have been oppressed. It is through rigorous research activity within qualitative work that we can work to bring awareness to thoughts, ideas and experiences not commonly heard. In order to do this, we must have strategies that will allow us to manage threats to trustworthiness including the ways in which who we are and what we have experienced may keep us from hearing what our participants are saying. References

Aguinaldo, J. P. (2003) ‘Rethinking Validity in Qualitative Research from a Social Constructionist Perspective: From “Is this Valid Research?” to “What is this Research Valid For?”’, The Qualitative Report 9(1): 127–35, http://www.nova.edu/ssss/ QR/QR9–1/aguinaldo.pdf (consulted August 2005). Bein, A. and Allen, K. (1999) ‘Hand in Glove?: It Fits Better Than You Think’, Social Work 44(3): 274–7. Borochowitz, D. Y. (2005) ‘Teaching a Qualitative Research Seminar on Sensitive Issues’, Qualitative Social Work, 4(3): 347–62. Brun, C. (1997) ‘The Process and Implications of Doing Qualitative Research: An Analysis of 54 Doctoral Dissertations’, Journal of Sociology and Social Welfare 24(4): 95–112. Creswell, J. W. (1998) Qualitative Inquiry and Research Design: Choosing Among the Five Traditions. Thousand Oaks, CA: Sage. Creswell, J. W. (2003) Research Design: Qualitative, Quantitative and Mixed Methods Approaches, 2nd edn. Thousand Oaks, CA: Sage. Creswell, J. W. and Miller, D. L. (2000) ‘Determining Validity in Qualitative Inquiry’, Theory into Practice 39(3): 124–30. Fawcett, B. and Hearn, J. (2004) Researching Others; Epistemology, Experience, Standpoints and Participation’, Social Research Methodology 7(3): 201–18.


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 457


Froggett, L. and Chamberlayne, P. (2004) ‘Narratives of Social Enterprise’, Qualitative Social Work 3(1): 61–77. Gambrill, E. (1995) ‘Less Marketing and More Scholarship’, Social Work Research 19: 40–7. Gilgun, J. (1994) ‘Hand into Glove. The Grounded Theory Approach and Social Work Practice Research’, in E. Sherman and W. J. Reid (eds) Qualitative Research in Social Work, pp. 115–25. New York: Columbia University Press. Guillemin, M. and Gillam, L. (2004) ‘Ethics, Reflexivity, and “Ethically Important Moments” in Research’, Qualitative Inquiry 10(2): 261–80. Horsburgh, D. (2003) ‘Evaluation of Qualitative Research’, Journal of Clinical Nursing 12(2): 307–12. Johnson, R. and Waterfield, J. (2004) ‘Making Words Count: The Value of Qualitative Research’, Physiotherapy Research International 9(3): 121–31. Langer, C. and Furman, R. (2005) ‘Beyond the Ethnographic Interview: The Research Poem as a Tool for Teaching Culturally Sensitive Social Work with Natives’, Arete 28(2): 93–9. Li, Defeng (2004) ‘Trustworthiness of Think-aloud Protocols in the Study of Translation Process’, International Journal of Applied Linguistics 14(3): 301–13. Lincoln, Y. S. and Guba, E. G. (1985) Naturalistic Inquiry. Beverly Hills, CA: Sage. Lowery, D. and Evans, K. (2004) ‘The Iron Cage of Methodology’, Administration & Society 36(3): 306–27. MacBeth, D. (2001) ‘On ‘Reflexivity’ in Qualitative Research: Two Readings, and a Third’, Qualitative Inquiry 7(1): 35–68. Mauther, N. S. and Doucet, A. (2003) ‘Reflexive Accounts and Accounts of Reflexivity in Qualitative Data Analysis’, Sociology 37(3): 413–31. Morgan, A. K. and Drury, V. B. (2003) ‘Legitimizing the Subjectivity of Human Reality through Qualitative Research’, The Qualitative Report 8(1), http://www.nova.edu/ ssss/QR/QR8–1/morgan.html (consulted August 2005). Padgett, D. K. (1998) Qualitative Methods in Social Work Research: Challenges and Rewards. Thousand Oaks, CA: Sage. Padgett, D.K. (2004) ‘Introduction: Finding a Middle Ground in Qualitative Research’, in D. K. Padgett (ed.) The Qualitative Research Experience, pp. 1–13. Belmont, CA: Thomson/Brooks & Cole. Reich, J. (2003) ‘Pregnant with Possibility: Reflections on Embodiment, Access, and Inclusion in Field Research’, Qualitative Sociology 26(3): 351–67. Russell, G. M. and Kelly, N. H. (2002) ‘Research as Interacting Dialogic Processes: Implications for Reflexivity’, Forum: Qualitative Social Research 3(3): 1–19. Rubin, A. and Babbie, E. (2005) Research Methods for Social Work. Belmont, CA: Thomson/Brooks & Cole. Sandelowski, M. (1993) ‘Rigor or Rigor Mortis: The Problem of Rigor in Qualitative Research Revisited’, Advanced in Nursing Science 16(2): 1–8. Seale, C. (2002) ‘Quality Issues in Qualitative Inquiry’, Qualitative Social Work 1(1): 97–110. Spry, T. (2001) ‘Performing Autoethnography: An Embodied Methodological Praxis’, Qualitative Inquiry 7(6): 706–32.


03 070288 Lietz (to_d)

27/10/06

1:32 pm

Page 458


Stein, H. F. (2004) ‘A Window to the Interior of Experience’, Families, Systems, & Health 22(2): 178–9. Szto, P., Furman, R. and Langer, C. (2005) ‘Poetry and Photography: An Exploration into Collaborative Research Processes’, Qualitative Social Work 4(2): 135–56.

Cynthia A. Lietz, PhD, LCSW, is an Assistant Professor in the Department of Social Work at Arizona State University. Dr Lietz has worked in practice with children and families. Her primary areas of research include the strengths perspective, family resilience and clinical supervision. She is also interested in the use of qualitative methodology in social work. Address: Arizona State University at the West Campus, College of Human Services, Social Work Department, PO Box 37100, Phoenix, AZ 85069-7100, USA. [email: [email protected]]

Carol L. Langer, MSW, PhD, is an Associate Professor and Department Chair at the Department of Social Work at the University of Wisconsin – Eau Claire. Dr Langer has practice experience in medical and school social work. She has more than 20 years of experience in post-secondary education. Her primary areas of research interest include American Indian traditional wellness and social work education. Address: Department of Social Work, 253 Human Sciences and Services Bldg., University of Wisconsin – Eau Claire, 105 Garfield Avenue, PO Box 4004, Eau Claire, WI 54702-4004, USA. [email: [email protected]]

Rich Furman, MSW, PhD, is an associate professor and coordinator of the BSW program in the Department of Social Work at the University of North Carolina at Charlotte. His areas of research include: transnational social work with Latinos; arts and humanities inspired qualitative methods; and friendship. Address: College of Health and Human Services, Department of Social Work, University of North Carolina at Charlotte, 9201 University City Blvd., Charlotte, NC 28223-0001, USA. [[email protected]]
