Anything You Can Do, I Can Do Better: Bias

JOURNAL OF FORENSIC PSYCHOLOGY RESEARCH AND PRACTICE https://doi.org/10.1080/24732850.2017.1413532

Anything You Can Do, I Can Do Better: Bias Awareness in Forensic Evaluators Marissa Zappalaa, Amanda L. Reeda,b, Amanda Beltrania,c, Patricia A. Zapfa, and Randy K. Ottod John Jay College of Criminal Justice, The City University of New York, New York, USA; bThe Graduate Center, The City University of New York, New York, USA; cDepartment of Psychology, Farleigh Dickinson University, Teaneck, USA; dDepartment of Mental Health Law & Policy, University of South Florida, Florida, USA

Downloaded by [47.197.132.55] at 10:28 22 December 2017

a

ABSTRACT

KEYWORDS

Psychologists and other mental health professionals are frequently called upon by the courts to provide professional opinions on a multitude of psycholegal issues in civil and criminal proceedings. With the expectation for sound, objective opinions and the regularity with which these forensic assessments occur, the crucial role of evaluators is undeniable; however, research has confirmed the presence of bias in some psychological evaluations (Murrie, Boccaccini, Guarnera, & Rufino, 2013; see Neal & Grisso, 2014 for a discussion). Investigations regarding bias awareness reveal that most individuals lack recognition of their own biases in comparison with their perceptions of bias in others—an issue known as the bias blind spot (Pronin, Lin, & Ross, 2002)—but persist in believing introspection to be an effective technique for reducing bias (Neal & Brodsky, 2016; Zapf, Kukucka, Kassin, & Dror, 2017), despite research indicating that introspection is ineffective for reducing bias (Pronin & Kugler, 2007). The current study surveyed a sample of forensic mental health professionals (N = 80) to assess for a “bias blind spot” among forensic evaluators and to explore the effectiveness of an intervention to reduce reliance on introspection as a debiasing technique. There was evidence for a bias blind spot among evaluators and a high level of endorsement for the use of introspection. However, the intervention used to decrease reliance on introspection did not have an effect, indicating the need for additional research on effective strategies for reducing the bias blind spot and educating evaluators about those strategies.

bias blind spot; cognitive bias; forensic evaluation; forensic mental health assessment

With increasing frequency, mental health professionals are called to proffer opinions on psychological issues pertaining to a host of legal matters relevant to civil and criminal justice systems (Melton, Petrila, Poythress, & Slobogin, 2007). Based on education, training, knowledge, and experience, forensic evaluators are entrusted with conducting comprehensive psychological evaluations and offering informed opinions with the potential to impact the legal CONTACT Patricia A. Zapf © 2017 Taylor & Francis Group, LLC

[email protected]

5668 FishHawk Crossing Blvd, #306, Lithia, FL 33547.


2

M. ZAPPALA ET AL.

fate of the evaluee. Research has demonstrated high rates of concordance— upwards of 95%—between expert opinions and legal adjudications, suggesting that the courts rely heavily on the opinions of forensic evaluators (Zapf, Hubbard, Cooper, Wheeles, & Ronan, 2004). Given the significant weight placed by the court on the opinions of forensic evaluators, objectivity and accuracy are crucial. When conducting psycholegal assessments, forensic evaluators are held to specific ethical and professional standards (e.g., Specialty Guidelines for Forensic Psychology; American Psychological Association, 2013). Complete objectivity in clinical judgment is aspirational; ideally, all evaluators would come to identical conclusions when provided the same case presentations (Murrie, Boccaccini, Zapf, Warren, & Henderson, 2008). However, one threat to impartiality comes in the form of bias, variously defined as a deviation from the norm or slanting one way rather than another, but evolving to imply an error in judgment (see Neal & Grisso, 2014 for a review). In the realm of forensic mental health evaluation, bias may be implicit—operating outside of an evaluator’s conscious awareness—or explicit and has come to have a negative connotation. Research has demonstrated that various biases can impact forensic evaluations, including adversarial affiliation (Chevalier, Boccaccini, Murrie, & Varela, 2015; Murrie, Boccaccini, Guarnera, & Rufino, 2013; Otto, 1989), personal attitudes and values (Deitchman, Wallace, Kennedy, & Beckham, 1991; Everson & Sandoval, 2011), evaluator discipline and training (Murrie et al., 2008), and variation in personality traits (Miller, Rufino, Boccaccini, Jackson, & Murrie, 2011). Zapf and Dror (2017) delineated a seven-level taxonomy of the various influences that can result in bias in forensic mental health evaluation, including issues relevant to human nature and the cognitive architecture of the brain, environment, culture, and experience (i.e., training and motivation; organizational factors; base rate expectations), and influences that stem from case-specific details. Specific forms of cognitive bias such as overreliance on memory, underutilization of base rates, anchoring, overconfidence, and overreliance on unique data have been identified as having the potential to impact the accuracy of forensic evaluations (see Borum, Otto, & Golding, 1993 for a review). Prominent biases that are of particular interest in the current study include hindsight bias, fundamental attribution bias, confirmatory bias, and the illusory correlation (see Borum et al., 1993). Hindsight bias refers to the tendency for outcome information to impact one’s perception and judgment about the predictability of that outcome. The fundamental attribution error is recognized as the tendency for a person to emphasize personcentered factors over contextual factors when explaining and judging the behavior of others. Confirmatory bias is the tendency to seek out data that

JOURNAL OF FORENSIC PSYCHOLOGY RESEARCH AND PRACTICE

3

aligns with or confirms one’s hypotheses or preliminary impressions, and to neglect (or not search for) information that might disconfirm these impressions. Finally, the illusory correlation is the tendency to see a relationship between two variables when no relationship exists.


Bias awareness and bias blind spot Little experimental research has examined forensic evaluators’ awareness regarding their susceptibility to bias. In a preliminary attempt to capture forensic psychologists’ beliefs about bias, Neal and Brodsky (2016) collected qualitative responses from a sample of 20 forensic psychologists who were certified by the American Board of Professional Psychology (ABPP) and found that bias awareness appears to fall along a continuum, with some professionals denying the potential for bias and others maintaining the inevitability of bias. Further research has investigated the presence of a bias blind spot among forensic evaluators. The bias blind spot, first identified by Pronin, Lin, and Ross (2002), refers to the tendency of individuals to see bias in others as a greater cause for concern than bias in oneself. Zapf and colleagues surveyed over a thousand mental health professionals who conduct forensic evaluations and found a response pattern suggestive of a bias blind spot, with 79% of respondents acknowledging bias as a cause for concern in forensic evaluation but only 52% acknowledging bias as a concern in their own evaluations (Zapf, Kukucka, Kassin, & Dror, 2017). Boccaccini, Chevalier, Murrie, and Varela (2015) surveyed 95 forensic evaluators about their use of the Psychopathy Checklist Revised (PCL-R) in sexually violent predator cases. They inquired about participants’ beliefs surrounding the PCL-R’s susceptibility to adversarial allegiance, generally, as well as a more nuanced inquiry of participants’ beliefs of their own susceptibility to bias. They found that participants’ responses reflected a bias blind spot, evidenced by participant reports that PCL-R scores in general may be influenced by adversarial allegiance, but their own PCL-R scores would not be impacted by the same potential bias. In addition, they found a smaller but similar effect for a bias blind spot with regard to the Static-99, an actuarial measure for risk (see also Chevalier et al., 2015). The bias blind spot is a robust phenomenon (see Pronin, 2007; Pronin, Gilovich, & Ross, 2004; Pronin & Kugler, 2007), which persists despite education about this tendency (see Pronin et al., 2002). In an attempt to further understand the bias blind spot and ways in which bias might be minimized, Pronin and colleagues (2007) inquired about various strategies that people use to combat bias. They found that thinking about one’s own biases—introspection—was identified as a primary strategy for bias reduction; however, research has demonstrated that introspection is not an effective bias reduction strategy, given human inability to access higher-order


4

M. ZAPPALA ET AL.

cognitive processes (see Nisbett & Wilson, 1977). Pronin and Kugler (2007) argued that this “introspection illusion”—the belief that one can combat bias simply by thinking about one’s own biases—results from the nonconscious operation of cognitive biases, which renders their influence hidden from introspection. That is, the mental contamination that results from implicit bias is difficult to control as a result of the fundamental feature of human cognition that makes us unaware of many of our mental processes (Nisbett & Wilson, 1977; Wilson & Brekke, 1994). Neal and Brodsky (2016) found this introspection illusion at play in their survey of 351 forensic psychologists regarding strategies used to minimize or combat the impact of bias in forensic evaluation. All of the respondents reported attempting to minimize bias by introspection. Similarly, Zapf and colleagues (2017) indicated that the majority of respondents (87%) believed they could minimize bias simply by consciously trying to set aside their biases and expectations. Boccaccini and colleagues (2015) also reported that the insufficiency of introspection for recognizing one’s own bias could be a contributor to the observed bias blind spot in their sample. In an attempt to determine ways in which cognitive biases might be mitigated, Pronin and Kugler (2007) successfully used education about the shortcomings of introspection to reduce the bias blind spot in a sample of undergraduates. A brief article, published in Science (“Unaware of Our Unawareness”; Mandel, 2005), discussing the effects of unconscious processes on human behavior and emphasizing the inability to access these unconscious processes, resulted in an increased awareness of the potential for unconscious bias in oneself. The current study sought to test whether this same article could be used to increase bias awareness in a sample of forensic evaluators. Specifically, we hypothesized that (a) evidence for a bias blind spot would be demonstrated by a discrepancy between participants’ self- and other-ratings for various biases, and that education about the fallibility of introspection as a bias reduction strategy would (b) increase bias awareness, and (c) result in a lower ratings of the efficacy of this technique for bias reduction.

Method Participants

Participants were 80 forensic mental health professionals who attended one or more continuing education workshops offered between June 2015 and June 2016. The sample was primarily female (72.5%), with an average age of 38.8 years (Range = 21–67; SD = 12.5), and an average of 10.7 years of experience (SD = 10.0). Most participants held a doctoral degree (41.3%) or a master’s degree (35.0%), with 3.8% of the sample having obtained a medical


5

Table 1. Demographic Data.


Gender n (%) Male Female Age mean (SD) Highest degree n (%) Doctoral Master’s MD Bachelor’s Missing Years of Experience Mean (SD)

Total Sample (n = 80)

Rate Self, Article (n = 23)

Rate Other, Article (n = 16)

Rate Self, No Article (n = 23)

Rate Other, No Article (n = 18)

22 (27.5) 58 (72.5) 38.8 (12.5)

7 (30.4) 16 (69.6) 37.3 (13.1)

6 (37.5) 10 (62.5) 40.2 (11.9)

9 (39.1) 14 (60.9) 41.3 (12.9)

0 (0) 18 (100) 36.4 (12.2)

33 (41.3) 28 (35.0) 3 (3.8) 5 (6.3) 11 (13.8) 10.7 (10.0)

9 (39.1) 8 (34.8) 0 (0) 1 (4.3) 5 (21.7) 11.0 (11.8)

6 (37.5) 6 (37.5) 2 (12.5) 1 (6.3) 1 (6.3) 11.6 (9.4)

10 (43.5) 9 (39.1) 1 (4.3) 1 (4.3) 2 (8.7) 10.8 (9.6)

8 (44.4) 5 (27.8) 0 (0) 2 (11.1) 3 (16.7) 9.0 (9.3)

Note. No significant differences were found between the four groups with respect to age, F(3) = .674, p = .571, or years of experience, F(3) = .162, p = .92.

degree (MD). Demographic data for the sample are presented in Table 1. Comparisons between randomly assigned groups, representing conditions in this study, indicated no significant differences between groups with respect to age, education, or experience.

Materials Demographic questionnaire Participants completed an extensive demographic questionnaire as part of a larger study, with items inquiring about a wide range of demographic, training, and professional variables. Only information relevant to this study was included in the analysis. Bias survey Each participant completed a survey inquiring about four biases that occur in forensic evaluation: Illusory Correlation, Hindsight Bias, Fundamental Attribution Error, and Confirmation Bias (see Borum et al., 1993)—with a brief definition of each bias followed by directions for participants to rate their own (half the sample) or their peers’ (half the sample) susceptibility or vulnerability to each bias. Ratings were made on a 9-point Likert-type scale, anchored at three points (1 = not at all, 5 = somewhat, 9 = strongly). All participants were then asked, “To what extent do you believe that self-reflection and introspection are effective at helping you remain aware of biases that might affect your work?” This question was rated on the same 9-point Likert-type scale.

6

M. ZAPPALA ET AL.


Introspection article Pronin and Kugler (2007) used a brief article on unconscious processes and behavior to educate participants about the shortcomings of introspection. This short article, published in Science by Mandel (2005)—“Unaware of Our Unawareness”—was used as an intervention to provide education concerning the flaws of human judgment, a training technique suggested by Neal and Brodsky (2016). Pronin and Kugler (2007) demonstrated that this short article appears to reduce the bias blind spot; therefore, it was included in half of the self-rating packets and half of the peer-rating packets. If this article was read and understood, readers should recognize their own vulnerability to biases and thus be more likely to assign themselves a bias rating similar to the one they assigned to their peers. In addition, they should also recognize the limits of their consciousness in identifying their own biases. Procedure

This project used data from a large-scale research project conducted across two series of professional development workshops between June 2015 and June 2016. A 2 (self-rating vs. other-rating) x 2 (received article on introspection vs. no article) between-groups design was used. Prior to the start of the workshops, survey packets for the larger study were assembled so that one of the four conditions was represented in each packet. The packets were randomly distributed among workshop attendees as part of the check-in process for the workshops. At the beginning of each workshop, participants were provided with a brief explanation of the research and a request to participate. Participation was voluntary and all participants were informed that their decision regarding whether or not to participate would not impact their training. They were asked to complete the consent form (included as part of the research packet) and the other questionnaires at their convenience and return their research packets at the completion of their workshop. Participants were offered a small token (value $25) in exchange for their participation. All procedures were approved by the Institutional Review Board of [Institution redacted for blind review]. Results All analyses were conducted using SPSS version 24. Participants’ ratings of self and others’ susceptibility or vulnerability to four cognitive biases are presented in Table 2. For each of the four types of biases, participants rated their own susceptibility lower than their peers’ susceptibility to that same bias. To evaluate the effects of the type of bias reference (self vs. other) and the presence of the introspection illusion article (article vs. no article), a two-way MANOVA was calculated. A statistically significant main effect for bias reference (self or other) on the bias scores was found, F(4, 73) = 8.878, p < .001, Wilks’ Λ = .673, partial η2 = .327, but


7

Table 2. Ratings of Self and Others’ Ssusceptibility to Cognitive Biases. Self Ratings (n = 46)


Illusory Correlation Hindsight bias Fundamental Attribution bias Confirmatory bias

M (SD) 4.13 (1.50) 4.65 (1.58) 3.91 (1.77) 4.54 (1.74)

Other (Peer)Ratings (n = 34) M 5.47 5.76 5.88 6.29

(SD) (1.50) (1.60) (1. 41) (1.59)

Cohen’s d [95% CIs] .90 [.40, 1.34] .71 [.17, 1.16] 1.23 [.75, 1.74] 1.06 [.52, 1.56]

no main effect was found for presence of the article on bias ratings, F(4, 73) = 2.076, p = .093, Wilks’ Λ = .898, partial η2 = .102. The interaction between article presence (yes or no) and bias reference (self or other) was not statistically significant, F(4, 73) = 0.069, p = .991, Wilks’ Λ = .996, partial η2 = .004. Hypothesis 1: Evidence for a bias blind spot would be demonstrated by a discrepancy between participants’ self- and other-ratings for the various biases. Each of the four biases had a statistically significant main effect such that ratings for these biases in others were higher than the ratings for these biases in oneself: illusory correlation bias, F(1, 76) = 17.184, p < .001, partial η2 = .184, Mdiff = 1.363, p < .001, d = 0.90, 95% CI [.40, 1.34]; hindsight bias, F(1, 76) = 10.097, p = .002, partial η2 = .117, Mdiff = 1.133, p = .002, d = 0.71, 95% CI [.17, 1.16]; fundamental attribution error, F(1, 76) = 29.944, p < .001, partial η2 = .283, Mdiff = 1.990, p < .001, d = 1.23, 95% CI [.75, 1.74]; and confirmation bias, F(1, 76) = 22.175, p < .001, partial η2 = .226, Mdiff = 1.766, p < .001, d = 1.06, 95% CI [.52, 1.56]. Hypothesis 2: Those who received education about the fallibility of introspection will have higher bias awareness as reflected by higher bias ratings than those who did not receive the article. The absence of a main effect for the article precluded further analysis regarding this hypothesis. Hypothesis 3: Individuals who are educated about introspection will rate its use as less effective. An independent samples t-test was used to test this hypothesis and revealed a nonsignificant difference between participants who received the article and those who did not regarding the effectiveness of introspection as a debiasing technique, t(78) = .051, p = .960, d = .01, 95% CI [–.55, .47].

Bias Reduction Techniques

Participants were asked about the techniques used to reduce the impact of bias in their work. Table 3 includes the percentage of participants endorsing each of the debiasing strategies. As a whole, evaluators in this sample believed introspection to be effective in helping them remain aware of their biases, regardless of condition (M = 7.38, SD = 1.64). Additionally, the sample appeared highly motivated to reduce the impact of bias in evaluation, with a mean rating of 8.40 (SD = .866) on the 9-point Likert-type scale given

8

M. ZAPPALA ET AL.

Table 3. Debiasing Strategies. Strategy No strategies Continued education and training Introspection and self-reflection Use of structured tools Reliance on empirical data Motivation of professional pride

Percentage of participants (n = 80) 3.8 86.3 78.8 63.8 56.3 51.3

Note. “I do not use any debiasing strategies” was offered as a choice. However, two of the people who chose this answer also endorsed other choices.


in response to the question, “How motivated are you to reduce the impact of bias in your evaluations?” In addition, there were no differences in the average rating on this item for those who rated themselves versus others, t (78) = .677, p = .500, d = .16, 95% CI [–.05, .50]. Discussion Consistent with research on the bias blind spot in community and student samples, this sample of forensic mental health professionals demonstrated a propensity to rate themselves as less biased than their peers and colleagues (Frantz, 2006; Pronin & Kugler, 2007; Pronin et al., 2002; West, Meserve, & Stanovich, 2012). Although there is little empirical research demonstrating these types of biases in forensic evaluation, participants who rated their colleagues indicated that cognitive bias was a significant factor in forensic evaluation, but those who rated themselves reported that these biases did not have a significant influence on their own evaluations. Thus, bias is a likely problem for one’s colleagues but not for oneself: a pattern of responding indicative of a bias blind spot. Despite this lack of insight into the influence of bias on their own work, participants endorsed the use of introspection as a useful strategy for reducing bias. Neal and Brodsky (2016) suggested the importance of teaching forensic mental health professionals about the ineffectiveness of introspection as a debiasing technique. In this study, however, the attempt to educate about this issue was unsuccessful. Despite having given half of the sample an article discussing the fallibility of introspection as a debiasing technique, the two groups rated introspection as similarly useful, although we note that participants were asked about the effectiveness of introspection with regard to aiding in the awareness of bias rather than the reduction of bias. This finding might be an indication that the belief in introspection as a useful debiasing technique is pervasive, despite the literature identifying it as ineffective.


9


Limitations and Future Directions

This study’s findings are important; however, several limitations must be acknowledged. First, the methodology was hindered by the failure to demonstrate bias in this sample prior to screening for a bias blind spot. In the future, it will be important to demonstrate the existence of bias in the sample prior to testing for a blindness to bias in order to draw more valid conclusions. Perhaps identifying bias in action would serve as a useful educational tool that could increase bias awareness and more effectively reduce the bias blind spot. Further research demonstrating bias in forensic evaluation practice would also help to raise awareness, as there currently exists only a small (but growing) literature on this topic. It is possible that participants assigned bias ratings in a socially desirable manner instead of in a truthful and honest manner. The finding that evaluators rated themselves low on biases might reflect impression management, where participants wanted the researchers to view them as objective and professional. Participants who were asked to rate their peers may not have felt the same impulse to portray themselves in a positive light since the questions were not about them. Future research might include methodology that masks the intention of the study better in an attempt to reduce socially desirable responding. It was also not possible to know whether participants thoroughly and attentively read the article provided to them since these materials were not provided as a part of the training workshops but, rather, as an optional research component. Future studies should include a manipulation check in order to ensure that only participants who demonstrate an understanding of the article are included for analyses. Alternatively, it might be useful to include this article or a discussion of its findings as a mandatory part of a relevant training. Although the current study simply gave participants an article about introspection, future research might consider providing formal training about bias and the ways which bias might manifest in clinical work. We also note that the methodology of asking half of the participants to rate themselves while asking half to rate their peers has benefits, but also disadvantages. Administering surveys in this way, as opposed to asking participants to rate themselves and their peers, helps to mask the true intention of the study. However, this methodology fails to capture one individual’s opinions about his or her own bias in direct comparison to his or her perception of bias in others. Future research might consider inquiring about a person’s bias relative to his or her peers, though this might induce socially desirable responding to a larger extent. Also important is the finding of Boccaccini and colleges (2015) that perceptions of bias may vary depending upon the perceived subjectivity of an assessment tool. It would be


10

M. ZAPPALA ET AL.

interesting to further investigate the bias blind spot with regard to specific measures of varying subjectivity. Finally, this study was conducted within the context of continuing education and professional training on topics in forensic psychology. It is plausible that individuals who seek out professional development are characteristically different from those who do not (e.g., more motivated, more professional, less biased). Future research should expand on the current findings and test for the presence of differences in bias blindness in a more diverse sample. Importantly, introspection is just one of the reported debiasing strategies offered by forensic evaluators; others include maintaining professional pride, relying on empiricism, and employing structured assessment tools (Neal, 2013; Neal & Brodsky, 2016). Future research should continue to gather information about these techniques and their effectiveness. Conclusion

These findings have important implications, as forensic evaluators are expected to provide knowledgeable and objective opinions regarding relevant psycholegal matters. The ability to conduct reliable evaluations is crucial for these evaluators; scrutiny on the grounds of biased decision making would debase the credibility of any individual who testifies in court proceedings. With evidence of biased decision making within the field of forensic psychology circulating (Chevalier et al., 2015; Deitchman, Kennedy, & Beckham, 1991; Everson & Sandoval, 2011; LeBourgeois, Pinals, Williams, & Appelbaum, 2007; Miller et al., 2011; Murrie et al., 2013; Neal & Grisso, 2014; Otto, 1989), it is essential for evaluators to find methods that can reduce the potential for bias to influence their work. The ethical requirement of objectivity in forensic clinical decision making requires enhanced efforts by professionals to reduce these tendencies. Bias awareness is necessary for improving clinical decision-making in the field of forensic psychology, and further research on debiasing strategies is an important next step in these advancements. References American Psychological Association. (2013). Specialty guidelines for forensic psychology. American Psychologist, 68, 7–19. doi:10.1037/a0029889 Boccaccini, M. T., Chevalier, C. S., Murrie, D. C., & Varela, J. G. (2015). Psychopathy Checklist–Revised: Use and reporting practices in sexually violent predator evaluations. Sexual Abuse: A Journal of Research and Treatment. doi:10.1177/1079063215612443 Borum, R., Otto, R., & Golding, S. (1993). Improving clinical judgment and decision making in forensic evaluation. Journal of Psychiatry and Law, 21, 35–76.



11

Chevalier, C. S., Boccaccini, M. T., Murrie, D. C., & Varela, J. G. (2015). Static-99R reporting practices in sexually violent predator cases: Does norm selection reflect adversarial allegiance? Law and Human Behavior, 39, 209–218. doi:10.1037/lhb0000114 Deitchman, M. A., Kennedy, W. A., & Beckham, J. C. (1991). Self-selection factors in the participation of mental health professionals in competency for execution evaluations. Law and Human Behavior, 15, 287–303. doi:10.1007/BF01061714 Everson, M. D., & Sandoval, J. M. (2011). Forensic child sexual abuse evaluations: Assessing subjectivity and bias in professional judgements. Child Abuse and Neglect, 35, 287–298. doi:10.1016/j.chiabu.2011.01.001 Frantz, C. M. (2006). I AM being fair: The bias blind spot as a stumbling block to seeing both sides. Basic and Applied Social Psychology, 28, 157–267. doi:10.1207/s15324834basp2802_5 LeBourgeois, H. I., Pinals, D. A., Williams, V., & Appelbaum, P. S. (2007). Hindsight bias among psychiatrists. Journal of the American Academy of Psychiatry and the Law, 35, 67– 73. Mandel, G. (2005). Unaware of our unawareness. Science, 308, 1624–1625. Melton, G. B., Petrila, J., Poythress, N. G., & Slobogin, C. (2007). Psychological evaluations for the courts: A handbook for mental health professionals and lawyers (3rd ed.). New York, NY: Guilford Press. Miller, A. K., Rufino, K. A., Boccaccini, M. T., Jackson, R. L., & Murrie, D. C. (2011). On individual differences in person perception: Raters’ personality traits relate to their psychopathy checklist-revised scoring tendencies. Assessment, 18, 253–260. doi:10.1177/ 1073191111402460 Murrie, D. C., Boccaccini, M. T., Guarnera, L. A., & Rufino, K. A. (2013). Are forensic experts biased by the side that retained them? Psychological Science, 24, 1889–1897. Murrie, D. C., Boccaccini, M. T., Zapf, P. A., Warren, J. I., & Henderson, C. E. (2008). Clinician variation in findings of competence to stand trial. Psychology, Public Policy, and Law, 14, 177–193. doi:10.1037/a0013578 Neal, T. M. (2013). The objectivity demand: Experiences and behaviors of psychologists in capital case evaluations (Doctoral dissertation).University of Alabama, Tuscaloosa, AL, USA. Neal, T. S., & Brodsky, S. L. (2016). Forensic psychologists’ perceptions of bias and potential correction strategies in forensic mental health evaluations. Psychology, Public Policy, and Law, 22, 58–76. doi:10.1037/law0000077 Neal, T. S., & Grisso, T. (2014). The cognitive underpinnings of bias in forensic mental health evaluations. Psychology, Public Policy, and Law, 20, 200–211. doi:10.1037/a0035824 Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259. doi:10.1037/0033-295X.84.3.231 Otto, R. K. (1989). Bias and expert testimony of mental health professionals in adversarial proceedings: A preliminary investigation. Behavioral Sciences and the Law, 7, 267–273. doi:10.1002/bsl.2370070210 Pronin, E. (2007). Perception and misperception of bias in human judgment. Trends in Cognitive Sciences, 11, 37–43. doi:10.1016/j.tics.2006.11.001 Pronin, E., Gilovich, T., & Ross, L. (2004). Objectivity in the eye of the beholder: Divergent perceptions of bias in self versus others. Psychological Review, 111, 781–799. doi:10.1037/ 0033-295X.111.3.781 Pronin, E., & Kugler, M. B. (2007). Valuing thoughts, ignoring behavior: The introspection illusion as a source of the bias blind spot. Journal of Experimental Social Psychology, 43, 565–578. doi:10.1016/j.jesp.2006.05.011


12

M. ZAPPALA ET AL.

Pronin, E., Lin, D. Y., & Ross, L. (2002). The bias blind spot: Perceptions of bias in self versus others. Personality and Social Psychology Bulletin, 28, 369–381. doi:10.1177/ 0146167202286008 West, R F., Meserve, R. J., & Stanovich, K. E. (2012). Cognitive sophistication does not attenuate the bias blind spot. Journal of Personality and Social Psychology, 103, 506–519. doi:10.1037/a0028857 Wilson, T. D., & Brekke, N. (1994). Mental contamination and mental correction: Unwanted influences on judgments and evaluations. Psychological Bulletin, 116, 117–142. doi:10.1037/ 0033-2909.116.1.117 Zapf, P. A., & Dror, I. E. (2017). Understanding and mitigating bias in forensic evaluation: Lessons from forensic science. International Journal of Forensic Mental Health. doi:10.1080/14999013.2017.1317302 Zapf, P. A., Hubbard, K. L., Cooper, V. G., Wheeles, M. C., & Ronan, K. A. (2004). Have the courts abdicated their responsibility for determination of competency to stand trial to clinicians? Journal of Forensic Psychology Practice, 4, 27–44. doi:10.1300/J158v04n01_02 Zapf, P. A., Kukucka, J., Kassin, S. M., & Dror, I. E. (2017). Cognitive bias in forensic mental health assessment: Evaluator beliefs about its nature and scope. Psychology, Public Policy, and Law. Advance online publication. doi:10.1037/law0000153.