Relative Effectiveness of the Standard Validity Scales

0 downloads 0 Views 1MB Size Report
revised Dissimulation scale in the detection of fake-good and fake-bad MMPI-2 ... in detecting faking bad, and the O-S index and the Mp and L scales were ...
Psychological Assessment 1995, Vol. 7, No. 1,84-92

Copyright 1995 by the American Psychological Association, Inc. !040-3590/95/$3.00

Relative Effectiveness of the Standard Validity Scales in Detecting Fake-Bad and Fake-Good Responding: Replication and Extension R. Michael Bagby

Tom Buis York University

University of Toronto

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Robert A. Nicholson University of Tulsa The effectiveness of the Minnesota Multiphasic Personality Inventory—2 (MMPI-2; J. N. Butcher, W. G. Dahlstrom, J. R. Graham, A. Tellegen, & B. Kaemmer, 1989) Lie (L) and Infrequency (F) scales, relative to the Obvious-Subtle Index (O-S), the Positive Malingering (Mp) scale, and the revised Dissimulation scale in the detection of fake-good and fake-bad MMPI-2 protocols was evaluated by asking college students to respond honestly, fake bad, or fake good on the MMPI-2. MMPI2 protocols of participants asked to fake bad were compared with protocols from general psychiatric and forensic inpatient samples, and MMPI-2 protocols of participants asked to fake good were compared with MMPI-2 protocols of students asked to respond honestly. The F scale was superior in detecting faking bad, and the O-S index and the Mp and L scales were equally effective at detecting faking good. However, we caution against the use of the O-S index in the detection of fake-bad and fake-good responding.

Malingering (Mp) scale (Cofer, Chance, & Judson, 1949)2 was developed to specifically address defensive responding. Of these various scales and indexes designed for detecting response distortion, the S-O scales have been one of the most widely used for the detection of fake-bad and fake-good MMPI profiles (Greene, 1991). The validity of the S-O scales to detect invalid responding, however, has been challenged recently (Timbrook, Graham, Keiller, & Watts, 1993; Weed, Ben-Porath, & Butcher, 1990). Timbrook et al. (1993), for example, found that the O-S index discriminated between valid psychiatric and fake-bad profiles at a level comparable to that reported previously for the standard validity scales. However, when the O-S index and the F scale were regressed against profile status (i.e., bona fide patients vs. students instructed to fake bad), the O-S index did not account for as much variance as the F scale. Moreover, the F scale made a significant contribution to classification of patient and fake-bad MMPI-2 profiles after the variance for the O-S index had been partialed from the model, whereas no such significant increase was found for the O-S index when the variance for F was partialed from the model. The F scale also produced a larger effect size than either the sum of the Obvious scales or the O-S index. This latter finding for the MMPI-2 F scale corroborates the findings from a recent meta-

Response distortion on self-report measures of personality and psychopathology continues to be an important issue in psychological assessment. From its inception, the Minnesota Multiphasic Personality Inventory (MMPI; Hathaway & McKinley, 1940) included scales that were designed to assess response distortion. The Lie (L) and Infrequency (F) scales were implemented initially as validity indicators and were soon followed by the Correction (K) scale (Meehl & Hathaway, 1946). These standard validity scales have been retained on the MMPI-2 (Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989). As testimony to the importance of assessing response distortion, and in response to the belief of many investigators that the standard validity scales were perhaps not wholly adequate in the detection of distortion, a second generation of validity indicators emerged (Greene, 1988). These indicators included additional indexes of overreporting (malingering), such as the Wiener-Harmon Subtle-Obvious (S-O) Scales (Wiener, 1948), from which the Obvious-Subtle (O-S) Index is derived; Cough's original and revised Dissimulation (Ds) scales (Gough, 1954; Ds-r; Gough, 1957);' the sum of the Lachar and Wrobel Critical Items (CI; Lachar & Wrobel, 1979); and Cough's F-K Dissimulation Index (F-K; Gough, 1950). The OS and F-K indexes are bidirectional, which also allows the assessment of defensive or fake-good responding. The Positive

1 The Ds-r is a revised version of the 74-item Gough Dissimulation (Ds) scale (Gough, 1954), designed to detect individuals who chose to simulate or exaggerate psychopathology (Graham, 1987). The items on the original Ds scale were those items that significantly differentiated a group of neurotic patients from groups of college students and clinical psychologists asked to feign neurosis (Greene, 1991). The MMPI-2 retains 34 of 40 Ds-r items; we have designated the modified scale as Ds-r2. 2 Twenty-seven of the 33 original MMPI Mp items have been retained on the MMPI-2.

R. Michael Bagby, Mood Disorders Program, Clarke Institute of Psychiatry, University of Toronto, Toronto, Ontario, Canada; Tom Buis, Department of Psychology, York University, Toronto, Ontario, Canada; Robert A. Nicholson, Department of Psychology, University of Tulsa. Correspondence concerning this article should be addressed to R. Michael Bagby, Mood Disorders Program, Clarke Institute of Psychiatry, 250 College Street, University of Toronto, Toronto, Ontario, Canada M5T1R8. 84

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

SUBTLE-OBVIOUS SCALES

analytic review of the MMPI scales used to detect malingering (Berry, Baer, & Harris, 1991). A similar pattern of results emerged with the L scale in the identification of fake-good profiles. In discriminating between valid "normal" and fake-good profiles, the L scale produced substantial increases in the variance accounted for over and beyond that of the O-S index, and the O-S index added no significant predictive effects over and beyond that of the L scale. Although the results from the Timbrook et al. (1993) study suggest that the O-S index may not be as useful as the standard validity scales L and F, a number of statistical and methodological issues warrant further investigation. First, the total sample size in their study was small, especially the psychiatric sample, which consisted of only 49 patients. Second, participants completed the MMPI-2 twice, once under standard instructions and once under fake-bad or fake-good instructions. The MMPI-2 is a lengthy protocol containing 567 items. To complete the inventory once is time consuming and tedious, but to redo the entire test from a different perspective may become boring or even extremely taxing for the average person. An additional problem with using the same participants for both conditions concerns experimental bias. The contrasting instructional sets under which participants took the test may have led to an exaggeration of differences in the profiles. Finally, Timbrook et al. (1993) did not instruct their participants to be believable, nor did they provide an incentive for their participants to feign successfully. The lack of an incentive renders the external validity of the results weak, as clients who deliberately distort their responses typically have much to gain (Rogers, 1988). One purpose of this study was to corroborate the results of Timbrook et al. (1993), but with sampling and procedural modifications designed to redress limitations of their study. First, we obtained a considerably larger sample of general psychiatric patients than Timbrook et al. (1993), and we included a large sample of forensic patients undergoing pretrial evaluation as a second patient comparison group. Second, we used a between-groups design in which separate samples of "normal" students were asked to take the MMPI-2 under one of three conditions: standard instructions, instructions to fake bad, or instructions to fake good. Third, we offered an incentive for successful feigning. A second purpose of this study was to evaluate two additional scales developed to assess faking bad (the Ds-r) and faking good (the Mp). Rogers, Bagby, and Chakraborty (1993) found the MMPI-2 version of the Ds-r (designated hereafter as Ds-r2) to be superior to the F scale in the accurate classification of simulators and schizophrenic inpatients. In two recent studies, Bagby, Rogers, and Buis (1994) and Bagby, Rogers, Buis, and Kalemba (1994) found the Ds-r2 and the Mp to have comparable sensitivity and specificity rates in the classification of fakebad and bona fide patients, and fake-good and honestly responding normals. Similar results have been reported by Wetter, Baer, Berry, Smith, and Larsen (1992) for the MMPI-2 version of the Ds.

85

pital. The mean age was 35.4 years (SD = 11.54). Racial composition was 98 (76%) White, 6 (5%) Asian, 2(3%) Black, 2(2%) Native Canadian, and 21 (16%) unreported. Forty-three (33%) were diagnosed as having mood disorders, 19(15%) anxiety disorders, 19(15%) personality disorders, 11 (9%) adjustment disorders, 11(9%) schizophrenia, 4 (3%) delusional disorders, 1 (.7%) polysubstance use, and 21 (16%) diagnosis deferred. The forensic psychiatric inpatient sample was drawn from a university-affiliated, pretrial, forensic inpatient hospital and consisted of 159 patients (131 men, 28 women). The mean age was 33.8 years (SD = 10.81). Racial composition was 80 (50%) White, 1 8 ( 1 1 % ) Black, 6 (4%) Asian, 2 (1%) Native Canadian, 1 (.6%) Hispanic, and 52 (33%) unreported. Fifty (31%) were diagnosed as having schizophrenia, 39 (25%) personality disorders, 20 (13%) mood disorders, 16 (10%) alcohol/substance dependence or use disorders, 11 (7%) delusional disorders, 3 (2%) adjustment disorders, 1 (.6%) psychosexual disorders, 1 (.6%) amnestic disorder, and 18(11%) diagnosis deferred. All diagnoses were provisional, as they were obtained from initial intake examinations and were based on nonstandardized psychiatric and psychological interviews using criteria from the Diagnostic and Statistical Manual of Mental Disorders—Third Edition—Revised (DSMIH-R; American Psychiatric Association, 1987). Diagnoses were made independently of the MMPI-2 results. The college sample, which was used for the honestly responding "normals" and fake-bad and fake-good conditions, consisted of 344 college students (102 men, 242 women) recruited from undergraduate psychology courses. The mean age of this sample was 22.0 years (SD =5.51). Although the ethnic and racial information was not obtained, the sample was predominantly White.3

Procedure We used the method of instruction used in an earlier study (Bagby, Gillis, Toner, & Goldberg, 1991) and informed students that the investigators were conducting a study examining the ability of computerscored personality tests to detect individuals who were trying to fake bad or fake good. Different scenarios were given to clarify under what kind of circumstances such dissimulation might occur. For the fake-bad condition, participants were given the examples of trying to beat a legal charge on the grounds of insanity, applying for rehabilitation services, or qualifying for disability benefits. For the fake-good condition, examples were trying to gain custody of one's children in a divorce case, attempting to secure release from a mental hospital, or applying for a good job. Participants chose in which of two conditions they wished to participate: the fake-bad or the fake-good condition. Participants were allowed to choose, rather than be randomly assigned to, the condition in which they wished to participate on the assumption that they would be more motivated and accomplished at a task that they chose to perform. All participants were paid $ 10 for completing the MMPI-2. As additional incentive, participants were told that if they were not detected as faking they would qualify for a lottery in which they could win $100. Participants in the honest condition were administered the MMPI-2 as part of another study. These participants were paid $10 for completing the MMPI-2 and were provided written feedback about their test results. The MMPI-2 scales L, F, K, Variable Response Inconsistency (VRIN), True Response Inconsistency (TRIN) S-O scales were computer scored using a program provided by National Computer Systems, Inc. For all analyses, raw scores for the L and F scales were used, and the linear T scores were used for the S-O. The Ds-r2 and the Mp scales

Method Participants The general psychiatric inpatient sample consisted of 129 patients (64 men, 65 women) drawn from a university-affiliated, inpatient hos-

3 Bagby, Rogers, & Buis (1994) and Bagby, Rogers, Buis, & Kalemba (1994) have reported data on some of these students and patients in an effort to establish cutting scores and sensitivity and specificity rates for the validity scales and indexes.

86

R. BAGBY, T. BUIS, AND R. NICHOLSON

Table 1 Means, Standard Deviations, and Analyses of Variance ofMMPI-2 Validity Scales and Indexes for Standard Instructions, Fake-Bad Instructions, and Psychiatric Patients Condition Psychiatric patients Standard instructions

Fake-bad instructions

M

SD

M

8.8.0 240. 1K 241.9abc 53.4.0, 10.0.^

6.4

33.2b 338.8b 232.9.b I89.8b 19.7b

Score

Nonforensic

M

SD

Forensic

SD

M

SD

7.4

12.6cd 263.4^ 256.9,* 73.7^ 10.0^

62.3 33.2 101.3

14.6^ 270.7* 244.5abK) 93.4^ 12.0.*

55.3 27.1 82.4

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Men (n = 279)

/"scale Obvious scale Subtle scale O-S index Ds-r2 scale

44.2 24.3 69.1

5.7

11.3 48.6 20.5 69.1

11. 7,0, 281.8acd 255.1.0, 90.0.*,

66.9 31.7 94.8

5.9

11.3^

6.7

8.7

6.2

61.4 19.6

7.3 15.8 22.0

Women (n = 278) /"scale Obvious scale Subtle scale O-S index Ds-r^ scale

7.1. 240.9, 244. U 51. 3.4 20.8^

5.8 48.8 26.5 71.3

5.3

33.6b 341.9b 231.4M 198.8b 21.0,,

6.7

10.9 44.9 18.4 59.9

H.3cd 287.5,* 259.7,* 88.2,3,

52.8 32.0 83.3

4.5

12.1acd

6.3

7.2

6.5

155.8 49.7 10.3 45.4 39.1

Note. All /"values differ significantly at thep < .05 level. Means with different subscripts differ significantly from one another. Number of participants per condition were as follows: standard instructions: 58 men, 140 women; fake-bad instructions: 29 men, 45 women; psychiatric nonforensic: 63 men, 65 women; psychiatric forensic: 129 men, 28 women. For men, df '= 3, 275; for women, df= 3, 274. MMPI-2 = Minnesota Multiphasic Personality Inventory—2; F = Infrequency; O-S = Obvious-Subtle; Ds-r2 = MMPI-2 version of the Dissimulation Scale—Revised.

were calculated by extracting the items making up the MMP1 versions of these scales that have been retained on the MMPI-2. The same scoring algorithm for the MMPI versions of these scales was used for the MMPI-2 scales. Raw scores were used. The honestly responding students' MMPI-2 profiles were subjected to the following exclusion criteria: In raw scores, cannot say (CS) score > 30, K> 26, TRIN^L 18, VRIN^ 18. This resulted in the exclusion of 2 students. For the patient samples exclusion criteria were CS score > 30, TRIN^ 18, VRIN^. 18. These criteria resulted in the exclusion of 1 psychiatric inpatient and 2 forensic inpatients.4

Results Group Comparisons and Effect Sizes Table 1 presents the means and standard deviations for the F sum of Obvious scales, sum of Subtle scales, the O-S index, and the Ds-r2 scale for the forensic psychiatric inpatient, general psychiatric inpatient, honestly responding, and fake-bad responding participants. For both the male and female samples, group differences for all the scores were statistically significant. The pattern of scores across the groups for the men and women indicated that F, mean sum of Obvious, O-S index, and Ds-r2 were lowest in the honest condition and highest in the fake-bad condition, with the two psychiatric patient sample scores falling between these groups. The two patient groups did not differ significantly from one another on any of the variables for either the men or women. Table 2 presents the means and standard deviations for the L, sum of Obvious scales, sum of Subtle scales, the O-S index, and the Mp scale for the honestly responding normals and fake-good

responding participants. For both the male and female samples, there were significant differences for L, sum of Obvious scales, sum of Subtle scales, the O-S index, and the Mp scale. Participants in the fake-good condition scored higher as a group than did honestly responding normals on L, Mp, and sum of Subtle scales; they scored lower on the O-S index and the sum of Obvious scales. Overall, these results are consistent with those reported by Timbrook et al. (1993) and provide support for the Ds-r2 and Mp scales in the evaluation of fake-bad and fake-good MMPI-2 profiles. The magnitude of the differences between the fake-bad condition versus the general and forensic psychiatric inpatient and the standard instruction groups, and the fake-good condition versus the standard instruction group were determined by calculating effect sizes (Cohen's d; Cohen, 1977). These effect sizes are summarized in Table 3. For the comparison between 4

These exclusion criteria differ slightly from those used by Timbrook, Graham, Keiller, & Watts (1993). In addition to using CS score, K, TRIN, and VRIN scores, Timbrook et al. (1993) eliminated honest patient profiles with raw F scores > 21 and raw L scores > 10, and, in addition to TRIN sad VRIN scores, they also used raw /"scores > 25 as an exclusion criterion for the patient samples. We chose not to exclude protocols on variables that were ultimately used as dependent variables in subsequent analyses. However, for comparison, we also analyzed the data using the criteria used by Timbrook et al. (1993). These criteria resulted in the elimination of 11 honestly responding students, 9 psychiatric inpatients, and 19 forensic patients but did not change substantially the pattern of the results or the magnitude of the effect sizes reported in the results of this article.

87

SUBTLE-OBVIOUS SCALES Table 2 Means, Standard Deviations, andt Tests for Participants Receiving Standard and Fake-Good Instructions Instructions Standard MMPl-2 scale

M

Fake good SD

M

SD

6.9 183.9

24.7

;

Men (n = 72)

L

This document is copyrighted by the American Psychological Association or one of its allied publishers. This article is intended solely for the personal use of the individual user and is not to be disseminated broadly.

Obvious Subtle O-S Mp

3.2 240.1 241.9 53.5 9.1

2.3 44.2 24.3 69.1 3.6

266.0 -36.2 12.4

26.2 47.2 5.4

4.28*** 4.57*** 3.28** 4.59*** 2.73**

3.3 36.4 24.1 54.9 4.7

9.97*** 7.94*** 4.98*** 8.27*** 10.75***

4.7

Women («= 196) L

Obvious Subtle O-S Mp

2.8

240.9 244.1 51.3 7.7

1.9 48.8 26.5 71.3 2.9

6.6 183.6

264.5 -36.4 13.7

Note. Number of participants in each condition is as follows: standard condition: 58 men, 140 women; fake-good condition: 14 men, 56 women. MMPI-2 = Minnesota Multiphasic Personality Inventory—2; L = Lie; O-S = Obvious-Subtle; Mp = Positive Malingering. **;>