the adp-iv questionnaire - UKM

3 downloads 0 Views 170KB Size Report
with the Wisconsin Personality Disorders Inventory (WISPI: Klein et al.,. 1993) as indicated by a ...... Millon Clinical Multiaxial Inventory: A concurrent validity ...
Journal of Personality Disorders, 18(4), 405-419, 2004 © 2004 The Guilford Press SCHOTTE THE ADP-IV ETQUESTIONNAIRE AL.

THE ADP-IV QUESTIONNAIRE: DIFFERENTIAL VALIDITY AND CONCORDANCE WITH THE SEMI-STRUCTURED INTERVIEW Chris K. W. Schotte, PhD, Dirk A. M. De Doncker, MA, Dorota Dmitruk, MA, Ineke Van Mulders, MA, Hugo D’Haenen, MD, PhD, and Paul Cosyns, MD, PhD

The Assessment of DSM-IV Personality Disorders questionnaire (ADP-IV) is a self-report measure of the DSM-IV Axis II personality disorders (PDs). The ADP-IV assesses for each DSM-IV criterion its typicality as well as the accompanying distress and impairment. This study investigates two important aspects of the construct validity of the ADP-IV: (a) the differential validity (i.e., the ability to differentiate between [1] a sample of the general Flemish population (n = 659) and a sample of psychiatric inpatients (n = 487) with a high prevalence of clinical PD diagnoses, and [2] patients with and without a PD in the psychiatric sample; (b) the convergent validity with the SCID-II semi-structured interview in a population of psychiatric inpatients (n = 59). The results indicate a good differential validity: the dimensional scales and the categorical measures discriminated well between both groups and between patients with and without a PD diagnosis in the psychiatric sample. Concerning the concordance with the SCID-II, a decent level of agreement is exemplified by a correlation of .67 between the dimensional total scores of both instruments and by kappa coefficients for an “any” Axis II diagnosis at the .50 level. In conclusion, the results indicate that the ADP-IV is an efficient method for assessing PD in dimensional and categorical ways.

It is a prerequisite for diagnostic instruments evaluating DSM-IV (APA, 1994) personality pathology that they take into account the trait aspect as well as the impairment and distress, inherently associated with and caused by the personality disorder (PD) traits (Schotte, De Doncker, Maes, Cluydts, & Cosyns, 1993). The ADP-IV questionnaire (Assessment of DSM-IV Personality disorders: Schotte & De Doncker, 1996; De Doncker, Schotte, Vertommen, & Van Kerkchoven, 1997; Schotte, De Doncker, Van Kerkchoven, Vertommen, & Cosyns,1998; Schotte & De Doncker, 2000; From UZA, University Hospital Antwerp, Department of Psychiatry, and CAPRI, Collaborative Antwerp Psychiatric Research Institute, University of Antwerp (C.K.W.S., D.A.M.D., D.D., P.C.); VUB, Free University Brussels, Faculty of Psychology (C.K.W.S., I.V., H.D.); AZ VUB, University Hospital Free University Brussels, Department of Psychiatry (I.V., H.D.). Address correspondence to Prof. dr. C. Schotte, Department of Psychiatry, University Hospital Antwerp (U.Z.A.), Wilrijkstraat 10, B-2650 Edegem, Belgium; E-mail: [email protected].

405

406

SCHOTTE ET AL.

Schotte, 2000; Schotte, De Donker, Dmitruk, De Valck, & Van Mulders, 2002) was developed by our research group specifically to assess these two central aspects of the DSM-IV PD definition: the typicality of each DSM-IV criterion is first assessed by means of a 7-point Likert scale and results in a Trait-score. When the subject judges the trait to be present, then the dysfunctionality—reflected by the suffering and problems caused by the presence of the trait—is assessed by means of a 3-point Distress-scale. This test design allows dimensional assessment formats, using only the Trait-scores, as well as traditional categorical measurements of the criteria and diagnoses, which are obtained using scoring algorithms (i.e., combinations of Trait and Distress cut-off scores). Previous research with the ADP-IV (Schotte et al., 1998; Schotte & De Doncker, 2000; Schotte et al., 2002) revealed that the dimensional Trait-scales are internally consistent (median Cronbach’s alpha: .76; range: .60-.84) and display good concurrent validity with the Wisconsin Personality Disorders Inventory (WISPI: Klein et al., 1993) as indicated by a median correlation of .61 between the corresponding scales. The Trait-assessment displayed factorial validity both at the item and at the subscale level (Schotte et al., 1998): a principal component analysis of the ADP-IV questionnaire at the item level revealed 11 psychologically relevant dimensions that only partially conformed to the organization of the items in categories and clusters in the DSM-IV. Adequate levels of short-term test-retest reliability and stability (6-month interval) were obtained: temporal stability is exemplified by a median correlation of .82 for the dimensional Axis II scales and by a kappa coefficient of .67 for an “any PD” diagnosis (Schotte, 2000). The purpose of the present study is to evaluate two further aspects of the construct validity of the ADP-IV: differential validity and concordance with the semi-structured interview method. First, the differential validity of a test refers to its power to discriminate between diagnostic groups with contrasting prevalences of the criterion disorder it is supposed to measure. This quality is investigated here in terms of the extent to which the ADP-IV differentiates (1) between a stratified sample of the general Flemish population and a population of Flemish psychiatric inpatients, and (2) between patients with and without a standardized clinical PD diagnosis in the psychiatric sample (Study 1). Second, concerning the convergent validity of instruments assessing PDs, it has been shown that the correspondence between questionnaire and interview methods is generally low with median kappa values at the .20 level and rarely higher than .40 for the “any PD” diagnosis (e.g. Perry, 1992; Clark, Livesley & Morey, 1997; Bronisch & Mombour, 1998; Schotte, 2000; Smith, Klein, & Benjamin, 2003). Taking this issue into account, Study 2 evaluates the concordance between the ADP-IV and the SCID-II semi-structured interview (Structured Clinical Interview for DSM-IV Axis II Personality Disorders: First, Gibbon, Spitzer, Williams, & Benjamin, 1997) in a psychiatric inpatient population. A reasonable level of convergent validity needs to be obtained and should be reflected in kappa coefficients above the .40 level for the categorical data, and in correlations around the .50 level for the dimensional measures.

THE ADP-IV QUESTIONNAIRE

407

METHOD: THE ADP-IV The ADP-IV, a 94-item questionnaire, assesses the criteria of the 12 PD categories, described in the DSM-IV. As illustrated in Figure 1, for each criterion the ADP-IV first investigates the self-judged typicality of the criterion by means of a 7-point Trait scale. If the Trait description is judged to be applicable, the distress, maladaptivity, and suffering of the subject or his/her environment as a consequence of the presence of the Trait criterion is assessed using a 3-point Distress scale. One could argue that this scoring procedure induces a response bias in the course of which respondents shun the Distress-scale question by avoiding Trait scores greater than 4. This is not the case: on average three to four items on each ADP-IV questionnaire are scored as being typical but without any accompanying distress (Distress score = 1). The original version of the questionnaire is in Dutch; English, German, Japanese, and French versions are available. The ADP-IV allows categorical and dimensional diagnostic formats. A dimensional assessment of the DSM-IV PDs is obtained by summing the ADP-IV Trait scores for the 12 PDs, for the 3 clusters, and for a total score. Norms obtained in a stratified sample of the Flemish general population (n = 659) serve as a guide for the interpretation of the dimensional scores (Schotte et al., 1998). Categorical assessments of the DSM-IV criteria are obtained by combining the Trait and Distress scores using scoring algorithms, based on combinations of cut-off scores for the ADP-IV Trait and Distress items. For instance, with the algorithm labeled as T>4&D>1, an item scores positive/pathological and is considered to indicate a DSM-IV criterion only when a Trait score of 5 (“rather agree”), 6 (“agree”), or 7 (“totally agree”), and a Distress score of 2 (“somewhat”) or 3 (“most certainly”) are obtained simultaneously. By using other algorithms it is possible to stress the Trait element or emphasize the Distress concept in the diagnostic evaluation. Summation of the number of criteria for each PD category, for the three clusters, and for the total number of criteria results in “pseudodimensional” scores, which give an impression of the extent to which an individual corresponds to the prototype of a PD category. Finally, categorical PD diagnoses are obtained according to the DSM-IV thresholds. An Internet application provides scoring, dimensional and categorical interpretations, and a narrative description of the typical and distressing traits.

METHOD: STUDY 1: DIFFERENTIAL VALIDITY The Study 1 population included (1) the “Flemish sample,” a stratified sample of 659 subjects of the Flemish population, described in detail previously (Schotte et al., 1998) and (2) the “Psychiatric sample,” a sample of 487 inpatients admitted to Flemish psychiatric facilities in general hospitals and psychiatric hospitals. In the Psychiatric sample, a clinical DSM-IV Axis II diagnosis was obtained for 348 of the patients, using a standardized procedure. Blind to the ADP-IV scores, the diagnosing clinicians—qualified psychologists or psychiatrists—evaluated on a computer programme the presence of each Axis II disorder by rating its core diagnostic characteristics on a 4-point scale: “none or very few traits,” “some traits,” “almost meets the

408

SCHOTTE ET AL.

Response format of the ADP-IV items:

TRAIT-QUESTION

DISTRESS-QUESTION

How much do you agree with this statement about yourself? 1= 2= 3= 4=

totally disagree disagree tend to disagree agree nor disagree

Has this trait ever caused you or others any suffering or problems? 1= not at all 2= to a degree 3= definitely

5= tend to agree 6= agree 7= fully agree

5. I really can't bear the thought that someone would leave or abandon me, I therefore would do anything to prevent this happening.

5. 1 2 3 4

5 6 7 1 2 3

FIGURE 1. Design of the ADP-IV.

criteria for the diagnosis,” and “meets the criteria for the diagnosis.” According to the fourth threshold of the rating scale, the prevalence of clinical personality disorder diagnoses in this psychiatric inpatient population was 58.9%; combining the “almost” and “certain” clinical diagnoses resulted in a prevalence of 89.7%.

METHOD: STUDY 2: CONVERGENT VALIDITY WITH THE SCID-II INTERVIEW The SCID-II (First et al., 1997) is one of the most commonly used semi-structured interviews for the assessment of the 12 DSM-IV Axis II disorders. Most studies on the reliability and validity of the SCID-II relate to the DSM-III-R version (e.g. Arntz et al., 1992; Renneberg, Chambless, Dowdall, Fauerbach, & Graceley, 1992; First et al., 1995; Dreessen & Arntz, 1998) and mention reasonable levels of interrater and test-retest reliability. Maffei et al. (1997) report adequate inter-rater and internal consistency reliabilities for the DSM-IV version. The Study 2 sample consisted of 59 inpatients, admitted to the psychiatric wards of the University Hospitals in Antwerp (U.Z.A.) and Brussels (AZ V.U.B.). Subjects with mental retardation or deterioration or an acute psychotic or agitated state were excluded. To minimize the effects of the turmoil of the admission and the influence of acute Axis I state disorders, Axis II assessments were performed in the second or third week of the admission. After completion of the ADP-IV by the patient, the full Dutch SCID-II interview (Weertman, Arntz, & Kerkhofs, 1997) was conducted using two evaluators, alternating between interviewer and observer roles. The interviewers—fi-

THE ADP-IV QUESTIONNAIRE

409

nal-year clinical psychology master’s students—were extensively trained and supervised in administering the SCID-II interview. Initial SCID-II scores were based on a consensus between interviewer and observer. Furthermore, to approach the LEAD standard (Longitudinal Expert using All Data; Spitzer, 1983) for the SCID-II results, the final results of each interview were obtained after a thorough discussion with the supervising psychologist, who had several clinical contacts with the patients. All researchers were blind to the ADP-IV results. Dimensional SCID-II scores were obtained by summing the raw scores (scores 1-3) of the criteria for the Axis II categories, clusters, and total number of criteria. In a similar way pseudodimensional scores were obtained by taking the positively scored items (score 3) into account. Pearson correlation coefficients were used to investigate the concordance between the (pseudo)dimensional scores; the agreement at the categorical level was evaluated with kappa coefficients (Cohen, 1960).

RESULTS: STUDY 1: DIFFERENTIAL VALIDITY DEMOGRAPHIC CHARACTERISTICS The mean age of the Flemish general population sample was 37.3 (SD: 12.8 years; range: 18-67), which is significantly higher (F(1,1125) = 13.75; p < 10-3) than the Psychiatric inpatient sample, which obtained a mean age of 34.7 (SD: 10.24 years; range: 18-68). The following demographic characteristics describe the Flemish sample: 50.8% female, 66.3% married or living together, 28.4% single, 4.6% divorced and 0.6% widowed. In the Psychiatric sample 49.8% were female, 39.6% were married or living together, 41.7% were single, 18.1% divorced, and 0.6% widowed. The educational level of the Flemish sample was relatively high: 40.5% of the subjects received education after high school (Psychiatric sample: 29.9%). In the Flemish sample 5.4% (n = 35) of the subjects had ever received a psychiatric or psychological treatment; 8 of these subjects had been admitted to an inpatient psychiatric facility. DIFFERENTIAL VALIDITY: COMPARING THE DIMENSIONAL, PSEUDODIMENSIONAL AND CATEGORICAL ADP-IV SCORES BETWEEN THE FLEMISH AND PSYCHIATRIC SAMPLES The MANOVA with the 12 ADP-IV dimensional scales representing the DSM-IV Axis II categories as the dependent variables and the Flemish and Psychiatric population subgrouping as the independent variable results in a significant differentiation (F(12, 998) = 54.07; p < 10-3). Table 1 presents the results of the individual ANOVAs and reveals highly significant effects for all dimensional ADP-IV scales, with the Psychiatric sample obtaining significantly higher scores. The MANOVAs with the 12 ADP-IV pseudodimensional Axis II scales, using the T>4&D>1 and T>5&D>1 scoring algorithms, also resulted in significant differentiations (e.g., T>4&D>1: F(12, 1133) = 59.68, p < 10-3). All individual ANOVAs showed significantly more criteria in the Psychiatric sample for the T>4&D>1 and T>5&D>1 scoring algorithms.

410

SCHOTTE ET AL.

TABLE 1. Study 1: Results of the analyses of variance with the ADP-IV dimensional Trait-scales as the dependent variables and the diagnostic subgroups as the independent variable

DSM-IV Categories PAR Mean SD SZ Mean SD ST Mean SD AS Mean SD BDL Mean SD HIS Mean SD NAR Mean SD AV Mean SD DEP Mean SD O-C Mean SD P-A Mean SD DE Mean SD CLA Mean SD CLB Mean SD CLC Mean SD TOT Mean SD

Flemish sample (n = 659) 16.92 6.44 15.96 5.88 19.41 7.79 12.43 4.92 23.81 8.71 18.29 7.05 18.82 6.88 18.04 0.83 18.00 7.14 23.63 7.51 14.66 5.49 16.76 7.60 52.13 16.70 72.72 23.15 59.43 19.18 182.16 52.09

Psychiatric sample (n = 487) 22.91 8.53 20.92 7.66 29.05 10.74 16.92 7.54 39.45 12.45 25.17 8.79 23.08 8.81 26.44 10.20 27.11 9.78 29.54 9.09 20.08 7.57 27.01 9.72 72.62 22.02 104.24 29.93 83.02 23.96 258.32 66.41

ANOVA F(1,1144) (1, 1129) = 181.27

p < 10-5

(1, 1131) = 151.97

< 10-5

(1, 1131) = 306.58

< 10-5

(1, 1141) = 147.67

< 10-5

(1, 1124) = 614.58

< 10-5

(1, 1116) = 210.98

< 10-5

(1, 1129) = 83.40

< 10-5

(1, 1128) = 241.78

< 10-5

(1, 1126) = 327.92

< 10-5

(1, 1134) = 143.94

< 10-5

(1, 1129) = 194.56

< 10-5

(1, 1139) = 399.51

< 10-5

(1, 1105) = 383.43

< 10-5

(1, 1102) = 330.14

< 10-5

(1, 1021) = 422.62

< 10-5

(1, 1129) = 415.14

< 10-5

Note. PAR: Paranoid; SZ: Schizoid; ST: Schizotypal; AS: Antisocial; BDL: Borderline; NAR: Narcissistic; AV: Avoidant; DEP: Dependent; O-C: Obsessive-Compulsive; DE: Depressive; P-A: Passive-Aggressive; CLA: Cluster A scale: Summation of PAR, SZ and ST; CLB: Cluster B scale: Summation of AS, BDL, HIS, and NAR; CLC: Cluster C scale: Summation of AV, DEP, and O-C; TOT: Total score: Summation of CLA, CLB, and CLC.

For all algorithms, the number of “any PD” diagnoses and diagnoses in Cluster A, B, or C in the Psychiatric sample was significantly higher, with χ2 tests significant below the 10-5 level. For example, the frequency of an Axis II diagnosis, according to the ADP-IV, for the Flemish population was 20.6% (T>4&D>1) and 5.9% (T>5&D>1), and for the Psychiatric sample 70.2% (T>4&D>1) and 43.3% (T>5&D>1). The number of subjects receiving more than one Axis II diagnosis, according to the ADP-IV, for the Flemish population was 8.5% (T>4&D>1) and 1.8% (T>5&D>1) in the Flemish sample, and 51.3% (T>4&D>1) and 25.3% (T>5&D>1) in the Psychiatric sample.

THE ADP-IV QUESTIONNAIRE

411

TABLE 2. Study 1: Results of the Analyses of Variance with the ADP–IV Dimensional Trait–Scales as the Dependent Variables and the Diagnostic Subgroups as the Independent Variable in the Psychiatric Sample ADP–IV Dimensional Trait–Scores with Corresponding Clinical Diagnosis Absent

Present

ANOVA

Scale

Mean

SD

n

Mean

SD

n

F(1,346)

PAR

21.77

8.3

290

7.28

8.41

58

21.15

10–5

p

SZ

19.29

7.02

273

24.59

7.88

75

31.68

10–6

ST

27.38

10.35

286

37.84

11

62

50.84

10–6

AS

16.47

7.19

328

23.8

9.6

20

18.8

10–4

BDL

36.23

11.33

226

44.16

12.6

122

35.84

10–6

HIS

24.61

8.44

262

27.33

10.34

86

5.98

0.15

NAR

21.87

8.62

274

26.82

9.43

74

18.37

10–4

AV

23.64

9.83

176

28.4

9.77

172

20.45

10–4

DEP

25.18

9.43

215

28.06

10.09

133

7.26

.008

0C

27.33

9.07

250

32.59

7.94

98

25.38

10–5

DE

24.11

8.99

211

29.55

9.31

137

29.58

10–6

PA

9.17

7.36

300

23.33

8.41

48

12.68

10–3

Note. Abbreviations, see Table 1.

DIFFERENTIAL VALIDITY: EFFECTS OF THE CLINICAL DIAGNOSIS IN THE PSYCHIATRIC SAMPLE Table 2 presents the effects of clinical diagnosis of specific Axis II categories on the corresponding dimensional ADP-IV scores. As a correction for experiment-wise error, the acceptable level of α was set at p < 0.004 (Bonferroni correction .05/12). Significant differences were found on 10 of the 12 dimensional scales; only the histrionic and dependent ADP-IV scales failed to differentiate significantly for the corresponding clinical diagnosis. Similar patterns were obtained for the pseudodimensional scores: for T>4&D>1 9 and for T>5&D>1 8 of the 12 correspondent pseudodimensional scales were significantly (p < 0.004) influenced by the clinical diagnosis.

RESULTS: STUDY 2: CONVERGENT VALIDITY DEMOGRAPHIC CHARACTERISTICS AND PREVALENCE OF DSM-IV DIAGNOSES The research sample consisted of 59 psychiatric inpatients with a mean age of 38.9 (SD = 11.3; range: 18-65 years); 62.7% (n = 37) were female; 40.7% were married or living together, 30.5% were single, and 28.8% divorced. Considering educational level, 1.7% attended only primary school, 74.6% went to high school, and 23.7% received education after high school. The clinical DSM-IV Axis I diagnosis revealed the following prevalences: affective disorders (71.2%), anxiety disorders (20.3%), substance dependence

412

SCHOTTE ET AL.

TABLE 3. Study 2: Prevalences of the DSM–IV Personality Disorders in Percentages, According to the SCID–II Interview and to the Scoring Algorithms of the ADP–IV. Concordance Table: Correlations between the Pseudodimensional (i.e. number of criteria) ADP–IV and SCID–II Scores (n = 59), and Kappa Coefficients between SCID–II and ADP–IV Categorical Diagnoses for the Borderline, Avoidant, Obsessive–Compulsive Axis II Diagnoses, for a Diagnosis in Cluster B, Cluster C, and for any Personality Disorder Diagnosis (n = 43) Concordance between SCID–II and ADP–IV According to the ADP–IV Scoring Algorithms Prevalence of PD in % According to ADP–IV scoring Algorithms SCID–II

T 4&D 1

T 5&D 1

PAR

0.00

8.47

1.69

SZ

1.69

0.00

ST

0.00

AS BDL

Pseudodimensional Scores

Categorical Diagnoses

Pearson’s correlations

Kappa coefficients*

T 4&D 1

T 5&D 1

T 4&D 1

T 5&D 1

PAR

0.47

0.34





0.00

SZ

0.22

0.07





0.00

0.00

ST

0.12

0.14





3.39

0.00

0.00

AS

0.41

0.29





11.86

30.51

15.25

BDL

0.57

0.65

0.54

0.66

HIS

1.69

15.25

3.39

HIS

0.45

0.45





NAR

1.69

1.69

1.69

NAR

0.14

0.19





13.56

20.34

11.86

AV

0.78

0.77

0.67

0.76

AV DEP

1.69

8.47

3.39

DEP

0.45

0.50





O–C

16.95

28.81

13.56

O–C

0.50

0.33

0.35

0.20

DE

3.39

11.86

1.69

DE

0.58

0.49





P–A

3.39

1.69

0.00

P–A

0.17

0.22





CLA

1.69

11.86

1.69

CLA

0.25

0.30





CLB

15.25

32.20

16.95

CLB

0.57

0.60

0.64

0.49

CLC

27.12

40.68

23.73

CLC

0.68

0.65

0.47

0.53

ANY PD

38.98

57.63

33.9





0.53

0.54

TOT

0.62

0.62

Median

0.46

0.40





Median*

0.57

0.65

0.53

0.54

ANY PD

Note. Abbreviations see Table 1; *Calculated for the diagnoses with a prevalence > 5: BDL, AV, O–C, CLB, CLC, ANY PD. Pearson correlations > 0.26 are significant at the p < 0.05 level; kappa values > 0.22 are significant at the p < 0.05 level.

(15.3%), adjustment disorders (13.6%), substance-related disorders (8.5%), and psychotic disorders (3.4%). Table 3 shows the prevalences of the DSM-IV PDs, diagnosed with the two instruments. A remarkable finding was the high prevalence of a NOS diagnosis according to the SCID-II: 27% of the patients met the general DSM-IV criteria for a PD diagnosis, but did not fulfill the criteria of a specific Axis II category. Compared with the SCID-II data, the ADP-IV T>4&D>1 scoring algorithm was too sensitive, whereas the best correspondence was obtained for the T>5&D>1 algorithm: respectively 38.9% (SCID-II) and 33.9% (ADP-IV) of the population received at least one PD diagnosis. The SCID-II resulted in a mean number of 0.59 (SD = 0.87) PD diagnoses per patient; the

413

0.33

0.33

0.27

0.05

0.24

0.49

0.24

0.37

0.43

0.12

0.18

0.10

0.10

0.28

0.22

0.17

0.40

0.12

0.35

0.23

0.39

ST

DEP

OC

DE

PA

CLA

CLB

CLC

0.46 0.45

0.15 0.36 0.12

0.34 0.39 0.34

0.14 0.33 0.32

0.18 0.14 0.43

0.05

0.42

–0.02

0.60

0.18

0.32

0.41

0.68

0.31

0.64

0.37

0.40

0.24

0.54

0.18

0.59

0.25

0.31

0.11

0.41

0.30

0.28

0.24

0.21

0.28

0.20

0.24

0.43

0.51

0.04

0.41

0.09

0.46

0.12

0.51

0.50

0.52

0.21

0.28

0.08

0.44

0.11

0.44

0.59

0.07

0.24

0.24

0.40

0.52

0.63

0.49

0.51

0.20

0.27

0.24

0.55

0.27

0.45

0.44

0.35

0.56

0.31

0.36

0.54

0.18

0.44

0.65

0.27

0.65

0.08

0.26

0.23

0.12

0.52

0.62

0.50

0.28

0.36

0.63

0.67

0.45

0.33

0.44

0.19

0.31

0.44

0.32

0.26

0.45

0.10

0.27

0.28

0.22 –0.01

0.24 0.32

0.04

0.20

–0.11

0.21

0.30

0.35 0.08

0.37 0.24

0.43

0.57

0.24

0.19

0.2

0.04 –0.02

0.17

0.16

0.38

0.11

0.36

0.48

0.35

0.38

0.06

0.29

0.5

0.67

0.13

0.27

–0.10

0.08

0.10

0.49 0.31

–0.02

0.11

–0.06

0.04

0.12

0.37 0.17

0.31

–0.11

0.23

0.47

0.27 0.24

0.53

–0.12

0.26

0.10

0.43 –0.05

0.65 0.36

0.09

0.16

0.24

0.48

0.38 0.11

0.27

0.46

TOT

–0.05

0.30

0.30

0.13

0.14

0.41

0.26

0.32

0.52

0.51

0.24

0.44

–0.10

0.19

0.31

0.17

0.25

0.23

0.15

0.22

0.20

0.20

0.32

0.34

AV

ADP–IV Scales

0.32

0.10

0.32

NAR

0.12

0.02

0.35

HIS

0.17

0.43

BDL

0.13

0.15

AS

Note. Abbreviations see Table 1. Pearson correlations > .26 are significant at p < 0.05. Correlations between corresponding scales are underlined.

0.29

0.57

CLC

TOT

0.51

0.45

0.44

CLA

CLB

0.28

0.40

0.40

DE

PA

0.03 0.24

0.29

0.05

DEP

0.26

0.12

–0.08

0.12

–0.08

OC

0.25

0.27

NAR

AV

0.40

0.28

BDL

HIS

0.24

0.21

ST

AS

0.49

0.13

SZ

0.35

0.58

PAR

SZ

PAR

SCID-II Scales

TABLE 4. Study 2: Concordance Table ADP-IV Scales: Correlations for the Dimensional SCID–II and ADP–IV Scales

414

SCHOTTE ET AL.

ADP-IV obtained a mean of 1.31 (SD = 1.70) for T>4&D>1, and 0.53 (SD = 0.95) for T>5&D>1. CONVERGENT VALIDITY BETWEEN THE SCID-II AND ADP-IV DIMENSIONAL SCALES Table 4 presents the correlation matrix between the dimensional scales of the two instruments. The Pearson correlation coefficients for the corresponding dimensional scales on the diagonal varied between .35 and .67, with a median value of .54. For the noncorresponding scales a median value of .23 was obtained, whereas for 9 of the 12 PD scales, for the 3 cluster scales, and for the total scores, the highest correlation was observed on the diagonal. Three scales, schizotypal, antisocial, and passive-aggressive, obtained correlations below .40 and did not display their highest correlation with the corresponding scale. Table 3 presents the concordance between the two instruments for the pseudodimensional scores and for the categorical diagnoses according to the T>4&D>1 and T>5&D>1 scoring algorithms of the ADP-IV. The highest correlations are at or above the .60 level and are observed for the avoidant and borderline PD, and for the Cluster B, Cluster C, and Total pseudodimensional scores. It is not coincidental that these scales represent the diagnostic categories with the highest prevalences for the SCID-II as well as for the ADP-IV. Unsatisfactory correlations are observed for the scales measuring the least frequently diagnosed PDs. A comparison of the scoring algorithms over the 16 scales indicates the highest median correlation of .46 for the T>4&D>1 algorithm. However, if one only takes the diagnostic categories with a prevalence of at least 5 cases according to the SCID-II into account, the T>5&D>1 algorithm obtains the most favorable median correlation of .65. Finally, Table 3 presents the level of agreement between the two instruments for the categorical diagnoses with a SCID-II prevalence of at least five cases and excluding the subjects with a NOS diagnosis. A median kappa value of .54 for the T>5&D>1 scoring algorithm reflects the highest level of concordance; the T>4&D>1 algorithm results in a median kappa above .50. Across all scoring algorithms a consistently low kappa value is observed for the obsessive-compulsive PD. The evaluation of the parameters of diagnostic efficiency using the SCID-II “any PD” diagnosis as reference diagnosis reveals that the T>4&D>1 algorithm results in a sensitivity of .83, a specificity of .70, a positive predictive value of .76, and a negative predictive value of .78. For the T>5&D>1 scoring algorithm values of .65, .90, .88, and .69, respectively, were obtained.

DISCUSSION The present study examined aspects of the construct validity of the ADP-IV by investigating its differential and convergent validity. Concerning differential validity, Study 1 first evaluated, using a straightforward research paradigm, the requisite that the ADP-IV should differentiate significantly between populations with low and high prevalences of PD diagnoses in the

THE ADP-IV QUESTIONNAIRE

415

Flemish sample and the Psychiatric sample, respectively. In samples of the general population, such as the Flemish sample, epidemiologists consistently estimate PDs to occur in the range of 10 to 15% of the population (Maier, Lichtermann, Klingler, Heun, & Hallmayer, 1992; Weissman, 1993; Oldham, 1994; Reich & de Girolamo, 1997). Studies in psychiatric populations assessing the DSM Axis II criteria with a semi-structured interview generally obtain prevalence rates around 50% (e.g. Reich & Troughton, 1988; Zimmerman, Pfohl, Coryell, Stangl, & Coranthal, 1988; Loranger et al., 1994). In the present study almost 60% of the psychiatric inpatient subjects were judged to meet the criteria for a clinical Axis II diagnosis. The analyses of all levels of dimensional and categorical measurement of the ADP-IV provided support in favor of its power to differentiate between the two groups: the ANOVAs revealed significant and substantially higher scores and higher frequencies of Axis II diagnoses in the Psychiatric sample. The value of the ADP-IV Distress scale was supported by the finding that in the Psychiatric sample the mean total ADP-IV trait score was approximately one and a half times that found for the stratified Flemish sample, whereas the total pseudodimensional score was, depending on the scoring algorithm, approximately three to five times that of the Flemish sample. Moreover, an interesting finding is the fact that—in contrast with other self-report methods—the ADP-IV does not result in the typical extremely high comorbidity rates among the PD categories (Loranger, 1992; Perry, 1992). Indeed, most of the subjects in the Flemish sample with an Axis II diagnosis, according to the ADP-IV, met the criteria for only one diagnosis, whereas a relatively low number of subjects generated more than one diagnosis (Schotte et al., 1998). In psychiatric samples one obviously expects higher rates of overlap: on average two-thirds of the patients obtain more than one Axis II diagnosis (Widiger et al., 1991). This was confirmed in our study: comorbidity rates were higher, although not exaggerated in the Psychiatric sample, ranging between approximately half (T>4&D>1) and 25% of the patients (T>5&D>1) obtaining more than one diagnosis. Second, Study 1 evaluated the effect of the clinical PD diagnosis on the corresponding dimensional and pseudodimensional scales. The clinical diagnosis comprised a 4-point scale for the assessment of the core characteristics of each DSM-IV PD. Scoring rules, clear definitions, anchoring points, and standardized assessment with the computer program were implemented in order to overcome the problematic reliability and validity associated with an unstandardized clinical diagnosis (Schotte, 2000). The ADP-IV scores reflected the effects of the clinical diagnosis well: in general, patients with a given PD diagnosis obtained significantly higher scores on the corresponding ADP-IV scales. In conclusion the findings of Study 1 are arguments that confirm the basic requisites for the differential validity of the ADP-IV. Study 2 examined the convergent validity of the ADP-IV by investigating the relationships with the SCID-II semi-structured interview in a population of psychiatric inpatients. A remarkable observation is that, using the SCID-II interview, a high proportion of patients (27%) obtained a NOS diagnosis on Axis II. Similar findings are associated with the criticism that the construct validity of the DSM-IV Axis II classification is too restricted: for ex-

416

SCHOTTE ET AL.

ample, Westen & Shedler (2000) state that approximately 60% of the patients receiving treatment in clinical practice for personality pathology cannot be diagnosed on Axis II. At the dimensional level, we computed the correlation matrix between the SCID-II and the ADP-IV Trait-scales. In such matrices convergent validity is reflected by the correlations on the diagonal between the corresponding scales, whereas the correlations between the nonrelated scales indicate the degree of divergent validity. Construct validity presupposes higher correlations on the diagonal compared to the coefficients outside the diagonal. Previous research comparing questionnaires with the SCID-II revealed a corresponding median value of .33 (range: .19 - .42) for the PDQ-4+ (Personality Diagnostic Questionnaire-4+: Fossati et al., 1998), whereas the WISPI-IV (Wisconsin Personality Disorders Inventory-IV; Smith et al., 2003) obtained a median correlation of .44 (range: .32 - .60) on the diagonal and of .17 as median off-diagonal value (range: -.24 - .75). The ADP-IV obtained a median of .52 (range: .35 - .67) for the convergent correlations of the 12 PD scales and a value of .23 (range: -.12 - .51) for the divergent correlations. Moreover, 9 of the 12 PD scales obtained their highest correlations on the diagonal of the correlation matrix. The high correlation—approaching the .70 level—between the total dimensional scores of both instruments is remarkable. In general, the correlation matrix offers arguments in favor of the convergent and divergent validity of the ADP-IV dimensional scales. Finally, at the diagnostic category level, the concordance for the diagnoses with a prevalence of at least five cases revealed kappa coefficients around the .40 to .50 level, reflecting a “reasonable” level of agreement. Consistent with reports in the literature (e.g. Clark et al., 1997), our results indicate the best coefficients (at the .60 -.70 level) for the borderline and the avoidant PDs, and a low level of concordance, with kappa’s below .40, for the obsessive-compulsive scale. Concerning the obsessive-compulsive PD, the correlation between the dimensional obsessive-compulsive scales amounts to .52, whereas at the pseudodimensional and at the categorical level coefficients of concordance drop as the scoring algorithms place more emphasis on the distress associated with the presence of the traits. These results indicate that on the one hand obsessive-compulsive individuals seem to underestimate the presence and especially the problematic or dysfunctional nature of their traits, while on the other hand the clinician administering the SCID-II interview not only seems to recognize these traits earlier but also evaluates them as problematic or dysfunctional more easily. Considering the efficiency of the different scoring algorithms, we found that the T>4&D>1 and T>5&D>1 scoring algorithms displayed adequate levels of concordance—at the .50 level—for an “any PD” diagnosis. Evaluation of the parameters of diagnostic efficiency indicates that the T>4&D>1 scoring algorithm is quite sensitive but may not have enough specificity, whereas the T>5&D>1 scoring algorithm displays more specificity with respect to the SCID-II. Therefore, at the moment, the T>5&D>1 scoring algorithm seems to be the best choice if one is aiming at a close correspondence with the interview method. The T>4&D>1 scoring algorithm seems more appropriate when the ADP-IV is used as a screening tool. Consequently, our

THE ADP-IV QUESTIONNAIRE

417

Internet application generates a report that contains the results obtained by both scoring algorithms. It is worth mentioning that the ADP-IV—in contrast to other Axis II self-report questionnaires (Loranger, 1992; Perry, 1992; Clark et al., 1997; Bronisch & Mombour, 1998)—does not lead to an overdiagnosis nor results in the typical high comorbidity between PD categories. In Study 2, the SCID-II interview produced a mean of .59 diagnoses: the somewhat too sensitive T>4&D>1 scoring algorithm resulted in a mean value of 1.31 diagnoses while the T>5&D>1 scoring algorithm yielded .53 diagnoses. Compared to other PD questionnaires, which diagnose up to five times more disorders in comparison with semi-structured interviews (e.g., Hunt & Andrews, 1992; Duijsens, Eurelings-Bontekoe, & Diekstra, 1996) the present results are quite exemplary. In summary, the results of the present studies support the construct validity of the ADP-IV and warrant research with the instrument in the field of the PDs (e.g., De Clercq & De Fruyt, 2003; Tenney, Schotte, Denys, van Megen, & Westenberg, 2003) and its use in clinical and therapeutical settings (e.g. Schotte, 2002; Schotte et al., 2002).

REFERENCES American Psychiatric Association (APA). (1994). DSM-IV: Diagnostic and Statistical Manual of Mental Disorders (4th ed.). Washington, DC: Author. Arntz, A., Beijsterveldt, B., Hoekstra, R., Hofman, A., Eussen, M., & Sallaerts, S. (1992). The interrater reliability of a Dutch version of the Structured Clinical Interview for DSM-III-R Personality Disorders. Acta Psychiatrica Scandinavica, 85, 394-400. Bronisch, T., & Mombour, W. (1998).The modern assessment of personality disorders: Part 2: Reliability and validity of personality disorders. Psychopathology, 31, 223-301. Clark, L.A., Livesley, W.J., & Morey, L. (1997). Personality disorder assessment: The challenge of construct validity. Journal of Personality Disorders, 11, 205-231. Cohen J.A. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46. De Clercq, B., & De Fruyt, F. (2003) Personality disorder symptoms in adolescence: A Five-Factor Model Persective. Journal of Personality Disorders, 17, 269-292. De Doncker, D., Schotte, C., Vertommen, H., & Van Kerckhoven, C. (1997). Assessment van de DSM-IV persoonlijkheidsstoornissen: Ontwikkeling en voorlopige resultaten van de ADP-IV

zelfbeoordelingsvragenlijst. Tijdschrift Klinische Psychologie, 27, 171-186. Dreessen, L., & Arntz, A. (1998). Short-interval test-retest interrater reliability of the Structured Clinical Interview for DSM-III-R Personality Disorders (SCID-II) in outpatients. Journal of Personality Disorders, 12, 138-148. Duijsens, I.J., Eurelings-Bontekoe, E.H.M., & Diekstra, R.F.W. (1996). The VKP, a self-report instrument for DSM-III-R and ICD-10 personality disorders: construction and psychometric properties. Personality and Individual Differences, 20, 172-182. First, M.B., Gibbon, M., Spitzer, R.L., Williams, J.B.W., & Benjamin, L. (1997). User’s guide for the Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II). Washington, DC: American Psychiatric Association. First, M.B., Spitzer, R.L., Gibbon, M., Williams, J.B.W., Davies, M., Borus, J., Howes, M.J., Kane, J., Pope, H.G., & Rounsaville, B. (1995). The Structured Clinical Interview for the DSM-III-R Personality Disorders (SCID-II): Part II: Multi-site test-retest reliability study. Journal of Personality Disorders, 9, 92-104. Fossati, A., Maffei, C., Bagnato, M., Donati, D., Donini, M., Fiorilli, M., Novella, L., & Ansoldi, M. (1998). Brief communica-

418 tion: Criterion validity of the Personality Diagnostic Questionnaire - 4+ (PDQ-4+) in a mixed psychiatric sample. Journal of Personality Disorders, 12, 172-178. Hunt, C., & Andrews, G. (1992). Measuring personality disorder: The use of self-report questionnaires. Journal of Personality Disorders, 6, 125-133. Klein, M. H., Benjamin, L. S., Rosenfeld, R., Treece, C., Husted, J., & Greist, J. H. (1993). The Wisconsin Personality Disorders Inventory: Development, relia b i l i t y an d v a l i d i t y . Journal of Personality Disorders, 7, 285-303. Loranger, A. W. (1992). Are current self-report and interview measures adequate for epidemiological studies of personality disorders? Journal of Personality Disorders, 6, 313-325. Loranger, A. W., Sartorius, N., Andreoli, A., Ber g er , P. , Buchheim, P. , Channabasavanna, S.M. et al. (1994). The International Personality Disorder Examination: The World Health Organization/Alcohol, Drug Abuse, and Mental Health Administration international pilot study of personality disorders. Archives of General Psychiatry, 51, 215-224. Maffei, C., Fossati, A., Agostoni, I., Barraco, A., Bagnato, M., Donati, D., Namia, C., Novella, L., & Petrachi, M. (1997). Interrater reliability and internal consistency of the Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II), Version 2.0. Journal of Personality Disorders, 11, 279-284. Maier, W., Lichtermann, D., Klingler, T., Heun, R., & Hallmayer, J. (1992). Prevalences of personality disorders (DSM-III-R) in the community. Journal of Personality Disorders, 6, 187-196. Oldham, J.M. (1994). Personality disorders: Current perspectives. Journal of the American Medical Association, 272, 1770-1776. Perry, J. C. (1992). Problems and considerations in the valid assessment of personality disorders. American Journal of Psychiatry, 149, 1645-1653. Reich, J. & de Girolamo, G. (1997). Epidemiology of DSM-III personality disorders in the community and in clinical populations. In (A.W. Loranger, A. Janca, & N. Sartorius, Eds.) Assessment and diagnosis of personality disorders. The

SCHOTTE ET AL. ICD-10 international personality disorder examination (IPDE) (pp. 10-17). Cambridge, UK: Cambridge University Press. Reich, J., & Troughton, E. (1988). Frequency of DSM-III personality disorders in patients with panic disorder: Comparison with psychiatric and normal control subjects. Psychiatry Research, 26, 89-100. Renneberg, B., Chambless, D.L., Dowdall, D.J., Fauerbach, J.A., & Gracely, E.J. (1992). The Structured Clinical Interview for DSM-III-R, Axis II and the Millon Clinical Multiaxial Inventory: A concurrent validity study of personality disorders among anxious outpatients. Journal of Personality Disorders, 6, 117-124. Schotte, C.K.W. (2000). New Instruments for diagnosing personality disorders. Current Opinion in Psychiatry, 13, 605-609. Schotte, C. K. W. (2002). Assessment of borderline personality disorder: Consideri n g a d i a g n o s t i c s t r a t e g y . Acta Neuropsychiatrica, 14, 55-59. Schotte, C., & De Doncker, D. (1996). ADP-IV Questionnaire: Manual and norms. Antwerp, Belgium: University Hospital Antwerp. Schotte, C. K. W., & De Doncker, D. (2000). De ADP-IV: Een vragenlijst voor een therapeutisch georiënteerde diagnostiek van DSM-IV persoonlijkheidsstoornissen? PsychoPraxis, 2, 267-273. Schotte, C.K.W., De Doncker, D., Dmitruk, D., De Valck, E., & Van Mulders, I. (2002). On der zoek n aar de begripsvaliditeit van de ADP-IV: Bevindingen betreffende discriminante v a l i d i t e i t , co n v e r g e n t i e m e t de s e m i - g e s t r u c t u r e e r d e interviewmethode en therapeutische bruikbaarheid. Tijdschrift Klinische Psychologie, 32, 187-205. Schotte, C., De Doncker, D., Maes, M., Cluydts, R., & Cosyns, P. (1993). MMPI assessment of the DSM-III-R histrionic personality disorder. Journal of Personality Assessment, 60, 500-510. Schotte, C.K.W., De Doncker, D., Van Kerckhoven C., Vertommen, H., & Cosyns, P. (1998). Self-report Assessment of the DSM-IV personality disorders: Measurement of trait and distress

THE ADP-IV QUESTIONNAIRE characteristics: The ADP-IV. Psychological Medicine, 28, 1179-1188. Smith, T.L., Klein, M.H., & Benjamin, L.S. (2003). Validation of the Wisconsin Personality Disorders Inventory-IV with the SCID-II. Journal of Personality Disorders, 17, 173-187. Spitzer, R.L. (1983). Psychiatric diagnosis: Are clinicians still necessary? Comprehensive Psychiatry, 24, 399-411. Tenney, N.H. , Schotte, C.K.W., Denys, D.A.J.P., van Megen, H.J.G.M., & Westenberg, H.G.M. (2003). Assessment of DSM-IV personality disorders in OCD: Comparison of clinical diagnosis, self-report questionnaire, and semi-structured interview. Journal of Personality Disorders, 17, 550-561. Weertman, A., Arntz, A., & Kerkhofs, M. (1997). Nederlandstalige versie van de SCID-II (DSM-IV versie). Maastricht: University of Maastricht, Department of Medical, Clinical and Experimental Psychology.

419 Weissman, M. M. (1993). The epidemiology of personality disorders: A 1990 update. Journal of Personality Disorders, 7(Suppl.), 44-62. Westen, D., & Shedler, J. (2000). A prototype matching approach to diagnosing personality disorders: toward DSM-IV. Journal of Personality Disorders, 14, 99-109. Widiger, T. A., Frances, A. J., Harris, M., Jacobsberg, L. B., Fyer, M., & Manning, D. (1991). Comorbidity among Axis II disorders. In (J.M. Oldham, Ed.) Personality disorders: new perspectives on diagnostic validity (pp. 163-194). Washington, DC: American Psychiatric Press. Zimmerman, M., Pfohl, B., Coryell, W., Stangl, D., & Coranthal, J. L. C. (1988). Diagnosing personality disorder in depressed patients: A comparison of pat i e n t an d i n f o r m a n t i n t e r v i e w s . Archives of General Psychiatry, 45, 733-737.