Pre-Publication DRAFT

Copyright © 1990 Demosthenes Lorandos

CHANGE IN ADOLESCENT BOYS AT TEEN RANCH: A FIVE-YEAR STUDY

Demosthenes A. Lorandos ABSTRACT

-

A five year study was undertaken to assess the effectiveness of the Teen Ranch

-

AF

-

T

70 bed residential treatment program for adolescent boys. To date, no long¬ term study of treatment effectiveness has been conducted; research has con ¬ centrated on recidivism, and while this provides some meaningful information, it provides little or no data to treatment directors, therapists, and adminis¬ trators concerning in-process therapy or client change issues. The present study compared 120 full psychological test and retest batteries of boys who were in the program for at least one year. Through test retest analysis, 27 intellectual and personality variables were compared to describe changes that took place . Nine intellectual and academic change variables and nine person ¬ ality change variables were found to be significant . These changes were utilized to examine the overall effectiveness of the Teen Ranch residential treatment program and the possibility of integrating evaluation and treatment processes.

Pre -Pu

bli

cat

ion

DR

In the last twenty years, there has been tremendous growth in the number of residential treatment programs for adolescents. This has particularly been the case for boys. In the past, boys referred to resi¬ dential treatment were orphaned, runaways, or involved in petty theft, while today’s referrals are involved in murder, criminal sexual conduct , serious drug offenses, and a variety of significant familial pathologies. With a 70-bed residential treatment program in three Michigan counties and a growing, intensive foster care and family counseling program, Teen Ranch has been providing high-quality residential and outreach treatment to adolescents and their families for a generation. Working to meet the needs of more seriously disordered adolescents, Teen Ranch initiated an intensive treatment program update in 1982 . At the core of this revision was a process of objective psychological assessments at onset of treatment and annually thereafter . These as¬ sessments were designed to inform program directors, counselors, and school personnel as to each resident’s strengths and weaknesses. In this way , an attempt was made to integrate assessment data into in ¬ dividualized treatment plans. A goal set by the authors of the program revision was to follow up on earlier theoretical research ( Lorandos, 1978; Strupp & Hadley,

1


- -

-

AF

T

1977a ) and implement a computer aided integration of day to day treatment data with objective assessment findings. Utilizing a pictorial representation of each resident’s daily treatment record, assessment data, and information from school, this evaluation process was envi¬ sioned as providing immediate micro assessments of each boy’s process and macro assessments of program effectiveness as a whole. Before work on the data development and integration software was started , a detailed review of the literature was undertaken . Administrators and counselors have decried the scarcity of valuable data from program evaluations. A search through 30 years of program evaluation literature underscored these complaints. Some of the early work done in goal attainment and behavior scaling was found to have value ( Kiresuk & Sherman, 1968; Struening & Guttentag, 1975; Millman & Pancost. 1977; Peckham 1977 ; Adams, 1980 ), but little or no information as to its integration into ongoing counseling was offered. Maloney , Timbers, and Maloney (1977 ) discuss a federally funded program similar to Teen Ranch , but do not offer information as to its evaluation. Most studies reviewed used after the-fact questionnaires and for¬ mulas ( Couturier, 1980; Kazdin , French, & Sherick , 1981; Rachlin , 1983; Velasquez & Lyle, 1985: Giacobbe & Schneider , 1986; Gilliland Mallo & Judd, 1986) or were questionnaire studies of a very small number of subjects ( Nelson & Johnson , 1975; Schaefer , 1977; Munson & Blineoe, 1984; Lampen & Neill, 1985; Cote , Harris, & Vipond , 1986; Jackson, Olsen, Schaefer, & Holmes, 1986). Confounding the use of these after the fact questionnaires, Lewis (1982 ) concluded that “ im ¬ provement made in residential treatment does not determine the de¬ gree of adjustment experienced upon return to home and community.” A number of researchers pointed to methodological problems in eval ¬ uations ( Bergin, 1971; Handler , 1975; Gross & Miller, 1975; Johnson et al., 1976; Strupp & Hadley , 1977b; Brown , 1980 ). Strupp and Hadley (1977b ) state: “ We contend that profound discrepancies in evaluative criteria are continuing to confound our best efforts at evaluating the outcomes of psychotherapy” ( p. 483). Elsewhere, Strupp and Hadley (1977a ) describe a three-part model of effectiveness evaluation, inter correlating societal variables, therapists’ views, and clients’ opinions: “ If one is interested in a comprehensive picture of the individual , eval¬ uations based upon a single vantage point are inadequate and fail to give necessary consideration to the totality of an individual’s func ¬ tioning” ( p. 190). Unfortunately, their work remained theoretical. No suggestions or field trials of this tripartite model were described. Faced with this lack of scientific data addressing specific changes resulting from residential treatment of adolescent boys, and seemingly

DR

-

Pre -Pu

bli

cat

ion

- -

2


little or no information describing the integration of evaluation data into the treatment process, Michigan Psychological Services aided Teen Ranch in seeking help to develop its own format . The Skillman Foun ¬ dation of Detroit provided funds for a pilot study to assess the effec¬ tiveness of Teen Ranch’s residential treatment program and to develop a program for integrating the findings into the ongoing treatment

process. To lay the foundation for the computer-aided integration program, a basal intellectual personality process for Teen Ranch clients was sought. Five hundred thirty four psychological test protocols were re¬ viewed, and their objective indicators in intellectual, academic , and personality variables were reduced to standard scores. These were com¬ pared and basic intellectual , academic , and personality profile data were developed for the typical Teen Ranch adolescent in residential therapy . From these 534 batteries, 120 protocols were selected because they offered complete test retest batteries of adolescents in their first and thirteenth month of treatment. These were utilized to provide an objective analysis of change in Teen Ranch boys. Positive change in a number of variables was expected. These were hypothesized to be within areas of intellectual and personality func¬ tioning most immediately affected by the residential treatment process.

T

-

METHOD

DR

AF

-

cat

ion

Subjects The subjects in this study were adolescent boys referred to the Teen Ranch residential treatment program from January 1983 to June 1988. They ranged in age from 11 to 17 years, with a mean age of 14 years 6 months at onset of treatment. Assessment revealed a mean WISC-R Full-Scale I.Q. of 98. Average length of stay was 15 months. The ju ¬ venile court system in Michigan was the typical referral mechanism.

Pre -Pu

bli

Procedure Within 30 days of treatment onset , each boy was administered a full battery of psychological tests, including a mental status examination , Bender-Gestalt Test , Slosson Drawing Coordination Test , Wechsler Intelligence Scale for Children-Revised , Wide Range Achievement Test-Revised , H-R Stress Test , Projective Drawings, Self-Description Inventory, Sentence Completion Test, and the Minnesota Multiphasic Personality Inventory ( Overall’s abbreviated “ 168” version ) . For the purpose of evaluating change in the boys, 120 protocols were selected because they represented an evaluation within the first 30 days of

3


entry and another identical battery on the same boys’ first year an ¬ niversary in residence. The most clearly objective indices (WISC R ,

-

-

WRAT-R, and MMPI 168 ) were utilized as the standard, and T-scores could be easily compared. These scales offered 27 variables to inter ¬ correlate between Test One and Test Two. With a sample size of 120 and 27 variables for comparison these 3,240 values were stored in the NH Analytical Softwares Statix II program within an IBM AT. A paried t test was run on each Test One versus Test Two correlation. ,

-

RESULTS

Table 1

Chances i n I n t e l l e c t u a l / Academic S k i l l s :

DR

Significant

-

AF

T

Comparison of 27 variables between Test One and Test Two provided the expected and some unexpected results. Seventeen of the 27 vari

Mean Standard Scores And Paired t'-Test Values

. Test

Test One vs

Variable

Full scale I.Q. Verbal I.Q.

Score

Test One

Test Two

t Values

96.98

100.7

- 6.93

90.53

Digit span

.

Pre -Pu

Performance I.Q

Picture completion

93.54

-

4.99

8.058

8.5

- 2.92

8.3

9.15

- 3.57

bli

Vocabulary

Coding

Score

cat

-

.

Mean Std

ion

Mean Std.

Two and t Values

105.2 11.12

9.101

11.88

- 5.12 - 3.29

10.16

-

108.7

3.47

Reading

89.43

87.76

+ 1.97

Arithmetic

85.46

88.72

- 2.96

4


ables compared demonstrated significant change. Eight of the 14 in¬ tellectual academic scores increased significantly. Reading recognition as measured by the WRAT R decreased. In the domains measured by the MMPI 168. eight of the 13 values measured changed significantly. With five years of data and over 530 cases, a normal distribution of scores within the Teen Ranch population was assumed. Therefore , to be significant at the .05 level, paired t test values of ± 1.96 were needed. Table 1 depicts significant changes in intellectual academic variables between Test One and Test Two. The changes in WISC R Full-Scale , Verbal , and Performance I.Q. were the most significant. Table 2 presents the significant changes in the domains measured by the MMPI-168. Most startling are the decreases noted in depression and the domain measured by the F scale. The increase noted in mas¬ culinity-femininity suggests a very positive result of the treatment process as well.

-

-

1

AF

T

-

Table 2

DR

Sianificant Chanaes In MMPI-168 Scores: Mean T-Scores And Paired t-Test .. Values

. Test

Test One vs

Mean

Mean

T-Score

T-Score

Test One

Test Two

ion

Variable

K scale

73.02

+ 4.40

47.35

49.24

-

1.99

64.73

57.80

+

5.59

2.50

bli

Depression

t Values

78.42

cat

F scale

Two and t Values

73.02

70.44

+

Mascul'ty/Femin * ty

57.26

59.07

- 2.60

Paranoia

67.37

63.45

+

Psychasthenia

65.84

61.37

+ 3.07

Social Inversion

58.35

56.53

+

Pre -Pu

Psychopathic Deviancy

5

2.83

1.87

Copyright © 1990 Demosthenes Lorandos DISCUSSION

A detailed review of the literature of program evaluation in resi¬ dential treatment settings offered little data to support the ongoing process of treatment. Program directors and counselors wishing to in ¬ tegrate program evaluative data into the process of treatment have been faced with mail out questionnaires, reports of recidivism , and little else to aid in this collaboration. Preliminary to a more formalized and computer assisted integration of detailed test protocol data into the ongoing treatment process , this researcher collated the results of 534 psychological evaluations in order to begin to objectively assess the Teen Ranch program’s effectiveness in helping adolescents to change. With funding from the Skillman Foundation of Detroit , computers and software were purchased and the data from over 15,000 psychological testing variables intercorrelated. The intention of the study was to lay the groundwork for the integration of computerized interpretation of evaluation processes into the treat ¬ ment milieu , and to provide an overall assessment of the way in which boys typically change during their stay at Teen Ranch . The results of this study were rewarding and somewhat startling. Paired t test analysis demonstrated significant positive change in in ¬ tellectual scores which therapists would say indicates a strong and enhanced ability to cope with the world . Startling changes in intellec¬ tuality were offset by a decrease in reading achievement. This may represent an indictment of the rural public school systems upon which the Ranch relies to provide formal education for its youngsters. Within the domains measured by the abbreviated version of the MMPI, dramatic decreases in pathological processes were noted. The significant decrease in depression over time seems to bear out the value of the Ranch’s counseling program. The domain measured by the F scale, which some feel has to do with rare and/or magical thinking, also was significantly decreased. Another positive effect was the in¬ crease in the masculinity -femininity scale, which is an indication of enhanced ability to be expressive. These results point to the effectiveness of Teen Ranch as a residential treatment setting for adolescents. The entire process of this study sug¬ gests that a strong foundation has been laid for going beyond the afterthe fact evaluative measures of the past and integrating treatment and program evaluation data into the ongoing process of day to day counseling. The next step will be to develop the software to allow program directors and therapists to access a boy’s entry psychological report, integrate this with his daily treatment record , and report a

-

-

Pre -Pu

bli

cat

ion

DR

AF

T

-

- -

6


pictorial representation of change in behavior , intellectuality, academ ¬ ics, community integration , and psychopathology . This process is un ¬ derway and will be discussed in a forthcoming paper . REFERENCES Adams. D. B. < 1980 ) . Adolescent residential treatment: An alternative to in ¬ stitutionalization . Adolescence. 25' 59 . 521-527. ' outcomes. In A. Bergin & B. Bergin . A . 1971 ). The evaluation of therapeutic ( Garfield Eds. ), Handbook of psychotherapy and behavior change. New York: Wiley. Brown , S. (1980 ) . An outcome evaluation of adolescents placed in a long term residential treatment center. Dissertation Abstracts International 41 ,

'

'

-.

'

DR

AF

T

683. Cote. J . . Harris, D. P.. & Vipond . E . ( 1986 ). A psychometric evaluation of a residential treatment facility: An illustration of an interpretable re ¬ search design without a control group. Adolescence . 22 ( 81 ). 67-79 . Couturier. L . C. ( 1980 ). An evaluation of group homes for delinquent male adolescents: The relationships of various program variables to the youths academic and talented achievements , school attendance , ego develop ¬ ment. moral development, and group home behavior . Dissertation Ab ¬ stracts International , 41 , 161. Giacobbe , G.. & Schneider. F. ( 1986 ). The success rate index: A method for evaluating residential treatment programs. Journal of Offender Coun ¬ seling , Services and Rehabilitation , 10 , 97-105. Gilliland -Mallo. D.. & Judd. P. ( 1986 . The effectiveness of residential care

bli

cat

ion

' facilities for adolescent boys. Adolescence , 22 ( 82 ) , 311-321. Gross, S.. & Miller. J. ( 1975). A research strategy for evaluating the effec ¬ tiveness of psychotherapy. Psychological Reports , 37 , 1011-1021. Handler . E . ( 1975 ) . Residential treatment programs for juvenile delinquents. Social Work , 20 , 217-222. Jackson . N.. Olsen. L . , Schaefer , C .. & Holmes. W. ( 1986 ). Evaluating the treatment of emotionally disturbed adolescents. Social Work , 31. 182 185. Johnson. H., Nutter . C.. Callan , L . . & Ramsey . R. ( 1976 ) . Program evaluation in residential treatment: Some practical issues. Child Welfare . 55, 279-287. Kazdin . A .. French , N .. & Sherick . R . ( 1981 ) . Acceptability of alternative treat ¬ ments for children: Evaluations by inpatient children parents, and staff . Journal of Consulting and Clinical Psychology. 49 . 900 907 . Kiresuk , T,. & Sherman, R. ( 1968 ) . Goal attainment scaling: A general method for evaluating comprehensive community mental health programs. Com munity Mental Health Journal , 4 , 443-453. Lampen , J ., & Neill. T. (1985 ) . A bucket of cold water: A follow -up study in a residential special school. Journal of Adolescence , 8 , 271-287 . Lewis. W. ( 1982 ). Ecological factors in successful residential treatment . Be ¬ havioral Disorders . 7 , 149-156. Lorandos, D. ( 1978 ). The integration of treatment and evaluation . Paper pre¬ sented to the National Institute on Drug Abuse National Convention , Seattle, WA.

-

,

-

Pre -Pu

-

7


Maloney. D . , Timbers, G.. & Maloney, K . < 1977). BIABH project: Regional adaptation of the teaching-familv model group home for adolescents. Child Welfare , 56 , 787 796 . Millman. H . , & Pancost , R. (1977). Program evaluation in a residential treat ¬ ment center. Behavioral Disorders . 2 , 66-75. Munson, R.. & Blincoe, M. ( 1984 ) . Evaluation of a residential treatment center for emotionally disturbed adolescents. Adolescence , 19 ( 14 ) . 253-261. Nelson. R .. & Johnson. L. ( 1975 ). Asking the children. Child Care Quarterly , 4 , 273-276. Peckham , R . ( 1977 ). Uses of individualized client goals in the evaluation of drug and alcohol programs. American Journal of Drug and Alcohol Abuse , 4 , 555 570. Rachlin, D. ( 1983). Evaluation of a short-term residential program for delin¬ quent adolescent males. Dissertation Abstracts International , 44 , 596. Schaefer. C . (1977 ) . Using a mail questionnaire to evaluate the community adjustment of children discharged from residential treatment . Journal of Genetic Psychology , 131 , 331-332 . Struening, E .. & Guttentag, S. ( 1975 ) . Handbook of evaluation research ( Vol. 1). Beverly Hills. CA: Sage. Strupp, H.. & Hadley, S. ( 1977a . A tripartite model of mental health and therapeutic outcomes. American Psychologist , 33 , 187-196. Strupp. H .. & Hadley. S. (1977b ) . Evaluations of treatment in psychotherapy: Naivete of necessity? Professional Psychology , 8 . 478-490. Velasquez, J .. & Lyle , C. ( 1985). Day residential treatment for juvenile of ¬ fenders: The impact of program evaluation. Child Welfare , 64 , 145 156.

-

Pre -Pu

bli

cat

ion

DR

AF

T

-

8

-