The Multidimensional Adolescent Assessment Scale - SAGE Journals

RESEARCH ON SOCIAL WORK PRACTICE

Mathiesen et al. / MULTIDIMENSIONAL ADOLESCENT ASSESSMENT SCALE

The Multidimensional Adolescent Assessment Scale: A Validation Study Sally G. Mathiesen Scottye J. Cash Walter W. Hudson

Florida State University

The Multidimensional Adolescent Assessment Scale is a tool for the assessment of the severity of personal and social problems in adolescence. It is composed of 16 subscales relevant to adolescent functioning, including depression; self-esteem; problems with mother, father, or family; personal stress; problems with friends or school; aggression; suicidal thoughts; feelings of guilt; confused thinking; disturbing thoughts; memory loss; and alcohol or drug use. The scale was designed for use by practitioners in a variety of disciplines to assess a comprehensive array of problem areas to facilitate diagnosis and targeted treatment planning. This article provides a description of the scale, explains the method of scoring, and presents the psychometric properties of the instrument.

Researchers and practitioners in the human services have long recognized the multifaceted nature of problems in living. In terms of adolescents, developmental issues as well as problems that may become pathological are important to consider. A great deal of attention has been given to the social work perspective of the person in the environment (Germain & Gitterman, 1980; Karls & Wandrei, 1995; Kemp, Whittaker, & Tracy, 1997). Kemp et al. (1997) noted that

Authors’ Note: Sally G. Mathiesen and Scottye J. Cash would like to acknowledge the extraordinary generosity of Walter Hudson in regard to his contribution to this article as well as to the profession of social work. The data collection, analyses, and interpretation of the results were completed after Hudson’s death. The description of the instrument and the method by which he sought to establish reliability and validity were created and submitted for this article by Hudson. To preserve the memory of his contribution to this article and to the methodology that he established, those sections of the article have remained in the words and style of Hudson. Research on Social Work Practice, Vol. 12 No. 1, January 2002 9-28 © 2002 Sage Publications

9

10


the ability to understand “environment” from the client’s perspective and to function effectively in the environmental domain is central to many emergent areas of practice, such as practice with extended families and personal networks, practice from a “strengths” perspective, and culturally competent practice. (p. xi)

Yet, full understanding of the relationship between the individual and the social environment will not be possible without reliable and valid measurement tools that can assess multiple problem areas. Standardized instruments have been developed that measure unidimensional aspects of adolescent functioning or problem areas such as depression. However, there are few standardized self-report instruments that capture the issues of personal and social functioning from a multidimensional perspective. Social workers continually face the issue of finding reliable and valid instruments that are readily accessible, can be interpreted by social workers, and address both the person and the social context in which the person functions. This is a critical gap, given the importance of personal and social relationships for social work practitioners. As noted, many measures have a unidimensional focus. For example, the Child Depression Inventory (Kovacs, 1985) is a 27-item self-report measure for children age 7 to 17. It was originally based on the Beck Depression Inventory (Beck, Ward, Mendelson, Mock, & Erbaugh, 1961) and was tested and refined for children. Even though this is a widely used instrument for assessing depression in clinical and nonclinical samples, reports of instrument stability range from poor to excellent. One study reported a reliability of .39 for school children over a 1-week interval (Saylor, Finch, Spirito, & Bennett, 1984), whereas Kovacs reported a test-retest coefficient of .84. Nelson and Politano (1990) examined children in an inpatient setting and found the correlation between scores was .62 after 10 days, which significantly decreased to .47 after 30 days. The authors noted that the decrease may reflect change in the child’s actual depressive state or “it may be an artifact of the scale itself” (p. 256). In terms of measuring multiple areas of behavior, the Child Behavior Checklist (CBCL) (Achenbach, 1991) uses parental ratings of children’s behaviors, emotional status, and strengths. The scale results in scores for eight dimensions of behavior, plus an internalizing, externalizing, and total problem score. A teacher rating format as well as a self-report version of the CBCL for youth age 11 to 18 are available to assess these domains. A significant advance in the area of social work measurement was made with the development of the Multi-Problem Screening Inventory (MPSI) (Hudson, 1990), a comprehensive assessment instrument with 27 subscales


11

that have been used in research and practice settings. The considerable length of the instrument makes it difficult to administer in many practice settings. In spite of its length, the MPSI has exhibited strong psychometric properties (Hudson & McMurtry, 1997) that make it an important measurement source for social workers. The Brief Adult Assessment Scale (BAAS) (Hudson, 1995) was created as a subset of the MPSI and significantly reduced the number of items to which a client was asked to respond. The BAAS contains 16 subscales that assess personal and social functioning in adults. A pilot study of data gathered with the BAAS indicated that aspects of social functioning explained as much as 65% of the variance in personal functioning (Hudson, Mathiesen, & Lewis, 2000). Hudson (1995) recognized the need for a similar reliable and valid assessment tool for adolescent functioning and created the Multidimensional Adolescent Assessment Scale (MAAS). The MAAS has a significant advantage over the CBCL. Although both instruments have a multidimensional perspective, the MAAS has a combined total of 16 subscales. The larger number of domains in the MAAS provides clinicians, researchers, and families with a greater and more comprehensive understanding of adolescent personal and social functioning. In addition, the MAAS explores areas that are not seen in the CBCL, such as suicidal ideation, and provides scales that differentiate between anxiety and depression. This article will describe the 16 subscales that comprise the MAAS. The psychometric properties of the MAAS, specifically reliability and validity, will be presented. Recommendations for using the MAAS in practice settings and educational environments will also be discussed.

THE MAAS SCALE

The MAAS is a paper-and-pencil self-report measure containing 177 items that requires an average of 15 to 20 minutes to complete. Clients (adolescents age 10-20) respond to each item by using a 7-point category-partition scale (Stevens, 1968). The inventory is divided into 16 different subscales ranging in length from 10 to 15 items, with each subscale producing its own score. These subscale scores are the main products of the MAAS because, as a multidimensional inventory, it does not produce a single composite score. Instead, subscale scores are used to develop a graphic profile of client problems that is then used in assessment and treatment planning. The nature of each subscale is described briefly by the subscale name. Readers may obtain a sample copy by writing to the publisher.

12


The principal aim of the MAAS is to serve as a tool for assessing diverse client problems during intake or early phases of treatment or service delivery. It can also be employed for periodic reevaluation by having clients complete it during later phases of treatment. Used in this manner, it provides a description of change (progress, stability, or deterioration) that may have occurred during the period between administrations. Whether it is used for interim assessments, it is well suited for use immediately before termination of services as a means of gauging progress over the entire period of clinical contact. Client performance on the subscales of the MAAS is designed to be easy to interpret, with each subscale producing a score ranging from 0 to 100. Lower scores represent the relative absence of problems in a specific area of personal and social functioning, and higher scores represent more serious problems in that area. Several of the subscales in the MAAS were derived from the MPSI, and others were specifically created to address adolescent functioning. Those subscales based specifically on the MPSI reported clinical cutting scores. It cannot be assumed that these same clinical cutting scores will be valid due to changes in the scale properties in the MAAS. For example, the number of scale items for each domain was reduced significantly (i.e., from 25 to 12), and the scaling was changed from a 5-point Likert-type scale to a 7-point Likert-type scale. For these reasons, this article will not suggest clinical cutting scores for the MAAS. Further validation studies are needed on the MAAS with both clinical and nonclinical populations to establish reliable and valid cutting scores. It is recommended that clinicians monitor the client’s status carefully and rely on conservative estimates of levels of risk depending on the nature of the problem area. The MAAS is intended to guide assessment and facilitate additional focused measures of problem areas identified in each of the 16 subscales.

THE MAAS SUBSCALES, SCORING BLANK, AND GRAPHIC PROFILES

Each subscale is scored by using a single formula and one set of scoring procedures for each of the 16 subscales. Although these procedures are not difficult, the inventory’s length does mean that scoring takes considerable time. On average, it takes about 25 to 30 minutes to score the MAAS manually and prepare a graphic profile. This increased demand on professional time is a disadvantage of using the MAAS or any such similar inventory. However, through use of available computer programs (including commercial spreadsheets), scoring time for the MAAS can be reduced substantially.


13

Clients respond to each of the 177 items using a 7-point category-partition scale (Stevens, 1968). Clients are also given the option of answering the item with an X, indicating that the item does not apply. To score these measures, the nonproblematic items, as indicated by an asterisk, are reverse scored by changing 1 to 7, 2 to 6, 3 to 5, 5 to 3, 6 to 2, and 7 to 1 (4 remains unchanged). Then, the total score is computed as S = (ΣY – N) (100) / (N*6), where Y represents the score on an item and N is the total number of items satisfactorily completed by the respondent. For some respondents, scales and/or items did not pertain to their life circumstances at the time and therefore were marked X. Omitted items and items scored outside the range of 1 to 7 are essentially ignored. A benefit to this scoring formula is that it produces scores ranging from 0 to 100 no matter how many items the respondent chooses to omit or score improperly. Lower scores represent the absence of or low levels for the specific problem area, and higher scores indicate the presence of a more serious problem in the area being assessed. Figure 1 displays a reproduction of the first page of the MAAS that contains subscale items. As can be seen, the first three subscales are depression, self-esteem, and problems with mother. As an illustration, scores from one respondent who participated in the testing of the instrument have been filled in for these three subscales. Figure 2 reproduces the MAAS subscale scoring blank, which is located near the end of the instrument. This page shows the title of all 16 subscales in the instrument, along with the number of items in each. Also included in Figure 2 are sample scores for the fictitious client data shown in Figure 1, completed for the first three subscales. Overall, the ability to spot clinically significant problems quickly and in relation to other areas of functioning is one of the key advantages of the MAAS and its graphic profile feature. Clients’ scores are most readily comprehended and applied to planning and assessment tasks by presenting them in the form of a graphic profile. As shown in Figure 3, the profile reveals that this client has major problems in several areas of personal and social functioning. Based on the 0 to 100 scale range, the clinician might pay particular attention to the participant’s scores on the following subscales: Scores were 48 on depression, 40 on self-esteem, 47 on father problems, 70 on personal stress, 36 on friend problems, 58 on aggression, and 55 on family problems. The highest score for this participant was a 70, on the personal stress subscale. Particular attention should be paid to the high-risk problem area of suicide, where this client reported a score of 33. An overview of the client’s profile graph reveals that the adolescent has problems in multiple areas of functioning. Although cutting scores have not been established, the suicidal ideation subscale always merits careful

14

Figure 1:


Depression, self-esteem, and problems with mother subscales of the Multidimensional Adolescent Assessment Scale.

attention as to the need for additional assessment. It is also evident that the client has significant problems with personal stress. The score of 70, the


Figure 2:

15

Multidimensional Adolescent Assessment Scale subscale scoring blank.

highest of all the subscale scores, may indicate a serious problem area. Problems with aggression and family problems were indicated. The family problems may be centered more on the father and other family members than on the mother. There are also indications of problems with guilt and confusion that may merit further attention. Areas that seem to be of less concern from the client’s perspective are drug and alcohol use. In other words, the scores alone are quickly suggestive of a diagnostic picture, which through discussions with the client will be clarified and used to create at least an initial treatment plan. The remainder of this article is devoted to describing the methodology of a recent study designed to validate the psychometric properties of the MAAS.

16

Figure 3:


Multidimensional Adolescent Assessment Scale score profile graph.

The reliability and validity of the MAAS subscales are then reported and interpreted.

METHOD

This study was conducted as a class project for bachelor of social work (BSW) and master of social work (MSW)–level courses in research methods at a large southeastern university, with the intent of training students to conduct basic social work research. Specific emphasis was placed on the method of data collection, protection of human participants, and data entry and analysis. MSW students in the research and development office located within the school of social work entered and cleaned all of the data. The purposes of conducting this study in this manner were to (a) provide BSW and MSW students with an applied example of conducting basic research, (b) produce useful new knowledge for the field, and (c) complete the tasks in a manner that made good use of the skills of all investigators involved in the study.


17

Procedure

Each BSW student contacted five adolescents and invited them to participate in the study. The age of the respondents ranged from 10 to 21. All students were trained to inform either the parent or guardian and the adolescent of the nature of the study, indicating that participation was voluntary and would be kept confidential. The adolescents who chose to participate in the study and who had parental permission were ensured that their parents or guardians would not be provided specific responses to any item. In addition, if the students elected to participate in the study by contributing their data to the study, they completed their own signed informed consent forms. If they chose not to participate, they did so without penalty and were offered another assignment to complete. This research did not involve greater than minimal risk ordinarily encountered in daily life or during the performance of routine physical or psychological examinations or tests. There was no expected psychological, legal, physical, or social harm to the respondents, and participation in the study was voluntary and confidential to the principal investigators. The respondents could withdraw from the study at any time. In addition, a list of counseling social service agencies was provided to all respondents in the instance that they would like to investigate the use of professional help. Parents were informed that they would not have access to their individual children’s responses to the items on the instrument. However, the parents were offered an opportunity to receive information from the study on the aggregated group results. The principal investigators were available to discuss with any respondent whatever anxiety or emotional distress that may have arisen in connection with the completion of the questionnaires. An informed consent form was obtained from both the parent and the adolescent prior to completion of any data collection, as required by the university’s institutional review board to protect human participants.

RESULTS Respondents

The BSW and MSW students contacted all 239 respondents. According to the criteria established, all respondents had to be between the ages of 10 and 21, and parental consent had to be given. Demographic information was obtained from the brief background questionnaire and is summarized in

18

TABLE 1:


Demographic Characteristics of Sample (N = 239)

Characteristic Gender Male Female Ethnic background White African American Hispanic Asian Other Age Current grade in school People living in household

n

%

M

SD

Mdn

Range

3

18

227 33.0 67.0 227 72.6 21.2 3.5 0.4 2.2 223 197 226

18.75 12.54

2.61 2.97

Table 1. Table 2 presents the mean and standard deviation for each subscale, as well as the number of respondents who completed each subscale. One reported standard for a scale to be used in practice is a reliability of .80 or greater (Carmines & Zeller, 1979). Hudson (1982) argued for a higher reliability standard of .90, and this was the target adopted for each subscale of the MAAS. However, such high reliability is very difficult to reach in practice. It should also be noted that reliability is only one measure of the usefulness of the scale. That scale reliability is sample dependent is one critique of classical measurement theory (Hambleton, Swaminathan, & Rogers, 1991). This is congruent with the authors’previous recommendation that the MAAS should be subjected to further research with different populations to establish additional levels of psychometric rigor and practical utility. The sample used for this study is a convenience sample based on respondents who were readily available to the students who administered the questionnaires. Thus, the question arises as to whether it was representative of the community from which it was drawn. For purposes of this article, it is not claimed that the sample is representative of any well-identified population in terms of community identification. This method of sampling is limited by its inability to generalize beyond the sample itself. For this reason, the results should be seen as an initial validation study of the MAAS, and again it is encouraged that further testing be conducted with representative samples. The respondents (N = 239) were predominantly female (67%) and White (72.6%), and the mean age of the sample was 18.75. The majority of the participants had completed high school, and the mean number of years of schooling was 12.54. The median total family size was three persons.


TABLE 2:

19

Mean Scores of Subscales with Standard Deviations

Subscale Depression Self-esteem Problems with mother Problems with father Personal stress Problems with friends Problems with school Aggression Family relationship problems Suicidal thoughts Feelings of guilt Confused thinking Disturbing thoughts Memory loss Alcohol abuse Drug use

n

M

SD

213 216 203 196 217 213 204 184 207 195 215 214 213 213 145 104

29.043 24.326 15.715 17.937 21.274 15.681 30.845 18.686 19.599 5.624 17.949 20.947 14.378 17.126 14.423 8.691

11.5042 13.7342 14.4825 17.6753 17.8740 13.8740 27.4495 18.0103 17.3738 13.5147 16.5075 18.8361 18.0815 13.8920 13.8767 17.4900

Psychometric Properties

Investigation of the psychometric properties of the MAAS subscales began by first examining their reliability, their standard errors of the mean, and basic descriptive statistics. Minimally, for a measurement tool to have any clinical or scientific utility, it must contain two psychometric properties. It must be reliable, and it must be valid. Without acceptable levels of reliability and validity, the researcher or clinician never can be confident that the measure accurately captures what it purports to capture. The following sections discuss the reliability and validity of the MAAS. Reliability

The principal device used in this study to evaluate the subscale reliability for the MAAS was Cronbach’s alpha coefficient. Each subscale was investigated separately, and the alpha coefficient for each subscale is shown in Table 3. Table 3 shows that all but 1 of the 16 subscales had alpha coefficients of .84 and above. The alpha for depression was still moderately high (.74), and 12 additional subscales were above .90. Ongoing investigation of the reliability of the MAAS will continue, but these alpha levels suggest that the scale is highly reliable in its present form.

20


TABLE 3:

Reliability Coefficients for Each Subscale


n 213 216 203 196 217 213 204 184 207 195 215 214 213 213 145 104

Coefficient Alpha .7429 .8566 .9081 .9143 .9461 .9301 .8466 .9040 .9413 .9749 .9060 .9148 .9596 .8493 .9258 .9513

Content and Factorial Validity

Content and factorial validity for an assessment tool such as the MAAS are generally related to one another and are therefore discussed together. Basically, the MAAS was designed to follow the principles of the domain-sampling model of measurement (Nunnally & Bernstein, 1994). According to this model, there is a theoretically infinite number of items that could be used to create each subscale. If such a population of items could be defined, the goal of instrument development would be to obtain a random sample of items from this population. The laws of probability sampling would thereby ensure that a random sample of such items would be acceptably representative of the domain of possible items comprising each dimension covered. In practice, however, it is impossible to define the total population of items that could represent each MAAS subscale. This means it is impossible to draw a true random sample of items from that population. Content validity must therefore be assessed in other ways. One approach is to measure the degree of consensus among competent judges as to whether each item on the scale is a member of the population of items that comprises the content domain. If high consensus is obtained, the scale can be considered to have acceptable content validity; if there is little consensus, no such assumption can be made. Although a study of this type is planned, no formal data are yet


21

available to represent the degree of consensus among judges as to the content validity of the MAAS subscales. A different approach to the evaluation of content validity is to examine a scale’s factorial validity, which is closely related conceptually to both content and construct validity. If a measurement tool is constructed as a multi-item device, the use of those items implies the hypothesis that each item in some way measures the construct in question and not some other construct. However, the use of an item in a scale also implies the hypothesis that it will have a higher correlation with an independent measure of the construct and that it will also have a lower correlation with other construct measures. The major task in investigating factorial validity is to find a way to test the hypotheses that items correlate well with the variables they are supposed to correlate with and that they correlate poorly with the constructs they are not supposed to correlate with. The multiple group method of factor analysis (Nunnally, 1978) is ideally suited for this task. The multiple group method is a type of a priori, confirmatory, or hypothesistesting factor analysis designed to show whether a well-specified hypothesis matrix will account for the pattern of correlation between a set of variables intended to represent a specific set of factors. One of the niceties of this method is that it reduces to a simple table that presents the correlation between scale items and subscale total scores (Hudson, 1982). In interpreting such a table, it should be recognized that the correlation of any scale item with its own total score is a part-whole correlation, or the correlation between an item and the sum of K – 1 items and itself (K is the total number of items) (Nunnally, 1978). Detailed findings concerning factorial validity are available from the authors in the form of 32 tables of item-total correlation. These findings are summarized in Table 4 due to space limitations that prevent the inclusion of such a large number of tables. The table presents results of factorial validity tables for all 16 subscales of the MAAS, described in terms of the percentage of factor-loading failures across subscales. These percentage failure rates were obtained for each subscale by dividing the number of loading failures by the number of loadings in the specific subscale and then multiplying by 100. The loading success rate is simply 100 minus the failure rate. The table shows each subscale name, the number of factor-loading failures, the total number of factor loadings, the percentage of failures, and the percentage of successes. When using the multiple group method of factor analysis, factor-loading criteria are not normally selected in terms of specified magnitudes. Instead, a factor-loading confirmation is achieved when a scale item correlates much better with its own total score as compared to its correlation with all other total scores and variables included in the analysis. Similarly, a

22

TABLE 4:


Percentage of Factor-Loading Failures and Successes for Subscale Item Loadings


Number of Number of Loadings % Loading Failures in Specific Scale Failure

% Success

3 0 0 0 0 0 6 0

192 192 192 208 192 208 160 160

1.6 0.0 0.0 0.0 0.0 0.0 3.5 0.0

98.4 100.0 100.0 100.0 100.0 100.0 96.5 100.0

0 0 0 0 0 0 0 1

208 176 160 112 144 128 240 160

0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0

100.0 100.0 100.0 100.0 100.0 100.0 100.0 99.0

disconfirmation is obtained when a scale item correlates less well with its own total score as compared to one or more other total scores or variables that are included in the analysis. Of the 16 subscales of the MAAS, 3 had loading failures. A factor failure indicates that an item within the scale loaded better with another scale. The drug abuse subscale had one factor failure, depression had 3 factor failures, and school problems had 6 factor failures. Regarding the school problems subscale, there was a consistent problem with factor failures on individual scale items and the aggression and confused thinking subscales. Although school problems had the highest number of factor failures, it still achieved a 96.5% success rate. This means that all the remaining 15 subscales had an even higher factor-loading success rate. Because the subscale items do a good job of correlating with their own total scores and not correlating with the total scores of all other subscales, it is appropriate to conclude that the MAAS has good factorial validity. In addition, because the items and the subscale total scores were constructed on the basis of the item contents, it then follows that these findings also provide strong support for the claim of good content validity for the inventory.


23

Some observers may be concerned that the ratio of observations to number of variables is quite small and does not conform to typical prescriptions of a ratio between 5 and 10 to 1. Like t tests with small samples, positive outcomes are very convincing, whereas negative findings pose an excessive risk of Type 2 error. In this case, the small cases-to-variables ratio augurs against confirmation of factorial validity. In any event, the results of this factor analysis need to be viewed cautiously, given the relatively small sample size. Construct Validity

Establishing construct validity requires the demonstration of both convergent and discriminant validity (Campbell & Fiske, 1959). In this regard, it is important to note that at the item level of analysis, the MAAS subscales correlate well with the things they are theoretically supposed to correlate with, their own subscale total scores, and this is the definition of convergent validity. Moreover, the MAAS subscale items tend not to correlate strongly with the things that theoretically, they should not correlate with, the remaining subscale total scores, and this is the definition of discriminant validity. Thus, the previous findings concerning factorial validity also provide good beginning evidence of both convergent and discriminant validity at the subscale item level of analysis. Another way to investigate discriminant validity is to specify a number of variables that theoretically should be unrelated to the MAAS subscales and to determine whether they indeed fail to correlate with the subscale scores. Data collected on the background variables are useful in this regard, as it can be contended that there is no good theoretical basis for believing that the respondents’gender, age, years of education, and family size are related to any of the MAAS subscales. It can thus be predicted that none of these variables will have meaningfully large correlations with the MAAS subscales. Table 5 reports the bivariate correlation computed between each of the background variables and the subscale scores. As can be seen, all of the 64 correlations are equal to or less than .289, meaning that the very largest correlation in the table accounts for no more than 8.35% of the variance. Because the majority of the correlations shown in Table 5 are much smaller, that means each variable in the pair accounts for less than 8% of the variance in the other variable in the pair (as indicated by squaring the coefficient). These findings provide convincing evidence of the discriminant validity of the MAAS subscales with respect to the background variables included in the study. Reviewing the means of the absolute values of the correlations in each row and column of the table further supports this conclusion. The mean of all

24

TABLE 5:


Correlations Between Multidimensional Adolescent Assessment Scale Subscales and Background Variables


Gender a .035 –.002 –.034 .112 .012 –.055 –.031 –.206 –.040 –.055 –.029 –.005 –.098 –.034 –.143 –.121

Age

Grade in School

People in Household

–.115 –.236 –.106 –.160 –.014 –.185 –.062 –.161 –.063 –.052 –.191 –.106 –.126 –.057 –.056 –.166

–.112 –.289 –.095 –.167 .009 –.176 –.075 –.189 –.065 –.025 –.172 –.126 –.145 –.056 .022 –.227

–.076 –.025 –.050 –.047 –.125 –.053 –.016 –.081 –.077 –.058 –.083 –.057 –.077 –.049 –.054 –.018

a. Gender was coded as 0 = male and 1 = female.

correlations in the table was only –.0841. It thus seems justifiable to conclude that this is evidence of acceptable discriminant validity for each of the MAAS subscales. As a test of convergent validity, it would be desirable to predict and test for relationships between the subscales that are presumably related. Unfortunately, testing such relationships on members of a nonclinical sample may produce inappropriate conclusions with respect to the performance of the instrument with clinical populations. For example, the presence of a problem on the part of a nonclinical respondent may be much less likely to be associated with the appearance of other problems than would be the case with a client who is experiencing clinically meaningful problems. In addition, no one has a good empirical basis for making strong or weak predictions concerning the theoretical relationships between the 16 subscales of the MAAS. The best that can be done is to provide the correlations between the subscales so some consideration may be given to these findings in relation to present knowledge in the field regarding interrelations among these various areas of the personal and social functioning of adolescents. Table 6 shows the correlations between the subscales of the MAAS. When examining these correlations, it is important to recognize that in this sample

TABLE 6:

Correlation Between Multidimensional Adolescent Assessment Scale Subscales

1. Depression 2. Self-esteem 3. Problems with mother 4. Problems with father 5. Personal stress 6. Problems with friends 7. Problems with school 8. Aggression 9. Family relationship problems 10. Suicidal thoughts 11. Feelings of guilt 12. Confused thinking 13. Disturbing thoughts 14. Memory loss 15. Alcohol abuse 16. Drug use

*p < .05. **p < .01.

1

2

3

4

— .599** .447** .323** .587** .548** .022 .205**

— .402** .313** .443** .635** .032 .142*

— .232** .420** .429** .143* .256**

.460** .281** .473** .389** .467** .329** .115 .356**

.466** .302** .413** .286** .358** .177** .137 .383**

.701** .533** .357** .226** .402** .227** .307** .271** .288** .274** .197** .187** .294** -.031 .421** .133

5

6

7

— .394** — .339** .482** — .207** .030 .099 — .146* .377** .166* .075 .439** .410** .434** .568** .592** .414** .210** .337

.474** .361** .413** .273** .352** .229** .183* .463**

.147* .219** .102 .102 .193** .162* .118 .153

8

9

10

11

— .376** .434** .270** .337** .200** .129 .245**

— .387** .334** .361** .446** .358** .478**

— .477** .460** .356** .291** .404**

12

13

14

15

16

— .164* .362** .296** .442** .386** .329** .427** .439**

— .616** — .544** .419** — .283** .304** .152 — .398** .350** .338** .617** —

25

26


there was adequate variance in many of the subscale scores to prevent undue truncation of subscale intercorrelations. In other words, these correlations may reasonably accurately reflect subscale relationships between those who participated in the study. Given these observations, we believe that the correlations shown in Table 6 provide beginning, tentative evidence to assess convergent and discriminant validity of the MAAS subscale scores. Much remains to be done in the validating process of the MAAS subscale scores. Statistical Norms

When considering the use of a new assessment scale such as the MAAS, many will ask for population norms with respect to background characteristics such as age, gender, ethnicity, marital status, and other characteristics often believed to have an impact on the interpretation of scores. Currently, there are no norms for either clinical or nonclinical populations when using the MAAS. Such norms have not been pursued in this study, but we are encouraging replication of this study with clinical populations.

DISCUSSION

This study examined the reliability and validity of 16 subscales of the MAAS, related to personal and social functioning in adolescents. The initial findings indicate that 12 of the 16 subscales had reliabilities above .90, which meets or exceeds the current standard set in the social sciences. Of the additional subscales, 3 had reliability coefficients above .80. Only the subscale of depression had a reliability coefficient below .80; the use of this subscale in clinical settings should be cautious. Overall, the high reliability coefficients indicate that the MAAS has very good reliability (Nunnally & Bernstein, 1994). The findings in this study seem to corroborate various levels of validity: factorial, content, and construct validity. It should be noted that due to the relatively older adolescent age of the sample (M = 18), the study should be repeated with a younger population to determine if developmental issues associated with age and maturation may influence a person’s score on each of the subscales. Of particular concern are the school problems and depression subscales, as they had the highest number of factor failures, the lowest reliability coefficients, and the highest standard deviations. Specifically, developmental issues may be associated with the older age of the population in these two subscales, as the average age and grade level of the sample place the respondents in college; therefore, school may be a different construct than


27

college. In future studies, these subscales require additional examination to determine if they are valid and reliable measures of these constructs. Another limitation of this study is that it used a convenience-sampling strategy. Related to this is the issue of ethnicity, as this sample is predominately White. The constructs being measured in the MAAS may have cultural implications that do not address or take into account cultural diversity. Future studies with randomly drawn, diverse samples would help to address these potential differences. As noted throughout, clinical and nonclinical random samples are needed to aid in the validation process of the MAAS. The MAAS provides a reliable and valid method of measuring multiple domains of functioning. As this initial validation study has noted, there are some subscales that require additional development. Practitioners and researchers should feel confident that the scale, taken as a whole, performs as intended but should cautiously interpret the subscale scores for depression and school problems. One recommendation in using this scale with either a clinical or nonclinical population is to use the scores as guides, rather than as diagnostics. The MAAS should guide the practitioner in determining areas that require additional assessment and possible referrals. The MAAS provides similar reliability coefficients as the well-established Child Depression Inventory and the CBCL. It assesses multiple constructs and was designed to address problems in 16 areas of the social and personal functioning of client populations often seen by social workers. Furthermore, the MAAS is moderate in cost, convenient to administer, and may be either hand scored or placed in a spreadsheet for the simple computation of the scores. The MAAS provides a significant advance in tools for use by social workers.

REFERENCES Achenbach, T. M. (1991). Manual for the Child Behavior Checklist and 1991 profile. Burlington: University of Vermont, Department of Psychiatry. Beck, A. T., Ward, C. M., Mendelson, M., Mock, J. B., & Erbaugh, J. K. (1961). An inventory for measuring depression. Archives of General Psychiatry, 4, 561-571. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81-105. Carmines, E. G., & Zeller, R. A. (1979). Reliability and validity assessment. Series: Quantitative applications in the social sciences (No. 17). Beverly Hills, CA: Sage. Germain, C. B., & Gitterman, A. (1980). The life model of social work practice. New York: Columbia University Press. Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage. Hudson, W. W. (1982). The clinical measurement package: A field manual. Homewood, IL: Dorsey.

28


Hudson, W. W. (1990). The MPSI technical manual. Tallahassee, FL: WALMYR. Hudson, W. W. (1995). The Brief Adult Assessment Scale. Tallahassee, FL: WALMYR. Hudson, W. W., Mathiesen, S. G., & Lewis, S. J. (2000). Personal and social functioning: A pilot study. Social Service Review, 74, 76-102. Hudson, W. W., & McMurtry, S. (1997). Comprehensive assessment in social work practice: The Multi-Problem Screening Inventory. Research on Social Work Practice, 7(1), 79-98. Karls, J. M., & Wandrei, K. E. (1995). Person-in-environment. In Encyclopedia of social work (Vol. 3, pp. 1818-1827). Washington, DC: National Association of Social Workers. Kemp, S. P., Whittaker, J. K., & Tracy, E. M. (1997). Person-environment practice: The social ecology of interpersonal helping. New York: Aldine de Gruyter. Kovacs, M. (1985). The Children’s Depression Inventory (CDI). Psychopharmacology Bulletin, 21, 995-998. Nelson, W. M., & Politano, P. M. (1990). Children’s Depression Inventory: Stability over repeated administrations in psychiatric inpatient children. Journal of Clinical Child Psychology, 19(3), 254-256. Nunnally, J. C. (1978). Psychometric theory. New York: McGraw-Hill. Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill. Saylor, C. F., Finch, A. J., Spirito, A., & Bennett, B. (1984). The Children’s Depression Inventory: A systematic evaluation of psychometric properties. Journal of Consulting and Clinical Psychology, 52, 955-967. Stevens, S. S. (1968). Ratio scales of opinion. In D. K. Whitla (Ed.), Handbook of measurement and assessment in behavioral science (pp. 171-199). Reading, MA: Addison-Wesley.