Journal of Emotional and Behavioral Disorders - Gevirtz Graduate ...

Journal of Emotional and Behavioral Disorders http://ebx.sagepub.com/

Using Surveillance of Mental Health to Increase Understanding of Youth Involvement in High-Risk Behaviors : A Value-Added Analysis Erin Dowdy, Michael J. Furlong and Jill D. Sharkey Journal of Emotional and Behavioral Disorders 2013 21: 33 originally published online 16 April 2012 DOI: 10.1177/1063426611416817 The online version of this article can be found at: http://ebx.sagepub.com/content/21/1/33 Published by: Hammill Institute on Disabilities

and http://www.sagepublications.com

Additional services and information for Journal of Emotional and Behavioral Disorders can be found at: Email Alerts: http://ebx.sagepub.com/cgi/alerts Subscriptions: http://ebx.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav

>> Version of Record - Feb 6, 2013 OnlineFirst Version of Record - Apr 16, 2012 What is This?

Downloaded from ebx.sagepub.com at UNIV CALIFORNIA SANTA BARBARA on April 9, 2013

416817 dy et al.Journal of Emotional and Behavioral Disorders © Hammill Institute on Disabilities 2010

EBXXXX10.1177/1063426611416817Dow

Reprints and permission: http://www. sagepub.com/journalsPermissions.nav

Using Surveillance of Mental Health to Increase Understanding of Youth Involvement in High-Risk Behaviors: A Value-Added Analysis

Journal of Emotional and Behavioral Disorders 21(1) 33–44 © Hammill Institute on Disabilities 2012 Reprints and permission: http://www. sagepub.com/journalsPermissions.nav DOI: 10.1177/1063426611416817 http://jebd.sagepub.com

Erin Dowdy1, Michael J. Furlong1, and Jill D. Sharkey1

Abstract This study examined the potential utility of adding items that assessed youths’ emotional and behavioral disorders to a commonly used surveillance survey. The goal was to evaluate whether the added items could enhance understanding of youths’ involvement in high-risk behaviors. A sample of 3,331 adolescents in Grades 8, 10, and 12 from four California school districts were coadministered a mental health screener and a youth surveillance survey. Items from both tools assessing chronic sadness and elevated mental health risk were significantly associated with increased risk of suicide ideation, cigarette use, alcohol use, binge drinking, marijuana use, physical fighting, being threatened or injured with a weapon, and skipping school. However, the addition of mental health content to the surveillance survey increased precision of understanding which youths were at the greatest odds of engaging in risk behaviors. Implications for practice and policy are discussed. Keywords mental health, assessment, screening, school psychology, high-risk behaviors, epidemiology, California Healthy Kids Survey

Introduction

understanding of the complexity of risk behaviors that manifest during adolescence.

Epidemiological studies have found that mental health disorders are prevalent, with estimates suggesting that half of all individuals in the United States will have a mental health disorder sometime during their lifetime (Kessler, Berglund, Demler, et al., 2005). Although the onset of mental health problems often occurs during childhood or adolescence (Kessler, Berglund, Demler, et al., 2005), long delays between the emergence of psychological symptoms and treatment are common (Kuehn, 2005), and a majority of youth are unidentified and underserved (Jamieson & Romer, 2005). Youth who experience mental health disorders, particularly when not identified and treated early, experience severely compromised life functioning (Kuehn, 2005). Despite an understanding of this public health crisis (Campaign for Mental Health Reform, 2005), few indicators of adolescents’ emotional and behavioral disorders are included in current strategies to monitor youth risk, including national and state surveillance programs (Freeman et al., 2010). This study explored the potential value of adding items that assess youth psychological health indicators to a surveillance survey. The aim was to investigate whether these additional psychological health items could enhance

Co-occurrence of EmotionalBehavioral Disorders and Risk Behaviors Mental health disorders often do not occur in isolation. For example, one study found that more than 80% of youth with an alcohol disorder had another psychiatric disorder and that these mental health disorders most often preceded the alcohol disorder (Rhode, Lewinsohn, & Seeley, 1996). Conversely, individuals with depression are more likely to take up the use of substances after the onset of the depressive symptoms (Kessler, Berglund, Demler, et al., 2005). Unfortunately, the interface between mental health functioning and other youth risk behaviors, such as substance use, 1

University of California–Santa Barbara, USA

Corresponding Author: Erin Dowdy, Gevirtz Graduate School of Education, University of California–Santa Barbara, CA 93106, USA Email: [email protected]


34

Journal of Emotional and Behavioral Disorders 21(1)

victimization, and involvement in aggression, is complex and has not been sufficiently investigated. Few surveillance studies have examined the co-occurrence of mental health and other risk behaviors such as substance use disorders (Jané-Llopis & Matytsina, 2006). This is potentially important because the treatments for co-occurring disorders differ from those for single-dimension disorders (E. K. Reynolds, Tull, Shalev, & Lejuez, 2010). Indeed, failure to recognize co-occurring problems can limit treatment success (Kirsh, 2010). Coadministering high-quality psychological health questions with other risk-behavior questions could provide information regarding the co-occurrence and incidence of such problems and increase awareness of the need for complex interventions. Moreover, regularly monitoring rates of emotional and behavioral disorders, risk behaviors, and their co-occurrence could allow health professionals to design more sophisticated prevention and intervention programs for youth that address multiple disorders and also monitor their success across broad populations (Dowdy, Ritchey, & Kamphaus, 2010).

Youth Surveillance Surveillance originated as a public health strategy to monitor infectious disease; it subsequently expanded to include chronic disease and, more recently, to encompass mental health disorders (Freeman et al., 2010). As health professionals recognize the interrelated effects of physical and mental health disorders on life functioning, there have been efforts to include mental health indicators in surveillance instruments. For example, the eight-item Patient Health Questionnaire depression scale was validated for use in the Behavioral Risk Factor Surveillance System (BRFSS), a population-based study conducted with adults (Kronke et al., 2009). In addition, the U.S. Department of Health and Human Services agencies are collaborating to integrate mental health and public health surveillance; a variety of ongoing psychiatric epidemiology and health surveys currently include a common nonspecific psychological distress scale (Freeman et al., 2010). This inclusion of a common approach to instrumentation across surveys facilitates comparisons and provides an opportunity for states and other interested parties to analyze data investigating the effects of co-occurring mental health problems, substance use disorders, and medical disorders (Freeman et al., 2010). Although progress has been made to include mental health content in adult surveillance instruments, youth surveillance surveys have yet to widely embrace this practice, and mental health content is not well integrated into commonly used youth surveillance systems. For example, of the U.S. surveys that contain publicly available mental health data, only two provide data for individuals of all ages, with the majority only collecting data for individuals above the age of 18 (Freeman et al., 2010). Unfortunately, this leaves

mental health surveillance largely ignored among child and adolescent populations (Romer & McIntosh, 2005), and few opportunities to explore the relationships among disorders exists. When screening for emotional and behavioral disorders occurs, screening often focuses on one specific disorder (e.g., depression) to the exclusion of others (e.g., conduct disorder), with results rarely aggregated to inform and monitor mental health needs across populations (Dowdy et al., 2010). The Youth Risk Behavior Survey (YRBS), developed by the Centers for Disease Control and Prevention (CDC), is the most widely recognized youth surveillance survey in the United States (CDC, 2010). The YRBS gathers valuable information regarding youth functioning and measures risk behaviors in six priority areas, including behaviors that contribute to the following: (a) violence and unintentional injuries, (b) tobacco use, (c) alcohol and other drug use, (d) sexual behaviors, (e) unhealthy dietary behaviors, and (f) physical inactivity. These priority areas were chosen because they “contribute to the leading causes of morbidity and mortality among youth and adults, often are established during childhood and adolescence, extend into adulthood, and are interrelated and preventable” (Eaton et al., 2010, p. 1). However, despite the interrelatedness and frequent co-occurrence of multiple youth risk factors (Freeman et al., 2010), few indicators monitoring mental health functioning are included in YRBS. Specifically, on the 2011 standard high school YRBS questionnaire, only 1 out of 86 items assesses mental health functioning—this item assesses feelings of sadness or hopelessness (CDC, 2010b). In comparison, 11 questions ask about tobacco use and 20 query alcohol and drug use.

Study Purpose There is no doubt that youth surveillance surveys provide useful information about the incidence of youth risk behaviors. However, these surveys may be underused by those interested in the emotional and behavioral health of students due to the current imbalance of information included. Nonetheless, due to the limited adolescent mental health surveillance data readily available, researchers have used YRBS and other surveillance survey items as indicators of mental health functioning. For example, information gathered from adolescents about suicidal ideation and attempts is often used as an indicator of extreme psychological distress (Kessler, Berglund, Borges, Nock, & Wang, 2005). Yet, it is unknown how questions assessing suicidal behaviors, or sadness and hopelessness, adequately measure the global mental health functioning of youth. In addition, suicide occurs outside of the context of depression (Verona, Sachs-Ericsson, & Joiner, 2004) and the link between suicide and depression is complex. For example, in a populationbased longitudinal study, individuals with depression comorbid


35

Dowdy et al. with another mental illness were found to be at an increased risk of suicide attempts, even above and beyond the risk associated with prior suicidal behaviors alone (Bolton, Pagura, Enns, Grant, & Sareen, 2010). Moreover, indicators of sadness and suicide ideation do not yield sufficient information about a host of other important emotional or behavioral problems, including anxiety, conduct disorder, or attention deficits. To date, a mental health screener has not been coadministered with a youth surveillance survey. This study contributes to the literature by examining the potential benefits of adding items to a surveillance survey to more fully assess youths’ emotional and behavioral needs. In an attempt to examine one way in which youth surveillance surveys might be enhanced, the current study (a) examined the associations between youth mental health status, chronic sadness, and other risk behaviors commonly assessed through surveillance techniques, and (b) examined the increase in variance associated with common risk behaviors by adding mental health screening items to surveillance items commonly used to assess for chronic sadness. Due to known differences in risk behaviors among males and females, gender was examined first to determine whether results should be aggregated or separated by gender.

Method Participants Participants included 3,331 students in Grades 8, 10, and 12. There were more females (51.5%) than males (48.5%) and 0.7% did not respond to the gender item. A majority of students identified as Hispanic (64.8%), followed by White (13.7%), Asian (3.4%), African American (2.1%), Pacific Islander (1.2%), Native American (1.1%), and 13.7% did not respond to the ethnic identification question. The sample included 61.8% of all 8th-grade students (n 1,226), 64.1% of all 10th-grade students (n 1,264), and 43.3% of all 12th-grade students (n 841) from four school districts serving a moderate-sized central California agricultural community.

Instruments Youth risk behaviors and experiences. The youth surveillance survey used in the study was the California Healthy Kids Survey (CHKS, Module A), which includes sections about diet and exercise, violence, perceptions of safety, harassment, bullying, and the use of alcohol and other drugs (California Department of Education, 2010; http://chks. wested.org/about). The CHKS (available in both English and Spanish) is used for surveillance purposes in California and in empirical research (Felix, Furlong, & Austin, 2009; Kim & McCarthy, 2006; Sharkey, You, & Schnoebelen,

2008; Waters & Cross, 2010). From its inception in 1998, CHKS items were matched to, or complement, those used in the YRBS. The CHKS items chosen for inclusion in this study assess mental health, substance use, aggression, and victimization, and are comparable with those in the YRBS. Various studies have provided support for the reliability of surveillance survey items (Brener, Billy, & Grady, 2003; Brener et al., 2002; Hanson & Austin, 2003). All students chose to complete the English version of the CHKS. See Table 1 for item wording for the CHKS items used in this study and their comparable YRBS items. Mental health risk. The Behavior Assessment System for Children–2 Behavioral and Emotional Screening System Student form (BESS Student) is a 30-item behavior rating screener measuring youths’ (Grades 3–12) self-reported levels of risk for behavioral and emotional or mental health problems (Kamphaus & Reynolds, 2007). The BESS Student form requires no informant training, can be completed in 5 min or less, and is available in both Spanish and English. Although both language versions were available, all students chose to complete the English version. Students report on their behavioral and emotional functioning using a 4-point response scale (“never,” “sometimes,” “often,” and “almost always”). The sum of the item raw scores is transformed to a total T-score, in which higher scores reflect more problems: scores of 20 to 60 suggest a “normal” level of risk, 61 to 70 (scores between 1 and 2 standard deviations above the mean) suggest an “elevated risk” level, and scores of 71 or higher (more than 2 standard deviations above the mean) suggest an “extremely elevated” level of risk. For the purposes of this study, students in the elevated and extremely elevated ranges were grouped together, resulting in the following dichotomized risk-level classification: normal (T-score of 20–60) or elevated mental health risk (EMHR; T-score of 61 and above). The psychometric properties of the BESS Student version are generally acceptable, having good split-half reliability (.90–.93) and test–retest reliability (.80). The manual reports classification accuracy when using the BESS Student form to predict student’s at-risk status. The form yielded moderate sensitivity (.59), high specificity (.95), moderate positive predictive value (.68), and high negative predictive value (.92). According to the BESS manual (Kamphaus & Reynolds, 2007), the BESS Student form has moderate correlations with other measures of behavioral and emotional problems: the Achenbach System of Empirically Based Assessment Youth Self-Report (rs .66–.77; Achenbach & Rescorla, 2001), Conners’s Rating Scales (rs .51–.68; Conners, 1997), Children’s Depression Inventory (r .51; Kovacs, 2001), and the Revised Children’s Manifest Anxiety Scale (r .55; C. R. Reynolds & Richmond, 2000). The BESS Student form had high internal consistency for this sample with a Cronbach’s alpha of .88.


36


Table 1. Wording of Items Included in Study Analyses and Percentage of Students Responding Affirmatively Compared With National YRBS Responses of Hispanic and White Students Study sample percentage responding affirmatively or 1 or more days/ times (Grades 8, 10, and 12) Female

Malea

Female

Maleb

During the past 12 months, did you ever feel so sad or hopeless almost everyday for 2 weeks or more in a row that you stopped doing some usual activities?

36.7

24.3

39.7

23.6

During the past 30 days, on how many days did you smoke cigarettes? Identical

13.4

14.7

16.7

19.4

39.7

28.6

43.5

42.4

Identical

23.8

19.3

23.3

25.1

During the past 30 days, how many times did you use marijuana?

17.9

19.2

18.2

25.0

During the past 12 months, how many times were you in a physical fight on school property? During the past 12 months, how many times has someone threatened or injured you with a weapon such as a gun, knife, or club on school property? Identical

16.0

27.8

9.3

17.7

5.2

10.5

6.3

12.0

11.9

12.4

8.3

7.9

Identical

16.9

11.5

20.2

10.7

California Healthy Kids Survey items

YRBS items

Mental health related During the past 12 months, did you ever feel so sad or hopeless almost everyday for 2 weeks or more that you stopped doing some usual activities?c Substance use related During the past 30 days, on how many days did you use cigarettes?d During the past 30 days, on how many days did you use at least one drink of alcohol?e During the past 30 days, on how many days did you use five or more drinks of alcohol in a row, that is, within a couple of hours?f During the past 30 days, on how many days did you use marijuana (pot, weed, grass, hash, or bud)?g Aggression and victimization related During the past 12 months, how many times on school property have you been in a physical fight?h During the past 12 months, how many times on school property have you been threatened or injured with a weapon (gun, knife, club, etc.)?i During the past 30 days, on how many days did you not go to school because you felt you would be unsafe at school, or on your way to school, or from school?j During the past 12 months, did you ever seriously consider attempting suicide?k

YRBS national Hispanic sample percentage responding affirmatively or 1 or more days/ times (Grades 9 to 12)

Note: YRBS Youth Risk Behavior Survey. YRBS national Hispanic data obtained from the Centers for Disease Control and Prevention, High School YRBS (http://apps.nccd.cdc.gov/youthonline/App/Default.aspx?SID HS). a n varies between 948 and 1,697 due to variability in response rate. b n varies between 2,037 and 2,429 due to variability in response rate. c 2 58.02 (1, N 3259), p .001. d 2 1.08 (1, N 3248), ns. e 2 44.24 (1, N 3286), p .001. f 2 9.73 (1, N 3282), p .002. g 2 0.86 (1, N 3281), ns. h 2 33.01 (1, N 3291), p .002. i 2 67.32 (1, N 3279), p .001. j 2 0.22 (1, N 3197), ns. k 2 11.85 (1, N 2014), p .001.


37

Dowdy et al.

Procedure and Data Analyses Analyses are based on cross-sectional data from the fall 2009 administration of the CHKS. All students in Grades 8, 10, and 12 from four school districts were eligible for inclusion. Students voluntarily completed the anonymous, selfadministered questionnaire in school following local parental permission procedures. The questionnaire consisted of the CHKS Module A, which contained 132 questions, and a 40-item supplement that included the selected mental health risk screener (BESS Student). Survey responses were recorded on scanable forms and processed by WestEd following standard procedures used with the CHKS since 1998. The data set was edited for inconsistencies by WestEd (see www.wested.org/hks). Missing data were not statistically computed and of the 4,055 completed questionnaires, 124 (3.1%) were excluded from analysis because they failed three or more of seven response quality control checks. Statistical analyses were conducted using SPSS (version 18). Logistic regressions were run to calculate odds ratios (ORs) and 95% confidence intervals (CIs). The items assessing aggression, victimization, substance use, and suicide ideation served as the outcome measures. The surveillance survey items assessing for feelings of sadness or hopelessness (yes/no) and mental health risk status (normal/elevated) were entered stepwise as predictors.

Results Descriptive Analyses The initial analyses explored whether males and females reported different rates of mental health symptoms, substance use, and aggression. Results of these analyses reveal that more females reported experiencing mental health symptoms and using substances than males, whereas more males reported engaging in aggressive behaviors. For example, significantly more females than males experienced chronic sadness (36.7% and 24.3%, respectively), χ2(1, N 3259) 58.02, p .001. Table 1 details all descriptive analyses. To provide a perspective on this study’s predominately Latino/Latina regional sample, Table 1 also shows the responses of the Latino/Latina youth in Grades 9 to 12 from the national YRBS 2009 sample (Youth Online: High School YRBS 2009 survey). This study’s female participants had significantly lower rates of sadness (z –2.38, p .017), cigarette use (z –2.85, p .002), alcohol use (z –2.36, p .009), and suicide contemplation (z –2.28, p .011), and higher rates of physical fighting (z 6.47, p .0001) and avoiding school because of fear (z 3.80, p .0001) than the Latinas in the national YRBS 2009 sample. The males in this study sample reported significantly

lower rates of cigarettes use (z –3.73, p .0001), alcohol use (z –8.60, p .0001), binge drinking (z –4.21, p .0001), and marijuana use (z –4.25, p .0001), and higher rates of physical fighting (z 7.26, p .0001) and avoiding school due to fear (z 4.61, p .0001) than Latinos in the YRBS 2009 sample. Both males and females reported comparable rates of suicide ideation. In sum, this study’s sample, in comparison with the YRBS national sample, showed less involvement in substance use and more involvement in aggression-related behaviors.

Mental Health Indicators and Risk Behaviors and Experiences Next, we examined the relationships between student responses to a question asking whether they had experienced sadness for a minimum of 2 weeks in the past 12 months (0 no sadness reported, 1 sadness reported) and their EMHR status, as assessed by the BESS Student, for eight risk behaviors or experiences (cigarette use, alcohol use, binge drinking, marijuana use, physical fighting, being threatened or injured with a weapon, skipping school, and suicide ideation; see Table 2). Chronic sadness and EMHR were significantly associated with the eight risk indicators— most strongly with suicide ideation and avoiding school because of fear. Youth with chronic sadness reported engaging in the risk indicators at a rate 1.3 to 1.9 times more than youth without chronic sadness, with the exception of suicide ideation, which was 2.9 times higher for youth with chronic sadness. The association between EMHR and the risk indicators was even stronger. Youth with EMHR reported engaging in the risk indicators at a rate 2.0 to 2.9 times higher than youth without EMHR, with the exception of alcohol use, which was 1.8 times higher. These results show that both chronic sadness and EMHR are sensitive to rates of involvement in high-risk behavior and experiences but that EMHR has a stronger association with risk behaviors. The individual and combined effects of chronic sadness and EMHR are examined in the next sections.

Logistic Regression Stepwise logistic regression was performed to assess the added predictive association of EMHR status (0 no risk and 1 elevated risk) with risk behaviors after entering student responses to chronic sadness. As shown in Table 3, chronic sadness was significantly associated with the eight risk behaviors for both females and males. After accounting for the variance attributable to chronic sadness (Step 1), the change in variance explained by EMHR status was significant (p .001) for the eight risk behaviors for both males and females.


38


Table 2. Percentage of Youth in Grades 8, 10, and 12 Reporting Participation in Risk-Related Behaviors or Experiences by Chronic Sadness and Elevated Mental Health Risk Chronic sadness (n Risk behaviors and experiences Cigarette use past 30 days Alcohol use past 30 days Binge drinking past 30 days Marijuana use past 30 days Physical fighting in school past year (1 or more times) Threatened or injured with weapon in school past year (1 or more times) Skipped school due to fear past 30 days (1 or more days) Seriously considered suicide past year

Elevated mental health risk (n 3,304b)

3,267a)

No (%)

Yes (%)

2

No (%)

Yes (%)

2

28.8 27.0 28.9 29.1 27.2 28.6

42.6 37.6 37.1 37.6 40.5 55.1

34.55*** 39.01*** 17.39*** 16.42*** 41.37*** 75.54***

17.4 16.3 16.9 17.3 16.6 18.2

39.4 28.6 33.9 35.2 34.2 49.0

116.08*** 69.72*** 98.88*** 98.52*** 106.90*** 138.15***

28.8

44.9

40.64***

17.4

45.0

158.73***

23.3

66.8

225.76***

16.6

47.6

144.73***

Note: Chronic sadness was measured by self-report of having 2 or more weeks of sadness at least once during the preceding year. Elevated mental health risk was a T-score of 61 or higher on the Behavioral and Emotional Screening System. a n varies between 2,022 and 3,267 due to variability in response rate. b n varies between 2,024 and 3,304 due to variability in response rate. ***p .001.

When EMHR status was entered (Step 2, see Table 3), chronic sadness was no longer associated with binge drinking for females (Wald 0.82, p .365) and males (Wald 1.75, p .186). In addition, for males only, chronic sadness was no longer associated with past 30-day cigarette (Wald 2.21, p .137) or marijuana (Wald 0.51, p .476) use. With the exception of suicide ideation, EMHR status was the strongest predictor for seven of the eight risk behaviors with ORs of 1.92 to 3.28 (alcohol use and threatened, respectively, small to medium effect sizes) for females and 1.79 to 3.79 (alcohol use and threatened, respectively, small to medium effect sizes) for males. Chronic sadness and EMHR status were strongly associated with student-reported suicide ideation. The odds of females with chronic sadness also reporting having suicide ideation were 4.4 times greater than the odds of those who did not have chronic sadness, a moderate effect size. For females with EMHR, after accounting for the effects of chronic sadness, their odds of reporting suicide ideation was 2.77 times greater than the odds of youth without EMHR, a small to moderate effect size. For males, an even stronger association was found with those reporting chronic sadness as shown by their being 7.46 times more likely to report having suicide ideation than their peers who did not experience chronic sadness (medium to large effect size). For males, the OR for EMHR status, after accounting for the effects of chronic sadness, was 4.33 (medium effect size).

Combinatorial Relationships of Chronic Sadness and EMHR The logistic regression showed that the combination of chronic sadness and EMHR increased precision of

understanding which youths were at the greatest odds of engaging in risk behaviors or experiences. To further explore this result, we identified youth who reported experiencing neither chronic sadness nor EMHR, either chronic sadness or EMHR, and finally the subgroup of youths who reported both chronic sadness and EMHR. Table 4 shows significant differences in the rates with which these three groups participated in the eight risk indictors. For all risk indicators, the youth who reported chronic sadness and EMHR had the highest absolute rates compared with youth without chronic sadness or EMHR—the most pronounced differences were for being threatened with a weapon (6.1:1) and suicide ideation (24.5:1).

Discussion To help contextualize the experiences of youth and provide a more complete and accurate understanding of the challenges and risks that students experience, additional surveillance information on youth is needed. Results demonstrated that the youth surveillance item assessing chronic sadness was significantly associated with a variety of risk behaviors, including substance use, aggression, and victimization. The results support the value of the chronic sadness YRBS item because of its significant and moderate association with other risk behaviors. In addition, after accounting for the variance attributable to this chronic sadness item, the change in variance explained by EMHR status was significant for the eight risk behaviors examined, which highlighted the value of including comprehensive mental health information in a surveillance survey. The inclusion of items assessing both sadness and additional mental health content dramatically increased the precision of understanding which



39

Cigarette use 30 days Step 1: ChrSad Step 2: ChrSad Step 2: EMHR Constant Cox Snell R2 Nagelkerke R2 (Step 2 – Step1) Alcohol use past 30 days Step 1: ChrSad Step 2: ChrSad Step 2: EMHR Constant Cox Snell R2 – Nagelkerke R2 (Step 2 – Step1) Binge drinking past 30 days Step 1: ChrSad Step 2: ChrSad Step 2: EMHR Constant Cox Snell R2 – Nagelkerke R2 (Step 2 – Step1) Marijuana use past 30 days Step 1: ChrSad Step 2: ChrSad Step 2: EMHR Constant Cox Snell R2 – Nagelkerke R2 (Step 2 – Step1) Physical fighting school past year Step 1: ChrSad Step 2: ChrSad Step 2: EMHR Constant Cox Snell R2 – Nagelkerke R2 (Step 2 – Step1) Threatened or injured weapon school past year Step 1: ChrSad Step 2: ChrSad Step 2: EMHR Constant

Risk behaviors and experiences

OR [95% CI]

.23 .24 .24 .20

1.15 0.86 1.19 –3.76

3.15 [2.00, 4.97] 2.35 [1.46, 3.78] 3.28 [2.06, 5.21]

25.18 1.97 [1.55, 2.57] 11.08 1.60 [1.21, 2.11] 38.44 2.51 [1.88, 3.36] 448.55 (.021 – .037)i

.14 .14 .15 .10

0.68 0.47 0.92 –2.12

24.47 12.43 25.25 358.38

12.87 4.05 33.11 411.37

0.47 .13 0.27 .14 0.83 .15 1.86 .09 (.018 – .030)g

1.59 [1.24, 2.05] 1.32 [1.01, 1.72] 2.30 [1.73, 3.06]

6.65 1.36 [1.08, 1.71] 0.82 1.12 [0.88, 1.43] 38.76 2.30 [1.77, 3.00] 318.72 (.022 – .033)e

.12 .12 .13 .08

.030 0.11 0.84 1.42

1.58 [1.29, 1.94] 1.38 [1.12, 1.71] 1.92 [1.50, 2.45]

27.67 2.16 [1.62, 2.88] 11.39 1.44 [1.24, 2.27] 36.71 2.95 [2.16, 4.03] 461.42 (.026 – .048)a

Wald

19.69 9.05 27.53 107.19

.15 .15 .08 .11

SE B

0.46 .10 0.32 .11 0.65 .12 .70 .07 (.016 – .022)c

0.77 0.52 1.08 2.42

B

Female

.000 .000 .000 .000

.000 .001 .000 .000

.000 .044 .000 .000

.010 .365 .000 .000

.000 .003 .000 .000

.000 .001 .000 .000

p .16 .17 .16 .10

SE B

.14 .15 .15 .08

1.33 1.05 1.26 –2.91

0.68 0.52 0.79 –1.28

.17 .18 .18 .13

.13 .13 .14 .07

0.36 .14 0.11 .15 1.03 .15 –1.74 .08 (.028 – .045)h

0.41 0.20 0.92 –1.72

0.43 .13 0.31 .13 0.58 .14 –1.12 .07 (.011 – .015)d

0.48 0.25 –1.01 –2.10

B

OR [95% CI]

1.54 [1.20, 1.97] 1.36 [1.05, 1.75] 1.79 [1.36, 2.35]

1.43 [1.08, 1.89] 1.11 [0.83, 1.50] 2.80 [2.08, 3.77]

60.16 33.93 47.50 501.93

3.77 [2.69, 5.26] 2.84 [2.00, 4.04] 3.52 [2.46, 5.04]

29.75 1.98 [1.55, 2.53] 15.76 1.68 [1.30, 2.17] 32.27 2.21 [1.68, 2.91] 310.49 (.020 – .029)j

6.13 0.51 46.46 427.47

8.36 1.51 [1.14, 1.99] 1.75 1.22 [0.91, 1.63] 37.08 2.51 [1.87, 3.37] 422.94 (.016 – .036)f

11.79 5.48 17.13 254.45

9.43 1.62 [1.19, 2.20] 2.21 1.28 [0.93, 1.77] 37.84 2.75 [1.99, 3.80] 476.63 (.022 – .040)b

Wald

Male

(continued)

.000 .000 .000 .000

.000 .000 .000 .000

.013 .476 .000 .000

.004 .186 .000 .000

.001 .019 .000 .000

.002 .137 .000 .000

p

Table 3. Summary of Final Stepwise Logistic Regression Analysis Model for Predicting Eight Risk Behaviors for Females and Males in Grades 8, 10, and 12 With Feelings of Sadness for 2 Weeks in the Past Year (ChrSad: 0 No Sadness, 1 Sadness Reported) Entered at Step 1 and EMHR Status (EMHR: 0 No Risk, 1 Elevated Risk) Entered at Step 2

40


2

.16 .16 .17 .12

.18 .18 .19 .15

1.65 1.48 1.02 –2.60

SE B

0.73 0.46 1.11 –2.55

B

OR [95% CI]

88.03 5.23 [3.70, 7.39] 66.98 4.40 [3.08, 6.27] 30.53 2.77 [1.93, 3.98] 309.43 (.025 – .042)o

22.07 2.07 [1.53, 2.81] 7.88 1.58 [1.15, 2.18] 44.23 3.04 [2.19, 4.22] 452.87 (.026 – .049)m

k

(.015 – .044)

Wald

Female

.000 .000 .000 .000

.000 .005 .000 .000

p

2.21 2.01 1.47 –3.33

0.72 0.38 1.36 –2.49

B

.22 .23 .23 .20

.17 .18 .17 .11

SE B

97.41 9.91 [5.87, 14.10] 75.46 7.46 [4.74, 11.75] 39.45 4.33 [2.74, 6.84] 291.17 (.035 – .071)p

18.63 2.04 [1.48, 2.83] 4.71 1.47 [1.04, 2.07] 62.32 3.91 [2.79, 5.49] 492.81 (.037 – .056)n

l

OR [95% CI]

(.027 – .056)

Wald

Male

.000 .000 .000 .000

.000 .030 .000 .000

p

Note: EMHR Elevated Mental Health Risk; OR odds ratio; CI confidence interval; ChrSad Chronic Sadness. ChrSad was measured by self-report of having 2 or more weeks of sadness at least once during the preceding year. EMHR was a T-score of 61 or higher on the Behavioral and Emotional Screening System. a 2 Step 2 model fit (2, N 1646) 76.26, p .001. b 2 Step 2 model fit (2, N 1605) 44.57, p .001. c 2 Step 2 model fit (2, N 1667) 47.23, p .001. d 2 Step 2 model fit (2, N 1565) 28.26, p .001. e 2 Step 2 model fit (2, N 1663) 44.19, p .001. f 2 Step 2 model fit (2, N 1566) 43.36, p .001. g 2 Step 2 model fit (2, N 1661) 44.27, p .001. h 2 Step 2 model fit (2, N 1566) 50.15, p .001. i 2 Step 2 model fit (2, N 1668) 61.76, p .001. j 2 Step 2 model fit (2, N 1571) 60.38, p .001. k 2 Step 2 model fit (2, N 1657) 50.11, p .001. l 2 Step 2 model fit (2, N 1571) 102.88, p .001. m 2 Step 2 model fit (2, N 1617) 64.27, p .001. n 2 Step 2 model fit (2, N 1528) 76.48, p .001. o 2 Step 2 model fit (2, N 1528) 76.48, p .001. p 2 Step 2 model fit (2, N 943) 142.56, p .001.

Cox Snell R – Nagelkerke R (Step 2 – Step1) Skipped school due to fear past 30 days Step 1: ChrSad Step 2: ChrSad Step 2: EMHR Constant Cox Snell R2 – Nagelkerke R2 (Step 2 – Step1) Seriously considered suicide past year Step 1: ChrSad Step 2: ChrSad Step 2: EMHR Constant Cox Snell R2 – Nagelkerke R2 (Step 2 – Step1)

2

Risk behaviors and experiences

Table 3. (continued)

41

Dowdy et al. Table 4. Percentage of Youth in Grades 8, 10, and 12 Reporting Participation in Risk-Related Behaviors or Experiences by Neither, Either, or Both Chronic Sadness and EMHR Risk behaviors and experiences 1. Cigarette use past 30 days 2. Alcohol use past 30 days 3. Binge drinking past 30 days 4. Marijuana use past 30 days 5. Physical fighting in school past year (1 or more times) 6. Threatened or injured with weapon in school past year (1 or more times) 7. Skipped school due to fear past 30 days (1 or more days) 8. Seriously considered suicide past year

No ChrSad and no EMHR (%)

Either ChrSad or EMHR (%)

Both ChrSad and EMHR (%)

9.6 28.8 17.4 14.4 16.7 3.6

16.6 39.1 23.7 21.1 24.2 10.4

29.9 51.3 37.7 32.8 40.8 22.0

109.98*** 80.62*** 77.09*** 74.29*** 107.92*** 158.96***

2 (3,214) 2 (3,250) 2 (3,247) 2 (3,245) 2 (3,257) 2 (3,245)

7.0

16.5

26.3

131.18***

2 (3,162)

5.0

21.2

49.0

310.46***

2 (2,015)

2

df

Note: EMHR Elevated Mental Health Risk; ChrSad Chronic Sadness. ChrSad was measured by self-report of having 2 or more weeks of sadness at least once during the preceding year. EMHR was a T-score of 61 or higher on the Behavioral and Emotional Screening System. ***p .001.

youths were most vulnerable to engaging in other risk behaviors. In addition, both chronic sadness and EMHR were strongly associated with suicide ideation. Although it might not be practical to include a comprehensive 30-item mental health screener in all surveillance activities, the results of this study suggest that a more balanced approach to surveillance assessment may be warranted. For example, the YRBS has only one item that is specifically and solely designed to assess mental health, and this focuses only on depression-related experiences. The mental health screener used to assess EMHR in this study includes not only depression content but also other internalizing (e.g., anxiety and self-esteem) and externalizing (e.g., conduct disorder and attention deficit) experiences. Furthermore, it has content that focuses on positive adaptation to school and the community. The results of this study suggest that a broader focus contributed uniquely to the understanding of which youths were at increased risk for involvement in substance use and aggressive behaviors, and most strongly in suicide ideation. This study also provides insight into areas for future research. The causal patterns between many mental health and other youth risk behaviors are unknown. Early intervention for EMHR may prevent later involvement in substance abuse and violence or vice versa. Some evidence suggests that anxiety and disruptive behavior disorders are more likely (87.5%, 80.0%) and that depression (58%) is as likely to precede alcohol disorders (Rhode et al., 1996). In addition, individuals with aggression, hyperactivity, and inattention in childhood have been found to be at increased risk for substance use in adolescence (Jester et al., 2008), suggesting that ameliorating symptoms of aggression or hyperactivity in childhood may decrease the likelihood of substance use in adolescence. However, additional studies

using longitudinal analysis to determine the development of mental health and other risk behaviors would be beneficial. The likelihood that mental health disorders precede other challenges adds incentive to including mental health screening information in universal assessment strategies for children and youth to catch emerging challenges as they develop.

Study Limitations A primary limitation of this study is that a regional sample was used, thus, the specific results do not generalize to national samples and need to be replicated. However, the intent was not to identify population patterns but to explore the potential usefulness of including mental health functioning content in surveillance surveys. The key question was whether or not more detailed mental health information could enhance understanding of which youths engage in the risk behaviors that the YRBS and other surveillance surveys aim to monitor. This sample provided an ample first test of this issue. Another limitation is that this study used a commercially available, but well-designed, mental health screener that includes 30 items. An instrument of such length will not commonly be amended to an existing publicly accessible youth surveillance survey due to administration constraints and cost. Although not specifically examined in this study, there is a need for future research to explore which type of items that assess the emotional and behavioral needs of youth might provide useful information in an efficient manner. For example, in this study, we found that the single YRBS item that asked youths about experiencing a 2-week period of sadness during the previous year explained unique variance associated with the self-reporting of engaging in a range of high-risk behaviors. Future research should explore


42


whether similar items (or brief scales) can be developed that assess other internalizing (e.g., anxiety) or externalizing (e.g., attention deficits) experiences. Such an effort could draw on other respected publicly accessible assessments, such as the Strengths and Difficulties Questionnaire (Goodman, Ford, Simmons, Gatward, & Meltzer, 2000), the Ohio Scales (Ogles, Melendez, Davis, & Lunnen, 2001), and the Positive and Negative Affect Scales (Watson, Clark, & Tellegen, 1988).

Implications The results of this study have implications for youth with emotional and behavioral disorders at the local school/ community level where services are planned and implemented. Similarly, implications are present at the broader state and national levels where policies are created that affect resource allocation and initiative development. At the local level, the addition of mental health content to surveillance surveys could increase awareness among community and school officials of the need to expand their understanding of the complex challenges facing their youth, particularly those with emotional and behavioral disorders. If health professionals and educators only, or mostly, ask questions about substance use and aggression, then the solutions may become primarily drug prevention and violence prevention programs. With current youth surveillance techniques largely dependent on negative developmental and behavioral indicators, such as alcohol and substance use, interventions may be necessarily tailored to these areas; that is, “what gets measured gets done” (Knopf, Park, Brindis, Mulye, & Irwin, 2007, p. 335). Awareness of these complex needs at the local level could encourage support for multimodal, wraparound programs that simultaneously address the mental health needs of youth (Randall & Vernberg, 2008). At the national and state level, the exclusion of psychological health and well-being content in surveillance surveys reinforces a silo approach to setting public policies related to addressing the needs of youth who engage in high-risk behaviors. Examples of this effect can be seen in how the issue of youth risk behaviors is managed in national reports issued in the United States. Neither the Indicators of School Crime and Safety (Robers, Zhang, & Truman, 2010) nor the Monitoring the Future (Johnston, O’Malley, Bachman, & Schulenberg, 2011) studies include any analysis of the relationships between emotional and behavioral disorders and substance use, aggressive behaviors, and victimization. Another example of how current youth surveillance survey practices limit awareness of the needs of youth with emotional and behavioral disorders comes from the Safe School/Healthy Students (SSHS) initiative (Sharkey et al., in press). This multiagency-funded SSHS initiative requires all grantees to report annually on several Government

Performance Reporting Act indicators. The national evaluation for the SSHS initiative uses items taken from the YRBS for violent behavior (e.g., physical fighting on school campus) and substance use (e.g., use of marijuana in the past 30 days), to address SSHS goals related to reducing violence and substance use. However, the YRBS, as noted previously, has limited mental health content. Subsequently, the SSHS goal associated with children’s mental health tracks only the number of referrals made for mental health services and the percentage of referrals that resulted in services being offered (Sharkey et al., in press). The SSHS initiative does not track information about presenting problems and needs, type of services provided, or any outcome measures. This is due, in part, to the lack of easily accessible information included in routinely administered youth surveillance surveys. This observation is not used to claim that current surveillance reports do not provide valuable information but only that surveillance instruments could be enhanced by including items that comprehensively assess youths’ emotional and behavioral health needs. Results highlight how the addition of mental health content could provide more precise information about the complexity of co-occurring disorders that youth are experiencing. This information can be used at the national level to inform policy changes and also at the local community level to provide information on how to target resources for maximum benefit.

Conclusion Overall, this study provides a step in the direction of considering additional mental health content in surveillance of youth behaviors. Youth with mental health problems are not being adequately served (Jamieson & Romer, 2005). Through an increased understanding of the problems facing youth, policies and practices can and should be modified to provide enhanced supports and services to identify and serve youth before mental health problems grow into major challenges that cost youth, their families, and their communities tremendous emotional and financial energy (Dickstein, 2009; Insel, 2008). Declaration of Conflicting Interests The author(s) declared no potential conflicts of interests with respect to the authorship and/or publication of this article.

Funding The author(s) received no financial support for the research and/or authorship of this article.

References Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA school-age forms and profiles. Burlington: University of Vermont, Research Center for Children, Youth, and Families.


43

Dowdy et al. Bolton, J. M., Pagura, J., Enns, M. W., Grant, B., & Sareen, J. (2010). A population-based longitudinal study of risk factors for suicide attempts in major depressive disorder. Journal of Psychiatric Research, 44, 817–826. doi:10.1016/j.jpsychires.2010.01.003 Brener, N. D., Billy, J. O. G., & Grady, W. R. (2003). Assessment of factors affecting the validity of self-reported health-risk behavior among adolescents: Evidence from the scientific literature. Journal of Adolescent Health, 33, 436–457. doi:10.1016/ S1054-139X(03)00052-1 Brener, N. D., Kann, L., McManus, T., Kinchen, S. A., Sundberg, E. C., & Ross, J. G. (2002). Reliability of the 1999 Youth Risk Behavior Survey questionnaire. Journal of Adolescent Health, 31, 336– 342. doi:10.1016/S1054–139X(02)00339–7 California Department of Education. (2010). California Healthy Kids Survey. Available from http://chks.wested.org Campaign for Mental Health Reform. (2005). A public health crisis: Children and adolescents with mental disorders. Congressional briefing. Retrieved from http://www.nami .org/Template.cfm?Section=May4&Template=/Content Management/ContentDisplay.cfm&ContentID=23007 Centers for Disease Control and Prevention. (2010). Youth Risk Behavior Survey (YRBS) Standard 2011 high school questionnaire item rationale. Retrieved from http://www.cdc.gov/ HealthyYouth/yrbs/questionnaire_rationale.htm Conners, C. K. (1997). Conners’ Rating Scales–Revised. North Tonawanda, NY: Multi-Health Systems. Dickstein, D. P. (2009). The costs of mental illness. Journal of the American Academy of Child & Adolescent Psychiatry, 48, 459–460. doi:10.1097/CHI.0b013e31819cb036 Dowdy, E., Ritchey, K., & Kamphaus, R. W. (2010). School-based screening: A population-based approach to inform and monitor children’s mental health needs. School Mental Health, 2, 166–176. doi:10.1007/s12310-010-9036–3 Eaton, D. K., Kann, L., Kinchen, S., Shanklin, S., Ross, J., Hawkins, J., . . . Wechsler, H. (2010). Youth Risk Behavior Surveillance—United States, 2009. Morbidity and Mortality Weekly Report, 59, 1–142. Felix, E. D., Furlong, M. J., & Austin, G. (2009). A cluster analytic investigation of school violence victimization among diverse students. Journal of Interpersonal Violence, 24, 1673–1695. doi:10.1177/0886260509331507 Freeman, E. J., Colpe, L. J., Strine, T. W., Dhingra, S., McGuire, L. C., Elam-Evans, L. D., . . . Croft, J. (2010). Public health surveillance for mental health. Preventing chronic disease, 7. Retrieved from http://www.cdc.gov/pcd/issues/2010/jan/09_0126.htm Goodman, R., Ford, T., Simmons, H., Gatward, R., & Meltzer, H. (2000). Using the Strengths and Difficulties Questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. British Journal of Psychiatry, 177, 534–539. Hanson, T. L., & Austin, G. (2003). Student health risks, resilience, and academic performance in California: Year 2 report, longitudinal analyses. Los Alamitos, CA: WestEd. Insel, T. R. (2008). Assessing the economic cost of serious mental illness. American Journal of Psychiatry, 165, 663–665. doi:10.1176/appi.ajp.2008.08030366

Jamieson, K. H., & Romer, D. (2005). A call to action on adolescent mental health. In D. L. Evans, E. B. Foa, E. Gur, H. Hendin, C. P. O’Brien, M. E. P. Seligman, & B. T. Walsh (Eds.), Treating and preventing adolescent mental health disorders: What we know and what we don’t know (pp. 598–615). New York, NY: Oxford University Press. doi:10.1093/9780195173642.003.0033 Jané-Llopis, E., & Matytsina, I. (2006). Mental health and alcohol, drugs, and tobacco: A review of the comorbidity between mental disorders and the use of alcohol, tobacco, and illicit drugs. Drug and Alcohol Review, 25, 515–536. doi:10.1080/09595230600944461 Jester, J. M., Nigg, J. T., Buu, A., Puttler, L. I., Glass, J. M., Heitzeg, M. M., & Zucker, R. A. (2008). Trajectories of childhood aggression and inattention/hyperactivity: Differential effects on substance abuse in adolescence. Journal of the American Academy of Child & Adolescent Psychiatry, 47, 1158–1165. doi:10.1097/CHI.0b013e3181825a4e Johnston, L. D., O’Malley, P. M., Bachman, J. G., & Schulenberg, J. E. (2011). Monitoring the future national results on adolescent drug use: Overview of key findings, 2010. Ann Arbor: Institute for Social Research, The University of Michigan. Kamphaus, R. W., & Reynolds, C. R. (2007). Behavior Assessment System for Children–Second Edition (BASC-2): Behavioral and Emotional Screening System (BESS). Bloomington, MN: Pearson. Kessler, R. C., Berglund, P., Borges, G., Nock, M., & Wang, P. S. (2005). Trends in suicide ideation, plans, gestures, and attempts in the United States, 1990-1992 to 2001-2003. Journal of the American Medical Association, 293, 2487–2495. doi:10.1001/ jama.293.20.2487 Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., & Walters, E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the national comorbidity survey replication. Archives of General Psychiatry, 62, 593–602. doi:10.1001/archpsyc.62.6.593 Kim, J., & McCarthy, W. J. (2006). School-level contextual influences on smoking and drinking among Asian and Pacific Islander adolescents. Drug and Alcohol Dependence, 84, 56–68. doi:10.1016/j.drugalcdep.2005.12.004 Kirsh, K. L. (2010). Differentiating and managing common psychiatric comorbidities seen in chronic pain patients. Journal of Pain & Palliative Care Pharmacotherapy, 24, 39–47. doi:10.3109/15360280903583123 Knopf, D. K., Park, M. J., Brindis, C. D., Mulye, T. P., & Irwin, C. E., Jr. (2007). What gets measured gets done: Assessing data availability for adolescent populations. Maternal and Child Health Journal, 11, 335–345. doi:10.1007/s10995–007–0179–2 Kovacs, M. (2001). Children’s Depression Inventory. North Tonawanda, NY: Multi-Health Systems. Kronke, K., Strine, T. W., Spitzer, R. L., Williams, J. B. W., Berry, J. T., & Mokdad, A. H. (2009). The PHQ-8 as a measure of current depression in the general population. Journal of Affective Disorders, 114, 163–173. doi:10.1016/j.jad.2008.06.026 Kuehn, B. M. (2005). Mental illness takes heavy toll on youth. Journal of the American Medical Association, 294, 293–295. doi:10.1001/jama.294.3.2


44


Ogles, B., Melendez, G., Davis, D., & Lunnen, K. (2001). The Ohio Scales: Practical outcome assessment. Journal of Child and Family Studies, 10, 199–212. doi:10.1023/A:1016651508801 Randall, C., & Vernberg, E. M. (2008). Evidence-based treatment for children with serious emotional disturbance. In R. G. Steele, T. D. Elkin, & M. C. Roberts (Eds.), Handbook of evidencebased therapies for children and adolescents: Bridging science and practice, Issues in Clinical Child Psychology, Part II (pp. 389–408). New York, NY: Springer. doi:10.1007/978–0– 387–73691–4_22 Reynolds, C. R., & Richmond, B. O. (2000). Revised Children’s Manifest Anxiety Scale. Los Angeles, CA: Western Psychological Services. Reynolds, E. K., Tull, M. T., Shalev, I., & Lejuez, C. W. (2010). Resolving treatment complications associated with comorbid anxiety and substance use disorders. In M. W. Otto & S. G. Hofmann (Eds.), Avoiding treatment failures in the anxiety disorders: Series in anxiety and related disorders (pp. 271–290). New York, NY: Springer. doi:10.1007/978–1–4419–0612–0_15 Rhode, P., Lewinsohn, P. M., & Seeley, J. R. (1996). Psychiatric comorbidity with problematic alcohol use in high school students. Journal of the American Academy of Child & Adolescent Psychiatry, 35, 101–109. doi:10.1097/00004583– 199601000–00018 Robers, S., Zhang, J., & Truman, J. (2010). Indicators of school crime and safety: 2010 (NCES 2011-002/NCJ 230812). Washington, DC: National Center for Education Statistics, U.S. Department of Education, and Bureau of Justice Statistics, Office of Justice Programs, U.S. Department of Justice.

Romer, D., & McIntosh, M. (2005). The roles and perspectives of school mental health professionals in promoting adolescent mental health. In D. L. Evans, E. B. Foa, R. E. Gur, H. Hendin, C. P. O’Brien, M. E. P. Seligman, & B. T. Walsh (Eds.), Treating and preventing adolescent mental health disorders: What we know and what we don’t know (pp. 598–615). New York, NY: Oxford University Press. doi:10.1093/9780195173642.0 03.0032 Sharkey, J. D., Furlong, M. J., Dowdy, E., Felix, E. D., Grimm, L., & Ritchey, K. (2011). Safe schools/healthy students initiative: turning a national initiative into local action. In S. R. Jimerson, A. B. Nickerson, M. J. Mayer, & M. J. Furlong (Eds.), The handbook of school violence and school safety: International research and practice. New York: Routldege. Sharkey, J. D., You, S., & Schnoebelen, K. (2008). Relations among school assets, individual resilience, and student engagement for youth grouped by level of family functioning. Psychology in the Schools, 45, 402–418. doi:10.1002/pits.20305 Verona, E., Sachs-Ericsson, N., & Joiner, T. E. (2004). Suicide attempts associated with externalizing psychopathology in an epidemiological sample. American Journal of Psychiatry, 161, 444–451. doi:10.1176/appi.ajp.161.3.444 Waters, S., & Cross, D. (2010). Measuring students’ connectedness to school, teachers, and family: Validation of three scales. School Psychology Quarterly, 25, 164–177. doi:10.1037/a0020942 Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS Scales. Journal of Personality and Social Psychology, 47, 1063–1070.
