Predicting Utilization of Evidence-Based Parenting ... - Springer Link

2 downloads 0 Views 205KB Size Report
Feb 13, 2009 - with Organizational, Service-Provider and Client Variables. Matthew R. Sanders Æ Ronald J. Prinz Æ. Cheri J. Shapiro. Received: 7 April 2008 ...
Adm Policy Ment Health (2009) 36:133–143 DOI 10.1007/s10488-009-0205-3

Predicting Utilization of Evidence-Based Parenting Interventions with Organizational, Service-Provider and Client Variables Matthew R. Sanders Æ Ronald J. Prinz Æ Cheri J. Shapiro

Received: 7 April 2008 / Accepted: 12 January 2009 / Published online: 13 February 2009 Ó Springer Science+Business Media, LLC 2009

Abstract Multidisciplinary service providers (N = 611) who underwent training in the Triple P-Positive Parenting Program participated in a structured interview 6 months following training to determine their level of post-training program use and to identify any facilitators and barriers to program use. Findings revealed that practitioners who had received training in Group Triple P, received positive client feedback, had experienced only minor barriers to implementation, and had consulted with other Triple P practitioners following training were more likely to become high users of the program. Practitioners were less likely to use the program when they had lower levels of confidence in delivering Triple P and in consulting with parents in general, had difficulties in incorporating Triple P into their work, and where there was low workplace support. These findings highlight the importance of considering the broader post training work environment of service providers as a determinant of subsequent program use. Keywords Parenting interventions  Dissemination  Program use  Barriers to implementation  Workplace support  Triple P

M. R. Sanders (&) Parenting and Family Support Centre, The University of Queensland, Brisbane, QLD 4072, Australia e-mail: [email protected] R. J. Prinz  C. J. Shapiro University of South Carolina, Columbia, SC, USA

Introduction The development of population based parenting programs as part of a comprehensive effort to promote better mental health outcomes in children is relatively new (Prinz and Sanders 2007; Sanders 1999; Shapiro et al. 2007). The successful dissemination of population level approaches is based on the assumption that a suitable workforce in a position to implement such programs can be trained and will subsequently implement the program with families they work with. While much has been written about dissemination of evidence based practice programs, there is little empirical data examining factors that are associated with the successful transfer of skills from a training environment to actual clinical practice (Conor-Smith and Weisz 2003; Hawley and Weisz 2002). The relative lack of research contributes to the gap between what has been proven effective in trials and what is currently used in the community; these are concerns for both the fields of prevention (Biglan and Taylor 2000; Pentz 2004) and treatment (Herscell et al. 2004; Ollendick and Davis 2004; Silverman and Kurtines 2004; Weisz et al. 2004, 2006). This concern is further amplified when evidence based programs are implemented at a population level, as a sufficient amount of parental exposure to an intervention is essential to achieving population level outcomes such as a reduction in the prevalence rates of specific problems (Sanders et al. 2007). There has been considerable theoretical speculation about how best to understand the diffusion of innovation process (e.g. Rogers 2000). The diffusion of innovation theory has remained one of the most influential models to explain the adoption of a new innovation. The model has evolved over time and includes several factors thought to mediate or moderate the adoption of new technologies

123

134

beyond the influence of empirical evidence These factors include the compatibility of the new innovation with the existing social system, the complexity of the innovation, the observability of the results of the innovation, the relative advantage of the innovation, and the ability of adopters to pilot or test the new innovation. In understanding the process of change underpinning the adoption by agencies of evidence-based practices (Seng et al. 2006; Turner and Sanders 2006) increasing attention has been given to the organizational social context within which service providers work (Glisson et al. 2008). The key elements from an organizational change perspective that need to be taken into account include the extent to which the organizational culture supports or embraces innovation, organizational climate of change, quality of training service providers receive in the new innovation, the characteristics of service providers themselves, and level of support for service providers after initial training to support the innovation (Aarons and Palinkas 2007; Backer et al. 1986; Henggeler et al. 1997). The adoption of evidence-based programs by agencies and service providers is influenced by a range of organizational, training, and practitioner variables (Aarons and Palinkas 2007; Glisson et al. 2008). Aarons and Palinkas (2007), in their research with child welfare, identified six primary factors that are critical in implementing evidencebased programs (EBP) including (1) acceptability of EBP; (2) suitability of EBP; (3) motivations of practitioner; (4) experiences with being trained in EBP; (5) organizational support of EBP; and (6) impact of EBP on process and outcome of services. Research has also explored factors such as fit with community organizations (Biglan and Taylor 2000), and collaborative relationships between researchers and communities (Herscell et al. 2004, Wandersman 2003; Weisz et al. 2004). However, once programs have been developed and relationships forged between researchers, program disseminators and communities, there are still the challenges inherent in training service providers in a community in a cost-effective manner (Wolfe 1999). Ensuring that providers are appropriately supported in their workplace for program delivery is an important element in successful dissemination of evidence-based programs. The return on an agency’s investment in having staff complete specific in-service training is diminished when practitioners fail to implement programs. However, there is little evidence examining the effects of these variables on the implementation of a comprehensive multidisciplinary public health model targeting parenting. By training existing service providers from a variety of agencies within health, education, welfare, and voluntary sectors to use evidence-based interventions, more parents can be exposed to these interventions thus enhancing the

123

Adm Policy Ment Health (2009) 36:133–143

population reach of the program to the level necessary to produce a reduction in the prevalence rates of targeted child problems (Sanders 2008). Professional training regimens that incorporate active skills training methods are well accepted by service providers and are associated with significant increases in observed and self reported skills in conducting parent consultations and an increase in self efficacy (Sanders et al. 2003b). However, not all providers who are otherwise satisfied with their training and who report increases in confidence initiate program use (Seng et al. 2006). Little is known about the factors that might contribute to sustained program use after initial training. The present study aimed to fill this gap. The model of training used in the Triple P system has been designed to provide relatively brief and cost effective training usually as an in-service training process (Mihalopoulos et al. 2007; Turner and Sanders 2006). Providers on average complete a 2–5 day training course depending on the level of intervention they plan to deliver; and then 6–8 weeks following initial training providers complete an accreditation process to become an accredited Triple P provider. Providers return to their own agency or organization with the expectation they will use the program with clients for whom they judge the program to be relevant. Subsequent program use may be influenced by both workplace and program specific factors. Organizational factors such as the extent to which service providers perceive their work environment to be supportive or contribute barriers to program use may be crucial in facilitating the adoption of evidence-based programs (Seng et al. 2006). For example, managers or supervisors who support the service provider’s use of the program are more likely to provide encouragement and to assist the service provider to overcome logistical barriers or obstacles to program use (e.g., permission to run after hours groups or programs, provision of child care). Another facilitator might be whether service providers receive support from peers or colleagues in the work place. Professional relationships with colleagues may affect adoption of new ideas and motivation to change work practices. Another driver for change concerns the clients themselves. Clients are rich sources of feedback for providers about the acceptability and effectiveness of programs that may influence future program use. Provider’s judgment of a programs’ value may be influenced via positive feedback received from clients about the intervention effects, cultural relevance, or acceptability of the program. The present study sought to extend the literature on the dissemination of evidencebased parenting interventions, within a large scale population level effort, by following the natural history of service providers’ program use in the 6 month period after initial Triple P professional training. We expected that greater program use would be associated with the presence

Adm Policy Ment Health (2009) 36:133–143

135

of organizational support and the relative absence of program barriers.

Triple P training with the service providers whom they interviewed. The interview team was able to complete interviews for 94% of the service provider sample.

Method

Training Program

Participants

Within the Triple P framework there are five different levels of intervention. These interventions include a universal population-level media information campaign targeting all parents (Level 1), two levels of brief primary care consultations targeting mild behaviour problems (Levels 2 and 3), and two more intensive parent training and family intervention programs for children at risk for more severe behavioral problems (Levels 4 and 5; for more information concerning the varying levels of Triple P see Sanders et al. 2003a). There was one prerequisite for the training levels of the intervention. Service providers had to be trained in Level 4 Triple P before being able to participate in the Level 5 Enhanced Triple P training course. Entry to the other levels did not require completion of prior Triple P training. See Table 1 for more information concerning the different levels of intervention in the Triple P system. After consultation with a training consultant practitioners nominated the level of intervention and type of training they wanted to attend. A small portion of participants (9.8%, n = 59) attended more than one level or type of training; 76.4% (n = 468) of participants attended only Primary Care Triple P training courses (which prepare a provider to deliver both Level 2 and Level 3 interventions); 13.8% (n = 84) attended only Level 4 Standard training courses, and 7.6% (n = 47) attended Level 4 Group Triple P. All training involved two parts. The first part was either 2 days (Level 2–3) or 3 days (Level 4 Standard, Level 4 Group) and consisted of the initial training where skills and knowledge in Triple P were acquired. At the end of partone training, practitioners were encouraged to bring the program back to their worksite to practice delivering the program with peers or clients. The second part of the training occurred 6–8 weeks after part-one of the training and consisted of competency evaluation activities. Providers were required to demonstrate their skills and knowledge of Triple P through role playing with other practitioners who stood in as parents. Providers demonstrated program skills in two mandatory scenarios (provided prior to part-two training) and a third scenario selected from three options. Competency assessment required that at least 80% of the requisite skills were demonstrated in each scenario, as assessed by a Triple P trainer. The assessment of the providers’ skills occurred only during the second part of the training. Trainers are permitted to provide limited additional coaching and feedback as needed for providers on any given competency

Six hundred and fifty service providers who underwent training in Triple P over a 2.5 year period were contacted to complete a 6 month post training survey. Of the 650 participants contacted, 611 (94%) completed the survey. The information obtained from the practitioner survey provided the data for the current study. Almost all of the providers (97%) supplied complete demographic data. The vast majority of providers were female (93%). Participants’ mean age was 44.62 (SD = 13.52), with an average of 10.14 years of experience working with parents. Information on the most recent professional background was available for 99.5% of the participants. Professional backgrounds included: counseling professionals (46.80%), health professionals (9.82%), education professionals (14.24%), and other professionals (e.g., parent educators, administrators, managers, victim advocates, civil service, military, law enforcement, clergy, volunteers; 28.64%). Participants learned of the training opportunities through a variety of sources, including supervisors, colleagues, media releases, or direct mail. Procedures This study was embedded in the context of a population level randomized trial of the Triple P Positive Parenting Program being conducted in one state in the Southeast region of the United States to test population level prevention of child maltreatment (Prinz and Sanders 2007). Triple P professional training was implemented in nine counties that were randomly assigned to the Triple P system condition, while nine counties in the comparison condition did not receive such training. In addition to Triple P training, counties in the Triple P system condition were exposed to a continuing series of media and public information strategies focused on positive parenting, as well as support for providers who work with parents of children aged 0–7 in the form of Triple P parenting materials and ongoing consultation from the dissemination staff. Service providers registered for training offered in the counties in which they worked. All service providers who had participated in the initial part of training were eligible to participate in a 20 min structured telephone interview 6 months after initial training. Interviews were conducted by trained interviewers who had not been involved in

123

136

Adm Policy Ment Health (2009) 36:133–143

Table 1 The level and type of Triple P intervention participants trained to deliver Level/type of intervention

Intervention methods

Level 1

Coordinated media and communication strategy raising awareness of parent issues and encouraging participation in parenting programs. May involve electronic and print media (e.g., community service announcements, talkback radio, newspaper and magazine editorials)

Media and communication strategy Universal Triple P Level 2 Brief selective intervention

Health promotion information or specific advice for a discrete developmental issue or minor child behavior problem. May involve a group seminar process or brief (up to 20 min) telephone or face-to-face clinician contact

Selected Triple P Level 3 Narrow focus parent training

Brief program (about 80 min over four sessions) combining advice, rehearsal, and self-evaluation to teach parents to manage a discrete child problem behavior. May involve telephone or face-to-face clinician contact or group sessions

Primary Care Triple P Level 4 Broad focus parent training Standard Triple P

Broad focus program (about 10 h over 8–10 sessions) focusing on parent-child interaction and the application of parenting skills to a broad range of target behaviors. Includes generalization enhancement strategies. May be self-directed or involve telephone or face-to-face clinician contact or group sessions

Group TripleP Level 5 Intensive family intervention

Intensive individually tailored program with modules (60–90 min sessions) including practice sessions to enhance parenting skills, mood management and stress coping skills, and partner support skills

Enhanced Triple P

that was not met during the competency assessment. However, despite possible additional coaching and feedback, providers were only passed during the second part of training if the required competency standards were met. Trainers were also available to address any concerns or issues that providers encountered in their initial efforts at program implementation. Accreditation as a Triple P provider required successful completion of the competency evaluation and a written test on Triple P principles and techniques (see Turner and Sanders 2006 for a more detailed description of the training process). Out of the 650 providers who initially started the Triple P training only 530 (81.54%) of the providers passed the accreditation process. Measures Sociodemographic Characteristics of Providers Providers’ sociodemographic characteristics including; age, gender, race/ethnic background, geographic location (urban, suburban, rural), and occupations were collected. The 21 different discipline groups were classified into 4 types including health, educational, counseling, and other. Confidence in Parent Consultation Skills Providers’ level of confidence in conducting parent consultations was self-assessed by the question ‘‘How confident are you in conducting parent consultation about

123

child behavior?’’ Providers responded on a seven-point scale ranging from 1(not at all confident) to 7 (very confident). Adequacy of Training in Parent Consultation Skills Providers’ self-report of adequacy in training of parent consultation skills was assessed by the question ‘‘Do you feel adequately trained to conduct parent consultations about child behavior?’’ Responses were noted on a sevenpoint scale from 1(no, definitely not) to 7 (yes, definitely). Self-efficacy in Triple P Self-efficacy in Triple P skills was assessed by responses on the training evaluation form to the question ‘‘Do you feel you now have the skills to implement Triple P in your work with families.’’ Again, providers responded on a seven-point scale of 1(no, definitely not) to 7(yes, definitely). Program Use Provider use of the program was assessed 6 months after the completion of part-one training during a structured telephone interview. Providers were asked whether they had used Triple P with families at any time since their initial training. Providers who identified themselves as having used the program at least once since training were defined as being Triple P initiators. Providers were defined

Adm Policy Ment Health (2009) 36:133–143

137

as being maintained users if they reported not stopping their Triple P usage after using it with at least one or two families. For this study, the vast majority of providers who initiated Triple P usage maintained their usage (see Table 2). Providers were further defined as a high or a low user on the basis of a median split of the total number of families that the providers reported having used Triple P with (range = 1–100 families; median split = 5 families). Specifically, providers were asked how many families they were using Triple P with, and based on the median split, providers who used Triple P with more than five families

were considered a high user, and those who used Triple P with less than five families were considered a low user.

Table 2 Demographic characteristics of sample

Analyses

Facilitators and Barriers to Use Facilitators and barriers to program use were also assessed during the structured telephone interview. Providers were asked to rate how much of a facilitator or obstacle each of 15 potential facilitators and 19 potential barriers had been to the use of Triple P. Responses were given on a 1–5 scale (1 = not an obstacle, 3 = a moderate obstacle, and 5 = an extreme obstacle). Items addressed workplace characteristics, program management difficulties, and program fit.

Frequency % (N = 611) Age Less than 30 years

85 (13.96%)

30–39 years

144 (23.65%)

40–49 years

165 (27.09%)

50–59 years

171 (28.08%)

60 years or older Unanswered

44 (7.22%)

Given the lack of prior evidence of predictors of Triple P program use, exploratory analyses were conducted with the data. Specifically, a series of binary logistic regressions were conducted to explore the univariate relationship between each predictor and both Triple P program use (i.e., use versus non-use) and amount of Triple P program use (i.e., high versus low use).

2 (0.06%)

Racial or ethnic group African-American or black

234 (38.42%)

Caucasian or white

346 (56.81%)

Biracial

3 (0.49%)

Hispanic

8 (1.31%)

Asian

1 (0.16%)

Native American

4 (0.65%)

Other Refused/unanswered

3 (0.49%) 12 (1.96%)

Work setting Urban (inner city)

147 (24.06%)

Suburban (away from city)

154 (25.20%)

Rural (away from city)

281 (45.99%)

Unanswered

29 (4.75%)

Practitioner profession Counseling professionals

286 (46.80%)

Health professionals

60 (9.82%)

Education professionals

87 (14.24%)

Other professionals Unanswered

175 (28.64%)

Female Unanswered

Sociodemographic Characteristics of Practitioners Table 2 presents data on the sociodemographic characteristics of all service providers. Service providers were relatively experienced with 62% being over the age of 40. While the majority of participants were Causasian (56%), a substantial number of African American providers were in the sample (38%). Many worked in a rural setting (48%), were from the counseling profession (47%) and were overwhelmingly female (92%). Table 2 also depicts the breakdown of all service providers who completed the survey (611 = 100%) into service providers who initiated Triple P use (384 = 62.85%), maintained use (356 = 58.27%) and who were non-users (227 = 37.15%) of Triple P. Therefore, out of the 384 service providers who initiated Triple P use, 356 maintained their usage (92.71%).

3 (0.5%)

Practitioner Characteristics and Program use

Practitioner gender Male

Results

49 (7.58%) 545 (91.75%) 17 (0.77%)

Program use Users initiated Triple P

384 (62.85%)

Maintained using

356 (58.27%)

Non-users

227 (37.15%)

Table 3 presents data on the relationship between practitioner characteristics and program use. There was no relationship between program use and practitioner’s age, or whether they worked in urban or rural settings. However, higher use was associated with being a Caucasian provider, and lower use was associated with being a health, education, or other professional.

123

138

Adm Policy Ment Health (2009) 36:133–143

Table 3 Practitioner characteristics: logistic regressions of facilitators of Triple P usage

Triple P usage N (611)

%

OR

95% CI

Practitioner profession Counseling professionals

286

52.7

Health professionals

60

6.8

Education professionals

87 175

1.0 0.318***

0.180, .563

12.8

0.536*

0.327, .879

27.7

0.639*

0.430, .949

49

10

1.0

545

90

0.509*

85

13.4

1.0

30–39 years 40–49 years

144 165

23.3 27.7

1.079 1.198

50–59 years

171

29.6

1.299

0.759, 2.223

44

6.0

0.730

0.351, 1.521

Other professionals Practitioner gender Male Female

0.260, .999

Practitioner age Less than 30 years

60 years or older

0.623, 1.868 .699, 2.052

Racial or ethnic group

* Signifies a statistically significant odds ratio at P \ .05 ** Signifies a statistically significant odds ratio at P \ .01 *** Signifies a statistically significant odds ratio at P \ .001

African-American or black

234

34.55

1.0

Caucasian or white

346

60.21

1.532*

1.089, 2.156

Biracial

3

0.8

0.000

0.00

Hispanic

8

1.57

2.318

0.458, 11.726

Asian

1

0

0.000

0.000

Native American

4

0.52

0.773

0.107, 5.579

Other

3

0.52

1.545

0.138, 17.282

12

1.83

1.803

0.455, 7.145

Refused/unanswered Work setting Urban (inner city)

147

23

1.0

Suburban (away from city) Rural (away from city)

154 281

26.3 50.7

1.241 1.451

Client Characteristics and Program Use Table 4 presents information of the relationship between providers’ client characteristics and program use. Practitioners were more likely to use Triple P when they had clients with presenting problems for which Triple P was appropriate. They were also more likely to use Triple P when they encountered parents whose needs were appropriate to the level of intervention that they were trained to deliver. Racial or cultural concerns, literacy level of the program, and language barriers were not significant predictors of Triple P usage. Providers’ Perceptions of Barriers and Program use Table 5 presents data on practitioners’ perceptions of barriers and organizational barriers, and program use. Lower program use was associated with the following barriers being reported as a medium or high obstacle: lack of access to consultation or supervision, lack of recognition by

123

0.781, 1.972 0.962, 2.188

colleagues, lack of overtime or time in lieu for after hours appointments, clash with other after hours commitments, lack of knowledge or skill in behavioral family intervention, clash with theoretical orientation or preferred approach, not being integrated with caseload or other work responsibilities, and difficulties coordinating with other practitioners involved with the family. Predictors of High Program Use Table 6 summarizes the significant predictors of high program use. The strongest predictors of high program use were being trained in the use of Group Triple P, receiving positive feedback from parents, exposure to only minor or moderate workplace barriers, seeing observable changes in children or families, and consultation with other Triple P practitioners. Inhibitors of high use were low confidence in parent consultation work and use of Triple P, difficulty incorporating Triple P into everyday work, and low workplace support.

Adm Policy Ment Health (2009) 36:133–143

139

Table 4 Client characteristics: logistic regressions of barriers to Triple P usage

Table 5 Providers’ perceptions of barriers: logistic regressions to Triple P usage

Triple P usage N (611)

%

Triple P usage OR

95% CI

Do any of the following keep you from using Triple P in specific cases or with specific families? Client needs higher level of Triple P than you are trained to provide Yes

267

43.70

1.0

No

333

54.50

1.645**

1.171, 2.311

Not appropriate for the presenting problem Yes

202

33.06

1.0

No

402

65.79

1.597*

86

14.08

1.0

No

521

85.27

1.107

437

71.52

1

Medium obstacle High obstacle

115 40

18.82 6.55

0.610* 0.486*

0.402, 0.928 0.254, 0.933

After hours appointments clashing with other commitments

High obstacle Low obstacle

0.687, 1.786

Literacy level Yes

177

28.97

1.0

No

430

70.38

1.385

456

95% CI

349

57.12

1

96

15.71

0.712

0.446, 1.136

119

19.48

0.581*

0.379, 0.889

Lack of recognition by colleagues for Triple P work

Yes

No

OR

Low obstacle

Medium obstacle 1.113, 2.294

%

Not enough knowledge and skills in behavioral family intervention

Low obstacle

Racial or cultural concerns

Language barriers Yes 150

N (611)

459

75.12

1

Medium obstacle

76

12.44

0.561*

0.344, 0.916

High obstacle

53

8.67

0.659

0.370, 1.172

Unavailability of overtime or comp time for after-hours appointments 0.956, 2.008

Low obstacle Medium obstacle

24.55

1.0

74.63

1.141

High obstacle 0.776, 1.679

61.87

1

63

10.31

0.670

0.388, 1.157

127

20.79

0.526**

0.349, 0.792

Insufficient access to consultation or supervision

** Signifies a statistically significant odds ratio at P \ .01

Low obstacle Medium obstacle

*** Signifies a statistically significant odds ratio at P \ .001

High obstacle

* Signifies a statistically significant odds ratio at P \ .05

378

440 106

72.01 17.35

1 0.935

0.600, 1.456

43

7.04

0.499*

0.266, 0.936

Difficulty coordinating with other practitioners involved with the family

Practitioner Behavior and Program Use All service providers (611) were asked as part of the survey if they wished to add any additional information regarding their experiences with Triple P; as a result, 166 service providers (both users and non-users) did offer additional supplementary information. Table 7 presents data on the relationship between practitioners behavior and program use which was derived from this additional, qualitative information obtained from a subsample of the larger provider pool. To check whether this subsample demographically differed from the whole sample Chi Square analyses were performed, see Table 8. These analyses revealed no significant differences between survey respondents and nonrespondents in terms of being a Triple P user, providers age, providers gender, and providers race. However, the respondents did significantly differ for work setting and job profession, although this was most likely due to the baseline sample size. A large number of providers reported that they had incorporated Triple P ideas and principles into their work in general (82%). Practitioners were significantly less likely to use the program if they had not incorporated Triple P principles and strategies in other aspects of their work, or had not incorporated Triple P informally with neighbours,

Low obstacle

458

74.96

1

Medium obstacle

73

11.95

0.458**

0.278, 0.753

High obstacle

39

6.38

0.580

0.300, 1.120

Clash with your theoretical perspective or preferred treatment approach Low obstacle 524 85.76 1 Medium obstacle

46

7.53

0.644

0.351, 1.182

High obstacle

27

4.42

0.433*

0.198, 0.944

Triple P not integrated with caseload or other responsibilities at work Low obstacle

372

60.88

1

Medium obstacle

104

17.02

0.874

0.557, 1.373

High obstacle

120

19.63

0.710

0.466, 1.081

Low availability of clients Low obstacle

396

64.81

1

Medium obstacle

97

15.88

0.877

0.554, 1.388

High obstacle

99

16.20

0.832

0.529, 1.309

Triple P resource materials not available Low obstacle

554

90.67

Medium obstacle

23

3.76

1 1.043

0.435, 2.503

High obstacle

18

2.95

0.695

0.270, 1.790

* Signifies a statistically significant odds ratio at P \ .05 ** Signifies a statistically significant odds ratio at P \ .01 *** Signifies a statistically significant odds ratio at P \ .001

123

140

Adm Policy Ment Health (2009) 36:133–143

Table 6 Logistic regressions for predictors of high Triple P usage High Triple P usage N (384) %

OR

Table 7 Practitioner behaviors: logistic regressions of facilitators of Triple P usage Triple P usage

95% CI

N (166)

%

OR

95% CI

Level 4 group Triple P use No

317

Yes

36

82.34 1 9.38 8.575*** 2.963, 24.813

Feedback from parents regarding the program Not helpful

37

Helpful

92

9.64 1 23.96 3.359**

1.459, 7.733

Very helpful 205 53.39 3.662** 1.684, 7.962 Extent of barriers to the use of Triple P in your workplace To a great extent

22

5.73 1 33.33 3.539*

1.300, 9.635

To a minor extent

52.08 2.832*

1.064, 7.533

Seeing observable change in children or families Not helpful Helpful Very helpful

35

9.11 1

0.455*

Yes

97

58.43

1

No

69

41.57

0.426*

Yes

137

82.53

1

No

28

16.87

0.357*

1.048, 5.417

Yes

50

30.12

1

55.99 2.332*

1.104, 4.925

No

116

69.88

0.541

Yes

249

64.84 1.593*

147

38.28 1

Somewhat confident

198

51.56 0.589*

Not very confident 8 2.08 0.217 Confidence in providing consultation to parents

1.005, 2.526

0.382, 0.908 0.042, 1.113

Very confident

213

55.47 1

Somewhat confident

136

35.42 0.550**

0.356, 0.850

1.04 0.718

0.099, 5.192

4

Ease of incorporating Triple P into your job activities Very easy

152

39.58 1

Somewhat easy

164

42.71 0.741

37

0.475, 1.157

9.64 0.255*** 0.115, 0.565

Workplace support of Triple P 274

Somewhat supportive

58

Not very supportive

15

0.221, 0.821

0.156, 0.819

0.255, 1.149

* Signifies a statistically significant odds ratio at P \ .05

Overall confidence in using Triple P Very confident

0.226, 0.918

Provided Triple P to neighbours, friends, adult family members, or anyone else in something other than your normal work setting

21.61 2.383*

27.08 1

** Signifies a statistically significant odds ratio at P \ .01 *** Signifies a statistically significant odds ratio at P \ .001

systems-contextual perspective. There are various phases in the adoption of an evidenced based program including initial adoption, implementation, and sustained use. This study focused on the first two of these phases over a 6 month period since initial training. For a population health approach to work, trained providers need to become regular users of the programs they have been trained to implement. In the present context two-thirds of service providers became users of Triple P and of those who commenced use after training, the vast majority maintained their program use when assessed 6 months post training.

71.35 1 15.10 0.886 3.91 0.207*

0.502, 1.562

Organizational Characteristics

0.057, 0.749

* Signifies a statistically significant odds ratio at P \ .05 ** Signifies a statistically significant odds ratio at P \ .01 *** Signifies a statistically significant odds ratio at P \ .001

friends, adult family members or used tipsheets and videos without follow up.

Discussion The present study confirms the central importance of viewing the preparation of service providers to implement evidence-based programs from an organizational

123

1

61.45

83

104

Very supportive

38.55

102

215

No

Not very easy

64

No

Incorporated parts of Triple P within another parenting program or curriculum

Consultation with other Triple P practitioners

Not very confident

Yes

Incorporated Triple P ideas or principles into your work in general

To a moderate extent 128 200

Given out tip sheets or shown videos even when there has not been an opportunity to discuss them with the parents

The present findings confirm the importance of the organizational social context for the adoption of evidencebased programs (Glisson et al. 2008). Several organizational barriers were identified as inhibitors of program use. These included after hours appointment clashes, difficulties getting compensated for after hours appointments, lack of workplace recognition by colleagues for Triple P work, inadequate consultation or support and difficulties coordinating with other service providers work on a case. Although some of these problems were experienced by only a minority of providers (e.g., inadequate supervision or support), in some instances these problems were experienced by many (e.g., after hours appointment clashes), and this result confirms that program use does

Adm Policy Ment Health (2009) 36:133–143 Table 8 Chi-Square analysis for sociodemographic characteristics for respondents versus non-respondents

141

Respondents (n = 166)

Non-respondents (n = 445)

Statistics

Work setting Urban

59 (37.3%)

88 (20.8%)

Suburban

34 (21.5%)

120 (28.3%)

Rural

65 (41.1%)

216 (50.9%)

Counselling

57 (34.8%)

229 (51.6%)

Health

17 (10.4%)

43 (9.7%)

Education

52 (31.7%)

35 (7.9%)

Other

38 (23.2%)

137 (30.9%)

13 (8.1%) 147 (91.9%)

36 (8.3%) 398 (91.7%)

v2 = 0.004, df = 1, ns

110 (66.3%)

274 (61.6%)

v2 = 1.140, df = 1, ns

56 (33.7%)

171 (38.4%)

v2 = 16.831, df = 2, P \ .001

Practitioner profession v2 = 57.224, df = 3, P \ .001

Practitioner gender Male Female Triple P usage User Non-user Age Less than 30 years

25 (15.1%)

60 (13.5%)

30–39 years

35 (21.1%)

109 (24.6%)

40–49 years

42 (25.3%)

123 (27.8%)

50–59 years

46 (27.7%)

125 (28.2%)

60 years or older

18 (10.8%)

26 (5.9%)

v2 = 5.249, df = 4, ns

Racial or ethnic group African-American or black

58 (34.9%)

176 (39/7%)

105 (63.3%)

241 (54.4%)

Biracial

0 (0%)

3 (0.7%)

Hispanic Asian

1 (0.6%) 0 (0%)

7 (1.6%) 1 (0.6%)

Caucasian or white

Native American

1 (0.6%)

3 (0.7%)

Other

0 (0%)

3 (0.7%)

Refused/unanswered

1 (0.6%)

9 (2.0%)

not occur in a social vacuum and that workplace barriers need to be addressed if programs are to reach enough parents. Provider Characteristics Service providers who became high users of Triple P and therefore intervened with more families, tended to be trained in Level 4 Group Triple P, had few barriers to implementation, received feedback from parents regarding the program, and tended to consult with other Triple P practitioners. The likelihood of being a high user diminished when practitioners had low self efficacy or confidence, found it difficult to incorporate Triple P into their job, and lacked workplace support. Provider barriers to program use were also identified, and about a fourth of the providers reported that insufficient knowledge and skills in the application of the

v2 = 7.400, df = 7, ns

behavioral family intervention approach was a barrier and was associated with lower program use. Although a clash with theoretical orientation was a barrier to program use, it was experienced as a problem by relatively few providers (11.95%). These findings are not expected given that many of the service providers in the study had a relatively low level of professional education and little prior exposure to evidence based parenting programs (Seng et al. 2006). The present findings show that providers who used Triple P were more likely to generalize their use of skills. For example, the finding that nearly two-thirds of the providers reported having given advice informally to neighbors, friends, and other adult family members suggests that the positive parenting principles and strategies were applied outside an intervention relationship and that this generalization across settings was associated with an increased likelihood of using Triple P with clients. However, it should be noted that this finding was based on a

123

142

subset of the full sample, and as a result caution is indicated when interpreting the generalizability of the findings. Providers who incorporated Triple P knowledge and skills in their general work were also more likely to use the program with clients. These findings suggest that when evidence based programs are introduced staff who are the most proactive in using the program also apply what they have learned in a variety of formal and informal contexts. These findings confirm the importance of provider attitudes in adopting and using evidence-based programs (Aarons and Palinkas 2007). Importantly, although some providers reported incorporating parts of Triple P into other parenting programs, it was not a significant predictor of program use. Collectively, these findings point to the need to acknowledge and potentially encourage service providers to apply what they have learned in diverse contexts just as parents are trained to do. However, there is also a risk that indiscriminate use of a program may result in ineffective or inappropriate applications. Perhaps supervision and quality assurance processes could be used to promote the appropriate generalization of knowledge and skills and discourage applications that are unlikely to be productive (e.g., blending the video resources from one program with activities from another). Client Characteristics Client characteristics such as racial and cultural background, literacy levels, and language barriers were unrelated to the level of use, suggesting that the program has a broad applicability across a wide range of families. Additionally, Triple P was more likely to be used when clients presented with problems that were appropriate for Triple P. These findings indicate that practitioners are likely to be appropriately utilizing Triple P with families that need it. Implications There are several important implications from this study. First, service providers are not passive recipients of a disseminator’s efforts to go to scale. These findings show that providers who become the heaviest users and enthusiasts for the program use the program in a range of contexts and incorporate guiding principles into the overall work they do with families, which supports the emphasis that Aarons and Palinkas (2007) place on provider attitudes in adopting and using evidence-based programs. As Triple P builds in a number of features that specifically promote transfer of learning into both the parenting intervention and the professional training course experience (e.g., training loosely, training sufficient exemplars, the importance of self regulation) this flexible application

123

Adm Policy Ment Health (2009) 36:133–143

can be viewed as a positive outcome. We are unable to determine, however, whether this generalized responding has been done appropriately (i.e., practitioners discriminate whether the context is suitable for generalized responding). Second, implementation of evidence-based programs occurs in a social context. Organizations wanting to adopt evidence-based approaches cannot simply send providers to be trained; if they want a return on investment they need to support their staff members in using the program posttraining. This may be achieved by eliminating organizational barriers to implementation (e.g., lack of supervision or peer support, ability to see families outside of typical daytime clinic hours). One way to address these social-contextual issues might be to establish formal agreements between a disseminator and an organization regarding program implementation, and having line managers trained in program delivery along with staff. These actions may help ensure that logistical barriers are eliminated or reduced. Such agreements also will aid in establishing local implementation targets so that a defined and agreed upon number of programs are delivered to families. However, as Aarons and Palinkas (2007) suggest, the eventual implementation of evidence-based programs by agencies is a result of the disseminator’s perseverance, experience, and flexibility. The present findings need to be interpreted in light of the study’s limitations. First, the sole informant was the trained service provider. We were unable to separately assess the organizational and other barriers or facilitators or the actual level of program use reported by providers. Second, this study did not attempt to assess clinical outcomes at an individual case level. The larger trial assessed population level outcomes through inspection of independent data systems relating to child maltreatment reports, hospitalization and emergency room visits due to maltreatment, and out of home placements (Prinz et al. 2009). Third, this study followed providers for a relatively brief period of time post-training; future work will focus on longer term follow up with these providers. Finally, in retrospect it would have been better to categorize the qualifications and disciplines of participants more fully. The parent education workforce represents a diverse range of occupational backgrounds and employment contexts that makes classification of a providers job by discipline difficult. To address this limitation in future research needs to use a more comprehensive system of the professional classification relevant to the providers of parenting services. Acknowledgments The research described in this paper was supported by grants U17/CCU422317 and 1R18CE001340 to Prinz and Sanders from the National Center for Injury Prevention and Control, Centers for Disease Control and Prevention. The findings and conclusions in this article are those of the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention.

Adm Policy Ment Health (2009) 36:133–143 The authors wish to acknowledge the contribution of James Kirby in the preparation and editing of the manuscript.

References Aarons, G. A., & Palinkas, L. A. (2007). Implementation of evidencebased practice in child welfare: Service provider perspectives. Administration and Policy in Mental Health and Mental Health Services Research, 34(4), 411–419. doi:10.1007/s10488-0070121-3. Backer, T. E., Liberman, R. P., & Kuehnl, T. G. (1986). Dissemination and adoption of innovative psychosocial interventions. Journal of Consulting and Clinical Psychology, 54, 111–118. doi:10.1037/0022-006X.54.1.111. Biglan, A., & Taylor, T. K. (2000). Increasing the use of science to improve child-rearing. The Journal of Primary Prevention, 21, 207–226. doi:10.1023/A:1007083203280. Conor-Smith, J. K., & Weisz, J. R. (2003). Applying treatment outcome research in clinical practice: Techniques for adapting interventions to the real world. Child and Adolescent Mental Health, 8(1), 3–10. doi:10.1111/1475-3588.00038. Glisson, C., Landsverk, J., Schoenwald, S., Kelleher, K., Hoagwood, K. E., Mayberg, S., et al. (2008). Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research, 35(1–2), 98–113. Hawley, K. M., & Weisz, J. R. (2002). Increasing the relevance of evidence-based treatment review to practitioners and consumers. Clinical Psychology: Science and Practice, 9(2), 225–230. doi: 10.1093/clipsy/9.2.225. Henggeler, S. W., Melton, G. B., Brondino, M. J., Schereer, D. G., & Hanely, J. H. (1997). Multisystemic therapy with violent and chronic juvenile offenders and their families: The role of treatment fidelity in successful dissemination. Journal of Consulting and Clinical Psychology, 65, 821–833. doi:10.1037/ 0022-006X.65.5.821. Herscell, A. D., McNeil, C. B., & McNeil, D. W. (2004). Clinical child psychology’s progress in disseminating empirically supported treatments. Clinical Psychology: Science and Practice, 11, 267–288. doi:10.1093/clipsy/bph082. Mihalopoulos, C., Sanders, M. R., Turner, K. M. T., Murphy-Brennan, M., & Carter, R. (2007). Does the Triple P-Positive parenting program provide value for money? The Australian and New Zealand Journal of Psychiatry, 41(3), 239–246. doi:10.1080/ 00048670601172723. Ollendick, T. H., & Davis, T. E. (2004). Empirically supported treatments for children and adolescents: Where to from here? Clinical Psychology: Science and Practice, 11, 289–294. doi: 10.1093/clipsy/bph083. Pentz, M. (2004). Form follows function: Designs for prevention effectiveness and diffusion research. Prevention Science, 5, 23–29. doi:10.1023/B:PREV.0000013978.00943.30. Prinz, R. J., & Sanders, M. R. (2007). Adopting a population-level approach to parenting and family support. Clinical Psychology Review, 27(6), 739–749. doi:10.1016/j.cpr.2007.01.005. Prinz, R. J., Sanders, M. R., Shapiro, C. J., Whitaker, D. J., & Lutzker, J. R. (2009). Population-based prevention of child maltreatment: The US Triple P system population trial. Prevention Science. doi:10.1007/s11121-009-0123-3.

143 Rogers, E. M. (2000). Diffusion theory: A theoretical approach to promote community-level change. In J. L. Peterson & R. J. DiClemente (Eds.), Handbook of HIV prevention (pp. 57–65). Netherlands: Kluwer Academic Publishers. Sanders, M. R. (1999). Triple P positive parenting program: Towards and empirically validated multilevel parenting and family support strategy for the prevention of behavioral and emotional problems in children. Clinical Child and Family Psychology Review, 2, 71–90. doi:10.1023/A:1021843613840. Sanders, M. R. (2008). The Triple P-positive parenting program as a public health approach to strengthening parenting. Journal of Family Psychology, 22(4), 506–517. Sanders, M. R., Markie-Dadds, C., & Turner, K. M. T. (2003a). Theoretical, scientific and clinical foundations of the Triple P-positive parenting program: A population approach to the promotion of parenting competence. Parenting Research and Practice Monograph, 1, 1–21. Sanders, M. R., Murphy-Brennan, M., & McAuliffe, C. (2003b). The development, evaluation and dissemination of a training programme for general practitioners in evidence-based parent consultation skills. International Journal of Mental Health Promotion, 5, 13–20. Sanders, M. R., Markie-Dadds, C., Rinaldis, M., Firman, D., & Baig, N. (2007). Using household survey data to inform policy decisions regarding the delivery of evidence-based parenting interventions. Child: Care, Health and Development, 33(6), 768–783. doi:10.1111/j.1365-2214.2006.00725.x. Seng, A. C., Prinz, R. J., & Sanders, M. R. (2006). The role of training variables in effective dissemination of evidence-based parenting interventions. International Journal of Mental Health Promotion, 8, 20–28. Shapiro, C.J., Prinz, R.J., & Sanders, M.R. (2007). Population-wide parenting intervention training: Initial feasibility. Journal of Child and Family Studies, published online 28 September. Silverman, W. K., & Kurtines, W. M. (2004). Research progress on effectiveness, transportability, and dissemination of empirically supported treatments: Integrating theory and research. Clinical Psychology: Science and Practice, 11(3), 295–299. doi:10.1093/ clipsy/bph084. Turner, K. M. T., & Sanders, M. R. (2006). Dissemination of evidence-based parenting and family support strategies: Learning from the Triple P-positive parenting program system approach. Aggression and Violent Behavior, 11, 176–193. doi: 10.1016/j.avb.2005.07.005. Wandersman, A. (2003). Community science: Bridging the gap between science and practice with community-centred models. American Journal of Community Psychology, 31, 227–242. doi: 10.1023/A:1023954503247. Weisz, J. R., Chu, B. C., & Polo, A. J. (2004). Treatment dissemination and evidence-based practice: Strengthening intervention through clinician-researcher collaboration. Clinical Psychology: Science and Practice, 11, 300–307. doi:10.1093/ clipsy/bph085. Weisz, J. R., Jensen-Doss, A., & Hawley, K. M. (2006). Evidencebased youth psychotherapies versus usual clinical care: A metaanalysis of direction comparisons. The American Psychologist, 6, 671–689. doi:10.1037/0003-066X.61.7.671. Wolfe, J. (1999). Overcoming barriers to evidence-based practice: Lessons from medical practitioners. Clinical Psychology: Science and Practice, 6, 445–448. doi:10.1093/clipsy/6.4.445.

123