Journal of Positive Behavior Interventions - Digital Chalkboard

Journal of Positive Behavior Interventions http://pbi.sagepub.com/

A Descriptive Analysis of Intervention Research Published in the Journal of Positive Behavior Interventions: 1999 Through 2005 Shelley Clarke and Glen Dunlap Journal of Positive Behavior Interventions 2008 10: 67 DOI: 10.1177/1098300707311810 The online version of this article can be found at: http://pbi.sagepub.com/content/10/1/67

Published by: Hammill Institute on Disabilities

and http://www.sagepublications.com

Additional services and information for Journal of Positive Behavior Interventions can be found at: Email Alerts: http://pbi.sagepub.com/cgi/alerts Subscriptions: http://pbi.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav

Downloaded from pbi.sagepub.com by guest on June 6, 2011

Forum The Forum section of the Journal of Positive Behavior Interventions is presented to encourage communication among readers and provide for an exchange of opinions, perspectives, ideas, and informative personal accounts. We welcome brief articles from family members, professionals, friends, advocates, administrators, researchers, and other individuals who are concerned with behavioral support issues. The purpose of the Forum is to facilitate a constructive dialogue among many stakeholders regarding important issues in practice, research, training, program development, and policy. Submissions to the Forum undergo an expedited review.

Journal of Positive Behavior Interventions Volume 10 Number 1 January 2008 67-71 © 2008 Hammill Institute on Disabilities 10.1177/1098300707311810 http://jpbi.sagepub.com hosted at http://online.sagepub.com

A Descriptive Analysis of Intervention Research Published in the Journal of Positive Behavior Interventions: 1999 Through 2005 Shelley Clarke Glen Dunlap University of South Florida Abstract: The Journal of Positive Behavior Interventions (JPBI) has been publishing reports of empirical intervention research since 1999, with a commitment to serve as a vehicle for dissemination of data and perspectives pertinent to positive behavior support (PBS). PBS is distinguished by an emphasis on certain features of interventions, such as ecological and social validity. The current analysis was undertaken as an effort to describe the characteristics of intervention research published in JPBI from 1999 through 2005 and to provide a comparison with other peer-reviewed journals that publish a large number of articles reporting intervention research with children and youth with disabilities. The data indicate that JPBI has been publishing research with comparatively high levels of ecological validity, social validity, and assessment-based interventions. The authors note other distinctive aspects of JPBI’s publication record and discuss the data with respect to the current and future character of PBS research. Keywords:

T

JPBI; descriptive analysis; applied research

he Journal of Positive Behavior Interventions (JPBI) was established as a peer-reviewed vehicle for disseminating research and perspectives relating to the emerging discipline of positive behavior support (PBS; Dunlap & Koegel, 1999). Positive behavior support has been defined as a pragmatic approach based on behavioral and biomedical sciences for enhancing the quality of life and reducing the problem behaviors of individuals with challenges of behavioral adaptation. The definition of PBS has been elaborated on by numerous authors (e.g., Horner et al., 1990; Carr et al., 2002) with lists of features that, in aggregate, essentially define the practice of PBS. These features include data-based accountability, an emphasis on broad outcomes reflecting lifestyle improvements, ecological and social validity, a collaborative approach to planning and implementation, and an emphasis on proactive interventions focusing on instruction and environmental redesign. It was reasonable to assume, therefore, that JPBI, which began publication in 1999, would include in its contents articles that reflected

these defining features to a greater extent than other journals concerned with intervention research. The current analysis was undertaken to provide an empirical description of intervention research articles that appeared in the first seven volumes of JPBI—from its inception in 1999 through 2005. Of particular interest was the extent to which JPBI’s published intervention research has reflected major features of PBS (e.g., ecological validity, social validity, use of assessment data to determine interventions) and the extent to which its

Authors’ Note: Development of this article was partially supported with funding from the U.S. Department of Education, Office of Special Education Programs, Grant H324Z010001. The opinions expressed by the authors are not necessarily reflective of the position of, or endorsed by, the U.S. Department of Education. The authors would like to thank Rebecca “Rainey” Frank and Michelle Marill for assistance with data collection and summation and JPBI’s Action Editor for this manuscript, Todd Risley, for constructive suggestions regarding the manuscript.


67

68 Journal of Positive Behavior Interventions

publication record differs from intervention research published in comparable peer-reviewed journals.

Method Definition of Database This descriptive analysis included data-based articles regarding interventions for children or youth with disabilities published in JPBI during the period noted previously. For purposes of comparison, two additional journals were examined: the Journal of Applied Behavior Analysis (JABA) and Education and Training on Mental Retardation and Developmental Disabilities (ETMRDD). These journals were selected for comparison because, of all peer-reviewed periodicals, they included the largest number of intervention research articles focused on children and youth with disabilities and because they maintained operation from 1999 through 2005. Each of the three journals was searched manually. Articles were included in the database if they met the criteria described below (cf., Clarke, Dunlap, & Stichter, 2002). Participants included in the articles had to be children or adolescents (i.e., 21 years of age or younger), with at least one participant identified as experiencing an intellectual, emotional, behavioral, learning, language, and/or attention disorder or disability. Articles were excluded if the participants did not have descriptive or diagnostic labels, unless the participants were enrolled in programs explicitly designed for children or youth with the pertinent labels. The articles had to contain original reports of experimental or quasi-experimental research, and the research design had to include manipulation of an independent (e.g., treatment) variable. Articles had to include descriptions of methods and results accompanied by some sort of data display. The independent variable or variables had to involve educational, behavioral, or psychological interventions; articles in which the independent variable was psychopharmacological were not included. At least one of the dependent variables had to involve observable performance or social behavior, viewed from a broad, applied perspective, as a target for modification.

Procedures and Reliability Each article included in the database was scored along a number of operationally defined dimensions by at least one reviewer. Data extracted from each article consisted of the following:

1. bibliographical information; 2. population characteristics, including diagnostic or descriptive label, age and gender; 3. the setting in which the research took place; 4. ecological context, whether typical or geared toward a special population in which the study took place, including physical location, activity presented, and social environment; 5. dependent variables; 6. independent variables (i.e., intervention type and purpose), as well as whether the intervention included more than one independent variable; 7. who was responsible for orchestrating intervention, including parents, teachers, residential staff, or clinician; 8. whether generalization or maintenance data were collected and/or had been collected for more than 6 months; 9. whether there was a functional link between assessment and the intervention implemented; and 10. whether social validation measures were collected (see Authors’ Note). To assess interobserver agreement, two data collectors recorded data independently for the following percentages of articles from each of the three journals: JPBI, 25%; JABA, 43%; and ETMRDD, 67%. Agreement was counted for each category when the coders entered identical information on the coding sheet. Agreement exceeded 85% for all variables except dependent measures (71%) and intervention type (78%). The relatively low agreement for these two variables appeared to be the result of vague descriptions in the text of some of the articles. In addition, for each of these variables, multiple categories could be scored for each article, and an agreement was counted only if both coders scored the same combination of categories.

Results Table 1 shows the summary statistics for each variable from the articles that met the inclusion criteria. The first row provides the total number of articles for each journal, while subsequent rows present the percentage of articles from each journal that was scored for each variable. The data points that distinguish JPBI are highlighted below.

Ecological and Social Validity An important tenet of positive behavior support is that interventions should have ecological validity, meaning


Clarke, Dunlap / Analysis of Intervention Research in JPBI 69

Table 1 Descriptive Statistics (Percentage of Articles) for Intervention Research Studies in Each Journal Variable Population Severe/profound disability Mild/moderate disability Autism Emotional/behavioral disorders Other Age (yrs) Birth–5 6–11 12–17 18–21 Setting Typical school program Special education Home Residential program Community Dependent variables Destructive behaviors Disruptive behaviors Engagement Social interaction Skill development Adult behaviors Independent variables Skill instruction/enhancement Teacher/parent training Self-management Antecedent manipulations Consequence manipulations Peer mediation Multicomponent interventions Intervention agent Parent/family Teacher Clinician Multiple agents Ecological validity Typical physical context Typical activity context Typical social context Extended data collection Maintenance data Generalization data Data after 6 months of intervention Social validation data collected Assessment-intervention linkage

JPBIa

Figure 1 Percentage of Intervention Research Articles in Each of Three Journals That Were Scored as Having Three Categories of Ecological Validity

JABAb ETMRDDc

12 10 71 5 12

27 23 56 5 17

30 57 32 0 8

51 34 14 2

25 50 21 4

13 35 40 12

31 19 34 2 7

7 9 13 15 2

22 37 8 5 12

44 44 37 15 58 19

55 35 24 6 38 8

8 10 8 10 68 3

41 14 17 44 20 5 29

25 3 1 46 70 1 41

63 3 13 32 22 5 30

10 15 47 27

2 4 90 3

2 17 73 7

63 69 69

19 24 22

33 50 42

44 34 15 31 54

13 6 4 3 45

45 40 2 20 22

Note: JPBI = Journal of Positive Behavior Interventions; JABA = Journal of Applied Behavior Analysis; ETMRDD = Education and Training on Mental Retardation and Developmental Disabilities. a. Total number of articles = 59. b. Total number of articles = 142. c. Total number of articles = 60.

Note: JPBI = Journal of Positive Behavior Interventions; JABA = Journal of Applied Behavior Analysis; ETMRDD = Education and Training on Mental Retardation and Developmental Disabilities.

that they should be implemented in typical contexts (Carr et al., 2002). In the current analysis, research articles were scored as to whether they were implemented in (a) typical physical contexts, meaning that the settings were not geared to a special population; (b) typical activity contexts, meaning that the activities were naturally occurring and not simulated for the purposes of intervention or intervention research; and (c) typical social contexts, meaning that the participants’ interaction occurred in socially inclusive circumstances. Contexts were scored as typical if at least half of the activities were considered typical in regard to the setting, the activity, and the participants involved. As shown in Table 1 and Figure 1, JPBI’s intervention articles were scored as having higher overall rates of ecological validity than those from the two comparison journals. JPBI’s rates were 63%, 69%, and 69% for typical ecological, activity, and social contexts, whereas the rates for JABA ranged from 19% to 24% and those for ETMRDD ranged from 33% to 50%. Each article in the database was also scored as to whether it included measures of social validation, including the acceptability and perceived viability of an intervention. Thirty-one percent of JPBI’s intervention articles included some measure of social validation, with the comparable figures being 3% for JABA and 20% for ETMRDD.

Assessment-Based Intervention Another commonly cited feature of positive behavior support is that interventions are to be preceded by assessments of the behavior’s function and the contexts in which the behavior occurs and does not occur. Data were obtained


70 Journal of Positive Behavior Interventions

on whether the study included an assessment that was directly related to the selection and/or design of the intervention. This functional linkage was scored if the article explicitly referred to a connection between assessment and intervention. The data showed that 54% of JPBI’s intervention articles were assessment based, while the same was true for 45% of JABA’s intervention articles and 22% of ETMRDD’s.

Other Data of Note A number of other variables in Table 1 are pertinent to PBS. For instance, authors writing about PBS often emphasize the importance of affecting broad areas of a participant’s life and doing so over extended periods of time. The three data categories under “Extended Data Collection” show that JPBI’s intervention research articles reported generalization and maintenance data at approximately the same rate as ETMRDD, while a higher percentage of JPBI’s articles included data that were collected 6 months following the beginning of intervention. Although not particular to PBS, some other data categories are distinctive. For instance, 71% of JPBI’s articles have included participants with autism, compared to 56% for JABA and 32% for ETMRDD. In terms of participants’ ages, JPBI has focused on the birth to 5-yearold group, seeming to indicate a tendency to publish early intervention and preschool interventions.

Discussion JPBI was founded to encourage positive behavior support, an approach that explicitly features in situ interventions and measures, extended data collection, social validity, and assessment-based interventions (Carr et al., 2002). This descriptive analysis was undertaken to examine the prevalence of these features in articles published over the first 7 years of the journal’s existence. For comparison, the prevalence of these features was also examined in two other journals: JABA, which represents the original research base for much of the work in JPBI; and ETMRDD, another applied research journal without the explicit PBS mission of JPBI. The following results were found: 1.

2.

JPBI has been most successful in encouraging ecological validity, with two thirds of the articles displaying in situ interventions and measures, a figure substantially higher than the two comparison journals. JPBI has been moderately successful in encouraging assessment-based interventions, with roughly half of its articles displaying this characteristic.

3.

4.

JPBI has been only somewhat successful in encouraging extended data collection, with less than half of its articles measuring short-term maintenance or generalization and even fewer measuring long-term maintenance (these figures are equivalent to ETMRDD, but much higher than JABA). JPBI has been marginally successful in the area of social validity, with less than one third of the articles including this feature (this is still higher than the comparison journals).

Overall, based on these data, we conclude that JPBI is meeting its purpose of publishing research articles that reflect some of the key tenets of PBS. Although no quantitative standards or benchmarks are possible, the comparative data indicate that JPBI has been publishing intervention research characterized by relatively high levels of definitional features, such as ecological validity and preintervention assessments. It is important to acknowledge some limitations with this analysis. One limitation is that the intervention research considered was restricted to articles that included children and youth with some type of identifiable disability. This was done, in part, because the authors had already developed a database with reliable definitions and articles that had previously been coded within the established population parameters (Clarke, Dunlap, & Stichter, 2002; Dunlap, Clarke, & Steiner, 1999). Of course, neither JPBI nor PBS itself is limited to these parameters, so it is possible that articles with older or nonlabeled participants could have been omitted, conceivably altering the nature of the findings. However, in reviewing the contents of JPBI, the actual number of articles excluded was small. Only one article was excluded because it had no participants less than 22 years of age, and four articles were excluded because no disability was identified for the participant or participants. A second limitation is that we did not analyze trends in the key variables. This is the case because the data are pulled from only seven volumes for each journal, making any analysis of trends tenuous at best. In addition, when the data were plotted by volume (year), no clear or conspicuous trends were apparent. Clearly, future analyses of publication records would be richer with analyses of trends (e.g., Dunlap et al., 1999). It is important to stress that the data presented in this article do not pertain to the quality or significance of intervention research published in JPBI or the other two journals. The analyses reported do not include effect sizes, methodological rigor, or practical impact, and the data do not permit inferences regarding the degree of


Clarke, Dunlap / Analysis of Intervention Research in JPBI 71

eventual contributions to the discipline of positive behavior support. Finally, the current data should not be interpreted to represent anything beyond a partial record of JPBI’s brief publication history. JPBI does not represent PBS or any part of PBS, just as no other journal can fully represent the discipline of its title. The contents of any journal depend upon the manuscripts that are submitted and all the factors that enter into the production of relevant research, choices related to submissions, and the peer review process. Still, JPBI is presently the only journal dedicated to dissemination of PBS research and practice and, as such, may be scrutinized in terms of the depiction and evolution of PBS research. The current data are encouraging in that certain features of PBS are evident more commonly in JPBI than in comparable journals.

Note 1. Complete definitions and scoring procedures for these variables are presented in Dunlap, Clarke, and Steiner (1999) and can be obtained by contacting the authors.

References Carr, E. G., Dunlap, G., Horner, R. H., Koegel, R. L., Turnbull, A. P., Sailor, W., et al. (2002). Positive behavior support: Evolution

of an applied science. Journal of Positive Behavior Interventions, 4, 4–16. Clarke, S., Dunlap, G., & Stichter, J. P. (2002). Twenty years of intervention research in emotional and behavioral disorders: A descriptive analysis and a comparison with research in developmental disabilities. Behavior Modification, 26, 659–683. Dunlap, G., Clarke, S., & Steiner, M. (1999). Intervention research in behavioral and developmental disabilities: 1980 to 1997. Journal of Positive Behavior Interventions, 1, 170–180. Dunlap, G., & Koegel, R. L. (1999). Welcoming editorial. Journal of Positive Behavior Interventions, 1, 2–3. Horner, R. H., Dunlap, G., Koegel, R. L., Carr, E. G., Sailor, W., Anderson, J., et al. (1990). Toward a technology of “nonaversive” behavioral support. The Journal of the Association for Persons with Severe Handicaps, 15, 125–132.

Shelley Clarke, MA, BCBA, is a faculty member in the Division of Applied Research and Educational Support at the University of South Florida. Her current research interests include applied research, positive behavior support, early intervention, family support, and content analyses reflecting current intervention research trends. Glen Dunlap, PhD, is a research professor at the University of South Florida. His interests include early intervention, family support, autism and developmental disabilities, and positive behavior support. Address: Shelley Clarke, MHC #2114 DARES, University of South Florida, 13301 N. Bruce B. Downs Blvd., Tampa, FL 33620; e-mail: [email protected]
