EDUCATION AND TREATMENT OF CHILDREN Vol. 31, No. 4, 2008
Training Elementary School General Educators to Implement Function‑Based Support Tyler L. Renshaw, Lynne/e Christensen, Michelle Marchant, and Tara Anderson Brigham Young University Positive Behavior Support Initiative Abstract Function‑based support (FBS) is an intervention strategy aimed at decreasing problem behaviors and increasing replacement behaviors via functional behavioral assessment and behavioral intervention planning. Although FBS is considered best practice for behavioral interventions in school se/ings, it is not mandated for students without disabilities and students with unidentified disabilities exhibiting problem behavior. Thus, many of these students receive inadequate interventions or none at all. Considering this context, this study investigated the effects of training elementary school general educators to independently implement FBS as a prereferral intervention. Results from four participating teachers and students suggest this may be a pragmatic and moderately effective practice. Confounds and limitations, as well as future directions for research and practice, are explored herein.
F
unction‑based support (FBS) is a strategic approach for decreasing problem behaviors and increasing replacement behaviors. It evolved from the tenets of applied behavior analysis (e.g., Baer, Wolf, & Risley, 1968, 1987) and is typically comprised of three consecutive processes: (a) conducting functional behavioral assessment (FBA), (b) utilizing FBA products to develop a behavior support plan (BSP), and (c) implementing and monitoring the BSP (Tobin, 2005). For FBS to be effective each process must be implemented; however, there is no consensus about the particular procedures comprising each process (Fox & Gable, 2004; Weber, Killu, Derby, & Barre/o, 2005). To date, lack of procedural uniformity has not limited the ef‑ fectiveness of FBS. Using varying procedures, researchers have dem‑ onstrated its effectiveness with individuals from preschool‑age to adulthood, with mild to severe disabilities, in clinical, community, and school se/ings (Umbreit, Ferro, Liaupsin, & Lane, 2007; e.g., Hughes, Alberto, & Fredrick, 2006; Liaupsin, Umbreit, Ferro, Urso, & Upreti, 2006). Furthermore, FBS appears to improve problem behavior more Correspondence to: Lynne/e Christensen, BYU Positive Behavior Support Initiative MCKB 301, Provo, UT 84602; e‑mail: lynne/
[email protected].
Pages 495–521
496
RENSHAW et al.
efficiently than comparable comprehensive, technically sound, non‑ function based approaches (e.g., Ingram, Lewis‑Palmer, & Sugai, 2005; Newcomer & Lewis, 2004). As a result, FBS may be considered best practice for behavioral interventions across se/ings and popula‑ tions (Iwata et al., 2000). Considering FBS to be best practice, the Individuals with Dis‑ abilities Education Improvement Act (IDEIA; U.S. Department of Education, 2004) requires that it be implemented in schools for stu‑ dents with disabilities who violate a code of student conduct and are thus at risk for placement change, and suggests it be considered for students with disabilities whose behavior impedes the teaching and learning processes. Researchers and practitioners agree that these two legal provisions are empirically beneficial (Fox & Gable, 2004); nevertheless, they do not account for all student problem behavior. Because IDEIA is solely applicable to students with disabilities who are identified as needing special education services (U.S. Department of Education), it is not intended to meet the needs of students with‑ out disabilities and students with unidentified disabilities who exhibit problem behavior. Problem behavior exhibited by students without disabilities and students with unidentified disabilities is usually less intense but more frequent than problem behavior exhibited by students with dis‑ abilities (Liaupsin, Jolive/e, & Sco/, 2004). Although these students are typically under‑identified by school discipline systems, teachers report that their high frequency problem behaviors are as disrup‑ tive to the teaching and learning process as the problem behaviors exhibited by students with severe disabilities (Liaupsin et al., 2004). Considering this, it is estimated that 5% to 15% of general education students are currently exhibiting mild problem behaviors but receiv‑ ing no special education or intervention related services (e.g., Walker et al., 1996). Without intervention, these mild problem behaviors can escalate in severity, placing these students at‑risk for several negative outcomes (e.g., Lane, Mahdavi, & Borthwick‑Duffy, 2003; Lane & We‑ hby, 2002). Thus, to reduce classroom disruption and prevent negative outcomes, several researchers suggest that such students receive FBS to decrease their problem behaviors (e.g., Gresham, Watson, & Skin‑ ner, 2001; Lane, Weisenbach, Li/le, Phillips, & Wehby, 2006; Umbreit et al., 2007). Instead of receiving FBS, students without disabilities and stu‑ dents with unidentified disabilities are most likely to receive support for their problem behaviors via prereferral intervention (e.g., Lane, Givener, & Pierson, 2004). The aim of prereferral intervention is to intervene in student problem behavior using means other than special education (Trusco/, Cohen, Sams, Sanbron, & Frank, 2005). Although
FUNCTION‑BASED SUPPORT TRAINING
497
prereferral intervention processes resemble FBS, they are typically not function‑based and have been implemented with only varying degrees of success (Lane et al., 2003). Thus, in response to this inad‑ equacy, researchers have recently investigated means for providing FBS as a prereferral intervention for at‑risk students (e.g., Lane et al., 2006; McDougal, Nastasi, & Chafouleas, 2006). Preliminary evidence suggests that elementary school general educators may be capable of providing FBS as a means of prerefer‑ ral, behavioral intervention for at‑risk students in their classes. Three recent studies support this practice, but each has limited generaliz‑ ability. First, Moore et al. (2002) trained a first, a fourth, and a filh grade teacher to each implement a function‑based intervention for one at‑risk student. Although the intervention was implemented in‑ dependently by each teacher, the FBAs were conducted and the inter‑ ventions were developed by the investigators. Results indicated that all teachers implemented the intervention conditions with high fidel‑ ity. But because no data were collected on student outcomes or social validity, the effectiveness, practicality and feasibility of these interven‑ tions are unknown. In another study, conducted by Maag and Larson (2004), a filh grade teacher was trained to independently implement all FBS pro‑ cesses with two at‑risk students. Following training, the teacher in‑ dependently conducted an FBA, developed a BSP, collected baseline data, implemented the BSP, and collected intervention data for both students. Results indicated significant decreases in both students’ tar‑ geted behaviors. Additionally, the teacher’s perception of FBS as an intervention, assessed via the Intervention Rating Profile for Teachers (IRP‑15; Martens, Wi/, Elliot, & Darveaux, 1985), was very favorable. Although this study succeeded at facilitating student behavior change, the generalizability of the results is limited due to lack of social valid‑ ity measures—specifically, no data were collected on teachers’ percep‑ tions of the practicality and feasibility of including the training within their typical school duties. Most recently, Lane et al. (2006) reported that two second grade teachers collaborated with a graduate student liaison to successfully implement FBS as a prereferral intervention for one at‑risk student in each of their classes. The main procedures of the FBA process were implemented primarily by the liaison, who collaborated with the teachers only to determine the functions of the problem behavior. During the BSP development, implementation, and monitoring pro‑ cesses, the teachers independently selected the intervention method and collaborated with the liaison to design and implement interven‑ tion procedures. Results indicated a functional relationship between
498
RENSHAW et al.
intervention implementation and decreases in the target behavior for both students. IRP‑15 results indicated that both teachers perceived FBS favorably as an intervention. Although this study succeeded at facilitating student behavior change, the teachers’ actual contributions to the FBS processes were olen ambiguous. Specifically, it is unclear how much they contributed when determining the function of the target behavior and designing and implementing intervention pro‑ cedures. This lack of specificity limits the possibility of replication. Again, these results also have limited generalizability because no so‑ cial validity data were collected on the practicality and feasibility of including the training and implementation methodology within the teachers’ typical school duties. Collectively, these three studies indicate that it may be possible for elementary general educators to independently implement FBS as a prereferral intervention. To date, Maag and Larson’s (2004) study is the best indicator of this. Thus, in an a/empt to build capacity in more teachers, the primary research question of this study was concerned with replicating Maag and Larson’s intent on a larger scale: What are the effects of training a group of elementary school general educators to independently implement all FBS processes as a prereferral inter‑ vention? Furthermore, researchers have recommended that this line of research pursue a balance between empirical rigor and social validity (Sco/ et al., 2004). If it is important for general educators to acquire the skills to implement FBS, it is then critical that these skills be ac‑ quired and implemented in a practical and feasible manner—or they will never be used. Thus, the secondary research question of this study was: What are the social validity (i.e., practicality and feasibility) out‑ comes of training elementary school general educators to implement FBS within their typical school duties? Method Se#ing This study was conducted at a suburban, Title I elementary school in Utah. At the time of the study the school employed thirty general educators. Of its 868 enrolled students, 75% were Caucasian, 20% were Hispanic, and 5% were of other ethnic groups; approxi‑ mately 18% were identified with disabilities; 15% were English lan‑ guage learners; and 40% received free or reduced price lunch. Participants Selection method. Recruitment involved one of the investiga‑ tors presenting a synopsis of the study in a faculty meeting. General
FUNCTION‑BASED SUPPORT TRAINING
499
educators were informed that participation would require nominat‑ ing a student meeting the following criteria: (a) exhibiting behavior disruptive to teaching or learning processes, (b) not receiving special education or intervention‑related services, and (c) not learning Eng‑ lish as a second language. Because of the time limitations of the study, and in an a/empt to facilitate a simple and direct implementation of FBS, general educators were discouraged from selecting their most difficult student. If they desired to participate, all general educators were invited to meet with one of the investigators aler school the fol‑ lowing day. Out of 30 total general educators, 13 (3 kindergarten, 3 first grade, 2 second grade, 2 third grade, and 3 filh grade) teachers volunteered to participate. A stratified random sampling procedure was used to select participants from the 13 volunteers. The volunteers’ names were sorted by grade level, and one name was randomly selected from each level, with the last participant (another first grade teacher) being ran‑ domly selected from the eight remaining names. Each participant nominated a student who met the specified criteria, and student as‑ sent and parental consent were obtained for each student. Because of external conflicts and time demands unrelated to this study, the third and filh grade teachers terminated their participation early. Mrs. Walker and Andrew. Mrs. Walker was a Caucasian, first‑year first grade teacher with a bachelor’s degree in Elementary Education and no prior teaching experience. She nominated Andrew, a Hispanic student on free or reduced price lunch, for exhibiting ina/entive be‑ haviors during direct literacy and math instruction. She reported that his behavior interfered with instruction and distracted his peers. Mrs. Oliver and Dominic. Mrs. Oliver was a Caucasian, first grade teacher with 15 years of previous teaching experience—across first, second, third, and fourth grades—and a bachelor’s degree in Elemen‑ tary Education. She nominated Dominic, a Caucasian student on grade level for reading and math, for participation in the study. Dominic was selected by his teacher for exhibiting ina/entive behaviors during whole‑group spelling and reading instruction. She reported that his behavior interfered with instruction and distracted his peers. Mrs. Anderson and Danny. Mrs. Anderson was a Caucasian, first‑ year kindergarten teacher with four years of experience teaching preschool, a bachelor’s degree in Early Childhood Education, and an English as a Second Language endorsement. She nominated Danny, a Caucasian student, on grade level for reading and math, for exhibit‑ ing off‑task behaviors during independent work periods. She report‑ ed that his behavior prevented him and his peers from completing tasks.
500
RENSHAW et al.
Mrs. Bailey and Landon. Mrs. Bailey was a Caucasian, second grade teacher who had had three years of previous teaching experi‑ ence and a bachelor’s degree in Elementary Education, and was com‑ pleting a master’s degree in Educational Leadership. She nominated Landon, a Hispanic student on free or reduced price lunch, who was above grade level in math and reading for intervention. Landon was selected for exhibiting off‑task behaviors during any combination of instruction and independent work periods. She reported that his be‑ havior prevented him and his peers from completing tasks. FBS Training, Materials, and Implementation Training content, materials, and implementation methodology were adapted and modified from a recently published text developed by Umbreit et al. (2007). Training consisted of three core components: (a) group training, (b) independent reading and applied activities, and (c) individual consultation. Unlike other methodologies, in which training is completed prior to FBS implementation (e.g., Maag & Larson, 2004), this methodology required general educators to dem‑ onstrate successful implementation of one process (e.g., conducting FBA) before being trained in the next process (e.g., developing a BSP). Thus, training and implementation were interactive and training was considered to be in process throughout the study. This approach to training and implementation was utilized to ensure quality skill ac‑ quisition. Group training. Four hour‑long training sessions were conducted aler school by one of the investigators, over a ten week period. Each training session covered the rationale, principles, and procedures of the FBS processes: Session 1 covered conducting FBA, Sessions 2 and 3 covered developing a BSP, and Session 4 covered implementing and monitoring the BSP. Independent readings and applied activities. Each general educator was assigned ten readings (one to five pages in length) from Umb‑ reit et al.’s (2007) text. These readings reviewed important principles covered in the training sessions and served as a primer for the in‑ dependent applied activities. The applied activities required general educators to respond to review questions, implement FBS procedures with nominated students, and report and summarize findings. These activities were completed at a rate of one per week. The applied activities were forma/ed as 10 skills‑based proce‑ dures: (1) identifying and defining the target behavior, (2) conducting teacher and student interviews, (3) conducting antecedent‑behavior‑ consequence (ABC) observations, (4) identifying behavioral function, (5) identifying and defining the replacement behavior, (6) selecting
FUNCTION‑BASED SUPPORT TRAINING
501
an intervention method, (7) selecting a data collection method, (8) de‑ veloping a BSP, (9) collecting and graphing data, and (10) implement‑ ing and monitoring the BSP. Implemented consecutively, these proce‑ dures guided general educators through all three FBS processes. To ensure fidelity, individualized rubrics were created for each activity delineating the expected format and content of the required data. At the conclusion of each activity, these rubrics were a/ached to the respective applied activity assignment sheets and submi/ed to one of the investigators for evaluation. For each activity, 100% mastery of the material was required before progressing to subsequent applied activities. Aler being evaluated, satisfactory assignments were imme‑ diately returned to the teacher, to serve as a reference in completing subsequent activities. Unsatisfactory assignments were returned with wri/en feedback, and correction, resubmission, and a satisfactory re‑ evaluation were required before progressing in training. Individual consultation. All general educators received two pri‑ vate consultation sessions, conducted aler school and lasting 5 to 15 minutes, depending on the teacher’s needs. During the first consulta‑ tion session, one of the investigators reviewed the teacher’s perfor‑ mance on the four applied activities comprising FBA and responded to questions about training content thus far. The second consultation session was structured as the first, but focused on the four applied activities comprising BSP development. FBS Knowledge Test The FBS Knowledge Test was created by the investigators to di‑ rectly assess the teachers’ knowledge of the rationale, principles, and procedures comprising the three FBS processes throughout the study. This test consisted of 30 multiple‑choice items and each item had four possible answer choices. Item content was derived from Umbreit et al.’s (2007) text and was aligned with material presented in the group trainings. The item prompts were one to four sentences long and took one of two formats: direct questions about rationale, principles, and procedures; or applied questions about the use of particular principles and procedures in context. An example of each item format (direct and then applied, respectively) is listed below: What is the function of baseline? a. To demonstrate a relationship between the ABC’s of behavior b. To operationally define the problem behavior c. To accurately predict and describe data before intervening d. To show experimental control The teacher gives an assignment to the student. The student
502
RENSHAW et al. finds the assignment challenging and kicks and screams. The teacher then tells the students that he doesn’t need to do the assignment. This is an example of what? a. Positive reinforcement b. Punishment c. Negative reinforcement d. Response cost
Each of the three FBS processes were represented by 10 items on the test and items were arranged in random order, so the teachers did not know which items were associated with which FBS processes. Tests were scored by one of the investigators, according to an answer key; each item was weighted equally as one point; and three scores were reported (one for each section) for each test. For each test ad‑ ministration, the mean score for all general educators was taken and graphed for each of the three sections. During weeks that group train‑ ing was held, tests were administered to the teachers before or aler the training. In the interim weeks, tests were self‑administered by the teachers and returned to the investigators. Variables and Experimental Designs The effectiveness of the training and implementation methodol‑ ogy was evaluated in two phases: first, by teachers’ knowledge gains on the FBS Knowledge Test (Phase I); second, by students’ behavioral outcomes (Phase II). Phase I. The independent variable in the first phase was the multi‑component FBS training. The dependent variable was the FBS Knowledge Test scores—reflecting the extent to which general educa‑ tors acquired and maintained knowledge of the rationale, principles, and procedures of the three FBS processes. These variables were ac‑ counted for via a multiple‑baseline design, pa/erned aler Miller and Weaver’s (1972) modified multiple‑baseline achievement test. In this study, each condition represents a section of training in one of the three FBS processes (that aligned to a section of items on the FBS Knowl‑ edge Test). In the baseline condition, each general educator took the FBS Knowledge Test prior to receiving training. In the following three conditions, the same test was taken repeatedly aler receiving training in one of the three FBS processes. The average scores for all general educators were graphed and compared across the 12‑week duration of the study. Phase II. The behavior support plans, developed using functional behavioral assessment and implemented by each teacher (see Table 1), were the independent variables for this phase. The dependent
FUNCTION‑BASED SUPPORT TRAINING
503
variables were either the replacement or target behaviors of each stu‑ dent—depending on which the teacher chose to observe—measured according to the data collection method selected by each teacher (see Table 1). While testing for a functional relationship was an initial aim of the study, time limitations, due to the end of the school year, made it impossible to do so. Therefore, an A‑B design was selected for the following reasons: to provide general educators with a brief, uncom‑ plicated means for evaluating the effectiveness of their interventions (Alberto & Troutman, 2006), and to investigate the variables of interest within a limited time frame. FBS Products: FBA and BSP Development The purpose of this study was to provide general educators with the requisite skills to enable them to conduct a FBA and to develop and implement a BSP at the prereferral level. Thus, the products from the FBA and BSP development processes represent the teachers’ initial efforts in these areas. Regarding these products, it is noteworthy that the teachers conceptualized replacement behaviors broadly, perceiv‑ ing them as behaviors serving the same function as the target behav‑ ior either directly (e.g., raising one’s hand to get the teacher’s a/en‑ tion instead of shouting) or indirectly (e.g., receiving teacher praise, or a/ention, for si/ing quietly). As such, each replacement behavior definition (see Table 1) is comprised of multiple options, allowing for either, or both, direct and indirect replacement of target behaviors. The general educators gradually developed BSPs via the afore‑ mentioned applied activities. Activities 1‑7 laid the foundation for these plans. Activity 8 required the teachers to compile all of the per‑ tinent information from prior activities onto a single BSP form, as well as generate specific procedures for implementing the interven‑ tion: detailing antecedent adjustments; noting when and how to teach replacement behaviors; identifying reinforcers, contingencies, and schedules; and stating extinction strategies. All intervention proce‑ dures were generated independently by the teachers—following the third training—to facilitate maximum practicality and feasibility in their classrooms. Aler the BSP form was successfully evaluated (ac‑ cording to the respective rubric), the teachers implemented their in‑ terventions. A brief overview of the main components of the teacher generated functional behavioral assessments and behavior support plans are detailed in Table 1. Observers, Data Collection, and Interobserver Agreement Observers. The four general educators, along with four under‑ graduate students from the local university, were observers in this
PR – A/ention
Ina/entive Behavior:
Talking only to participate in class
Entire Class: Reinforcer Ignore exchange inappropriate behavior of other students
Praise note at end of class
Verbal
Praise
Reinforcement Adjustment
Hands and feet to self
Reminder of expectations
Antecedent Adjustment
Student: Review of replacement behavior
Adjust the contingencies
Intervention Method
BSP
Si/ing quietly, facing teacher, eyes on teacher
A/entive Behavior:
Replacement Behavior
Note: PR = Positive Reinforcement; NR = Negative Reinforcement
Touching, whispering to and making faces at peers
Clicking tongue
Humming
Function
Target Behavior
Teacher/Student: Walker & Andrew FBA
Data Collection Method
Teacher and Rate students ignore target behavior
Extinction
Table 1 Overview of Functional Behavioral Assessment Results and Behavior Support Plan Components
504 RENSHAW et al.
Playing with non‑ instructional items
Talking only to participate in classroom discussion
Looking at the teacher
Turning around to look at or make faces at peers
A/entive Behavior:
Si/ing in learning position
PR – A/ention
Ina/entive Behavior:
Replacement Behavior
Talking to peers during instruction
Function
Target Behavior
Teacher/Student: Oliver & Dominic FBA
Improve the environment
Intervention Method
BSP Reinforcement Adjustment
Reminder to student of reinforcement possibilities
Reminder to class Wri/en of expectations
Moved seating to Praise close proximity with teacher Verbal
Antecedent Adjustment Teacher ignoring behavior
Extinction Rate
Data Collection Method
FUNCTION‑BASED SUPPORT TRAINING 505
Talking to peers only to receive help
Looking at and/or working on the task at hand
NR – Activity
Playing with objects on desk Using materials inappropri‑ ately
On‑task:
PR – A/ention
Off‑task: Talking to peers
Replacement Behavior Adjust the contingencies
Intervention Method
BSP
Function
Target Behavior
Teacher/Student: Anderson & Danny FBA
Reminder of reinforcement possibilities
Reminder of expectations
Antecedent Adjustment
Get out of a Center free card
Reinforcer exchange
Tokens
Verbal
Praise
Reinforcement Adjustment
Not included in plan
Extinction
Duration
Data Collection Method
Overview of Functional Behavioral Assessment Results and Behavior Support Plan Components (continued)
Table 1
506 RENSHAW et al.
Not looking at task at hand
Talking to peers only for task‑ related help
Looking at teacher during instruction
Leaving seat to talk to peers or talking to peers
On‑Task Behavior:
Adjust the contingencies
Intervention Method
BSP
Following along with teacher and class in the morning workbook
PR – A/ention
Off‑Task Behavior:
Replacement Behavior
Playing with instructional and non‑ instructional objects
Function
Target Behavior
Teacher/Student: Bailey & Landon FBA
Reminder to student of expectations
Antecedent Adjustment
Reinforcer exchange
Tokens
Verbal
Praise
Reinforcement Adjustment Not included in plan
Extinction Duration
Data Collection Method
FUNCTION‑BASED SUPPORT TRAINING 507
508
RENSHAW et al.
study. Each general educator served as an observer during baseline and intervention conditions for her nominated student, and each ob‑ servation lasted approximately 15 minutes. However, to prevent ob‑ server bias and facilitate fidelity, two undergraduates served as pri‑ mary observers during both conditions and two served as secondary observers (for interobserver agreement). Data collection. As part of the training and implementation pro‑ cesses, each general educator selected the data collection method she perceived as most feasible to implement within the classroom context. Mrs. Walker and Mrs. Oliver chose to record the target (ina/entive) behavior and selected rate recording as the most feasible data collec‑ tion method. Their rationale for choosing to measure the target behav‑ ior was that it would be easier to measure during intervention, as it would be occurring less olen than the replacement behavior. For each observation, both teachers carried a clipboard, a data collection form (which included the teacher‑defined operational definition of the tar‑ get behavior), and a pencil. They noted the times the observation be‑ gan and ended; they recorded tally marks, while teaching, each time the target behavior was observed; and, at the end of each observation session, they hand‑graphed each data point. Mrs. Anderson and Mrs. Bailey chose to record the replacement (on‑task) behavior and selected duration recording as the most feasible data collection method. Their rationale for selecting duration record‑ ing was that they perceived it as the most feasible method to use dur‑ ing classroom instruction. For each observation, both teachers used a stopwatch: they would press ‘start’ when the replacement behavior was observed, press ‘stop’ when the target behavior was observed, and press ‘start’ again when the replacement behavior resumed. At the end of each observation session, they hand‑graphed each data point. The undergraduate observers were trained, by one of the inves‑ tigators, in rate and duration data collection methods. To ensure com‑ parability of the observations, these observers used the same materi‑ als, procedures, and operational definitions of the target/replacement behaviors as defined and used by the teachers. Interobserver agreement. It was assumed that data collected by the general educators would differ from—and likely underrepresent—the data collected by the undergraduate observers. Thus, interobserver agreement (IOA) was assessed only for data collected by the pairs of undergraduate observers. For baseline and intervention, IOA was cal‑ culated by dividing the number of agreements by the number of agree‑ ments plus disagreements X 100. IOA data were collected for 45% of the sessions across both conditions. For Danny’s target behavior, the
FUNCTION‑BASED SUPPORT TRAINING
509
mean IOA across sessions was 95%, with a range of 93% to 98%; for Andrew’s, the mean was 95%, with a range from 85% to 100%; for Dominic’s, the mean was 91%, with a range from 85% to 100%; and for Landon’s, the mean was 92%, with a range from 86% to 96%. Treatment Fidelity Treatment fidelity data were collected during each intervention session by the general educators and independent observers. Using a checklist comprised of the unique intervention components for each student (designed by each classroom teacher), the observers placed check marks next to the components that were implemented each ses‑ sion. To prevent interference with the collection of student outcome data, all observers marked the checklists immediately following each session. Social Validity and Feasibility At the completion of the study all general educators completed the FBS Training and Implementation Social Validity Questionnaire. This questionnaire was designed by the investigators to assess the practicality and feasibility of the training and implementation meth‑ odology used in this study. It was comprised of 16 items: 11 items assessed the teachers’ level of agreement on a six‑point Likert scale (“strongly agree” to “strongly disagree”) to positive statements about the training and implementation methodology, and 5 free‑response items inviting feedback or querying for a rationale of an agreement item. Each item was assessed independently across teachers, and a total social validity score was derived for each teacher by summing the agreement items. Results Phase I: General Educators’ FBS Knowledge Gains General educators’ knowledge gains regarding FBS rationale, principles, and procedures were assessed throughout the study via the FBS Knowledge Test. The baseline scores for each test section and the baseline composite score are presented below (see 1). Following this, pre‑ and post‑training scores for each section, as well as the post‑ training composite score, are presented. All scores are presented as percentages of correct responses and represent the combined average performance of all general educators. Baseline. On average, the general educators answered 63% of the questions from Section 1 correctly, 58% of the questions from Section 2 correctly, and 63% of the questions from Section 3 correctly dur‑ ing the baseline administration (see Figure 1). The average baseline
510
Figure 1. General educatior’s FBS knowledge gains
RENSHAW et al.
FUNCTION‑BASED SUPPORT TRAINING
511
composite score was 61%. Section 1: Conducting FBA . Before receiving training in this area, the general educators answered 63% of the questions in this section correctly (i.e., in the baseline test). Aler receiving training in this area, the general educators answered an average of 94% (with a range of 88% to 98%) of the questions in this section correctly (i.e., in Tests 1 to 12), an improvement of 31%. Section 2: Developing a BSP. Before receiving training in this area, the general educators answered an average of 64% (with a range of 58% to 70%) of the questions in this section correctly (i.e., in baseline to Test 4). Aler receiving training in this area, the general educators answered an average of 85% (with a range of 75% to 90%) of the ques‑ tions in this section correctly (i.e., in Tests 5 to 12), an improvement of 21%. Section 3: Implementing and Monitoring the BSP. Before receiving training in this area, the general educators answered an average of 64% (with a range of 58% to 68%) of the questions in this section cor‑ rectly (i.e., in baseline to Test 8). Aler receiving training in this area, the general educators answered an average of 83% (with a range of 80% to 85%) of the questions in this section correctly (i.e., in Tests 9 to 12), an improvement of 19%. Post‑training composite score. Aler the general educators received training in each of the three FBS processes, their average composite score was 89% (with a range of 88% to 90%) for the remaining admin‑ istrations of the test (i.e., Tests 9 to 12), an improvement of 28%. Phase II: Student Outcomes Outcome data was collected by the general educators and un‑ dergraduate observers via baseline and intervention observations for each student. Because the teachers conducted observations while per‑ forming typical classroom duties, and considering that interobserver agreement was not calculated for data derived from their observa‑ tions, their data are considered less reliable, less accurate, and thus less representative of actual student outcomes. However, their data is presented here for comparison purposes: to highlight the varying degrees of accuracy among the general educator participants. Teacher and observer outcome data are presented as averages for each student during each condition. See Figure 2 for graphical representations of the outcome data. Andrew. During baseline, Mrs. Walker observed an average of 0.47 (with a range of 0.33 to 0.63) occurrences of the target behavior per minute per session, while the independent observer recorded an average of 1.06 (with a range of 1.00 to 1.25) occurrences of the target
512
RENSHAW et al.
Figure 2. Each graph represents a teacher‑student dyad. The dashed line represents the mean for observer data and the do/ ed line represents the mean for teacher data in each graph.
FUNCTION‑BASED SUPPORT TRAINING
513
behavior per minute. During intervention, Mrs. Walker observed an average of 0.19 (with a range of 0.1 to 0.28) occurrences of the target behavior per minute per session, while the independent observer re‑ corded an average of 0.23 (with a range of 0.00 to 0.45) occurrences of the target behavior per minute—an improvement of 0.83. Thus, while the independent observer noted approximately twice as many occur‑ rences of the target behavior during baseline, the differences in obser‑ vations during intervention were negligible. Dominic. During baseline, Mrs. Oliver observed an average of 0.47 (with a range of 0.33 to 0.76) occurrences of the target behavior per minute per session, while the independent observer recorded an average of 1.07 (with a range of 1.00 to 1.10) occurrences of the target behavior per minute. During intervention, Mrs. Oliver observed an average of 0.49 (with a range of 0.27 to 0.84) occurrences of the target behavior per minute per session, while the independent observer re‑ corded an average of 0.75 (with a range of 0.53 to 1.23) occurrences of the target behavior per minute—an improvement of 0.32. Although the teacher’s and independent observer’s baseline observations appear disparate, their intervention observations illustrate a similar trend. Danny. During baseline, Mrs. Anderson observed Danny exhib‑ iting the replacement behavior an average of 68% (with a range of 34% to 79%) of each session, while the independent observer recorded an average of 70% (with a range of 40% to 82%) of each session. During intervention, she observed Danny exhibiting the replacement behav‑ ior an average of 85% (with a range of 80% to 96%) of each session, while the independent observer recorded an average of 86% (with a range of 77% to 97%) of each session—an improvement of 16%. Thus, for Danny the differences between the teacher and independent ob‑ servations were negligible. Landon. During baseline, Mrs. Bailey observed Landon exhibit‑ ing the replacement behavior for an average of 94% (with a range of 93% to 99%) of each session, while the independent observer recorded an average of 83% (with a range of 64% to 91%). During intervention, Mrs. Bailey observed him exhibiting the replacement behavior for an average of 98% (with a range of 96% to 99%) of each session, while the independent observer recorded an average of 89% (with a range of 82% to 97%)—an improvement of 6%. It appears that the teacher’s observations actually overstate Landon’s baseline and intervention performance by approximately 10%. Treatment fidelity. Across all intervention sessions, all general educators self‑reported 100% treatment fidelity, and the independent observers reported the same.
514
RENSHAW et al.
Social Validity The FBS Training and Implementation Social Validity Question‑ naire was designed to assess the practicality and feasibility of the methodology used in this study. Scores below 43 represented an un‑ favorable view, scores from 44‑55 represented a favorable view, and scores from 56‑66 represented a very favorable view of the method‑ ology. Total questionnaire scores are presented below, followed by a delineation of responses for each item. Total questionnaire scores. Total scores indicate that Mrs. Walker, Mrs. Oliver, and Mrs. Bailey (scoring 53, 50, and 48, respectively) each had a favorable perception of the FBS training and implementation methodology used in this study. These three teachers responded to all items by marking either slightly agree or agree. The total score for Mrs. Anderson (58) indicates that she had a very favorable perception of this methodology. She responded to one question by marking slightly agree and the others by marking either agree or strongly agree. Individual item results. All general educators indicated some lev‑ el of agreement to all but one item. All agreed that the FBS training helped them to improve their student’s problem behavior and that such training would be beneficial for all elementary general educa‑ tors. Additionally, they all agreed that the group trainings, reading as‑ signments and applied activities, and brief consultation sessions were beneficial. Moreover, all agreed that the time required to participate in each level of training and apply the procedures in the classroom was feasible within their typical school duties and time limitations. Fur‑ thermore, all agreed that these time requirements would be feasible for most other elementary general educators. The one item lacking unanimous agreement from all general educators was: “I feel that I would be willing to spend more time learning this material, if need‑ ed.” In response to this item, two general educators marked slightly disagree and two marked agree. Responses to the five free‑response items varied among the gen‑ eral educators. When queried about the most beneficial part of the training sequence, each responded differently: one found learning how to define target behaviors most beneficial; another found more meaning in learning how to identify behavioral function; another learned more from instruction on how to design an intervention; and the fourth, more generally, reported that learning “how to effectively manage problem behavior” was most beneficial for her. When asked to offer a rationale for the level of agreement they indicated on items regarding time feasibility, all teachers noted that the training would have been more beneficial and feasible if conducted earlier in the school year.
FUNCTION‑BASED SUPPORT TRAINING
515
Discussion This study investigated two research questions regarding the application of FBS to improve student problem behavior: (1) What are the effects of training a group of elementary school general educators to independently implement all FBS processes as a prereferral inter‑ vention? (2) What are the social validity (i.e., practicality and feasibil‑ ity) outcomes of training such teachers to implement FBS within their typical school duties? Findings and Conclusions The effects of training a group of elementary school general educators to independently implement all FBS processes were evalu‑ ated via teachers’ knowledge gains (Phase I) and student behavioral outcomes (Phase II). Additionally, the social validity outcomes were evaluated via results from the FBS Training and Implementation Questionnaire. Effects of training: Phase I. Substantial and consistent gains on the FBS Knowledge Test suggest that the streamlined, skills‑based train‑ ing and methodology was an effective means for facilitating knowl‑ edge acquisition and maintenance of knowledge in FBS rationale, principles, and procedures. It is noteworthy, however, that score in‑ creases following training were differential, not uniform, across sec‑ tions. One possible explanation for this is that FBS rationale, princi‑ ples, and procedures vary in complexity and therefore difficulty of acquisition. Taking this explanation, it is possible that the complexity of such knowledge increases as the three FBS processes progress—re‑ sulting in less robust increases in Knowledge Test scores over time. Effects of training: Phase II. Student behavioral outcomes indi‑ cate that two students made moderate gains and two relatively small gains, but all gains were positive. Lack of large gains may be due to the fact that teachers chose students with less severe behavior prob‑ lems (i.e., they did not have as much room to improve). We suggest that the size of these gains may be accounted for via methodological confounds, discussed later herein. With this caveat, the unanimous positive outcomes suggest that the FBS training and implementation methodology used in this study may facilitate moderately positive student outcomes at the prereferral level. Social validity outcomes. Results from the FBS Training and Imple‑ mentation Social Validity Questionnaire indicate that general educa‑ tors’ perceptions of the training and implementation methodology used in this study were favorable and highly favorable. Because the outcomes on this particular questionnaire were so positive, we suggest that this methodology strikes an appropriate balances between em‑
516
RENSHAW et al.
pirical rigor and social validity (as suggested by Sco/ et al., 2004) and therefore may be a legitimate model for successfully training future general educators. Furthermore, considering that two general educa‑ tors expressed reluctance to spend more time acquiring such skills, we caution against increasing the complexity or time involvement of this methodology—as it may lower its practicality and feasibility. Confounds and Limitations As aforementioned, we suggest that two extraneous factors may have confounded students’ behavioral outcomes. Furthermore, we posit that a primary limitation hinders the generalizability of all re‑ sults and the general scientific rigor of this study. Higher performing students. First, participating general educators were encouraged not to nominate their most difficult student. This request was made in consideration of time limitations (with the end of the school year approaching), as well as to make the teachers’ first a/empts at implementing FBS less complex. However, it appears Mrs. Anderson and Mrs. Bailey selected students who were already per‑ forming well at the baseline level (see Figure 2). Thus, it is likely that only minimal improvements were observed during intervention be‑ cause these students were already performing at or above an average on‑task level. In future studies, this confound may be remedied by allowing teachers to select their most difficult students or requiring that baseline observations demonstrate legitimate excesses or deficits in target behavior prior to nominating a student for intervention. Limited intervention time. Second, the general educators’ inter‑ vention implementation time was limited. This confound was a di‑ rect result of real time limitations due to the end of the school year. By the time the general educators had conducted an FBA, developed a BSP, and collected baseline data, they had an average of only five school days to implement the intervention under normal classroom conditions. Thus, it is possible that more robust behavioral outcomes may have been observed for Andrew and Dominic (see Figure 2) if the interventions had persisted longer. This confound may be easily remedied in the future by conducting the study earlier in the school year. Beyond confounds, however, this study also has a primary limi‑ tation. Lack of experimental control. The primary limitation of this study is the use of an A‑B research design to account for the relationship between the teacher‑implemented BSPs and the students’ behavior‑ al outcomes. This design resulted in a lack of experimental control and therefore an inability to demonstrate a true functional relation‑ ship between the variables. However, despite the lack of experimen‑
FUNCTION‑BASED SUPPORT TRAINING
517
tal control, studies using A‑B designs can still contribute important and useful findings (Cooper, Heron, & Heward, 2007). Specifically, use of an A‑B design in this study provided a brief, uncomplicated means for teachers to evaluate the effectiveness of their interventions by identifying the possible correlations between the variables of in‑ terest (as suggested by Alberto & Troutman, 2006, and Cooper et al., 2007). Indeed, visual inspection of the outcome data suggests positive correlations—two moderate and two minimal—between the teacher‑ implemented interventions and students’ behavioral outcomes. While the A‑B designs in this study do not allow for determination of a func‑ tional relationship, they do demonstrate replication. The social valid‑ ity outcomes corroborate these findings, as all teachers agreed that FBS improved their student’s problem behavior and that intervention implementation and data collection methods were practical and fea‑ sible. These findings are important, despite the lack of ability to dem‑ onstrate a functional relationship, because they highlight the investi‑ gators’ aims in selecting an A‑B design. Specifically, the investigators posited that—given the research questions and the context within which the study was conducted—this design would facilitate an ap‑ propriate balance between social validity and empirical rigor (as sug‑ gested by Sco/ et al., 2004), and it appears to have done so. In sum, the design was purposefully matched to study conditions, not vice versa. Thus, we suggest that, at the preferral level, an A‑B design could be an efficient and effective means for teachers to quickly evaluate the effec‑ tiveness of a BSP. Nevertheless, it must be emphasized that, without true experimental control, these positive findings must be considered preliminary and may be considered premature. Considering this, we suggest that future research investigate the practicality and feasibility of more complex research designs in similar contexts. Considerations for Future Research and Practice In culmination, we offer two suggestions for researchers to con‑ sider as they pursue further investigations to improve this practice. Improve data collection methods and technologies. First, we suggest that researchers target means for improving data collection methods and technologies for general educators who are independently imple‑ menting FBS amidst typical school duties. Despite moderate student outcomes in this study, the data do indicate something noteworthy: although exact data points collected by general educators and inde‑ pendent observers differed, the general trends in the data are similar (when disregarding outliers). Furthermore, when the individual data points do differ significantly, data collected by the general educators
518
RENSHAW et al.
tended to understate the amount of behavioral change (i.e., Andrew’s and Dominic’s data). Because the general educators in this study col‑ lected data while simultaneously participating in class or individual‑ ized instruction, it appears reasonable that they would observe fewer exhibitions of target or replacement behaviors and therefore under‑ state behavioral outcomes. Thus, future investigations should seek to develop and validate more feasible methods and technologies to en‑ sure more accurate assessment. Establish a standardized FBS training methodology. Second, as a general guiding recommendation, we reiterate the ideas of previ‑ ous researchers (e.g., Maag & Larson, 2004; Sasso, Conroy, Stichter, & Fox, 2001) and suggest that future studies aim to refine and estab‑ lish a standardized methodology for training general educators to independently implement FBS—especially for at‑risk students at the prereferral level. In doing this, we suggest that the overall aim of this line of research should be to develop and validate a practical, feasible, scientifically sound, best‑practice methodology. This study and Maag and Larson’s study represent initial efforts in this direction, but the findings are preliminary and limited. Therefore, refinement and rep‑ lication are needed using these methodologies, or new and enhanced methodologies could be developed and investigated. Summary Although FBS is considered best practice for behavioral interven‑ tion across populations and se/ings, there are no legal requirements mandating the provision of FBS to at‑risk students in general educa‑ tion se/ings. Thus, this study was undertaken to explore the capabil‑ ity of elementary school general educators to provide FBS as a means of prereferral intervention for such students. Results from four teach‑ ers and four students indicate that the FBS training and implementa‑ tion methodology used in this study has the potential to: (a) increase elementary general educators knowledge of FBS rationale, principles, and procedures; (b) facilitate moderately positive student behavioral outcomes; and (c) accomplish these aims in a practical and feasible manner amidst teachers’ typical school duties. Considering the small percentage of school personnel currently trained in FBS methodology, we suggest that it is imperative that future investigations seek to ad‑ vance this emerging practice for general educators. Furthermore, we emphasize that future inquiries must continually seek to strike a bal‑ ance between empirical rigor and social validity—or this practice will lose its purpose and may never be adopted.
FUNCTION‑BASED SUPPORT TRAINING
519
Acknowledgment The authors wish to thank John Umbreit for sharing ideas and materials in the early planning stages of this study. This research was supported by a mentoring grant from the Office of Research and Cre‑ ative Activities at Brigham Young University. References Alberto, P. A., & Troutman, A. C. (2006). Applied behavior analysis for teachers. Upper Saddle River, NJ: Pearson Education, Inc. Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimen‑ sions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91‑97. Baer, D. M., Wolf, M. M., & Risley, T. R. (1987). Some still‑current di‑ mensions of applied behavior analysis. Journal of Applied Be‑ havior Analysis, 20(4), 313‑327. Cooper, J. O, Heron, T. E., & Heward, W. L. (2007). Applied behavior analysis (2nd ed.). Upper Saddle River, NJ: Pearson Education. Fox, J. J., & Gable, R. A. (2004). Functional behavioral assessment. In R. B. Rutherford, M. M. Quinn, & S. R. Mathur (Eds.), Handbook of research in emotional and behavioral disorders (pp. 143‑162). New York: The Guilford Press. Gresham, F. M., Watson, T. S., & Skinner, C. H. (2001). Functional be‑ havioral assessment: Principles, procedures, and future direc‑ tions. School Psychology Review, 30(2), 156‑172. Hughes, M. A., Alberto, P. A., & Fredrick, L. L. (2006). Self‑operated auditory prompting systems as a function‑based intervention in public community se/ings. Journal of Positive Behavior Inter‑ ventions, 8(4), 230‑243. Ingram, K., Lewis‑Palmer, T., & Sugai, G. (2005). Function‑based inter‑ vention planning: Comparing the effectiveness of FBA func‑ tion‑based and non‑function‑based intervention plans. Jour‑ nal of Positive Behavior Interventions, 7(4), 224‑236. Iwata, B. A., Wallace, M. D., Kahng, S. W., Lindberg, J. S., Roscoe, E. M., Conners, J., et al. (2000). Skill acquisition in the implemen‑ tation of functional analysis methodology. Journal of Applied Behavior Analysis, 33(2), 181‑194. Lane, K. L, Givner, C. C., & Pierson, M. R. (2004). Teacher expectations of student behavior: Social skills necessary for success in el‑ ementary school classrooms. The Journal of Special Education, 38(2), 104‑110.
520
RENSHAW et al.
Lane, K. L., Mahdavi, J. N., & Borthwick‑Duffy, S. A. (2003). Teacher perceptions of the prereferral intervention process: A call for assistance with school‑based interventions. Preventing School Failure, 47(4), 148–155. Lane, K. L., & Wehby, J. H. (2002). Addressing antisocial behavior in the schools: A call for action. Academic Exchange Quarterly, 6(2), 4‑9. Lane, K. L., Weisenbach, J. L., Li/le, M. A., Phillips, A., & Wehby, J. (2006). Illustrations of function‑based interventions imple‑ mented by general education teachers: Building capacity at the school site. Education and Treatment of Children, 29(4), 549‑ 571. Liaupsin, C. J., Jolive/e, K., & Sco/, T. M. (2004). Schoolwide systems of behavior support. In R. B. Rutherford, M. M. Quinn, & S. R. Mathur (Eds.), Handbook of emotional and behavioral disorders (pp. 487‑501). New York: The Guilford Press. Liaupsin, C. J., Umbreit, J., Ferro, J. B., Urso, A., & Upreti, G. (2006). Improving academic engagement through systematic, func‑ tion‑based intervention. Education and Treatment of Children, 29(4), 573‑591. Maag, J. W., & Larson, P. J. (2004). Training a general education teach‑ er to apply functional behavioral assessment. Education and Treatment of Children, 27(1), 26‑36. Martens, R. K., Wi/, J. C., Ellio/, S. N., Darveaux, D. X. (1985). Teacher judgments concerning the acceptability of school‑based inter‑ ventions. Professional Psychology: Research and Practice, 16(2), 191‑198. Miller, L. K., & Weaver, F. H. (1972). A multiple baseline achievement test. In G. Semb (Ed.), Behavior analysis & education (pp. 393‑ 399). Lawrence, KS: University of Kansas. McDougal, J. L., Nastasi, B. K., & Chafouleas, S. M. (2006). Bringing research into practice to intervene with young behaviorally challenging students in public school se/ings: Evaluation of the behavior consultation team (BCT) project. Psychology in the Schools, 42(5), 537‑551. Moore, J. W., Edwards, R. P., Sterling‑Turner, H. E., Riley, J., Dubard, M., & McGeorge, A. (2002). Teacher acquisition of functional analysis methodology. Journal of Applied Behavior Analysis, 35(1), 73‑77. Newcomer, L. L., & Lewis, T. J. (2004). Functional behavioral as‑ sessment: An investigation of assessment reliability and
FUNCTION‑BASED SUPPORT TRAINING
521
effectiveness of function‑based interventions. Journal of Emo‑ tional and Behavioral Disorders, 12(3), 168‑181. Sasso, G. M., Conroy, M. A., Stichter, J. P., & Fox, J. J. (2001). Slow‑ ing down the bandwagon: The misapplication of functional assessment for students with emotional or behavioral disor‑ ders. Behavioral Disorders, 26(4), 282‑296. Sco/, T. M., Bucalos, A., Liaupsin, C. J., Nelson, C. M., Jolive/e, K., DeShea, L., et al. (2004). Using functional behavioral assess‑ ment in general education se/ings: Making a case for effec‑ tiveness and efficiency. Behavioral Disorders, 29(2), 189‑201. Tobin, T. J. (2005). Function‑based support in home, school, and commu‑ nity. Eugene, OR: Author. Trusco/, S. D., Cohen, C. E., Sams, D. P., Sanborn, K. J., & Frank, A. J. (2005). The current state(s) of prereferral intervention teams: A report from two national surveys. Remedial and Special Edu‑ cation, 26(3), 130‑140. Umbreit, J., Ferro, J. B., Liaupsin, C. J., & Lane, K. L. (2007). Func‑ tional behavioral assessment and function‑based intervention: An effective, practical approach. Upper Saddle River, NJ: Pearson Education, Inc. U.S. Department of Education. (2004). Individuals with Disabilities Ed‑ ucation Improvement Act 2004. Retrieved May 31, 2007 from: h/p://idea.ed.gov Walker, H. M., Horner, R. H., Sugai, G., Bullis, M., Sprague, J. R., Bricker, D., et al. (1996). Integrated approaches to preventing antisocial behavior pa/erns among school‑age children and youth. Journal of Emotional and Behavioral Disorders, 4(4), 194‑ 209. Weber, K. P., Killu, K., Derby, K. M., & Barre/o, A. (2005). The status of functional behavioral assessment (FBA): Adherence to stan‑ dard practice in FBA methodology. Psychology in the Schools, 42(7), 737‑744.
522
2@