learn to master the code with intensive ... in the intervention program to the lan- guage of instruction in the core reading program. First-grade students whose.
Effectiveness of Spanish Intervention for First-Grade English Language Learners at Risk for Reading Difficulties Sharon Vaughn, Sylvia Linan-Thompson, Patricia G. Mathes, Paul T. Cirino, Coleen D. Carlson, Sharolyn D. Pollard-Durodola, Elsa Cardenas-Hagan, and David J. Francis
Abstract The effectiveness of an explicit, systematic reading intervention for first-grade students whose home language was Spanish and who were at risk for reading difficulties was examined. Participants were 69 students in 20 classrooms in 7 schools from 3 districts who initially did not pass the screening in Spanish and were randomly assigned within schools to a treatment or comparison group; after 7 months, 64 students remained in the study. The intervention matched the language of instruction of their core reading program (Spanish). Treatment groups of 3 to 5 students met daily for 50 min and were provided systematic and explicit instruction in oral language and reading by trained bilingual intervention teachers. Comparison students received the school’s standard intervention for struggling readers. Observations during core reading instruction provided information about the reading instruction and language use of the teachers. There were no differences between the treatment and comparison groups in either Spanish or English on any measures at pretest, but there were significant posttest differences in favor of the treatment group for the following outcomes in Spanish: Letter-Sound Identification (d = 0.72), Phonological Awareness composite (d = 0.73), Woodcock Language Proficiency Battery–Revised Oral Language composite (d = 0.35), Word Attack (d = 0.85), Passage Comprehension (d = 0.55), and two measures of reading fluency (d = 0.58–0.75).
R
esearch with native English speakers suggests that those at risk for reading difficulties make significant progress when they are provided with systematic and explicit interventions in reading (Fletcher & Lyon, 1998; O’Connor, 2000; Snow, Burns, & Griffin, 1998; Torgesen, Mathes, & Grek, 2002; Vellutino et al., 1996). This research has influenced public policy, assessment, early intervention, and reading instruction for students at risk for reading problems. Although many issues regarding reading interventions for monolingual English students at risk for reading problems require further study (e.g., sequencing of instruction, effects over time), we know a good deal about the effectiveness of interventions for these students. Vellutino et al. (1996) examined the effectiveness of an intervention aimed at very poor first-grade
readers. Tutors provided either one or two semesters (depending on progress) of 20-min daily, one-to-one tutoring in letter identification, phoneme awareness, and word reading skills. The results revealed that the majority of these students became average readers. Torgesen et al. (1999) found that daily one-to-one intervention for 88 hr enabled most of the first graders who were in the bottom 10% for reading ability to move into the average range. Similarly, Mathes et al. (2005) demonstrated that all but 2% of children could attain reading levels within the average range by the end of first grade when high-quality classroom instruction and intensive small-group intervention were paired. Similar findings for the effectiveness of early interventions for monolingual students at risk for reading problems have been reported by others (O’Connor, 2000; Sim-
mons, Kame’enui, Stoolmiller, Coyne, & Harn, 2003). In summary, native English speakers at risk for reading difficulties benefit from intensive, smallgroup instruction that focuses on building skills in phonemic awareness, orthographic processing, phonics and decoding, fluency, vocabulary, and comprehension (Ball & Blachman, 1991; Bradley & Bryant, 1983; Byrne & Fielding-Barnsley, 1991; Foorman, Francis, Novy, & Liberman, 1991; Foorman & Torgesen, 2001; Lundberg, Frost, & Peterson, 1988; Swanson, Hoskyn, & Lee, 1999). Furthermore, these interventions can reduce the gap between current student performance and performance of typically achieving peers in reading. Distinctly missing from these syntheses of effective interventions for students with reading difficulties is an understanding of the effectiveness of
JOURNAL OF LEARNING DISABILITIES VOLUME 39, NUMBER 1, JANUARY/FEBRUARY 2006, PAGES 56–73
VOLUME 39, NUMBER 1, JANUARY/FEBRUARY 2006
interventions for English language learners who are at risk for reading problems. To illustrate, the National Reading Panel stated, “The panel did not focus on special populations such as children whose first language is other than English and children with learning disabilities” (National Reading Panel, 2000, p. 4-2). Although it is often assumed that much of what is known about teaching reading to native English speakers applies to teaching reading to English language learners (e.g., Gersten & Jimenez, 1998; Goldenberg, 1998, 2001), there are still gaps between what we know about monolingual readers and what we know about bilingual readers— particularly for students at risk for reading difficulties. Spanish is the language spoken by the largest population of English language (EL) learners in the United States. Literacy skills that are significant predictors of later reading success and response to instruction are similar for English and Spanish, and include skills in phonological processing (BravoValdivieso, 1995; Carrillo, 1994; Defior & Tudela, 1994; González & Garcia, 1995; González & Valle, 2000), decoding skills (Bravo-Valdivieso, 1995; Lindsey, Manis, & Bailey, 2003; Signorini, 1997), and verbal activities (BravoValdivieso, 1995). Basic segmenting ability is important in the beginning stages of literacy acquisition, but by first grade, phoneme manipulation is a better predictor (Carrillo, 1994), with some forms of phoneme awareness developing after the onset of reading instruction. There are also strong correlations between phonological skills in Spanish and English (English, Leafstedt, Gerber, & Villaruz, 2001). Given these similarities, there is reason to assume that effective reading instruction will share many characteristics in both English and Spanish. However, syntheses on effective reading instruction for EL learners have revealed few empirical intervention studies with students with reading difficulties in Spanish. Thus, many unanswered questions with respect to early reading instruction remain, particularly for students
with reading problems (August & Hakuta, 1997; Fitzgerald, 1995a, 1995b; Gersten & Baker, 2000a, 2000b).The importance of assisting struggling beginning readers in Spanish to become competent Spanish readers cannot be overstated. Literacy instruction contributes to the development of foundation skills that lead to proficient literacy skills in Spanish, which can later transfer to English literacy (SavilleTroike, 1984). Proficient Spanish readers transfer phonological awareness skills (Quiroga, Lemos-Britton, Mostafapour, Abbott, & Berninger, 2002) and comprehension skills to English reading (Jiménez, 1994; Jiménez, Garcia, & Pearson, 1996). Thus, it is important that effective interventions for students who have difficulties learning to read in Spanish be identified. Experimental studies of the effectiveness of intensive intervention in early reading with EL learners at risk for reading problems who are learning to read in Spanish are needed, to provide reliable information about effective practice. Educators have limited knowledge about the effectiveness of early interventions for EL learners that could guide them in providing interventions to reduce the number of students who are later identified with reading problems and even long-term reading disabilities. Contributing to the lack of research on EL learners at risk for reading difficulties is the challenge of determining whether the reading difficulty with beginning readers is due to literacy difficulties, language difficulties, or other difficulties (Lundberg, 2002).
Pedagogical and Conceptual Framework The framework for the intervention used in this study reflects (a) the research on effective interventions for students with reading difficulties who are English speakers; (b) the phonology of the Spanish writing system, in which letter–sound correspondence is predictable and apparent (Carreiras, Perea, & Granger, 1998; Cuetos, 1993;
57
Signorini, 1997); and (c) the fact that Spanish has many more multisyllabic words and fewer monosyllabic words than other alphabetic languages, such as English and French. Decoding is used as an instructional strategy primarily at the sound and syllable level, due to the syllabic structure of the language (Honig, Diamond, & Gutlohn, 2000). Spanish-speaking students, like monolingual English speakers, learn to read through phonological recoding and spelling–sound patterns (Lopez & Greenfield, 2004; Signorini, 1997; Treiman, 1984). Although reading instruction in Spanish tends to focus on the syllable as a unit, students who have difficulties in learning to read in Spanish also benefit from instruction at the phoneme level (González, González, Monzo, & Hernandez-Valle, 2000; Signorini, 1997). In general, less skilled students who are learning to read in alphabetic languages have difficulties because they have not mastered the alphabetic principle (Paulesu et al., 2001), so that initial intervention instruction addresses individual sounds and syllables. Simultaneously, instruction addresses sight recognition, a strategy often used when teaching students to read irregular words in English (Ehri & Wilce, 1983; Goswami, 1993; Rayner, Foorman, Perfetti, Pesetsky, & Seidenberg, 2001). Sight words or highfrequency words are introduced prior to reading them in stories. Because students who are struggling readers often learn to master the code with intensive intervention, but fail to acquire the comprehension skills needed to understand and profit from text reading, the intervention is also aligned with current research on developing vocabulary and comprehension (Beck, McKeown, & Kucan, 2002; Fitzgerald, 1995a; Gersten & Baker, 2000a, 2000b; Snow, 2002; Ulanoff & Pucci, 1999). The instructional design principles are based on the converging research on the benefits of explicit and systematic instruction in beginning reading that provide high opportunities for student response with teacher feedback. Students are engaged in reading text very
58
JOURNAL OF LEARNING DISABILITIES
early (after 7 lessons), and high-level expository text is used to increase language, vocabulary, and comprehension of text. The framework has compatible interwoven elements that include building skills in the alphabetic principle from beginning decoding (sound to letter; sound to syllable), to regular and irregular word reading, to sentences and longer texts (stories and small books), combined with ongoing instruction in vocabulary and comprehension taught daily through expository text. Our framework guides pedagogical decisions and includes provisions for teaching phonemic awareness, phonemic decoding skills, fluency in word recognition and text processing, construction of meaning, vocabulary, spelling, and writing (see Foorman & Torgesen, 2001; National Reading Panel, 2000; Pressley, 1998; Rayner, Foorman, Perfetti, Pesetsky, & Seidenberg, 2001; Snow et al., 1998). Instruction in these areas needs to be explicit. It also appears that for some students, instruction must be intensive to facilitate adequate reading development. By intensive instruction, we mean that students are highly engaged in learning critical content and that the ratio of teachers to students is relatively small.
Study Purpose We were interested in how an intervention in Spanish would influence outcomes in Spanish reading and in English and Spanish oral language skills. This study was designed to control for effects of the language of instruction by matching the language of instruction in the intervention program to the language of instruction in the core reading program. First-grade students whose parents and schools selected Spanish as their language for core reading instruction were eligible for inclusion in this study. We established two other criteria for selecting students for this study. First, schools were identified for inclusion in the study if their overall ratings based on state tests indicated
that they were successfully teaching the majority of their students to read. We prioritized effective schools because we were interested in determining the effectiveness of intervention programs in contexts that supported literacy acquisition. Second, students eligible for the intervention needed to be significantly at risk for reading problems. We established criteria for identifying students at risk for reading problems in first grade based on their reading performance in the language in which they were being taught to read (i.e., Spanish). Specifically, we examined the effectiveness of an explicit, systematic reading intervention program in Spanish for first-grade EL learners (Spanish/ English) who were at risk for reading difficulties. EL learners who were significantly at risk for later reading difficulties were randomly assigned within each school to either (a) researcherprovided daily intervention, for 50 min per day, 5 days per week (October through April), or (b) a comparison group not provided with intervention by the research team. All EL learners were maintained within their core reading instruction, which was provided in Spanish. Observations during core reading instruction provided contextual information about the reading instruction and language use of the teachers providing core reading instruction to our target students.
Method Participants School Sites. This intervention study was part of an overall program project investigating bilingual literacy and oral language skills in EL learners (Spanish/English). This intervention study occurred at three sites in Texas that were selected because they were representative of the population areas where large numbers of EL learners go to school (border district, large urban district, and middle-size urban district). We purposely selected seven schools within these districts that were
considered effective for EL learners using the following a priori selection criteria: (a) Schools were participating in a transitional bilingual model; (b) at least 60% of the school population was Hispanic; and (c) schools’ state-level reading achievement test at third grade indicated that 80% or more of the students passed the state-level reading test. The average percentage of Hispanic students across these seven schools ranged from 63% to 100%, and 0% to 29% of students were African American; generally, the proportion of European American students was lower than the proportion of African American students. The average English language learner (EL) population in kindergarten and Grade 1 ranged from 77% to 100%. Four of the seven schools had state ratings of exemplary, and three were rated as recognized. For these schools, pass rates on the thirdgrade State Bilingual Reading Assessment ranged from 87.5% to 99% (third grade is the first grade when this assessment is administered). All schools participated in the free or reducedprice lunch program, and the majority of the schools had more than 90% of students qualify. Books in the school libraries that were printed in Spanish ranged from 5% to 75%. Student Participants. Students in each of the seven schools were selected through a screening completed by all first-grade students at the beginning of the school year. The Spanish screening consisted of two subtests: (a) the Letter–Word Identification (LWID) subtest from the Woodcock Language Proficiency Battery (Spanish; described later), and (b) the first five words from an experimental word reading list (in Spanish) used to assess initial word reading ability (see Note). The first five words were the easiest and consisted of two- to four-letter words. Criteria for inclusion into the intervention were determined as (a) a score below the 25th percentile for the first grade on the LWID subtest, and (b) inability to read more than one of the simple words from the word list.
VOLUME 39, NUMBER 1, JANUARY/FEBRUARY 2006
A total of 361 students were administered the Spanish screening at the seven target schools. Of these, 73 (20%) met the intervention inclusion criteria; however, 4 of these 73 students withdrew or transferred from their schools prior to randomization for treatment and comparison conditions. The 69 students who remained were matched and then randomly assigned within their schools to treatment or comparison groups. The composition of the randomized groups changed for 2 intervention students whose schedules could not be accommodated, and these were replaced by their matched pair. We analyzed primary results both with and without these 4 students, and the results were not substantively different. Therefore, the results presented throughout the rest of this article include all children who did (treatment group) and did not (comparison group) receive the intervention. This study began with 35 treatment students and 34 comparison students and ended with 31 treatment students and 33 comparison students (11% and 3% attrition, respectively, due to students’ leaving the school). The mean age of the final sample (N = 64) at pretest was 6.60 years (SD = 0.37). All students were Hispanic, and 45% of the students (n = 31) were girls. Classroom Teachers. The 64 children came from 21 first-grade classrooms across the seven schools. The 21 bilingual teachers (20 female, 1 male) who provided the core reading instruction in Spanish to these students averaged 11.7 years of teaching experience (SD = 8.4) with most teachers having taught first grade for an average of 9.2 years (SD = 7.2). Overall, 80% (n = 17) had credentials as bilingual teachers, providing instruction in the primary language (Spanish), and 8 were certified to teach English as a second language.
Measures Prior to the onset of intervention (pretest; October) and following its completion (posttest; May), all students
were assessed using a comprehensive battery of language- and literacyrelated measures in both Spanish and English. Of the students completing the posttests, two treatment students completed an assessment in Spanish but refused to complete the assessment in English. Students were assessed in both languages because, when determining the testing battery, we were unsure as to how much time the classroom-based reading program would actually be provided in the designated language, Spanish. Previous observation reports had revealed that teachers often provided instruction in both languages or switched to English after the first half of first grade. Also, it seemed reasonable to think that several of the foundation skills in reading (e.g., phonological awareness and letter naming) might have effects in Spanish that also generalized to English. Letter Naming and Sound Identification. Children were asked to identify each of the 26 letters of the English alphabet and each of the 30 letters of the Spanish alphabet. Children were also asked to provide at least one sound for each of the 26 letters of the English alphabet and at least one sound for each of the 30 letters of the Spanish alphabet. These measures were not timed. Dependent measures were the raw score totals for each measure. Comprehensive Test of Phonological Processing. The Comprehensive Test of Phonological Processing (CTOPP; Wagner, Torgesen, & Rashotte, 1999) has nine subtests measuring phonological awareness (PA), rapid naming (RN), and phonological memory (PM). The normative base was similar to the 1997 U.S. Census statistics. Coefficient alpha reliabilities for all three composites in the normative sample ranged from .83 to .95, and from .83 to .92 in the age range of this sample; test–retest estimates in a small sample (n = 32) of children ages 5 to 7 ranged from .70 to .92 for the three composites. Furthermore, content, concurrent, predictive, and construct validity data are provided in
59
the CTOPP manual (Wagner et al., 1999). Seven subtests of the CTOPP were used, including Elision, Blending Words, Blending Nonwords, Segmenting Words, Sound Matching (First Sound and Last Sound), Nonword Repetition, and Rapid Letter Naming (Form A or B). Although age-based standard scores are available for the CTOPP, raw scores were used in the analyses to compare performance with a Spanish-language version of this instrument (see next section). A phonological awareness (PA) composite score was created from the CTOPP subtest scores of Sound Matching, Blending Words, Blending Nonwords, Segmenting Words, and Elision, as these were the five subtests that are categorized by the CTOPP as PA subtests. The composite was an average of these performances, corrected for the number of items in each subtest (percentage correct). Where the Sound Matching subtest was not administered because students met performance criteria on the Blending Words subtest, a perfect score was imputed for purposes of calculating this composite (although analyses were also performed in which a composite was computed without the Sound Matching subtest, and results were highly similar). This and other subtest routing rules to reduce student frustration and testing time were derived from earlier research with the predecessor to the CTOPP and empirical modeling of performance on this test using item response methods (Schatschneider, Francis, Foorman, Fletcher, & Mehta, 1999), and from work on a measurement development aspect of a related project examining the properties of this assessment in a larger sample (n = 1,600) of EL learners. Test of Phonological Processing– Spanish. The Test of Phonological Processing, Spanish version (TOPP-S), was developed to align with the Englishlanguage CTOPP in terms of the skills addressed and the linguistic complexity of the items in each subtest, while
60
still being appropriate for the Spanish language. Each subtest consists of comparable numbers of items as those in the CTOPP. With the exception of Sound Matching, all TOPP-S subtests were built entirely of productionbased items, and items were targeted to match CTOPP items in task demands and linguistic complexity (e.g., number of phonemes and syllables, area of manipulation) but relied on phonemes and syllables appropriate for the Spanish language. Reliability estimates for the TOPP-S were determined on a sample of approximately 1,500 students, and the coefficient alphas were very high, ranging from .93 to .97. Raw scores comparable to those calculated for the CTOPP were used for data analysis; the same branching rules for the CTOPP were also used for the TOPP-S. Woodcock Language Proficiency Battery–Revised. The Woodcock Language Proficiency Battery–Revised (WLPB-R), English Form (Woodcock, 1991) was normed on a sample of 6,359 participants (3,245 in K to 12)—the same normative sample as that of the Woodcock-Johnson Psychoeducational Battery–Revised (Woodcock & Johnson, 1989). Median coefficient alphas ranged from .81 to .92 across all age ranges (and from .77 to .96 at ages 6 to 9) for the subtests used; test–retest measures for selected subtests in a sample of 504 participants ranged from .75 to .95. The WLPB-R Spanish Form (Woodcock & Munoz-Sandoval, 1995) was derived for 3,911 native Spanish-speaking individuals from 22 countries (with 1,325 from the United States and 1,512 from Mexico) who were nearly monolingual Spanish speakers; median coefficient alphas ranged from .84 to .92 across all age ranges (and from .68 to .95 at ages 6 to 9; Woodcock & MunozSandoval, 1995). The scaling process on the WLPB-R allows scores on the English and Spanish language assessments to be directly compared, in the sense that it places the Spanish language norms on the same scale as the English language norms.
JOURNAL OF LEARNING DISABILITIES
WLPB-R subtests used in this study were Letter–Word Identification (at screening only), Word Attack, Passage Comprehension, Listening Comprehension, Picture Vocabulary, Verbal Analogies, and Memory for Sentences (at pretest only). Letter–Word Identification requires the student to match a rebus to an actual picture of that object (beginning items), then to read aloud individual letters, and then to read aloud words that increase in length and complexity. Word Attack requires the student to read aloud nonsense words or unfamiliar words that are linguistically logical. Passage Comprehension first requires students to point to a picture represented by a phrase in a multiple choice format, then to read a sentence or short passage and provide a missing word that is appropriate for the context of the passage. Listening Comprehension is similar to Passage Comprehension in the oral domain; it asks the student to listen to a passage and supply the missing word at the end using an oral cloze procedure. Picture Vocabulary requires the student to name familiar and unfamiliar pictured objects and is primarily an expressive semantic task. Verbal Analogies requires a student to provide verbal answers to questions about logical relationships that increase in difficulty. Finally, Memory for Sentences requires a student to repeat phrases or sentences that increase in length. Dependent measures were age-based standard scores only, although raw scores were analyzed with similar results. Dynamic Indicators of Basic Early Literacy Skills. The Dynamic Indicators of Basic Early Literacy Skills (DIBELS; Good & Kaminski, 2002), and its Spanish version, the Indicadores Dinámicos del Éxito en la Lectura (IDEL; Good, Bank, & Watson, 2003), are reading fluency measures requiring the student to orally read a passage geared to the student’s grade level. Children were given a maximum of 3 seconds per word, and a maximum of 60 seconds for the entire passage. At pretest, the
Grade 1 beginning-of-year passage was administered, and at posttest, both the Grade 1 beginning- and end-ofyear passages were administered, in both Spanish and English. The dependent measures were the number of words read correctly.
Intervention Students met with a bilingual certified teacher in small groups of three to five students for 50 min a day, 5 days a week, from October through May. Instruction was provided at a time during the school day that did not conflict with the core reading lessons offered in the general education classroom. The six intervention teachers who delivered this small-group instruction received 12 hr of professional development from the authors of the intervention prior to implementation, and they received an additional 6 hr after 6 weeks of implementation. Teachers also participated in frequent 1- to 2-hr staff development sessions at each site, during which they (a) viewed videotaped lessons with discussion and feedback, (b) discussed issues regarding the implementation of the interventions, and (c) collaborated in problem solving to plan for accelerating the growth of specific students. The frequency of these sessions varied across the year. During the first month of implementation, teachers met weekly with site coordinators. These sessions were later reduced to only once per month, unless deemed necessary. Teachers also received frequent onsite coaching that varied from weekly to monthly depending on the needs of individual teachers. Teachers were also videotaped frequently and required to view their own tapes and critique their own teaching. Curriculum for the Intervention. In designing the reading intervention, we applied research from several sources: (a) effective reading instruction in English with native English speakers with reading disabilities and reading difficulties; (b) the sequence
VOLUME 39, NUMBER 1, JANUARY/FEBRUARY 2006
and development of Spanish literacy acquisition; and (c) principles of effective instruction for developing oral language skills. Specifically, we developed our intervention following the same instructional design principles used to create an effective beginning reading intervention for struggling native English readers (i.e., Proactive Beginning Reading; Mathes, Torgesen, Wahl, Menchetti, & Grek, 1999). The result was a curriculum (Lectura Proactiva; Mathes, Linan-Thompson, Pollard-Durodola, Hagan, & Vaughn, 2003) that was different in terms of the sequence and focus of instructional content, but similar in terms of instructional design and delivery (Carnine, Silbert, & Kame’enui, 1997). Lesson Format. Instruction was provided at a quick pace that gave students many opportunities to respond and to receive feedback. There was ongoing interchange between the instructor and the students. In a typical activity, the teacher asked all students to respond to letters or words and provided opportunities for each student to respond to demonstrate knowledge and progress. Moreover, the 50-min lessons were organized around 7 to 10 activities, promoting quick movement from one activity to the next. The teaching routine included the teacher modeling new content, providing guided practice for students, and implementing independent practice. Instructors consistently monitored students’ responses, providing positive recognition for correct responses and feedback if an error occurred. Instructional Design. Teachers provided explicit instruction following these predetermined lesson plans, with lessons organized so that various content strands (i.e., letter-sound knowledge, phonemic awareness, speeded syllable reading, word recognition, fluency, and comprehension strategies) were integrated. Alphabetic Knowledge and Skills. In a typical lesson, students
practiced previously taught letter– sound correspondences, including writing these letters, and learned the sound of a new letter. In terms of phonemic awareness instruction, students were taught in the initial lessons to segment words into phonemes and to blend phonemes back into words. These skills were then used to facilitate understanding of the sounding-out process and as a tool for spelling. Because of the syllabic nature of Spanish, teaching students to read syllables was an early focus of instruction. Within the first few lessons, students read syllables composed of previously taught letter–sound correspondences by sounding out the syllable, then reading the syllable as a whole. Within a short time, students were asked to read syllables as a unit, rather than phoneme by phoneme. In these “speeded” activities for reading syllables, the placement of vowels varied from day to day to ensure that students were processing individual phonemes within syllables rather than memorizing a specific pattern. Likewise, teaching students to decode multisyllabic words began almost immediately. The basic strategy was to read an unknown multisyllabic word syllable by syllable, then to put the syllables together to read the whole word. Initially, students sounded out syllable parts, then read the syllable, then read the whole word. Over time, students were reading multisyllabic words quickly and were decoding unknown words fast and efficiently. At the same time that students were asked to improve their decoding speed, the complexity of words that they were reading increased both in terms of length (i.e., number of syllables) and in the complexity of the syllable type (i.e., VCV, CVC, CVV, CCV). Connected Text Practice. Beginning on the seventh day of instruction, students began reading connected text daily. This text was fully decodable, meaning that all phonetic elements and high-frequency words appearing in the text had been taught previously.
61
Although this text was stilted in the beginning, as students’ ability to decode more difficult words improved, the text became richer in terms of language and story complexity. By the end of the intervention, students were reading grade-level books with complex word and sentence structures. A primary objective in the design of Lectura Proactiva (Mathes et al., 2003) was to promote text fluency. Our goal was to prepare students to read 75 words per minute correctly on gradelevel text by the end of first grade. To achieve this goal, each story was read repeatedly, requesting greater fluency after each reading. Typically, the first reading was read as a group in unison, followed by each member of the group reading a section of the story. In later lessons, teachers timed individual students on entire stories while the remaining students read in pairs. Comprehension. A second objective of connected text reading was to teach comprehension strategies. From the beginning, students were asked to make predictions or tell what they knew related to the story before reading, using a modified K-W-L procedure (Ogle, 1986). After reading a story, students were asked to retell and sequence events in the story. Students were then asked to identify story grammar elements and, later, to identify main ideas. Finally, summarization was introduced, using either story grammar for narrative text or simple content webs for expository text. Oral Skills and Vocabulary Development. Because the participating students were EL learners at risk for reading problems and with overall low language proficiency scores in both English and Spanish (see Table 1), we prioritized the development of oral language skills and vocabulary development in Spanish. Every day for 10 min, the instructors provided students with a book-reading and vocabulary activity. All instructors used the same expository books (n = 25) in Spanish, which were centered on eight informa-
62
tion themes (e.g., pets, bugs). Each theme was addressed in three or four books. The only exception was the first theme, “families,” which was a narrative theme. Books were selected based on reading at the second- to thirdgrade level and were aligned with students’ interests. Each day, two to three key vocabulary words were selected (identified for each segment of the book read that day) and were taught prior to listening to the passage from the book. Teachers read passages to the students each day and then asked questions about the vocabulary and key ideas. Teachers used probes to guide students in story retelling, providing opportunities for each student to participate. During this time, teachers did not use a direct instruction model, and students dialogued with the teacher about the story using complete sentences and new vocabulary terms. Hickman, Pollard-Durodola, and Vaughn (2004) have provided a detailed description of the oral language skills and vocabulary development intervention.
Intervention Instructors and Validity Checks All interventionists were bilingual (Spanish/English), had an undergraduate degree, were hired by the research team, provided the intervention outside of the core reading curriculum, and were well prepared to provide the intervention (see Intervention section). All but two of the intervention instructors were certified to teach elementary or EL learners. During the year, two observers, in consultation with the primary author of the intervention, worked closely to obtain interrater reliability using videotapes of bilingual intervention teachers implementing the Spanish intervention curriculum. Upon obtaining an interrater reliability of 95%, both observers conducted intervention validity checks during the beginning, middle, and end of the year, so that each instructor was observed for fidelity of implementation a total of three times.
JOURNAL OF LEARNING DISABILITIES
Interrater reliability was reestablished prior to each intervention validity check. The intervention validity instrument allowed for the collection of both quantitative and qualitative data, which focused on the following observable teacher behaviors at each observational point: (a) instructional pacing, (b) providing independent practice, (c) presenting the lesson appropriately, (d) providing error correction, (e) providing appropriate scaffolding, (f ) teaching concepts to mastery, (g) maintaining student attentiveness, and (h) eliciting student responses. Using specific guidelines, the observers assigned one of the following numerical ratings to each of the eight aforementioned areas for every activity observed: 1 = poor; the instructional behavior greatly deviated from specified guidelines; 2 = average; the instructional behavior met most but not all guidelines specified; or 3 = excellent; the instructional behavior met all guideline specifications. Field notes were also written by observers to provide further details on each of the eight aforementioned instructional behaviors. Across numerous activities and observation points, the average rating scale (maximum possible = 3) for teachers providing the intervention ranged from 1.93 to 2.97, with an overall average of 2.21 (SD = 0.85). Lower scores occurred earlier in the intervention validity checks, and teaching behaviors that contributed to these low scores were corrected. In addition to the aforementioned eight instructional categories, teachers were rated using a list of nine questions that addressed general teacher preparedness related to teaching the intervention: (a) materials ready, (b) materials visible to students, (c) students seated appropriately, (d) instructor’s enthusiasm/warmth, (e) ongoing monitoring of student performance, (f ) checking of practice items for correctness
and providing feedback, (g) redirection of off-task behavior, (h) communication of clear expectations and learning goals for activities, and (i) participation of each student during the story retelling. Each of these nine global teacher behaviors was marked as being present (yes) or not present (no). Across independent observations, instructors received an average of more than 90% yes responses. When no responses were received, appropriate support and feedback were provided to the instructor.
Core Reading Program and Classroom Observations Observation Measure. The observation schema was developed by Foorman and colleagues (Foorman, Goldenberg, Carlson, Saunders, & PollardDurodola, 2004; Foorman & Schatschneider, 2003) to record time by activity during reading instruction. Observers make on-the-minute observations of the teacher and students during reading and language arts instruction and English language development. Observations were conducted three times across the year (beginning, middle, and end of the year). Mean reliabilities were 80% or higher. Trained bilingual researchers who were not working directly on this intervention study and were unfamiliar with which students were assigned to treatment and comparison conditions collected all data. Core Reading Program. The core reading curriculum used in the large city was ¡Vamos de Fiesta! (Ada, Campoy, & Solis, 2000), supplemented with Estrellita (Myer, 1990). The core reading program in the border city was Esperanza (Hagan, 1998). The core reading program in the midsize urban setting was Lectura: Scott-Foresman (Blanco et al., 2000), supplemented with Estrellita (Myer, 1990) or ¡Vamos de Fiesta! (Ada et al., 2000). As the core reading materials provided little information about what the teachers actually taught and what lan-
63
VOLUME 39, NUMBER 1, JANUARY/FEBRUARY 2006
guage of instruction was used during core reading, the 21 teachers who provided the primary core reading instruction for our target students (treatment and comparison) were observed during their instruction time three times throughout the school year. Using the timing activity during reading schema (Foorman et al., 2004; Foorman & Schatschneider, 2003), independent observers recorded on the minute the subject and content area taught and the language used by the teacher during instruction. All content codes were grouped into the following eight categories: oral language, reading, reading comprehension, word work, writing and spelling, giving directions, providing feedback, and nonreading instruction. The total number of minutes spent on each content area and the time spent using each language were presented as a percentage of the total time observed. Based on independent observations, the amount of time that classroom teachers taught reading and language arts averaged 183 min per day (SD = 38 min). Approximately 92% of the time observed consisted of actual instruction time. During the instruction time observed, there was a relatively equal distribution of instruction across the categories of oral reading (e.g., students read aloud from either books or pages; M = 12.7%, SD = 5.3%); reading (e.g., students read silently from either books or pages; M = 15%, SD = 5%); writing/spelling (e.g., students were writing at their desks, in groups, or copying from the board or practicing spelling words; M = 13.6%, SD = 7.4%), and word work (e.g., sounding out words, reading words in isolation, reading word families; M = 14.2%, SD = 6.7%). The remaining instructional time was spent giving directions (M = 18.6%, SD = 4.4%), providing feedback (M = 4.2%, SD = 4%), or in nonreading instruction (M = 15.5%, SD = 5.2%). Although all schools provided Spanish reading instruction as the focus, a significant amount of the instruction observed was provided by
the teacher in English. On average, teachers used Spanish 54% of the time (SD = 23.9%), English 19% (SD = 20.1%), and a mix of Spanish and English 10% of the time (SD = 3.6%). Reading Intervention for Comparison Students. To determine the extent to which comparison students were provided with reading interventions (instruction additional to the core curriculum) by the schools, the students’ classroom teachers were individually interviewed by a member of our research team three times over the school year. A standardized form with specific questions was completed for each student to determine the type of additional instruction provided, if any, and the amount of time it was provided. Researchers also met with the personnel providing the intervention to document the accuracy of reporting, including the amount of time and the type of instruction provided. Of the 34 comparison students, 29 received one or more types of reading intervention in addition to their core reading instruction. The total amount of time that reading intervention was provided to these students ranged from 9 hr 2 min to 227 hr 29 min, with a mean of 83 hr 58 min (SD = 49 hr 44 min). The types of instruction provided included guided reading (n = 9), Esperanza (n = 8), Reading Recovery (n = 7), and tutoring (n = 11), among others.
Results Plan of Analysis First, preliminary analyses were conducted for all subtest scores to examine performance distributions. Next, treatment and comparison groups were compared on all dependent measures prior to the onset of intervention. The next set of analyses examined posttest performance as a function of group (intervention or comparison), controlling for pretest performance levels. Standardized effect sizes (Cohen’s d; Cohen, 1988) were computed using differences in mean performance divided
by the pooled within-group standard deviation on the unadjusted posttest score for each measure; these values were then adjusted for sample overestimation bias (Hedges & Olkin, 1985). Confidence intervals (95% limits) were calculated based on the standard error of the corrected d values. The final group of analyses was exploratory and examined the performance of treatment group students who showed positive and negative responsiveness to intervention based on their posttest performance relative to the treatment group as a whole.
Preliminary Analyses: Sample Distributions Examination of preintervention score distributions through box plots, stemand-leaf plots, and other univariate statistics indicated that the majority of students were performing at the floor on subtests assessing phonemic awareness, rapid letter naming, and wordreading-fluency skills in Spanish; a similar but more pronounced pattern was seen in the students’ English language performance. This pattern is not surprising, given the selection criteria for the study and the fact that most measures were related to reading or language skills. Average age did not differ between the treatment and comparison groups, F(1, 62) < 1, ns. Most students were age appropriate for first grade in Texas (6 years old), with only a few students being 1 year older.
Pretest Performance (Spanish and English) Pretest performance means for treatment and comparison groups are presented in Table 1. As expected, given the random assignment of students to treatment and comparison groups, there were no significant group mean differences in performance on either of the skills used in the intervention screening (WLPB-R Letter–Word Identification and experimental word reading list). Furthermore, mean comparison of skill performance on the larger
64
TABLE 1 Pretest Performance on Language and Reading Measures in Spanish and English by Treatment and Comparison Groups Spanish Measure
n
M
English
SD
n
M
SD
Letter Naming Letter-Name Identification Treatment Comparison Rapid Letter Naming (CTOPP/TOPP-S) Treatment Comparison
35 34
18.17 19.44
8.5 7.7
35 34
7.46 7.18
7.5 7.2
35 34
0.29 0.31
0.4 0.4
34 33
0.10 0.06
0.2 0.1
Phonological Processing Letter-Sound Identification Treatment Comparison Nonword Repetition (CTOPP/TOPP-S) Treatment Comparison PA Composite (CTOPP/TOPP-S)a Treatment Comparison
35 34
18.97 20.50
8.9 8.1
35 34
11.06 13.65
7.9 6.8
35 34
5.77 6.74
4.0 4.5
35 34
9.23 9.91
2.8 3.1
35 34
29.83 27.53
15.1 14.8
35 34
27.56 24.88
15.7 15.7
Language Related Listening Comprehension (WLPB-R) Treatment Comparison Picture Vocabulary (WLPB-R) Treatment Comparison Verbal Analogies (WLPB-R) Treatment Comparison Memory for Sentences (WLPB-R) Treatment Comparison
35 34
80.74 81.79
14.0 14.6
33 33
39.15 39.91
22.8 20.5
35 34
79.34 78.32
21.8 30.8
28 31
32.93 36.32
18.9 21.6
34 34
80.94 84.18
18.2 16.5
32 31
72.16 75.94
15.2 16.2
35 34
76.40 80.71
13.9 11.4
35 34
60.82 54.60
12.2 15.8
Reading Letter–Word Identification (WLPB-R)b Treatment Comparison Experimental Word Listb Treatment Comparison Word Attack (WLPB-R) Treatment Comparison Passage Comprehension (WLPB-R) Treatment Comparison Oral Reading Fluency (DIBELS BOY) Treatment Comparison
35 34
8.54 8.94
2.9 2.3
— —
— —
— —
35 34
0.26 0.24
0.4 0.4
— —
— —
— —
34 34
72.68 73.88
16.7 17.7
32 32
81.53 83.94
4.3 6.9
35 34
70.80 76.03
10.3 15.3
32 33
74.44 77.61
8.0 8.4
33 31
1.79 2.29
2.5 3.1
30 29
1.33 0.28
2.3 0.6
Note. CTOPP = Comprehensive Test of Phonological Processing (English version; Wagner, Torgesen, & Rashotte, 1999); TOPP-S = Test of Phonological Processing, Spanish version; PA = phonological awareness; WLPB-R = Woodcock Language Proficiency Battery–Revised (English version: Woodcock, 1991; Spanish version: Woodcock & Munoz-Sandoval, 1995); DIBELS = Dynamic Indicators of Basic Early Literacy Skills (English version; Good & Kaminski, 2002) or Indicadores Dinámicos del Éxito en la Lectura (Spanish version; Good, Bank, & Watson, 2003); BOY = beginning of year story. There were no pretest differences (all p s > .05) between treatment and comparison groups on any measure in either language, with the exception of DIBELS BOY scores in English (see text). TOPP-S/CTOPP subtests, Letter-Sound Identification, WLPB-R Letter–Word Identification, Experimental Word List, and DIBELS BOY data are raw scores; Rapid Letter Naming is a letters per second measure; PA Composite is an average proportion correct score (%); and the remaining WLBP-R subtest scores are standard scores. a The PA Composite is generated from the Sound Matching, Blending Words, Blending Nonwords, Elision, and Segmenting Words subtests of the TOPP-S or CTOPP. b Students were administered Letter Word Identification and Experimental Word List in Spanish only.
65
VOLUME 39, NUMBER 1, JANUARY/FEBRUARY 2006
battery administered prior to the onset of treatment indicated that students in the treatment and comparison groups performed at comparable levels on all Spanish language skills assessed; reading and language performances were approximately 1.5 to 2 SD below normative levels for both groups. Performances on subtests assessing English skills were also relatively comparable across the two groups at pretest; the only significant group difference was performance on the English DIBELS (word reading fluency), F(1, 67) = 5.93, p < .03, where the treatment group was able to read significantly more words per minute (M = 1.3) than the comparison group (M = 0.28), although clearly both values were quite low. Also, reading performances were 1 to 1.5 SD below normative levels for both groups, and language performances were 2 to 4 SD below normative levels for both groups.
Posttest Performance (Spanish) The results of posttest performance in Spanish are presented in Table 2. This table includes means, effect sizes, significance tests, and gain scores for students who had test data at both time points. Performances are discussed by area. Letter Naming and Letter Naming Fluency. Treatment and comparison students did not differ in their ability to name Spanish letters (p > .05) after adjusting for pretest performance on this measure. Performance on the TOPP-S Rapid Letter Naming subtest also did not reach significance, F(1, 61) = 3.86, p < .06. Phonological Processing. As noted in the Method section, a Phonological Awareness (PA) composite score was created from five subtests of the TOPP-S. Performance on the PA composite measure indicated that treatment group students outperformed comparison students on these measures after adjusting for pretest perfor-
mance level, F(1, 61) = 10.05, p < .003; treatment group students correctly answered an average of 63% of the items for each subtest administered, relative to 52% for comparison group students, and the standardized effect size of the difference between groups was large (d = +0.73). The results were highly similar for performance on the LetterSound Identification subtest, F(1, 61) = 12.28, p < .001, d = +0.72. On a measure of phonological memory, however, the results were different; on the TOPP-S Nonword Repetition subtest, treatment and comparison students did not differ after adjusting for pretest performance level, F(1, 61) < 1, p > .05; in fact, their means were virtually identical. Oral Language. On the WLPB-R oral language subtests, different patterns of performances were noted. For Picture Vocabulary and Verbal Analogies, there were no differences between treatment and comparison students after adjusting for pretest performance level (both ps > .05), whereas treatment group students outperformed comparison group students on Listening Comprehension after adjusting for pretest performance, F(1, 61) = 7.98, p < .007, and the effect size was moderate (d = +0.43). Reading and Academic Achievement. On the WLPB-R Word Attack subtest, there was a significant difference between groups after adjusting for pretest performance, F(1, 60) = 14.27, p < .001, such that treatment group students demonstrated a greater ability to apply phonic and structural analysis skills to pronounce phonetically regular nonsense words in Spanish, and the effect size of this difference was large (d = +0.85). Moreover, on the WLPB-R Passage Comprehension subtest, there was a strong difference between groups after adjusting for pretest performance, F(1, 60) = 8.46, p < .006, with treatment students showing greater ability to supply missing words to demonstrate comprehension in a cloze procedure; the effect size of this difference was strong (d = +0.55).
Dictation. No significant differences were noted on the WLPB-R Dictation subtest, F(1, 61) = 2.51, p > .05. Students also completed two word reading fluency stories of the DIBELS (at levels gauged to correspond to the beginning and the end of Grade 1). As expected, students appeared to read more words from the beginning-ofyear story than from the end-of-year story in general. However, treatment group students were able to more fluently decode Spanish words in context relative to comparison students, after adjusting for pretest reading level; beginning of year story, F(1, 55) = 11.59, p < .002; end of year story, F(1, 55) = 7.02, p < .02; the effect sizes of the differences between groups were large for these measures (d = +0.75 and +0.58, respectively).
Posttest Performance (English) The results of posttest performance in English are presented in Table 3, which includes means, effect sizes, significance tests, and gain scores for students who had test data at both time points. Although the posttest performances of the treatment group students across Spanish outcome measures were consistently, significantly, and meaningfully greater than those of comparison group students, few differences were observed between treatment and comparison group students on English outcome measures. In fact, across the domains of Letter Naming, Phonological Processing, and Reading and Academic Achievement, there were no differences in performance levels at posttest on any measure after adjusting for pretest performance; furthermore, the effect sizes of the nonsignificant differences that did arise were generally small and often negative. Within the English oral language domain, however, some differences were noted. At posttest, after adjusting for pretest performance levels, comparison students outperformed treatment group students on WLPB-R Listening Comprehension, F(1, 56) = 4.77, p < .04, d =
66
TABLE 2 Posttest Performance on Language and Reading Measures in Spanish by Treatment and Comparison Groups Gaina
Performance Measure
n
M
SD
d
95% CI
F
df
p
1, 61
ns
M
SD
6.61 4.21
7.8 6.8
0.72 0.53
0.5 0.5
9.26 4.55
8.6 6.6
0.45 0.30
2.3 2.0
32.87 23.47
17.4 11.9
7.26 1.24
10.5 6.8
6.26 3.36
24.1 25.5
8.97 7.39
15.8 14.7
51.07 30.84
22.8 21.7
— —
— —
33.52 20.59
11.8 18.7
32.28 20.07
13.3 15.5
— —
— —
Letter Naming Letter-Name Identification Treatment Comparison
31 33
25.29 23.70
3.7 5.7
+0.32
Rapid Letter Naming (TOPP-S) Treatment Comparison
31 33
1.04 0.84
0.3 0.5
+0.46
−0.17 to +0.82
−0.04 to +0.95
2.71
3.86
1, 61
.06
Phonological Processing Letter-Sound Identification Treatment Comparison
31 33
28.71 25.50
2.0 5.8
+0.72
Nonword Repetition (TOPP-S) Treatment Comparison
31 33
9.84 10.12
2.8 3.2
PA Composite (TOPP-S)b Treatment Comparison
31 33
62.59 51.65
14.8 14.8
−0.09
+0.73
+0.21 to +1.23
−0.58 to +0.40
+0.23 to 1.24
12.28
