The Anatomy of Retelling Scores

1 downloads 0 Views 518KB Size Report
33-34). Research documents that comprehension is central to reading (Braunger & Lewis, 2006; ... Court Phonics Kits (SRA/McGraw-Hill, 2000). She taught ...
The Anatomy of Retelling Scores: What These Scores Do (and Don’t) Reveal about Readers’ Understandings of Texts G. Pat Wilson

University of South Florida

Prisca Martens Towson University

Poonam Arya

Wayne State University

Lijun Jin

Towson University

Retellings are one means of assessing students’ comprehension of texts and a valid source of information about students’ abilities to reconstruct meaning (Barnhart, 1990). Providing the fewest constraints on readers, they are the most authentic means of assessing comprehension, and, thus, frequently used in research studies to evaluate readers’ understandings of what they read (Goodman, 1996). Nevertheless, how retellings are analyzed and scored varies from study to study. Methods of scoring retellings described in the literature include checklists of items corresponding to parts of the story (Bui, 2002), story propositions listed and rated for their importance (Lipson, Mosenthal, & Mekkelsen, 1999), elements of story structure mentioned in the retelling (Morrow, Sisco, & Smith, 1992), analysis of the units of thought revealed in the retelling (Feathers, 2002), holistic scores representing the richness and completeness of the retelling (Irwin & Mitchell, 1983), and the use of retelling guides that include character recall and development as well as story events (Goodman, Watson, & Burke, 2005). Information on why a particular method is used and how the retellings are scored often are not specified (Golden & Pappas, 1990). In addition, studies rarely contextualize the method theoretically, leaving readers of the study to infer that information. In their study of retelling procedures, Golden and Pappas (1990) identify the issue this creates: Claims about comprehension/recall of text are based upon how the texts and retelling/recall protocols are analyzed…In studies utilizing retelling or recall, the assessment of text use (i.e., how it is comprehended or remembered) is based upon an examination of how a particular reader’s reconstructed text corresponds to the original text. What a researcher elects to score in the retelling or recall protocol is dependent upon how the text structure has been defined and what the researcher believes about the reader’s contributions to, or interpretations of, the text. The final score for each protocol in a study represents (either solely or in combination with other measures) what the researcher perceives the reader understands or remembers about the text (pp. 33-34). Research documents that comprehension is central to reading (Braunger & Lewis, 2006; Flurkey & Xu, 2003; RAND Reading Study Group, 2002; National Reading Panel, 2000). If retelling scores represent researchers’ perspectives of readers’ comprehension, understanding the impact of retelling scoring protocols on the representations of readers’ comprehension is critical. We located no studies that have examined this impact. 362

Retelling Scores

363

This study investigates three retelling protocols with contrasting perspectives of retellings to discover what the protocols reveal and do not reveal about readers’ comprehension of texts. The protocols are Morrow’s (2005) Sense of Story Structure; Goodman, Watson, and Burke’s (2005) Reading Miscue Inventory: Construction of Meaning; and Irwin and Mitchell’s (1983) Holistic Scoring. We begin with the methodology of our study.

Method The retellings we analyzed were part of a larger study of the impact of texts with contrasting features on second graders’ readings and retellings (Martens, Arya, Wilson & Jin, 2005). The students in the study attended a school in a large metropolitan area on the east coast. Classrooms were homogeneously grouped based on the county curriculum benchmark tests, teacher judgment, and standardized tests. For our current inquiry, we analyzed the retellings of the 13 students who read and retold Flossie and the Fox (McKissick, 1986). Their teacher, Ms. Standish, graciously accepted us into her classroom and identified these readers, who were tracked to enter the gifted program in third grade, as the strongest readers in her class of 25. We decided Flossie and the Fox (McKissick, 1986) would be a good match for them as challenging but not frustrating text. The ethnic composition of these 13 students was 77% white and 13% African American. Six students were male and 7 were female. The students spoke English proficiently and were not receiving special education services. Ms. Standish used Houghton Mifflin Reading: A Legacy of Literacy curriculum (Houghton Mifflin, 2000) for reading instruction during a 90 minute literacy block in the morning as well as Open Court Phonics Kits (SRA/McGraw-Hill, 2000). She taught comprehension skills in the Houghton Mifflin theme book, but retellings were not part of the reading curriculum. In this report, we draw on the readings and retellings of Flossie and the Fox (McKissack, 1986) (Flossie). We selected Flossie for the larger study because it was an authentic piece of literature (authored trade-book), had unique features (i.e., text structure, dialect, and figurative language), and was unfamiliar to these students. Fountas and Pinnell (2001) leveled Flossie as late third/early fourth grade. For a synopsis of the story, see Appendix A. Data Collection We collected the data in one-on-one sessions (researcher and student) that were audio-taped and transcribed. Before reading Flossie, we told the students, “I will be asking you to tell me the story after you read it.” In the unaided retelling, the students shared, without being interrupted, what they remembered. In the aided retelling, we asked questions but avoided giving information about the text. These questions were re-phrasings of the readers’ comments or general prompts, such as “Can you tell me more about…?” (Goodman et al., 2005). We coded and scored each retelling according to three retelling evaluation methods: Morrow’s (2005) Sense of Story Structure (Story Structure), Goodman et al.’s (2005) Reading Miscue Inventory: Construction of Meaning (RMI), and Irwin and Mitchell’s (1983) Holistic Scoring (Holistic). We constructed retelling guides according to the procedures of the respective researcher(s). For example, to prepare for the Story Structure protocol, we parsed the story events into setting, theme, plot episodes, and resolution. We then assigned points for each part. According to the procedures,

364

National Reading Conference Yearbook, 56

up to four points are awarded for setting (introduction, characters, and time/place); one point for theme; one for plot episodes; two for resolution and two for sequence for a possible total score of ten points. Appendix B is a sample for Flossie. For the RMI procedure, we constructed a retelling guide that included character analysis (40 points) and story events (60 points) (see Appendix C). For narrative texts, character analysis generally includes characters’ names (20 points) and characterization (i.e., physical appearance, attitudes, feelings, behavior, relations to others, morals, personality) (20 points). Events include major and minor happenings or ideas broken into plot episodes, with more or fewer points assigned to each episode depending on the number of events in the episode and their importance to the story. The Holistic protocol did not require a retelling guide. The general holistic scoring guide identifies five distinct levels of richness for retellings (see Appendix D). These levels involve the ratings of eight characteristics: generalizations beyond the text, a thesis statement, major points, supporting details, supplementations, coherence, completeness, and comprehensibility. Retellings earn a score of 5 to 1 with a 5 representing a retelling that is well developed in all eight characteristics while a 1 contains details, irrelevant supplementations, and poor coherence, completeness and comprehensibility. We converted the scores of 1 to 5 scores to percentages (20% - 100%) for ease in comparison between the protocols. Layers of Analysis We have two layers of analysis in this study. On the first layer, two of us individually scored each of the 13 retellings. In instances of disagreement, a third or fourth researcher scored. Interrater reliability was between .95 and .98 for the different protocols. We used descriptive statistics to obtain means and ranges for each protocol. This allowed us to explore how each individual child scored across each of the three protocols as well as consider the trends of the whole group with any of the protocols. The second layer of analysis involved looking closely at the protocols themselves. This allowed us to investigate, as Golden and Pappas (1990) assert, “how the text structure has been defined and what the researcher believes about the reader’s contributions to, or interpretations of, the text” (p. 34). We used two complementary processes to analyze the protocols. First, we reviewed literature regarding each protocol’s theoretical underpinnings. Second, we used the constant comparative method (Hubbard & Power, 2003) to determine themes and patterns as to what the protocol actually represented of readers’ understandings. In other words, we read and re-read the scored protocols to establish what of the retellings were made visible and what remained unseen. In addition, we reviewed our research logs, minutes from our meetings and emails to identify issues that we had discussed regarding preparation, scoring or interpretation.

What We Learned For each protocol, we summarize our learnings from both layers of analysis. We first introduce each protocol by addressing its underlying theoretical assumptions regarding text and comprehension. We then use our scoring of Don’s retelling to demonstrate how each protocol represents a reader’s understanding (see Appendix E for transcript). Don’s retelling scored high with

Retelling Scores

365

one protocol and lower on the other two, illustrating variations we saw in scores. We also use Neil’s retelling (Appendix E) because he understood an aspect of Flossie that was important to the story and is reflected differently in each protocol. We include trends across the 13 readers, an evaluation based on our experiences with the protocol, and our conclusions regarding what the protocol reveals about readers’ understandings. Table 1. Retelling Scores for “Flossie and the Fox” (McKissack, 1986) Morrow

Goodman et al.

Irwin & Mitchell

Mean Retelling Scores for all 13 students

56%

36%

42%

Don’s Scores

75%

32%

40%

Neil’s Scores

50%

36%

20%

Note: We converted the holistic (1 to 5 scale) Irwin & Mitchell scores to percentages to facilitate comparisons. Don’s holistic score was 2. Neil’s holistic score was 1. The mean was 2.1.

Morrow’s (2005) Sense of Story Structure Drawing on research regarding the role of story grammar in comprehension, the Story Structure protocol focuses on the reader’s sense of story grammar (see Appendix C). Story grammar refers to the elements of a narrative text: characters, setting, a goal or problem to solve, episodes leading to meeting the goal or resolving the problem, and the resolution. Thorndyke (1977) finds that readers develop a sense of story structure through reading. They then use their knowledge of story grammar to guide their comprehension during (Englert & Thomas, 1987; Mandler & Johnson, 1977; Stein & Glenn, 1979) and after reading (Morrow, 1985, Pappas, 1991; Scott, 1988). In Don’s retelling, he speaks of the road and the places Flossie is going and names Flossie as well as four out of five characters. He refers to four of the six episodes by mentioning the animals in them. Although Don does not provide the resolution to the story, he relays that the fox can’t convince Flossie, and that Flossie reaches her destination, thereby ending the story and earning a point. Don’s retelling, although sparse, follows the sequence of the text and so he earns 2 points. Don’s score of 75% is the highest of the 13 Story Structure scores and is his highest across all three protocols (see Table 1). Based on the theoretical underpinnings, the score represents the percentage of story elements included, that is, he included 75% of the story elements as described and weighted in the protocol. However, the score does not reveal the extent or depth of his reference to particular elements. The mean score across the 13 readers is 56% with a range from 26% to 75%. Overall, we find that the children do not use story grammar to frame their retellings. Rather they tend to summarize the story and provide details in the aided retelling, thereby earning points in the protocol. Nine of the children refer to at least four of the six episodes. Despite not relying on story grammar to structure their retelling, for 11 children their story structure retelling score is the highest across the three protocols. The story element the children have the most difficulty with is “theme,” scored by the readers’ reference to Flossie’s goal of delivering the eggs safely to Miz Viola. Flossie’s problem is how to get

366

National Reading Conference Yearbook, 56

past Fox. To solve the problem, she uses her intelligence and wit to deflect the Fox’s attention from the eggs by persuading him that she doesn’t believe he is indeed a fox. Neil clearly understands part of the theme, stating, “Flossie was trying to trick the fox,” but he doesn’t related that the reason she is tricking the fox is to protect the eggs. We find that the goal or problem to which most children refer is that of Fox convincing Flossie that he is a fox. Don, for example, says, “Fox can’t prove to Flossie that he is a real fox.” Mike (34%) notes, “It is about a fox that the girl didn’t believe was a real fox.” Even Neil, who understands a major part of the theme, focuses his retelling around Fox needing to prove he is a fox “because Flossie wouldn’t take his proveness.” We find this protocol easy to use. We are able to tally across retelling score sheets to reveal group tendencies regarding use of story structure in retelling, which is helpful in planning class instruction. We find, though, that if we are only looking at the score sheets, we have little indication of the depth of the readers’ understanding or the detail they provide for each plot episode. In addition, the protocol does not reveal ideas that are not within a story grammar frame, such as opinions, inferences, or associations. Goodman et al.’s (2005) Reading Miscue Inventory: Construction of Meaning (RMI) The RMI is grounded in socio-psycholinguistic theory (Flurkey & Xu, 2003; Marek & Edelsky, 1999). This theory views reading as a language process in which readers draw on their experience with and knowledge of language and their sociocultural worlds to actively predict and construct meaning as they read (Goodman, 1994; Rosenblatt, 1978; Smith, 2004). Comprehension is “the reader’s cumulative interpretation of the text” (Goodman et al., 2005, p. 56) and is evaluated through retellings. However, while retellings offer insights into the “depth and breadth of comprehension,” the researchers state that retellings “[never represent] a reader’s total understanding of the text; readers rarely tell all they know” (Goodman et al., 2005, p. 55). Don’s retelling score using the RMI protocol is 32% (see Appendix D). For Character Analysis (21/40) he names the two main characters, Flossie and Fox, and three of the animals in the story. However, he does little to describe and develop Flossie and Fox, providing no characterization of Flossie and, for the fox, only mentioning “The Fox can’t prove to Flossie that he is a real fox.” Under Events (11/60) he refers to five of the seven episodes, earning the most points (5/8) for Episode 2 when Flossie first meets the Fox and for Episode 7 (3/9) when they arrive at the farm and the Fox must run from the hounds. He receives one point for naming something in each of the other three episodes he mentions. Don scores the lowest on this protocol compared to the other two (see Table 1), reflecting the lack of detail in his retelling. According to the theoretical foundation for this protocol, Don’s 32% his interpretation and understanding of the text’s content related to the characters and events, as specified on the retelling guide. The mean retelling score for all 13 readers is 36%, with a range of 23% to 52%. Of this score, the mean for character recall is 17% and for character development 3.9 %. The children tend to recall the characters, but provide very little information about their characteristics. Neil is an exception because he talks about Flossie tricking the fox, earning 7 points under character development. For Events, the mean score is 15.76%. Overall, the children tend to talk most about the first and last events, in addition to event five (see Appendix C), capturing about one third of the details (and points) for each of these episodes. For Flossie, the majority of these children’s points are earned for naming characters and providing some information of the events.

Retelling Scores

367

Overall, we find that the RMI protocol reveals the depth of the children’s understandings of the characters and plot episodes. The retelling guide clearly shows what readers share and the richness of the details and insights related to each point. In addition, under Character Development this protocol provides a place to indicate whether the reader understands that Flossie is tricking the Fox, as Neil does. The issue we struggled with in using this protocol is how to allot points in each area. Goodman et al. (2005) suggest dividing narrative texts into 40 points for Character Analysis and 60 for Events, which we did. However, it took numerous conversations for us to agree on how much weight to give particular characters and events. Goodman et al. also state that how points are distributed depends on the passage and how the retelling score will be used. Since the point distribution and weight impacts scoring, it requires careful consideration. Irwin and Mitchell’s (1983) Holistic Scoring The holistic scoring evaluates retellings in an integrated holistic manner that allows for the uniqueness of individuals’ retellings (see Appendix D). Irwin and Mitchell (1983) question whether readers who “[restate] the passage content in accurate, precise detail” have retellings superior to readers who “[make] in depth generalizations about life as [they summarize] the text content” (p. 391). Thus, they argue against tools that “assign points to reflect the relative importance of various elements of retelling” (p. 391), feeling that “point systems do not capture the interrelationships of all the individual factors nor the individuality of a student’s point of view” (p. 392). In this protocol, comprehension is evaluated in terms of the depth and richness in the retelling as a whole. The format of the protocol is similar to tools used in holistic scoring of compositions (Irwin & Mitchell, 1983). Don’s retelling scores a 2 (40%) (see Appendix D). He only relays a few major ideas, and through the aided retelling, supplies some supporting details. His retelling has some cohesiveness (the information he relays is in sequence, he has the beginning, middle and end), but is only somewhat comprehensible because he does not provide a lot of information to the listener. The mean across the 13 retellings is 2 (42%) with a range of 1 to 4 (20% to 80%) (see Table 1). Ten of the retellings are scored 2 (40%) or lower. This score captures the tendency of the group to tell the gist or a summary of the story. Neil’s understanding that Flossie is tricking the fox is included as a generalization (he states he knows Flossie is tricking the Fox because she reminds him of a Pokemon character) and as a relevant supplementation. The categories focus us on the composition of the retelling and provide a place to look at the associations and inferences readers make, though those are not recorded specifically. In other words, as the scoring is holistic, it provides only a generalized indicator of the richness of the retelling as a whole. It does not readily reveal details about the understandings held by the reader. Learnings across the Protocols The results for each protocol clarify what is shown and not shown about readers’ comprehension and the theoretical underpinnings represented. Yet, as we read and re-read retellings and protocols, we found aspects of readers’ understandings that remained in the background with all three protocols because they are designed to tap “how a particular reader’s reconstructed text corresponds to the original text” (Golden & Pappas, p. 33). Remaining hidden from consideration

368

National Reading Conference Yearbook, 56

are the readers’ constructed stories when their stories deviate from that of the author’s story. The reader, in transaction with text, constructs “a text parallel and closely related to the published text” (Goodman, 1994, p. 1114); through the retellings, we glimpse the readers’ text world (Langer, 1995). There is information within retellings that reveal the reader’s understanding of the text, how texts work and of the wider sociocultural world, despite not matching the author’s text that show individual differences in experience and interpretation. For example: • Dana’s constructed text includes “and she went by a fox and she kept making fun of it because she thought it wasn’t a real fox.” • Thomas includes, “and, this part isn’t in the story, but…if the story was longer, she

would give the fox one last and final try and Flossie would believe the fox was a fox…by showing that it’s the fox that always stole the eggs.”

• Adam says, Flossie has eggs “to give to the fox,” but she doesn’t give them to him

because, “she didn’t believe it was a real fox.”

We could simply say that Dana, Thomas and Adam didn’t understand the story. Although their reconstructed texts don’t match the author’s text, their texts are windows to their thinking. They provide information about how they process the story. Dana, for example, taps her experience with her sister, saying “she’ll say mean things about me like Flossie did to the fox.” Both Thomas and Adam make predictions that differ from the author’s story. Thomas believes Fox almost has Flossie convinced he’s a fox, that the story just ended too soon, while Adam believes Flossie intends to give the eggs to the fox but Fox isn’t the real fox. Dana, Thomas and Adam provide examples of how beliefs and assumptions guide comprehension. In addition, the protocols do not shed much light on expected inferences, such as the one the author implies—that Flossie knew he was a fox and kept him confused so she could protect the eggs. Neil understands Flossie tricked Fox, but the protocols provide little weight to this understanding, although it is the major point in the story. Each protocol has a place to record this insight, but the significance is not highlighted. Neil earns 50% with Story Structure, 36% with the RMI and a 1 on the Holistic Scoring. The scores alone indicate moderate to weak comprehension, yet he understands the overarching point of the story.

Closing Thoughts In this study, we investigated three retelling protocols in order to learn what they reveal and don’t reveal about readers’ comprehension of texts. We found that like the blind men describing an elephant, the representation of comprehension varies depending upon the facet being considered by the given protocol. This is not surprising, as each protocol’s design follows a different theory about how to evaluate comprehension, even though all three use verbal retelling as a data source. Theory “illuminates some aspects of social phenomena for consideration and leaves others in the background” and each theory can lead to assessment tools that guide teachers’ and researchers’ interpretations, decisions and actions, which will vary based on the theory driving them (Moss, Girard & Haniford, 2006, p. 109). It isn’t that each of the retelling protocols in our study does not provide a valid assessment of comprehension. The protocols are valid, but researchers and teachers must understand the perspective they represent in order to understand the impact on interpretations of the results, decisions made based on those results and subsequent instructional

Retelling Scores

369

actions based on those decisions. Thus, we support Golden and Pappas’ (1990) recommendation that studies contextualize their research methods theoretically. It is also necessary for teachers and other consumers of research to actively search out the theoretical perspectives behind their tools for assessing comprehension. Retellings offer researchers a great deal of information. Further, they offer children benefits, for they allow the reader active participation, language development, and a way to organize thoughts (Morrow, 2005). Nevertheless, limitations should be noted. Although retellings provide a more reallife representation of understanding than do comprehension questions, they are not an authentic activity; adults rarely engage in a retelling after reading. Further, this research should be extended. The reading and retellings of other groups of children, with other books, should be studied. The children who participated in this study were being tracked for a gifted and talented program. The question arises as to differences in results with other groups, with other books. Indeed, we will explore that question through another facet of this study. We also see the need for similar studies across different school settings since the instructional program in place impacts children’s reading (Arya et al., 2005; Wilson, Martens, & Arya, 2005; Wiltz & Wilson, 2006), as do sociocultural factors (Heath, 1994; Smagorinsky & O’Donnell-Allen, 1998). We find that the protocols we studied offer researchers and teachers valuable frames for interpreting retellings because they do illuminate aspects of the reader’s comprehension. Furthermore, we realize an advantage to having different types of protocols that allow consideration of varied facets of readers’ understandings. Researchers and teachers using retelling protocols need to be aware of what is not revealed as well as what is in order to use the tools with proper respect for the implications of the final score.

References Arya, P., Martens, P., Wilson, G. P., Altwerger, B., Jin, L., Laster, B., et al. (2005). Reclaiming literacy instruction: Evidence in support of literature-based programs. Language Arts, 83(1), 63-72. Barnhart, J. E. (1990). Differences in story retelling behaviors and their relation to reading comprehension in second graders. In J. Zutell & S. McCormick (Eds.), Literacy theory and research: Analysis from multiple paradigms (pp. 257-266). Chicago, IL: National Reading Conference. Braunger, J., & Lewis, J. P. (2006). Building a knowledge base in reading. Newark, DE: International Reading Association and Urbana, IL: National Council of Teachers of English. Bui, Y. N. (2002). Using story-grammar instruction and picture books to increase reading comprehension. Academic Exchange Quarterly. Retrieved September 4, 2004, from http://www.highbeam.com/doc/1G189970448.html Englert, C. S., & Thomas, C. C. (1987). Sensitivity to text structure in reading and writing: A comparison between learning disabled and non-learning disabled students. Learning Disability Quarterly, 10, 93-105. Feathers, K. (2002). Young children’s thinking in relation to texts: A comparison with older children. Journal of Research in Childhood Education. Retrieved September 4, 2004, from http://www.highbeam.com/ doc/1G1-94765455.html Flurkey, A., & Xu, J. (2003). On the revolution of reading: The selected writings of Kenneth S. Goodman. Portsmouth, NH: Heinemann. Fountas, I., & Pinnell, G. S. (2001). Guiding Readers and Writers (Grades 3-6): Teaching Comprehension, Genre, and Content Literacy. Portsmouth, NH: Heinemann. Golden, J. M., & Pappas, C. C. (1990). A sociolinguistic perspective on retelling procedures in research on children’s cognitive processing of written text. Linguistics and Education, 2, 21-41.

370

National Reading Conference Yearbook, 56

Goodman, K. (1994). Reading, writing, and written texts: A transactional sociopsycholinguistic view. In B. R. Ruddell, M. R. Ruddell & H. Singer (Eds.), Theoretical models and processes of reading. Newark, DE: International Reading Association (pp. 1093-1130). Goodman, Y. M. (1996). Retellings of literature and the comprehension process. In S. Wilde (Ed.), Notes from a kidwatcher: Selected writings of Yetta M. Goodman (pp. 70-81). Portsmouth, NH: Heinemann. Goodman, Y. M., Watson, D., & Burke, C. (2005). Reading miscue inventory: From evaluation to instruction. Katonah, NY: Richard C. Owen. Irwin, P. A., & Mitchell, J. N. (1983). A procedure for assessing the richness of retellings. Journal of Reading, 26(5), 391-396. Heath, S. B. (1994). Separating “things of the imagination” from life: Learning to read and write. In W. H. Teale & E. Sulzby (Eds.), Emergent literacy: Writing and reading (pp. 156-171). Norwood, NJ: Ablex Publishing Corporation. Houghton Mifflin. (2000). Houghton Mifflin reading: A legacy of literacy. Boston, MA: Houghton Mifflin. Hubbard, R. S., & Power, B. M. (2003). The art of classroom inquiry: A handbook for teacher-researchers (2nd ed.). Portsmouth, NH: Heinemann. Irwin, P. A., & Mitchell, J. N. (1983). A procedure for assessing the richness of retellings. Journal of Reading, 26(5), 391-396. Langer, J. A. (1995). Envisioning literature: Literacy understanding and literature instruction. New York: Teachers College. Lipson, M., Mosenthal, J., & Mekkelsen, J. (1999). The nature of comprehension among grade 2 children: Variability in retellings as a function of development, text, and task. In T. Shanahan & F. V. RodriguezBrown (Eds.), National Reading Conference Yearbook, 48, pp. 104-119. Chicago, IL: National Reading Conference. Mandler, J. M., & Johnson, N. S. (1977). Remembrance of things parsed: Story structure and recall. Cognitive Psychology, 9(111-151). Marek, A., & Edelsky, C., (1999). Reflections and connections: Essays in honor of Kenneth S. Goodman’s influence on language education. Cresskill, NJ: Hampton Press, Inc. Martens, P., Arya, P., Wilson, G. P., & Jin, L., (2005). The impact of text characteristics on second graders’ reading and retelling of texts with contrasting story structures. Paper presented at the National Reading Conference, Miami, FL. McKissack, P. C. (1986). Flossie and the Fox. New York: Dial Books for Young Readers. Morrow, L. M. (1985). Retelling stories: A strategy for improving young children’s comprehension, concept of story, and oral language complexity. The Elementary School Journal, 85(5), 647-661. Morrow, L. M. (2005). Literacy development in the early years: Helping children read and write (5th ed.). Needham Heights, MA: Allyn & Bacon. Morrow, L. M., Sisco, L. J., & Smith, J. K. (1992). In C. K. Kinzer & D. J. Leu (Eds.) Literacy research, theory, and practice: Views from many perspectives. Forty-first Yearbook of the National Reading Conference (pp. 435-443). Chicago, IL: National Reading Conference. Moss, P. A., Girard, B. J., & Haniford, L. C. (2006). Validity in educational assessment. Review of Research in Education, 30, 109-162. National Reading Panel. (2000). Teaching children to read, an evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: National Institute of Child Health and Human Development, National Institutes of Health. Pappas, C. C. (1991). Young children’s strategies in learning the “book language” of information books. Discourse Processes, 14, 203-225. RAND Reading Study Group. (2002). Toward an R & D program in reading comprehension. Santa Monica, CA: RAND Corporation. Rosenblatt, L. M. (1978). The reader, the text, and the poem: The transactional theory of literary work. Carbondale, IL: Southern Illinois University Press. Smagorinsky, P., & O’Donnell-Allen, C. (1998). Reading as mediated and mediating action: Composing meaning for literature through multimedia interpretative text. Reading Research Quarterly, 33(2), 198226. Smith, F. (2004). Understanding reading. Hillsdale, NJ: Lawrence Erlbaum. Scott, C. M. (1988). A perspective on the evaluation of school children’s narratives. Language, Speech, and Hearing Services in Schools, 19, 67-82.

Retelling Scores

371

SRA/McGraw-Hill. (2000). Open Court Phonics Kits. DeSoto, TX: SRA/McGraw Hill. Stein, N., & Glenn, C. (1979). An analysis of story comprehension in elementary school children. In R. O. Freedle (Ed.), New directions in discourse processing (pp. 53-120). Norwood, NJ: Ablex Publishing Corporation. Thorndyke, P. (1977). Cognitive structures in comprehension and memory of narrative discourse. Cognitive Psychology, 9, 77-110. Wilson, G. P., Martens, P., & Arya, P. (2005). Accountability for reading and readers: What the numbers don’t tell. The Reading Teacher, 58(7), 622-631. Wiltz, N., & Wilson, G. P. (2006). An inquiry into children’s reading in one urban school using SRA Reading Mastery. Journal of Literacy Research, 34(4), 493-528.

Appendix A Synopsis of Flossie Flossie and the Fox (McKissick, 1989) is the tale of Flossie, who has been given eggs to deliver to Mz. Viola’s. Seems her hens have been troubled by a fox. Flossie has never seen a fox before, but she meets one on the road. She convinces him that he must offer proof that he is a fox. He shows her his thick fur, pointy nose and bushy tail, but she says he is tricking her—that he could be a rabbit, a rat or a squirrel (respectively). In the end, Flossie delivers the eggs safely and when the fox asserts yet again that he is a fox, she responds, “I know, I know.” In this story, the author never says explicitly that Flossie was tricking the fox. The reader must infer this.

Appendix B Retelling Scores 24

Morrow’s (2005) Sense of Story Structure Retelling Evaluation – Don Appendix B Morrow’s (2005) Sense of Story Structure Retelling Evaluation – Don Give one point for each element included as well as for “gist.” Give one point for each character named as well as for words as “boy, girl, dog.” Credit plurals (e.g. friends) with 2 points under characters. Setting a. begins story with an introduction

1

b. names main character (s)

1

c. number of other characters named fox, Big Mama, cat, squirrel, hounds d. actual number of other characters

4 5

e. score for “other characters (e/f) f. includes statement about time or place Theme

.80 1

d. actual number of other characters

372

e. score for “other characters (e/f) f. includes statement about time or place

5 .80

National Reading Conference Yearbook, 56 1

Theme Refers to main character’s primary goal or problem to be solved

0

Plot Episodes a. number of episodes recalled

4

b. number of episodes in story

6

c. score for “plot episodes” (a/b)

.66

Plot Episodes 1. Flossie meets the Fox, who eyes the eggs and tells her he is a fox so she will be frightened. Flossie says she doesn’t believe he is a fox. He asks if he must offer proof he is a fox and she says yes and skips off.

Retelling Scores 25

2. Fox shows her his thick luxurious fur for proof, but Flossie says that isn’t proof, as it feels like rabbit fur. She says he is a rabbit trying to fool her that he’s a fox. Fox shouts. She skips away. 3. Fox shows her his long pointed nose as proof, but Flossie says rats have long pointed noses and accuses him of being a rat trying to pass himself off as a fox. Fox gasps. She skipped on down the road. 4. Flossie sees an orange tabby (cat); Fox says the cat will tell her he is a fox. The cat says he is a fox because he has sharp claws and yellow eyes. Flossie says it doesn’t prove anything except that he is a cat. Fox howls. Flossie skipped away. 5. Fox begs to be believed and points out he has a long bushy tail. Flossie says squirrels do, too. Fox cries. Flossie offers him a bit of peach. 6. Coming out of the woods, fox asks for another chance, saying he has sharp teeth and can run fast. Flossie says so can hounds, and there is one after him. Hound chases Fox. Resolution a. Names problem solution/goal attainment

0

b. Ends story

1

Sequence Retells the story in structural order: setting, theme, plot episodes, resolution (score 2 for proper, 1 for partial, 0 for no sequence evident).

2

a. Names problem solution/goal attainment

0

b. Ends story

1

Retelling Scores

373

Sequence Retells the story in structural order: setting, theme, plot episodes, resolution (score 2 for proper, 1 for partial, 0 for no sequence evident).

Final Percentage: total earned over total possible.

2

7.46 / 10 = 75%

(Underline italics = Don’s responses that were used in scoring)

Appendix C

Retelling Scores 26

Appendix C

Don’s Retelling Scored on Goodman, Watson, & Burke (2005): 32 / 100 Don’s Retelling Scored on Goodman, Watson, & Burke (2005): 32 / 100 Character Analysis (40 points) Recall (Total: 20 pts)

Character Analysis Total for Don: 21/40

Don: 19/20

Development (Total: 20 pts) Don: 2/20

8 – Flossie

10 – Flossie

8 – Fox

• is smart

1 – Big Momma

• tries to confuse fox; pretends not to know

1 – Squirrel with bushy tail

fox is a fox

1 – Hound that runs fast

• Plays with words

1- Cat with yellow eyes and sharp claws

10 – Fox • fox wants to scare Flossie

(mentioned in story: Miz Viola; Mr. McCutchin; rat; rabbit)

• fox wants the eggs and will most anything to get them • fox tries to convince Flossie he is a fox (to scare her so she gives up the eggs) but can’t

Retelling Scores 27

374

National Reading Conference Yearbook, 56

Events (60 points)

Events Total for Don: 11/60

Episode 1

• Big Mama called Flossie; Flossie tucked away her straw doll

(7 pts total)

• Big Mama told Flossie to take a basket of fresh eggs to Miz Viola’s over at the McCutchin Place.

Don:

• Miz Viola’s chickens be scared of a fox. The fox is a sly critter.

1 point

• Flossie starts through the woods with the basket tucked under her arm. • She wonders what will happen if she comes upon a fox. She’s never seen one.

Episode 2 (8 pts total) Don: 5 points

• Flossie meets a critter she couldn’t recollect ever seeing. The fox said he is a fox, • but Flossie said “Nope, I just purely don’t believe it” –she doesn’t believe he is a fox. • He told her she should be terrified of him, and was shocked when she isn’t. • Fox asked her if he must offer proof he was a fox. Retelling She said Scores yes. She28 skipped off and he looked for proof.

Episode 3

• Fox says he has proof, “thick, luxurious fur.”

(10 pts total) • Flossie rubbed his back and said, “feels like rabbit fur to me…Shuck’s you aine no fox. • You a rabbit, all the time trying to fool me.” Don: 0 points

• Fox shouts that his reputation precedes him; he is third generation of foxes who out-smarted and out-hunted Mr. J.W. McCutchin’s hunting dogs, “I am a fox,” and you (Flossie) “will act accordingly.” Flossie said, “..Unless you can show you a fox, I’ll not accord you nothing!” She then skips away.

Episode 4

• Flossie stopped by a stream.

(9 pts total)

• Fox said he as a long pointed nose, that should be proof enough. • Flossie said rats got long pointed noses.

Don:

• “That’s it! You a rat trying to pass yo’self off as a fox.”

0 points

• Fox gasped. Flossie skipped down the road. “I’ll teach you a thing or two,” he called after her.

Episode 5 (9 pts total)

• In a clearing, Flossie sees a large orange tabby (cat). Fox says, “Since you won’t believe …perhaps you will believe that fine feline creature…”

Don:

• “That’s it! You a rat trying to pass yo’self off as a fox.”

0 points

• Fox gasped. Flossie skipped down the road. “I’ll teach you a thing or two,”

Retelling Scores Episode 5 (9 pts total)

375

he called after her.

• In a clearing, Flossie sees a large orange tabby (cat). Fox says, “Since you won’t believe …perhaps you will believe that fine feline creature…” • The cat purred, “This is a fox because he has sharp claws and yellow eyes.”

Don: 1 point

• Flossie looks at cat and at Fox and says, “…both y’all got sharp claws and yellow eyes…that don’t prove nothing ‘cep’n bothRetelling y’all be cats. Scores 29 • Fox started howling and running in circles. Flossie skipped away.

Episode 6 (8 pts total)

• Fox follows Flossie, pleading, and tells her “I I I have a bushy tail…that has got to be adequate proof.”

Don:

• “Aine got to be…so do squirrels.”

1 point

• Flossie offered him a bite of her peach, but Fox was crying like a baby. • “If I promise you I’m a fox, won’t that do?” Flossie shook her head no.

Episode 7

• Coming out of the woods, Fox asks for one more chance.

(9 pts total)

• “I have sharp teeth and I can run exceedingly fast.” • Flossie rocks back and forth, and said, “it don’t make much difference what I think anymore…’cause there’s one of Mr. J.W. McCutchin’s hounds behind you…it’s all over for you.”

Don: 3 points

• Fox, with a quick glance back dashed toward the woods shouting, “The hound knows who I am.”

Fox can’t outrun or out-smart the dogs

• Flossie said, “I know” and turned toward Miz Viola’s with the eggs tucked safely under her arm. TOTAL SCORE: Character Analysis _21_ / 40

Events _11_ / 60 TOTAL: 32 / 100 = 32%

(Underline italics = Don’s responses that were used in scoring) TOTAL SCORE:

Character Analysis _21_ / 40

Events _11_ / 60

TOTAL: 32 / 100 = 32%

Appendix D

(Underline italics = Don’s responses that were used in scoring)

Irwin & Mitchell’s (1983) Holistic Scoring Retelling Evaluation Criteria for establishing the level of a retelling (5 = highest level, 1 = lowest level) 5 Student generalizes beyond text; includes thesis (summarizing statement), all major points, and appropriate supporting details; includes relevant supplementations; show high degree of coherence, completeness, comprehensibility.

376

National Reading Conference Yearbook, 56

4 Student includes thesis (summarizing statement), all major points, and appropriate supporting details; includes relevant supplementations; shows high degree of coherence, completeness, comprehensibility. 3 Student relates major ideas; includes appropriate supporting details and relevant supplementations; shows adequate coherence, completeness, comprehensibility 2 Student relates a few major ideas and some supporting details; includes irrelevant supplementations; shows some degree of coherence; some completeness the whole is somewhat comprehensible. 1 Student relates details only; irrelevant supplementations or none; low degree of coherence; incomplete; incomprehensible. (Underline italics = Don’s responses that were used in scoring. Don scored a 2.)

Appendix E Don’s retelling: (R = Researcher; D = Don) D: Flossie has eggs and she’s skipping down the road and she sees a fox but she doesn’t really think it’s a fox cause she never saw one before. And then Flossie keeps skipping down the road and she finds other animals that she thinks is a fox but she doesn’t believe the real Fox. And then when she gets to Miss Villa’s and Miss Mitchins’s farm the fox can’t out-run or out-smart Mr. McChutins’s dogs and Miss Violet’s. The end. R: Anything else? D: That the Fox can’t prove to Flossie that he is a real fox. R: You said that she finds other animals. Do you remember any of those animals? D: Uhuh (+) …A cat, an orange cat, and a squirrel. Neil’s retelling: (R = Researcher; N = Neil) N: Um, the fox kept on proving to Flossie that he was a fox but she wouldn’t take it. And the fox kept on trying but the last time a hound came and chased the fox. And the girl had some eggs. And the fox tried to prove because, um, … Flossie wouldn’t take her, his, um, proveness. R: I didn’t understand that. What was Fox trying to prove? N: That he was a fox. R: What else happened in the story? N: …She was going to Mc-something…They couldn’t shop (?) to give the eggs. R: Do you remember the different things that Fox did to prove that he was a fox? N: Pointy nose, fluffy tail, yellow eyes, sharp teeth, yellow eyes and the sharp teeth, the cat did. … Pretty much it. R: Who do you think was trying to trick who in this book? N: Flossie was trying to trick the fox. R: Why do you say that? N: Because she already knew he was a fox and at the end she said she knew already. R: So why was she trying to trick him if she knew he was a fox? N: I don’t know.