built around the same human biology terms used in developing the domain knowl- .... tors of above- and below-average performance were the analogic passage ...
Journal ofhttp://jlr.sagepub.com/ Literacy Research
Domain Knowledge and Analogic Reasoning Ability as Predictors of Expository Text Comprehension Patricia A. Alexander and Jonna M. Kulikowich Journal of Literacy Research 1991 23: 165 DOI: 10.1080/10862969109547735 The online version of this article can be found at: http://jlr.sagepub.com/content/23/2/165
Published by: http://www.sagepublications.com
On behalf of: Literary Research Association
Additional services and information for Journal of Literacy Research can be found at: Email Alerts: http://jlr.sagepub.com/cgi/alerts Subscriptions: http://jlr.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://jlr.sagepub.com/content/23/2/165.refs.html
>> Version of Record - Jun 1, 1991 What is This?
Downloaded from jlr.sagepub.com by guest on October 11, 2013
Journal of Reading Behavior 1991, Volume XXIII, No. 2
DOMAIN KNOWLEDGE AND ANALOGIC REASONING ABILITY AS PREDICTORS OF EXPOSITORY TEXT COMPREHENSION
Patricia A. Alexander Texas A&M University
Jonna M. Kulikowich University of Connecticut
ABSTRACT In this investigation, we analyzed the role domain knowledge, analogic reasoning ability, and interactive knowledge play in the comprehension of scientific exposition. Data from three experiments involving sixth graders, high-school students, and college undergraduates are examined via regression and discriminant analysis procedures. The dependent variables in these analyses were student scores on expository passages on topics in human biology/immunology each prepared in two versions: with and without supporting analogy. The independent variables were subjects' performance on: (a) multiple-choice tests of human biology/immunology terminology (domain knowledge test), (b) figurai analogy items from the Advanced Progressive Matrices (analogic reasoning test), and (c) A:B::C:? analogy problems composed of human biology/human immunology terms (interactive knowledge test). Results showed that the variables predicting performance on the nonanalogic and analogic versions and the variables discriminating between above-average and below-average performers differed across the grade levels with less integration of content knowledge and analogic reasoning ability occurring for college students. Implications of these results for theory and practice are discussed. Prior research has firmly established the individual importance of domain knowledge (e.g., Alvermann, Smith, & Readence, 1985; Anderson, Reynolds, Schallen, & Goetz, 1977; Rabinowitz & Chi, 1987) and strategic processing (e.g., Garner, 1987; Hare & Borchardt, 1984; Paris, Lipson, & Wixson, 1983; Pressley, Johnson, Symon, McGoldrick, & Kuria, 1989) in text learning. However, their interactive contributions to the understanding of exposition have been explored in 165
166
Journal of Reading Behavior
only a handful of studies (e.g., Alexander, Pate, Kulikowich, Farrell, & Wright, 1989; Hasselhorn & Korkel, 1986; Walker, 1987). When students fail to comprehend exposition, we cannot say with much assurance whether this failure was due to a deficit in domain knowledge, strategic processing, or some interplay between the two. In the present investigation, we were interested in achieving a greater understanding of expository text comprehension by disentangling the unique and interactive effects of domain knowledge and strategic processing. Domain knowledge refers to the body of knowledge individuals possess relative to a particular field of study, such as biology or economics (Alexander & Judy, 1988; Alexander, Schallert, & Hare, 1989). Strategies are defined here as effortful, planful procedures that are consciously initiated in the completion of a cognitive task (Garner & Alexander, 1989). Although there are many domains and many strategies that could be examined in research of this kind, we focused our efforts on the domain of human biology/ immunology and on the strategy of analogical reasoning. There were several reasons for this decision. First, human biology/immunology is a domain rich in highly organized content (Kulikowich & Alexander, 1990). This organization permitted the development of assessment measures that were instructionally sound and psychometrically valid. Second, because human biology/immunology is basic to school curricula, it would ensure some subject familiarity with the information being tested. Our interest in analogical reasoning ability was predicated upon its assumed value in text comprehension (e.g., Bean, Singer, & Cowen, 1985; Hayes & Tierney, 1982; Judy, Alexander, Kulikowich, & Willson, 1987; Tierney & Cunningham, 1984), knowledge acquisition (Vosniadou & Brewer, 1987; Vosniadou & Ortony, 1983), and knowledge transfer (Crisafi & Brown, 1986; Gick & Holyoak, 1980). Finally, the long history of analogy problems as measures of aptitude and achievement (Embretson & Schneider, 1989; Spearman, 1927; Stemberg, 1985) allowed us to identify or to devise assessment measures that were not only informative, but also reliable and valid. As indicators of text comprehension, we administered expository passages dealing with topics in human biology/immunology. Each passage was presented in two versions, one without a supporting analogy (nonanalogic) and one with (analogic). In this manner, the contributing effects of domain knowledge and analogic reasoning ability could be investigated under conditions in which one may prove more influential in comprehension than the other. Based on existing research (e.g., Vosniadou & Ortony, 1983), we hypothesized that both domain knowledge and analogic reasoning ability would differentially impact comprehension performance of the nonanalogic and analogic versions of the expository passages. In this investigation, we examined the comprehension performance for middle-school (Experiment I), secondary (Experiment II), and college (Experiment III) students. By studying text comprehension at three different grade levels, we
Text Comprehension
167
could begin to produce a developmental picture of how domain knowledge and strategic processing (i.e., analogic reasoning ability) affect reading performance. EXPERIMENT I: MIDDLE-SCHOOL STUDENTS Method Subjects The first experiment involved 211 sixth graders enrolled in a middle school in southcentral Texas. There were 104 males and 107 females in the sample which also consisted of a representative number of minority students and those from low, middle, and upper-middle socioeconomic class backgrounds. According to teacher judgment and standardized test scores (e.g., Otis-Lennon), a cross section of cognitive ability groups was represented in this sample. To ensure a variability in reading performance, the sample was drawn from below-average, average, and aboveaverage reading classrooms. However, no special education or Limited English Proficiency (LEP) students were included in the sample. Materials To assess the unique and interactive contributions of domain knowledge and analogic reasoning ability, the following three tests served as the independent measures in analyses: (a) domain knowledge test, (b) analogic reasoning test, and (c) interactive knowledge test. Additionally, the dependent measures were expository passage tests. Domain knowledge test. Although various means of ascertaining subjects' content knowledge were considered (e.g., verbal reports, concept mapping), we concurred with Freebody and Anderson (1983) that vocabulary knowledge serves as an effective gauge of subjects' content knowledge. Following the work of Anderson and Freebody (1979) and others (e.g., Herman, Anderson, Pearson, & Nagy, 1987), therefore, we developed a multiple-choice vocabulary test to assess students' knowledge of human biology. Each of the 25 items was constructed from a corpus of human biology terms and their definitions drawn from five leading science textbooks used at the sixth-grade level. Alternate forms of the test were prepared. To ensure reliability and validity, the measure underwent extensive pilot testing (e.g., Alexander, Kulikowich, Willson, & Goetz, 1987). Internal consistencies for the alternate forms were .76 and .73, respectively, as determined by the Kuder Richardson (KR 20) formula. In addition, experts in science education validated the accuracy and appropriateness of test items (cf. Alexander, Pate, Kulikowich, Farrell, & Wright, 1989). All expert validations exceeded .80. A sample item from the domain knowledge test appears below:
Journal of Reading Behavior
168
5
6
7
8
Figure 1. An approximation of an item from the Advanced Progressive Matrices. Sample Item from the Domain Knowledge Test (Sixth Grade) Bladder is: *a. a sac that stores urine b. the fat of a whale c. the tissue found in the heart d. a language spoken by babies Analogic reasoning test. As a measure of sixth graders' strategic processing (i.e., analogic reasoning abilities), we administered 20 items from the Advanced Progressive Matrices (APM), Sets I (Raven, 1958) and II (Raven, 1962). Specifically, all of Set I was given along with 8 randomly selected items from Set II. An APM-like item appears in Figure 1. Since no normative data were used in our analyses, we were comfortable with this selective use of the APM. Although the APM is used as a test of general intelligence, it is composed entirely of figurai, matrix analogy problems. Much research has been performed on these items which supports their use as measures of the component processes
Text Comprehension
169
underlying analogical reasoning (e.g., Mulholland, Pellegrino, & Glaser, 1980; Willson, Alexander, Kulikowich, & Pate, 1990). According to Sternberg (1985) and others (Embretson & Schneider, 1989; Pellegrino, Mumaw, & Shute, 1985), these component processes remain consistent across verbal and nonverbal analogy problems. Indeed, recent research has found facilitative effects for training in the component processes of analogical reasoning on measures of verbal comprehension (Pate, Alexander, & Kulikowich, 1989). The nonverbal nature of the APM problems, therefore, permitted us to assess analogical reasoning processes while controlling for the influence of domain knowledge. The reliability for the analogic reasoning test as determined by the Kuder Richardson formula was .66. Interactive Knowledge (science analogies) test. Sixth graders' application of both domain knowledge and analogical reasoning was tested via A:B::C:? problems built around the same human biology terms used in developing the domain knowledge test. We felt that this type of test was an effective measure of interactive knowledge since it required not only an understanding of the human biology terminology but also the capability of employing that knowledge in the completion of analogy problems. As with the domain knowledge test, the interactive knowledge measure was prepared in alternative forms with KR 20s of .78 and .82, respectively. Sample items from this measure appear below: Sample Items from the Interactive Knowledge Test (Sixth Grade) WEAKER STRONGER : : RECESSIVE : OVARIES EGG : : TESTES : MUCOUS RESPIRATORY SYSTEM : : SALIVA :
Expository passage tests. We measured expository text comprehension by means of 17 multiple-choice questions that followed each of two passages about a human biology topic. Both passages were prepared in two versions: one without a supporting analogy (hereafter referred to as the nonanalogic version) and one with a supporting analogy (the analogic version). Specifically, one expository passage dealt with the body's defense mechanisms. In the analogic version, these mechanisms were compared to components in the video game of Pacman. The other expository passage dealt with the topic of infection. The analogic version of this passage related the elements of the immune system to aspects of war. Although the content of the two versions of the same passage differed somewhat (i.e., did or did not contain a supporting analogy), the questions remained consistent. The number of science concepts and word length were maintained across all passages, with KR 20 reliabilities of .76 (defense mechanisms) and .80 (infection). Because traditional readability formulas were considered inappropriate for calculating the reading levels of the resulting passages, more suitable methods were sought (e.g., Klare, 1984). Specifically, content validation and expert validation of the reading passages were conducted. That is, two effective ways to determine if passages could be read by sixth graders of various ability levels would be to: (a)
170
Journal of Reading Behavior
have students of similar age/ability read them and assess their comprehension (content validation), and (b) have those with experience working with these students rate the readability (expert validation). Thus, versions of the passages were pilot tested on three occasions with several hundred students of similar age/ability. Changes suggested by the pilot studies were incorporated in subsequent versions. In addition, five reading teachers and reading educators judged the suitability of the final version for use with sixth-grade students. A segment of the nonanalogic and analogic versions of the infection passage and a related question appear in Appendix A. Procedure During testing, the order of the domain knowledge, analogic reasoning, and interactive knowledge tests was counterbalanced. After the three independent measures were given, the students read one randomly assigned passage (i.e., immune system or defense mechanisms) in its nonanalogic form and the other passage in its analogic form. As with the prior testings, the order of presentation was counterbalanced. Results and Discussion Our objectives in this experiment were twofold. First, we wanted to assess the influences of domain knowledge, analogic reasoning ability, and interactive knowledge on reading comprehension. Second, we sought to determine how these three cognitive constructs discriminate between those of above-average versus below-average comprehension ability. We first inspected the distributions of our variables so that we would not violate any statistical assumptions related to the use of parametric tests. Using the skewness and kurtosis indices as guides, we found the distributions for the variables of interest to be approximately normal. Thus, we used parametric statistics. Means and standard deviations for all independent and dependent measures are presented in Table 1. As these data suggest, sixth graders appeared most capable in selecting the correct response on the domain knowledge test and least capable in generating answers for the interactive knowledge test. An independent t test determined that there were no statistical differences found for passage topics (i.e., immune system or defense mechanisms), rs(l).18. Therefore, we employed subsequent analyses without regard for this variable. Next, we performed a dependent t test on the passage data to find out whether sixth graders process nonanalogic and analogic text differently. The dependent measures were the differences calculated between nonanalogic and analogic scores within each subject. Results showed no significant difference f(l)= —1.56, p>. 11. We hypothesized, however, that the lack of such a difference did not necessarily imply that the same independent variables would affect the comprehension of the
Text Comprehension
111
Table 1 Means and Standard Deviations for Independent and Dependent Measures for Experiment I Measures Domain Knowledge Analogic Reasoning Ability Interactive Knowledge Expository Passages Nonanalogic Analogic
Total Items
M
SD
25 20 30
15.29 10.29 9.16
4.72 2.63 4.27
17 17
8.98 8.63
3.86 4.09
two passage versions in the same manner. Thus, we tested two stepwise regression equations, one per passage, with the nonanalogic and analogic passage scores serving as the dependent measures, respectively. For these two linear models, the measures of the domain knowledge, analogic reasoning ability, and interactive knowledge served as predictor variables. In the nonanalogic passage model, the domain knowledge test [ß= .29, F(l, 207) = 29.36, p . 0 8 . One plausible explanation for this lack of predictive effects is that the undergraduates we tested did adequately on unidimensional tests of domain knowledge and analogic reasoning ability, but scored well below expectations on the interactive knowledge test. This suggests that they may be less capable of integrating their knowledge of science with their reasoning abilities; an integration that seems requisite to the comprehension of demanding scientific exposition. Another possible reason for the nonsignificant findings in this experiment is the relatively low power in the statistical analyses that may have resulted from the small sample size (n = 35). Stepwise discriminant analyses were then performed on the nonanalogic and analogic passage scores. The means and standard deviations for the nonanalogic and analogic discriminant models are displayed in Tables 9 and 10, respectively. For the nonanalogic passage, the analogic passage significantly discriminated between above-average and below-average performance, F(l, 33) =106.80, p
