Student Engagement and Learning

Moving Authentic Soil Research into High School Classrooms: Student Engagement and Learning

k–12 education

Bianca N. Moebius-Clune,* Irka H. Elsevier, Barbara A. Crawford, Nancy M. Trautmann, Robert R. Schindelbeck, Harold M. van Es ABSTRACT Inquiry-based teaching helps students develop a deep, applied understanding of human–environmental connections, but most high school curricula do not use inquiry-based methods. Soil science topics, which are also generally lacking from curricula, can provide hands-on model systems for learning inquiry skills. We report on the implementation of a soil science inquiry unit for high school earth science classrooms. Teams in three classes participated in open inquiry about water runoff from, and infiltration into soils. Students learned how scientists conduct research by asking their own research questions, designing and conducting investigations, presenting findings to their peers, and peer-reviewing each other’s work. Student engagement and learning were assessed through testing, final projects, a student survey, and observations of student attitudes. Pre- to post-test gains (17% average gain from 63% average pre-test score, with 71% maximum gain) showed significant student science-contentlearning. Generally lower scores on final projects (61% average) than on post-tests (80% average) suggest the need for more teacher-scaffolding in complex, open-ended assignments. Students reported enjoying the unit and learning essential inquiry skills, such as experimental design, scientifically based teamwork and group-learning, and real world applicability of concepts. Observations suggest that students were motivated and substantively engaged. One-third of students reported increased excitement about science. We conclude that inquiry-based units should be more commonly used in science classrooms, to enable students to learn how to think critically, develop collaborative teamwork skills, take ownership of their learning, and be substantively engaged in authentic tasks applicable in later life.

Over the next decades, natural resource management and policy decisions will increasingly be made by our younger generations: today’s K–12 students. Schools need to use teaching methods that prepare our students to have the understanding and skills to grapple with contemporary agricultural and environmental challenges. Inquiry-based curriculum, like the soil inquiry unit demonstrated here, can stimulate student interest and teach essential professional skills. Such open-ended, student-centered approaches should be more commonly used in science classrooms.

Impact Statement

T

wo gaps have been identified in current high school science education. First, most high school curricula do not effectively employ scientific inquiry-based teaching (Crawford, 2007); second, explicit lessons in soil science content are rare (Collins, 2008). Students often lack interest in science, particularly in middle and high schools (Linn et al., 2000; Schmidt et al., 1999). Effective inquiry-based teaching can capture student interest and facilitate students in developing important skills, such as critical and logical thinking, and a deep and applied understanding of human–environmental connections (Backus, 2005; Brown and Campione, 1994; Crawford, 2000). Soil science–related topics, while being important in their own B.N. Moebius-Clune, R.R. Schindelbeck, and H.M. van Es, Dep. of Crop and Soil Sciences, Cornell Univ., Ithaca, NY 14853; I.H. Elsevier, Penn Yan Academy, Penn Yan, NY 14527; B.A. Crawford, Dep. of Ecology and Evolutionary Biology, Cornell Univ., Ithaca, NY 14853; N.M. Trautmann, Dep. of Natural Resources, Cornell Univ., Ithaca, NY 14853. Received 5 July 2010. *Corresponding author ([email protected]). J. Nat. Resour. Life Sci. Educ. 40:102–113 (2011) doi:10.4195/jnrlse.2010.0019k • http://www.JNRLSE.org © American Society of Agronomy 5585 Guilford Road, Madison, WI 53711 USA

right, can provide a concrete and relevant model system for learning inquiry skills. This article addresses both gaps.

Need for K–12 Soil Science Education Soil is a key player in today’s most pressing global issues, from global food security to energy demand and climate change (Brady and Weil, 2002; Lal, 2007, 2008; Magdoff and van Es, 2009; Montgomery, 2007). Yet agriculture—the profession perceived to be most directly related to soiltopics—is being practiced by a shrinking percentage of the population (U.S. Department of Agriculture, 2010). The average child is thus no longer growing up with an awareness of the importance of soil, as there are few other concrete ways for students to learn this. A group of 16 incoming freshmen in arts and sciences majors at Cornell University, taking part in a New Student Reading Project on John Steinbeck’s The Grapes of Wrath, for example, stated that they felt no connection to agriculture (Crawford, unpublished data, 2009). In higher education the noticeable decline in undergraduate enrollment in soil science courses and majors may well be

Abbreviations: avg., average; SD, standard deviation; min., minimum score; max., maximum score.

Copyright © 2011 by the American Society of Agronomy. All rights reserved. No part of this periodical may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher.

Jo urnal of Natura l Re s o u r c e s & L i f e S c i e n c e s E d u c a t i o n • Vo l u m e 4 0 •   2 0 1 1

102

related to the limited emphasis on soil science in K–12 science classes (Collins, 2008). Many citizens may lack an understanding of the soil beneath their feet as a three-dimensional volume with pore spaces that can hold, move, or fail to absorb water. Belowground processes of the water cycle are often difficult for students to grasp (Elsevier, personal communication, 2009). Without such understanding, and thus ability to make appropriate management decisions, land managers and policy makers may inadvertently contribute to soil degradation and loss of essential soil functions (Karlen et al., 2003; Lal, 1998; Magdoff and van Es, 2009; Pimentel et al., 1995). It is crucial that future citizens (today’s high school students) develop a deep understanding of the basic functions of environmental systems, so that they will be equipped to make wise decisions about contemporary environmental problems and food security issues as they arise (National Research Council, 2000). To this end, it is an important goal of science education for all students to become “scientifically literate.”

Characteristics of Scientific Inquiry Scientific literacy necessitates understanding and being able to do scientific inquiry. Scientific inquiry is “what scientists and engineers do” (Shipman, 2004) as they develop scientific knowledge by asking new questions, designing experiments or organized ways of making observations of the natural world, and coming to conclusions by giving priority to evidence in interpreting the results (Capps and Crawford, 2009). Essential features of inquiry, as discussed in the National Science Education Standards, include the learner (1) being involved in scientifically oriented questions, (2) giving priority to evidence in responding to questions, (3) using evidence in developing explanations, (4) connecting explanations to scientific knowledge, and 5) effectively communicating and justifying explanations and scientific ideas in varied settings (National Research Council, 2000). Further skills needed for inquiry are critical and independent thinking, scientific reasoning, creativity, and collaboration (AAAS, 1989; National Research Council, 1996, 2000; Shipman, 2004). These goals are difficult to attain in the traditional classroom, where the teacher’s role is that of the main authority, and students have limited opportunities to take charge of their own learning (Crawford, 2008). Students must practice the skills that create scientific knowledge; in other words they must learn to do scientific inquiry and this requires appropriate teaching techniques (AAAS, 1989; National Research Council, 2000; Reeves et al., 2007; Rutledge, 2005).

Need for Inquiry-Based Teaching Inquiry-based teaching is instruction that reflects the nature of science and scientific inquiry and integrates these characteristics into classroom content and dynamics (Anderson, 2002; National Research Council, 1996, 2000). Inquiry-based classrooms are thus student- rather than teacher-centered. Students collaborate in teams and engage in authentic activities, which either relate to their lives or are similar to what scientists or other professionals might encounter (Backus, 2005; Crawford et al., 1999; National Research Council, 1996). In open or full inquiry, students choose their own scientific questions, while in guided inquiry

students are given a question, but still design and conduct their own experiments (National Research Council, 2000). Students use their evidence to develop answers to their questions and debate and write about the relevance of their findings to real-world issues. An inquiry-based classroom allows both students and teachers to take on non-traditional roles (Crawford, 2000). The teacher facilitates student activities and discussions, guides and mentors student progress, and models behaviors and attitudes of a scientist by being an innovator, experimenter, and active learner in collaboration with student teams. Students take on roles of apprentices, experimenters, learners, and collaborators. Within their research teams, they take on roles traditionally reserved for the teacher, such as teacher, leader, and planner. Students thus practice skills needed in adult workplaces. Such skills cannot be learned individually, or abstractly, but only by experiencing and developing them in groups (Brown et al., 1989). This approach tends to engage students substantively in their own learning, by encouraging them to develop personal sustained commitment to understand and interact with the content matter for its inherent interest (Nystrand and Gamoran, 1991), and by giving them ownership of their discoveries and the confidence and motivation to be curious and seek answers (Crawford, 2000). Students taught using inquirybased approaches may perform better on science achievement tests, in addition to gaining other important skills and a deeper understanding of the subject matter and the nature of science (AAAS, 1993; Brown and Campione, 1994; National Research Council, 1996; Schneider et al., 2002). Despite the focus of the science content standards on inquiry, and the development of teaching standards promoting inquiry-based teaching (AAAS, 1993; National Research Council, 2000), there are few actual examples of effective inquiry-based teaching at the K–12 education level. Inquirybased instruction is still uncommon in classrooms for many reasons. It is a complex and sophisticated way of teaching that demands significant professional development (Capps and Crawford, 2009; Crawford, 2000, 2007). Many teachers are inadequately prepared in science generally (Krajcik et al., 2000), lack effective inquiry-based teaching materials, feel pressure from time constraints due to high-stakes testing, or are unfamiliar with how science is practiced (Deboer, 2004).

Addressing Two Gaps This article simultaneously addresses the lack of soil science content and the lack of inquiry teaching materials for high school settings. Facilitated by the NSF GK–12 sponsored Cornell Science Inquiry Partnerships program, a graduate student researcher (the first author) and a high school earth science teacher (the second author) collaboratively designed and implemented a scientific inquiry unit in which students explore water runoff from and infiltration into soil, and the ways that human activities affect these soil–water dynamics. We adapted an experimental set-up from authentic research methods used at Cornell University to investigate runoff and infiltration (van Es and Schindelbeck, 2003), such that it could be cheaply and easily implemented in the classroom. Our objectives were to describe (1) the field testing of the unit, (2) findings about stu-


103

Table 1. Sequence of 40-min-lessons in the inquiry unit, as field tested. Day

Class content

Day 1

Pre-test.

Day 2

Split classes into teams to build rain-makers. Discussion of importance of precision and accuracy in tools/measurements.

Day 3

Introduction: Sponge as soil exercise, demo of rain-maker and sample experimental set-up. Share ideas/observations from journal entries. Hand out methods reading.

Homework: students read methods and write in journals about what jobs these methods could be divided into in their team. Day 4

Discussion to split methods into jobs to be performed by team members.

Day 6

Brainstorm what makes up soil. Discuss runoff/infiltration related soil issues. Rotate through 10–15 different soil materials and make observations to help visualize and decide on questions/hypotheses. Student teams pick questions, write hypotheses, and start research proposals. Teacher collects proposals to provide feedback. Hand out assessment rubric to students and announce project.

Day 7

Teams finish research proposals, incorporate written/verbal feedback from instructors and class discussion. Teams receive final approval from instructors.

Day 8

Instructors provide all materials. Teams set up their experiments and make as many measurements as are possible in allotted time.

Day 9

Teams calculate total runoff, total infiltration, percentages, averages, and draw bar graphs of their results.

Day 10

Students finish/revise calculations and graphs, and analyze their data, guided by worksheets. Instructors discuss final project and rubric.

Homework: students start their final projects. Day 12

About one full class worth of time allotted over several days for students to work on final projects, and consult with the teacher about progress.

Homework: students finish projects. Day 13

Short discussion of role of conferences in science. Classroom conference—each student reviews at least three projects of their peers.

Day 14

Post-test and Student Survey.

dent learning of soil science concepts and science inquiry skills, and (3) student attitudes and engagement while undertaking this unit, and (4) to derive some conclusions about the usefulness of the teaching style based on this case study.

MATERIALS AND METHODS Unit Development and Field Testing Over 2 consecutive years, we developed and field-tested a 3-week-long unit for classroom instruction on runoff and infiltration. In the first year, we used portable research equipment from Cornell University (van Es and Schindelbeck, 2003). We involved three earth science classes from a rural high school in an outdoor, guided inquiry unit, in which students were given a question and the experimental set-up, and were responsible for analyzing and interpreting the data.

top using rubber stoppers with movable tubes, such that the amount of rainfall per unit time could be held approximately constant (Ogden et al., 1997; van Es and Schindelbeck, 2003). We built soil microcosms by drilling drainage holes into the bottoms of 7.6- to 18.9-L (2- to 5-gallon) plastic buckets, and a hole on the side of each bucket to hold a tube for diverting runoff into a receptacle (Fig. 1). In preparation for their experiments, students filled buckets with soil materials they had chosen in their research proposals up to the level of the runoff tube before making measurements.

In the second year, we modified the methods and approach. We revised the unit so that the necessary materials were more readily available to teachers, and so that students followed a more open inquiry process. This revised unit is the focus of this article. Table 1 shows the sequence of lessons in this revised second year of unit development. We designed multiple, smaller, indoor experimental set-ups (conceptually pictured in Fig. 1), requiring only basic equipment and materials, such as milk jugs, plastic buckets, rubber stoppers, tubing, an electric drill, carpet knives, pins, and a variety of soil materials with varying runoff and infiltration characteristics, all easily accessible to teachers. Student teams built rain-makers by adding a volumetric scale to the outside of a 3.8-L (1-gallon) milk jug, and poking small pin-holes into the bottom. Rain-makers were capped at the

10 4

Fig. 1. Conceptual diagram of runoff and infiltration measurement set-up that can be built for classroom experimentation out of available materials.

J o u r n a l o f N a t u ra l Re s o u r c e s & L i f e S c i e n c e s E d u c a t i o n • Vo l u m e 40 •  2011

In April 2007, 48 students, mostly 10th graders from three earth science classes in the same rural high school (all taught by the second author) were divided into teams of three to four students, strategically selected by the teacher to include a diversity of academic proficiency and perspective in each group. Student teams were introduced to the instrumentation through constructing the rain-makers, and to the subject matter of the unit through a hands-on activity that involved sponges as analogies for soils (Table 1, Moebius and Elsevier, 2008). Students then asked their own research questions and designed their experiments with the available equipment. Questions chosen by the teams included assessing the influence of compaction, vegetation, rock content, particle size, slope, and prior water content among others on runoff and infiltration partitioning. Throughout the unit, students completed a series of worksheets to guide their open inquiry, some individually, and some in teams. Students also completed journal entries, in which they answered a series of questions. These questions addressed the experiment performed in class and made connections to real-world issues, similar to those suggested on the unit website (Moebius and Elsevier, 2008). We added two further components: a final project to be completed by each individual student and a conference in which students displayed their projects and shared their findings with each other, and answered questions regarding observations and inferences made by their peers. Final project options for students included a research report; a letter to a curious friend; designing a poster, cartoon, or similar artistic representation; and the option to propose their own idea for the project format. An assessment rubric (Fig. 2), applicable to all options, was also provided. After students completed their final projects, the class conference was held during one class period. Students displayed their own projects around the classroom, and read and answered questions about several of their peer’s final projects. The goal of adding these two components to the unit was to help students integrate and communicate about their gained understanding and to help them establish a more complete perspective on the process scientists undergo in developing scientific knowledge. The unit was taught collaboratively by an earth science teacher and a graduate student during this case study, so that in-class observations could be made during teaching, but the unit can easily be taught by one teacher. The teacher’s roles included giving short lectures, referred to as “lecturettes” by Shipman (2004), as necessary, to introduce each day’s activities, summarize learning, draw connections to real-world events and scientific processes, or clear up common misunderstandings. Other roles of the teacher were those of mentor and collaborator and so forth as described by Crawford (2000). All teaching materials, including a detailed teacher’s guide, were updated using student feedback and experience from the second-year field-testing of this unit and were made available online (Moebius and Elsevier, 2008).

Student Assessment and Survey We administered a pre- and post-test to all students to assess student achievement, using 13 New York State standardized test questions from old multiple-choice Regents exams (New York State Education Department, 2009) and four short-answer questions of our own design. Questions

demanded a contextual understanding and some application of water runoff and infiltration concepts. Post-test questions used were identical to those on the pre-test. The purpose of these paired tests was to evaluate student gains, or learning, as a result of the unit, using authentic classroom assessment tools that teachers consider to be important indicators. For New York teachers to consider implementing new teaching strategies in the classroom, these must generally contribute to improving students’ Regents scores. Test questions generally assessed whether goals for gains were met in knowledge, comprehension, and some application, which are lower-order skills according to Bloom’s Taxonomy of learning (Santrock, 2006). Final projects were graded using the provided rubric. These allowed for assessment of whether proficiency goals in higher-level skills including analysis, synthesis, and evaluation according to Bloom’s Taxonomy of learning (Santrock, 2006) were met. These higher-level skills are generally not targeted by standardized testing, but are arguably essential skills for citizens to acquire. We also designed and administered a partially open-ended survey of student agreement or disagreement with a series of statements. Questions were targeted to assess students’ self-reported ability to carry out various aspects of scientific inquiry, their perceptions of learning from peers and the unit, and the extent to which they enjoyed the unit. Survey results were used as indicators of how well the activities engaged the students cognitively and emotionally in learning the broader unit content related to the process of scientific inquiry and authentic applications in contemporary environmental challenges.

Education Research Design and Data Sources The research design used for this case study involved a mixed methods approach (Creswell, 1998; Krathwohl, 1998; Miles and Huberman, 1994). Thus, in addition to the assessments described above, additional data included the authors’ journal entries written after each lesson and teacher observations of student attitudes, engagement, and abilities. The teacher contributed insights on these students, whom she had observed during regular class and laboratory work for at least 8 months before teaching this unit. At the end of the unit we solicited teacher feedback regarding necessary unit design changes.

Data Analysis Pre- and post-test questions were used to calculate a preand post-test grade (%) for each student (n = 48). Descriptive statistics were calculated for (1) the total test score, (2) the subset of Regents-only questions, (3) the subset of nonRegents-only questions, and (4) final project grades (n = 27). A one-tailed paired t-test of students’ pre- and post-test grades was used to determine whether overall test scores had improved (gain) for each grouping of test questions. The net % of students with gain for each question was reported separately as:

Net % gain = Improved % – Inferior % where Improved % is the percentage of students who improved their answer to the question on the post-test, and Inferior % is the percentage of students who gave a poorer


105

Fig. 2. Final project grading rubric for runoff and infiltration investigation.

10 6

J o u r n a l o f N a t u ra l Re s o u r c e s & L i f e S c i e n c e s E d u c a t i o n • Vo l u m e 40 •  2011

answer on the post-test. We ranked questions from most to least net student improvement. Regression r2 values for final project grades vs. test scores and test gains were determined. The Wilcoxon signed-rank test (Darlington, 1996) was used to assess average agreement or disagreement with survey statements. Average rank scores were compared to the neutral H0 of X = 2.5, and a table of Wilcoxon p values was used to determine statistical significance (Darlington, 1996). Two open-ended survey questions, in which students shared what they especially liked, and what suggestions for improvement they had, were analyzed by coding (Guptill, 2008; Strauss and Corbin, 1990). According to this method, investigators determined categories, reported in the results section, from student responses while reading their comments. The number and percentage of student responses fitting into each category were determined.

RESULTS AND DISCUSSION Pre-Tests and Post-Tests Pre- vs. post-test results (Table 2, Fig. 3) indicate that, on average, students performed significantly better on the posttest. The fact that a large number of students knew some of the material well enough to excel at the pre-test (average of 63%, and maximum of 98%) may in part be explained by the high school’s location in an agricultural area such that more students were somewhat knowledgeable about the concepts addressed by this unit. However, even with relatively high pretest scores, total test score average increased by 17%, while the standard deviation, and therefore the gap between mostand least-knowledgeable students, decreased (Table 2). Regents pre-test scores were higher than non-Regents scores, and gains were greater in the non-Regents (20%) than Regents questions (15%). There may be several reasons for this: Many students have a keen ability to read and eliminate multiple-choice options, allowing for better Regents pre-test scores. Conversely, our own questions applied more Table 2. Pre-test and Post-test score (%) outcomes, n = 48.

Pre-test Post-test Gain†

Total Test (17 questions) Avg. ‡

63

80

SD ‡

21

12

17§ 19

Min. ‡

10

40

–24

Max. ‡

98

100

71

Avg.

67

82

SD

23

16

21

Min.

0

31

–31

Max.

100

100

77

Avg.

55

75

SD

27

16

30

Min.

0

25

–50

Max.

100

100

94

Regents (13 questions) 15§

Non-Regents (4 questions)

20§

† Gain = post-test score – pre-test score. ‡ Avg. = average, SD = standard deviation, Min. = minimum score, Max. = maximum score. § All gains were significant with a one-tailed paired t-test at α < 0.0005.

Fig. 3. Gain from pre-test to post-test vs. pre-test score, with linear regressions for total scores (solid line), regents scores (dotted line), and non-regents scores (dashed line). directly to unit activities, so there was more overall room for improvement from lower scores, contributing to greater gain on non-Regents questions. These results suggest that most students learned a significant amount of science content material measurable by this test, including both content required in the state’s standards as measured by Regents questions (New York State Education Department, 2009) and further content deemed important by the teacher. A net 21 to 38% of students improved their answer to each of 7 out of the 17 questions on the post-test (Table 3). All four non-Regents short answer questions were among the most improved questions, as they were specifically created by the teacher to test for learning and ability to apply concepts taught during this unit. Two of these asked for a real-world implication of water runoff and infiltration, and one required a basic understanding of the water cycle that was reviewed and applied during the experiment. These questions required fairly simple short answers. The fourth short-answer question tested students’ understanding of observation and inference. This required a more sophisticated answer than any of the other questions, and seemed to be a difficult concept for the students to grasp. The low success rate (2%) on the pre-test, and the fact that few students received full points by the end of the unit, suggest that these concepts would have needed more discussion in class. However, nevertheless, 29% of students improved their answer to this question. One might underestimate the importance of these overall findings, and expect students to gain more understanding throughout a unit. However, the experienced teacher’s perspective is: “If a quarter of your students have improved on anything, then you’ve done a really good job in education.” These results are quite significant, especially when comparing the percentage of students with improved answers to the percentage providing imperfect answers on the pre-test (Table 3). When not including students who answered perfectly pre- and post-unit, improvement rates (net % improved per % with imperfect pre-test score) range from 19% (Question 5) to 100% (Question C), with on average 54% of students, who needed to learn more, improving their answers. Lower net improvement rates (4–19%) were found for


107

Table 3. Total increase from all classes in the net % of students that improved their answer for each question (Gain) from pre- to post-test, given in order from highest to lowest improvement by question. Letters designate short answer questions we designed, numbers designate Regents questions. Test question % Students with imperfect no. score on pre-test

Net % students with pre- to post-test gain

Question topic

A

48

38

Name three places where runoff goes

B

42

35

Give example of positive effect of infiltration

9

42

31

Choose condition causing increased surface runoff

D

98

29

Define difference between observation and inference

11

38

23

Choose graph relating particle size and infiltration

8

54

23

Choose situation with greatest surface runoff

C

21

21

Give example of negative effect of runoff

12

29

19

Relate particle size and infiltration

7

23

17

Choose condition to produce most runoff

1

42

15

Relate infiltration to saturation and permeability

2

44

15

Relate runoff and infiltration in graph

6

21

13

Choose runoff results with defined surface characteristic

5

56

10

Apply relationship between runoff and infiltration numerically

10

21

10

13

17

8

Choose probability of flooding related to runoff Relate infiltration to type of particle size mix

4

23

6

Relate infiltration to multiple concepts (particle size, slope, permeability)

3

15

4

Relate slope and permeability to infiltration

each of the remaining 10 questions. Several factors explain this: (1) More students already received perfect scores on the pre-test for some questions. (2) Some questions required especially careful reading, or a mixing of concepts that may have been more difficult. (3) We did not prepare students for specific types of test questions during this unit.

reasoning, synthesis and integration, evaluation, and expression of concepts based on an understanding of the nature of science. The skills students practiced here were thus much higher on Bloom’s Taxonomy of learning (Santrock, 2006) than the multiple choice or short answer questions asked by standard tests and more common end-of-unit assessments.

Overall, the data suggest that 4 to 38% of all students (19–100% of those who had room for improvement) improved their understanding of every question asked on the test. Most remaining students began with the right answers, whether as a result of guessing or prior knowledge is hard to tell, and thus could not have improved. However, 7 out of 48 students (15%) selected enough wrong answers on the posttest to receive lower post-test scores than pre-test scores (Fig. 3). This may be attributed to apathy sometimes found in high school students, or possibly differences in student health status between test dates. Seven students (15%) with pretest scores of less than 30% improved by more than 50% (Fig. 3), showing that this unit had the potential to significantly improve student understanding of the subject matter.

Final project scores were found to be bimodally distributed. The teacher noted she has seen bimodal distributions when there is either a lack of effort or extra effort on the part of some of the students. The distribution of final scores is normal (Shapiro-Wilk test, p = 0.44, JMP Statistical Software, Version 7) when the eight students with the grades in the highest range (75–81%) were not included, suggesting that these eight put in extra effort beyond the remaining students. Even the projects prepared by students with the greatest effort nevertheless only met 75 to 81% of the criteria. On the lower end of the spectrum (grades of 31–46%) a number of projects had obviously been given very little effort.

Final Projects Final project scores on average were much lower than post-test scores (avg. of 61%, with a SD of 14%, min. of 31%, and max. of 81%), despite a clearly laid out grading rubric (Fig. 2) that students received early in the unit, and some available class-time for consulting with the teacher, although students took advantage of this to a limited extent. The fact that students were only given one class period to perform the experiment itself is likely one contributing factor to lower project grades, as more time spent doing and revising the experiment would likely have contributed to higher quality discussion (see further discussion on this topic in student surveys and unit improvement). Low project scores indicate that this kind of exercise is much more challenging for students than a standardized test. The final project was a difficult task, demanding organization, critical thinking, planning,

10 8

Learning to communicate about scientific findings is often best achieved by a collaborative effort and for an authentic audience (Hand et al., 2004). A peer-review session of draft projects might have motivated students who contributed little effort to take the rubric and final project more seriously, by providing a more authentic process and audience during project preparation. Discussion and feedback might have allowed all students to develop a better ability to understand and communicate their findings in general, to better enable them to deliver projects that met the criteria. Assessing their peers’ projects would also have allowed students to grapple with the grading rubric from the other side, before using it to finalize their own projects. There was no significant correlation between final project scores and any pre- or post-, total or partial test scores, used alone or jointly, with r2 values of non-significant linear regressions ranging from