Trends in Excellence Gaps

1 downloads 19 Views 272KB Size Report
continued to grow while the proportion of lower SES 8 th graders performing at the same level had stagnated, leading to a growing gap over time between these ...
Trends in Excellence Gaps: A 12-Year Inte rnational Pe rspective

Authors: Leslie Rutkowski, Indiana University David Rutkowski, Indiana University Jonathan Plucker, Indiana University

International Excellence Gap Introduction Educational reforms and initiatives in many countries around the world are aimed at narrowing achievement gaps between important demographic groups. For example, the No Child Left Behind Act in the U.S. places a focus on minimum competency with the goal of raising achievement for the lowest performers to an agreed upon basic level of educational achievement. Other countries have used the Organisation for Economic Co-operation and Development’s (OECD’s) Programme for International Student Assessment (PISA) results to enact similar changes. For example, findings from PISA 2000, which ranked Germany’s school system 21st (out of 32 participating countries) (OECD, 2001), vigorously challenged the highly tracked German secondary education system. Specifically, PISA findings indicated that students in general and vocational tracks were far behind their academically tracked peers. In response to a call by German educationalists and policy makers to close the achievement gap of the lowest performing students, reforms included nation-wide education standards and a four-billion euros expansion of all-day schooling, and other initiatives intended to equalize education throughout the country (German Federal Ministry of Education and Research, n.d.). Few would argue that providing quality education for all students and raising achievement for low performers is critically important; however, a recent study (Plucker, Burroughs, & Song, 2010) showed that in the U.S., gaps between high achieving students from certain demographic groups continue to grow in many cases. For example, the authors showed that the proportion of high SES 8th graders performing at the advanced level in mathematics continued to grow while the proportion of lower SES 8th graders performing at the same level had stagnated, leading to a growing gap over time between these two groups. These findings suggest that not all students are being provided with the resources needed to reach their full 2

International Excellence Gap educational potential. Importantly, neglecting or under-serving groups of high performing students might have long-term economic consequences (Dillon, 2010; Hanushek & Rivkin, 2009). Further, the demand for a mathematically and scientifically highly-skilled workforce is projected to rise significantly both in the U.S. and abroad (National Science Board, 2003). And as many countries increase their investment in research and development to entice educated workers, there is increased global competition for this limited pool of employees (National Science Board). As such, it is important to understand whether math and science gaps among the highest performing groups exist and whether these gaps are changing over time. Background The investigation of excellence gaps is a relatively new area of inquiry with, to our knowledge, no international investigations regarding the current state of excellence gaps around the world. As such, this paper aims to analyze data from all four cycles of the International Association for the Evaluation of Educational Achievement’s (IEA’s) flagship study, the Trends in International Mathematics and Science Study (TIMSS) for the presence of excellence gaps in mathematics and science internationally. To this end, we concentrate our analysis on traditionally under-represented demographic groups, including girls, students with an immigration background, and students from less educated families. With critical and persistent shortages in jobs that require advanced levels of mathematics and science skills (National Science Board, 2003), understanding trends in mathematics and science excellence gaps is unquestionably an important line of inquiry. Although the investigation of excellence gaps has received little attention until recently, related U.S.-based studies have shown that achievement gaps tend to grow between Black and

3

International Excellence Gap White students over time (Hanushek & Rivken, 2006), even for students who otherwise begin school with equal levels of achievement (Reardon, 2008). Similar findings emerged in a review of the achievement gap literature (Farkas, 2009). The author documented emergent achievement gaps for U.S. ethnic minorities as early as the preschool. Further, these gaps were found to generally increase over time through high school. And the achievement gaps found in standardized tests extended to lower rates of college enrollment and persistence for minority students. Findings from these studies suggest that the most able students in vulnerable groups tend to underachieve, possibly limiting their long-term prospects for education and employment. Achievement gaps are also found to be a persistent problem between males and females across the educational trajectory, especially in the areas of science and mathematics. Similar to ethnic achievement gaps, gender-based differences in skills, particularly those associated with mathematics, have been found to emerge early in children’s development (e.g. Levine, Huttenlocher, Taylor, & Langrock, 1999). Further, mathematics gaps worsen as students progress through the educational system (e.g. Penner & Paret, 2008). These gaps have lasting effects as far fewer women enter academic fields that are in high demand such as science and engineering (National Science Foundation, 2002). Our study seeks to extend recent U.S.- focused excellence gap work (Plucker, Burroughs, & Song, 2010) to include an international focus. To that end, we examine excellence gaps in mathematics and science achievement in 16 countries that participated in all four TIMSS cycles (1995, 1999, 2003, and 2007). In particular, we examine the proportion of students from two policy-relevant demographic groups that achieve at the TIMSS advanced benchmark in mathematics and science (discussed subsequently) at each of four study cycles. That is, we compare the proportion of high achieving students in the focus group of interest (e.g. girls) to the 4

International Excellence Gap proportion of high achieving students in the associated reference group (e.g. boys). Given that international studies do not measure race or ethnicity, we instead focus on students from an immigration background. Specifically, we examine those students that were born outside of the country of the test. We then examine changes in these proportions over time. Given several time points and a relatively long time span from a policy perspective (12 years), changes in these proportions provide some evidence internationally of the expansion or contraction of excellence gaps in math and science within two highly relevant demographic groups.

Methods Our study uses a straightforward analysis of trends over time. In each country, we examine the proportion of study participants in important demographic groups who achieve at the TIMSS advanced benchmark in each of four study cycles for both math and science. As an example, our study estimates the proportion of girls vs. boys achieving at the advanced math and science benchmark in each country from 1995 to 2007. Differences in these proportions with standard error estimates are used to measured excellence gaps – the difference in the proportion of the focal group achieving at the advanced benchmark compared to the reference group. Excellence gaps are reported such that a positive gap indicates that the reference demographic group (e.g. boys in a gender comparison) has a higher proportion of examinees achieving at the advanced benchmark than does the focal group (e.g. girls). Statistically significant differences in proportions between groups at a given point in time are measured with a standard error estimated via the jackknife repeated replication (JRR) method (Rutkowski, Gonzalez, Joncas, & von Davier, 2010). Changes over item in the proportion of boys or girls achieving at this level are indicative of a growing or shrinking excellence gap across gender. Statistically significant

5

International Excellence Gap growth or shrinkage in the excellence gap over time (between the 1995 cycle and the 2007 cycle) is also measured with a JRR standard error. Trend tables and graphs are reported. For all analyses, achievement levels are measured using all five plausible value estimates and sampling weights are incorporated (Rutkowski et al.). Data TIMSS is an internationally comparable study of mathematics and science achievement conducted at the fourth and eighth grade in 59 countries (in the 2007 cycle) around the world. The unique design of TIMSS allows for an analysis of trends in achievement over time, with assessment cycles including 1995, 1999, 2003, and 2007. The target population of TIMSS is all students at the end of 4th and 8th grades in participating educational systems. In addition to assessing mathematics and science achievement of 4th and 8th graders internationally, TIMSS also collects a wealth of background data from students, teachers and principals of participating schools. The resulting database is a rich resource for policy makers and researchers interested in educational achievement and possible correlates. For our analysis, we chose to use student level data from the 1995, 1999, 2003, and 2007 TIMSS databases. Measures for this analysis include math and science achievement, whether the child was born in or outside of the country of the test and the gender of the child. Specifically, we compared the proportion of high achieving girls (vs. boys) and children born outside the country of the test (vs. domestically born children born). Because four cycle trends are only possible at the eighth grade (fourth grade was not included in 1999), we chose to analyze only grade eight data for our study. According to the TIMSS 2007 Assessment Framework and Specifications (Mullis et al., 2005), the 8th grade sample includes children aged 13 and 14,

6

International Excellence Gap where grade eight is defined as the upper of the two adjacent grades with the most 13-year-olds. As noted above, 16 countries participated in all four TIMSS cycles. These countries include Australia, Cyprus, England, Hong Kong, Hungary, Iran, Israel, Italy, Japan, Korea, Lithuania, Romania, Russia, Singapore, Slovenia, and the United States. Given our focus on excellence gaps at the student level, we classified a student as high achieving if they reached the TIMSS advanced benchmark. The TIMSS advanced international benchmark in mathematics is defined as an achievement level where: Students can organize and draw conclusions from information, make generalizations, and solve non-routine problems. They can solve a variety of ratio, proportion, and percent problems. They can apply their knowledge of numeric and algebraic concepts and relationships. Students can express generalizations algebraically and model situations. They can apply their knowledge of geometry in complex problem situations. Students can derive and use data from several sources to solve multi-step problems (Mullis, Martin, & Foy, 2008, p. 69). And the TIMSS advanced international benchmark in science is defined as an achievement level where: Students can demonstrate a grasp of some complex and abstract concepts in biology, chemistry, physics, and Earth science. They have an understanding of the complexity of living organisms and how they relate to their environment. They show understanding of the properties of magnets, sound, and light, as well as demonstrating understanding of structure of matter and physical and chemical properties and changes. Students apply knowledge of the solar system and of Earth’s features and processes, and apply 7

International Excellence Gap understanding of major environmental issues. They understand some fundamentals of scientific investigation and can apply basic physical principles to solve some quantitative problems. They can provide written explanations to communicate scientific knowledge (Martin, Mullis, & Foy, 2008, p. 96).

All other benchmarked achievement levels were omitted for the current analysis.

Results Mathematics In this section, we present the trends in mathematics excellence gaps between students who were born in the country of the test and students who were born abroad and between boys and girls. Table 1 includes the proportion (with standard errors) of native and foreign born students over time who achieve at the high benchmark in each country. Here, we can see that in general, the proportion of both native- and foreign-born grade eight students achieving at the high benchmark has been more or less stagnant or has seen modest upward growth internationally. A notable exception includes Israel, which has experienced a slight decrease in high achievers in both demographic groups. Table 2 specifically looks at the difference in proportions of boys and girls achieving at the high math benchmark over time. Figure 1 supports the tabled results and several findings are worth noting. In most countries, there is little evidence of changing math excellence gaps based on immigration background, which is in line with the findings that most countries have shown little progress in advanced achievement; however, exceptions exist. In particular, a growing and statistically significant excellence gap that favors native born students can be seen in Hong Kong, Romania and the U.S. The largest of these gaps is in Hong Kong, where in 2007 ten percent more native born students achieved at the high

8

International Excellence Gap benchmark than their foreign-born peers (33.6 percent compared with 23.3 percent). Large changes from cycle to cycle exist in Korea and Singapore; however, these findings are likely due to small sample sizes in the immigrant groups in these countries that achieved at the high benchmark. In Australia, a persistent excellence gap in favor of foreign-born students exists. Although, only in 1999 and 2007 were the gaps associated with immigrant status in Australia statistically significant. Also notable in these results are the countries for which no excellence gaps between native- and foreign-born students exist. These countries include Hungary, Iran, Japan, and England. It is important to note, however, that no statistics are available for Japan in 1995, since the question was not asked of Japanese participants during that cycle. Finally, the changes in excellence gaps between 1995 and 2007 generally support the finding that excellence gaps based on immigration status are generally static. Two countries showed significant growth in excellence gaps that favor native-born students: Hong Kong and Romania. The large changes noted in Korea and Singapore are, again, likely attributable to small sample sizes. The results for mathematics excellence gaps by gender can be found in Tables 3 and 4 and Figure 2. In contrast to the results on immigration-based excellence gaps, a fairly clear pattern of shrinking excellence gaps based on gender is notable in many countries. Specifically, Hong Kong, Hungary, Israel, Italy, Japan, Korea, Romania, and Russia are countries that demonstrate a shrinking gap that was formerly in favor of boys. Only one country, Singapore, exhibits a statistically significant and growing gap in favor of girls. Countries for which evidence of an excellence gap by gender has been absent across all four study cycles include Australia, Cyprus, and Lithuania. It is worth noting, however, that Australia demonstrated a general, although not statistically significant pattern of a growing gap that favors boys. Finally, countries with persistent if stagnant excellence gaps based on gender include the U.S. and Slovenia.

9

International Excellence Gap Table 1. Percent of native- and foreign-born students achieving at the TIMSS high benchmark in mathematics. 1995

1999

2003

2007

Native (SE)

Abroad (SE)

Native (SE)

Abroad (SE)

Native (SE)

Abroad (SE)

Native (SE)

Abroad (SE)

Australia

6.5 (0.8)

8.6 (1.5)

9.4 (1.6)

11.8 (3)

5.8 (0.9)

12.9 (3.3)

5.6 (1.2)

8.4 (2.6)

Cyprus

1.9 (0.3)

2.2 (0.9)

2.2 (0.4)

4.4 (1.5)

1.3 (0.2)

1.4 (0.7)

2.5 (0.4)

1.1 (0.7)

Hong Kong

19.3 (2.2)

22.6 (3.1)

28.7 (2.2)

26.9 (2.9)

33 (1.8)

26.2 (2)

33.6 (2.2)

23.3 (2.4)

Hungary

7.4 (0.7)

5.1 (2.7)

12.9 (1.2)

9.3 (5.9)

11.6 (1.1)

8.4 (3.6)

10 (1)

5.2 (4.1)

Iran

0.2 (0.1)

0.2 (0.2)

0.6 (0.2)

0 (0)

0.4 (0.2)

0.4 (0.4)

0.8 (0.2)

1.2 (1.2)

Israel

7.3 (1.4)

4.7 (1.8)

3.6 (0.5)

5.1 (1.4)

6.4 (0.7)

4.2 (1)

4.2 (0.6)

2.9 (1.1)

Italy

3.4 (0.4)

0.7 (0.7)

3.8 (0.5)

3 (2.8)

2.8 (0.6)

2.1 (1.2)

2.6 (0.6)

0.8 (0.8)

Japan

0.0

0.0

28.8

15.5

24.4

18.3

26.1

26

()

()

(0.9)

(6.9)

(1)

(6.7)

(1.3)

(8.7)

Korea

27.1 (0.9)

16 (5.7)

32.4 (0.9)

36.3 (7.6)

35.6 (1.3)

15.2 (4.9)

40.4 (1.1)

61.7 (13.9)

Lithuania

1.4 (0.3)

1.1 (1.7)

3.4 (0.6)

0 (0)

5.8 (0.6)

1.7 (0.9)

6.7 (0.7)

3.1 (1.4)

Ro mania

2.7 (0.4)

3.2 (2.1)

4.1 (1)

0.4 (0.6)

4.3 (0.6)

0.9 (1.7)

3.9 (0.6)

0 (0)

Russia

7.1 (0.8)

6.1 (1.5)

12.5 (1.7)

7.5 (2)

6.2 (0.8)

6.3 (2.1)

8.7 (0.9)

4.9 (2.4)

Singapore

33 (2.7)

35.2 (3.1)

40.5 (3.5)

52 (5.2)

45.4 (2.1)

39.1 (3.1)

38.4 (1.9)

53.9 (3.4)

Slovenia

6.9 (0.7)

4.7 (1.8)

12.6 (1)

7.3 (3)

3.2 (0.5)

2.6 (2.1)

4.3 (0.6)

1.6 (0.9)

USA

3.9 (0.6)

3.2 (1.3)

7.7 (1)

6.1 (1.4)

7.0 (0.8)

3.2 (0.8)

6.5 (0.6)

3.4 (1.1)

England

4.8 (0.7)

4.2 (2.2)

6.2 (0.9)

7.2 (2.1)

5.6 (1.1)

5.0 (2.2)

8.5 (1.5)

9.2 (3.7)

International

8.9 (0.3)

7.9 (0.6)

14.2 (0.3)

12.9 (0.9)

13.4 (0.3)

9.1 (0.7)

14.2 (0.3)

12.4 (1)

Country

10

International Excellence Gap Table 2. Trends in mathematics excellence gaps based on students’ immigrant status. 1995 Gap (SE) -2.12** (0.98)

1999 Gap (SE) -2.47 (2.81)

2003 Gap (SE) -7.06** (2.66)

2007 Gap (SE) -2.86 (2.14)

Cyprus

-0.28 (0.88)

6.96 (5.73)

-0.05 (0.60)

1.38** (0.63)

1.66 (1.08)

Hong Kong

-3.26 (2.13)

-2.13** (1.05)

6.85** (1.98)

10.35** (2.12)

13.61** (3.00)

Hungary

2.31 (2.09)

1.82 (2.29)

3.13 (3.17)

4.79 (3.12)

2.49 (3.76)

Iran

0.00 (0.22)

3.64 (4.16)

0.02 (0.16)

-0.40 (0.81)

-0.40 (0.84)

Israel

2.58 (2.04)

0.59** (0.12)

2.29** (0.98)

1.33 (0.92)

-1.25 (2.23)

Italy

2.72** (0.79)

-1.52 (1.25)

0.71 (0.94)

1.79** (0.82)

-0.93 (1.14)

Japan

0.00 (0.00)

0.78 (2.42)

6.08 (5.67)

0.13 (8.26)

0.13 (8.26)

Korea

11.17** (4.90)

13.30** (5.34)

20.34** (4.72)

-21.33 (12.82)

-32.50** (13.73)

Lithuania

0.33 (1.47)

-3.89 (4.70)

4.12** (0.71)

3.53** (1.17)

3.19 (1.88)

Ro mania

-0.49 (1.91)

3.36** (0.57)

3.40** (1.71)

3.94** (0.43)

4.42** (1.96)

Russia

0.97 (1.22)

3.65** (1.05)

-0.12 (1.71)

3.80 (2.14)

2.83 (2.46)

Singapore

-2.23 (2.31)

5.03** (1.49)

6.33** (2.68)

-15.44** (2.80)

-13.21** (3.63)

Slovenia

2.20 (1.57)

-11.50** (3.05)

0.59 (2.03)

2.71** (0.90)

0.51 (1.81)

USA

0.73 (0.92)

1.61 (0.91)

3.78** (0.89)

3.12** (1.01)

2.39 (1.37)

England

0.62 (1.71)

-0.99 (1.96)

0.62 (1.78)

-0.76 (2.53)

-1.38 (3.05)

International Average

1.02** (0.51)

1.34 (0.71)

4.35** (0.58)

1.72 (0.94)

0.70 (1.07)

Country Australia

Gap Change (SE) -0.74 (2.36)

**p