learning to teach for social justice: measuring

Measuring Changes 1/33

LEARNING TO TEACH FOR SOCIAL JUSTICE: MEASURING CHANGES IN BELIEFS SARAH ENTERLINE LARRY H. LUDLOW EMILIE MITESCU MARILYN COCHRAN-SMITH

Boston College Lynch School of Education

Paper prepared for: Annual Meeting of the American Educational Research Association Division K Symposium: “Teacher Education for Social Justice: When Ideology Meets Accountability” New York City, March 2008

Contact info: [email protected]


Introduction Preparing teachers to teach for social justice is becoming a common goal among teacher education programs throughout the country. While there are common themes that run across programs (e.g. recognizing disparities, enhancing the life chances of all pupils, diminishing inequities), there is considerable variation in what is meant by “teaching for social justice” (North, 2006). Furthermore, there are few sound instruments that attempt to measure teacher candidates or graduates with regard to this complex construct (Hollins & Guzman, 2005). In response, our research group, the Boston College Teachers for a New Era Evidence Team (BCTNE ET), has taken on the challenge of defining a measurable construct of “learning to teach for

social justice” and, correspondingly, has developed a series of instruments that can be used both reliably and validly to measure teacher candidates and graduates. One of these instruments is the subject of this paper. Specifically, we developed a 12-item “learning to teach for social justice-beliefs” (LTSJ-B) scale to assess teacher candidates’ and graduates’ beliefs and practices as they relate to teaching for social justice. This scale has been administered several times as part of a series of five surveys. These surveys are administered upon entry into the teacher education program (Entry Survey), upon graduation (Exit Survey), and one, two and three years after graduation when these graduates are in their first few years of teaching (One, Two and Three Year-Out Surveys). This survey system is a powerful mechanism for assessing teacher candidates and graduates over time, and has provided our teacher education program with significant evidence from which decisions have been are made (Ludlow et al., 2007). Due to the fact that the 12-item scale appears on all surveys within the survey system, we can assess LTSJ-B in a number of different ways. We can compare entering candidates’ beliefs about social justice to graduating candidates’ beliefs, which can then be compared to teachers’ beliefs in the field. Furthermore, we can compare beliefs about social justice to teaching practices using a modification of the LTSJ-B scale--the “learning to teach for social justice-practices” scale. These comparisons can assess the same people at different points within their


preparation and into their teaching, can be made across cohorts of candidates and graduates, and can compare the same peoples’ beliefs to their practices at the same point in time. Initial analysis of the LTSJ-B scale (Ludlow, Enterline, & Cochran-Smith, 2008) demonstrated the complexity of this construct and the various comparisons and analyses that are possible. These analyses are beyond the scope of one paper, but will continue to drive our research in the future. We are, however, at a point where we can make some of these comparisons to understand how “learning to teach for social justice” develops and changes across teacher preparation and into the profession. This paper, therefore, presents a series of analyses focused on the results from several administrations of this scale and specifically focuses on an assessment of changing beliefs as they relate to teacher candidates’ and graduates’ commitment to teaching for social justice. These changes are assessed both longitudinally (i.e., the same cohort of candidates/graduates at different points during preparation and beginning teaching) and crosssectionally (i.e., different cohorts of candidates/graduates at the same points in preparation or teaching career). The findings presented in this paper focus on responses from the Entry, Exit, and One Year Out surveys.

Background

Defining “Learning to Teach for Social Justice” The “teaching for social justice” agenda is grounded in scholarship, practice, and grassroots efforts that share common themes about teaching and learning. Ludlow, Enterline and Cochran-Smith (2008) argue that, “teaching for social justice builds on and requires knowledge (i.e., knowledge of content, pedagogy, learners, cultures, schooling, communities, as well as knowledge of self), interpretive frameworks (i.e., ways of understanding and acting on the events and processes of schooling based on the integration of knowledge with beliefs, values, ethics, moral commitments, and attitudes), and practices (including subject -specific pedagogies and strategies for supporting the learning process of English language learners [ELL], pupils with special needs, and pupils from a range of socioeconomic backgrounds).


Teaching for social justice also involves teacher commitment to being part of larger social movements by working as advocates and activists for their pupils.” (p. 194)

In addition to these ideas, we draw on previous conceptualizations of “teaching for social justice,” which includes teaching practice that is reflective and maintains an inquiry stance; that builds on the cultural, linguistic and experiential prior knowledge of pupils in the class; that enacts curriculum and pedagogy responsive to all pupils; that provides social supports for learning of all pupils; that enhances educational opportunities and life chances of all pupils; that questions, criticizes and actively works to break down institutional structures which maintain socioeconomic, racial, gender, cultural and language disparities; that prepares pupils to become active citizens in a democratic society; that partners with families and communities to promote pupil learning; and that challenges and changes the curriculum reflective of these beliefs (Ludlow, et al, 2008; Adams, Bell & Griffin, 1997; Cochran-Smith, 1999, 2004; Michelli & Keiser, 2005; Oakes & Lipton, 1999; Sleeter, 1996; Villegas & Lucas, 2003; Zeichner, 2003). Taken together, these ideas characterize a construct that is by no means simple to define; however, the growing body of literature regarding teaching for social justice does provide a series of components that can be addressed, we believe, on a survey in a reliable and valid way.

Measuring ““Learning to Teach for Social Justice” In developing the scale to measure this construct, our research group (the BC-TNE ET), comprised of experts in both measurement and evaluation as well as teacher education, came to a common understanding that a construct as complex as “learning to teach for social justice” may be conceptualized as a continuum along which teacher candidates and graduates differ. In other words, candidates and graduates differ in the extent to which they understand, accept, and are prepared to teach in ways consistent with social justice principles. Furthermore, not only do the respondents to our surveys differ from one another, the very nature of teaching for social justice assumes that candidates and graduates develop their commitment to these principles over time, where novice (or untrained) teachers would have less understanding of, and therefore exhibit less commitment towards, teaching for social justice. In contrast, experienced teachers, especially those prepared through a program with an explicit focus on


social justice, like Boston College, would exhibit a stronger commitment and understanding of teaching for social justice. This understanding of the construct was essential to our approach of how to begin to measure it. The current field of psychometrics consists of two primary measurement models, classical test theory (CTT) and item response theory (IRT). The basic psychometric tools of CTT include: factor analysis, reliability analyses (e.g., test-retest, internal consistency, and inter-and intrarater reliability), and validity analyses (e.g. content, construct, discriminant, divergent, predictive, and consequential). The limitations of CTT have been widely recognized (e.g., the ability estimate of a person is dependent on the difficulty of the items, the standard error applies equally to all ability levels, item discrimination can be too high) (Wainer, 1986; Masters, 1988; Traub, 1997; Hattie, Jaeger & Bond, 1999; Brennan, 2001) and have led many investigators to employ the principles underlying IRT. Researchers who embrace the principles of the Rasch IRT measurement model assume that every variable stretches across a continuum of simple-to-complex tasks, or levels of knowledge, or affective characteristics, or cognitive abilities, depending on the variable one is aiming to measure. With regard to instrument development, there is a very deliberate attempt to construct a hierarchical arrangement of tasks, like a ladder, that a person is located along and who can then be expected to advance along (or up the ladder) over subsequent examinations. No matter the construct being measured, a tim e-consuming, literature-exhausting variable definition, which commences in item development, is conducted. Following this, the instrument is administered to an appropriate sample of people (usually it is first administered as a series of pilot studies), responses are analyzed, followed by a determination of the extent to which the instrument functioned as intended, or the extent to which the data fit the model. The results are evaluated to see if there is evidence to support the existence of a unidimensional variable stretching across the a priori defined continuum, the extent to which people have given responses consistent with their estimated locations on the variable, and the extent to which items provoked responses consistent with their location on the variable. With regard to measuring “learning to teach for social justice,” a Rasch model was employed not because it “fit” the data better than any other model, but because when the data fit the


model, teacher candidates and graduates can be ordered along a continuum based on their endorsement of simpler to m ore complex beliefs and we can measure changes in their commitment over time as they progress up the scale. Specifically, once “learning to teach for

social justice” was operationally hypothesized a pr iori in a continuous and hierarchical manner, we were able to conceptualize beliefs about this construct (through the scale items) that corresponded to varying levels of commitment. By constructing the scale in this fashion, we gained the opportunity to both conceptually and literally locate and position a teacher candidate or graduate on the variable and describe the types of beliefs, or items, that are most closely associated with that person’s score (defining their position or location) on the instrument (the variable being measured).

Scale Development From a set of over 200 potential social justice-related items gathered from the literature on teaching for social justice, a subset of 25 items comprised the initial pilot administration of the LTSJ-B scale that was administered to 284 graduating teacher education students in the spring of 2004. Numerous subsequent analyses led to the final set of 12 social justice items now integrated into the BC TNE Entry Survey, Exit Survey, One Year-Out Survey, Two Year-Out Survey, and Three Year-Out Survey system (Ludlow, et al, 2007). The LTSJ-B scale is administered to the teacher candidates and graduates at crucial times in their preparation and practice. Two versions of the scale exist. The items are exactly the same on the Entry, Exit, and One Year-Out Surveys, where they address candidates’ and graduates’ beliefs regarding teaching for social justice. While the concepts and subjects addressed in the items remain the same, the One, Two, and Three Year-Out Surveys include the second version of the scale where graduates are asked about their classroom practices regarding social justice. Thus, with the exception of the One Year-Out Survey, only one version of the LTSJ scale appears on each survey; the One Year-Out S urvey contains both the beliefs and practices scales. The items from the Beliefs scale, which is the focus of this paper, are presented in Figure 1. For each item, respondents are asked to answer using a 5- point rating scale (1=Strongly Disagree, 2=Disagree, 3=Uncertain, 4=Agree, and 5=Strongly Agree). Some items are positively;


others are negatively worded. Candidates and graduates were expected to agree with items 1, 2, 4, 7, and 8 and disagree with items 3, 5, 6, 9, 10, 11, and 12. The items where respondents were expected to disagree with were reverse-scored (“R”) in order for higher scores (e.g. closer to 5) to correspond to a stronger commitment to teaching for social justice. Once the items are coded in the appropriate direction, a higher total score corresponds to a higher level of commitment to teaching for social justice. Figure 1. The Learning to Teach for Social Justice—Beliefs scale Respond to the following statements regarding your beliefs about teaching. ab 1

An important part of learning to be a teacher is examining one’s own attitudes and beliefs about race, class, gender, disabilities, and sexual orientation

2

Issues related to racism and inequity should be openly discussed in the classroom.

3R

For the most part, covering multicultural topics is only relevant to certain subject areas, such as social studies and literature.

4

Good teaching incorporates diverse cultures and experiences into classroom lessons and discussions.

5R 6R 7

The most important goal in working with immigrant children and English language learners is that they assimilate into American society. It’s reasonable for teachers to have lower classroom expectations for students who don’t speak English as their first language. Part of the responsibilities of the teacher is to challenge school arrangements that maintain societal inequities.

8

Teachers should teach students to think critically about government positions and actions.

9R

Economically disadvantaged students have more to gain in schools because they bring less into the classroom.

10R

Although teachers have to appreciate diversity, it’s not their job to change society.

11R

Whether students succeed in school depends primarily on how hard they work.

12R

Realistically, the job of a teacher is to prepare stude nts for the lives they are likely to lead.

a b

Likert response categories: Strongly Disagree=1, Disagree=2, Uncertain=3, Agree=4, Strongly Agree=5 R: denotes the categories were reverse scored.

The first administrations of the Beliefs scale defined a baseline for the “learning to teach for

social justice” scale. Results demonstrated that the scale was invariant across time of testing and experience with teaching. This property of invariance is a necessary condition for subsequent efforts to measure change. In addition, the mean scores were relatively low for incoming classes as measured by the Entry Survey; graduating classes scored higher than the entering class as measured by the Exit Survey, establishing evidence of discriminant validity of the scale (Ludlow, et al, 2008).


Specifically, factor analyses identified the factor structure of the items, with the purpose of seeing if a unidimensional construct could be supported and lend itself well to Rasch analysis. Using principle axis factoring (thereby choosing to analyze only common variance) with a varimax rotation, two factors were extracted; one factor defined by the seven reverse-scored items; one factor defined by the remaining five items. Given these two factors, initially we were uncertain whether the two clusters of items actually measured two underlying constructs, or whether the resulting two-factor solution was a methodological artifact of the wording of the items. Further analyses provided evidence that while the two clusters of items address different aspects of learning to teach for social justice, the two clusters of items measure simple and complex aspects of a common construct (see Ludlow, Enterline & Cochran-Smith, 2008 for a detailed presentation of this finding and interpretation of the differences between these two sets of items). After the factor structure was established, reliability analyses were conducted. For respondents who completed one of the 2005-2007 Entry surveys, the responses to the 12 items of the Beliefs scale generated Cronbach alphas consistently around 0.80 at each administration. For those who completed the Exit Survey between 2005 and 2007, the responses consistently generated alphas around .74. Although the survey items were constructed to be comparable across different surveys, there was no prior evidence to suggest how comparable, if at all, they would be across cohorts of teacher candidates and graduates at different time points in their teacher education program and teaching career. Consistently we found the same “two factor” solution for all of the exit, entry and one year out surveys. Coefficients of congruence were computed to examine the extent to which the two factors were similar across surveys. Cureton and D’Agostino (1983) suggest that if a set of items administered in two studies (i.e. the same scale used on different surveys) really represent the same factor, the coefficient should be at least 0.90. The coefficients of congruence, calculated across entry and exit surveys for the positively and negatively worded items were 0.95 and 0.97 (Ludlow, et al, 2008), respectively. The Rasch rating scale model was then applied to these data (Andrich, 1978; Wright and Masters, 1982). This model is appropriate when the scoring categories have rigorously defined


scoring rubrics that do not depend upon the characteristics of the specific items. The WINSTEPS software package was used (Wright & Linacre, 1998). xni

∑ [ β n − (δ i +τ j )]

π nix =

e j =0

k

m

∑e

[ β n − ( δ i + τ j )] ∑ j =0

k =0

The π nix is the probability of person n responding in category x to item i where

δ

i

is the location

(generically referred to as item ”difficulty”) of item i on the variable, τ j is the location (“threshold” parameter) of the kth transition from one scoring category to the next for the m+1 rating categories, and

β

n

is the parameter for an individual’s "level of commitment to teach for

social justice". These location estimates are reported in a metric referred to as logits (Ludlow & Haley, 1995). These estimates simultaneously portray the structure of the Learning to Teach for

Social Justice-Beliefs variable and the location of each teacher candidate along the variable.

A Baseline Measure for Beliefs about Teaching for Social Justice: The 2005 Exit Survey Figure 2 contains the variable map for the 2005 Exit survey. The items are ordered from easiest to endorse (bottom right of the map) to hardest to endorse (top right of the map). The teacher candidates are ordered from lowest scoring (bottom left of the map) to highest scoring (top left of the map) in terms of learning to teach for social justice-beliefs. As demonstrated by the easiest to endorse items, it is easiest to strongly agree with items SJ1 and SJ4 (examining one’s own beliefs, and incorporating diverse cultures), followed by items SJ8 (think critically about government policies), SJ2 (racism and inequity openly discussed), and SJ7 (challenge school arrangements). At the top of the map are the hardest items to endorse (by “disagreeing”). Item SJ12R (prepare students for lives they are likely to lead), is harder to reject than item SJ11R (success in school depends primarily on how hard they work). This is followed by item SJ5R (assimilate immigrant children and ELL into American society) and item SJ3R (covering multicultural


topics is only relevant to certain subjects). In the middle of the distribution are items SJ6R, SJ9R, and SJ10R (lower expectations for ELL, economically disadvantaged bring less and gain more, and it is not the teacher’s job to change society, respectively). Proceeding up the scale means addressing increasingly more complex, controversial, and debatable choices. Figure 2. 2005 Exit Survey Variable Map. INPUT: 224 Persons 12 Items MEASURED: 224 Persons 12 Items 5 Categories EACH '#' IS 3 candidates

5

4

3

2

1

0

-1

-2

Persons - MAP - Items | . + | | . | | | + | . | | | .# | + .# T| | #### | | #### | ### S+ ##### | .# |T .##### | .######## M| sj12 ##### | sj11 .######### + .###### |S ######### | .# S| sj10 .## | sj9 ### | sj6 . +M sj3 | sj8 T| sj7 .# | sj2 | |S + | | | sj1 |T | + |

sj5

sj4


The operational definition of the scale revealed in this variable map is meaningful and consistent with a priori expectations. The separate person and item “fit” analyses revealed occasional odd patterns of responses by individual candidates but no evidence of systematic misfit. The two sets of item and person “separation” statistics suggested reasonable spread in the person and item location estimates for both administrations. The category threshold estimates confirmed that the response categories had performed in the monotonic order expected of them. In order to measure change, however, the measure must remain fixed. This means that some solution must serve as the frame of reference either as a baseline from which change may be marked or as a baseline towards which change may proceed. A reasonable argument can be made that the LTSJ-B scale, and all corresponding item and person estimates, should be anchored on the 2005 administration of the Exit Survey. This administration represents a reasonable baseline measure of “learning to teach for social justice” since it was the first time the

Beliefs scale was administered to a cohort of graduating teacher candidates. This solution thus represents the desired outcome to which the entering candidates would be compared in terms of “how far they have to go” (Ludlow, Haley & Andres, 2005) or, in the present context, the extent to which they would strive to “learn to teach for social justice”. Anchoring on the Exit estimates is also reasonable, strictly from the scale definition perspective, since the items at Exit form a clearer variable continuum than they did on the 2005 Entry Survey--an expected outcome, since many entering students are simply unfamiliar with many of the aspects of social justice presented on the Entry Survey. Hence, the item and category threshold estimates from the 2005 Exit Survey were used as “anchor” estimates for all other applications of the LTSJ-B scale. This procedure was performed with WINSTEPS.

Classical Approaches to Measuring Change

Changes in Scale and Item Variances An interesting way to examine changes in responses is by examining the reliability and scale score variance estimates. The reliability estimates for the Entry Surveys, (α approximately .80), mentioned above, were noticeably higher than the reliability estimates for the Exit Surveys


(α approximately .74). Interestingly, the scale variance estimates on all Entry Surveys (between 35 and 42) were also considerably higher than the scale variance estimates on the Exit Surveys (approximately 30 on all administrations). Recalling that the magnitude of the Cronbach alpha is a function of scale variance, this slight decline in alpha between Entry and Exit Surveys is understandable and meaningful. This reduction in scale variance may itself be understood as a program effect of having brought about a measurable degree of homogeneity to the students after four years of study. Because the new teacher candidates may not have had similar past experiences that would influence their attitudes toward social justice, it would be understandable that there would be some initial greater variability within their responses. The graduating candidates, however, have engaged in similar experiences at Boston College as part of the teacher education curriculum, and it would be plausible that their beliefs and attitudes toward teaching for social justice would not vary as much as the entering teacher candidates (who may not have had similar experiences). Thus, the lower scale variance on the Exit Surveys than the Entry Surveys contributed to the lower reliability estimates on the Exit Surveys, given the relation between Cronbach’s alpha and scale score variance.

Changes in Mean Person Estimates In addition to assessing the scale reliability and variance, a more traditional analysis of the average scores on the "learning to teach for social justice-beliefs" scale demonstrates changes in teacher candidates' and graduates' commitments towards teaching for social justice. Rather than computing total scores based on the 1-5 range of the scale, however, we computed mean person estimates based on the Rasch analysis. The mean person estimates provide substantively more information than a simple total score for two reasons: first, the estimates are scaled using a logit scale, which allows for common metric comparisons to be made across people and items; second, these estimates are based on anchored item and threshold estimates as determined by the 2005 Exit Survey. Accordingly, the mean person estimates provide a richer representation of the candidates and graduates than can be done using simple total scores.


To understand the utility of these estimates, one must understand the logits on which they are scaled. In general, a logit is a unit of measurement that can be applied to person estimates as well as item estimates. Logits are referred to as "log odd units," and correspond to the natural log of the odds associated with a person's or item's responses. The "odds" refers to the probability of an event occurring versus not occurring, such as a response corresponding to a strong commitment to teaching for social justice. Logits are computed to place both person estimates and item estimates on the same interval-level scale of measurement, thereby establishing the capability to compare across people and items, as well as across different groups of respondents and administrations. A high positive person logit corresponds to a high level of commitment to teaching for social justice; a low negative logit corresponds to a lower level of commitment. A high positive item logit corresponds to a hard to endorse social justice item; a low negative logit corresponds to an easy to endorse item. The following discussion focuses on the mean person estimates for three administrations of the Entry Survey (2005, 2006, and 2007), three administrations of the Exit Survey (2005, 2006, and 2007) and two administrations of the One Year-Out Survey (2006 and 2007) in order to demonstrate the magnitude of the changes that we observed in commitments towards teaching for social justice across teacher preparation and into the field. These estimates are presented in the table below. Table 1. Changes in Mean Person Estimates across Surveys. Mean Person Estimates 2005

2006

2007

Entry

.80

.79

.86

Exit

1.30

1.35

1.41

One Year-Out

N/A

1.33

1.27

Initially, we hypothesized that, although entering candidates may vary in their commitment to teach for social justice (as measured by the variance estimates), the mean person estimates for entering candidates would remain approximately the same across 2005, 2006, and 2007 Entry Surveys. This is because recruitment and admissions procedures did not change greatly in these years (even though selectivity of the undergraduates did increase slightly), and there would be


no reason to expect a great difference among these groups. As hypothesized, we found that the mean person estimates, presented in Figure 3 were consistent between 2005 and 2007 in the 0.80 range; however, it is important to note that the mean person estimate for the 2007 Entry Survey was higher than the estimates for the previous two administrations. Figure 3. Mean Person Estimates for 2005, 2006, and 2007 entering candidates.

Mean Person Estimates

1.5 1 0.5

0.79

0.8

0.86

2006

2007

0 -0.5 -1 -1.5 2005

Entry Surveys

We also examined the mean person estimates for teacher candidates on the Exit Survey. Between 2005 and 2007, the teacher education department’s curriculum emphasized teaching for social justice in a more explicit and coherent manner as an intentional component of the TNE project. Specifically, the faculty in the teacher education department formed a committee around the social justice theme of the Lynch School of Education, and intensified their emphasis of teaching for social justice in coursework by explicitly discussing principles and practices of teaching for social justice in methods and theory courses. For example, with regard to this scale in particular, the complexity of certain issues relating to teaching for social justice (e.g. incorporating diversity, challenging societal inequities) as well as the complexity of measuring this construct (e.g. what is the “right” answer?) were each discussion points within courses. Thus, we hypothesized that the mean person estimates for teacher candidates on the Exit survey would increase between 2005 and 2007. Again, as expected, the mean person estimates for graduating teacher candidates, presented in Figure 4, increased from 2005 to 2007, from 1.30 in 2005 to 1.41 in 2007.


Figure 4. Mean Person Estimates for 2005, 2006, and 2007 exiting candidates.

Mean Person Estimates

1.5

1.29

1

1.35

1.41

2006

2007

0.5 0 -0.5 -1 -1.5 2005

Exit Surveys

To measure change in commitment to teaching for social justice, we compared the mean person estimates of the teacher candidates at Entry and Exit between 2005 and 2007. It is important to note that when examining the differences between the mean person estimates at entry and exit, we are not comparing the same people. However, because the mean person estimates at Entry are consistent between 2005 and 2007, we expect that the mean person estimates from the Entry survey are representative of entering teacher candidates and would be appropriate for comparison. As demonstrated in Figure 5, the mean person estimates for entering teacher candidates are considerably lower at Entry than they are at Exit. Moreover, Figure 5 again highlights the increase in mean person estimates across the Exit surveys between 2005 and 2007. Figure 5. Comparison of mean person estimates at Entry and Exit between 2005 and 2007.

Mean Person Estimate

1.5

0.5

0.79

1.41

1.35

1.29

1

0.86

0.8

0 -0.5 -1 -1.5 Entry

Exit 2005

Entry

Exit 2006

Entry

Exit 2007

Surveys

We also examined the mean person estimates on the “learning to teach for social justice” scale for those who responded to the One Year-Out survey. These responses reflect the views of


those who graduated one year prior and who had taught for one year when they completed the survey. Across 2006 and 2007, the mean person estimates for BC teachers after one year of teaching was approximately 1.30, slightly less than the mean person estimates for teacher candidates at the time of graduation. This finding demonstrates that teachers’ commitments towards teaching for social justice after being in the field for one year are similar to their beliefs when they graduated from their teacher education program. This finding will be further investigated in a later section of this paper.

Measuring Learning to Teach for Social Justice As presented and discussed in the previous section, the 2005 Exit Survey serves as the baseline from which we measure “learning to teach for social justice-beliefs.” This administration first establishes the initial level of the graduating teacher candidate’s commitments to teaching for social justice, as determined by the beginning of the Teachers for a New Era project. Second, it establishes the point at which we expect entering teacher candidates to get to as a result of their teacher education program at Boston College. Third, and finally, it establishes the level of commitment a beginning teacher has prior to their entry into their full-time teaching positions and roles. There are some exceptions to this use of the 2005 Exit Survey (for example, impact of previous teaching experience, time spent in preparation, whether or not the graduate enters the teaching field immediately); however, we are confident that these exceptions do not limit our ability to draw conclusions about teacher candidates’ and graduates’ changing commitments to teaching for social justice as measured by this Beliefs scale. The following discussion aims to present these changes across three time points for different cohorts of teacher candidates and graduates. Specifically, we present results from the 2005, 2006, and 2007 Entry Survey, the 2005, 2006, and 2007 Exit Survey, and the 2006 and 2007 One Year-Out Survey. Figure 6 places the mean person estimates, presented in the previous section of this paper, on the variable map for the 2005 Exit Survey in order to demonstrate the location of each of these cohorts. The differences in location for these cohorts comprise the focus of the discussion below.


Figure 6. Mean Person Estimates on the Learning to Teach for Social Justice—Beliefs scale

2007 Exit Survey: 1.41 2006 Exit Survey: 1.35 2005 Exit Survey: 1.29

2006, 2007 Year-Out Survey: 1.33, 1.35

2005, 2006, 2007 Entry Survey: .80, .79, .86

Entering teacher candidates’ commitments to teaching for social justice Even though there are differences between graduates and undergraduates, entering teacher candidates at Boston College are not unfamiliar towards, nor do they lack opinions about, the concepts addressed on the “learning to teach for social justice-Beliefs” scale. However, as presented earlier, their commitment to teaching for social justice is weaker than graduating candidates or practicing teachers, on average. By employing a Rasch measurement model, we can not only better estimate the magnitude of these differences, through anchoring techniques, but can determine the items on this scale that more directly reflect these changes. The variable map for the 2005 Exit Survey, shown above, arranges the 12 items on the scale from those corresponding to weaker commitments to teaching for social justice to stronger commitments, where the first question (sj1) is easiest for respondents to agree with and the last question (sj12) is hardest. The respondents are placed along this same continuum in order to assess their beliefs as determined by their collective responses to certain items. Across three


cohorts of entering teacher candidates, the mean person estimates were similar: 0.80 for the 2005 Entry, 0.79 for the 2006 Entry, and 0.86 for the 2007 Entry. Even though the 2007 entering candidate’s demonstrated somewhat stronger commitments than the 2005 and 2006 cohorts did, these estimates correspond to the same relative placement on the variable map, and, more importantly, the same response patterns for the 12 items. Specifically, entering students are expected to respond to items 1-10 in ways that demonstrate they are committed to teaching for social justice. For example, entering students, on average, believe “an important part of learning to be a teacher is examining one’s own attitudes and

beliefs… ,” “issues related to racism and inequity should be openly discussed…,” “teachers should teach

students to think critically about government positions and actions,” and that it is not “reasonable for teachers to have lower classroom expectations for students who don’t speak English as their first

language,” (sj1, sj2, sj8, and sj6, respectively). On the other hand, the responses of entering students to items 11 and 12 do not reflect strong commitments to teaching for social justice. These students, on average, believe that success is based primarily on effort (sj11) and that teaching based on any preconceived notions towards the lives of their students is realistic (sj12)—both beliefs are contrary to the principles of teaching for social justice (Cochran -Smith, 1999; 2008). As mentioned above, these results are based on the average response across all entering candidates. However, these candidates are spread across this continuum, such that some candidates provided response patterns that related to stronger commitments to teaching for social justice than their peers do, and conversely, some candidates provided response patterns that related to weaker commitments to teaching for social justice. Specifically, the person estimates ranged from a low of -1.30 to a high of 4.45 across these three Entry Surveys. Due to the fact that the lowest item estimates were for item 4, “good teaching incorporates

diverse cultures and experiences…” (-1.54), and item 1, “An important part of learning to be a teacher is examining one’s own attitudes and beliefs…” (-1.46), it can be concluded that all entering teacher candidates can be expected to agree with these two items, as demonstrated across three subsequent cohorts, regardless of their responses to the remaining ten questions. This conclusion is based on the fact that the lowest person estimate is higher than the item estimates


for both sj4 and sj1. Conversely, approximately 40 entering candidates, depending on the cohort, provided responses that correspond to the strongest commitm ent possible (i.e. they disagreed with items 11 and 12), because the highest person estimate (4.45) was markedly higher than the highest item estimate of 1.31 for sj12. To summarize, entering teacher candidates demonstrate that they are at least somewhat committed to teaching for social justice, as it is defined by these twelve items, with several demonstrating a strong commitment. This is understandable, given that the mission of our preparation program explicitly includes the goal of teaching for social justice; therefore students who apply to our program do so most likely because they agree, at least somewhat, with this mission. However, the responses of entering candidates stretch across a wide range for most of the twelve items, indicating not only that their commitments to teaching for social justice can be strengthened but their understanding of these specific concepts can be enhanced.

Exiting teacher candidates’ commitment to teaching for social justice While entering teacher candidates across the three cohorts demonstrated that they were fairly committed to teaching for social justice, on average, graduating teacher candidates demonstrated considerably stronger commitments. We interpreted this finding to mean that these teacher candidates have “learned to teach for social justice” as a result of the teacher preparation program —they understand the issues addressed by the 12 items more completely, and their response patterns more often indicate that they provide responses that we attribute to a strong commitment towards teaching for social justice. The mean person estimates for the 2005, 2006, and 2007 Exit Surveys were 1.29, 1.35 and 1.41, respectively, which were at least 0.43 logits above the highest mean estimate for the Entry Survey (0.86). These person estimates ranged from a low of -0.70 to a high of 4.45 across the three Exit Surveys. Therefore, the lowest scoring person at Exit was 0.60 logits higher than the lowest scoring person at Entry (-1.30), whereas the highest scoring person was equal for both Entry and Exit Surveys (4.45). This finding illustrates an interesting limitation of this Beliefs scale, and Rasch scaling in general; these 12 items appear to do a good job of spreading respondents out along the lower end of the scale (i.e. where the item location estimates are), but


are limited in their ability to locate respondents at the higher end whose estimates are higher than the average estimate of the items. In other words, it is possible that responses to these 12 items are demonstrating a ceiling effect. With regard to the specific items, only the mean person estimate for the 2005 Exit Survey (1.29) was at or below the highest mean item estimate (sj12 = 1.31); both the 2006 and 2007 administrations of the Exit Survey had mean person estimates above all twelve mean item estimates (1.35 and 1.41, respectively). In other words, candidates who graduated in 2005 struggled with the concept of whether or not teaching based on preconceived notions towards student lives was realistic, and, on average, did not respond in a way that demonstrated a strong commitment towards teaching for social justice (i.e. they did not “strongly disagree” with this item). The responses from candidates who graduated in 2006 and 2007, on the other hand, did suggest that they understood this concept and, on average, disagreed with, “Realistically, the job of a teacher is to prepare students for the lives they are likely to lead.” This finding leads to an additional conclusion based on the three Exit Survey administrations. Each year, we found that the mean person estimate rose by 0.06 logits. This demonstrates an increased change from Entry in commitment towards teaching for social justice each year. This change is understandable given several programmatic changes that have occurred within the teacher education program that reflect an increased emphasis on teaching for social justice (e.g. increased requirements for teaching ELL students); therefore, candidates have had increased exposure towards and coursework related to the concepts, issues, and theories of teaching for social justice each year. In general, the Teachers for a New Era project itself has initiated several discussions, among candidates and faculty, around the meaning of teaching for social justice with regard to underlying theory and philosophy, as well as the challenges of measuring and assessing this complex construct. By understanding the context of this program, then, it is clear that increases in these mean person estimates over subsequent cohorts of graduates were likely and were expected. Furthermore, the increased emphasis on teaching for social justice, which has also been apparent in our publications and recruitment materials, may help explain why the 2007 entering cohort demonstrated a stronger incoming


commitment to teaching for social justice than the 2005 and 2006 entering cohorts. We expect this pattern to continue through the 2008 Exit Survey.

Practicing teachers’ commitment to teaching for social justice The Beliefs scale also appeared on the 2006 and 2007 One Year-Out Surveys, alongside the 12-item Practices scale. This allows for comparisons to be made between beliefs and practices as they related to social justice by assessing the same people at the same time. Unfortunately, this comparison is beyond the scope of this paper; however, we can garner some sense of how practicing teachers’ beliefs change once they are in their own classrooms by comparing results from the Beliefs scale at One Year-Out to the Beliefs scale at Exit. In general, practicing teachers, after they have had one full year of teaching experience, are equally committed to teaching for social justice as graduating candidates. The mean person estimate for the 2006 and 2007 One Year-Out Surveys was 1.33 and 1.35, respectively, which is at or near the same estimate as the 2006 Exit Survey. In other words, we conclude that our graduates maintain the level at which they are committed to teaching for social justice at graduation into at least their first year of teaching. Plus, practicing teachers, on average, are still grappling with similar concepts regarding teaching for social justice as much as they were at graduation. While the mean person estimates provide one picture of the degree to which practicing teachers are similar to graduating candidates, the range of individual person estimates on the One Year-Out Surveys paints a somewhat different picture. The lowest person estimate across three administrations of the Exit Survey was -0.70; however, the lowest person estimate for the One Year-Out Surveys was -0.61. This indicates that there was some movement upwards amongst individual respondents; however, due to the fact that none of the item estimates were between -1.46 (sj1) and -0.56 (sj2), we are unable to draw any conclusions regarding changes in response patterns from these estimates alone. In other words, because none of the items were located within this range, we cannot assess the specific concepts regarding teaching for social justice that these different respondents are struggling with. In addition, the highest mean person estimate remained at the same level as was observed for both the Entry and Exit Surveys (4.45).


These results paint an interesting picture. When we assess these candidates and graduates with regard to the item estimates it appears as if the limitless number of contextual variables that are present within the school context during a teachers’ first year did not significantly influence a teachers’ beliefs towards teaching for social justice—responses to the Exit and One Year-Out Surveys were similar with regard to the mean person estimate as well as the lowest and highest estimates. Additionally, while there were considerable changes in the mean person estimates across the three surveys (generally from Entry to Exit), the lowest person estimate always fell in between the item estimates for items 1 an d 2, indicating that candidates and graduates were always expected to agree with “an important part of learning to be a teacher is

examining one’s own attitudes and beliefs… ” (sj1) regardless of when they were surveyed, and always struggled with their responses to “issues related to racism and inequity should be op enly

discussed in the classroom” (sj2). These findings speak to the limitation of only assessing items using their estimates on the logit scale, especially when the items are not dichotomous items. In general, the rating scale used on the “learning to teach for social justice-beliefs” scale can elicit a great deal more information about the way respondents answer these questions than can be seen by assessing the individual item locations themselves. The next section of this paper presents these findings again, but through the lens of changing responses on a 5-point scale.

Changes within the Rating Scale The scale structure portraying the lowest and highest response category thresholds is presented in Figure 7. In this map the locations of the candidates are the same as in Figure 6. And the column labeled “Mean Item estimate” places the items in the same locations. The column labeled “1 st threshold for Items”, however, places the items at positions defined as “mean item estimate plus the 1st threshold estimate”, in this case -1.71. This set of locations shows the .5 probability of the candidates responding with a “2” on the items. The column labeled “4 th threshold for Items” places the items at positions defined as “mean item estimate plus the 4 th threshold estimate”, in this case 2.17. This set of locations show the .5 probability of the candidates responding with a “5” on the items.


This map clearly shows that most candidates, at Entry, are in the “agree” to “strongly agree” range on the familiar items addressing broad based issues of equity and diversity—the desired response. But they are in the “uncertain” to “disagree” range on the more focused political action items (recall that a “5” on the reversed items corresponds to a strongly disagree response). At Exit and One Year-Out, however, most candidates and graduates are in the “disagree” to “strongly disagree” range on the more complex and potentially controversial political action items—the undesired response. We use this map to chart not only progress over time but the strength of commitment associated with that progress.


Figure 7. Rating Scale Variable Map for the Learning to Teach for Social Justice—Beliefs scale INPUT: 224 Persons 12 Items MEASURED: 224 Persons 12 Items 5 Categories EACH '#' IN THE Person COLUMN IS 3 Persons MAP OF Persons AND Items MEASURE | BOTTOM P=50% | MEASURE ------Persons-+-Items -+-Items 5 . + + | | | | . | | | | | | 4 + + | | . | | | | | | .# | | 3 + + .# | | 2007 Exit Survey: 1.41 2006, 2007 Year| | #### | | 2006 Exit Survey: 1.35 Out Survey: 1.33, | | 2005 Exit Survey: 1.29 1.35 #### | | 2 ### + + ##### | | 2005, 2006, 2007 Entry Survey: .# | | .80, .79, .86 .##### | | .######## | | X ##### | | X 1 .######### + + .###### | | ######### | | .# | | XX .## | | X ### | | X 0 . + + X | X | X | X | X .# | | X | | | | -1 + XX + | X | | X | | X | XX | X | | X | -2 + X + | | | | | | | | | | -3 + XX + | | | | | | | | | | -4 + + ------Persons-+-Items -+-Items

| TOP P=50% -+-Items + | | | | | + | | | | | X + X | | | | XX | X + X | X | X | X | X | + | | | | XX | + | | | | | + | | | | | + | | | | | + | | | | | + -+-Items

MEASURE --- 5

4

3

2

1

0

-1

-2

-3

-4


Category cha racteristic curves Category characteristic curves (“CCCs”) display the probabilities of responding to each category for any person on any item. From a measurement and instrument development perspective, the ideal pattern in the CCCs occurs when each response category is the expected response for some level of person-by-item interaction. This pattern occurs when the threshold estimates follow a strictly increasing order. As seen in Figure 8, the spread in the CCCs and category threshold estimates (where the CCCs intersect) is excellent for the Entry survey. That is, when a candidate with a relatively low estimate of commitment to teaching for social justice answers an item with a relatively high estimate of commitment to teaching for social justice and the difference is, say -1.0, the expected response is “2”. Likewise, when a candidate with a relatively high estimate responds to a relatively lower level item and the difference in the estimates is, say 2.0, the expected response is “5”. The pattern for the Exit threshold estimates is noteworthy because it differs from the Entry estimate’s pattern. The Entry threshold estimates were (-1.7, -.5, .05, 2.2)—this is the desired pattern. The Exit thresholds were (-1.3, -.07, -.4, 1.7)—here there is a disordinal pattern in the estimates. The middle category of “uncertain” is not the expected response for any level of person-by-item interaction. This difference in the patterns is, however, a reasonable and expected result. It means that incoming students were relatively “uncertain” about some concepts and ideas when they entered the program. They may not have had much experience with these concepts and may not have thought much about these specific issues. Graduates, in contrast, have spent up to four years in a program (or five years if enrolled in the 5 th-year M.A. option) that is known for its commitment to social justice, and have spent a great deal of time addressing these ideas. These graduating teachers are no longer as “uncertain” about their beliefs towards learning to teach for social justice. This same pattern was seen for the 2006 Entry and 2007 Entry estimates (strictly increasing thresholds) versus the 2006 Exit and 2007 Exit estimates (disordinal where the “uncertain” category was not frequently chosen).


Exit Survey

2005

CATEGORY PROBABILITIES: MODES - Structure measures at intersections P ++-------+-------+-------+-------+-------+-------+-------++ R 1.0 + + O | | B | | A | 555| B .8 + 55 + I |11 55 | L | 11 55 | I | 11 55 | T .6 + 1 5 + Y | 1 444444444 55 | .5 + 11 44 4445 + O | 1222222 4 544 | F .4 + 2221 22 44 55 44 + | 22 1 **333*333 5 4 | R | 22 11 33 224 33 55 44 | E | 222 *3 442 33 55 444 | S .2 +2 33 11 4 22 3* 44 + P | 33 ** 22 555 333 444| O | 333 44 11 ** 333 | N | 333333 44444 1***55 22222 333333 | S .0 +***********555555555555 111111111**********************+ E ++-------+-------+-------+-------+-------+-------+-------++ -3 -2 -1 0 1 2 3 4 Person [MINUS] Item MEASURE

CATEGORY PROBABILITIES: MODES - Structure measures at intersections P ++--------- +--------- +---------+---------+---------+---------++ R 1.0 + + O | | B | | A |11 | B .8 + 11 + I | 111 555| L | 11 55 | I | 11 55 | T .6 + 11 55 + Y | 1 4444444 55 | .5 + 11 444 444455 + O | 1 44 5544 | F .4 + 222**222222 44 5 444 + | 222 1 22 4 55 44 | R | 22 11 ** 55 44 | E | 222 1 3*33**333 55 444 | S .2 + 2222 33**4 22 333*5 4+ P |22 333 44 11 2*55 333 | O | 3333 44 111 555 222 33333 | N | 3333333 44444 5555*1111 22222 3333333 | S .0 +***************55555555555 11111111******************+ E ++--------- +--------- +---------+---------+---------+---------++ -3 -2 -1 0 1 2 3 Person [MINUS] Item MEASURE

CATEGORY PROBABILITIES: MODES - Structure measures at intersections P ++------+------+------+------+------+------+------+------ ++ R 1.0 + + O | | B |11 | A | 11 55| B .8 + 11 55 + I | 11 55 | L | 11 5 | I | 1 55 | T .6 + 1 5 + Y | 1 44444444 5 | .5 + 11 44 4455 + O | *2222222 4 544 | F .4 + 22 1 2 3 4 5 4 + | 22 1 **3 **33 55 44 | R | 22 1 33 24 33 5 44 | E | 22 133 422 33 5 44 | S .2 + 22 3311 4 2 ** 44 + P | 222 33 1*4 22 55 33 44| O |22 333 44 11 ** 333 | N | 3333 4444 1***55 2222 33333 | S .0 +*****************5555555555 11111111*******************+ E ++------+------+------+------+------+------+------+------ ++ -4 -3 -2 -1 0 1 2 3 4 Person [MINUS] Item MEASURE

CATEGORY PROBABILITIES: MODES - Structure measures at intersections P ++-------+-------+-------+-------+-------+-------+-------++ R 1.0 + + O | | B | 55| A |1 555 | B .8 + 11 55 + I | 11 55 | L | 11 55 | I | 1 55 | T .6 + 11 5 + Y | 1 44444444 55 | .5 + 1 44 44 5 + O | 1 44 5*4 | F .4 + 22**22222 4 5 44 + | 22 1 2 4 55 44 | R | 222 1 2* 5 44 | E | 22 1 33*3**333 55 44 | S .2 + 22 33**4 2 33*5 444 + P |22 33 4 1 2255 333 444 | O | 3333 444 111 5522 333 44| N | 33333 4444 555**1 22222 333333 | S .0 +***********5555555555 11111111**********************+ E ++-------+-------+-------+-------+-------+-------+-------++ -3 -2 -1 0 1 2 3 4 Person [MINUS] Item MEASURE

2007

Entry Survey

2006

Figure 8. Category Characteristic Curves for the Learning to Teach for Social Justice—Beliefs scale.

CATEGORY PROBABILITIES: MODES - Structure measures at intersections P ++-------+-------+-------+-------+-------+-------+-------++ R 1.0 + + O | | B | | A | 555| B .8 +1 55 + I | 1 55 | L | 11 55 | I | 11 55 | T .6 + 1 5 + Y | 1 44444444 55 | .5 + 11 44 44 5 + O | 1 2222 44 *44 | F .4 + 222* 222 3 44 55 4 + | 22 1 **33 *333 5 44 | R | 22 11 33 2 4 33 55 44 | E | 222 *3 4*2 33 55 444 | S .2 +2 33 11 4 22 3*3 44 + P | 333 ** 22 555 33 444| O | 333 44 11 ** 333 | N | 33333 44444 1***55 22222 333333 | S .0 +***********555555555555 111111111**********************+ E ++-------+-------+-------+-------+-------+-------+-------++ -3 -2 -1 0 1 2 3 4 Person [MINUS] Item MEASURE

CATEGORY PROBABILITIES: MODES - Structure measures at intersections P ++-------+-------+-------+-------+-------+-------+-------++ R 1.0 + + O | | B | 55| A |11 555 | B .8 + 11 55 + I | 11 55 | L | 11 55 | I | 1 55 | T .6 + 1 5 + Y | 11 44444444 55 | .5 + 1 44 44 5 + O | 1 44 5*4 | F .4 + *22 4 5 44 + | 2222 1 222 4 55 44 | R | 22 1 ***333 5 44 | E | 222 1*34422 333 55 44 | S .2 + 22 333 * 2 3*5 444 + P |222 33 44 11 22 55 333 444 | O | 333 44 11 5*22 333 44| N | 33333 4444 555**1 2222 3333333 | S .0 +***********5555555555 1111111***********************+ E ++-------+-------+-------+-------+-------+-------+-------++ -3 -2 -1 0 1 2 3 4 Person [MINUS] Item MEASURE


Longitudinal Assessment of Master’s Students We ultimately wanted to track a cohort of students’ beliefs about teaching for social justice from the time of entry, through graduation and into the first years of teaching. As the first administration of the LTSJ-B scale occurred in 2005, it would be impossible to compare undergraduates at this point in time as the first cohort for comparison would be in 2009, when these undergraduates graduate from their teacher education programs. Accordingly, we examined one cohort of Master’s level students who entered the program and took the Entry survey in 2005, graduated and took the Exit Survey in 2006, and taught their first year and completed the One Year-Out Survey in 2007. While over 100 Master’s level candidates responded to these three surveys, this analysis focused specifically on the students who completed all three of these surveys (N = 50). Therefore, we avoided any bias due to attrition, or the result of losing respondents across the three surveys. In 2005, the mean person estimate for entering Master’s students was 1.22, considerably higher than the mean person estimate for all entering teacher candidates (0.80) who started the program at that time. In 2006, at graduation, the mean person estimate for Master’s students on the Exit survey was 1.38, slightly higher but similar to all 2006 graduating teacher candidates (mean person estimate of 1.35). Finally, in 2007, after one year of teaching, the mean person estimate for the same cohort of Master’s level students was 1.39, again notably higher than the mean person estimate for all respondents on the 2007 One Year Out survey (1.27). These findings are interesting for two reasons. First, the Master’s students exhibit the same pattern that was found across all respondents—weaker commitment to teaching for social justice upon entry into teacher education, and a stronger commitment upon graduation that is maintained after one year of teaching. Second, while the pattern is similar to that found for all respondents, there are important differences between the mean person estimates for the Master’s students and those of the entire cohort—the large difference between the estimates for the graduate students and the entire cohort at Entry and at One Year-Out and the similarity between these two groups at Exit indicates that undergraduate and graduate students differ significantly when they begin their program as well as once they are in their own classrooms, while they are similar to one another at graduation.


With regard to this second point, at the beginning of their teacher education program, Master’s level students demonstrated considerably stronger commitments towards teaching for social justice than the undergraduate students who started the program at the same time. This is not surprising considering that, on the 2005 Entry survey, Master’s level entering teacher candidates reported being significantly more influenced by the schools’ social justice mission than undergraduate students. Similarly, Master’s level students may have chosen the teacher education program because of the social justice mission more often than undergraduate students in the same program. This could be a result of maturity or because of the differing variety of factors undergraduate and Master’s level students may take into account when deciding on a teacher education program , as well as the degree to which they have previous teaching experience. At the time of graduation, both the Master’s level and undergraduate students demonstrated similar levels of commitment to teaching for social justice. This finding is understandable given that, at graduation, both Master’s level and undergraduate students have experienced the teacher education curriculum, bringing a level of homogeneity to their understanding of commitment to teaching for social justice. However, while undergraduate experience four years of the teacher education program Master’s level students complete the program in on average 1 to 1 ½ years. Thus, the increase for Master’s level students in mean person estimate from 2005 Entry to 2006 Exit may not be as dramatic as the increase for undergraduate students who spend four years in the program, particularly because they started at a point in which they were more familiar and more committed to social justice at the beginning of their program. After one year of teaching, the Master’s students again demonstrated a similar commitment to teaching for social justice as they did at graduation. Again, we interpret this finding positively; graduates, regardless of their specific teacher education program, remain committed to teaching for social justice even after they have completed their first year of teaching. However, the mean person estimate for these Master’s students was considerably higher than the average for all respondents. This finding again indicates that there is a significant difference between graduate and undergraduate candidates after one year.


Figure 9. Master’s students Mean Person Estimates on the Learning to Teach for Social Justice—Beliefs scale

2007 Year-Out Survey: 1.39 2006 Exit Survey: 1.38 2005 Entry Survey: 1.22

Conclusion In summary, we found that entering teacher candidates were more varied than graduating candidates, based on the differences in reliability estimates and scale variance for these two surveys. Plus, responses to the 12 items on the LTSJ-B scale indicated that entering candidates were more “uncertain” while Exit and Year Out respondents, on the other hand, struggle with strongly agreeing/disagreeing versus simply agree/disagree. The results also demonstrated that entering candidates were less committed to teaching for social justice than graduating and practicing teachers, who demonstrated they had “learned” to teach for social justice as a result of their teacher education program. Furthermore, increases in the mean person estimates across three administrations of the Exit Survey suggest that changes within the program, as a result of the TNE project, have affected graduating candidates’ commitments in a measurable way. Results from the One Year-Out Survey indicated that practicing teachers are as committed as


when they graduated and this pattern remained consistent across two administrations. Finally, by focusing on one cohort of Master’s students, we found that graduate students appear to start out more committed at Entry but end up in the same place as the rest of the respondents at Exit. This finding is understandable given the qualitative differences between undergraduates and graduate students as graduate students, by definition, have made a formal commitment to a higher level of education within a specific field, beyond that which is required of an undergraduate. These analyses allow us to look at more than simple pre-post mean comparisons, which are generally done when assessing change using a survey-based measure. By using the Rasch measurement model, we were able to anchor both item estimates and thresholds on a baseline measure (the 2005 Exit Survey). This allowed for a more direct comparison across multiple cohorts and years, specifically with regard to the items respondents were uncertain about and the issues which candidates and graduates were struggling with at different points in time. We identified items that were more difficult/easy to endorse at each point in time, and identified shifts in the probability of certain response patterns. Thus, we used sophisticated analyses to examine a complex construct, allowing for a targeted approach to understanding “teaching for social justice” which makes a significant contribution to the field of teacher education. While these findings are quite powerful, there are some important limitations that should not be overlooked. In general, it is important to keep in mind that while we consistently found similar patterns across surveys and people, these results, with the exception of the longitudinal assessment of the Master’s students, are based on different people. Additionally, while we argue that the 2005 Exit Survey is a reasonable baseline administration of the LTSJ-B it is important to note that the order of the items on the variable map, and therefore the item estimates and thresholds are somewhat different across all of the surveys. These differences do provide interesting comparisons across people and survey, but do not serve our current purpose for measuring change. As has been alluded to several times within this paper, m any more comparisons across surveys and cohorts are possible. Now that the “learning to teach for social justice-Beliefs” scale has been defined and been established as a stable and reliable measure, we aim to focus our


future research on a comparison between beliefs and practices using the “learning to teach for

social justice-Practices” (LTSJ-P) scale, which has yet to be introduced in our teams’ publications. Seeing as we assume that a teachers’ commitment towards teaching for social justice often would not determine whether or not their practices reflected their beliefs (preliminary evidence from the LTSJ-P scale indicates that teachers are less likely to engage in these activities, even though their beliefs may indicate that they are strongly committed to the principles of teaching for social justice), the results obtained from the LTSJ-P scale are a necessary next step in evaluating teaching for social justice. This goal would introduce a future definition of the baseline for the LTSJ-P scale, which probably will be the 2005 One Year-Out Survey, and additional complexities with anchoring. Specifically, simply anchoring on items and thresholds would not allow for comparisons across beliefs and practices, because these estimates would differ. Therefore, we would need to anchor across people. To conclude, this paper highlighted the strength of the “learning to teach for social justice-

Beliefs” scale, by continuing the research begun by the BC TNE-ET (Ludlow, et al, 2007, 2008). Our findings demonstrated that the scale was invariant across time of testing and experience with teaching and ensured the property of invariance, a necessary condition in subsequent efforts to measure change. Furthermore, the development of this scale demonstrates the challenge of measuring a complex construct in a reliable and valid way; however, rigorous piloting as well as consistent results over time across surveys and cohorts of candidates and graduates has led to a scale that we believe measures “learning to teach for social justice” accurately, and we are confident in our ability to measure changes in commitment to teaching for social justice across candidates and graduates as they progress through their teacher education and enter the field of teaching.


References Adams, M., Bell, L. A., & Griffin, P. (Eds.). (1997). Teaching for diversity and social justice. New York: Routledge. Andrich, D. (1988). Rasch Models for Measurement. Newbury Park: Sage. Brennan, R.L. (Winter 2001). Some problems, pitfalls, and paradoxes in educational measurement. Educational Measurement: Issues and Practice, 6-18. Cochran-Smith, M. (1999). Learning to teach for social justice. In G. Griffin (Ed.), The education of teachers: Ninety-eighth yearbook of the National Society for the Study of Education (pp. 114-144). Chicago: University of Chicago Press. Cochran-Smith, M. (2008). Toward a theory of teacher education for social justice. Paper presented at the American Educational Research Association Annual Meeting. New York, NY. Crocker, L. & Algina, J. (1986). Introduction to Classical & Modern Test Theory. NY: Holt, Rinehart & Winston. Cureton, E.E.; D'Agostino, R.B. (1983). Factor Analysis: An applied approach . Lawrence Erlbaum. Gulliksen, H. (1950). Theory of Mental Tests. Hillsdale, NJ: Lawrence Erlbaum. Hattie, J., Jaeger, R.M. & Bond, L. (1999). Persistent methodological questions in educational testing. Review of Research in Education, 24, Chapter 11. Washington, DC: AERA. Hollins, E., & Guzman, M. (2005). Research on preparing teachers for diverse populations. In M. Cochran-Smith & K. Zeichner (Eds.), Studying teaching: The report of the AERA panel on research and teacher education (pp. 477-548). Mahwah, N.J.: Lawrence Erlbaum. Ludlow, L.H., Pedulla, J., Enterline, S., Cochran-Smith, M., Kafka, A., Loftus, F., Mitescu, E. & Solomon-Fernandez, Y. (2007). From students to teachers: Teacher candidates’/graduates perceptions and experiences. Paper presented at the American Educational Research Association Annual Meeting. Chicago, IL. Ludlow, L.H., Enterline, S., Cochran-Smith, M. (2008). Learning to Teach for Social Justice—Beliefs Scale: An Application of Rasch Measurement Principles. Measurement and Evaluation in Counseling and Development, 40(4). Ludlow,L.H. & Haley,S.M. (1995). Rasch model logits: Interpretation, use, and transformation. Educational and Psychological Measurement, 55, 967-975.


Ludlow LH, Haley SM & Andres P. (2005). Measuring change: Functional activity item calibrations at admission vs. discharge. In Bezruczko, N. (ed.) Rasch Measurement in Health Sciences. Maple Grove. MN: JAM Press. Lord, F.M. & Novick, M.R. Statistical Theories of Mental Test Scores, Reading, MA: AddisonWesley, 1968. Masters, G. (1988). Item discrimination: When more is worse. Journal of Educational Measurement, 25, 15-29. Michelli, N., & Keiser, D. (Eds.). (2005). Teacher education for democracy and social justice. New York: Routledge. North, C. (2006). More than words? Delving into the substantive meaning(s) of “social justice” in education. Review of Educational Research. 76(4), 507-535. Oakes, J., & Lipton, M. (1999). Teaching to change the world . Boston: McGraw Hill Publishers. Spearman, C. (1904). "General intelligence" objectively determined and measured. American Journal of Psychology, 15, 201-293. Traub, R.E. (1997). Classical test theory in historical perspective. Educational Measurement: Issues and Practice , 8-14. Van der Linden, W. & Hambleton, R. (Eds.). (1997). Handbook of Modern Item Response Theory. NY: Springer. Villegas & Lucas (2002). Educating Culturally Responsive Teachers: A Coherent Approach. Albany, NY: SUNY Press. Wainer, H. (1986). Can a test be too reliable? Journal of Educational Measurement, 23, 171-173. Wright, B.D. & Linacre, M. (1998). WINSTEPS. Chicago: MESA Press. Wright, B.D. & Masters, G.N. (1983). Rating Scale Analysis. Chicago: MESA Press. Zeichner, K. (2003). The adequacies and inadequacies of three current strategies to recruit, prepare, and retain the best teachers for all students. Teachers College Record, 105(3), 490-515.