Women Catch Up: Gender Differences in Learning Programming Concepts Laurie Murphy
Renée McCauley
Pacific Lutheran University
[email protected]
College of Charleston
Suzanne Westbrook University of Arizona
Brad Richards
[email protected] Briana B. Morrison
[email protected] Timothy Fossum
University of Puget Sound
Southern Polytechnic State Univ
SUNY at Potsdam
[email protected]
[email protected]
[email protected]
specific concepts (e.g., selection, loop, procedure, arrays and pointers in [11]).
ABSTRACT This paper describes a multi-institutional study that used categorization exercises (known as constrained card sorts) to investigate gender differences in graduating computer science students’ learning and perceptions of programming concepts. Our results show that female subjects had significantly less pre-college programming experience than their male counterparts. However, for both males and females, we found no correlation between previous experience and success in the major, as measured by computer science grade point average at graduation. Data also indicated that, by the time students completed their introductory courses, females reported nearly equal levels of mastery as males of the programming concepts. Furthermore, females generally considered the programming concepts to be no more difficult than did the men.
Previous studies investigating gender differences in experience and confidence levels among students have focused primarily on students in introductory-level computer science or non-major courses at a single institution (e.g., [2, 5, 13, 15]). This study was unique in that it focused on students who have persisted and succeeded in the major, those graduating with baccalaureate degrees in CS. It was also multi-institutional, involving 73 students from eight colleges and universities across the United States. Furthermore, the primary methodology used to elicit data did not ask students to self-report or subjectively rate their general knowledge or experience, but rather required that they characterize their learning and perceptions of 26 specific programming concepts using a categorization technique adopted from knowledge acquisition research. Subjects were required to sort the concepts using prescribed criteria and category names. This technique is useful in measuring agreement between subjects [10]. The prescribed criteria focused on when subjects first learned specific programming concepts, when they mastered those concepts, and how difficult they had been to learn. This was done within the context of a study [8] of students’ general knowledge of programming concepts and was not specifically designed to investigate gender differences.
Categories and Subject Descriptors K.3.2 [Computers & Education]: Computer & Information Science Education – Computer Science Education.
General Terms Human Factors
Keywords card sort, gender differences, programming experience
Asking students to identify when they were introduced to or mastered specific concepts or how difficult they perceived the concepts to be may have several advantages over more subjective self-ratings. First, this technique is likely to result in a more precise assessment of experience. For example, a student who mastered concepts such as objects and recursion “before college” comes to an introductory college level computer science class with a more sophisticated background than one who has not, even if both students have taken a pre-college programming course. Comparing which and how many concepts are assigned to the “before college” category by men and women gives a more accurate view of how pre-college computing experiences may differ for these two groups. Secondly, such an approach may be less susceptible to gender bias caused by male students’ general tendency to rate their own abilities more highly than equally able female students, as well as the tendency for both men and women to rate men’s abilities higher than women’s [2]. The premise is that even a student with little confidence will find it difficult to substantially underestimate his or her knowledge of specific concepts, such as if-then-else or encapsulation. Furthermore, focusing on students’ perceptions of programming concepts,
1. INTRODUCTION It is a well-known phenomenon that women students come to introductory computer science (CS) classes with less pre-college programming experience than do men (e.g., [5, 7, 11, 13]). There is also considerable research suggesting that women students have less confidence in their computing abilities than their male peers [1, 7, 13]. Data on experience and confidence have typically been obtained through student self-reports, including survey questions that ask students to “measure pre-college experience on a scale from 1 to 7” [13] or respond to statements such as “I have studied computer science in school” using a Likert-type scale [5]. Very few studies have asked students to rate their familiarity with Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SIGCSE’06, March 1–5, 2006, Houston, Texas, USA. Copyright 2006 ACM 1-59593-259-3/06/0003…$5.00.
17
rather than their abilities compared to those of others, and asking the questions in a gender-neutral manner, may be less likely to bias their responses. A study by Spencer et al. [14] investigating stereotype threat showed that the mere suggestion that a math test has gender differences can cause women to perform worse and men to perform better on the test. This suggests the possibility that surveys including questions such as “In general, men are better than women at programming” [3] might influence the accuracy of men’s and women’s self-assessments of knowledge or ability. The gender-neutral protocol used in the study reported here is likely to result in more accurate results in terms of gender.
2.1 Constrained Sorts Subjects were asked to perform four sorts using specific categories and criteria provided by the interviewer. (See Table 2.) Table 2: Constrained sort criteria and categories When I was first introduced to it before college, lower-level CS courses, upper-level CS courses, on the job, on my own, don’t know the term When I mastered it before college, lower-level CS courses, upper-level CS courses, on the job, on my own, haven’t mastered it yet, don’t know the term Language paradigm procedural, functional, object-oriented, logic, not sure, don’t know the term Difficulty level easy, intermediate, advanced, don’t know the term
We investigated gender differences by examining graduating students’ experience and knowledge of programming concepts. This research project is an adaptation of an earlier multi-national study [12] that employed the same concept categorization exercise to investigate the knowledge structures of 243 novice programmers and 33 educators at 22 universities. The novices’ understanding of programming concepts was elicited through a knowledge-acquisition technique called a repeated singlecriterion card sort [10]. While the novice study showed little difference between sorts produced by male and female students or between students with very little or considerable programming experience, it was useful in focusing the study of graduating students reported herein. Specifically, the criteria and categories chosen for the constrained sorts, discussed in this paper, were selected based on data observed during analysis of the data from the study of novices.
For each criterion, subjects were asked to group the cards into categories for the given criterion. For example, for the “When I was first introduced to it” criterion, one subject provided the following groupings of cards into categories: Before college: Lower-level CS courses: dependency, object, abstraction, scope, list, recursion, state, encapsulation Upper-level CS courses: tree, thread, event On the job: On my own: function, method, procedure, if-then-else, boolean, choice, parameter, variable, constant, type, loop, expression, iteration, array Don’t know the term: decomposition
2. STUDY METHODOLOGY Interviews were conducted with graduating CS students. The interviews consisted of two types of categorization tasks requiring subjects to sort programming concepts into categories based on a single criterion. In the first type, subjects were asked to articulate their own criteria and category names. There is evidence to suggest that the way subjects categorize concepts reflects their internal representation of those concepts [6]. We refer to these sorts as unconstrained sorts.1 The second type, discussed herein, asked subjects to perform constrained sorts in which the criteria and category names were provided.
This paper focuses on the analysis by gender of data from three of the four constrained sorts: “When I was first introduced to it”, “When I mastered it”, and “Difficulty level”.
2.2 Subjects The subjects were 73 undergraduate CS students at eight colleges and universities in the USA, including both private and public institutions, ranging from small liberal arts colleges to large research universities. All subjects were eligible to complete baccalaureate degrees in CS at some time during the calendar year in which they were interviewed. At each school, the study was advertised and students volunteered to participate. An effort was made to recruit female students and students with a range of academic abilities. The recruitment techniques, research protocol and interview procedures were approved by the Institutional Review Boards at all institutions.
During all card sorts subjects categorized the same set of 26 index cards, each containing a prompt for a programming concept. (See Table 1) These concepts ranged from specific programming entities, such as if-then-else and variable, to more abstract concepts, such as decomposition and encapsulation. However, all were general in nature and not limited to a particular language syntax or programming task. Table 1: Stimuli used in card sorts function method procedure dependency object decomposition abstraction
1
scope list recursion choice state encapsulation parameter
type loop expression tree thread iteration
constant boolean event variable if-then-else array
2.3 Data Collection Data were collected during Spring 2004 and Spring 2005. Data collection followed the standard protocol established for the earlier study of novice programmers [12]. All investigators had participated in the novice study and were familiar with the sorting and data collection procedures. The following data were collected for each of the 73 subjects:
See [8] and [9] for analysis of the unconstrained sorts
18
Figure 1: Mean concepts per category for introduction and mastery sorts Demographic and background data included expected graduation date, age, gender, first spoken language, first and second programming languages, age when they began programming, level of experience with a variety of programming languages, overall grade point average (GPA2), CS GPA, and grades in CS courses.
shows females designated slightly more concepts as “easy” (females 54%, males 49%) and slightly fewer as “intermediate” (females 25%, males 29%) than did the males. On average, male subjects considered slightly more (18%) of the terms to be “advanced” than female subjects did (15%). Both groups reported knowing all but a very few of the terms.
Constrained card-sort data included a record of the cards sorted into each given category for each of the four constrained sort criteria. (See Table 2.)
Table 3: Mean concepts per difficulty category
3. RESULTS AND DISCUSSION 3.1 Demographic variables The subjects were 22 (30%) females and 51 (70%) males. While the mean age of male and female subjects was nearly the same (24.5 years for females and 24.0 years for males), on average the females began programming at a later age (18.4 years of age) than the males (16.5 years of age). This disparity in ages at which males and females begin to program is similar to those reported in other studies [1]. For this group, the mean ages imply that on average these females began programming in college, while males began while still in high school.
Intermediate
Advanced
Don’t Know
Females
14.0
6.5
4.0
1.5
Males
12.8
7.5
4.8
0.9
To quantify difficulty ratings for individual concepts each category was assigned a numeric value (1=easy, 2=intermediate, 3=advanced, 4=don’t know) and the mean difficulty rating of each concept was calculated for male and female subjects. Little difference was observed in male and female subjects’ difficulty ratings of the individual concepts, with both rating the concepts decomposition, abstraction, dependency and thread as most difficult and constant, boolean and variable as easiest. The mean difficulty ratings for all 26 concepts by individual subjects was nearly identical for males (N=51, M=1.76, SD=.249) and females (N=22, M=1.73, SD=.341).
Academic performance was nearly the same for both male (GPA = 3.28, CS GPA = 3.29) and female (GPA = 3.34, CS GPA = 3.32) subjects. It should be noted here, that an effort was made to recruit additional female subjects with high CS GPAs in Spring 2005 since they were not well represented in Spring 2004.
3.3 Introduction and Mastery Sorts The categories for both the “When I was first introduced to it” and “When I mastered it” sorts were “before college,” “introductory CS classes,” “upper-level CS classes,” “on the job,” “on my own,” and “don’t know.” The category “haven’t mastered yet” was also used in the mastery sort. Figure 1 reveals the substantial differences between males and females, with males being both introduced to and mastering many more programming concepts than females before college. Results of chi-square tests on the number of concepts assigned to the “before college” category, versus those assigned to one of the categories representing college or after, indicated these differences were significant (p
