A Case Study from Computer Science - CiteSeerX

Session 13a4 Progressing from Small Group Work to Cooperative Learning: A Case Study from Computer Science Deborah A. Trytten School of Computer Science University of Oklahoma Norman, Oklahoma, 73019 Abstract - Although the industrial partners of academe are unanimous in their desire to hire engineering graduates who are experienced in working productively in small groups, implementing small group work in a computer science class can be difficult. The obvious assignment, a group programming project, proved to be a poor choice when implemented in my computer graphics class. An examination of the literature in this area shows that a group programming project has many features in common with a group term paper, the assignment which has been uniquely identified as the worst choice for small group work. Fortunately, there are better choices for cooperative learning in computer science. Assignments with “the three S’s”: Same problem, Specific choice, and Simultaneous reporting of group choices, work well. This was implemented in my class by having students work multiple choice quizzes designed to require high level learning skills. Quizzes were first worked by individuals, then by small groups. The small group answers are then compared and discussed in class. This generates the type of interaction between the professor and students which creates positive cooperative learning experiences. Promising preliminary results have been seen with this method, from both the student and the professor’s perspective.

Introduction The industrial demand for software engineers who are well prepared to work in groups led me to integrate group programming projects in a computer graphics course. Even though the course was carefully designed and planned, the experiment was not initially successful. This wasn’t unexpected given the observations that in the early phases of team learning, students initially go through the classic grief process [12] to mourn being forced to bear more responsibility for their own learning. The degree to which it occurred, however, was unexpected and alarming. After consultation with experts in cooperative learning, it was discovered that the group programming project assignment was, in fact, a poor choice. This result is surprising since group programming projects are such an obvious choice given the industrial context of computer science. The small group assignments were restructured to create a cooperative learning environment which has been much more suc-

cessful. This structure of assignment is applicable to many computer science classes, as well as classes in other engineering disciplines.

Small Group Work Since computer scientists in industry typically work in teams, industrial representatives have been encouraging faculty in computer science to integrate more small group projects into classes [3], [4]. This section will describe methods for managing small group learning into the classroom. Assigning Small Groups There are many elements which must be considered in the selection of small groups. The ideal group size is between four and seven members, inclusive [2]. Groups of five or more students may be desirable in classes where a significant number of students typically withdraw during the term, since they will not leave behind a group disadvantaged by being of insufficient size. Groups of size six or seven have higher transaction costs (the amount of time and energy which must be spent arranging and attending group meetings), if a significant portion of the work is to be done outside of the allotted class time. Group membership should not be altered during the term to allow group members a chance to learn to work together better over time. Heterogeneous small groups have been consistently found to be more effective than homogeneous small groups. A study of five methods for assigning small groups found that groups function best when they are heterogeneous with respect to GPA and homogeneous with respect to interest (as is common in engineering classes). Groups which were self selected by students were found to have the poorest attitudes about their instructor, the projects, their classmates, and the course [5]. Heterogeneous groups were also found to be superior when GPA, outside activities, practical experience, and learning style differences were considered [6]. There is one dimension along which heterogeneity may not be desirable. When members of under-represented groups are in the class, it has been shown that they should be clustered together in small groups [11], [9]. To avoid social loafing (the tendency of people to work less when grouped), it is essential that individual contributions are evaluated and graded in addition to group work. A

07803-5643-8/99/$10.00 c 1999IEEE November 10 - 13, 1999 San Juan, Puerto Rico 29th ASEE/IEEE Frontiers in Education Conference 13a4-22

Session 13a4 standard method is to perform peer evaluation by having group members assign 100 points among the other members of the team. This forces team members who wish to unfairly reward team members with lesser contributions to punish the most significant team contributors. For peer evaluation to be meaningful, it must be a significant factor in the final grade.

The Original Design In Spring of 1997 I began working with small groups in my CS 4053/5053 Computer Graphics course. This course is offered once a year and has both undergraduate and graduate students enrolled. The prerequisites for the course are data structures and linear algebra. The class size was in excess of seventy students. Groups consisted of four to five students. Groups were created to maximize diversity, except that the four women in the class were all placed in a single group. There were no other members of under-represented groups in the class. The dimensions which were used to determine diversity were whether the student was a graduate student or undergraduate student, GPA, full time or part time student, native or non-native speaker of English, and whether the students had passed a software engineering class. Small groups were created in such a way that every group except one had at least four hours during the week when all members reported that they were available, although this made assigning small groups feel as if it were NP-complete. Groups stayed together for the duration of the term. After an ungraded in-class team building exercise, small groups were asked to perform large programming projects collaboratively. Each group worked on essentially the same project. The only distinction between group projects was that groups could choose to compete for 10 of the 100 points for original additions to the projects, called enhancements. Enhancements were graded on the basis of originality. Groups which chose not to do enhancements, would receive at most 90/100 points. These programming projects made up half of the grade in the class. For each programming project, students performed a peer evaluation of their small group members by distributing 100 points among the other members of the group. The peer evaluation scores were used as a multiplier on the project score. An example of the calculation is shown below. Suppose that there are four members of the group, and their peer evaluation is as given below. If the group grade for the project was a 90, the individual grades would be as given in the last row of the table below. Member D receives the full 100 points, even though the group received only 90, because

of strong peer evaluations, while. Member C receives only 81 points. Evaluation A B C D P.E. Total Project Grade

A 31 30 32 93 85

B 34 35 36 105 95

C 30 28 32 90 81

D 36 41 35 112 100

The peer evaluations for the first few projects were done by having students email their peer evaluation numbers to the instructor. It soon became apparent that the raw numbers did not provide sufficient feedback to students, and a more comprehensive peer evaluation form which required students to defend their numerical ratings was created and used. Results The results were not promising. Students complained continuously and bitterly about having to work together. Many groups reported difficulties with social loafing and excessive transaction costs. Students continually begged to be released from small group work in favor of individual work. There were several small groups with interpersonal problems significant enough to require intervention by the instructor. One group became so dysfunctional that these interventions were held weekly. The projects which were finished were clearly being accomplished by the strongest individuals in the groups working in isolation, depriving other group members of the necessary understanding of class material. Several groups failed to submit a working version of several projects. The peer evaluations generated considerable controversy, and students asked the instructor to edit them for a variety of invalid reasons. Some days it seemed that every one of the unwelcome group members described by Lerner (from Nola No-Can-Meet to Do-It-All Dottie) were in each and every small group in my class [7]. During the course of the semester, a number of attempts were made to fix the problems after consultation with an expert in small group learning. Group project demonstrations were scheduled simultaneously and groups were required to evaluate each other’s base project and enhancements, in the hope of building healthy competition between groups and establishing group identity. The same multiple choice quiz was given both to individuals and groups (in sequence) before the programming projects were begun to force students to develop and demonstrate basic knowledge of the topic. These quizzes became 20% of the project grade. Although these measures did pacify some students, perhaps because they felt that the instructor was actively working to


Session 13a4 address problems, the vast majority of small group problems were unaffected. The one bright spot in the semester was that the group which consisted of all women consistently outperformed all other groups, reinforcing the controversial idea of clustering members of under-represented groups. A recent discussion with one of the members of this group confirmed that being in a group where women were in the majority was one of the most significant and unique experiences in her education. After the semester was over, several students reported that their group used clandestine means to conceal the lack of group interaction. These groups “diversified” code written by a single individual so that it would appear to be a group effort by having other group members reformat the code to appear to be in a different programming style. Several students reported that this tactic is regularly used by small groups when performing programming projects in other classes. Another student reported that his group had elected to have each group member create one project, even though the projects were thought to be too large for this to be physically possible. The student evaluations of the instructor were significantly lower than previous years, although the in-class content of the course had not changed. In short, the group programming projects were a disaster. The benefits of small group learning as reported by Felder [8] were not being seen in my class.

Cooperative Learning When I had assigned group programming projects, I had hoped to gain the well publicized benefits of cooperative learning. Although cooperative learning requires that students work in small groups, requiring students to work together in a small group does not necessarily generate a cooperative learning experience. Cooperative learning occurs when a group of people have [1]: • Joint Goals • Mutual Rewards • Shared Resources • Complementary Roles Small group projects do not necessarily have these attributes. As an example, two students may have very different goals for a group project. One student may have a heavy class load and want to spend as little time as possible on the projects, even if his grade suffers. Another student may view the project as a way to defend against weak test grades and not be satisfied with any grade less than 100, even if that means spending an unreasonably large amount of time on projects. A small group which contains these two individuals may have difficulty with social loafing since the first student may be happy to allow the second student to do a disproportionate amount of the work, and the second student may oblige. Peer evaluation can lower the grade of the first stu-

dent, but this student may not be dissatisfied with a lower grade. The presence of enhancement points in the original project assignment exacerbated this problem. Programming as a Group Writing Project As a group assignment, creating a computer program is similar to writing a term paper in many aspects. Having a group write a term paper together is the assignment which has been identified as the least effective cooperative learning assignment [10]. To understand why this is so, examine four models for group writing given by Schultz and Ludlow [13]. In the Sequential Model, the first author writes a complete document, subsequent group members edit the document in sequence. Notice that the first author is, in all likelihood, allowing subsequent group members to socially loaf. If this model is applied to a programming project, the first author is doing almost all of the work. Subsequent edits will be unlikely to be made since few students will meddle with a working programming project. As a result, no cooperative learning occurs. In the Stratification Model, each student takes on one role. The initial assignment goes to the project manager. The project manager asks the data gatherer to collect and summarize data. This data goes to the writer or editor who actually writes the document. Another team member incorporates graphics resulting in a finished product. This method again results in little resource sharing, e.g. the data gatherer may learn little about how the graphics person selected and created the figures and the graphics person may not learn where the data gatherer found the information. It also has a unequal division of labor, particularly if applied to a programming project, which will be likely to result in social loafing. The Horizontal Division Model takes a large project and breaks it into separate pieces of approximately equal size. These pieces are then accomplished separately by individuals in the group and combined. This model is similar to a common industrial software engineering process. As with the other models, there is little resource sharing going on. The difficulty of combining programming segments into a cohesive whole is also substantial, particularly when individuals have little experience with team programming, resulting in substantial transaction costs. If the size of the separate pieces are not comparable, social loafing may also result. The Group Composition Model, which is used by groups of two, has both members sitting at the computer writing the paper. This could result in true collaborative learning, except that the sizes of small groups are too large to allow it to be used. Groups of two are known to be too small for cooperative learning to occur. The result of this analysis is that group writing assignments, whether they be term papers or programming projects do not result in cooperative learning. This does not mean that these assignments should not be given. In fact, the industrial


Session 13a4 need for engineers who have experience with group writing and group programming may justify the inclusion of this assignment whether it results in cooperative learning or not. An instructor who uses this assignment should not be surprised, however, when significant problems with social loafing, transaction costs occur, and the well publicized benefits of cooperative learning disappear.

Selecting Assignments: The Three S’s A good structure for cooperative learning assignments has been developed by Michaelsen [14]. He recommends that all cooperative learning assignments be characterized by “The Three S’s” • Same problem • Specific choice • Simultaneous report An example of an exercise which has all of these attributes is a single multiple choice question given to all groups. When the time allotted is finished, the groups should report their results by either shouting out their choice to the class or holding up their choice written on a piece of paper. Class discussion of the question can then focus on the choices made by different groups. The competition between groups to give the most common answer, particularly if there are several answers of differing degrees of correctness, is intense and results in a lively intragroup discussion. When the group answers are compared in class discussion, everyone in the class is well prepared to vigorously defend their response and the class discussion is intense. A simple, factual multiple choice question will not generate this type of discussion. To get a good discussion, one needs to focus on high level learning objectives in the Bloom taxonomy [15] such as analysis, synthesis, and evaluation. Low level learning objectives such as knowledge, comprehension, and application will not create multiple choice problems which are challenging enough to generate the interchange of ideas and sharing of resources which characterizes successful cooperative learning assignments. My original group programming project with enhancements had none of the three S’s. The groups were working on slightly different problems because of the enhancements. Although they were making hundreds of choices in implementation, most of these choices were made by individuals instead of being the product of thoughtful collaboration. There was no simultaneous report since there was no single choice which was a product of the collaboration. When the enhancements were removed from the project, the assignment had only one of the three S’s. The structure of the assignment led to the failure of group programming assignments in my class. Feichtner and Davis [2] offer a summary of ways in which faculty members can cause small group learning to fail

in their classroom, particularly when used in combination. Their list includes the following items: • Structure the group assignments so that students can work independently and get the job done • Assign four or more cases or written reports • Give no group examinations • Use the absolute minimum of class time for group work Each of these items is directly related to the structure of the cooperative learning assignment. A group which has to make a specific choice cannot have individuals working independently unless it simply divides up the problems amongst its members. Since this process is occurring during class time (as it must be for simultaneous report to occur), an instructor can easily spot this aberration by the absence of communication and suggest interaction. No group has ever chosen to work this way in my class. Written reports are missing two of the three S’s, just as group programming projects were. Since group examinations are one of the easiest and most effective ways to create learning activities which have the three S’s, failing to give group examinations may mean that suboptimal assignments are being chosen. As mentioned above, the three S’s require a significant amount of class time to be used for group work. As a result, each of these items identified as a potential problem with small group work could be avoided if the three S’s had been followed.

A Better Assignment In the Spring of 1999, I had another opportunity to teach CS 4053/5053. Instead of using group programming projects to meet the industrial demand for students who had experience working with teams, I elected to create in-class assignments which would generate cooperative learning. Groups were created exactly as they had been previously with only one alteration. Each student was asked to describe themselves as either an introvert or an extrovert, using the Myers-Briggs definition. This addition was made after it was observed that several dysfunctional groups consisted only of introverts, with the most problematic group of people being so shy that they would not turn their chairs to face each other during discussions. The vast majority of my students labeled themselves introverts. The few extroverts which self reported were distributed throughout the groups to maximize diversity. Good fortune provided one extrovert for each group this semester. Had there been a shortage, as might be expected, I would have left the small group with the strongest individual student(s) without an extrovert. The structure of the assignments was built to have all three S’s. For each major topic, a multiple choice quiz was created which required that students meet high level learning objectives to answer correctly. These quizzes typically had six to eight problems. Students first took the quiz as individuals. Then the students took the same quiz as a group. The


Session 13a4 individual and group quizzes made up 40% of the grade each. The remaining 20% of the grade came from peer evaluation. After the quizzes are finished, the class discusses the correct answers collectively, when time permits (timing the multiple choice quizzes has been a problem since they always take substantially more time than it seems they should). This exercise takes an entire class period. This assignment has all three S’s and has been very effective in generating intense intragroup discussion, as well as productive class discussions. This structure of small group work was pioneered by Larry Michaelsen at the University of Oklahoma College of Business. Individuals working alone have been averaged 4/8 questions correct on the quizzes. Small groups averaged 7/8 questions correct. This assignment avoids the two best known problems with small group work: social loafing and transaction costs. Social loafing is severely punished in this format since individuals lose points both on their quiz and on their peer evaluation. These two elements together make up 60% of the grade on group work. This is a powerful incentive to be a productive individual and group member. Since I prowl the room listening in on group discussions during the group quiz taking process, I can also identify social loafers. With so many students actively engaged, spotting disengaged students is easy. When confronted with a professorial comment about apparent disengagement, many students will confess that they were socially loafing. I did not find the same individual disengaged twice. Transaction costs are also avoided since all group work is occurring during class time. There is no additional student time spent arranging and attending meetings. Groups which contain members whose busy schedules preclude meeting outside of class are neither disadvantaged nor resentful. During the course of the term, I’ve made a few small corrections to the initial plan. The early quizzes were done with closed books and notes. After the second quiz, students requested that the quizzes be open book and notes. As the quiz questions require students to meet high level learning objectives, and are therefore cannot be answered easily by reference to the book, this request was granted. This improved the quality of intragroup discussion significantly since students could read and interpret the text to reinforce and correct their arguments. When small groups were first assigned, the same team building exercise was performed as in Spring 1997. Five quizzes were given. The first quiz grade was not recorded or counted because a secretarial error caused some students to have missing pages in their quizzes. Results My subjective evaluation is that students enjoyed this form of small group work, and learned more about computer graphics using the improved assignments. A direct comparison of final

examination scores was not done since other factors in the class had changed significantly (e.g. the textbook, the class size, the teaching assistant structure). There were no student complaints, and no dysfunctional groups. I did not have to schedule a single group counseling session. The peer evaluations were strikingly different than with the previous format. Almost every group divided points nearly evenly. There were no student complaints about peer evaluation, and no student requested that their evaluation be changed. When students were in my office asking questions about the programming projects, which were done individually instead of collectively, they were clearly more knowledgeable about the topics and the questions were more focused and of higher quality. I received a substantial quantity of informal verbal and email feedback from students. Several students commented that they feel like they are “a real part of the class” instead of being a passive observer. One student emailed that he had never “fit in” to computer science, and that working with this group has allowed him to see that he could. When asked for informal feedback on the cooperative learning exercises, students consistently commented that they couldn’t pass the class without the group quizzes. There was palpable excitement on the days quizzes are given. Students were on time, and immediately engaged. After class was over, members of groups were often seen continuing to discuss class material in the hallway, sometimes for extended periods. From the student perspective, this experiment was a success. The only significant difficulty reported by students was that reading the text independently, as is necessary to pass the quizzes, is difficult. This may be caused by the shortage of computer science texts that are both technically correct and pedagogically well designed. Since the rapid pace of change in computer science often precludes the creation of these definitive texts, this difficulty will probably continue. In other engineering disciplines, where these definitive texts are more frequently available, this form of cooperative learning would be expected to work even better. From the professor’s perspective, this assignment has been successful. Class discussion after the quizzes was directed towards areas where students are having difficulty and was therefore more productive. The quality and quantity of class discussion also changed. There was more student interest and focus, and students are asking more sophisticated questions and bringing up subtle and interesting points. Although I initially feared that the breadth of coverage might have to be sacrificed for depth of coverage, this was not the case. My explanation for this is that students are spending more time outside of class studying the book and notes. This decreases the amount of class time which must be spent introducing elementary material, which is undoubtedly the least productive use of classroom time. On the negative side, substantial faculty time must be spent creating quizzes. Although this will probably improve


Session 13a4 as I become more experienced in asking questions which represent high level learning outcomes in multiple choice format, I expect that it will always take more time than writing a simple lecture. Of course an insignificant amount of time is spent on grading these quizzes, which is partial compensation. It is my opinion that the gains in student learning are significant enough to justify this investment. An interesting anomaly occurred during the first two weeks of the semester. Enrollment for this class has typically been large, initially in excess of eighty students this semester. When the collaborative learning exercises were explained to students in class, more than twenty students elected to immediately drop the class. This left me with a class size of only fifty two students. Most of the students who dropped were international graduate students. When asked why they dropped the class, the most common response was that it “seemed like it was going to be too much work”. This is similar to students “voting with their feet” and taking a biology class in the standard lecture format instead of the cooperative learning format as reported by Miller and Groccia [16].

Conclusions There is little doubt that the industrial demand for engineers and computer scientists who are experienced in small group work will continue to grow. This places demands on academe to find good ways to give students experience working in small groups while retaining or improving the intellectual content of courses. By improving the structure of a cooperative learning assignment in my computer graphics course, I have given students the chance to develop skill in working with small groups without sacrificing coverage of the material or creating unpleasant and unproductive learning environments for students.

Acknowledgments My interest in small group learning was generated, fostered, and supported by the Instructional Development Program at the University of Oklahoma, headed by L. Dee Fink. The dual multiple choice quiz format model is from Larry Michaelsen of the University of Oklahoma Michael J. Price College of Business.

References [1] Zin, Q., Johnson, D. and Johnson, R., “Cooperative versus competitive efforts and problem solving”, Review of Educational Research, Vol. 65, No. 2, pp129-143. [2] Feichtner, S. B. and Davis, E. A., “Why some groups fail: A survey of students’ experiences with learning groups”, The Organizational Behavior Teaching Review, Vol. IX, Issue 4, pp58-73.

[3] Black, K. M., “An Industry View of Engineering Education”, Journal of Engineering Education, Vol. 83, No. 1, pp 26-28. [4] Lang, J.D., Cruse S., McVey, F. D., and McMasters, J, “Industry Expectations of New Engineers: A Survey to Assist Curriculum Designers”, Journal of Engineering Education, Vol. 88, No. 1, pp 43-52. [5] Brickell, J. L., Porter, D. B, Reynolds, M. F., and Cosgrove, R. D., “Assigning Students to Groups for Engineering Design Projects: A Comparison of Five Methods”, Journal of Engineering Education, Vol. 83, No. 3, pp 259-262 [6] Hunkeler, D. and Sharp, J.E., “Assigning Functional Groups: The Influence of Group Size, Academic Record, Practical Experience, and Learning Style”, Journal of Engineering Education, Vol. 86, No. 4, pp 321-332. [7] Lerner, L.D., “Making Student Groups Work”, Journal of Management Education, Vol. 19, No. 1, pp 123-125. [8] Felder, R. M., Felder, G. N. and Dietz, E. J., “A Longitudinal Study of Engineering Student Performance and Retention V. Comparison with Traditionally-Taught Students”, Journal of Engineering Education, Vol 87, No. 4, pp 469480. [9] Felder, R. M., Felder, G. N., Mauney, M., Hamrin, C. E., and Dietz E. J., “A Longitudinal Study of Engineering Student Performance and Retention III: Gender Differences in Student Performance and Attitudes”, Journal of Engineering Education, Vol. 84, No. 2, pp 151-164. [10] Michaelsen, L. K., Fink, L. D. and Knight, A., “Designing Effective Group Activities: Lessons for Classroom Teaching and Faculty Development”, published in To Improve the Academy, Vol. 16, pp 373-398, New Forums Press and the Professional and Organizational Development Network in Higher Education. [11] Tonso, K. L., “The Impact of Cultural Norms on Women”, Journal of Engineering Education, Vol. 85, No. 3, pp 217-226. [12] Felder, R. M., “Random Thoughts... We Never Said it Would be Easy”, Chemical Engineering Education, Vol. 29, No. 1, pp 32-33. [13] Schulz, K.H., and Ludlow, D. K., “Incorporating Group Writing Instruction in Engineering Courses”, Journal of Engineering Education, Vol. 85, No. 3, pp 227-232. [14] Michaelsen, L., “Three Keys to Using Learning Groups Effectively”, published in Toward the Best in the Academy, a periodical published by The Professional and Organizational Development Network in Higher Education, Vol 9, No. 5. [15] Bloom, B. S., Taxonomy of Educational Objectives. Handbook i: Cognitive Domain, David McKay Co., 1956. [16] Miller, J. E., and Groccia, J.E., “Are four heads better than one? A comparison of cooperative and traditional teaching formats in an introductory biology course”, Innovative Higher Education, Vol. 21, No. 4, pp 253-273.
