our three-course computer science introductory sequence in order to introduce ... 1 provides a high-level outline of the external documentation requirements.
Teaching Software Engineering Early - Experiences and Results Renée McCauley and Ursula Jackson Computer Science Department University of Southwestern Louisiana Lafayette, LA 70504-1771 {mccauley, jackson}@usl.edu Abstract In the fall of 1994 we reorganized the content of our three-course computer science introductory sequence in order to introduce software engineering concepts early and provide a consistent software engineering focus from one course to the next. We also established documentation and design standards that would serve as a framework for teaching the software engineering principles and techniques that we considered appropriate and essential to novice software developers. In an attempt to assess the impact of this new teaching process, we compared the performances in upper-level project-oriented courses of students who had been exposed to the new introductory sequence to those of students who had not. This paper describes the documentation and design standards established in 1994, their evolution over the past four years, and how these standards can be used as a framework for teaching software engineering concepts early in the curriculum. It also reports on what we have learned through tracking our students. We found that, in three upper-level courses, project grades for students exposed to software engineering concepts early averaged as much as half a letter grade higher than those of other students.
Introduction In the fall of 1994 we reorganized the content of our threecourse computer science introductory sequence in order to introduce software engineering concepts early and provide a consistent software engineering focus from one course to the next. The immediate objectives were to teach students software engineering techniques and provide them with practical software engineering experiences as they learned to write computer programs. The long-term objective was to adequately prepare students for the project-oriented courses that they would be required to take as upper-classmen. These courses are intended to further prepare them for the software production environment that they will likely enter upon graduation. In an effort to achieve these objectives, we established documentation and design standards that have served as a framework for teaching software engineering principles and techniques that are considered appropriate and essential to the novice software developer. This framework allows us to expose students to practical software engineering experiences such as testing techniques, teamwork, and working with large software systems, designs and code written by others.
Since its implementation in the fall of 1994, we have found this framework extremely beneficial for incorporating software engineering concepts into the introductory course materials. In an attempt to assess the impact of this new teaching process, we tracked our students and observed their performance on projects in upper-level project-oriented courses in comparison to students who did not come through the new introductory sequence. Results indicate that students with the software engineering background are indeed better prepared for these courses.
Motivation Traditionally, software engineering principles have been taught in dedicated upper-level courses or as an optional area of focus in curricula [1]. However, based on the student product turned out over the years, this approach has been perceived as being inadequate in several respects [2, 3] and industry has called on educators to provide students with more practical software engineering experiences [4]. In our degree program, we offer a traditional senior-level software engineering course as well as several other projectoriented courses at the upper level. However, we were not convinced that our students were entering these courses with adequate preparation in terms of their ability to work alone or with others in managing large, practical software systems. Also, prior to this time, we did not have any real consistency or quality control among the courses in our introductory sequence, i.e. faculty members taught the required materials in the way they saw fit. Thus, teaching approaches and coverage of software engineering concepts differed from course to course and from semester to semester.
Beginnings We began our overhaul of the introductory computer science courses by establishing a set of detailed documentation standards to be used in the project-oriented courses in our curriculum [5]. Our objective in taking this approach was to provide students with a set of clear, consistent, and systematic guidelines to assist them in developing software. Also, we wanted to provide a mechanism that would strongly encourage students to spend time and effort designing and documenting software solutions before they began coding those solutions. Modeling our standards after those presented by Gersting [6], we specified the content and
I. Requirements Documentation 1. Problem Description 2. Input Information 3. Output Information 4. User Interface Information II. Design Documentation 1. System Architecture Description 2. Object Information 3. Diagrams illustrating relationships among objects III. Implementation Documentation 1. Program Code IV. Verification and Validation Documentation 1. Test data 2. Test results 3. Operating directions Figure 1. Outline of the External Documentation
format of the external documentation to be submitted with each software solution. Our requirements for external documentation, like Gersting’s, are subdivided into four parts: Requirements Documentation, Design Documentation, Implementation Documentation, and Verification and Validation Documentation, which correspond to the software development activities of analysis, design, implementation, and verification, respectively. Figure 1 provides a high-level outline of the external documentation requirements. In addition to specifying details of the external documentation, the standards include information on the physical organization of system components, the internal documentation of developed systems, and style considerations to be used in writing code. Complete information on what is specified in the documentation standards is available elsewhere [5]. The original standards that we established in the fall of 1994 specified design information related to procedureoriented development techniques. In the fall of 1995, we made a transition from teaching procedure-oriented to objectoriented design and development in the introductory sequence. Therefore, in the summer of 1995, we rewrote the necessary portions of the standards in order to produce a version suitable for documenting object-oriented solutions. Over the course of the 1995-96 academic year, we further refined those standards based on information gathered as we used them. The only difference in the outlines for the procedure- and object-oriented versions of the standards for external documentation is in the information required in the design section. Additional details and copies of both versions of the standards are available electronically [7].
Using the Framework
In order for these standards to have the desired effect, we require that students submit solutions in an incremental manner: that is, the requirements portion of the documentation (along with test plan) is submitted, graded, and returned to students before they are expected to begin working on a design, and so on. This establishes a quality assurance process that encourages students to think about and document plans for solutions before beginning to code them. It also enables us to provide steady feedback on the systems students are developing. This is in contrast to our previous practice of making assignments, having students complete and submit them for grading, then having students begin working on the next project as we begin grading the previous one. Thus, in the past, students were required to begin working on the next assignment before receiving any feedback on the previous one. The standards quickly became an effective means of communication among faculty and students about software products. What we learned through the use of the standards was that they could easily be used as a framework to expose students to a variety of software engineering principles. This went beyond our initial objective of having students pursue a systematic, incremental approach to software development. Thus, we perused the literature to identify additional software engineering concepts that were considered appropriate and essential for novice programmers. Concepts that can be incorporated naturally into our introductory sequence using this documentation framework include [8, 1, 9]: • The need for software engineering; • Characteristics of good software; • The software development process; • Specification of software requirements; • Software design quality and documentation; • Software testing plans and techniques; • Maintenance; and • Aspects of teamwork. We began using the framework to teach these software engineering concepts in our introductory courses by designing a variety of types of software development assignments. In addition to allowing us to make assignments that are submitted incrementally, a framework of documentation standards makes it possible to develop programming assignments which focus on subsets of these software engineering principles. Assignments might require students to work through all phases of the software development process and submit the deliverables for each phase separately and incrementally. On the other hand assignments might include providing students with one or more parts of a solution and requiring them to work on other parts of a problem solution. That is, every assignment need not require students to build problem solutions by starting with requirements analysis and ending with complete solutions. In making assignments, for example, we have tried the following: • Present students with design documentation and code written by others and assign them the task of structurally testing that code.
•
Provide students with requirements for a system and the working system (executable code only) and ask them to do the system testing. • Provide students with requirements and design documents written by others and ask them to implement and test the system. • Provide students with working systems and complete documentation packages and ask them to modify/extend the system in some way. • Provide students with the requirements for a system and ask them to produce a high-level design without giving them details of the physical implementation of the system. • Require students to develop a design for an abstract data type (ADT) without providing them with details of the eventual application of the ADT. • Ask students to work with poorly developed documents or samples of poorly-engineered systems in order to emphasize the need for software engineering and adequate documentation. • Use design documents written by others to teach students to assess, compare, and critique designs. • Allow students to work in small teams to develop various portions of the systems and the related documentation. This approach of using a variety of types of assignments is beneficial in that we can expose students to many more different software development problems and also give them the experience of working with large software systems. It avoids the drawback of developing all systems from scratch, which takes a good amount of time and limits the number and nature of assignments to which we can expose students.
Perceived Benefits We have perceived both costs and benefits to this approach in teaching the introductory courses. The primary costs have been in the form of additional instructor time required to prepare, grade, and provide feedback on assignments. In order for the incremental quality assurance process to work, students require fairly rapid feedback on their deliverables. However, it is our opinion that the resulting benefits have made the additional time worthwhile. One of the major benefits of introducing software engineering early in the curriculum and using it coherently throughout the introductory sequence is that students recognize the importance of software engineering and formal documentation while they are still novice software developers. They acquire good software development habits and emerge from the introductory sequence having been exposed to a variety of software development problems, including relatively large systems that would have been impossible to build from scratch in the time allowed. The full benefit of this method of teaching software engineering can only be experienced when there is extensive cooperation among faculty teaching the courses. We have periodic meetings (formal and informal) to discuss and agree upon the topics to be covered in each of the three courses in the
sequence. The second course expects certain outputs in terms of student learning from the first course; the third course expects certain outputs from the second. This faculty cooperation has allowed us to provide students with a consistent approach from one course to the next and to use each subsequent course to reinforce software engineering concepts learned in the earlier course(s). A side effect and benefit of this faculty teamwork is that it has made teaching of the courses in the sequence much more enjoyable.
Tracking Our Students All of the faculty involved in teaching courses in the new introductory sequence believe that students leave these courses as more mature software developers than students did when they completed the courses in the old sequence. Indeed, our hypothesis from the beginning has been that students who complete the new introductory sequence would be better prepared for the project-oriented upper-level courses than those whose introductory courses lacked the software engineering focus. In an effort to test the validity of this hypothesis and assess the effectiveness of our teaching efforts, we conducted a study of student performance in required, project-oriented upper-level courses from the 1995 spring semester through the 1997 fall semester. Our data were obtained from two groups of students as they progressed through the project-oriented upper-level courses. The two groups were: those who took the introductory sequence before and those who took it after the modifications were made. We did not carry out a formal experimental study with an experimental group taking courses in the new introductory sequence and a control group taking courses in the old sequence because it would not have been ethical to preclude some students from the perceived benefits in order to have a formal experimental design. Moreover, from a practical perspective, it would have been hard to teach multiple sections of the courses at the introductory level given our departmental constraints. We chose to measure student performance based on project grades earned by students taking project-oriented courses. For each class studied, we excluded the grades earned by very good students and very weak students. Our reasoning for this was that we expected that the very good students would be very likely to do good work and the very weak students would be likely to do poor work despite their earlier preparation, and we did not want the very good and very weak students to bias the results we observed. For the purposes of this study, we identified students to be very good or very weak based on their overall grade point average (GPA) at the time they enrolled in the course under study. Therefore, we eliminated all students with GPAs of 3.7 or above and those with GPAs of 2.7 or below from our student pool. We also eliminated any students who stopped participating in class (attendance, assignments, etc.) before the end of the semester. Finally, we eliminated all graduate students from the subject pool as we did not believe that their performances and GPAs could be compared against
Table 1: Student Performance on Projects in Project-Oriented Courses COURSE
YEARS
Without-new-sequence
With-new-sequence
GPA
Project
N
GPA
Project
N
Software Engineering
1995-1997
3.24
76.80
32
3.26
80.53
28
Operating Systems
1995-1997
3.18
80.48
25
3.20
83.17
20
Compiler Construction
1995-1997
3.23
88.38
33
3.27
94.00
17
1995 1996 1997
3.21 3.20 3.61
85.71 89.22 99.82
14 17 2
3.22 3.32
91.62 96.25
0 8 9
those of the undergraduates. Thus, for this study we considered only the performances of undergraduate students with GPAs greater than 2.7 and less than 3.7. There are four required upper-level courses in our curriculum that include a large software development project: Compiler Construction, Database Management Systems, Operating Systems, and Software Engineering. We excluded the Database Management Systems course from this analysis because it was taught by three different instructors over the period of study. Additionally, the course enrollment each semester was dominated by graduate students leaving us with few (sometimes as few as two) undergraduate students per semester to include in the student pool. Each of the other three courses, however, was taught by the same instructor using a similar format in each year under study. For these courses, we subdivided the students in the study pool into two groups: students who had taken at least one course in the new introductory sequence and students who had not. Hereafter, we refer to these groups as withnew-sequence and without-new-sequence, respectively. In Table 1 we report the average GPA, the average project grade, and the number (N) of students in each group for each course over the study period of spring semester of 1995 through fall semester of 1997. For the Compiler Construction course, we further report these measures for each semester/year because they allow us to discuss more subtle benefits of introducing software engineering concepts early in the curriculum. It should be noted here that the project-oriented courses require/allow students to work in teams to complete their projects. In no course did we make any effort to group students according to their backgrounds in terms of enrollment history in the introductory sequence. Team formation differs from instructor to instructor, some allowing students to self-select teams and others making team assignments. We do not have specific team-composition
information, but know that some teams were composed of a mix of students, some with-new-sequence and some withoutnew-sequence. The results in Table 1 support the hypothesis that introducing software engineering early in the curriculum better prepares students to work on the larger projects in the upper-level courses. In the Software Engineering course, for example, the with-new-sequence group outperformed the without-new-sequence group by more than a third of a letter grade (or 3.73 points); in the Operating Systems course by a quarter of a letter grade (or 2.69 points); and in the Compiler Construction course by half a letter grade (or 5.62 points). The average GPA for each group was comparable in each course. Additionally, comparison across years during the transition period suggests that students without the new sequence benefited from being in project groups with students who had the new sequence. For example, in the Compiler Construction course in 1995, none of the students enrolled in the class had been through our new introductory sequence. In 1996 and 1997 the classes were populated by a mix of students with and without the new sequence. For the 1996ñ1997 period, the average GPA for all students in the Compiler Construction course in the without-new-sequence group was 3.24 and the average project grade was 90.34. Thus performance by students without-new-sequence, but mixed with the with-new-sequence students, was more than half a letter grade (or 6.63 points) stronger than in the 1995 sample (when the average was 85.71) when there was no opportunity for cultural diffusion from the with-newsequence students to the without- new-sequence students. Finally, in looking further at student performance on the individual deliverable/components that contributed to the final project grades earned in the Software Engineering course, we observed that students in the with-new-sequence group scored on average almost a whole letter grade (or 8.54
points) better than those in the other group on the first component of the semester project. Typically, this first deliverable of the semester project is completed and submitted very early in the semester, before much software engineering is actually taught. We believe that this large difference in performance between the two groups over the three years of the study is further evidence that the students exposed to software engineering principles early are better prepared for project work that those who are not. We find these results reassuring. Although there are many factors that might influence student performance such as quality of teaching, textbook, and assignments, as well as student motivation and interest in the subject matter, we tried to minimize the impact of these factors in this study by considering only project grades and not exam scores and final course grades. We believe that the results of this study validate our original hypothesis. We do not expect, however, that we will be able to collect much more data for this study, as the number of students enrolling each semester in the projectoriented courses without-new-sequence is diminishing rapidly. The majority of students who went through the introductory sequence before we modified the courses have since graduated. For example, in the spring 1998 offering of the Compiler Construction course there were 22 undergraduates enrolled, but only one of them was a member of the without-new-sequence group.
Summary and Future Plans This paper describes how we modified our introductory computer science courses to provide a consistent software engineering focus throughout the sequence. The pedagogic framework that we use to teach these courses is dependent upon a set of well-defined documentation standards that can be used to structure assignments in a variety of ways. This allows us to expose students to different types of problems and practical software engineering experiences. We have been teaching the introductory courses using this framework since the fall of 1994. We reported on a study carried out in order to assess the merits of this approach and validate the hypothesis that students who are exposed to software engineering concepts early in the curriculum are better prepared for success in later courses that involve large software development projects. The results of this study indicate that project grades of students exposed to the new sequence were higher on average than those not exposed to software engineering concepts early. In conclusion, we have found that the use of documentation standards improves faculty-student communication about software products. Also, an early and consistent emphasis on software engineering concepts creates a student culture that values the principles and practices of software engineering, and based on the above results, produces students who are better prepared to deal with large software projects.
We plan to continue use of this framework of documentation standards to teach appropriate software engineering principles and techniques in the introductory sequence. These documentation standards have been used or adapted for use by faculty at other institutions as well. See Almstrum [10] for an example. We hope to work with these colleagues to assess the merits of this framework for teaching software engineering concepts in a variety of ways throughout the curriculum.
References 1. Hilburn, T.B., I. Hirmanpour and A. Kornecky. The integration of software engineering into a computer science curriculum, Proceedings of the 8th SEI Conference on Software Engineering Education, 1995, pp. 87−97. 2. Jones, C. Gaps in programming education, IEEE Computer, 28(4), 1995, pp. 70−71. 3. Weide, B.W. Challenges of software design and the undergraduate computing curriculum, IEEE Computer, 28(8), pp. 31−32. 4. Wall Street Journal. “More practice, less theory,” say software companies, February 21, 1995, Section B1. 5. McCauley, R.A., U. Jackson and B. Manaris. Documentation standards in the undergraduate computer science curriculum, Proceedings of the 27 th SIGCSE Technical Symposium on Computer Science Education, 1996, pp. 242−246. 6. Gersting, J.L. A software engineering ‘frosting’ on a traditional cs-1 course, Papers of the 25th SIGCSE Technical Symposium on Computer Science Education, 1994, pp. 233−237. 7. McCauley, R. Documentation Standards, 8. Robergé, J. and C. Suriano. Using laboratories to teach software engineering principles in the introductory computer science sequence, Papers of the 25 th SIGCSE Technical Symposium on Computer Science Education, 1994, pp. 106−110. 9. Jackson, U., B. Manaris and R. McCauley. Strategies for effective integration of software engineering concepts and techniques into the undergraduate computer science curriculum, Proceedings of the 28th SIGCSE Technical Symposium on Computer Science Education, 1997, pp. 360−364. 10.Almstrum ,Vicki. CS328 Abstract Data Types: Programming Documentation Standards, .
