Sep 20, 1994 - Professorial intervention in the classroom was minimal. ... paper will encourage other professors to teach formal methods in the undergraduate curriculum, ... the input-output relationship the program is intended to produce. ... We also use the multiple assignment statement, where all of the expressions on ...
(In)Formal Methods: Teaching Program Derivation via the Moore Method James A. Foster Karen Van Houten a
Michael Barnett Luke Sheneman
a
a
Department of Computer Science University of Idaho Moscow, Idaho 83844-1010
September 20, 1994 Technical Report LAL 94-01 a Member,
Laboratory for Applied Logic
Acknowledgements: We would like to thank John Dickinson, the chair of the Department of Computer Science at the University of Idaho and the College of Engineering for providing the time necessary to teach this course. We also gratefully acknowledge the help and support of Dr. James Alves-Foss and Dr. Deborah Frincke and the Laboratory for Applied Logic, where formal methods are appreciated daily. And, most importantly, we thank the students who took our course|without whom the experiment would surely have failed. c 1994. Laboratory for Applied Logic. All rights reserved. Copyright This document may be copied in whole or in part for educational or research purposes.
1
Abstract Formal methods, the application of mathematical tools to software development, are underrepresented in the typical Computer Science curriculum. This is due in part to the diculty of teaching this material to students in an engaging and meaningful way. We o ered an experimental course on derivational programming, one approach to formal methods, using a primarily non lecture-oriented pedagogy. The teaching technique, known as the Moore method, requires students to discover the subject matter being taught for themselves. In our speci c case, the students learned the underlying mathematics of program derivation and learned to apply it, by presenting proofs and derivations on a daily basis. Professorial intervention in the classroom was minimal. Our experience has been that students learn otherwise dicult material better, and are better able to put it into practice, with this teaching technique than they would have been able to do in the typical classroom.
1 INTRODUCTION
2
1 Introduction During the fall semester of the 1993-4 academic year, two of us taught an experimental course at the University of Idaho in the Computer Science Department, entitled The Future of Computing. It was intended to teach the derivational style of programming using Cohen's text [3]; this is the style of programming pioneered by Dijkstra [5, 6] as well as others [12]. The course was taught using the Moore method, [1, pp. 287{294], a method that places a premium on student participation. The respective roles of the authors were: Barnett and Foster taught the course (Barnett led discussions, Foster usually observed and kept a journal), Sheneman was an undergraduate student in the course, Van Houten was an impartial observer. It is our belief that the experiment was a success. In this paper, we provide a review from our combined perspectives. We believe very strongly in the application of formal methods to programming; for programming \in the small", program derivation allows students to write imperative programs with a high degree of certainty in the correctness of their e orts. However, formal methods, and program derivation in particular, require the devotion of a large amount of time to practice the techniques: in this it shares much with other forms of calculus. We have been disappointed with our experiences teaching formal methods courses that rely on traditional pedagogics. Our experience with the Moore method, in contrast, has been highly encouraging. We hope this paper will encourage other professors to teach formal methods in the undergraduate curriculum, especially using the Moore method, or some other non-lecture based pedagogy. In our opinion, formal methods are a very important part of computer science which are often under-represented in the curriculum. In our department, if students see any formal methods at all, it is late in their careers. However, the only real prerequisite to a course emphasizing a practical introduction to program derivation is some familiarity with the predicate calculus, such as that found in any typical discrete mathematics course. Unfortunately, such courses sometimes emphasize mathematical results and facts over the act of proving theorems. We chose the Moore method for several reasons. First, formal methods are not signi cant if they are of only theoretical interest. We wanted our students to have hands-on experience with this methodology. Second, we are convinced that most students retain information better if they discover it for themselves, than if they hear it second hand in lectures. Third, we are strongly committed to undergraduate research, and the collaborative nature of classes taught
1 INTRODUCTION
3
with this method require a great deal of discovery in groups. We realized that this approach would require a great many oce hours, perhaps even an \open door" policy (which was customary with us in any case). This expectation did in fact prove true. Students often came to the professors' oces to work through proofs, more for con rmation than because the students were completely stumped. The painfully shy students tended to show us work in our oces before showing it to the class. We team-taught this course in order to support the extra-classroom requirements of the students, and to allow one instructor to be free during class to be an observer while the other functioned as a discussion prompter. We anticipate that this will not be necessary were the course to be repeated. In the rest of this section, we provide an introduction to the di erent aspects of the course: Section 1.1 explains what formal methods are, Section 1.2 presents the approach of derivational programming, and Section 1.3 describes the Moore method. Readers familiar with any of the topics may skip those sections. The course itself is described in detail in Section 2. Section 3 contains evaluations of the course from our three perspectives: the instructors, an independent CS educational professional, and a student. Selected notes adapted from a contemporaneous journal kept by one of us (Foster) are in Section 4. Conclusions and future plans are in Section 5.
1.1 Formal Methods The area of formal methods is not, in itself, a subject matter, but an approach to any subject matter within Computer Science. [4, 10, 11]. A good source for current information is the World-Wide Web Virtual Library on Formal Methods located at: http://www.comlab.ox.ac.uk/archive/formal-methods.html
There is not a single, agreed-upon de nition, but in the broadest terms it is the application of appropriate mathematical tools for solving problems (in computer systems). Of course, such a broad de nition can hardly be argued with: but in terms of programming, it refers to an attempt to nd the right mathematical tool(s) with which to treat programs as mathematical objects. In a fundamental sense, this is unavoidable|programs are formal mathematical objects, since a computer is just a machine that performs formal symbol manipulation. Formal Methods are most often identi ed with the area of program correctness:
1 INTRODUCTION
4
using mathematical analysis to predict program properties. The common starting point for all formal methods is a formal speci cation, i.e., a formula in the predicate calculus that expresses the input-output relationship the program is intended to produce. Formal methods aim for a rigorous, mathematical correspondence between the formal speci cation of what a program is intended to do and the program itself, which speci es what the program actually will do. It is in this sense that the resulting programs can be said to be \provably correct".
1.2 Derivational Programming In this space, it is not possible to do more than give a brief overview of this topic: complete treatments can be found in [3, 5, 6, 12]. Derivational programming is a particular methodology within formal methods that uses the tools of the predicate calculus to create programs hand-in-hand with their proofs|indeed, the search for a proof is what guides the writing of the program. This contrasts with attempting to prove that a program corresponds to its speci cation in a post-hoc manner. Using a set of heuristics as a guide, a programmer decomposes the speci cation into subproblems, letting the shape of the formulas drive the search for program structure. For example, a universally quanti ed formula indicates that a predicate must hold over an entire domain. A loop is the most common program structure causing iteration over a domain (recursion being the other). The methodology provides a checklist, which if met, guarantees a loop will satisfy its speci cation. Each step in a program derivation is a calculation in the predicate calculus; just as students cannot solve physics problems without practice in the integral calculus, students cannot hope to derive programs without being well-versed in the predicate calculus.
1.2.1 An Example We use an example adapted from the text to illustrate the basic techniques of derivational programming. A familiarity with the predicate calculus is assumed. The table of squares of the natural numbers is the sequence 0, 1, 4, 9, 16, : : : The problem is: Given integer N 0, write a program that prints the rst N entries in the tables of squares. (We refer to this as the postcondition of the program; the result desired from
1 INTRODUCTION
5
the execution of the program.) Further, no exponentiation or multiplication is allowed in the solution. (Without this restriction, the program is trivial. The reader is invited to solve the problem | with the restriction | before continuing.) Obviously, a program containing a loop is required, with one iteration for printing each entry in the table; we use an alternative notation for a while loop with B as the condition (guard) and S the body: do B --> S od
We also use the multiple assignment statement, where all of the expressions on the right-hand side are evaluated, then assigned to the variables on the left-hand side. When faced with a postcondition that cannot be immediately met, a heuristic suggests replacing the constant N by a fresh variable n the rst n entries in the table of squares have been printed ^ n = N which decomposes the problem into two smaller ones: how to print the rst n entries of the table, and then how to reach a state where n = N . When the conjunction holds, then it is equivalent to the postcondition. When it fails to hold because of the second conjunct, the program is not nished; we have not printed all of the entries in the table. The negation of the second conjunct can function as the guard of the loop. The rst conjunct becomes the invariant: a condition that encodes the relationships maintained during the execution of the program. After each iteration of the loop, the rst n entries will have been printed, but execution should increase n and print more entries. An invariant does not mean that there are no values being changed, else the program would be useless! In this case, two invariants are created: 0 : the rst n entries in the table of squares have been printed P1 : n0
P
(P 1 guarantees that P 0 is de ned.) This immediately yields the program: var n : int; n := 0;
f
P0
^
P1
g
1 INTRODUCTION do n
6=
6
N -->
print(n^2); n := n + 1 od
f
P0
^
)
P1
postcondition
g
Of course, this is not much better a solution, because it contains an exponentiation, which is disallowed. Again, we follow the heuristic by introducing another fresh variable, x, with the invariant: P
2 :
x
= n2
2 can be initialized before the loop by an assignment; the \trick" is to see if it can be maintained during the execution of the loop. Since the incrementing of n violates it, a new value for x must be derived that re-establishes the invariant:
P
var n, x : int; n, x := 0, 0; do n
6=
f
^
P0
P1
^
P2
g
N -->
print(x); n, x := n + 1, E od
f
P0
^
P1
^
P2
)
postcondition
g
The unknown value E is calculated by using the weakest precondition calculus which has the axiom: wp :x := E :P =
x
PE
(read the left-hand side as \the weakest precondition of the assignment of E to x in order to achieve the postcondition P ") where P represents the expression obtained by replacing all (free) occurrences of x in P by the expression E . One way to understand this axiom is that after the assignment statement the predicate x = E is certain to hold, so for P to hold afterwards, whatever conditions it imposes on x must hold for E before the assignment statement. (Because x E
1 INTRODUCTION
7
we assume that the evaluation of the expression E has no side-e ects.) The calculation proceeds as: := n + 1; E :P 2 h Axiom 1 i
wp:n; x
=
2 +1 = h Substitution i 2 E = (n + 1) = h Arithmetic i 2 E = n + 2n + 1 = h Invariant P 2 i E = x + 2n + 1 = h Introduce a new variable y i E = x+y P
n;x n
;E
Of course, in order to introduce a new variable y , we must have an invariant that goes with it: P
3 :
y
= 2n + 1
It is easy enough to initialize P 3 by assigning 1 to it before the loop. Once again, the incrementing of n violates the invariant, so a new value must be assigned to it that re-establishes it: var n, x, y : int; n, x, y := 0, 0, 1; do n
6=
f
P0
^
P1
^
P2
^
P3
g
N -->
print(x); n, x, y := n + 1, x + y, E od
f
P0
^
P1
^
P2
^
P3
)
postcondition
g
And again, we have to calculate the value for the expression E :
1 INTRODUCTION
8 := n + 1; x + y; E :P 3 h Axiom 1 i
wp:n; x; y
=
3 +1 + = h Substitution i E = 2(n + 1) + 1 = h Arithmetic i E = 2n + 3 = h Invariant P 3 i E = y +2 P
n;x;y n
;x
y;E
The expression y +2 can be implemented directly, so no new variables are required; the program is complete. var n, x, y : int; n, x, y := 0, 0, 1; do n
6=
f
P0
^
P1
^
P2
^
P3
g
N -->
print(x); n, x, y := n + 1, x + y, y + 2 od
f
P0
^
P1
^
P2
^
P3
)
postcondition
g
The soundness of the axioms with which we justi ed the steps of the calculations guarantees that if the loop ever terminates, all of the invariants will still hold, and together they imply that the desired postcondition will hold. The only uncertainty is whether the loop will terminate or not. For that, we need a metric: this is a function, t, which acts as a yardstick to measure how much longer the program will execute. (We do not explicitly indicate t's parameters; it may be any of the program variables.) The metric and the program must meet all of the following conditions: 1. t is an integer-valued function. 2. Execution of the body will de nitely decrease the value of t. 3. Decreasing the metric is guaranteed to falsify the loop condition, i.e., P ^ t 0 ) :B . (P is the conjunction of all of the invariants.)
1 INTRODUCTION
9
Given those properties, we are guaranteed that the loop will terminate. If the loop condition holds, then by the last item, t has some positive value. When the body is executed, by the second item, t will decrease. At some point in time, it must drop to zero or below, at which time the last item guarantees that the condition will become false, and the loop will stop. For this problem, the obvious metric is N ? n. An important point to notice is that the introduction of each new variable did not require us to go back and re-prove all of the previous invariants were maintained. We assume that assigning a value to a new variable cannot change any of the values of other variables.
1.3 The Moore Method The Moore method [1, pp. 287{294] was created by R.L. Moore, a professor of mathematics at the University of Texas at Austin in the 1940s, and is still in use in some courses within that department. One of us (Barnett) took courses taught using the method in the departments both of Mathematics and of Computer Sciences while in Austin; the following is a description re ecting the interpretation of the method in the late 1980s. The method forces students to learn a subject by rediscovering it from rst principles. It is related to the Socratic method in its reliance on eliciting the answer from the students. To assist the students|and to prevent their becoming tainted by having the material explained to them|Moore would remove all relevant texts from the library. (Clearly, this was before the current age of computer networks and interlibrary loans.) Students, upon entering the course, are given a packet of the theorems constituting the eld, and have to prove the theorems at the board during class. The professor's r^ole is that of passive observer: to sit at the back of the class and merely mark down the names of the students who go to the board. A mark is made regardless of the success or failure of the student, and encouragement, criticism, and help are all allowed from the other students. Obviously, such a course is extremely sensitive to the dynamics between the students and the professor, as well as those between the students themselves.
2 THE COURSE
10
2 The Course 2.1 Prerequisites We had no formal prerequisite to this course, since we wanted to address the course to students at every level, from freshman to graduate student. We provided the necessary practice with the predicate calculus in the rst few weeks of the course.
2.2 The Syllabus Our semesters are 15 weeks, which we split into two main parts: learning the propositional and predicate calculus (about 6 weeks), and then applying them to the problem of program derivation (about 9 weeks). Figure 1 shows how much time was spent on each chapter of Cohen's [3] text. We had time only to begin Chapter 11. The rst part of the course was oriented to teaching not only the two calculii, but also enforcing a uniform proof format. The format, shown in Figure 2, serves as a concise vehicle for performing proofs, and is a useful pedagogical tool. Requiring a hint in between each step, and allowing at least an entire line for it encourages giving a full justi cation for each step. At the same time, using a standard layout made it easier for the students to follow each other's proofs.
2.3 Grading Grading in the course was done by keeping track of the number of times that a student went to the board. It made no di erence if the student completed a problem, or if the solution was incorrect, or had already been presented in a di erent manner; a student received a \board credit" regardless. A tally sheet of the credits was available at each class for them to see. They were told that grades would be given based on an absolute number of visits to the board: around 15 to 20 credits would be enough for an \A". (This was to prevent them from comparing themselves to others in the class and playing an endless game of \catch up".) In the later part of the course, as the problems became larger and more involved, we allowed a group of students to get credit by solving a problem together with one representative presenting the solution at the board.
3 EVALUATION
11
2.4 The Faculty The two instructors are junior research-oriented faculty. One (Barnett) is trained in formal methods, and specializes in parallel systems. The other (Foster) is a computational complexity theorist who is also trained in mathematical logic. The third faculty member (Van Houten), observing the course, is a senior teaching-oriented faculty member with 20 years of college level teaching experience, and has been chair of, or has served on, curriculum committees at the Department, College, and University levels.
2.5 The Students The students ranged from beginning CS students to graduate students. There were more upper class students than rst or second year students; probably a result of upper class students having fewer required courses in their schedule. Another contributing factor is that many of them were about to take, or had just taken, our more theoretical upper division courses, and felt they needed the preparation. Figure 3 presents the data on the class composition. Figure 4 shows the math, computer science, and overall grade point averages for all of the students prior to the semester the course was o ered. The averages for math and computer science included any F's that students had on their transcripts, even if they repeated the course. The overall GPAs do not include any such F's. It is apparent that we had a broad range of scholastic aptitudes in the classroom.
3 Evaluation In this section we present three perspectives on the successes and failure of this approach to teaching this material.
3.1 From the Instructors' View 3.1.1 Class Composition There were fteen students in the class. This was slightly smaller than a typical upper division class in our department. One might be tempted to attribute our success to the 7.5:1 student to teacher ratio. However, it must be borne in mind that the professors rarely lectured, and
3 EVALUATION
0 1
12
2
3
4
5
6 7
9
10
11
Figure 1: Amount of time spent on each chapter in the text. The numbers within the boxes are chapter numbers. Each tick mark on the scale represents one week.
Theorem: X _ true true X _ true hP P true, with P := Y i X _ (Y Y ) h_ distributes over i
X
_ _ h , with := _ i
true
Y
X
P
P
Y
P
X
Y
Figure 2: A Proof in the standard format. In this case, it is a sequence of equivalence preserving transformations, with the justi cation for each step in angle brackets following the equivalence symbol that connects adjacent lines. Other connectives than equivalence are also allowed.
8
Number of Students
7 6 5 4 3 2 1 0 1
2
3 4 Year in School
5
Figure 3: Year in School of Students in the Class
3 EVALUATION
13
one professor was almost exclusively an observer and chronicler. Given these considerations, the e ective professor to student ratio was typical for our department. However, we feel that the teacher to student ratio was about 5:1, since students tended to form small groups of about ve people and to teach each other. We attribute much of the course's success to this fact. Often, experimental courses have a built in bias: the best students are attracted to them, these students do well (as they would in any course), and so the course succeeds. In our case, we had some excellent students, some hackers, and some who were academically very poor, as Figure 4 shows.
3.1.2 Course Organization Initially, students were uncomfortable with the idea of a course without lectures. They were suspicious of a grading system that did not depend on homework and tests. They were hesitant to present proofs in front of their peers. In short, the idea of a course structured entirely around publicly proving theorems was daunting. However, the rst several problems were very simple applications of logical axioms. With little prodding, the students realized that solving these problems was a trivial matter of nding which rules applied, then applying them. Our standardized proof format (see Figure 2) also helped persuade them that whatever they presented would at least look like what anyone else had done. With such rmly established ground rules, the process of proving theorems became more like a game than like a chore. What's more, the rules of the game assured the students of success, as they quickly found. In fact, several students commented that they found the exercises addictive. They viewed nding a proof like solving a puzzle, and they knew that they could solve the puzzle. Consequently, they spent much time proving exercises on their own. Our experience with other courses leads us to believe that this is unusual. Students rarely volunteer to do mathematical proofs for the simple enjoyment of it. It is also interesting to note that there were at least four faculty members to whom we described this approach who also became addicted to nding these proofs|to the point of losing sleep over them!
3 EVALUATION
14
3.1.3 Course Evaluations We take student evaluations of every course taught in our department. This evaluation asks students to rank ve statements according to how strongly they agree or disagree with the statements. The actual questions on the evaluation form are presented in Appendix A. Each student indicated whether they Strongly Agree, Agree, were Neutral, Disagree, or Strongly Disagree with each of the questions. These answers were given values from 4 to 0, with 4 corresponding to Strongly Agree, and their average was computed. We had students ll out separate forms for each professor, even though we were both relatively non-intrusive in this class, and one of us was almost entirely so. The results are given in Table 1. For comparison, averages for all CS courses, for upper-division CS courses, and college-wide averages are included. (The department is within the College of Engineering.) It is clear from these evaluations that the students thought they had bene ted a great deal from the course. We are tracking the student's grades in math and computer science courses as they progress through our curriculum in order to determine if they actually did receive any measurable bene t from the course|at least relative to their grades. We have not collected enough data yet for any de nitive conclusions. In addition to these standardized measurements, we asked the students to write essays on their impressions of the course, once at approximately mid term and once during the last class. We used these essays as evidence in our evaluation of the course, and many of our conclusions are based on their contents. One of us (Sheneman) was a student in the class and was invited to extend his comments into subsection 3.3. We felt this was a more e ective way to present the student's view than including excerpts from several student essays.
3.2 An Educational Professional's View 3.2.1 Introduction to My Interest and Involvement I was intrigued by the technique that was planned in teaching this course, so I decided to sit in and see how it worked. I was also interested in the material that was to be covered. Other professors in the department have been lobbying for the incorporation of formal methods in the freshman level CS courses and I wanted to know what they were talking about. This was particularly important to me, as I have been chair of the University Curriculum Committee and the Department Curriculum Committee, and have also served on the College of Engineering
3 EVALUATION
15
Number of Students
10
Math CS Overall
8 6 4 2 0 0-1.9
2.0-2.4
2.5-2.9 Student GPA
3.0-3.4
3.5-4.0
Figure 4: Math, CS, and Overall Grade Point Averages. GPAs are on a 0{4 scale.
Question Inst 1 Inst 2 Instructor Prepared 3.7 3.9 Informed of Progress 3.4 3.5 E ective Instruction 3.8 3.8 Relevant Material and Concepts 3.8 3.9 Would recommend to others 3.9 3.9 a Average b Average c Average d Average
CS CS College 3.1 3.3 3.1 3.0 3.0 3.0 2.9 3.1 2.9 2.9 3.4 3.0 2.9 3.2 3.0 a
b
c
College 3.3 3.0 3.1 3.2 3.1
d
of all Freshman and Sophomore CS courses for Fall 1993. of all Junior and Senior CS courses for Fall 1993. of all Freshman and Sophomore College of Engineering courses in Fall 1993. of all Junior and Senior College of Engineering Courses courses in Fall 1993.
Table 1: Average scores from student evaluations, from 4.0 (Strongly Agree) to 0 (Strongly Disagree).
3 EVALUATION
16
Curriculum Committee. I have a keen interest in the undergraduate curriculum and am always looking for ways to improve it. This course was taught as a three credit class. At the University of Idaho, three credit courses typically meet 150 minutes per week for 14 or 15 weeks with another week for nals. This is normally a lecture format with each student expected to spend twice that time studying for the class each week. This class actually met each week for 60 minutes one day and 90 another. Because of my schedule, I was only able to attend the 90 minute session each week.
3.2.2 Some Observations Foster describes the student reactions and the classroom ambiance very well in Section 4. The chaos was e ective most of the time, but I found it a bit too chaotic at times. Some Computer Science faculty were concerned that the advertisement of no exams would attract students looking for an easy A. I doubt that this was the case after the rst week when they found that they would be graded on doing proofs at the board. The course did an excellent job of making students comfortable with proofs and propositional logic. The derivational programming was not covered as thoroughly as planned because of time constraints. A solution to this would be to have the course meet longer than the usual 50 minutes per credit per week. I would classify the course format as a mix of lecture, recitation and lab, which typically meet longer each week for a credit. Even though a lot of the proofs became repetitive, I think it is important that all of them be presented and this takes time. The class progress was also slowed by all the class discussion, which is a nice problem to have. I was somewhat uncomfortable with the grading scheme. As indicated elsewhere (see section 5.2.2, the instructors had a good idea of which students had mastered the material, but there was no objective way to grade. I would be more comfortable with this type of class if a better way of grading were found. One idea I have thought of is to have the students keep a notebook of all the proofs they have done and that were done in class. This would be turned in at various times during the course. The notebook and the number of times at the board would be used to grade the students. There were times that the students in the class were not paying attention to those at the board, when they should have been. Keeping a notebook would have the side bene t of keeping them involved in the current discussion.
3 EVALUATION
17
3.3 From the Students' Perspective 3.3.1 First Impressions As a student, the idea of participating in a course which used a non-traditional teaching style was very appealing to me. Because of this, and at the advice of my peers and my advisor, I enrolled in the Future of Computing. During the rst class, when the guidelines and the rules were laid down by the faculty \coaches", I became very wary of the new teaching technique in which I had become involved. The concept of students teaching students seemed far too alien to ever work. I had a tremendous amount of anxiety about arguing and explaining proofs from the front of the class. I speak for most students when I state that being selected to \go to the board" is one of the most undesirable activities in a traditional classroom setting. The fear of making obvious errors and being ridiculed by both teachers and peers is a de nite cause for concern for most students. Although we were told by the faculty members involved in the class that negative comments and negative peer pressure were strictly not allowed, this did little to curb my anxiety and fear of the class. I am sure that many other students shared my feelings during the rst few sessions of this course. Quite honestly, I was afraid of proofs. During my years as a student, I have never been required to mathematically or rigorously prove anything. I had serious doubts about my ability to perform satisfactorily in class, let alone in front of my peers. This insecurity was enough to make me question my desire to be in the class, and I considered dropping the course after only two sessions. Fortunately, I stuck with it. After speaking to other students in the class, weeks later, I found that they had exactly the same doubts about their abilities, and about their willingness to do proofs on the board.
3.3.2 A Second Look Before the fourth session of the class, I prepared a couple of proofs outside of the classroom. Although the concepts were simplistic at the beginning of the course, I was not accustomed to thinking of problems in the manner in which they were presented in the textbook. The technique of looking at a logical expression without understanding its meaning was foreign to me, and perhaps the most dicult adjustment for me. All operators and symbols
3 EVALUATION
18
were meant to be taken at face value, without attempting to assign a deeper meaning to them. Instead, we were given, by the text, a set of actions and rules that could be applied to the otherwise meaningless symbols. It literally took me hours to perform my rst proof. I struggled with the idea of logical expressions and operators without meaning and the notation used in the text. Eventually, after hacking the problem apart and trying a variety of di erent things, I was able to prove one of the simple exercises in the book. I had an immense amount of satisfaction at having proved something, and I was eager to share what I had done with the class. After explaining my rst proof to the class, I quickly lost my fear of presenting and explaining things on the board.
3.3.3 Addicted To Puzzles? As the course progressed, I became progressively better and better at solving the exercises in the text. The idea of proving theorems, and proving them in di erent ways, was quite an exhilarating experience. I started viewing proofs as puzzles rather than work. I saw them as intellectual challenges which in many situations required ingenuity and creativity. From some of my interactions with my peers, I believe they felt exactly the same way. The work that I put forth for the class hardly felt like work. Instead, it felt like a game. Because of this, I was quite eager to do more than enough work to get an A in the course. I would often lose sleep over some of the more dicult problems, and nd myself running to a fellow student's room in the middle of the night to show him my proof. Sometimes, I simply couldn't wait to show my proofs in class. In retrospect, I nd this simply amazing. In no other class have I found such eagerness to participate and do homework. Traditional classroom scenarios have practically no student interaction and students have very little desire to do the assigned homework. Interestingly, the techniques for developing proofs and solving problems that I mastered in this course proved useful in other non-related classes. For example, I was enrolled in a second-semester engineering physics course during the same semester in which I was enrolled in the course. My physics class proved to be challenging for me, and I had diculty solving the homework problems. After familiarizing myself with the general problem-solving techniques taught in the class, I discovered that I could apply many of the techniques taught in this
4 JOURNAL NOTES
19
Computer Science course to my physics course. In the class, proofs are broken down into small pieces, and a concise set of abstract tools are then applied to these small pieces. The partial solutions are then added together to solve or prove the problem at hand. This technique for divide and conquer, as well as the rigorous and exhaustive use of small sets of prede ned tools applied quite well to my physics course-work. I am con dent that the problem-solving skills that I developed through the class will have an impact on the way I will approach any problem. I consider the development of general problem solving skills to be the largest bene t that I derived from taking the course.
3.3.4 Students Working With Students I rst believed that this course would provoke extremely competitive sentiments among the students. I thought that competition for board space and negative comments between students would cause feelings of hostility and frustration. In reality, however, things happened di erently. After students placed their solutions (proofs) on the board, and explained them, other students would ask questions and make comments. If the proof was awed in any way, students constructively attempted to explain why. Negative and derogatory comments were rare. While a student was explaining their proof, the classroom atmosphere was almost always one of support. It was refreshing to participate in a classroom atmosphere where the students were interested primarily in the truth, and were excited to share what they knew. There were also some problems with student interaction. The most eager and ambitious students (myself included) tended to get to the board before the shy students. Some eager students would go very early to class to put proofs on the board before anybody else. This caused some visible discomfort and frustration for some students. I believe that a rule insisting that proofs could be put on the boards only during class, and not before, might have been a good solution to the problem.
4 Journal Notes This section includes comments which are adapted from a journal, which one of us (Foster) kept contemporaneously with the course.
4 JOURNAL NOTES
20
Week 1: By the end of the week, students were convinced by experience that they could present
an acceptable proof, which they had discovered. These early successes were critical. Unless every student, including even the most mathematically unsophisticated, had been convinced that proof was no more than \mindless" application of rewriting rules, they would have been discouraged from participating, and the necessary learning environment would not have arisen. Psychologically, by emphasizing the symbolic, rather than semantic, nature of proofs we were able to get the students to assume that this was how proof derivations should be done. This symbolic stance would later apply to programs as well, and would enable the students to derive correct programs which would otherwise have eluded them.
Week 2: During the week, students became con dent that they could understand each other's
proofs, and that others would understand theirs. This encouraged them to work at the board and to discuss their solutions with each other. The few students who had not standardized their proof format by this time found their discussions more involved: others understood their work less well. After this class, all proofs were in the standardized format (see Figure 2), not because the instructors required it, but because the students had learned by interacting with other students that standardization disambiguates. What would have seemed arbitrary and boring, had it been described in a lecture and enforced by the instructor thus appeared natural and necessary. The students had taught themselves an unforgettable lesson about the importance of using formal notation carefully and precisely.
Week 3: Even the shiest and most reticent students were participating, both by presenting
their own work and by working in groups. Students had begun to show up before class to work on the board, and to stay after class. Again, this was not our typical experience with students' reactions to formal proofs! Occasionally, the instructors had to step in and present some material which was confusing students|usually confusion over notation. However, the dynamics of these \lectures" were unusual: the professor talked at the board, the students who were confused listened, others worked in small informal groups on other problems. This had the appearance of chaos, but it was e ective chaos. Students worked with whomever could help them,
4 JOURNAL NOTES
21
whether that was the professor, another student, or even themselves. A student could move from group to group at will, or even just sit back and eavesdrop on whichever conversation seemed most appropriate for their needs that day. This was very directed, and individualized learning. We can only imagine what a casual visitor would have thought. Imagine stepping into a classroom with ve or six di erent groups at the chalk board, all talking and scribbling. A couple students are just sitting quietly, staring rst at one group, then at another. Another small clique is huddled over a table, stabbing at a pile of scrap paper. Over all this, one professor is at the board \lecturing" about notation, as all but two or three students ignore him, as the other professor is scribbling furiously in a journal to record the scene. Our classes began to resemble this mayhem (though almost always without the \lecture") this early in the semester, and never changed thereafter.
Week 8: The problems had become much more challenging. The students were no longer
prepared with several solutions to present to the class. The professors had to perform one of the most dicult of all pedagogical feats: sit silently. The students did not have enough material prepared for board work, but we had told them we would not solve the problems for them. So, the instructors had to sit quietly, and the students had to endure the silence. After a few minutes, a couple brave souls volunteered to try to solve a problem from scratch. The problems were dicult, so they turned frequently to the class for help. Discussion ensued, and the class ended with a deeper level of interaction, and experimentation. The students now knew that they could solve dicult problems by working together. Students also gained a new appreciation for the power of formal methods. The same techniques which they had found attractive for easy problems worked for dicult ones. Twenty-line proofs, the students now knew, were essentially no more dicult than fourline proofs. The students also began at this point to develop an ugly proof, then to simplify it. The students showed obvious satisfaction in nding an elegant solution to a problem for which the previous solutions had been tedious and involved.
Week 9: We began to derive programs from speci cations. Until this point, we had been
proving theorems which would be useful when we nally began deriving programs, so
4 JOURNAL NOTES
22
the course resembled a discrete mathematics course more than one on programming. The students were a bit confused about the di erence between discovering a proof and deriving a program, at rst. Signi cantly, whenever there was any confusion, a discussion was initiated by the students. They were no longer shy about admitting ignorance and asking each other, and as a last resort the professors, for help. These discussions led to a fuller understanding of the material than could have been achieved with the traditional lecture followed by homework and tests.
Week 10: Proofs and derivations were so involved that only one or two could be presented per class. The students adapted to this by working in groups of up to six people.
An interesting event happened about this time. One student was displeased that some others would show up before class and put their solutions on the board. He was concerned that he would not be able to go to the board often enough to get an \A"|despite assurances to the contrary. One day he arrived an hour and a half early|and covered every board with solutions. The other students took him aside privately after this incident, and politely suggested that he had been to the board often enough and shouldn't provide any more solutions! The professors assured the student that he had indeed been to the board often enough to receive an \A". So he missed the next couple classes. This was an instance where the group dynamics actually caused a problem. However, after missing a couple classes, this student began showing up regularly and working with the groups. He had been working solo before. At the end of the semester, he told the professors that the class had been a very valuable experience. Our conclusion is that the group dynamics worked as a self-corrective. Again, the professors only role in solving this problem was assuring the student that we meant what we had said about our grading criteria. The rest of the situation was corrected by the students themselves.
Week 11: Students had become a little less enthusiastic as material became more involved. It
now took more time for a single problem, and few students came to class with solutions to every problem, as had happened before. Several problems from Chapter 9 were skipped while students began work on Chapter 10. Classroom discussion seemed to drag, and one student complained that the current material was hard.
4 JOURNAL NOTES
23
Week 12: The students had become comfortable with jumping around. Consequently, results
from Chapter 10 were presented, though not in sequential order. Many problems from Chapter 9 were skipped, though a student returned to nish them toward the end of this week. The level of excitement had risen to new heights. The boards (our classroom was lined with wall-to-wall chalk-boards) were covered within ten minutes. There were many small groups forming, dissolving, and re-forming to discuss particular topics. As one of our better students commented If I had a problem with something a group was doing on the board, I'd be listening to them. Our observer (Van Houten) called this mayhem \e ective chaos". One passage from my journal seems quite apt, so I include it here verbatim: This is distributed, targeted learning. It's important that people be working at the board|even if they don't have nished problems. Then the creative and discovery process is public, and those who will bene t can do so.
One algorithm which the student's derived in class was Horner's method for evaluating polynomials. I had previous experience presenting this algorithm in our CS I class, and found it dicult for some students to grasp with traditional lectures. In this class, the students understood the method thoroughly with less time by deriving it themselves.
Week 13: One of the students, who was working as a systems programmer for the CS depart-
ment, was currently writing a program to identify password le di erences. He was not satis ed that his e orts were correct, and was concerned that they should be so. He discussed formal speci cations, and e orts to derive his program, with a small group during the class. This was a wonderful, applied discussion of the importance and use of these techniques. It was also unplanned.
Another student volunteered to work on a problem at the board which no one in the class had been able to solve. He worked by himself for a while, thinking aloud, then gave up and asked the class for suggestions. A discussion ensued which lasted most of the class
5 CONCLUSIONS
24
period. By the end, the entire class, directed by a student volunteer, had solved this dicult problem.
Week 14: I was too excited to take thorough notes! Students were handling recursion in their derivations much easier by this time.
5 Conclusions In this section, we draw some conclusions from our experiment, and point out some further areas for work.
5.1 Program Derivation and Formal Methods Teaching this course has left us convinced that students need to be taught programming using some sort of formal method [7, 8]. For sequential, imperative programs, we feel that program derivation using the weakest precondition calculus is the most appropriate. However, we believe that it requires a radical change in the computer science curriculum. We spent a well-needed, but unfortunately long, time on teaching the propositional and predicate calculi, topics so basic to computer science that they should be taught at the beginning of the curriculum. But calculi are tools, and like all tools, they must be used constantly for students to remain skilled in their use. Just as the integral calculus is used throughout the curriculum in other engineering disciplines, the computer science curriculum must be restructured so that the use of formal methods is reinforced in each class. Otherwise, it is hard for students to understand their relevance.
5.2 The Moore Method When the Moore method works, it works fantastically well. Our expectations were surpassed in several ways.
5.2.1 Class Dynamics For the most part, the students responded very well to the idea that they were responsible for the entire course: its content, its pace, its entire dynamic.
5 CONCLUSIONS
25
Students quickly began to work in groups, and even to interact with the student at the board. This helped shy students overcome their shyness. In e ect, the class spontaneously developed study groups for individual diculties and \tutors" appeared as needed. With much of the pressure of tests and homework gone, many less talented students felt easier about participating. In the end, some of these students exceeded their own expectations. However, student expectations can be inconsistent with the actual course dynamics when the course format is radically di erent from anything which the students have experienced before (see the entry for Week 10 in Section 4). It is also a problem that shy people are at a disadvantage. The combined fear of doing mathematical proofs and of performing at the board reportedly scared away many of the female undergraduate students in the department. We discovered this during casual conversations with two of our female undergraduate students at the local student union building. There was only one female student in the course, a very shy graduate student.
5.2.2 Grading Grading can be a very dicult issue when using a pedagogy that is as radically di erent as the Moore method. Without (traditional) homework or exams, one option was to make the grading entirely subjective. We decided that would create too uncertain a situation for the students. Our decision to use a \public" scoreboard resulted in students knowing exactly what they had to do in order to receive the grade they wanted. This is despite the fact that allowing the students to work in groups created an occasional \free rider" problem: a student doing poorly would join a group so that they would get a \board credit". By not putting a limit on the size of the groups, they quickly grew to a size that seemed too large to re ect actual collaborative work. Yet, except for our tally of their trips to the board, we do not have any objective measure of what any individual student learned. As long as the course remains an elective, this is unlikely to pose a problem, but it poses a major impediment to getting the course (or the Moore method) adopted as part of the standard curriculum.
5 CONCLUSIONS
26
5.2.3 Suitability The Moore method is applicable to any technical material which consists of \turning the crank": mastering semi-mechanical techniques that are needed in order to progress to a deeper understanding. Our standard model for this is the integral calculus. In the nal analysis, teaching any subject using the Moore method runs into the diculty of passive direction: \herding cats" is an apt metaphor. This is more a problem with instructors expectations and habits than with the Moore method, however.
5.3 Future Directions There are many others who are teaching derivational programming without the use of the Moore method, e.g., Southwestern University has restructured their entire lower-level curriculum. CCNY teaches inner-city college students program derivation. It would be interesting to compare their experiences with ours. It would be good to target students better, distributing the material in our course across their rst two semesters of computer science. We failed to target any particular audience, accepting everyone from Freshmen to Graduate students. The rst half of the material in our course (predicate and propositional logic) should be integrated into a discrete mathematics course for CS students, with a large dose of handson practice. Gries and Schneider [9] would be an excellent textbook for this approach. The second half of our material (derivational programming) could be used as a supplement to a programming course, preferably at the CS1 level. The key is to have students working on formal materials early, before they are told that they are hard, and to engage their interests directly rather than passively (board work, not lectures). This will not only allow them to learn fundamental material in a lasting way, but will help shape their attitudes toward learning the rest of the computer science curriculum. We also failed to measure the course in any objective way. Our conclusions are based on subjective impressions and on student essays, taken twice during the semester. However, it would be good to teach a similar course with a large enough sample size to be signi cant and to take more scienti c statistics, perhaps using the approach of Almstrum and Wu [2]. Objective measurement is needed for convincing colleagues and accreditation boards.
A STUDENT EVALUATION FORMS
27
A Student Evaluation Forms These are the actual questions which the students answer on their evaluation form. Students were asked to indicate for each of the following statements whether they Strongly Agree, Agree, were Neutral, Disagree, or Strongly Disagreed with each statement. 1. The instructor prepared and organized the course e ectively. (When considering this statement, think about the following areas: were the course goals clear? Did the class sessions, readings, assignments, exams, and other activities seem purposefully and meaningfully selected and sequenced?) 2. The instructor provided adequate opportunity throughout the course for you to be informed of your progress in meeting course goals. (Consider the following questions: Were assignments returned in a timely way? Were the instructor's comments/assessments helpful? Was the instructor available to discuss your progress or assist you in getting additional help if requested?) 3. Overall, the instruction was e ective. (Think about the skills that, in your view, constitute e ective teaching. They might include these: Did the instructor communicate clearly, e ectively? Was she or he enthusiastic, receptive to student questions and comments? Did he or she attempt to make the material relevant, interesting? Was the system of evaluation fair and e ective?) 4. The course concepts and material seemed relevant and meaningful. (Think about the value of the course content. Was the material presented relevant in terms of your personal goals, career plans, or educational needs, or interests? Did the course seem worthwhile?) 5. Based on what I learned in this course, I would recommend this instructor to other students.
B TABLE OF CONTENTS FROM THE TEXTBOOK
B Table of Contents from the Textbook This is the table of contents from Cohen [3], which was the textbook for this course.
0 What can we learn from a cake? 1 Preliminary notions, notations, and terminology 2 Predicates A | Boolean operators 3 Predicates B | Quanti ed expressions 4 Speci cations 5 The shapes of programs 6 Intermezzo on calculations 7 Developing loopless programs 8 Developing loops | an introduction 9 Loops A | On deleting a conjunct 10 Loops B | On replacing constants by fresh variables 11 Mainly on recursion 12 Back to scratch 13 Where to go from here
28
REFERENCES
29
References [1] D.J. Albers, G.L. Alexanderson, and C. Reid. More Mathematical People. Harcourt Brace Jovanovich, 1990. [2] V.L. Almstrum and C.-C. Wu. The CSedRes toolbox: A resource to support research in computer science education. SIGSCE Bulletin, 25(4):21{26, Dec. 1993. [3] E. Cohen. Programming in the 1990's: An introduction to the calculation of programs. Springer-Verlag, 1990. [4] Computer. IEEE Computer Society Press, Sept. 1990. Special Issue on Formal Methods. [5] E. W. Dijkstra. A Discipline of Programming. Prentice-Hall Inc., 1976. [6] E.W. Dijkstra and C.S. Scholten. Predicate Calculus and Program Semantics. SpringerVerlag, 1990. [7] D. Gries. In uences (or lack therof) of formalism in teaching programming and software engineering. In E.W. Dijkstra, editor, Formal Development of Programs and Proofs, chapter 18, pages 229{236. Addison-Wesley, 1990. [8] D. Gries. Teaching calculation and discrimination: A more e ective curriculum. Communications of the ACM, 34(3):45{55, Mar. 1991. [9] D. Gries and F.B. Schneider. A Logical Approach to Discrete Math. Springer-Verlag, 1993. [10] IEEE Software. IEEE Computer Society Press, Sept. 1990. Special Issue on Formal Methods. [11] IEEE Transactions on Software Engineering. IEEE Computer Society Press, Sept. 1990. Special Issue on Formal Methods. [12] A. Kaldewaij. Programming: The Derivation of Algorithms. Prentice-Hall Inc., 1990.
