CUTTING-EDGE RESEARCH BY UNDERGRADUATES ON A SHOESTRING? Katrin Becker Computer Science Education Group Department of Computer Science, University of Calgary 2400 University Drive NW, Calgary, Alberta Canada, T2N 1N4 (403)220-5769 email:
[email protected]
Abstract This paper outlines an approach to helping mid-program undergraduates conduct cutting-edge research that can be incorporated into almost any program with no additional resource requirements. A key feature of this approach is a mechanism that allows for a thorough assessment of students’ work, while still permitting what is typically classified as failure with respect to the production of research results. A brief review of some of the literature along with its benefits and concerns is presented first, followed by an outline of a model for implementing a student-centered research project that can be offered within the context of most traditional courses, at no extra costs in terms of manpower or funds. The focus of this approach is on helping students learn to be researchers, rather than furthering faculty members research agendas. 1. Introduction Knowledge can be categorized into two main groups: explicit knowledge, which includes concepts, facts and theories, and tacit knowledge, which includes those things that allow us to actually do things [1]. The latter has more to do with process, and many agree that the most effective way to acquire this kind of knowledge is, not surprisingly, to actually do things [2-4]. Orey and Nelson [5] elaborate, stating that "learning requires more than just thought and action, or a particular physical or social situation, or just receiving a body of factual knowledge; it also requires participation in the actual practices of the culture" (p. 623). Project work has become a common in many undergraduate programs and it addresses some of these aspects, but few projects involve new work. True research, however does involve new work, and this makes it different from most other project scenarios. One key aspect is the risk of failure. It seems clear that learning to conduct research is valuable for anyone continuing to post-graduate work, but, as most college graduates do not go on in formal studies a valid question is: “Do all students benefit from learning to conduct research?” 2. The Typical Undergraduate Research Model The National Science Foundation actively promotes the involvement of undergraduates in research, and there has been considerable activity in support of this initiative over the last 10 years or so [6]. Through participation in research, undergraduates can attain a higher level of competence in the science, mathematics, engineering, and technology (SME&T) fields, and they will be better able to make informed decisions, communicate and work in teams, and solve complex problems. Various projects have been reported, both with and without NSF funding, involving undergraduates, typically working one on one with individual faculty, outside of a recognized course [7-9]. In other cases, the work is completed within the
context of a (usually) senior project course [9-11], and in a few instances, a researchoriented approach was tried in a freshman class [12-14]. Common concerns include: undergraduates lack the depth and breadth necessary to conduct meaningful research; lack of time for adequate background work necessary to the development of a project, and the loss to faculty who have invested their time into mentoring students when those students do not complete their projects, or do not go on to post-graduate work. Benefits for students have largely focused on helping students become better qualified to get into grad school [10, 15, 16]. Since an increasing number of undergraduates are applying to graduate schools, grades alone are no longer sufficient. Experience related to internship and funded research has also been described [8, 9]. The other primary reported benefit is for faculty, whose own research can be advanced through the use of undergraduates [8, 9, 15, 17]. 3. Should All Undergrads Learn to Do Research? Although the commonly reported benefits are valid and worthy, there are benefits beyond graduate school and faculty research. Amid the pressure to publish, and develop their graduate programs, the benefits to the students beyond the confines of academia are sometimes overlooked. Engagement in authentic, relevant work related to one’s chosen field can go a long way towards making the undergraduate experience more meaningful, and helping to prepare future professionals for a career likely to be filled with change. Further, as more information becomes available on the web, it becomes more and more important for all CS students to be familiar with how to conduct research. Research is not simply about preparation for a future as an academic. It has been generally recognized that research activity increases the likelihood of developing critical, creative, and complex inquiry skills. The integration of research into the teaching process promotes more meaningful understanding, and greater relevance for the learners [18], and research activities promote critical thinking and help learners develop flexible strategies for tackling ill-structured problems. Professional practice in computer science is still very dynamic in form, so it stands to reason that graduates who have experience with inquiry-based approaches to learning will have a distinct advantage, as the philosophy of life-long learning becomes the norm. 4. Addressing Concerns Many models that involve undergraduates in research activities require substantial departmental or institutional support, are labour-intensive, and assume a faculty with diverse and active research portfolios. Many smaller institutions do not have graduate programs, so they lack grad students to act as mentors; their faculty often lack both the time and funding to offer individualized research support, and many institutions whose primary focus is on teaching lack a research infrastructure. These are all things that can discourage faculty from incorporating research activities into their teaching practices. Many have still managed to incorporate research-oriented projects, and these are certainly valuable, but there is a difference between researchstyle projects and cutting-edge research. Often faculty-lead student research projects involve small incremental advances to heavily-researched problems. Undergraduates are rarely encouraged to pursue the “big questions”, as it is felt that they are not adequately equipped. This may
be true, but it is not necessarily a reason for them not to try. Just because a student is naïve, as most novices are, does not mean they are unable to cope with complex or difficult problems. The history of scientific research is not short of examples of individuals who made significant discoveries when, by modern standards, these individuals were seriously lacking in both breadth and depth in their fields. In fact, an enormous advantage that novices bring to research is that they have not yet been encultured to a particular way of thinking or working. This can result in some surprising approaches to unsolved problems. An effect of this is that students may ask questions that a ‘seasoned’ researcher would discard, not because the question is not worth asking, but because they have been trained to work in certain ways, and those ways have so far not been fruitful when applied to the given questions. There are plenty of ‘undiscovered countries’ when it comes to CS research, especially with the explosion of web-based, and wireless applications. Some examples of areas with plenty of room for new work include: non-text-based searches (i.e. finding images, sounds, video without the benefit of text annotations), filtering, security, gaming (both serious and for entertainment), mobile computing, and robotics. Most of these are areas where students already have experience as users, and so will likely have questions, desires, and opinions regarding areas of need. Many students in CS bring a great deal of prior knowledge and skill with them – providing a more freeform approach allows them to draw upon these skills, which is beneficial in connecting their formal educational experience to their actual lives. Most current students are also already familiar with open-source approaches to the development and sharing of knowledge; they are accustomed to being able to contribute to online communities, and are comfortable with the accessibility of blogs, game communities, etc. They should be encouraged to make use of and further develop these skills during their undergraduate careers. 5. Evolution of the Model, 2003-2004 The approach to be described has been developed over five years in a course dealing with data architecture and information storage, manipulation and retrieval. The course is offered in the fall session at the third year level, so students are no longer rank novices, but most are just halfway through their degrees. Course topics include units on file processing, data compression, and file formats, as well as other data structures. The project has been assigned in four different iterations of this course in its current form (Spr/02; Fall/02; Spr/03; Fall/03 1 ). The context provided for the research project is that of an Educational Object Repository, which can hold digital objects of many different and mixed types, and the goal is to examine ways of searching this repository that do not involve performing text-based searches of metadata or other annotations. Choices included examining and analyzing images, sound, and even video to facilitate searches for objects similar to a sample object. In essence, the problem they were trying to solve is: given some image, sound clip, or video clip, propose and test ways of identifying more that are similar to the sample. Within this context, students are given numerous possible scenarios, although they are encouraged to choose and research a question that is meaningful to them. The reason this particular context was chosen is that opportunities for inquiry and new research within this area are both rich and vast, and likely to remain so for many years 1
It was not assigned in 2004 as the instructor was on sabbatical.
to come. It should also be added that this research task was expressly designed to be one for which the students themselves could take ownership, as it is not the author’s research area. This project was about helping the students learn how to be researchers, and not about furthering faculty research agendas. This assignment has several distinguishing features that can be applied in many different research contexts. Almost all of the following are considered necessary in creating a well supported, and student-centered research experience, although the exact details of implementation may vary: 1. “Answers” to potential research questions are not known by anyone – that’s what makes it cutting-edge research. People may be actively working in the area, but there are many unknowns, and students cannot be told if they “are doing it right”, because no one knows. 2. Students are actively encouraged to pursue areas of inquiry for which there are NO known solutions. They are expected to make a reasonable attempt at a literature review, but a lack of prior work does not render the proposal non-viable. A perfectly acceptable result in this assignment is the discovery that their approach does not lead to any resolution. This is the single most important aspect of this entire exercise: it must be made clear to the students that “failure IS an option.” Some of the most exciting submissions have come from students who tried something that did NOT work. Along with environments where students can succeed, we must also provide ‘safe’ environments where they can fail. This scenario provides a framework that allows students the freedom to try, and to fail, and still earn top marks. They are not simply marked for effort, but they are awarded marks for the process, and their ability to reflect upon and articulate what they have learned. 3. The assessment is worth only a portion (30%) of the final course mark, and is distributed so that only 1/3 of the assignment mark is allotted for code and results. In many assignments, this part gets the bulk of the marks. The other two parts here consist of a research proposal worth 1/6 of the total, and a “post mortem” worth fully half. This sends a clear message to students that the process rather than the program is the key component in this project. It also places a firm emphasis on the practice of writing and reporting. 4. Groupwork is permitted, but not required, or even particularly encouraged. Groupwork skills are extremely important in CS, but other courses already address this aspect of professional practice. This project is intended to focus on research processes and skills from the personal perspective of the student, and a deemphasis on groupwork underlies that. 5. The details of the assessment are provided well in advance, so students can use them as guides during their work. Since the details of each project are up to the student, the assessment guides are quite general. 6. Assignment requirements are communicated through three rubrics that address aspects of each part of the process. The marking guide includes such questions as: Does the researcher provide support for his/her knowledge claims? Were there Special Problems encountered? How were they dealt with? Did the researcher critique his/her own work? What was learned from this? How did the project change the way in which you think about your educational situation?
7. Active reflection and self-assessment is required. Students are required to assess themselves, and must fill in the rubrics before submitting their assignments. Honest assessments are encouraged, and an assessment that deviates radically from that assigned by the markers will be discussed with the students involved. 8. The “post mortem” is designed to provide a mechanism for students to reflect on what they have learned from the process and experience, rather than what they produced. They are to describe such things as what went right, what went wrong, and what they would do differently if they were to do it again. The actual results, which would normally constitute the program output or final application in many other assignments, is clearly designated as secondary to the goal of articulating more intangible but personally relevant gains. 9. Students are given a great deal of choice, and those who really don’t want to attempt their own research tasks, may opt to replicate the results of another study or published paper. They are given several papers to choose from, this alternative is not necessarily going to be easier, simply better structured. For many, this is the first time they have read a scholarly paper – and for even more it is the first time they have had to read a paper with the goal of understanding it well enough to replicate it. The experience for students who choose this approach will be different from those doing cutting-edge research, but it is no less valuable. 10. Although course content is a necessary component of the solution, no specific aspects are guaranteed to be common to all – so the instructor must ensure that the required content is covered in other ways beyong this project. 11. Students are actively encouraged to make use of existing applications and tools found locally, or on the web, as well as to share items among classmates, provided proper citations and acknowledgements are observed. They are expected to concentrate on their chosen research task, so any tools that will help them further their research agenda are fair game. 6. Student Reaction and Results Quantitative results proving the advantages of a course like this are virtually impossible to produce, since the skills learned through this process involve the cognitive skills of synthesis and evaluation, described at the start as tacit knowledge. The skills learned through this exercise typically become evident in subsequent tasks, but not on exams. As a result, only qualitative results are presented here. Interestingly, the response from many students was decidedly negative while the project was in progress, as this work forces most out of their comfort zones, yet at the end of the course, many found the experience to be quite valuable, and some have since contacted the instructor (sometimes two years later) to report that what they had learned doing the research project has turned out to be very useful. A number reported that the experience they acquired on a small-scale research project helped them to be better organized and gave them the confidence to tackle a senior project, which they would not have otherwise done. In fact, quite a number have elected to continue the project begun in this course through the senior project course. Some students found this project to be frightening, and future versions will include early feedback to help identify these students so they can be offered more guidance and support, particularly in defining and focusing their projects. However,
not knowing when students are done, or if their answer is right is an unfamiliar feeling for many, but must be recognized as valid. Students need support in this, but they do not need to be prevented from encountering these issues. These outcomes are a common aspect of both academic and professional practice. Enrollment in the senior project course has grown noticeably (unfortunately, enrollment details are unavailable at this time), and some of the work that was produced by students is publishable. One student, for example, wanted to devise a method for classifying music genres. He realized the task was way too big for this project so narrowed it to classifying rock music by locating and identifying drum beats. His early results were very promising. Another invented a 256-bit signature for images based on combinations of RGB values, which she combined with several other measures to come up with a matching algorithm for image searches that easily rivals early Google text searches. Our institution does have a sizeable graduate program so teaching assistants were themselves graduate students actively involved in research activities. Even so, many have commented on how much they themselves learned through this work – it helped them focus their own efforts, and provided some instructional support for the process of research that they had not been given in their other graduate work. 7. Conclusion While students should be afforded plenty of support in attempting a project such as this, they also need to be given opportunities to attempt work that may fail. In our efforts to provide our students with a useful and valuable education, many of us have a tendency to avoid or play down the value of failure. Sometimes a particular solution may already have been tried, but this is not known to the student (or instructor). This can not always be avoided. Discovering that a proposed problem has been solved by someone else is something researchers face from time to time, and so students can learn that it’s not the end. When it comes to conducting research, most students have little experience, so feel very hesitant to take chances – they need encouragement, and to be offered multiple opportunities to practice. This approach gives them one more. In order for it to be a worthwhile learning experience; they need to be convinced that learning is not always about getting the right answer, and the grades need to reflect this philosophy – the end result cannot be the prime target for marks, instead, the process must be the focus. One of the most challenging tasks for the instructor is to find ways of helping students pair down their project so they will have an opportunity to try and produce some results. Given the extreme time pressures (this is a 6-week project), many of the traditional CS research and assignment procedures are simplified or even abandoned so students can concentrate on the work itself. This may include preparing a thorough proposal and literature reviews, creating code that is well documented, and robust, producing well-designed program interfaces, and many other aspects of the process, which are discussed in class, but not emphasized in the project. Almost every topic in CS has its undiscovered countries and under-explored regions – the task is simply to find some, list them, and provide the students a framework within which to work. Finally, this approach can encourage novel approaches and a free exchange of ideas, in other words, it gives students a glimpse of
that which can make research so completely engrossing and satisfying. Sometimes, when we give our students some freedom and trust, they surprise us. Note: All Assignment Outlines and Rubrics, and other support materials are available on request from the author. 8. References [1] J. S. Brown, S. Denning, K. Groh, and L. Prusak, "Storytelling: Passport to the 21st Century," vol. 2004, 2001, [Electronic Source] Retrieved from: http://www.creatingthe21stcentury.org/Intro0table.html, accessed: June 30 2004. [2] R. C. Schank, T. R. Berman, and K. A. Macpherson, "Learning by Doing," in Instructional-design theories and models : vol. 2, a new paradigm of instructional theory. Mahwah, N.J.: Lawrence Erlbaum Associates, 1999, pp. 161-181. [3] J. S. Brown, A. Collins, and P. Duguid, " Situated Learning and the Culture of Learning," Education Researcher, Vol. 18 No. 1, pp. 32-42, 1989. [4] J. Lave and E. Wenger, Situated learning : legitimate peripheral participation. Cambridge [England] ; New York: Cambridge University Press, 1991. [5] M. A. Orey and W. A. Nelson, "Situated Learning and the Limits of Applying the Results of These Data to the Theories of Cognitive Apprenticeships. Access ERIC: FullText," Georgia, 070 Information Analyses; 150 Speeches/Meeting Papers 1994. [6] NSF, "Shaping the Future, Volume II: Perspectives on Undergraduate Education in Science, Mathematics, Engineering, and Technology," National Science Foundation, 1998, [Electronic Source] Retrieved from: http://www.nsf.gov/pubs/1998/nsf98128/contents.pdf, accessed: Mar. 3 2005. [7] A. Gates, P. J. Teller, A. Bernat, N. Delgado, and C. K. Della-Piana, "Meeting the Challenge of Expanding Participation in the Undergraduate Research Experience," Journal of Engineering Education, Vol. 88 No., pp. 409-414, 1999. [8] E. Wenderholm, "Challenges and the elements of success in undergraduate research," presented at 9th annual Conference on Innovation and Technology in Computer Science Education (ITiCSE), Leeds, United Kingdom, 2004. [9] A. Koeller, "Experiences with student research at a primarily undergraduate institution," Journal of Computing Sciences in Colleges, Vol. 20 No. 3, pp. 181 - 187, 2005. [10] K. Ward, "The fifty-four day thesis proposal: first experiences with a research course," Journal of Computing Sciences in Colleges, Vol. 20 No. 2, pp. 94-109, 2004. [11] R. Tashakkori, "Encouraging undergraduate research: a digital image processing approach," Journal of Computing Sciences in Colleges, Vol. 20 No. 3, pp. 173 - 180, 2005. [12] U. Melin and S. Cronholm, "Project oriented student work: learning & examination," presented at 9th annual Conference on Innovation and Technology in Computer Science Education (ITiCSE), Leeds, United Kingdom, 2004. [13] J. P. Bernick, "The Flo-and-Mac Problem: A Tool for Encouraging Undergraduate Research," Inroads - The SIGSCE Bulletin, Vol. 35 No. 2, pp. 103-106, 2003. [14] D. Brent, D. Seneker, R. Reimer, and L. Gaudet, "Using an Academic Content Seminar to Engage Students with the Culture of Academic Research," in submitted to: Journal of the First-Year Experience and Students in Transition, vol. 2005, 2004, [Electronic Source] Retrieved from: http://www.ucalgary.ca/~dabrent/art/FYSpaper2.html, accessed: Mar. 2 2005. [15] R. A. Madler, "Genesis of an undergraduate research experience," presented at Frontiers in Education Conference FIE '98. 28th Annual, Tempe, AZ , USA, 1998. [16] C. Leggett, "The Benefits of Undergraduate Research," in Minority Scientists Network, vol. 2005, A. A. f. t. A. o. Science, Ed., 2003, [Electronic Source] Retrieved from: http://nextwave.sciencemag.org/cgi/content/full/2003/05/21/5?ck=nck, accessed: Mar. 1 2005. [17] T. Greening and J. Kay, "Undergraduate research experience in computer science education," presented at 7th annual Conference on Innovation and Technology in Computer Science Education (ITiCSE), Aarhus, Denmark, 2002. [18] H. Kanuka and D. R. Garrison, "Making a Teaching - Research Connect: The Critical Element to Facilitating Effective Inquiry Approaches," Learning Commons, University of Calgary, 2003, [Electronic Source] Retrieved from: http://commons.ucalgary.ca/documents/TeachingResearchConnect.pdf, accessed: Jan 2, 2005.
