Emulating human tutor empathy S. T. V. ALEXANDER Institute of Information & Mathematical Sciences Massey University at Albany, Auckland, New Zealand † An important factor in the success of human one-to-one tutoring is the tutor’s ability to empathise with students by understanding their non-verbal behaviour. However, current Intelligent Tutoring Systems (ITSs) model students solely on their answers to questions, which makes it impossible to adapt to the affective state information carried by the non-verbal behaviour of students. The aim of this paper is twofold: firstly to present a plan for a new ITS that will adapt to non-verbal behaviour through an animated pedagogical agent, and secondly to outline a study of human tutor empathy that will identify how this agent ought to adapt.
1 Introduction Human one-on-one tutoring has always been, without a doubt, an exceptionally effective form of instruction. Compared to classroom instruction, the effectiveness of one-on-one tutoring really is striking – it is well established that students tutored one-on-one score an impressive two standard deviations higher than students receiving classroom instruction alone (Bloom, 1984; Cohen, Kulik, & Kulik, 1982). What is it that makes human one-on-one tutoring so much more effective than classroom instruction? The answer is at once straight-forward and obvious: that good tutors can adapt their tuition to precisely meet the needs of individual students, whereas classroom teachers are forced to cater as best they can to the varied needs of a larger group (Rosé & VanLehn, 2003). The exciting potential of Intelligent Tutoring Systems (ITSs) is founded on their ability to emulate, or even surpass, this fundamental capability of human tutors – to adapt in the best way possible to meet the needs of individual students. As yet, this potential remains largely unrealised; the first aim of this paper is to present a plan for a new affect-sensitive ITS that may help to close the gap between humans tutors and their artificial cousins. The second aim of this paper is to outline a study of human tutor empathy that will provide a basis for the affect-sensitive component of this new ITS.
†
[email protected]
Emulating human tutor empathy
27
2 The importance of non-verbal behaviour ITSs are so called for their ‘intelligent’ ability to adapt to the needs of individual students (Urban-Lurain, 2003), and indeed, many ITSs have proved to be effective (e.g. Anderson, Corbett, Koedinger & Pelletier, 1995; Conati & VanLehn, 2001; Mitrovic, Martin, & Mayo, 2002). Traditionally, ITS research has assumed that students are modelled according to their answers to questions. These models represent different aspects of the student’s cognitive state. However, many researchers now agree that to restrict student modelling to simply interpreting answers is to overlook one of the human tutor’s greatest allies, an appreciation of the student’s non-verbal behaviour (e.g. Picard, 1997; Kort, Reilly, & Picard, 2001). Such is the nature of human communication, that tutors unconsciously process a continuous stream of rich non-verbal information that can suggest improved tutoring strategies. Competent human tutors adapt their tutoring according to the real time nonverbal behaviour of their students, as well as their answers to questions. Since adapting to the non-verbal behaviour of students is key to the success of human tutoring, it follows that perhaps ITSs could be significantly improved if they too could recognise and adapt to the affective information carried largely by the non-verbal behaviour of students. This conclusion has spawned a developing field of artificial intelligence: Affective Tutoring Systems. Affective Tutoring Systems adapt to the affective state of students just as effective human tutors do (de Vicente, 2003; Alexander, Sarrafzadeh, & Fan, 2003). Or at least, they will as soon as someone develops one – the field of Affective Tutoring Systems is sufficiently young that not a single Affective Tutoring System has actually been implemented. Of those who plan to, Kort et al. (2001) propose to build a Learning Companion, which will initially use eye-gaze as an input of affective state. Litman and Forbes (2003) propose an ITS that adapts to the emotions revealed by the acoustic and prosodic elements of a student’s speech.
3 Proposal: an Affective Tutoring System for maths 3.1 Domain The aim of the current research is to develop an Affective Tutoring System to help 8 to 9 year old students understand the concept of part-whole addition (New Zealand Ministry of Education, 2003). The tutoring system will be based on an existing exercise developed by the New Zealand Numeracy Project that encourages students to add numbers by transforming the initial equation to make the first addend up to the next 10. A simple example of the desired reasoning is given in Figure 1. Students will learn
28
S. T. V. Alexander
this reasoning in the tutoring system by manipulating tens frames and counters, as shown in Figure 2.
Question: 16 + 7 = Solution: 20 + (7 – 4) = 20 + 3 = 23
Figure 1: An example of part-whole reasoning in the Numeracy Project exercise.
Figure 2: Tens frames and counters in the Affective Tutoring System for maths 3.2 Animated pedagogical agents The system will adapt to the student through an animated pedagogical agent. Animated pedagogical agents are “lifelike autonomous characters that cohabit learning environments with students to create rich, face-to-face learning interactions” (Johnson, Rickel, & Lester, 2000). Animated agents carry a persona effect that has been shown to increase learner motivation, especially in technical domains, although its overall benefits remain unclear (van Mulken, Andr, & Muller, 1998). 3.3 Affective component The affective component of the tutoring system will be based on input from automated facial expression and gesture analysis systems, currently being developed in-house (Fan, Johnson, Messom, & Sarrafzadeh, 2003). These systems identify discernable expressions in images taken in real time by a web-cam mounted on the student’s monitor. This means that the tutoring system will be able to “see” the expressions of students, and thus to adapt
Emulating human tutor empathy
29
accordingly through the animated agent. How the system will adapt is briefly discussed in Section 4 of this paper. 3.4 Benefits of the system Therefore the student model of the Affective Tutoring System will maintain affective state information derived from the student’s non-verbal behaviour, as well as keep track of how quickly and accurately the student answers questions. The animated agent will adapt in two ways: - by presenting the most appropriate material, and giving the most appropriate help, and - by empathising with the student, and providing verbal and nonverbal encouragement. The agent will be able to adapt nonverbally through its own facial expressions and gestures. Therefore the agent will be potentially beneficial in at least two ways: - if the tutoring material most appropriate to the student’s affective state is presented, then this should facilitate learning, and - if the agent is believably sincere in its empathy, then this could amplify the motivational benefits of the persona effect. Increasing the student’s motivation to use the system should result in an increase in learning.
4 A study of human tutor empathy 4.1 Rationale for the study The proposed Affective Tutoring System will have the real time affective state of the student as an input, as detected by automated facial expression and gesture analysis systems. However, it is clearly essential to know what to do with this information before the animated agent can adapt its tutoring in a genuinely useful and believable manner. Therefore, it is proposed to study how human tutors adapt their tutoring based on the affective state (as betrayed by the non-verbal behaviour) of students. The ways in which human tutors adapt their tutoring based on their empathy with students can then be used as a platform on which to build the affect-sensitive component of the new ITS. 4.2 Methodology Several professional tutors will be videoed while tutoring participants from a local primary school one-on-one. About ten participants will be tutored on the same Numeracy Project exercise that will be used in the Affective Tutoring System, and was briefly outlined in this paper in Section 3.1. The
30
S. T. V. Alexander
participants will be 8 to 9 year old students at a local primary school, and will be selected solely on the basis of their maths ability. The participants should be at a high enough level so that the exercise is achievable, but not so high that they encounter no difficulties at all. A coding scheme will be developed based on similar work by Person and Graesser (2003). This scheme will be used to extract data from each video to describe: - the facial expressions and gestures of the student, - the affective state of the student, - the reactive facial expressions and gestures of the tutor to this state, and - the reactive tutoring adaptations of the tutor. It is hoped that several rules of human tutor empathy may be generalized from the data, and thus incorporated in the affect-sensitive component of the Affective Tutoring System.
5 Future work The next step in the current research is to undertake the study of human tutor empathy as outlined in Section 4. The results of this study will form the basis for the empathetic qualities of the proposed animated pedagogical agent. Given this foundation, the animated agent itself may be implemented. Colleagues at Massey University are already working on the automated facial expression and gesture analysis systems that will provide an input of the student’s non-verbal behaviour, as well as a non-affective ITS of the Numeracy Project exercise. What will need implementing is the affectsensitive component of the tutoring system, and the animation of the agent itself. Once the Affective Tutoring System is complete, it will be tested to measure its effectiveness. It will be tested against a non-affective, no agent version of the tutoring system, and a non-affective, but animated agent version of the system. This will illustrate both the impact of an affective component on the efficacy of tutoring systems, and the impact of an animated pedagogical agent in a children’s ITS for maths.
References Alexander, S.T.V., Sarrafzadeh, A., & Fan, C. (2003). Pay Attention! The Computer is Watching: Affective Tutoring Systems. Proceedings of ELearn 2003, Phoenix, Arizona.
Emulating human tutor empathy
31
Anderson, J.R., Corbett, A.T., Koedinger, K.R., & Pelletier, R. (1995). Cognitive Tutors: Lessons Learned. Journal of the Learning Sciences, 4(2):167-207. Bloom, B.S. (1984). The 2-sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6):4-16. Cohen, P.A., Kulik, J.A., & Kulik, C.C. (1982). Educational outcomes of tutoring: A meta-analysis of findings. American Educational Research Journal, 19:237-248. Conati, C., & VanLehn, K. (2001). Providing adaptive support to the understanding of instructional material. Proceedings Intelligent User Interfaces, Santa Fe, New Mexico. Fan, C., Johnson, M., Messom, C., & Sarrafzadeh, A. (2003). Machine Vision for an Intelligent Tutor. Proceedings of the International Conference on Computational Intelligence, Robotics and Autonomous Systems, Singapore. Johnson, W.L., Rickel, J.W., & Lester, J.C (2000). Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments. International Journal of Artificial Intelligence in Education, 11:47-78. Kort, B., Reilly, R., & Picard, R.W. (2001). An Affective Model of Interplay Between Emotions and Learning: Reengineering Educational Pedagogy - Building a Learning Companion. IEEE International Conference on Advanced Learning Technologies. Litman, D., & Forbes, K. (2003). Recognizing Emotions from Student Speech in Tutoring Dialogues. Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), St. Thomas, Virgin Islands. Mitrovic, A., Martin, B., & Mayo, M. (2002). Using evaluation to shape ITS design: results and experiences with sql-tutor. International Journal of User Modeling and User-Adapted Interaction, 12(2-3):243-379. van Mulken, S., Andr, E., & Muller, J. (1998). The persona effect: How substantial is it? Proceedings of Human Computer Interaction, Berlin, Germany. New Zealand Ministry of Education, (2003). Book 1, The Number Framework. Numeracy Professional Development Projects. Wellington: Ministry of Education.
32
S. T. V. Alexander
Person, N.K., Graesser, A.C., & The Tutoring Research Group (2003). Fourteen facts about human tutoring: Food for thought for ITS developers. AI-ED 2003 Workshop Proceedings on Tutorial Dialogue Systems: With a View Toward the Classroom, Sydney, Australia. Picard, R.W. (1997). Affective Computing. Cambridge, Mass., MIT Press. Rosé, C.P., & VanLehn, K. (2003). Is Human Tutoring Always More Effective than Reading?: Implications for Tutorial Dialogue Systems. Supplementary Proceedings of Artificial Intelligence in Education, Sydney. Urban-Lurain, M. (2003). An Historic Review in the Context of the Development of Artificial Intelligence and Educational Psychology. Retrieved June 24, 2003 from http://www.cse.msu.edu/rgroups/cse101/ITS/its.htm. de Vicente, A. (2003) Towards tutoring systems that detect students' motivation: an investigation. Ph.D. Thesis, School of Informatics, University of Edinburgh, UK.
