Virtual Environments Supporting Learning and Communication in ...

LWW/TLD

LWWJ350-02

July 24, 2007

6:14

Char Count= 0

Top Lang Disorders Vol. 27, No. 3, pp. 211–225 c 2007 Wolters Kluwer Health | Lippincott Williams & Wilkins Copyright

Virtual Environments Supporting Learning and Communication in Special Needs Education Sue V. G. Cobb, PhD Virtual reality (VR) describes a set of technologies that allow users to explore and experience 3-dimensional computer-generated “worlds” or “environments.” These virtual environments can contain representations of real or imaginary objects on a small or large scale (from modeling of molecular structures to buildings, streets, and scenery of a virtual city). Potential use in education was first considered in the early 1990s, when it was suggested that VR technology could provide powerful learning environments not available through other means (M. Bricken, 1991). A growing VR research community has since sought to examine the benefits of using this technology in mainstream and special needs education, as well as other learning environments. This article first presents a brief description of VR technology and its use in teaching and learning. VR can be used as a tool for communication, as a medium through which individuals express ideas, and for learning about communication, through guided training and rehearsal in simulated social scenarios. This article concentrates on the latter application and presents 4 research projects conducted by the Virtual Reality Applications Research Team and associated colleagues in special needs education since 1991. These projects involved the development and evaluation of virtual environment applications intended to support training for individuals with learning needs and communication difficulties in preparation for more independence in their everyday activities and communications in the community. Key words: communication, social skills, special needs education, task-based learning, virtual environments, virtual reality

VIRTUAL ENVIRONMENTS Virtual reality (VR) refers to a set of computing technologies used to create, and allow users to experience, three-dimensional

From the University of Nottingham, Nottingham, United Kingdom. This article is based on a series of research studies conducted between 1991 and 2003 by members of the Virtual Reality Applications Research Team (VIRART) and collaborating partners. The author thanks all past and current members of VIRART, collaborating partners, professionals, parents, students, and end-users who contributed to the projects described in this article. Corresponding author: Sue V. G. Cobb, PhD, VIRART, Human Factors Research Group, School M3, University of Nottingham, Nottingham, NG7 2RD, United Kingdom (e-mail: [email protected]).

(3D) simulated digital or “virtual” environments (VEs). A VE is initially empty 3D computer space that then is populated with 3D objects that are given behavioral characteristics replicating some aspect of real-world behavior (e.g., a doorbell chimes, a door handle turns, a door opens). A 3D navigation controller, such as a joystick, is used to move around the virtual world, and a cursor controller, such as a standard computer mouse, is used to interact with virtual objects. Interaction will cause a response in the VE in accordance with programmed object behaviors. To explore a virtual house, for example, a user may move toward a door by pushing forward on the joystick controller. Placing the mouse cursor over the door handle, she can click and drag to turn the handle (the door opens), and then use the joystick to move through the door into another room. 211

LWW/TLD

LWWJ350-02

212

July 24, 2007

6:14

Char Count= 0

TOPICS IN LANGUAGE DISORDERS/JULY–SEPTEMBER 2007

These VEs are distinct from traditional computer programs in several ways; the digital environment is 3D rather than 2D, users can move around in the digital environment in any direction they choose rather than following a preset route, users may use a special display to become “immersed” in the digital environment “as if they were really inside it.” Features of VEs useful to education include the following (Cobb, Neale, Crosier, & Wilson, 2002): • Representation of objects and environments: These may be representations of real things that exist in the real world or things that do not exist or cannot ordinarily be seen (such as molecules or particles). • Different viewpoints: Users can take different perspectives and travel around the VE appropriately (e.g., as a wheelchair user, eye-height would be at a fixed seated level and movement control restricted to forward/back with limited rotation. As a helicopter pilot, the viewpoint would be able to “fly” around the VE using the full 6 df of movement). • Reality and super-reality: VEs allow behaviors not normally possible in real time (e.g., to “fly”inside a machine to view the inner workings to gain an understanding of how it works). VEs may be explored via a standard desktop computer or projected screen (similar to standard computer use), or via immersive display devices. Using these latter devices, users wear a head-mounted display system so that they see only the VE. When they move their head from side to side, it is as if they are looking around inside the VE. VEs may be designed for single-user interaction (SVEs), in which a single-user controls navigation and interaction with the objects in the VE. In collaborative virtual environments (CVEs), several users share the same VE and each user is represented by an avatar: a virtual character that can be seen by other users. Users may communicate with each other via audio headphones with microphones and collaboratively interact with virtual objects.

VIRTUAL REALITY IN EDUCATION VR was first discussed as a possible medium for mainstream teaching in the early 1990s when it was suggested that characteristics of VR learning environments, specifically the facility to move around a simulated environment and explore conceptual information in an intuitive manner, could offer powerful learning experiences not available through other means (Bricken, 1991). Numerous studies throughout the 1990s considered the utility of VR technology in mainstream and special needs education and were reported in a variety of publications including a new journal, VR in the Schools (Auld & Pantelidis, 1995), the Virtual Reality in Education and Training conference (VRET, 1997), a special issue of the VR journal Presence: Teleoperators and Virtual Environments (1999), as well as other education journals. Reported projects examined the use of VR for teaching mainstream curriculum subjects including mathematics (Bricken & Winn, 1992), science (Byrne & Furness, 1994; Dede, Salzman, & Bowen Loftin, 1996; Salzman, Dede, Bowen Loftin, & Chen, 1999), and engineering (Bell & Fogler, 1995). The first substantial review of educational uses of VR technology was reported in 1998. Citing more than 50 studies, Youngblut (1998) concluded that there was some evidence of improvements in learning from using VR. In the ScienceSpace project, for example, a series of VEs was constructed for teaching science relating to physics (Newton World), electrical fields (Maxwell World), and chemistry (Pauling World). Subject knowledge tests applied to 18 students studying high school physics found that learning in 3D immersive Maxwell World was significantly better than with traditional teaching methods (Dede et al., 1996; Salzman et al., 1999). A more recent review of VEs used in mainstream education (Moshell & Hughes, 2002) defined different ways in which VEs may support learning; constructivist learning may be supported via student self-directed exploration of a VE (e.g., the Virtual RadLab radioactivity laboratory

LWW/TLD

LWWJ350-02

July 24, 2007

6:14

Char Count= 0

Virtual Environments for Learning and Communication for testing the shielding properties of different materials with a range of radioactive sources; Crosier, Cobb, & Wilson, 2000), constructionist learning may be supported when students work together to create their own virtual models (e.g., construction of the Solar System to illustrate how eclipses occur; Barab et al., 2000), and situated learning may be supported by encouraging interactive role-play in simulated scenarios (e.g., the ExploreNet 2D collaborative improvisational drama simulation; Moshell & Hughes, 1996). The first conference on virtual reality and disability (Murphy, 1993) identified potential application of the technology for improving accessibility, mobility, learning, rehabilitation, and assessment. Many of these have successfully been demonstrated at the now established international conference series on disability, virtual reality, and associated technologies (ECDVRAT, 1996). An extensive range of assistive devices and display systems has been developed to allow access to multisensory digital media used for a variety of applications including assessment and rehabilitation, behavior therapy and phobia treatment, and training and education for individuals with disabilities, and there is growing evidence of successful application of VEs and VR technologies in these areas (Cobb & Sharkey, 2007). A review of VEs used in training and education of people with intellectual disabilities highlights specific advantages of VEs in providing a safe place for people to learn about and practice skills, behaviors and activities that they may need to do in the real world, such as preparation for a courtroom appearance (Standen & Brown, 2005). Transfer of training has been found following the use of VEs for vocational training of students with intellectual disabilities in kitchen skills (Rose, Brooks, & Attree, 2000) and workplace tasks in sheltered factories (Mendozzi et al., 2000). VIRART PROJECTS The Virtual Reality Applications Research Team (VIRART) was established at the University of Nottingham, UK, in 1991 to explore

213

and develop applications of VR technology. A specific area of research interest was the use of VR technology for education and training. Demonstrator projects have been developed for industrial training, rehabilitation and mainstream, and special needs education (VIRART, 2007). This article presents an overview of four projects aimed at supporting learning and communication skills for children and adults with special educational needs (SEN).1 One of the challenges in design of VEs for SEN was that there were no guidelines available to inform content and interface design, and very little understanding of how individuals would interpret and react to VEs. User-centered design methods were applied and evolved to ensure that the VEs developed were appropriate for the intended users and teaching professionals. Throughout this period of application development and exploration, our research team has addressed a number of research aims including the following: -- Design and development of VEs to meet user requirements for special populations -- User understanding of the technology and what the VE represents -- Observation of user interaction with VE -- Evidence of learning gain or other benefit and transfer of learning Table 1 presents a summary of the selected projects, listing the VEs developed, main research questions, and outcomes. Detailed descriptions of these projects have been published previously, and readers are invited to refer to cited papers in Table 1 for further information. The first three projects were conducted in partnership with the Shepherd School, a UK specialist school for children with severe and profound learning disabilities. The last project involved the Rosehill School and Sutherland House for children

1 In the context of this article SEN includes severe intellec-

tual or cognitive disability. Readers should be aware that, in this article and others describing this work, the term severe learning difficulties (SLD) also refers to individuals with intellectual or cognitive disabilities.

Publications

Rutten et al. (2003) Parsons and Mitchell (2002) Parsons, Mitchell, and Leonard (2004) Parsons, Mitchell, and Leonard (2005)

Can students with ASD interact with VEs? Do students with ASD understand VEs as representations of social scenarios? Is there any evidence of social skills learning? Is there any evidence of skills transfer between contexts?

Parsons et al. (2000) Cobb et al. (2002) Neale et al. (2002) Kerr, Neale, and Cobb (2002)

6:14

Social skills training for adolescents with autistic spectrum disorder (ASD) (2000–03) Single-user VEs Virtual bus Virtual cafe´ Collaborative VEs Social cafe´ Interview room

July 24, 2007

Meakin et al. (1998) What life skills do students need to Brown, Kerr, and Bayon (1998) learn? Cobb, Neale, and Reynolds (1998) How effective is the VE in terms of Neale, Brown, Cobb, and Wilson (1999) Usability Brown, Neale, Cobb, and Reynolds Enjoyment (1999) Skill learning Neale, Cobb, and Wilson (2000) Skills transfer

Some children recognized objects, whereas others had difficulty Children copied virtual signer and repeated words Peer tutoring occurred Interface design and need for reward system Usability issues raised Students found it easier to use VEs than mentors believed they would Control action and decision-making shift from teacher-initiated to student-initiated Anecdotal reports of benefit to performance Student feedback on user-centered design method positive Usability issues raised Students with ASD learned to use VEs quickly Interpreted the VE nonliterally Recognized basic differences between VEs and videoed scenes Students learned appropriate responses within context Transfer of learning demonstrated between media (VE to video) within context No transfer of learning between contexts

Can we successfully bring the outside world into the school classroom? How will this support student learning?

LWWJ350-02

Life skills education (1998–99) Virtual house Virtual transport Virtual cafe´ Virtual supermarket

Children could interact with VE Children displayed enjoyment Teacher reaction positive Usability issues raised

Outcomes

Can children with learning and multiple disabilities control interaction within a virtual environment?

Research questions

214

Experiential VEs for children with Brown, Kerr, and Wilson (1997) special needs (1993–95) Virtual car driving Virtual ski slope Virtual house Virtual supermarket Virtual environments for teaching sign Brown et al. (1997) language and Makaton symbols Brown, Cobb, and Eastgate (1995) (1993) Makaton themed “warehouses”

Project and VEs

Table 1. Summary of VIRART projects exploring virtual environments (VEs) for learning and communication

LWW/TLD Char Count= 0


LWW/TLD

LWWJ350-02

July 24, 2007

6:14

Char Count= 0

Virtual Environments for Learning and Communication

Figure 1. Virtual skiing.

with autistic spectrum disorder (ASD) and the UK’s National Autistic Society (NAS). Summary and outcomes of the projects are discussed below. Experiential VEs for children with special needs At the outset, teachers suggested that one of the most appealing features of VR was that it could be used to bring the outside world into the classroom. It was considered that a VE could model places and activities that students with severe learning difficulties (SLD) may not have access to perhaps in their lifetime and so the VE could provide some experiences that would normally be beyond the reach of these students. A set of experiential VEs was developed that was intended to allow the user to control movement within VE. These included a virtual ski slope (Figure 1) and a virtual city (Figure 2). The intended purpose of these VEs was to provide students

Figure 2. Virtual driving.

215

Figure 3. Student and facilitator.

with the opportunity to control the environment and to explore features with which they could interact. One of the primary research aims was determining the suitability and ease of use of the input devices. Observation of students using alternative control devices concluded that specialist VR-control devices were not suitable for students with SLD as they were too sensitive and difficult to control (Hall, 1993). A further study observed six students, using standard ICT control devices (joystick, mouse, and keyboard), and found that mouse control of navigation was extremely frustrating for students with SLD. Standard joystick controllers were found to be much more suitable for controlling movement around the VE (Crosier, 1996). Observation of how these early VEs were used in the classroom revealed that students did not use the VE on their own, but were supported by a facilitator or teacher seated alongside them (Figure 3). Observation of student– teacher interactions, when using the experiential VE, found that individual characteristics had more of an effect on user behavior than did VE design, with individual students’ abilities influencing their awareness of position in the VE, whether they needed instruction for the next task, and their contribution to collaboration (Neale, 1997). Conclusions drawn from these early studies were that students with SLD appeared to enjoy having control over the VE and could successfully interact with it but required support from a colocated

LWW/TLD

LWWJ350-02

216

July 24, 2007

6:14

Char Count= 0


facilitator to guide them through interactive tasks (Neale et al., 1999). VEs for teaching sign language and Makaton symbols Development of the experiential VEs had demonstrated the feasibility of using VR to “bring the outside world into the classroom.” Teachers considered that a useful application of this finding would be to support teaching and learning of the Makaton language of symbols and signs used for communication throughout the school. Makaton comprises a library of 350 picture-based icons (Grove & Walker, 1990) and teaching of Makaton is traditionally achieved by association— showing an object, stating the name of the object, showing the Makaton symbol representing the object, and then performing the sign. One of the problems highlighted was how to teach associations for objects that cannot be brought into the classroom (e.g., vehicles, buildings). Traditionally, teachers overcame this by using representations of real items such as toys or photographs, but they expressed concern that students may not understand the function of objects if they could not explore and interact with them. It seemed sensible to utilize VR to accomplish the task, allow students to explore a VE, and gain an understanding of object function by interacting with virtual objects. The Makaton project was constructed using a “split screen” interface design comprising on one side an explorable VE containing a variety of virtual objects and, on the other, a permanent representation of the Makaton symbol for the object of interest and an animated virtual signer. Figure 4 shows the “boat warehouse”—one of the four components in the transport VE. The student could move around the VE to view the 3D objects from different angles and could also activate interactive features (e.g., the boat sails around the VE). Placing the cursor in the box containing the Makaton symbol triggered an audio file stating the word represented (in this case “boat”). If the cursor was placed in the box containing the manikin, then the virtual

Figure 4. VE, Makaton symbol, and signer representing “boat.”

signer would perform the appropriate sign for that word. Thus, the student could activate the signer and listen to the word as many times as necessary. Evaluation of the Makaton project took different forms. Multiple activity analysis examined interaction behavior between student– teacher pairs and found that, for this application, there was not much need for teachers to guide interaction (Neale et al., 1999). Although some students found it difficult to use the mouse to control the cursor, most demonstrated that they knew what to do. Spontaneous peer tutoring also was observed. A teacher commented that she had seen “an older student guiding a younger and less able student through a Makaton environment, helping her to form the hand sign and say the word associated with each symbol. The advantages for both are manifold”(Brown, 1994, p. 7). A later experimental study conducted independently found that, with practice, students took more control over their interaction with the VE and that their Makaton vocabulary increased (Standen & Low, 1996). Although the interface itself was easier to use than in the previous project, possibly because the activities in this VE were more structured, usability issues were identified particularly relating to the system feedback to students. This was resolved by introducing a positive reward system for correct choices and

LWW/TLD

LWWJ350-02

July 24, 2007

6:14

Char Count= 0


217

encouragement to try again when incorrect choices were selected. VEs for learning independent living skills The practice element of exploration and interaction in a virtual world was regarded as a potentially useful mechanism for students to learn how to engage in activities for themselves. The Life Skills Education project was a community-based research project intended for students with SLD aged 16+ to learn about and practice skills needed for independent living (Brown et al., 1999). A virtual city was constructed comprising four main elements: a virtual house, virtual supermarket, and virtual cafe´ linked together by a virtual transport system. Students were given a task scenario that required them to make decisions and perform tasks in the virtual world, including making a shopping list and planning a bus journey to the supermarket to buy food. At the bus stop, they needed to compare the name on the front of the bus with that displayed in the information bar at the bottom of the screen to check that they were getting on the correct bus (Figure 5). In the virtual supermarket (Figure 6), the viewpoint was fixed so that the students could move around the supermarket pushing the shopping trolley in front of them. A split-screen interface was used from which the shopping list could be selected to full-screen view allowing the users to check what items they needed to find (Figure 7).

Figure 5. At the bus stop.

Figure 6. Virtual Supermarket.

A user-centered design methodology involved users directly in design decisions concerning content and the VEs (e.g., the choice of learning scenarios) and evaluation of the VE interface (Meakin et al., 1998). Teachers and training professionals informed the structure and presentation of tasks embedded within the learning scenarios. Evaluation studies examined usability, enjoyment, demonstration of skill learning, and transfer of learning to real-world activities. Observational analysis, questionnaires, and interviews were used to assess usability and enjoyment. An experimental study was conducted in which 14 student–teacher pairs were observed in task performance in five weekly sessions using the VE compared with pre- and post-VE task performance in the real

Figure 7. Virtual shopping list.

LWW/TLD

LWWJ350-02

218

July 24, 2007

6:14

Char Count= 0


world (Cobb, Neale, & Reynolds, 1998). For each task component the level of support provided by the teacher was recorded. This ranged from “no support given,”through “verbal,” “visual,” and “physical prompts,” to “the teacher does the task.”The importance of this measure was that it allowed a comparison between the support given in the VE and support given to carry out the same task in the real world. It was important to ensure that the training tool used was not more difficult to use than to carry out the task in the real world. If this were the case, it would invalidate the use of the VE. The results showed that the students found the programs accessible and enjoyable. One interesting result was that students with SLD performed much better at VE interaction than their support workers had predicted (Cobb et al., 1998). This result might reflect a generation gap between students and their teachers with regard to acceptance of new technologies, highlighting the importance of user involvement in design and evaluation. Positive feedback was received with support workers commenting upon improved confidence and enthusiasm for real-world activities after they had used the VE for training. Students reported that the VE had helped them make decisions and know what to do in different situations. These anecdotal reports suggest potential for VE training to transfer into real-world behavior. Observation of student behavior, while using the VE, led to the identification of three types of usability problems: understanding task instructions, moving around the VE, and interacting with virtual objects (Neale et al., 2000). For example, one of the tasks in the virtual cafe´ was to find a table to sit at. To successfully achieve this task, students needed to understand that they had to look for an empty seat at one of the tables, use the joystick to move their viewpoint to the table, and then use the mouse to select the seat that they wanted to sit on. Classification of problems experienced helped identify issues with the design of the VE, and solutions to usability problems were implemented to ensure wider

accessibility. For example, many users experienced problems reading texts when creating their shopping list, reading menus, and when text overlays were presented to them. This was resolved by replacing text menus with pictures of the items on the shopping list (as seen in Figure 7), making the search task easier for students as they could directly match the picture with objects on the shelves. VEs for social skills training The Life Skills Education project had demonstrated that VEs could successfully be used for learning of procedural tasks and that students with SLD could transfer learning to real-world behavior (Cobb et al., 1998). The Asperger’s Syndrome (AS) Interactive project examined the suitability of VR technology and VEs to support learning of social communication skills for teenagers and adults with ASD (Cobb et al., 2002). This was another community-based project with partners from the UK National Autistic Society (NAS), autism training professionals, an autism specialist, and other special needs schools in Nottinghamshire, UK. The primary characteristics of ASD include inappropriate use of language, overly literal interpretation of words and phrases, and limited understanding of social norms and expectations leading to inappropriate behavior in social contexts (Wing, 1996). It has been suggested that VEs might be an ideal medium through which individuals with ASD can learn behavioral skills (Clancy, 1996) and early studies indicate successful interpretation of VE simulated scenarios (Strickland, 1996). The rationale behind the AS Interactive project was that, if social scenarios could successfully be replicated within VEs, the limited personal interaction afforded by the computer interface would be inherently more attractive to children with ASD and therefore provide a safe and supportive environment for learning (Parsons et al., 2000). In addition to the use of VEs for individual student training using single-user VEs (SVEs), as had been used in the previous projects, it was considered that this project would benefit from the use of CVEs,

LWW/TLD

LWWJ350-02

July 24, 2007

6:14

Char Count= 0


219

Figure 8. Virtual cafe´.

Figure 9. Choosing what to say.

allowing several users to share the same VE. One of the objectives of the AS Interactive project was to examine the feasibility of using CVE technology for social skills training and interaction in ASD schools. Two scenarios were chosen representing typical social situations that would be familiar to most users (a cafe´ and a bus). The objective was to support social interaction behavior specific to two tasks: queuing (i.e., taking one’s place in line) and finding somewhere to sit. This required students to control movement of their virtual character (“avatar”) through the VE, to respond appropriately to other avatars, to make decisions about when they should communicate with others in the VE, and decide what they should say. Again, a split screen interface was used, this time with the VE in the central part of the screen. The difficulty level for the VE was selected from tabs at the top of the screen, and instructions, prompts, and feedback presented on the information bar at the bottom of the screen. Figure 8 shows the virtual cafe´. Depending upon the VE level selected, there were empty tables (providing the option to sit down at a free table) or empty seats only (so that it would be necessary to sit down at a table with other people). To sit down at an empty table, the student had to navigate to the table and click a chair to sit on it. However, it was not permissible to sit at an occupied table without asking a question to the avatars seated at the table. A choice of questions was presented (Figure 9), and if the student selected an ap-

propriate question, then he or she could complete the task. The SVE was designed to support learning with regard to asking appropriate questions and interpretation of responses. For example, a user might approach a table with three avatars seated and one empty seat and ask whether she could sit down. Sometimes the response would be “yes”but at other times she would be told “No, that seat is taken.” Classroom observation found that teachers were using this program for group teaching sessions, using a projection screen so that up to six students could view the SVE while one student took control over navigation and interaction with the VE (Neale, Cobb, Kerr, & Leonard, 2002). Teachers commented that the VE provided a visual tool that could be used to support teacher-led discussion with the students about what to do in social situations. In the example above, an empty seat was described as “taken.”Using a “pause”button to “freeze”the program, leaving the visual scene on the screen, the teacher could discuss with students what this response might mean. At other points of the program, this facility was used to stop the program and ask students what they thought that characters in the VE might be thinking or what they thought would happen next. It was expected that CVEs may offer a different type of learning support than the SVE due to the facility for interaction with other people participating in the shared VE. Thus, this type of interaction would be closer to

LWW/TLD

LWWJ350-02

220

July 24, 2007

6:14

Char Count= 0


Figure 10. CVE social cafe´.

social interaction but, instead of direct faceto-face contact with another person, the student would communicate with them indirectly through the medium of the VE interface. The social cafe´ was constructed as a CVE replication of the single-user virtual cafe´ (Figure 10). Students were asked to find their friend and then buy a drink. One of the advantages of the CVE was that a teacher could facilitate student learning in a more natural way, themselves acting as an avatar in the VE and leading conversations, rather than sitting alongside users offering them prompts for interaction choices in the SVE. In the case of CVE, a facilitator played the role of a “friend” in the VE. In this situation, the teacher and the student colocated the VE; however, in the real environment, they were seated in different rooms. User testing of the CVEs conducted with adults with Asperger’s syndrome observed that they successfully navigated the CVE and appeared to interpret the VE as representative of the real world. However, it was not possible to examine social interaction skills as the participants avoided each other in the CVE (Rutten et al., 2003). Attempts to conduct CVE trials in schools found that the technology was not sufficiently robust to work successfully in situ. It is hoped that as these technologies mature, they will become sufficiently stable and portable to be tested in a teaching environment. The SVE tasks of queuing (joining a line) and finding somewhere to sit also were presented in a different context—traveling on a bus (Figure 11). In this example, two avatars are already seated on the bus, but there are lots of empty seats. Figure 12 shows the view-

Figure 11. Virtual bus.

point from the user’s seat once he or she has sat down. As can be seen, it is apparent that the user chose to sit next to one of the avatars. This was not prevented (as it would be in the virtual cafe´) as it is more permissible to sit where you like on public transport. However, although the avatar in this scenario makes no comment to the user, in this situation the person would most likely be wondering why the user chose to sit next to him when there were other seats available. The information bar displays the thoughts of the avatar. The AS Interactive project used a combination of experimental and qualitative approaches to investigate use and understanding of VEs and the potential they offer for learning of specific social skills. At the outset it was not clear whether ASD students would recognize the VE as a representation of a realworld scenario and whether their behavior

Figure 12. What are they thinking?

LWW/TLD

LWWJ350-02

July 24, 2007

6:14

Char Count= 0

Virtual Environments for Learning and Communication would reflect their understanding and behavior in the real world. An experimental study compared responses from ASD students with matched students with and without learning difficulties when asked to describe objects, characters and activities in the VE and video images of real-world equivalents (Parsons et al., 2004). This study found that most of the students with ASD did interpret the VE in a nonliteral way and were able to verbalize differences between the real world and the VE representation. However, in a follow-up study, students with ASD, low verbal IQ, and weak executive function skills demonstrated limited understanding of the VE and required additional support from a facilitator to complete tasks (Parsons et al., 2005). Although students could not generalize their learning to a different social context (e.g., cafe´ to bus), successful skill learning and transfer was evident from VE to real behavior within the same social context (Mitchell, Parsons, & Leonard, 2007). It was concluded that VEs could be used for education and training but not as a stand-alone teaching method. Teaching expertise is required to ensure appropriate use for individual learners (Neale, Cobb, & Wilson, 2001). Following this project, anecdotal reports also indicated that use of the VE could improve student confidence and learning (Wiederhold, 2004). DISCUSSION This article has presented a summary of progressive projects aimed at exploring the application of VR technology to develop VEs supporting learning and communication in special needs education. As VR is an emerging technology, initial research questions were concerned with utility and accessibility; could this medium be used as a teaching resource in schools, and would students with SEN be able to use it successfully? The studies presented in this article provide evidence that VE technology is accessible to (and acceptable by) students with special education needs, including SLD (intellectual/cognitive impairment) and ASD.

221

It is important to ensure that learning within a VE is not more difficult than it would be through other means (i.e., that control of the VE itself does not present a barrier to learning). Contrary to expectations of learning facilitators, the initial studies found that students with SLD could use 3D control devices to move around the VE. More recent research has found that dual control devices are confusing for users with learning difficulties and that it is better to separate control-action by using a joystick for navigation and mouse for interaction selection (Standen, Brown, Anderton, & Battersby, 2004). Positive outcomes were reported in all of the projects, and anecdotal evidence indicates some degree of learning following use of VEs. However, it was not possible within these studies to comment with any degree of authority concerning what type of learning is supported by VEs for special education needs. One reason for this was the methodological approach taken—it was unrealistic to expect that use of any training medium over a short period would generate demonstrable learning acquisition, particularly for students with SLD. Subsequent studies conducted by other research teams have found evidence of learning gain resulting from the use of these and similar VEs. Standen and Low (1996) observed an increase in Makaton vocabulary in students with SLD, and Standen, Cromby, and Brown (1998) found that the use of a VE for rehearsal of a similar supermarket shopping task resulted in significantly faster and more correct selection of items in performance of a real-world shopping compared to students who had not received VE training. Despite these and other examples of successful learning and training transfer (e.g., Mendozzi et al., 2000; Rose et al., 2000), still not enough is known about the pedagogical basis through which VEs support learning in special needs education. Domain knowledge acquisition, supported by constructivist and constructionist principles, has been demonstrated in the use of VEs in mainstream education. Of course, it is probable that these methods of learning are not appropriate for the

LWW/TLD

LWWJ350-02

222

July 24, 2007

6:14

Char Count= 0


learning of skills and behaviors. Examination of evidence of constructivist principles (as defined by Jonassen, 1994) in the use of the Makaton and Life Skills projects found that the different VEs met these principles in different ways; the virtual supermarket supported reflection on the real world and unaided exploration whereas the Makaton environment supported less independent exploration but more collaborative instruction (Neale et al., 1999). It was difficult to determine knowledge construction through observation of student performance in use of the VEs largely due to difficulties in control of the interaction devices. However, there was an observed trend toward student initiation of decisions and actions throughout the VE training period and from pre–post real-world observation of tasks in the Life Skills project (Cobb et al., 1998). Constructivist learning is more likely to be supported by VEs that replicate real-world scenarios or behaviors since students will be able to draw upon their understanding of the wider context. Abstract VEs, such as the Makaton environment, have no real-world counterpart for the students to draw upon. However, there is a danger that the more closely a VE does replicate the real world, features of the VE that do not behave as expected may serve to confuse or irritate students and this may detract from the learning experience. CONCLUSIONS Application of VR technology in special needs education has not yet realized the dream of the “Virtual Visualization Room” in which skills learned in virtual situations transfer to real situations (Brown, Stewart, & Mallett, 1997). Virtual reality and its associated technologies have developed rapidly since 1991; still little is known about how to apply these technologies for learning. Education theorists can indicate what is required of the learning experience, but few studies have been conducted to measure learning gain. The experience of VIRART’s research in this field over the last 15 years has been one of realization of the importance of human factors and principles of human–computer inter-

action in designing applications of new technology. In all of the studies reported in this article, usability and interface issues were identified and, in many cases, resolved by iterative design improvement. However, there are still no comprehensive guidelines for the design of VEs for these user populations, and it is recommended that researchers still need to include a variety of stakeholders in the design of new VEs for learning (Neale, Cobb, & Kerr, 2003). Long-term integration within the teaching curriculum requires successful implementation within the teaching environment and uptake from teaching staff. One of the barriers encountered when developing a new technology is that they may be unstable and not compatible with the existing technology infrastructure. To overcome this, it is essential that researchers work closely with teaching professionals and focus on the context of use, experience, and expectations of technology among the teaching staff. Successful implementation requires teachers to take ownership of the teaching program. More successful outcomes will be achieved if teachers and training professionals take ownership of the research from the outset (Cobb, Neale, & Stewart, 2001). Direct input from teaching professionals is essential to ensure that VE development leads to a useful and usable teaching aid. The AS Interactive project employed a teacher to develop a teaching resource pack to accompany the VE, guiding teachers how to use the system (download available from www.virart.nott.ac.uk/asi/). Of course, the technology must meet the needs of the teaching requirements if it is to be adopted as a teaching aid and, unfortunately, this required iterative development and evaluation extending beyond the timescale of these early projects. VR technology has a lot to offer. There appears to be potential for VEs to benefit learning but further work is needed to direct VE development and methods for evaluation (Winn, 2002). It is hoped that these early studies will provide encouragement and insight to others seeking to utilize VEs for children with special education needs.

LWW/TLD

LWWJ350-02

July 24, 2007

6:14

Char Count= 0


223

REFERENCES Auld, L. W. S., & Pantelidis, V. S. (Eds). (1995, June). VR in the Schools, 1(1). Barab, S. A., Hay, K. E., Squire, K., Barnett, M., Schmidt, R., Karrigan, K., et al. (2000). The virtual solar system: Learning through a technology-rich, inquiry-based, participatory learning environment. Journal of Science Education and Technology, 9(1), 7–25. Bell, J. T., & Fogler, H. S. (1995, June). The investigation and application of virtual reality as an educational tool. Paper presented at the annual conference of the American society for Engineering Education, Session No. 2513, Anaheim, CA. Bricken, M. (1991). Virtual reality learning environments: Potentials and challenges. Computer Graphics, 25(1), 178–184. Bricken, W., & Winn, W. (1992). Designing virtual worlds for use in mathematics education: The example of experiential algebra. Educational Technology, 32, 12– 19. Brown, D. J. (1994, September). Special computers for special children. Open Learning Systems News, pp. 7–8. Brown, D. J., Cobb, S. V., & Eastgate, R. M. (1995). Learning in virtual environments (LIVE). In R. A. Earnshaw, J. A. Vince, & H. Jones (Eds.), Virtual reality applications (chap. 14, pp. 245–252). San Diego, CA: Academic Press. Brown, D. J., Kerr, S. J., & Bayon, V. (1998, September 1011). The development of the virtual city: A user centred approach. In Proceedings of 2nd European conference on disability, virtual reality and associated ¨ vde, Sweden: Mount technologies (pp. 11–16). Sko Billingen. . Brown, D. J., Kerr, S. J., & Wilson, J. R. (1997). Virtual environments in special-needs education. Communications of the ACM, 40(8), 72–75. Brown, D. J., Neale, H. R., Cobb, S. V., & Reynolds, H. (1999). Development and evaluation of the virtual city. International Journal of Virtual Reality, 4(1), 28–41. Brown, D. J., Stewart, D., & Mallett, A. (1997, Spring). Virtual rooms. The SLD Experience, 17, 15–16. Byrne, C., & Furness, T. (1994). Virtual reality and education. IFIP Transactions, Computer Science and Technology, 58, 181–189. Clancy, H. (1996). Medical field prescribes virtual reality for rehabilitation therapy. Computer Reseller News, 698, 76. Cobb, S., Beardon, L., Eastgate, R., Glover, T., Kerr, S., Neale, H., et al. (2002). Applied virtual environments to support learning of social interaction skills in users with Asperger’s syndrome. Digital Creativity, 13(1), 11–22. Cobb, S. V. G., & Sharkey, P. M. (2007). A decade of research and development in disability, virtual reality and associated technologies: Review of ICDVRAT 1996–2006. International Journal of Virtual Reality, 6(2), 51–68.

Cobb, S. V. G., Neale, H. R., Crosier, J. K., & Wilson, J. R. (2002). Development and evaluation of virtual environments for education. In K. Stanney (Ed.), Virtual environment handbook (chap. 47, pp. 911– 936). Hillsdale, NJ: Erlbaum. Cobb, S. V. G., Neale, H. R., & Reynolds, H. (1998, September 10–11). Evaluation of virtual learning environment. In: The 2nd European conference on disability, virtual reality and associated technologies (pp. 17–23). Skovde, Sweden: University of Reading. Cobb, S. V. G., Neale, H. R., & Stewart, D. (2001, August 8– 10). Virtual environments—Improving accessibility to learning? In Proceedings of 1st international conference on universal access and human computer interaction (pp. 783–787). New Orleans, LA: Erlbaum. Crosier, J. (1996). Experimental comparison of different input devices into virtual reality systems used by children with severe learning disabilities. Unpublished dissertation, University of Nottingham. Crosier, J. K., Cobb, S. V. G., & Wilson, J. R. (2000). Experimental comparison of virtual reality with traditional teaching methods for teaching radioactivity. Education and Information Technologies, 5(4), 329– 343. Dede, C., Salzman, M., & Bowen Loftin, R. (1996). ScienceSpace: Virtual realities for learning complex and abstract scientific concepts. Paper presented at the Proceedings of Virtual Reality Annual International Symposium (VRAIS) 1996. pp. 246–253. ECDVRAT. (1996, July 8–10). Paper presented at the 1st European conference on Disability, Virtual Reality and Associated Technologies Maidenhead, UK. Retrieved from www.icdvrat.rdg.ac.uk/archive.htm Grove, N., & Walker, M. (1990). The Makaton vocabulary: Using manual signs and graphic symbols to develop inter-personal communication. AAC Augmentative and Alternative Communication, 15–28. Hall, J. (1993). Explorations of population expectations and stereotypes with relevance to design. Unpublished dissertation, University of Nottingham. Jonassen, D. (1994). Thinking technology? Towards a constructivist design model. Educational Technology, 34(4), 34–37. Kerr, S. J., Neale, H. R., & Cobb, S. V. G. (2002). Virtual environments for social skills training: The importance of scaffolding in practice. In: Julie Jacko (Ed.), The 5th international ACM conference on Assistive Technologies (ASSETS 2002) (pp. 104–110). ACM Press. Meakin, L., Wilkins, L., Gent, C., Brown, S., Moreledge, D., Gretton, C., et al. (1998, September 10–11). User group involvement in the development of a virtual city. In P. Sharkey, D. Rose, & J. I. Lindstrom (Eds.), The 2nd European conference on Disability, Virtual Reality and Associated Technologies (pp. 1–9). Skovde, Sweden: University of Reading. Mendozzi, L., Pugnetti, L., Barneiri, E., Attree, E. A., Rose, F. D., Moro, W., et al. (2000, September 23–25).

LWW/TLD

LWWJ350-02

224

July 24, 2007

6:14

Char Count= 0


A generic productive process to train persons with disabilities. In Proceedings of the 3rd international conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT) (pp. 115–122). Sardinia, NY. Mitchell, P., Parsons, S., & Leonard, A. (2007). Using virtual environments for teaching social understanding to adolescents with autistic spectrum disorders. Journal of Autism and Developmental Disorders, 37, 589–600. Moshell, J. M., & Huges, C. E. (1996). The virtual academy: A simulated environment for constructionist learning. International Journal of Human–Computer Interaction, 8(1), 95–110. Moshell, J. M., & Hughes, C. E. (2002). Virtual environments as a tool for academic learning. In K. Stanney (Ed.), Handbook of virtual environments, design, implementation and applications (pp. 893– 910). Hillsdale, NJ: Erlbaum. Murphy, H. J. (1993, June). Virtual reality and persons with disabilities. Proceedings of the first annual conference: Virtual reality and persons with disabilities. Northridge: California State University Center on Disabilities. Neale, H. (1997). A structured evaluation of virtual environments in special needs education. Unpublished dissertation, University of Nottingham. Neale, H. R., Brown, D. J., Cobb, S. V. G., & Wilson, J. R. (1999). Structured evaluation of virtual environments for special needs education. Presence: Teleoperators and Virtual Environments, 8(3), 264– 282. Neale, H., Cobb, S., & Kerr, S. (2003, March 25–28). An inclusive design toolbox for development of educational virtual environments. In Include2003 (pp. 230–235). London: Royal College of Art. Neale, H. R., Cobb, S. V., Kerr, S., & Leonard, A. (2002, September). Exploring the role of virtual environments in the special needs classroom. In Proceedings of the 4th international conference on disability, virtual reality and associated technologies (ICDVRAT) (pp. 259–266). Veszprem, Hungary: University of Reading. Neale, H. R., Cobb, S. V. G., & Wilson, J. R. (2000, September 23–25). Designing virtual learning environments for people with learning disabilities: Usability issues. In Proceedings of the 3rd international conference on disability, virtual reality and associated technologies (ICDVRAT) (pp. 265–272). Sardinia, NY: University of Reading. Neale, H. R., Cobb, S. V. G., & Wilson, J. R. (2001). Involving users with learning disabilities in virtual environment design. In Proceedings of universal access in HCI (pp. 506–510). New Orleans, LA: Erlbaum. Parsons, S., Beardon, L., Neale, H. R., Reynard, G., Eastgate, R., Wilson, J. R., et al. (2000). Development of social skills amongst adults with Asperger’s syndrome using virtual environments: The AS interac-

tive project. Paper presented at the 3rd international conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT), Sardinia, NY, September 23–25, 2000, pp. 163–170. Parsons, S., & Mitchell, P. (2002). The potential of virtual reality in social skills training for people with autistic spectrum disorders. Journal of Intellectual Disability Research, 46(Pt. 5), 430–443. Parsons, S., Mitchell, P., & Leonard, A. (2004). The use and understanding of virtual environments by adolescents with autistic spectrum disorders. Journal of Autism and Developmental Disorders, 34(4), 449– 466. Parsons, S., Mitchell, P., & Leonard, A. (2005). Do adolescents with autistic spectrum disorders adhere to social conventions in virtual environments? Autism, 9(1), 95–117. Rose, F. D., Brooks, B. M., & Attree, E. A. (2000). Virtual reality in vocational training of people with learning disabilities. Paper presented at the 3rd international conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT), Sardinia, NY, September 23–25, pp. 129–135. Rutten, A., Cobb, S., Neale, H., Kerr, S., Leonard, A., Parsons, S., et al. (2003). The AS interactive project: Single-user and collaborative virtual environments for people with high-functioning autistic spectrum disorders. Journal of Visualization and Computer Animation, 14, 233–241. Salzman, M. C., Dede, C., Bowen Loftin, R., & Chen, J. (1999). A model for how VR aids complex conceptual learning. Presence, 8(3), 293–316. Standen, P. J., & Brown, D. J. (2005). Virtual reality in the rehabilitation of people with intellectual disabilities: Review. Cyberpsychology & Behaviour, 8(3), 272– 282. Standen, P. J., Cromby, J. J., & Brown, D. J. (1998). Playing for real. Mental Healthcare, 1, 412–415. Standen, P. J., & Low, H. L. (1996). Do virtual environments promote self-directed activity? A study of students with severe learning difficulties learning Makaton sign language. Paper presented at the 1st European conference on Disability, Virtual Reality and Associated Technologies, Maidenhead, UK, July 8–10, pp. 123–127. Standen, P. J., Brown, D. J., Anderton, N., & Battersby, S. (2004). Problems with control devices experienced by people with intellectual disabilities using virtual environments: A systematic evaluation, 2004. Paper presented at the 5th international conference on Disability, Virtual Reality and Associated Technologies (ICDVRAT), Oxford, UK, September 20–22, 2004, pp. 299–304. Strickland, D. (1996). A virtual reality application with autistic children. Presence, 5, 319–329. VIRART (2007). Available from Virtual Reality Applications Research Team, University of Nottingham, UK, Web site: www.virart.nottingham.ac.uk

LWW/TLD

LWWJ350-02

July 24, 2007

6:14

Char Count= 0

Virtual Environments for Learning and Communication Virtual reality and education. (1999, June). Presence: Teleoperators and Virtual Environments, 8(3, Special issue). VRET (1997). Proceedings of Virtual Reality Education and Training, University of Loughborough, U.K. Retrieved from http://www.vrnews.com/vret97.html. Wiederhold, B. K. (2004). Virtual healing. San Diego, CA: Interactive Media Institute.

225

Wing, L. (1996). Autistic spectrum disorders. British Medical Journal, 312, 327–328. Winn, W. (2002). Current trends in educational technology research: The study of learning environments. Educational Psychology Review, 14(3), 331–351. Youngblut, C. (1998). Educational uses of virtual reality technology (Tech. Rep. No. D-2128). Alexandria, VA: Institute for Defense Analysis.