Me and My VE, Part 2

Proceedings of the Human Factors and Ergonomics Society Annual Meeting http://pro.sagepub.com/

Me and My VE, Part 2 Laura Strater, Jennifer Riley, Fleet Davis, Katelyn Procci, Clint Bowers, Deborah Beidel, Brian Bunnell, Alex Proaps, Ganesh Sankaranarayanan, Baichun Li, Suvranu De and Caroline G. L. Cao Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2013 57: 2127 DOI: 10.1177/1541931213571474 The online version of this article can be found at: http://pro.sagepub.com/content/57/1/2127

Published by: http://www.sagepublications.com

On behalf of:

Human Factors and Ergonomics Society

Additional services and information for Proceedings of the Human Factors and Ergonomics Society Annual Meeting can be found at: Email Alerts: http://pro.sagepub.com/cgi/alerts Subscriptions: http://pro.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://pro.sagepub.com/content/57/1/2127.refs.html

>> Version of Record - Sep 30, 2013 What is This?

Downloaded from pro.sagepub.com at University of Central Florida Libraries on October 6, 2013

PROCEEDINGS of the HUMAN FACTORS and ERGONOMICS SOCIETY 57th ANNUAL MEETING - 2013

2127

Me and My VE, Part 2 Laura Strater, Jennifer Riley, Fleet Davis, SA Technologies, Inc.; Katelyn Procci, Clint Bowers, Deborah Beidel, Brian Bunnell, University of Central Florida; Alex Proaps, Old Dominion University; Ganesh Sankaranarayanan, Baichun Li, Suvranu De, Rensselaer Polytechnic Institute & Caroline G. L. Cao, Wright State University Virtual environments, simulations and serious games are increasingly being employed for research, training, education, evaluation, and various business endeavors. In a redux of a favorite session from HFES 2012, this session will describe and demonstrate some of the diverse uses for virtual environments (VEs) in an alternate demonstration format. The session will begin with demonstrators providing a brief description of their VE, and how they’ve used it to answer a critical research question or address a unique need, including a video demonstration of the VE in action. After these introductions, all demonstrations will be set up around the room, and session attendees can move around the room for more direct interaction with both the demonstrations and the demonstrators. The objective of the session is to provoke ideas among the attendees for how VEs, simulations and serious games can help address their clinical, research, training, education, evaluation or business needs.

Copyright 2013 by Human Factors and Ergonomics Society, Inc. All rights reserved. DOI 10.1177/1541931213571474

INTRODUCTION While there are many human factors (HF) professionals, both researchers and practitioners, who use virtual environments (VE), simulations and serious games on a daily basis in pursuit of their work objectives, there are many more who have a vague sense that these tools might be useful, but aren’t really sure how to apply them to their particular tasks or the problems they must address. The objective of this unique demonstration session is to introduce attendees to several VEs, and the creative ways our demonstrators employ these VEs to solve their work problems. After seeing and interacting with the variety of demonstrations in the Me and My VE session, session attendees may conceive of novel ways that they can use VEs, serious games and simulations to address their real world concerns. VEs, simulations and serious games are increasingly popular among researchers, trainers, educators, evaluators and business people for a number of reasons. First, many tasks can be done for a lower cost using a VE. Training, for instance, can be developed once in a VE or serious game, and administered over and over with minimal ongoing, valuable instructor time required. This training can then be delivered to trainees at their desks or local computer training labs, minimizing travel and overall training costs. Moreover, the training can be done at any time, rather than only being available during a specific time frame. A critical benefit of VE is that many tasks that may be hazardous in a real world environment can be attempted with impunity in a VE or serious game. Driver training is familiar example, but the military is increasingly using

game-based training to familiarize personnel with combat skills (e.g., Strater & Bolstad, 2008). Moreover, often a VE can be used to either train or evaluate team or individual skills in a manner that is hard to recreate in a live setting and that can be reliably and precisely replicated for each trainee. By using avatars, video, and game components such as rewards for performance opportunities for interaction can be provided in a VE much more easily and at a lower cost than live training. In addition, VE or serious games, games developed with a serious objective, can deliver content (e.g., training, education, research) in a more engaging and entertaining format than traditional methods. Game elements or characteristics are frequently incorporated in VEs, and these game elements produce a more engaging environment for the participant, patient, trainee, or user. Some of the most common game characteristics used to engage the recipient are fantasy, control and challenging activities, rules, competition and goals, conflict and closure and rewards that lead to player motivation and attachment. Incorporation of many or all of these elements into a VE makes the final product more appealing to the target audience. In this paper, we don’t distinguish between VE, simulation and serious games, since most VEs today incorporate game elements. The format for the session will include a brief introduction of each VE by the demonstrator to include a video of the VE in action. After the demos are complete, each demonstrator will have a demo set up in the room, where attendees can either interact with the demos or view a more detailed video of the demo system. Demonstrators will describe their systems and their research in detail, and answer attendee questions. We hope that by seeing and interacting with the


demonstrations in this session, attendees will be inspired to develop new and creative methods for using VEs to solve their business and research questions.

2128

The demo will show both how clinicians will use the iPad app for patient management as well as the games developed for treatment of selective mutism.

DEMONSTRATIONS Demonstration 1 – Clinical Treating Selective Mutism with iPad Games Katelyn Procci The Selective Mutism App is a prototype iPad application that assists in the treatment of children with selective mutism, a form of social anxiety disorder where children are unable to speak in social situations (such as school), despite their ability to speak normally when at home or in other comfortable environments (Krysanski, 2003). Treatment is difficult and involves carefully shaping speaking behaviors, such as opening the mouth in front of others or making breath sounds. Selective mutism is most commonly found in children who are age 7 or younger and who typically do not understand the purpose or need for therapy. In order to better engage children in the therapeutic process, clinician researchers began to use iPad apps that mimicked the behaviors needed to produce speech. For example, one of the first goals of treatment is to have children produce vocal sounds such as “shhh.” Some available apps allow children to blow into the iPad microphone to blow out a virtual candle, which produces the “sh” sound. They found that using off-the-shelf iPad apps quickly engaged the children, made therapy fun, and hastened the therapeutic process. Encouraged by these results, the team proposed the development of an iPad app that could be integrated into a clinician’s treatment protocol. This would enable clinicians who did not specialize in anxiety disorders to deliver effective treatment to children using the app as both a guide and a toolbox. A patient management system would allow clinicians to add patients, launch therapy sessions, and track their patients’ progress over time. Once a therapy session was launched, the clinician would have several minigames to be used as tools. A prototype app was initially developed as a proof-of-concept and features both the framework for patient management and one minigame, Lift-Off!, to be used during therapy. Lift-Off! recreates one step in therapy, having the patient become comfortable with playing their recorded voice for others, by transforming it into a fun game. The patient records their voice and plays it back with varying levels of volume and distortion in order to inflate a hot air balloon and win the game. With this type of behavioral shaping therapy, compliant responses are rewarded, in this case with experience points and coins, which can then be used on real-world rewards as a part of the treatment protocol.

Figure 1: Selective Mutism iPad App screenshots show the patient management interface and the Lift Off! Minigame.

Demonstration 2 - Research GAMBIT Game-Distributed Interactive Simulation for Human Performance Research Alex Proaps Researchers in many lab facilities currently investigate various human performance issues using a virtual environment called GAMBIT. The modeled environment in this Game-Distributed Interactive Simulation (G-DIS) resembles the Fort McKenna Military Operations in Urban Terrain (MOUT) site at Fort Benning, Georgia. A research team from the game manufacturer, two universities, and the government investigated training potential using this modified version of the first person shooter game, Half Life 2™, developed by Valve Software (Lampton, Bliss, Orvis, Kring, & Martin, 2009). The game platform provides user-friendly avatar and object creation, scenario development, and route management. The modeled environment, though not as complex as a full-fledged, first-person shooter game, is an ideal platform for behavioral research because it allows for (1) team task collaboration, (2) team task performance assessment, and (3) real-time interaction with game players through a functional chat. Many researchers have used this G-DIS research platform to address critical research questions, including: (1) a study assessing alert characteristics and avatar identification, (2) the effects of unreliable information on Infantry situation awareness (SA), (3) the relationships among task difficulty, team search task performance, and emergent leadership. GAMBIT has also been used to conduct a series of experiments to explore the impact of rapid serial visual presentation on individual search task performance. The demonstration will show how the capabilities afforded by GAMBIT


allow researchers to address a broad variety of research questions in a dynamic, adaptable, and engaging research venue.

2129

Gen2 simulator consists of a Head Mounted Device that provides full immersion into the virtual environment inside which the surgical scene is projected onto a virtual monitor. A haptic feedback interface with instrumented surgical tools is used to interact and train in the simulator. The demo will allow attendees to interact with Gen2 to experience the capabilities of this immersive surgical simulation environment.

Figure 2: GAMBIT interface

Demonstration 3 – Research and Training Evaluating the Impact of Stressors on Surgical Performance Ganesh Sankaranarayanan, Baichun Li, Suvranu De & Caroline G. L. Cao Though Virtual Reality (VR) based surgical simulators have been used for training for a long time, so far they have been tailored to focus on developing psychomotor skills. In the real environment such as the Operating Room (OR) and the Emergency Room (ER), there are many factors that affect the surgeon’s performance, such as distractions, stress, cognitive overload, etc. What human factors researchers and cognitive psychologists in the field of surgical skills training would want is a fully immersive realistic VR system where the environment can be controlled to investigate the effects of distractions, stress and cognitive overload on the surgeon’s performance. Until now, there is no VR simulator capable of providing such a research environment. Our team of universities and medical facilities has developed a new VR simulator called the Generation (Gen) 2 simulator, which is a paradigm shift compared to current research and commercial simulators. The unique features of the Gen2 simulator are (1) its ability to realistically portray operating room environments, where multiple sources of distraction can be programmed (through audio and video simulation); (2) its ability to simulate various surgical procedures and patient scenarios using advanced computational mechanics algorithms; and (3) its ability to provide accurate haptic (force) feedback. In addition, the Gen2 simulation environment allows multiple persons to collaborate in the same VR environment for the study of group and team dynamics in surgery. The

Figure 3: Gen 2 user interacting with the system using both the HMD and haptic feedback system, with the operating room environment (inset)

Demonstration 4 – Training and Peer Tutoring Cognitive Skills Training for Novice Drivers Laura Strater While many of the training tools using VE for rapid skill acquisition focus on psychomotor skills, there is also a critical need for cognitive skills training programs that allow trainees to improve their situation assessment and decision making skills. Moreover, training is needed that focuses on the behaviors and actions trainees should take related to observing, gathering information, and communicating to others. To address this need, we have developed a computer-based training method and software tool, the Situation Awareness Virtual Instructor (SAVI), to capitalize on the benefits of peer tutoring and video modeling in developing situation assessment and decision-making skills. Trainees in complex operations often struggle to develop the deep understanding of “good” operational behaviors associated with consistent expert-level decision making and performance. These trainees benefit from direct instruction on and critical review of cognitive processes that help develop SA, the foundation of good decision making.


SAVI trains situation assessment behaviors and team communication skills by allowing trainees to play the role of the instructor-observer/trainer (King et al., 1998; Whitman & Fife, 1988). Trainees observe key actions, behaviors, and communications demonstrated by others during realistic scenarios and then critically review the quality of the actions. Trainees rate what they observe and then must provide their rationale for the ratings. Their results are compared to ratings and rationale provided by experts who have examined the same scene. The process of critically evaluating the actions portrayed in the scenes, and then evaluating and calibrating their responses through comparison with the expert, allows trainees to refine their mental models to more closely match those of experts regarding how these cognitive processes support information processing and impact decisions. SAVI utilizes live action video content or computer generated imagery from VR or gaming technologies to create realistic scenes for lessons. Trainees observe and rate both good and poor behaviors, which also engages them in modified video modeling training, shown to be effective in changing behavior (Struve & Wandke, 2009; Catania et al., 2009). This further promotes effective and appropriate behavior modification by supporting reflection on desired behaviors and the purpose of changing behaviors. This demo will present a prototype SAVI lesson developed to train novice drivers who often lack robust situation assessment skills. Experienced drivers use a high-degree of automaticity for both the physical tasks of driving (e.g., steering, starting stopping, signaling) and the cognitive tasks (e.g., scanning, recognizing and avoiding hazards). The prototype training shown in this demo focuses on helping less experienced drivers develop the cognitive skills and strategies of more experienced drivers.

Figure 4: SAVI screenshot showing interface for identifying trainee rationales for the selected rating

Demonstration 5 – Performance Assessment Automated Query Delivery and After Action Review in VBS2 with VESARS Jennifer Riley and Fleet Davis

2130

The Virtual Environment Situation Awareness Rating System (VESARSTM) is a software suite designed to assess situation awareness (SA) in both VE and realworld training of individuals or teams. The tool captures data on trainee SA and rapidly analyzes and presents it in a format which can be used by instructors for standardized coaching and feedback throughout a training exercise and/or during an after action review (AAR). The SA of each trainee is assessed during a training exercise using the following SA metrics: • Real-time SA Queries – provides real-time, objective assessment of trainee mission knowledge. VESARSTM allows trainers to assess individual, shared, and team SA. • Real-time SA Behavior Ratings – used for rating standard behaviors related to gathering and supporting SA. An expert rater observes an individual or team and uses rating scales to assess the trainees on a set of predefined, subject matter expert (SME)-validated SA behaviors. • Real-time SA Communication Ratings – used for rating communications specific to the acquisition and dissemination of SA. An expert rater evaluates trainees’ communications for utterances that fall into one of the predefined, SME-validated SA communication categories. The most recent VESARS design provides automatic query delivery and real-time scoring via direct integration with Virtual Battlespace 2 (VBS2). The software actively listens for data packets sent over a network by the simulator and uses this information to automatically activate and present SA queries to trainees while they are engaged in this serious game. Currently, VESARS is able to activate queries based on virtual entity health, entity fire events, entity location, and time. This functionality allows instructors to focus on monitoring the trainees rather than query delivery and provides them with a tremendous amount of flexibility for performance assessment. VESARS also scores the trainees’ responses and presents the scores in real-time via a performance data dashboard. This dashboard helps instructors to quickly review and analyze trainee performance in real time, thus allowing them to provide immediate feedback and rapidly adapt the material to the trainees’ skill level rather than waiting to the end of the exercise to interpret the results and revise their approach. VESARS’ capabilities allow instructors to offload tasks and workload by automating lower-level tasks and allowing instructors to focus on delivering the training that only they can provide. In this demonstration, we will show how VESARS interacts within VBS2 to improve training evaluation and AAR


through automated query delivery and a standardized, comprehensive performance review application.

Figure 5: VESARS interface for VBS52

DISCUSSION In Me and My VE, we have introduced you to several potential uses for virtual environments, simulations and games. In the first demonstration, we present a clinical application for an iPad application that can be used to treat selective mutism, improving the social interaction skills of children with this condition. Our next demo shows how the GAMBIT G-DIS gaming environment has been used to support critical research into the impact of various alert characteristics, information reliability, and rapid serial visual presentation, along with the relationships between task difficulty, team search task performance, and emergent leadership. Our third demo provides a venue for both research to investigate the impact of environmental stressors on surgical performance and training coping skill to ameliorate the impact. Next, we present a demo that shows a method for using the benefits of peer review and video modeling to provide behaviorally based training that can improve the performance of teen drivers. Our final demonstration provides support for instructors in using VE for training as it provides both automated query delivery based on scenario events as well as a comprehensive, standardized after action review to insure robust feedback to trainees. The variety of uses for VE extends beyond these demos; however, the demos were selected to give some ideas of the scope that VEs can provide for clinical use, systems evaluation, prototyping, research, and training. Many of the demos presented here show the advantages of VEs. Specifically, using an iPad for the clinical treatment of selective mutism provides a portable solution that can go where the patient goes, even easing some of the social pressure through the use of a popular

2131

device. For research, use of a VE affords a much greater degree of experimental control than is possible in a real world environment, allowing greater replicability and fewer confounds, and often allows research to be conducted for a much lower cost. This is also true for training applications. Training can be delivered in VE to many more students for a significant cost reduction, as compared to live training or even classroom training. Once the training is developed, it can be delivered again and again with minimal cost. In addition, more consistent and comprehensive evaluation is possible in VE. Finally, as in these demos, many training environments have a level of risk for trainees (e.g., military operations and driving), and providing as much initial training as possible in VE can minimize these risks to the trainee. Overall, VE provides a number of significant benefits for researchers, trainers, clinicians and practitioners, and the demos included here are intended to demonstrate those benefits. CONCLUSION In this paper, we’ve presented descriptions of how several researchers, practitioners and developers are using VEs in diverse ways to advance the state of the art in clinical practice, research, training, and evaluation. The objective of this paper is to encourage readers to consider ways in which VEs, serious games and simulations can address the work issues they face. As the capabilities, availability and cost effectiveness of VEs continue to increase, their use will extend to increasingly diverse areas, to address increasingly complex problems. If you are not using VEs at present in your work, we hope that this session will help you identify ways that you can use VEs in the future to solve the challenging problems HF practitioners and researchers address daily. REFERENCES Catania, C. N., Almeida, D., Liu-Constant, B., & DiGennaroReed, F. D. (2009). Video modeling to train staff to implement discrete-trial instruction. Journal of Applied Behavior Analysis, 42, 387–392. King, A., Staffieri, A., & Adelgais, A. (1998). Mutual peer tutoring: Effects of structuring tutorial interaction to scaffold peer learning. Journal of Educational Psychology, 90(1), 134 - 152. Krysanski, V. L. (2003). A brief review of selective mutism literature. The Journal of Psychology, 137(1), 29-40. Struve, D. & Wandke, H. (2009). Video modeling for training older adults to use new technologies. ACM Transactions on Accessible Computing, Vol. 2, No. 1, Article 4, May 2009. Whitman, N. A., Fife, J. D. (1988). Peer Teaching: To Teach Is To Learn Twice. ASHE-ERIC Higher Education Report No. 4, (ED305016).