Believe What You Hear, Not What You See – Vision Interferes with Auditory Route Guidance in Complex Environment Ying Wang1, Huiting Zhang2, Lu Yu2, Kan Zhang2, Xianghong Sun2, and Thomas Plocher3 1
Key Laboratory of Mental Health, Institute of Psychology, Chinese Academy of Sciences, China 2 Institute of Psychology, Chinese Academy of Sciences, China 3 Honeywell Labs, Minneapolis, Minnesota, USA {zhanghuiting,yulu,zhangk,sunxh}@psych.ac.cn,
[email protected],
[email protected]
Abstract. Auditory route guidance has a potential use for sighted people who have to conduct emergent real-world task during navigation. Despite its affordance in assisting people in the absence of vision, it may receive interference from vision under normal visual condition. The present study tested the effect of vision on auditory route guidance using different display modes. Normalsighted firefighters were instructed to navigate within a virtual building following auditory commands from a navigation aid, either under normal (high-visibility) or smoked (low-visibility) visual condition. Navigation in normal visual condition was faster but less accurate than that under low-visibility, and was characterized by unique walking patterns. Moreover, it resulted in worse spatial memory and less positive experience toward the system. These results suggest that the interaction mode of human and auditory route guidance system could be modified by vision. Clear visual inputs boost risk-taking behaviors in route following, which might lead to dangerous consequence in specific navigation tasks. Furthermore, the interference from vision was not restricted to specific display mode, indicating that it might be a general problem for auditory route guidance. As a challenging and primary human factor issue, it should attract more attention and caution in future research and design work. Keywords: auditory route guidance, vision, firefighter, human-computer interaction, visual auditory interaction.
1 Introduction As a species on the move, humans are adept in planning routes based on their spatial knowledge and the visual surroundings. However, this ability is often challenged by unfavorable conditions like a complex and unfamiliar environment, lack of valid visual cues, or engaging in a high demanding task. Personnel of specific occupations, for example the firefighters, may encounter all the mentioned problems in their work. Navigation aids with route guidance functions are urged for these people. J.A. Jacko (Ed.): Human-Computer Interaction, Part III, HCII 2011, LNCS 6763, pp. 346–354, 2011. © Springer-Verlag Berlin Heidelberg 2011
Auditory Route Guidance in Complex Environment
347
Route guidance should have minimal interference with the interaction between the user and the real-world task, suggested by the previous work [1]. According to this principle, an auditory interface is a good candidate for navigation aids which are designed for sighted people with particular tasks. It does not require the users to take their eyes off the visual surroundings. Neither does it conflict with hand-operations. The affordance of auditory route guidance has been established through the past decades, mostly in aiding the visually impaired [2]. People with permanent or temporary visual impairment were used to evaluate the effects of different auditory display modes in terms of spatial perception [3] [4], route guidance [5] [6] [7] [8] [9], spatial learning [10] [11], and cognitive loads [12]. Nevertheless, from the other hand, navigation is essentially active and vision dominated for sighted people, yet the role of vision in auditory route guidance remains largely unknown. Here we assessed the performance of an auditory route guidance system with a simulated firefighting task. Normal-sighted firefighters were recruited to accomplish the task with four auditory display modes either under high-visibility (normal) or lowvisibility (smoked) condition. We aimed to investigate whether the effect of auditory route guidance would be affected by the presence of clear visual surroundings and whether the role of vision was dependent on display mode. We hypothesized that full visual inputs would enhance navigation performance to some extent compared with the low-visibility condition, but might also cause problems for the interaction between human and route guidance system. For instance, a conflict might arise when visual cues suggested a plausible alternative route different from the auditory commands.
2 Method 2.1 Participants Twenty-four male firefighters with normal vision and hearing took part in the experiment. Ages ranged from 20 to 25. All had no experience with navigation aid in firefighting work. They were randomly assigned to high-visibility and low-visibility conditions in equal proportion. 2.2 Virtual Environment The test environment was constructed based on the layout of a corporate office and laboratory building with a desktop virtual reality technique. The virtual building has irregular and complex internal structures (room, stairway etc.), monochrome walls and floors, and no other landmarks. In the high-visibility condition, which simulated the normal visual environment, the end of the route on which one was standing and all the non-obstructed structures in open areas could be clearly seen. In the low-visibility condition the environment was filled with grey smoke of uniform density, allowing participants to see only within several feet around them. We planned four routes through different parts of the building, each composed of nine segments joined by 8 waypoints. The lengths were 429.6ft, 483.9ft, 501.0ft, and 390.6ft. There were six fires randomly located along or several feet away from each planned route. See example of a floor plan with a predefined route and fires in Figure 1.
348
Y. Wang et al.
2.3 Auditory Guidance A prototype of an indoor route guidance system was created. During the test, the guidance system guided the participants with auditory commands as they walked from one waypoint to another along a planned route from one end to the other. The participant’s position within the virtual environment was recorded every 50ms. The course from the current position to the next waypoint was a derived straight line. The next waypoint was determined by two simultaneously satisfied rules: among the two that were closest to the participant’s current position and nearer to the destination. Two types of commands were given respectively. Once the angle between the derived path and the participant’s heading direction was larger than four degrees, turning commands were triggered. Otherwise (deviation less than 4 degrees), forward commands were given. A musical tone announced the arrival at each waypoint at the capture radius of 3.281 feet (1m). For each visibility condition, there were four auditory display modes, with different combinations of turning and going-forward commands. The pure speech mode guided the user to turn (e.g., “turn right, 60 degrees”) and go forward (e.g., “go forward, 10 meters”) with non-spatialized verbal commands. The pure 3D-audio mode used a “whoop” sound coming from the direction of the next waypoint to indicate turning, and a “ding” sound from ahead of the user for going forward. The CombinedA mode instructed turning with the spatialized `whoop' sound as in the pure 3D mode, while the subject was led going forward with 3D-speech (e.g., “go forward, 10 meters” as coming from ahead). The CombinedB mode, opposite to CombinedA, guided turning with 3D-speech (e.g., “turn right, 60 degrees” as coming from 60 degrees right) while directed going forward with the `ding' sound coming from ahead. 2.4 Apparatus and Parameters The experiment was run on a DELL OptiPlex 745 Desktop with a 1.86GHZ Intel Core2 Duo Processor. The program was developed in C with Visual Studio 2005 and rendered in MOGRE (Managed OGRE). The 3D-audio effect was implemented by OpenAL and was created using Creative Sound Blaster X-Fi Surround 5.1 soundcard and LTB Magnum 5.1 AC97 Headphones. The speech commands were in clear female voices and synthesized by the “Fang Zheng Chang Ting” software (Founder Tech). All the 3D sounds were virtually 1 meter away from the user. The “whoop” sound lasted for 100ms with emission interval of 1 second. The “ding” sound was 900ms long. Its emission interval changed from 3 seconds to 0 second gradually from the position where the going forward commands were triggered to the next waypoint. All the sounds were displayed with a background white noise at its 1/20 intensity. 2.5 Procedure The participants were instructed to rescue a person trapped by fire in a particular room inside the test building with the instructions of auditory guidance. They also needed to extinguish fires encountered on the way of rescue without running into them. They were informed that the navigation aid would generate a relatively short and safe route based on stored map but could not predict the number and locations of
Auditory Route Guidance in Complex Environment
349
fires. Each participant finished four trials on four display modes under the same visibility condition, either with smoke or not. The two pure modes came first, followed by the two combined modes. The orders within each pair were counterbalanced. The combinations of routes and display modes were pseudo-randomized. Before the first two formal trials, participants practiced the task within another virtual environment until they got familiar with the task. During the task, they continuously received auditory commands from the headsets and used “up", “down", “left", “right" keys to move forward, backward, turn left, and turn right. They pressed a “space" key to extinguish the fires detected on their way. Immediately after they reached the trapped person, a feedback sound announced success and a window popped up presenting the post-task questionnaire. In the questionnaire, they were asked to choose a route, from four standardized bird-view routes on a blank background, that represented the path that they had followed on that trial. Then they did five-point ratings on A) the difficulty of accomplishing the task under guidance, and B) their preferences for guidance mode.
3 Results 3.1 Task Completion With the help of our navigation aid, all the participants were able to reach the destination efficiently in most of the trials, despite the fact that it was a very difficult task to find a designated location within an unfamiliar complex environment, particularly with limited visual cues. Two out of forty-eight trials (twelve participants by four modes) under the high-visibility condition and four under low visibility were failed. The failures were largely due to the computing limit of the navigation system. As a prototype, for algorithm economy, the system did not take account of the physical occlusions (walls) in route computing. While getting into a wrong enclosed region accidentally, the participant had to find his way back by bypassing the wall and ignoring the auditory commands temporarily. Failure to find the right exit back to the route potentially lead to a long detour and eventually quitting the trial. We excluded the failed trials from the following analysis and replaced each missing value by the mean of the other three trials from the same individual. Meanwhile, the mean numbers of extinguished fires were high in all the conditions (ranged from 4.3 to 5.8, with a total number of 6), suggesting that the participants had paid attention to the visual surroundings during route, following as required even when the visibility was low. 3.2 Navigation Time We standardized the time of each trial by dividing the length of the planned route. As shown in Table 1, the participants spent less time to complete the task in all the four high-visibility conditions than in the low-visibility conditions. Mixed ANOVA showed significant main effects of both visibility: (F(1,22)=26.47, p
