Did You See Me? Assessing Perceptual vs. Real ... - Prospect Project

Proceedings of the 9th ACM International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI ’17), September 24–27, 2016, Oldenburg, Germany .

Did You See Me? Assessing Perceptual vs. Real Driving Gains Across Multi-Modal Pedestrian Alert Systems Coleman Merenda, Hyungil Kim, Joseph L. Gabbard, Samantha Leong Virginia Tech {cjm120, hci.kim, jgabbard, sam94}@vt.edu

David R. Large, Gary Burnett University of Nottingham {david.r.large, Gary.Burnett}@ nottingham.ac.uk

one third of those incidents being pedestrian collisions in urban areas [2]. Historically in the US, pedestrians represent 11-13% of those killed in collisions and are 1.5 times more at risk than vehicle passengers to be fatally injured [3, 4]. A majority of these fatalities involve improper pedestrian behavior, such as crossing at the incorrect place or time. When examining the total number of pedestrian fatalities, 80-93% of the collisions occurred outside of a marked crosswalk [5]. These high percentages are attributed to drivers not expecting to see a pedestrian; which may result in considerably slower braking reaction times and less time and distance to brake overall. Thus, there is an increasing need for strategies that reduce the number of driverpedestrian collisions without attempting to directly change pedestrian behavior, as directly interfering with pedestrians is often either an unfeasible or unreliable tactic to employ in more quickly moving and complex city environments.

ABSTRACT

In-vehicle support systems have the potential to reduce the risk of pedestrian collisions and promote gains in braking performance and visual attention when scanning for threats on the road. This study investigated changes in driver behavior in pedestrian collision scenarios with increasing urgency while using varying levels of pedestrian alert system (PAS) support in a medium fidelity driving simulator. During pedestrian collision scenarios, we assessed drivers’ eye gaze behavior, braking performance, and acceptance ratings across three levels of PAS and four levels of increasing urgency, defined as time to collision (TTC). Results suggest that both audio- and visually-based PAS do not produce gains in the localization of pedestrians, but can nevertheless improve drivers’ braking performance in events where pedestrians may pose a threat. Our results further suggest that drivers exhibit both innate and direct confidence in visually-based PAS support, despite no concurrent gains in visual scanning performance.

Past research has focused on designing vehicle features, such as visual or auditory alert systems, that can quickly respond to potentially dangerous pedestrian behavior and help drivers avoid subsequent collisions. Several already existing pedestrian alert systems (PAS) employ long-range radar that claims to detect pedestrians within 3 seconds of the time to collision (TTC), which is the time difference between the moment a pedestrian begins moving in a collision intercept with a vehicle and the instance of collision itself [6]. Other in-development PAS predict a further capability to detect pedestrians within an 80-meter range while reducing the amount of false positives close to 0%. [7].

Author Keywords Human-Machine Interfaces; Augmented Reality; Head-Up Displays; Pedestrian Alert Systems; Visual Attention; Simulation. CCS Concepts

• Human-centered computing~Empirical studies in HCI INTRODUCTION

Despite advances in automotive technology, pedestrians remain a high at-risk group when considering many of the common dangers relating to general vehicular use. U.S annual reports show that in 2013 alone over 66,000 pedestrians were injured in vehicle collisions and around 4,000 collisions resulted in pedestrian fatalities [1]. In that same year, similar annual rates of pedestrian collisions by population density were reported in the UK, with approximately 21,000 seriously injured collisions and up to

Limitations of Pedestrian Alert Systems

While PAS technologies are making progress towards preventing pedestrian collisions, their detection capabilities and subsequent accuracy remain underdeveloped in comparison to human perception [8]. Even the most recent and advanced detection algorithms exhibit high false positive rates. This means that the system mistakenly reports a pedestrian when not actually present. Moreover, current pedestrian detection algorithms miss approximately 20-30% of all pedestrians (false negatives), and issue approximately one false alarm (false positives) for every ten presented [9]. Thus, driver trust in PASs is likely limited by these relatively high rates of false alarms. Past research also indicates that a general lack of human factors considerations limits the overall performance of more advanced alert

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. AutomotiveUI '17, September 24–27, 2017, Oldenburg, Germany © 2017 Association for Computing Machinery. ACM ISBN 978-1-4503-51508/17/09...$15.00 https://doi.org/10.1145/3122986.3123013.

40

Session 2: Pedestrians: Communication and Alerts

AutomotiveUI ’17, Oldenburg, Germany

systems when poorly integrated into known driving systems [10].

successful implementation in automobile interfaces, since higher levels of driver distrust is known to increase the likelihood that drivers will ignore alerts when critical events occur [20]. Consequently, more research is required to better understand the perceptual limits and subsequent decision making cognitive processes drivers engage in when using pedestrian alert systems in order to ultimately recommend better HMI methods to help mitigate imperfect system accuracy and improve driver’s acceptance of the system concordantly.

Driver Trust

Regardless of the technical capabilities of PAS support, the successful future implementation of PAS technology will depend largely on the degree to which the driver is ultimately willing to place their trust in the system over their own judgement [11]. In order to effectively increase users’ level of acceptance of PAS technologies, it is important to understand how that trust operates. Five widely acknowledged geospatially-based factors that influence the acceptance of driver alerts include the: (a) locale, (b) pedestrian location, (c) direction of pedestrian motion, (d) own-vehicle’s path, and (e) curvature of the road [12-14]. For instance, drivers are more likely to accept pedestrian alerts when walkers are close to, or moving toward, the vehicle path [12]. Conversely, indistinct or vague pedestrian movements may decrease driver trust as actual likelihood of collisions become more uncertain [12].

Delivery via Augmented Reality

Previous research has examined auditory and visual cues in traditional (head-down) PASs to assist in detection and localization of pedestrian threats [18, 21, 22], but fewer have explored the use of multimodal PAS with head-up HMI. Past work has shown that auditory-only forward crash alerts can provide better recognition of warnings, afford better understanding of a warning’s meaning, and improve the ensuing driver response [23]; however these alerts do not give visual guidance towards threats. In contrast, transparent see-through head-up displays (HUDs) can maintain or direct drivers’ visual attention on the road [24]. Augmented reality (AR) is defined as a, “class of displays that consists primarily of a real environment, with graphic enhancements or augmentation” [25]. In the last decade, the concept of AR is progressively moving toward real-world use, with some technologies already developed for vehicle interfaces with the aim of reducing driver distractions and collisions. AR-enabled HUDs can reduce the amount of time that a driver’s sight deviates from the forward roadway and can further guide driver’s attention to localize threats earlier [22, 26]. Additionally, the presence of a HUD can also influence the allocation of drivers’ visual attention when drivers’ actions are specific to a maneuver, and can command attention more strongly during a decision making phase [27-29]. Past work has also indicated that AR-enabled PAS may be an effective means of keeping drivers’ attention properly on the road and can further help mitigate other functional weaknesses (such as poorly designed controls or a suboptimal field of view of the road) that may exist between the driver and human-machine interface (HMI) [30].

A driver’s acceptance of a PAS is further impacted by the level of intrusion, or annoyance, present when using the alarm itself. It has been recommended that PASs not be equipped with the ability to turn alerts on and off (as drivers will often terminate the system if sufficiently frustrated [15]), but rather with the capacity to control the sensitivity thresholds, auditory intensity, and visual luminance [16]. These adjustments provide users with an important degree of control over a PAS. Technologies that lack these features may feel too invasive and can possibly increase distraction rather than reducing it. It is important therefore to consider and confirm what acceptable limits are for annoyance and disruption when designing PAS support in vehicles. Lastly, the urgency of issued alerts may impact a driver’s resulting perception of a PAS’s effective value. In this regard, a driver’s confidence in PAS may be related to perceived threat, and further depend on the relationship between alert onset time (e.g., as measured by time to collision) and the onset of dangerous pedestrian behavior. For example, earlier PAS alerts may often be interpreted as false positive alarms (the system alerts but no threat is visually detected or acted on) since a driver’s perception is less accurate at far distances [10]. Though early alarms can provide drivers with proper time to react, they increase this likelihood of false positives and often result in subsequent reductions in driver attentiveness [17]. On the other hand, PAS alerts issued too late can be viewed as useless, startling or even detrimental to the driver’s reaction and performance during the braking process [18].

Although AR-enabled HUDs show promise in improving current pedestrian alert systems, improper HMI design may result in significant perceptual issues when used on the road. Perceptual mismatches between what AR HUD designers intend to be perceived and what drivers actually perceive result from a variety of factors which include, but are not limited to: calibration errors, distance discrepancies, restricted field of view, misleading depth cues, and luminance mismatches [25, 31]. Furthermore, drivers operating a HUD-equipped vehicle commonly suffer from both cognitive/attention capture (i.e. unconsciously shifting attention from surrounding world towards the HUD visualization), and perceptual tunneling (i.e. excessive narrowing their peripheral awareness) [32]. The transition of

Thus, PAS Human Machine Interface (HMI) designers strive to balance driver acceptance with system effectiveness to allow enough time for the driver to brake the vehicle safely while also avoiding excessive negative emotional responses [19]. This balance of design between PASs that are effective at producing performance gains while still preserving driver trust is critically important to their

41



AR from testbed to execution is non-trivial. When deploying AR in dynamic real-world settings, the HMI must be both easy to understand and provide tangible benefits, or users might not feel value in the AR system and disengage from it entirely [33]. It is important to carefully consider the many perceptual and cognitive limitations of human-machine interactions with AR systems when designing them into vehicular interfaces or risk subsequent pitfalls during real world use.

in the United Kingdom. All participants were required to have either perfect or corrected-to-perfect vision, which they self-reported prior to testing via screening questionnaire. The survey also screened for vulnerability to simulator sickness, dizzi ness, and common visual conditions in order to minimize the potential impact of common negative side effects (such as nausea or dizziness) experienced driving in a simulator or using a head-up display. All participants were compensated with a £15 (GBP) shopping voucher for their time.

Study Objectives

Materials and Equipment

To provide better insight on the extent to which both audio and AR-enabled HUDs could affect driver trust and behavior, we evaluated driver performance, visual scanning behavior, and attitudes towards the PAS in pedestrian collision scenarios with increasing levels of HMI support. Drivers experienced a series of forward pedestrian collision events while navigating a single lane road within an urban setting using a medium fidelity simulator. Drivers experienced three levels of increasing HMI support (none, audio alerts only, and audio plus visual alerts projected via see-through HUD) during pedestrian collision events with four increasing levels of urgency (operationally defined TTC). We expected that the inclusion of audio and HUD alert support within a PAS would 1) significantly improve driver braking behavior, 2) produce gains (measured by decrease in time) in a driver’s efficiency when scanning for pedestrians, and, 3) support and improve drivers’ acceptance of a PAS. To better understand the dynamics of driver performance and trust in PASs, we examined how braking performance, gaze semantics, and subjective driver confidence varied during pedestrian collision scenarios with increasing alert urgencies (decreasing TTCs) and levels of HMI support.

The study was conducted as part of an international collaboration between Virginia Tech and the University of Nottingham Human Factors labs, with both labs contributing equipment and personnel to its completion. We employed a medium-fidelity driving simulator which projected simulated driving scenarios using STISIM Drive (V3) simulation software suite. Simulated driving scenarios were projected in a 270-degree view around an Audi TT model car using three overhead HD projectors (Figure 1). Drivers operated the simulations from inside the vehicle using an integrated 500RS Thrustmaster force feedback steering and pedal system. The vehicle was equipped with a Pioneer laser scanning HUD mounted above the steering wheel, approximately in line and centered with the drivers’ line of vision (Figure 2). Additionally, during testing all participants wore a pair of SensoMotoric Instrument (SMI) eye tracking glasses to collect gaze pattern and fixation data during drives. The eye trackers were calibrated for each participant prior to data collection. Experimental Design

Participants completed drives for pedestrian alerts in a 3x4 two-factor within-subjects design, including: 1) three levels PAS HMI support (no support, directional audio alerts only, and directional audio plus directional visual alerts via HUD), and 2) four levels of urgency defined by pedestrian onset and quantified using TTC from pedestrian onset. Each drive

METHODS Participants

A total of twenty-four participants (15 males, 9 females) completed the series of driving tasks. Each participant ranged in age from 18 to 55 years, and had a UK driver’s license and at least one year of previous driving experience

Figure 2: The positioning (shown on a sample multi-lane road) of the Pioneer HUD from the driver’s point of view enabled images to be projected on either side of a given lane.

Figure 1: Driving scenarios were simulated using an urban street scenario during which drivers were sequentially presented with randomized crossing pedestrian events.

42


AutomotiveUI ’17, Oldenburg, Germany either stop just before stepping into the road (at TTC = 1s) or continue to cross and directly intersect the vehicle’s path. In both cases, the PAS would issue an alert, resulting in either a false (no walk) alarm or correct (true walk) alarm respectively. We paired each increasing level of urgency (i.e., TTC with target pedestrian) with linearly increasing rates of false positive alarms (Table 1). The assignments shown in Table 1 are based on past research for false positive alarm rates commonly triggered in PASs [39]. To operationalize the false alarm rate, drivers experienced five pedestrian events per drive during which a potential pedestrian collision could occur, with false alarm rate increasing by increasing TTC condition. For the purposes of this specific study, false alarm rate was not considered to be an independent factor but rather a means of increasing the realism of the simulated driving scenario.

Figure 3: The alert shown via HUD (right) was adapted from a master alarm icon (left), using a modified color scheme based on recommendations to maximize saliency.

occurred in an urban city-street scenario, with sidewalks moderately populated with walking pedestrians. HMI Directional Visual Alerts

HMI Support Level

We designed the study’s HUD visual alert using recommendations from past work, utilizing an icon based on a common “master alert” icon [34]. The final visual alert design (see Figure 3) aimed to maximize ease of recognition and saliency when displayed via HUD [35], and used modified coloring and contrast (red exterior, white interior). Per design recommendations for static HUD alerts [21, 36], the visual alert was statically displayed for 4.0 seconds. The HUD displayed visual icons with a directional cue (lefts side or right side) in order to help drivers localize pedestrian threats.

TTC

False Positive Alarm Rate

None | Audio | Audio+HUD

2.0 s

20% (1 in 5)


3.0 s

40% (2 in 5)


4.0 s

60% (3 in 5)


5.0 s

80% (4 in 5)

Table 1. Participants completed twelve drives (3 levels of HMIs x 4 levels of TTCs) with increasing rates of false alarms (pedestrians who walked towards the road but stopped prior to crossing into the vehicle’s path)

We chose the pedestrian TTC levels based on previous design recommendations [40]. Participants completed all TTC and HMI conditions as a complete within-subjects design for twelve drives in total. TTC and HMI conditions were protected against ordering effects using a nested counterbalancing structure; that is, all HMI conditions were counterbalanced using a 3x3 Latin Square design. Within each HMI condition, TTC conditions were also counterbalanced by a Latin Squares (nested in each HMI)".

HMI Directional Auditory Alerts

Our aural HMI conditions employed an audio icon alert (a short car horn sound) that was based on past work recommending non-speech icons for better universal recognition [21]. We projected an audio alert icon from a forward speaker at a sound level intensity of 75dB, using recommended design parameters for forward collision audio warning icons [37]. This alert intensity fell within common noise safety standards (85dB) [38] yet still could be clearly and easily distinguished against measured background noise (55dB) during driving. In the audio only condition, the audio icon was played at the instant the pedestrian began to walk toward the road. In the audio-visual condition, the audio icon was similarly played at the same moment the pedestrian began walking but in conjunction with the visual alert displayed to drivers via HUD. Like the visual alerts, each audio signal contained a directional component (projected via a right aligned speaker or left aligned speaker) to match the side of the road from which the pedestrian approached via an adjacent sidewalk.

Primary Task

Each driving scenario took place in a simulated urban setting with one lane of traffic and no additional vehicles. In each scenario, the driver was instructed to keep their lane and maintain the simulation’s 25 mph speed limit, conveyed using standard European 25 mph speed limit signs displayed approximately every 500 ft. on the road. The simulation itself limited the vehicle to a maximum speed of 25 miles per hour in order to ensure a near-constant vehicle speed. This allowed drivers to have a consistently timed pedestrian event experiences when driving under different TTC conditions. The nature of the driving task forced drivers to divide their attention between inspecting the speedometer to ensure proper speed, keeping their vehcile within the lane markings, and attending to pedestian events and PAS alerts as appropriate.

TTC and False Positive Alarm Rate

Drivers experienced a “pedestrian event” when a target pedestrian located on the sidewalk (and within a set of other “noise” pedestrians) began to walk towards the road. Target pedestrians could walk from either the right or left side of the road, with each instance of orientation being selected randomly. Additionally, to support the ecological validity the simulated PAS usage scenario, target pedestrians could

Secondary Task

As a secondary task, drivers were instrucuted to behave according to common safe driving practices and brake for

43



any unexpcted events, objects or threats which appeared in the road. Participants drove on the road for an initial minium stretch of 500 ft (or approximately for 45 seconds when traveling at 25 mph), after which they would randomly experience a pedestrian event within a subsequent 500 ft range of roadway. At each pedestrian event, drivers stopped and resumed their drive only after they felt sure the roadway was safe to drive on (when the pedestrian had either stopped or finished crossing the road). In the event that the driver collided with a pedestrian, the simulation stopped, played a brief simulated crash sequence and then reset the scene at the same location in the drive. During this time, the experimenter ensured the participant was ready to continue and allowed them to rest if needed before resuming.

We also collected user feedback scores for usability ratings for audio-only and audio+HUD HMI conditions. In this study, the analysis and following discussion focused primarily on differences between HMI conditions and not particularly on differences between TTC conditions. Unless otherwise stated, all data met normality and independence assumptions initially after standard normality tests (ShapiroWilk) or after undergoing logarithmic translation. Gaze Reaction Time

Eye fixation patterns were analyzed using SMI BeGaze 3.5 software. Using semantic gaze mapping analysis in BeGaze, we first defined Areas of Interest (AOIs) within the driver’s field of view and subsequently mapped all driver fixations to these AOIs. Based on the AOIs, drivers fixations were donated as being on: the road, adjacent sidewalk areas (right and left sides), pedestrian threats (right or left approaching), or, visual alerts presented by HUD. The gaze analysis examined only the fixations contained within a defined time interval. The start of the gaze analysis was defined as five seconds before a theoretical collision would occur regardless of pedestrian TTC condition, corresponding to an approximate distance of 187.5 ft at 25 mph. This distance equates to the approximate boundary at which a pedestrian threat could be feasibly detected and identified given the visual fidelity of the simulator. This TTC limit also corresponded to the farthest distance that the driver could experience a possible pedestrian collision scenario (i.e., in conditions where TTC= 5s). We denoted the end of the gaze analysis interval as the instant that a pedestrian threat moved out of the field of view (FOV) captured by the eye tracking forward looking camera.

Procedure

Participants sat in the driver seat of the simulator and were allowed time to familiarize themselves with the driving scenario and the controls. Each participant was told to complete the primary task of navigating through the city while abiding by the city speed limits while also braking for any unexpected events based on their own judgement or due to alert from an enabled PAS. Each participant completed their series of twelve drives after a short practice scenario during which they familiarized themselves with both the audio and visual PAS. During each drive, participants were tasked to navigate the urban road unless an unexpected event occurred, at which time they should behave normally and brake to avoid any potential collisions or road threats. If completing a drive with either an audio or audio + visual HMI, the participant would also experience the alert at the same instance that the pedestrian began to walk toward the road. Between each drive, the experimenter asked participants to rate their trust, desirability and acceptance of the HMI (if present during the drive) via a commonly recommended usability scale [41]. The questionnaires were formatted on a seven point Likert-type scale based on past design recommendations [42]. Between each drive participants were given a rest, and between each HMI block (every four drives), participants were given a slightly longer rest at which point the eye tracking calibration was checked for continued accuracy.

For each trial, participants often fixated on a sequence of AOIs; for example, first fixating on the right sidewalk, then a right approaching pedestrian, then alternating between scanning the road again and the pedestrian until it moved out of view. To analyze eye gaze behavior, we defined gaze reaction time as the time difference between the moment that a pedestrian begins moving toward the road (which coincides with the issued audio and audio+HUD alert

During data collection, the experimenter checked participants for simulator sickness and allowed them the opportunity to cease the experiment at any time due to simulator sickness, fatigue, or any other reason. Total simulated driving time for each participant generally was kept to an hour or less to help minimize fatigue and boredom. At the conclusion of all drives, participants completed a post questionnaire screening for any residual simulator sickness and providing space for general feedback and were provided with a 10£ shopping voucher. RESULTS AND ANALYSIS

Figure 4: Gaze reaction time across both HMI and TTC conditions denotes the difference (in msec) between the instance the pedestrian began walking toward the road and the driver’s first fixation on the potential threat.

To evaluate differences between HMI and TTC conditions, we observed eye gaze behavior as well as driver braking patterns. The details of these metrics are described below in sections 3.1 and 3.2 respectively.

44



Figure 6: Stopping distance (in feet) denotes the stopping distance drivers needed to reach a local speed minimum before resuming road navigation.

Figure 5: Driver brake reaction time (in msec) showed more pronounced differences between HMIs as TTC increased.

conditions) and the first fixation on the pedestrian (i.e. either a right or left approaching pedestrian depending on the trial).

travel distance in both the audio and audio+HUD conditions as compared to the none HMI condition for TTC = 3s, 4s, and 5s. No significant differences between HMIs were found for TTC = 2s (Figure 6).

A two-way repeated measures ANOVA test on gaze reaction time revealed a significant main effect for TTC ( F(3,48.7) = 23.1, p < 0.001), but found no significant effects for HMI (Figure 4). In order to account for missing data, we used Satterthwaite approximation for degrees of freedom.

Subjective Feedback

We performed paired sampled t-tests on subjective ratings from the following usability scale questions:

Braking Performance

• Trust – “Overall how much do you trust the system?” • Confidence – “How confident are you that the system will be able to cope with all situations in the future?” • Annoyance – “How annoying was the system?” • Desirability – “How likely would you be to use the system if it was available in your own car?” Similar ratings levels were found for comparisons between levels of HMI support for the majority of the usability scales. However, a significant difference was found in ratings of confidence (t(23) = 2.14, p = .043), between the audio and audio+HUD HMI conditions (M=6.20, SD=1.83), indicating that participants had more confidence in the audio and HUD combination than the audio only HMI.

All braking behavior responses are shown in Figures 5 and 6. A two-way repeated measure ANOVA revealed a significant main effect of HMI (F(2,46.5) = 36.3, p < 0.001), TTC (F(3, 65.6) = 295.01, p < 0.001) and interaction effects between HMI and TTC (F(6,132.3) = 6.0, p < 0.001). We operationally defined a driver’s breaking reaction time as the difference between the first moment of onset of the pedestrian event (coinciding with when an alert occurred when an HMI was present) until the moment that the brake pedal inclined over 5 degrees, and obtained specific time difference stamps using logging data from STISIM simulation output. Post-hoc comparisons were performed using the Tukey adjustment for multiple comparisons. Results indicated that braking reaction time for both audio only and audio+HUD HMI conditions was significantly faster as compared to the none HMI condition for TTC= 3s, 4s, and 5s (p < 0.01). Additionally, significant differences were detected between audio only and audio+HUD HMI conditions for TTC= 5s (p = 0.007) (Figure 5). However, no significant differences were found between any HMI conditions for TTC = 2s.

DISCUSSION

When comparing results across the three categories of collected driver behavior measures, several interesting trends come to light. Although drivers are not particularly prone to changes in gaze response times to fixate on target pedestrians, their braking performance suggests drivers are still able to experience some performance gains in braking regardless of whether or not they demonstrate concurrent improvements in visually localizing pedestrian. In other words, drivers are capable of actually reacting to threats faster using PASs despite being unaware or unable to pinpoint where the actual pedestrian threat exists in the driving scene. We suspect that these differences between gaze and braking behavior occur because drivers have a tendency to immediately act upon the onset of PAS alerts regardless of the HMI modality. This finding is especially interesting as drivers did not actually rate the PASs as fostering high trust . However, in spite of these rather low subjective ratings, drivers actually do exhibit an inherent trust in both the audio and audio+HUD HMIs as inferred

Additionally, to further characterize driver braking behavior after reacting to each pedestrian event, we also calculated stopping distance required for each event. We define stopping distance as the distance the car traveled in the simulation from the moment of start of the brake pedal incline (i.e. the moment of braking) to the moment at which the vehicle stopped (or reached a minimum speed before beginning to accelerate again, indicating the threat had passed). Similarly, we found a significant main effect of HMI (F(2,46.3) = 25.3, p < 0.001), TTC (F(3, 63.3) = 334.8, p < 0.001) and interaction effects between HMI and TTC (F(6,125.3) = 6.4, p < 0.001). Specifically, drivers took less

45



through both direct braking measures: braking reaction time and stopping distance. Based on our analyses, at times, drivers would act upon the alert before visually confirming that the threat was present. Additionally, the results from both braking reaction time and stopping distance show that gains in braking performance are mediated by the urgency of the collision situation, and these differences in gains become more pronounced as TTC increases. That is, drivers find greater benefit from PAS HMI support when they have more time to process the HMI alert but also when they cannot see the threat as well. This suggests that drivers are more likely to rely or put their trust in the PASs as their own ability to visually confirm a threat diminishes.

experienced in both their visual scanning and braking behavior. Specifically, although drivers did not actually receive any notably better gains from the audio-only and audio+HUD alerts, they nevertheless reported a greater level of confidence in the combined alerts PAS to handle collision events than the audio PAS support alone. This suggests that drivers may feel safer and more confident with PAS present; especially when PAS includes a visual cue, even if that visual cue does not actually contribute value to their localization or braking ability. The inclusion of a visual HUD element is therefore likely to increase overall driver acceptance of PAS technology, even at a lower fidelity where it may be functionally ineffective.

One interesting finding is how drivers’ braking reaction time diverges from being consistent with their stopping distance across HMI conditions at TTC= 5s; the same point in time which target pedestrians become visibly indiscernible in our simulator. Here we see that the presence of audio alerts in conjunction with visual HUD alerts actually do produce significant gains in drivers’ braking reaction time as compared to their braking reaction gains from audio alerts alone. We speculate that the simulated nature of the study could explain these divergences in performance at long range (when the fidelity of the simulator would be outperformed by a driver’s visual acuity in the real world). at TTC= 5s, drivers’ visual uncertainty reached a threshold and they likely chose to employ a “rather safe than sorry” strategy and simply complied with the alert given to them by the visual HMI cues versus hovering over the brake as they continued to visually scan for hazards. After their initial braking reaction, drivers’ stopping distance remained more consistent between both audio and audio+HUD HMI conditions. It is important to note however that performance differences between HMI conditions still become more pronounced for both braking measures as TTC increased. This may indicate a relationship between a driver’s visual uncertainty of a collision threat and their willingness to rely on or trust a visually-oriented PAS HMI alert. In contrast, HMI presence does not actually appear impact of drivers’ glance and brake performance significantly in collision scenarios with high urgency (TTC = 2s). Therefore, PAS support at close proximity to threats may not actually impact drivers’ behavior as much as past work has indicated.

LIMITATIONS

Given the environment and level of available resources during design and data collection, some limitations with this study exist. Because of the limited field of view of the Pioneer HUD, visual warning graphics could not be displayed directly on or next to pedestrian threats. Therefore, participants did not experience truly conformal AR graphics but rather directional static graphics more akin to traditional HUDs. We were surprised that no significant differences were detected for gaze reaction time across HMI conditions, and believe that it is likely that using geospatially conformal graphics to more accurately direct visual attention toward pedestrians could have produced more clear differences in both gaze and braking performance across HMI types. However, small fields of view in existing HUD prototypes continue to significantly limit the extent to which conformal graphics can be tested. Another possible explanation for the lack of differences in gaze behavior was that drivers’ visual behavior was already finely tuned during the experiment; that is, drivers expected pedestrian events and were therefore already scanning effectively. Consequently, no differences were apparent in this simulated study, but may be more evident in a real-world situation (when the rate and number of warnings/interventions would be much lower, and unexpected). Low lighting may also have significantly impacted drivers’ perceptions of simulated pedestrians since the experiment took place in a simulator and not in a real world location or outdoor environment. There is a high likelihood that the saliency of the images projected via HUD would greatly diminish in an outdoor testbed due to lower luminance contrast and fluctuations due to effects of natural lighting [43]. Additionally, because no other vehicles were present within each simulation, the scenario tested may have been somewhat simplified compared to a real world counterpart. A following study using a car-following task or presenting noise using additional simulated vehicles could improve the validity of existing results. The design itself could be improved; for example, the nested counterbalancing of TTC within HMI conditions could have been instead designed using full counterbalancing, which may have better reduced expectancy and ordering effects. However, we believe that blocking HMI conditions allowed driver performance to be generally more consistent as they

Some interesting findings can be noted in regards to driver perceptions and their interaction with actual performance. Drivers often commented that they did not perceive a difference between a false alarm (during which no collision could occur) and a true alarm (during which a collision was actually possible), which may indicate that drivers generally overlooked small details and concentrated mainly on maximizing performance in their drives. It is likely that drivers’ general unfamiliarity with a simulator or perception of being in a “game scenario” may have caused them to treat and thus perceive their drives slightly differently than a real world scenario. Additionally, Drivers’ perceived gain from PASs were not consistent with the actual gains they

46



only used one type of HMI at a time (which they then used within alternating levels of urgency). Ultimately the study was forced to balance both the benefits of maintaining consistency for driver experience and minimizing expectancy and learning effects.

program from Virginia Tech University (NSF award 1261162) and the European Community’s Eighth Framework Program (Horizon2020) (agreement no. 634149) as part of the Proactive Safety for Pedestrians and Cyclists (PROSPECT) project. We would like to thank Nicole Sanderlin and Jack Lesko for their support and assistance throughout the project and Catherine Harvey for assisting in the experimental design.

It should also be noted that this study examined these PASs in context to drivers perceiving them as novel technologies with little or no prior exposure to in-vehicle technologies such as HUDs using AR graphics. A longitudinal study could likely better examine possible long term effects of using AR enabled PASs. Additionally, the presence of “false alarms” (i.e. pedestrians who walked toward, but not fully across the road) may have impacted drivers’ perception of those events and differentiated them to a degree from fullwalk scenarios. However, as noted before feedback showed that most drivers did not actually notice any differences between the two scenarios, indicating they perceived the act of initially walking as a defining characteristic of a roadway “threat”.

REFERENCES

1.

2. 3.

4.

CONCLUSIONS AND OUTLOOK

The results of this study show that the value of each PAS HMI is highly dependent on the type of gain that a designer is most interested in supporting. Analyses show that drivers may not actually experience improvements in visual scanning or threat localization even with the addition of PAS support. Furthermore, their lack of gains in gaze reaction time is unaffected by the addition of directional auditory and visual cues. It is possible that more rigorous visual cues like conformal or conformal hybridized graphics (such as a static image with more specific directional cues) may be needed to sufficiently justify the use of visually-based PASs if designers prioritize gains in visual attention. However, our results indicate that an audio+HUD PAS HMI is still able to contribute to a driver’s sense of wellbeing and are therefore likely to contribute to higher levels of driver acceptance overall even without associated gains in performance. The mere presence of any PAS (be it audio or visual) is capable of improving drivers’ braking performance despite not improving their localization abilities, and may be inherently relied on by a driver regardless of whether they show real improvement in actually detecting threats earlier. However, designers of PAS HMIs must be aware that both the performance gains and confidence that drivers exhibit when using a threat detection system may not necessarily correspond to an increased ability to actually detect those same threats the system seeks to identify. Though PAS systems with visual cues may not hinder driving performance as a whole, by no means should they be universally relied on when attempting to improve and guide visual attention towards dangers on the road. Continued research should focus on identifying specific elements within a PAS that can more authentically support a driver’s visual capabilities.

5.

6.

Administration, N.H.T.S., Traffic safety facts 2013 data. Rural/urban comparison. Washington, DC: Department of Transportation, 2015. Graves, F., et al., Reported Road Casualties Great Britain: 2013 Annual Report. 2014. Clifton, K.J., C.V. Burnier, and G. Akar, Severity of injury resulting from pedestrian–vehicle crashes: What can we learn from examining the built environment? Transportation research part D: transport and environment, 2009. 14(6): p. 425-436. Beck, L.F., A.M. Dellinger, and M.E. O'neil, Motor vehicle crash injury rates by mode of travel, United States: using exposure-based methods to quantify differences. American Journal of Epidemiology, 2007. 166(2): p. 212-218. Yanagisawa, M., E. Swanson, and W.G. Najm, Target Crashes and Safety Benefits Estimation Methodology for Pedestrian Crash Avoidance/Mitigation Systems. 2014. Mobileye Foward Collision Warning (FCW). [cited 2016 May 31]; Available from: http://www.mobileye.com/technology/applications/vehicledetection/forward-colision-warning/.

7.

8.

9.

10.

11.

ACKNOWLEDGEMENTS

12.

This study was jointly funded by the NSF funded International Research Experience for Students (IRES)

47

Broggi, A., et al., A new approach to urban pedestrian detection for automatic braking. Intelligent Transportation Systems, IEEE Transactions on, 2009. 10(4): p. 594-605. Zhang, S., et al., How Far are We from Solving Pedestrian Detection? arXiv preprint arXiv:1602.01237, 2016. Dollar, P., et al., Pedestrian detection: An evaluation of the state of the art. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2012. 34(4): p. 743-761. Abe, G. and J. Richardson, Alarm timing, trust and driver expectation for forward collision warning systems. Applied Ergonomics, 2006. 37(5): p. 577586. Large, D.R., Harvey, C., Burnett, G., Merenda, C., Leong, S., Gabbard, J. 2017, Exploring the relationship between false alarms and driver acceptance of a pedestrian alert system during simulated driving. Road Safety and Simulation International Conference 2017, 2017. Källhammer, J.-E. and K. Smith, Assessing contextual factors that influence acceptance of pedestrian alerts


13.

14.

15.

16.

17.

18.

19.

20.

21.

22.

23.

24.


by a night vision system. Human Factors: The Journal of the Human Factors and Ergonomics Society, 2012. 54(4): p. 654-662. Källhammer, J.-E., H. Toghyani, and K. Smith. A technique for evaluation of pedestrian warning conditions with high driver acceptance. in Proceedings. 23rd International Technical Conference on the Enhanced Safety of Vehicles (Seoul, Republic of Korea). 2013. Langham, M. and N. Moberly, Pedestrian conspicuity research: a review. Ergonomics, 2003. 46(4): p. 345363. Lubbe, N. and J. Davidsson, Drivers’ comfort boundaries in pedestrian crossings: A study in driver braking characteristics as a function of pedestrian walking speed. Safety Science, 2015. 75: p. 100-106. McCallum, M., Integrated Vehicle-Based Safety System heavy truck driver vehicle interface (DVI) literature review. 2006. De Boer, J., et al. The accuracy and timing of pedestrian warnings at intersections: The acceptance from drivers and their preferences. in Intelligent Transportation Systems (ITSC), 2010 13th International IEEE Conference on. 2010. IEEE. Lee, J.D., et al., Collision warning timing, driver distraction, and driver response to imminent rear-end collisions in a high-fidelity driving simulator. Human Factors: The Journal of the Human Factors and Ergonomics Society, 2002. 44(2): p. 314-334. Lubbe, N., Brake reactions of distracted drivers to pedestrian Forward Collision Warning systems. Journal of Safety Research, 2017. 61: p. 23-32. Lee, J.D. and B.H. Kantowitz, Perceptual and cognitive aspects of intelligent transportation systems. Human factors in intelligent transportation systems, 1998. Reader, N.T., Development and validation of functional definitions and evaluation procedures for collision warning/avoidance systems. 1999. Liu, Y.-C. and M.-H. Wen, Comparison of head-up display (HUD) vs. head-down display (HDD): driving performance of commercial vehicle operators in Taiwan. International Journal of Human-Computer Studies, 2004. 61(5): p. 679-697. Sayer, T.B., J.R. Sayer, and J.M. Devonshire, Assessment of a driver interface for lateral drift and curve speed warning systems: mixed results for auditory and haptic warnings. Proceedings of Driving Assessment, 2005: p. 218-224. Livingston, M.A., et al. Resolving multiple occluded layers in augmented reality. in Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality. 2003. IEEE Computer Society.

25.

26.

27.

28.

29.

30.

31.

32.

33.

34.

35.

36.

37.

38.

48

Drascic, D. and P. Milgram. Perceptual issues in augmented reality. in Electronic Imaging: Science & Technology. 1996. International Society for Optics and Photonics. Kim, H., et al. Look at Me: Augmented Reality Pedestrian Warning System Using an In-Vehicle Volumetric Head Up Display. in Proceedings of the 21st International Conference on Intelligent User Interfaces. 2016. ACM. Eyraud, R., E. Zibetti, and T. Baccino, Allocation of visual attention while driving with simulated augmented reality. Transportation research part F: traffic psychology and behaviour, 2015. 32: p. 46-55. Rane, P., et al. Virtual Road Signs: Augmented Reality Driving Aid for Novice Drivers. in Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2016. SAGE Publications Sage CA: Los Angeles, CA. Kim, H., J.D. Isleib, and J.L. Gabbard. Virtual Shadow: Making Cross Traffic Dynamics Visible through Augmented Reality Head Up Display. in Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2016. SAGE Publications Sage CA: Los Angeles, CA. Shneiderman, B., Designing the user interface: strategies for effective human-computer interaction. 2010: Pearson Education India. Gabbard, J.L., G.M. Fitch, and H. Kim, Behind the Glass: Driver Challenges and Opportunities for AR Automotive Applications. Proceedings of the IEEE, 2014. 102(2): p. 124-136. Savino, M.R., Standardized names and definitions for driving performance measures. 2009, TUFTS UNIVERSITY. Colley, A., et al. Insights from Deploying SeeThrough Augmented Reality Signage in the Wild. in Proceedings of the 4th International Symposium on Pervasive Displays. 2015. ACM. Thoma, S., T. Lindberg, and G. Klinker, Evaluation of a generic warning for multiple intersection assistance systems. Human factors, security and safety, 2009: p. 173-188. Campbell, J.L., et al., In-Vehicle Display Icons and other Information Elements. Volume I: Guidelines. 2004. Huang, C.H., Chao, C. W., Tsai, T., & Hung, M. H, The Effects of Interface Design for Head-Up Display on Driver Behavior. Life Science Journal, 2013. 10(2): p. 2058-2065. Campbell, J.L., et al., Crash warning system interfaces: human factors insights and lessons learned. 2007, US Department of Transportation, National Highway Traffic Safety Administration. Salvendy, G., Handbook of human factors and ergonomics. 2012: John Wiley & Sons.

Session 2: Pedestrians: Communication and Alerts 39.

40.

41.

42.

43.


Keller, C.G. and D.M. Gavrila, Will the pedestrian cross? A study on pedestrian path prediction. Intelligent Transportation Systems, IEEE Transactions on, 2014. 15(2): p. 494-506. Hoffmann, E.R. and R.G. Mortimer, Drivers' estimates of time to collision. Accident Analysis & Prevention, 1994. 26(4): p. 511-520. Brooke, J., SUS-A quick and dirty usability scale. Usability evaluation in industry, 1996. 189(194): p. 47. Vagias, W.M., Likert-type scale response anchors. Clemson International Institute for Tourism & Research Development, Department of Parks, Recreation and Tourism Management. Clemson University, 2006. Merenda, C., et al. Effects of real-world backgrounds on user interface color naming and matching in automotive AR HUDs. in Perceptual and Cognitive Issues in AR (PERCAR), IEEE VR Workshop on. 2016. IEEE.

49