Method Development and Interaction Cognitive Driver

C OVER STORY Automated Driving

Method Development and Interaction Cognitive Driver Take-Over Ability After Piloted Driving Further development progress in the field of automated driving continues to open up new free-doms for the driver, such as a higher engagement in non-driving related (NDR) tasks during auto-mated driving. However, the associated reduction in attention level begs the question of when the driver is cognitively back “in the loop” and suitably able to take over the driving task, in the event of a take-over request. The significance of the cognitive processes of situational processing and decision-making have been examined within the scope of various studies. Volkswagen and WIVW GmbH have developed an innovative methodology for recording cognitive take-over ability.

AUTHORS

Dr. Ina Othersen Project Manager focusing on the Human Machine Interface for Augmentation and Driving Functions at Volkswagen AG in Wolfsburg (Germany).

Dr. Ina Petermann-Stock Head of Subdepartment in the Field of Lighting and Display Technologies at Volkswagen AG in Wolfsburg (Germany).

28

Dr. Nadja Schoemig Senior Researcher at WIVW GmbH (Würzburg Institute for Traffic S ciences) (Germany).

Tanja Fuest, M.Sc. is a Former Employee of Volkswagen AG and Master’s Graduate of the Technische Universität Braunschweig (Germany).

MOTIVATION

Increasing automation in the auto motive context, which performs large parts of the driving task autonomously with no further need for monitoring activity on the part of the “pilot” [1], allows the driver to temporarily engage in secondary activities [2, 3] or to relax. This can result in the driver being “out of the loop” while driving [4]. Banks & Stanton [5] describe this state as a driver no longer having control over

© chombosan | iStock

their car, while simultaneously being in a passive state with a low attention level. The driver’s only task is to be aware of possible system transitions and to take over manual driving within a time frame of a few seconds. It can be assumed that the amount of time the driver requires to be back in the loop depends on how far out of the loop they were and how demanding the take-over situation is. Take-over ability is therefore significantly influ enced by the driver’s state and by the ATZ elektronik worldwide

necessary cognitive processes of situa tional processing and decision-making. The response to a take-over request (TOR) is typically described in the body of literature with indi-vidual visual or motor actions [6]. The driver must first perceive the TOR and appreciate the present situation (visually back in the loop), and return their hands to the steering wheel and their feet to the pedal cluster (motorically back in the loop). While doing this, the driver must integrate the driving information

and decide on the best response. Cur rent research findings document these visual and motor reaction times with values of 0.9 to 1.1 s for the first glances toward the road, and 1.8 to 2.6 s for gripping the steering wheel [7, 8]. This results in total take-over times of 3 to 9 s [7, 8, 9, 10]. These reaction times can be impacted both by a NDR task [11, 12, 13] and by increased stress or situational complexity [10, 11, 13]. The decision-making processes for cognitive take-over ability, which are

29


FIGURE 1 Schematic sequence of a test drive (© VW)

represented by information processing theory [14] and lie between p erceptual processing and executing a response [15], extend beyond purely visual per ception of the situation and have not been considered to date. Therefore, the objective of this research contribution is to enhance take-over ability at the point when the driver exhibits cogni tive readiness after a period of auto mated driving, and to make this mea surable. In addition, the influence of situational complexity, the performance and positioning of a NDR task, and its ability to be interrupted at the same time as a TOR must be considered in greater detail. METHOD DEVELOPMENT

In developing a procedural method for the experimental recording of cognitive take-over ability, four pilot studies were carried out in coopera tion with the Würzburg Institute for Traffic Sciences (WIVW GmbH). Building on the aforementioned t heoretical principles, an innovative method was used for this pur-pose, with the aid of driving-simulator stud ies: As an a dditional secondary task, the test person had to verbalize the necessary action (evasion direction) resulting from the decision-making process when faced with a TOR.

30

TEST SETTING

The implemented automation was a prototype L3 system, which com pletely took over lateral and longi tudinal control on a section of the f reeway with a set speed of 100 km/h. The driver did not have to monitor the system and was able to remove their hands from the steering wheel. The Human Machine Interface (HMI) con sisted of icons and text instructions in the instrument cluster, as well as acoustic elements. The test drive took place on a three-lane freeway, and the driver was instructed to drive in the center lane. The TOR was issued after 1 300 m of automated driving, based on a closure of the driver’s own lane. Starting from the center lane, the driver had to make a lane change to the left or right. The test person was also instructed to verbalize the eva sion direction (“left” or “right”) imme diately after making a decision. To heighten the effect of the driver being out of the loop, projection of the driv ing scenery was deactivated when the system was activated (occlusion of the visual scene). After a driving time of 47 s with the system, projec tion was re-activated at the same time as the TOR, which consisted of visual and acoustic feedback, FIGURE 1. The lane change to be carried out

in the appropriate direction was intended to be performed de-pending on the s urrounding traffic and the d ifferent relative speeds, so that no hazard arose from the ego vehicle. By varying the surrounding traffic at the moment of the lane change, three different degrees of complex ity were generated for the take-over situation (easy, moderate, difficult). The NDR task in this case was a quiz. Based on the quiz show “Who Wants to Be a Millionaire?”, a correct response had to be selected from three multiple-choice answers to the corre sponding question, by means of an entry on the touch screen in the c enter con sole. The quiz was only a ctivated during automated driving, and it was deacti vated automatically during the TOR. PILOT STUDIES

The test setting and its iterative adapta tion were examined in four pilot studies. The situational complexity (three levels), the evasion direction (left vs. right), and concentration on a NDR task (only in preliminary study 4; with vs. without quiz) were implemented as independent variables. The following dependent vari ables were considered: –– Reaction time until the first glance at the left or right exterior mirror (via eye tracking)

–– Reaction time until hands-on (via capacitive sensors in the steering wheel) –– Reaction time until the first spoken response (analysis of the recorded audio files) –– Reaction time until the first steering response (steering angle > 3°). In the first two pilot studies, the described procedure was carried out with two different take-over times of 5 and 8 s. Six test persons (employees of WIVW GmbH) participated in each of these studies. The results show that the spoken response and the steering response are sensitive to and react pro portionately with the complexity of the situation. It was also determined during the longer take-over time that the spoken response took place 300 ms before the steering response. In the 5-second TOR, both responses often took place simulta neously (with an average difference of 100 ms, FIGURE 2). In order to evaluate any potential influ ences of the take-over time by the spoken response, a third pilot study with six test persons was conducted without any speech output. Statistically no significant influences on hands-on times and on steering reaction times could be shown. The fourth pilot study examined any potential influencing of cognitive take-over ability by a NDR task. 15 test persons (average age 35 years, 47 %

FIGURE 2 Reaction times for hands-on, spoken response, and steering response in comparison (5-second vs. 8-second TOR) (© VW)

women) participated in the study. The situational complexity had an influence here on the point in time of the spoken response (F[2;28] = 71.071; p < .000) and on the steering response (F[2;28] = 32.629; p < .000), FIGURE 3. In processing the NDR task, an earlier glance at the right-hand exterior mirror (F[1;9] = 7.291; p < .05) and quicker hands-on times (F[1;14) = 9.414; p < .01)

were identified, but no influence on the spoken response. MAIN STUDY

Building on the findings from developing the methodology, an additional study was implemented in the static driving simula tor of Group Research at Volkswagen AG. The test setting was kept identical for this

FIGURE 3 Reaction times for all dependent variables, depending on situational complexity (© VW)

ATZ elektronik worldwide

31


study. In the main study, there was a more detailed consideration of the interactions between cognitive take-over ability, the situational complexity, and various dimensions of a NDR task: –– Execution of a NDR task (“within”; with vs. without quiz) –– Placement of the NDR task (“between”; on-board infotainment vs. tablet) –– Interrupt capability (“between”; with vs. without automatic interruption of the NDR task during the TOR). 64 test persons (average age 36 years, 36% women) participated in the study. They were recruited from the Volkswa gen AG pool of test persons. The results showed that the situational complexity (F[1;342] = 9.56; p < .000) and the execution (F[1;342] = 21.06; p < .000) and positioning of the NDR task (F[1;342] = 28.30; p < .000) have an inf lu-ence on cognitive take-over ability. In a more detailed consideration of the lat ter two factors, FIGURE 4, it was shown that the spoken response was later when playing the quiz on a tablet (3.1 s) than with the conditions of on-board infotain ment (2.7 s) or without a quiz (2.7 s). Likewise, all other reactions took place later when engaging in a NDR task on a tablet as compared to on-board infotain ment. However, no significant difference could be determined for the factor of interrupt capability (F[1;342] = 1.11; n.s.). DISCUSSION

The aim of this contribution was to examine the effects of cognitive aspects on the driver’s take-over ability. Factors such as the situational complexity of the take-over situation and the state of the driver influence the cognitive pro cesses of information processing and decision-making, and are therefore key factors for take-over times. As a result, the following implications can be made, among others, for the use of the developed methodology (see also [16]). The occlusion of the driving scene to generate an out-of-the-loop feeling for the test persons proved to be successful, despite the very short driving time of 47 s. Likewise, it was possible to vary the take-over complexity very well by varying the number and relative speeds of other vehicles. In addition, in the main study with the underlying test setting, it was shown that preoccupation with a NDR task and

32

FIGURE 4 Reaction times for all dependent variables, depending on the execution and placement of the NDR task (© VW)

the position of this activity have an impact on cognitive take-over ability. The results proved that a driver requires a longer time for the take-over process when engaging in a NDR task on a tab let. Thus, the findings from p revious studies with respect to longer take-over times due to a secondary activity can be reproduced and are expanded by the driver’s cognitive state [11, 13]. The results illustrate that the spoken response is well suited as a measure for the driver’s cognitive take-over ability, and it can model the process of selecting a response before the actual motor reac tion takes place. The method itself does not appear to influence the phases of the take-over process, which is an additional indication for an independent process of cognitive processing. This method is also applicable to other situations that require a decision between two or more options. This research contribution helps to better understand the underlying psy chological constructs in the take-over process during automated driving. The objective parameters affecting the pro cess of the take-over, from the orienta tion response through to stabilizing the vehicle, can also be supplemented with the cognitive component. This enables optimum adaptation of human-machine interaction to the user’s abilities.

REFERENCES [1] SAE International (2014). Taxonomy and definitions for terms related to on-road automated driving systems (SAE Standard J3016). Available at http:// standards.sae.org/j3016_201401/ [2] Ebner, H.-T. (2013). Motivation und Handlungsbedarf für Automatisiertes Fahren: DVR Kolloqui-um Automatisiertes Fahren. Bonn. Available at http://www.dvr.de/download2/p3745/3745_0.pdf [3] Wickens, C. D. & Holland, J. G. (2000). Engineering Psychology and Human Performance (3 rd edition) New Jersey: Prentice-Hall Inc. [4] Endsley, M. R. & Kiris, E. O. (1995). The Out-ofthe-Loop Performance Problem and Level of Control in Automation. Human Factors, 37 (2), 381–394. [5] Banks, V. A. & Stanton, N. A. (2014). Hands and Feet Free Driving: Ready or Not? In AHFE Conference (publisher), Advances in Human Aspects of Transportation, Part II (vol. 8, pp. 55-63). Louisville, KY: AHFE Conference. [6] [6] Zeeb, K., Buchner, A., & Schrauf, M. (2015). What determines the take-over time? An integrated model approach of driver take-over after automated driving. Accident Analysis & Prevention, 78, 212-221. [7] Damböck, D. (2013). Automationseffekte im Fahrzeug - von der Reaktion zur Übernahme (Dissertation). Technical University of Munich, Munich. [8] Gold, C. & Bengler, K. (2014). Taking Over Control from Highly Automated Vehicles. In AHFE Con-ference (publisher), Advances in Human Aspects of Transportation, Part II (vol. 8, pp. 64-69). Louis-ville, KY: AHFE Conference. [9] Damböck, D., Farid, M., Tönert, L. & Bengler, K. (2012). Übernahmezeiten beim hochautomatis-ierten Fahren. At 5th Conference on Driver Assistance – Focus: Connectivity. Munich: TÜV SÜD Akademie GmbH. [10] Petermann-Stock, I., Hackenberg, L., Muhr, T. & Mergl, C. (2013). Wie lange braucht der Fahrer eine Analyse zu Übernahmezeiten aus verschiedenen Nebentätigkeiten während einer hochautomatisierten Staufahrt. At 6 th Conference on Driver Assistance Systems – The Path to Automated Driving. Munich: TÜV SÜD Akademie GmbH.

[11] Makishita, H. & Matsunaga, K. (2008). Differences of drivers’ reaction times according to age and mental workload. Accident Analysis & Prevention, 40 (2), 567–575. doi: 10.1016/j. aap.2007.08.012 [12] Winner, H., Hakuli, S., Lotz, F. & Singer, C. (2015). Handbuch Fahrerassistenzsysteme. Wiesbaden: Springer Fachmedien Wiesbaden. doi: 10.1007/978-3-658-05734-3


[13] Young, K., Regan, M. & Hammer, M. (2007). Driver distraction: A review of the literature. In I. J. Faulks, M. Regan, M. Stevenson, J. Brown, A. Porter & J. D. Irwin (publisher), Distracted driving (pp. 379-405). Sydney. [14] Freitag, C. (2014). Entwicklung, Informations verarbeitungsansätze. In M. Wirtz & J. Strohmer (publisher), Dorsch - Lexikon der Psychologie. Bern: Hans Huber.

[15] Streitz, N. A. (1987). Die Rolle der Psychologie. In K.-P. Fähnrich (publisher), Software-Ergonomie (vol. 5, p. 43-53). Munich and Vienna: Oldenbourg. [16] Othersen, I., Petermann-Stock, I., Schömig, N., Neukum, A. & Fuest, T. (in press). Cognitive driver takeover capability after piloted driving - method development and interaction with a side task. In VDI Wissensforum GmbH (publisher), VDI Congress on Electronics in Vehicles, Bonn. Düsseldorf: VDI Verlag GmbH.

33


34


35