Institute of Ergonomics
Technische Universität München
Testing Open-Source Implementations for Detection Response Tasks Michael Krause, Antonia Conti, Moritz Späth, Klaus Bengler Institute of Ergonomics Technische Universität München, Germany
Institute of Ergonomics
Technische Universität München
Overview • Motivation • Experiment/Setup • Results • Conclusion • Outlook
12th September 2014
Michael Krause, Interaccion 2014
2
Institute of Ergonomics
Technische Universität München
Motivation • Measuring the effects of cognitive workload ISO/TC 22/SC 13/WG 8 ISO/CD 17488 Detection Response Tasks (DRT) => continuous stimulus - reaction test • Basic concept: If the reaction time (RT) for the DRT is prolonged the subject likely has a higher cognitive workload • Expensive proprietary equipment is available => Let’s try it free and OpenSource (modifiability often need for experiments, able to trace back functionality, no black box)
12th September 2014
Michael Krause, Interaccion 2014
3
Institute of Ergonomics
Technische Universität München
Arduino-DRT • Implementation of the DRT protocol on an Arduino microcontroller board • Documentation, schematics and source code available: http://www.lfe.mw.tum.de/arduino-drt/
12th September 2014
Michael Krause, Interaccion 2014
4
Institute of Ergonomics
Technische Universität München
Smartphone - Mobile Detection Task (MDT) • Implementation of the DRT protocol on an Android smartphone • The experimental setup via smartphone is not within the ISO drafts! Therefore, we call it Mobile Detection Task (MDT) • Documentation, schematics and source code available: http://www.lfe.mw.tum.de/mdt
12th September 2014
Michael Krause, Interaccion 2014
5
Institute of Ergonomics
Technische Universität München
Comparision
(Regular) Arduino DRT
Smartphone MDT
Visual stimulus
LED on a cap
Smartphone display
Tactile stimulus
Tactor taped to shoulder
Palm rest on smartphone loosely
Response
Reponse button on the finger
Thumb on touch screen
12th September 2014
Michael Krause, Interaccion 2014
6
Institute of Ergonomics
Technische Universität München
Procedure • Conditions (3) – Single condition: baseline (Detection Response Task only) – Double condition: DRT + Driving (AAM) – Triple condition: DRT + Driving (AAM) + Cognitive task (2-back task) • On different systems (Arduino-DRT, Smartphone-MDT) • In different modes (Visual or Tactile) • Full factorial, randomized, within-subject design • Each condition 1 Minute (~15 stimuli) 12th September 2014
Michael Krause, Interaccion 2014
7
Institute of Ergonomics
Technische Universität München
Static Driving Simulator
12th September 2014
Michael Krause, Interaccion 2014
8
Institute of Ergonomics
Technische Universität München
Participants • 36 subjects (50% male; 50% female) • Age: 19 - 31 years; M = 22 (SD 3) • Mainly students and researchers
12th September 2014
Michael Krause, Interaccion 2014
9
Institute of Ergonomics
Technische Universität München
Reaction times for tactile stimulus presentation on both systems
12th September 2014
Michael Krause, Interaccion 2014
10
Institute of Ergonomics
Technische Universität München
Reaction times for visual stimulus presentation on both systems
12th September 2014
Michael Krause, Interaccion 2014
11
Institute of Ergonomics
Technische Universität München
Effect size
12th September 2014
Michael Krause, Interaccion 2014
12
Institute of Ergonomics
Technische Universität München
Effect size
about the same effect sizes
12th September 2014
Michael Krause, Interaccion 2014
13
Institute of Ergonomics
Technische Universität München
T-test probabilities • Higher bar => lower probability (logarithmic)
12th September 2014
Michael Krause, Interaccion 2014
14
Institute of Ergonomics
Technische Universität München
T-test probabilities • Higher bar => lower probability (logarithmic) • Smartphone setup typically 1-2 orders of magnitude inferior and fails in one condition to reach alpha level 5% (5.2%)
12th September 2014
Michael Krause, Interaccion 2014
15
Institute of Ergonomics
Technische Universität München
Correlation of individual subject‘s RTs between conditions
12th September 2014
Michael Krause, Interaccion 2014
16
Institute of Ergonomics
Technische Universität München
Correlation of individual subject‘s RTs between conditions
used smartphone setup do not reach high correlations of RTs
12th September 2014
Michael Krause, Interaccion 2014
17
Institute of Ergonomics
Technische Universität München
Conclusions • Tests with the smartphone (MDT) shows the same trend (longer RTs) as the classical DRT and can detect cognitive effort. Nevertheless, for very similar conditions a higher number of participants is needed. Regular DRT setup is significant more quickly, with less participants and measurement points, than the MDT. • Constant low correlations in RT of the test subjects between similar experimental conditions for MDT indicates, that simple attachment of the smartphone to the steering wheel is a lower quality setup relative to the classical DRT setup => Improving the setup/mounting seems crucial. 12th September 2014
Michael Krause, Interaccion 2014
18
Institute of Ergonomics
Technische Universität München
Outlook (MDT improvements for the next experiment) 1) External response button
2) Mounting direct to the test subject. Similar to a regular DRT setup
12th September 2014
Michael Krause, Interaccion 2014
19
Institute of Ergonomics
Technische Universität München
Outlook (Follow-up Experiment)
• Questions/Motivation: Is the correlation between similar experimental conditions higher with the new setup; is the new setup better? We will track what the smartphone is doing (vibrate, subject response) in parallel with a microcontroller. Is the (in)accuracy of the smartphone sufficient?
12th September 2014
Michael Krause, Interaccion 2014
20
Institute of Ergonomics
Technische Universität München
Testing Open-Source Implementations for Detection Response Tasks
Michael Krause, Michael Krause
12th September 2014
Antonia Conti,
Moritz Späth,
Klaus Bengler
Graduate Research Assistant Faculty of Mechanical Engineering Institute of Ergonomics Boltzmannstraße 15 D-85747 Garching Tel +49 89 289-15404 Fax +49 89 289-15389
[email protected] http://www.ergonomie.tum.de Michael Krause, Interaccion 2014
21