Open-Source variants of Detection Response Task (DRT) were implemented on an Arduino board and Android smartphone. These systems were tested in an ...
Testing Open-Source Implementations for Detection Response Tasks Michael Krause, Antonia Conti, Moritz Späth, Klaus Bengler Technische Universtität München, Institute of Ergonomics Boltzmannstr. 15, 85747 Garching, Germany
{krause,conti,spaeth,bengler}@lfe.mw.tum.de ABSTRACT Open-Source variants of Detection Response Task (DRT) were implemented on an Arduino board and Android smartphone. These systems were tested in an experiment under single task (only the DRT), double task (DRT + driving simulation), and triple task conditions (DRT + driving simulation + 2-back cognitive task). All DRT variants reflected different task set load, nevertheless, the smartphone setup exhibited lower internal correlations and in one condition failed to reach significance.
the IVIS while performing the measurement task (viz. DRT). In this study, three DRT systems were evaluated through a withinsubjects design The comparison of the two open-source DRT systems are discussed in this paper. Former studies by different laboratories typically used custom-made DRT systems and exchanged hardware between each other on request. With this paper, we propose a new, open source approach to DRT hardware. Information on the background and hardware of these systems (e.g. schematics) can be found online and the software has been released as open source.
Categories and Subject Descriptors H.5.2 [Information Interfaces and Presentation]: User Interfaces— ergonomics, evaluation/methodology, theory and methods
General Terms Experimentation, Human Factors
Keywords Detection Response Tasks, DRT, Tactile, Visual, Open Source
1. INTRODUCTION The Detection Response Task (DRT) method is sensitive to differences in task load [1, 2, 4, 5, 6] and are currently being standardized (ISO/CD 17488). The basic concept of the DRT is that a subject has to react as quickly as possible to a randomly recurring stimulus. Stimuli are often either tactile, where the signal is delivered by a vibrating motor (a.k.a. Tactile DRT or TDRT), or visual, where the signal is delivered by an LED mounted to the head of the test subject (Head mounted DRT or HDRT). The main results of the DRT are hit rates and reaction times (RT) of the test subjects per task condition. The reaction times to the DRT are slower if the subject experiences workload caused by a task under evaluation. In very cognitively demanding conditions, the test subjects are more likely to stop performing the DRT and, therefore, stop reacting to presented signals. Thus, the DRT is performed in addition to another task or other tasks in combination. A typical scenario would be the assessment of an invehicle information system (IVIS). Since the system is used while driving, the DRT would include the following task combination, which results in a triple task setting: driving a car and operating
© Michael Krause et al. 2014. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive version was published in Interaccion 2014, http://dx.doi.org/10.1145/2662253.2662313
2. METHOD 2.1 Systems Both systems randomly issued a stimulus every 3-5 seconds (signal onset to signal onset). Signals were active for 1 second or until button press. Only the RTs of hits were analyzed. Responses between 100 ms and 2500 ms post stimulus onset were registered as hits.
2.1.1 System 1 - Arduino This DRT is an open source implementation on an Arduino Uno 1. Data is logged directly onto an SD card, eliminating the need of an extra computer to run this task. A computer connected via USB or Ethernet receives status messages from the Arduino, without real-time constrains. The advantage of this open source implementation is the accurate interrupt-based measurement on the Arduino, which arguably outweighs the disadvantage of having to construct the physical system itself. The motor used for the vibrating stimulus was from an old mobile phone (Alcatel One Touch Easy 302). The vibration frequency was around 115 Hz. A piezo sensor connected to the motor measured around 6-7 m/s². The motor was attached to the left shoulder of the test subjects by medical tape. The visual stimulus was attached to the head of the persons with a cap (see Fig.1). Figure 2 shows how the micro switch was attached and fastened to the left index finger. To respond, participants pressed their index finger against the steering wheel.
1
Further information and the source code for this Arduino implementation can be retrieved from: http://www.lfe.mw.tum.de/arduino-drt
Figure 3, Smartphone mounted to the steering
Figure 1. HDRT
2.2 Conditions The results reported in this paper are based on three experimental conditions: • Single In this baseline condition the participant performed the DRT alone. Thus, we refer to it is as single task condition.
Figure 2. Response button attached to left index
2.1.2 System 2 - Smartphone The open source Java implementation on an Android smartphone 2is a simple implementation with the advantage of fully integrated hardware (e.g. vibrating motor, display, WiFi, SDcard).The disadvantages are a non-realtime system with timing inaccuracies. As can be seen in Figure 3 in this setup the vibrating and visual stimuli are applied by a smartphone (Samsung S5830, Android 2.3.6, brightness full on) mounted on the steering wheel. The test subjects react by tapping on the touch screen. Since this system is dissimilar from the other ”traditional” DRT setups, we refer to it as Mobile Detection Task (MDT) setup rather than DRT. To prevent unintentional actuation, some parts of the phone were covered with plastic shields. For the tactile stimulus, the test subjects held a part of their hand to the side of the smartphone and responded with their thumb. The smartphone vibrated around 225 Hz and has, on most locations of the housing, a strength of about 7-15 m/s². For the visual stimulus, the complete display changed from gray to red for the signal duration.
•
Double In this condition, the participant performed the DRT together with a driving task; a dual-task condition. The driving scenario simulated a straight highway drive with a carfollowing paradigm. Particpants were to follow a car travelling at 80 km/h.
•
Triple The triple task condition involved the same tasks as per the dual-task with the addition of the 2-back task. The n-back task [3] have been used in many DRT studies. In this task, the participant is to repeat a number that was said “n” steps prior to the current number. This task places demands on working memory and can be considered a surrogate task for a cognitively loading invehicle task
All conditions were performed for 1 minute each.
2.3 Procedure The experimental procedure was a within-subjects design. DRT system order was randomized across participants. The experimental setup included some further conditions (LEDs in different distances to the test subject and a vibrating motor at the wrist) as well as a third proprietary system (realized with a USB breakout board). As these conditions are beyond the scope of the current paper, they will not be further discussed. After completing a demographic questionnaire and signing a consent form, the subjects were trained on the n-back task. When they felt comfortable, they were trained on the operation of the visual and tactile DRT, and then on the driving task. The last step in the training was the practice of the triple task setting. All subjects participated in the experiment on a voluntary basis and received no compensation. The experiment lasted up to and around 1 hour.
2.4 Participants 2
Further information and source code for the smartphone implementation can be retrieved from: http://www.lfe.mw.tum.de/mdt
In this experiment, 36 subjects participated on a purely voluntary basis.(50% male, 50% female). The average age was 22 years (SD = 3 years).
3. RESULTS & DISCUSSION The following results are based on individual means. For each participant, an individual mean RT and hit rate were calculated per condition, and then averaged across the group. These values were then further processed and graphed in the following Figures.
Figure 6. Hit rates for both systems and the three conditions
Figure 4. Reaction times for tactile stimulus presentation on both systems
Figure 7. Standardized effect size for reaction times between single/double and double/triple condition
Figure 5. Reaction times for visual stimulus presentation on both systems Figure 4 (tactile) and Figure 5 (visual) show the mean reaction times for the three conditions (single, double, triple). In Figure 6, it can be seen that the hit rates (i.e. hits divided by presented stimuli) are above 90% for each condition. Therefore, only the reaction times are further discussed. The difference between the baseline (single) condition and the double condition is smaller than the difference between double and triple. This can be seen for both systems. The difference in the visual baseline condition between Arduino and the smartphone could possibly be related to a systematical measurement inaccuracy 3. The standardized effect size for the RTs between the condition single/double and double/triple is shown in Figure 7. The effect size from single to double is around 0.5 and from double to triple around 1.5-2.0 (Figure 7); the visual setups have a greater effect size from single to double than the tactile setups. This seems plausible as the additional workload (driving) relies heavily on resources from the same domain (visual). Therefore, such an interference is likely.
3
Further information and source code for the smartphone implementation can be retrieved from: http://www.lfe.mw.tum.de/mdt
For Figure 8, paired sample one-tailed t-tests were calculated between the reaction times of conditions single/double and double/triple. The results are plotted logarithmic with inversed axis. Therefore, a higher bar indicates higher significance. Except for one t-test, all were significant (p
