using EO/IR sensors or Night Vision Goggles to carry out air-to- ... Goggles (NVG) are both used to conduct a simulated airborne Search and Rescue (SAR) ...
A comparison of task-related physiological responses in operators using EO/IR sensors or Night Vision Goggles to carry out air-toground search Nicholas J. Berezny, Gregory L. Craig, Madeline Lee, Heather E. Wright Beatty, Russell Thomas, Paul Kissmann, Jocelyn M. Keillor. Flight Research Laboratory, National Research Council Canada, Ottawa
The Advanced Integrated Multi Sensor Surveillance (AIMS) electro-optical system and Night Vision Goggles (NVG) are both used to conduct a simulated airborne Search and Rescue (SAR) search task aboard NRC’s DHC-6 Twin Otter. The following is a case study comparing the psychological and physiological impacts associated with either form of search and rescue. Two operators performed detection/discrimination search tasks, alternating between AIMS and NVGs, over 11 flights. Subjective data, such as the Dundee Stress State Questionnaire (DSSQ) and Psychomotor Vigilance Task (PVT) were collected both pre- and post-flight. Physiological data, including continuous Electrocardiography (ECG) and respiration, was collected throughout each flight. Heart Rate Variability (HRV) was derived from the ECG. Differences in metrics across tasks suggest that physiological arousal was significantly higher when performing the task with the AIMS system compared to performing the task with the NVGs for the two operators evaluated. 1. Introduction The Advanced Integrated Multi-Sensor Surveillance (AIMS) System was developed by Defence Research and Development Canada – Valcartier, to examine issues associated with airborne search and rescue operations using an electro-optical/infrared (EO/IR) sensor. The infrared imaging on the AIMS system can help detect objects (such as downed aircraft) in a wide variety of illumination and weather conditions. In assessing the overall benefit of using an EO/IR system in a night-time search and rescue context, a comparison of the operator’s effectiveness using the AIMS system and Night Vision Goggles (NVGs) was undertaken. While the assessment of detection performance with each system was the main focus of the test, this paper focuses on a range of operator physiological parameters and traditional self-report metrics that we obtained in order to gauge operator workload and stress. 2. Methods The search task involved detecting simulated downed aircraft objects and discriminating between target objects and distractor objects. Search objects were created using 10 highly reflective, low emissivity aluminized polyethylene tarps. Targets were placed into a generic plane-like configuration. Distractors were placed in a similar configuration. Two sensor operators alternated between using the AIMS system and NVGs for 11 flights on board the NRC’s DHC-6 Twin Otter. The operators differed in age (midforties and mid-twenties) and experience. One operator was highly experienced with AIMS and other EO/IR sensors; the other had no experience with AIMS and limited sensor experience. Search objects were placed in a range located at the Canadian Forces base in Suffield, Alberta. The objects were distributed along 15 lines, each containing a different number of targets and distractors. The search task was completed in-flight, at an altitude of 1500ft, with each flight lasting a little over 2 hours. A detailed description of the methods and an analysis of the combined effectiveness of sensors, with operators in the loop, for search and rescue tasks are provided by Strang et al., (2015).
1 Berezny et al.
During the flights, continuous electrocardiography (ECG) and respiration data was collected using the EquivitalTM EQ02 LifeMonitor. The ECG data was analysed using Kubios HRV software (available at http://kubios.uef.fi/) to derive a number of measures including basic heart rate (HR). Beat to beat intervals (RR intervals) were calculated and then arranged into an RR time series, from which the Heart Rate Variability (HRV) was derived. Heart rate variability has been used as a measure of stress in many studies (e.g., Castaldo, et al., 2015; Delaney & Brodie, 2000). The human cardiac system is modulated by the autonomic nervous system (ANS), which is sensitive to changes in physiological and cognitive arousal. Increases in arousal (i.e. stress) activate the sympathetic branch of the ANS while depressing the parasympathetic branch, which decreases heart rate variability. The root mean square of successive differences (RMSSD) is a measure of short term HRV, and thus indicative of stress (Orsila, et al., 2008). Frequency domain analysis of the RR time series data involved transforming the data using an autoregressive model to obtain the power spectrum, and separating it into three distinct frequency bands: very low frequency (VLF), low frequency (LF) and high frequency (HF). HF is modulated primarily by the parasympathetic branch of the ANS, and LF is predominated by sympathetic activity in conjunction with a smaller parasympathetic influence. The LF/HF ratio is seen as an estimate of sympathovagal tone, and again is indicative of stress. Non-linear HRV metrics were also calculated by using Kubios HRV software, specifically the correlation dimension (D2), which indexes the degrees of freedom of the RR time series. As stress increases and the sympathetic branch predominates cardiac control, a number of other components are marginalized (e.g. the parasympathetic branch of the ANS, hormonal influences, etc.). This shift towards a single dominant component manifests as a decrease in the degrees of freedom in the RR time series (Schubert, et al., 2009) In short: acute stress decreases D2 (Castaldo et al., 2015). Respiration data was recorded using a strain sensitive belt. Studies have reported higher breathing rates during stressed states to accommodate the increased metabolic rate (e.g. Suess, Alexander, Smith, Sweeney, & Marion, 1980). Both operators completed the Dundee Stress State Question (DSSQ), the Psychomotor Vigilance Task (PVT), the Karolinska Sleepiness Scale (KSS), and Subjective Fatigue scales. The short DSSQ, like the standard DSSQ, includes three second order factors: task engagement, distress and worry. It has previously been used in studies assessing subjective stress states and their relation to vigilance tasks (e.g., Temple et al., 2000). The highest reported correlation of performance is to post-task engagement (Helton & Russell, 2010). Distress and worry have been shown to increase from pre- to post-task, although they are not reliable predictors of performance (Warm, Matthews, & Finomore, 2008). The KSS is a self report 9-point scale ranging from extremely alert to extremely sleepy. It has been used to assess subjective sleepiness and has been validated against various sleep sensitive physiological measures (Putilov & Donskaya, 2013). The subjective fatigue scale is a seven point scale ranging from fully alert to completely exhausted. The Psychomotor Vigilance Task is a reaction time test which presents a series of visual stimuli and measures the subject’s response time. The PVT was administered pre- and post-flight to both operators. Task related fatigue is expected to increase post-flight reaction time (Lamond, Dawson, & Roach, 2005).
2 Berezny et al.
3. Results ECG and respiration data was segmented into epochs corresponding to each of the 15 target lines. A statistical comparison of tasks was done using the non-parametric Mann–Whitney U test. Significance levels were set to α = 0.05. An item-analysis approach was used in order to test significance within each subject across target lines.
Table 1: A comparison of physiological, performance, and questionnaire metrics between tasks Experienced (older) Operator
Inexperienced (younger) Operator
Metric HR (1/min)
AIMS 75.55
NVG 75.28
AIMS 69.44
NVG 70.00
RMSSD (ms)
28.82
31.62
33.54
41.46
LF/HF
6.96
6.75
3.96
2.61
Correlation Dimension
2.47
2.69
2.84
3.03
Heart Rate Slope (BPM/line)
-0.16
-0.45
-0.23
-0.62
Respiration (1/min)
15.33
13.63
18.03
15.50
Pre-Flight Reaction Time (ms)
238.07
239.72
231.83
248.33
Post-Flight Reaction Time (ms)
240.85
236.96
273.00
276.76
DSSQ Task Engagement
24.33
23.4
31.6
29.5
Karolinska Sleepiness Scale
6.2
5.8
2.8
3.0
Subjective Fatigue Scale
4.0
4.0
2.4
3.0
Task (AIMS vs. NVG) had no effect on HR for either the experienced operator (U = 2675.5, p = 0.2) or the inexperienced operator (U = 3165.5, p = 0.7). Task did have an effect on HRV, which tended to be lower for the NVG task. For example, the difference in RMSSD’s between the AIMS and NVG tasks approached significance for the experienced operator (U = 2476.5, p = 0.06) and was significant for the inexperienced operator (U = 2207.5, p < 0.001). The LF/HF ratio was significantly higher during the AIMS task for the inexperienced operator (U = 1809.0, p < 0.001), but was no different for the experienced operator across tasks (U = 2950.0, p = 0.82).
3 Berezny et al.
8
40
6
35
AIMS
30
NVG
LF/HF
RMSSD (ms)
45
AIMS
4
NVG
2
25
0
20 Experienced
Experienced
Inexperienced
Inexperienced
Operator
Operator
Figure 1: Short term heart rate variability as a function of task and experience level, as measured by RMSSD.
Figure 2: Autonomic balance as a function of task and experience level, as measured by the LF/HF ratio.
3.5
D2
3 AIMS
2.5
NVG
2 1.5
Respiration Rate (1/min)
The correlation dimension (D2) was higher for the NVG task, but this relation was only significant for the inexperienced operator (U = 2537.0, p = 0.01). Heart rate decreased as a function of time over most flights. The rate of decrease in the inexperienced operator was significantly higher for the NVG task (U = 0.0, p = 0.014). The rate of decrease in the experienced operator was higher for NVG as well, but this failed to reach significance (U = 4.0, p = 0.25). Respiration Rate was significantly higher during the AIMS task for both experienced (U = 1564.0, p < 0.001) and inexperienced operators (U = 942.0, p < 0.001). Post task engagement approached significance for the inexperienced operator (U = 4.5, p = 0.06), being on average higher after using the AIMS. No significant or reliable trends across search tasks were found for reaction time, subjective fatigue, subjective sleepiness, or from the DDSQ factors of distress and worry.
20 15 AIMS
10
NVG
5 0
Experienced Inexperienced Operator
Figure 3: Degrees of freedom in the RR time series, as measured by the correlation dimension.
Experienced Inexperienced Operator
Figure 4: Respiration rate as a function of task and experience level.
4 Berezny et al.
Heart Rate Trends Over Flights Heart Rate (BPM)
80 Inexperienced, AIMS
75
Inexperienced, NVG
70 Experienced, AIMS
65
Experienced, NVG
60 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Line Number
Figure 5: Average heart rate trend over all 15 lines. Darker lines indicate the trend during the NVG task; lighter lines indicate the trend during the AIMS task. Red lines are for the experienced operator and blue lines are for the inexperienced operator. Both operators showed a greater decline in heart rate for NVGs compared to AIMS.
4.
Discussion
Evidence from HRV metrics (figures 1 – 3), respiration rate (figure 4), and post task engagement suggest an increase in physiological arousal during AIMS operation as compared to the standard NVG search task. This trend was especially prevalent in the inexperienced operator. Heart rate fell at a higher rate during NVG operation (figure 5). There are several caveats that must be considered when interpreting the results. Firstly, stress as measured by HRV metrics may not be reflective of the physiological response to real-life search and rescue due to the artificially high target rate used for this experiment, which, coupled with highly visible targets and the certainty of the existence of said targets, could have increased operator vigilance to above-normal levels (Baddeley & Colquhoun, 1969). Secondly, HRV metrics such as the LF/HF ratio may be influenced by other factors in conjunction with the ANS. Lastly, differences in age and health between the two operators make any differences between the two individuals difficult to interpret and to generalize to a larger population.
5. Conclusion A search task was performed by two operators over 11 flights, alternating between two night-time search methods: AIMS and NVGs. Physiological data was recorded, including ECG and respiration, as well as subjective questionnaires. HRV metrics, respiration rate, and task engagement all suggested that using the AIMS system induced higher physiological arousal in the operators than did NVG usage.
5 Berezny et al.
Acknowledgments This study was funded in part by the National Search and Rescue Secretariat SAR New Initiative Fund and in part by the NRC Working and Travelling on Aircraft program. We thank our sensor operators for their time and patience.
Bibliography Baddeley, A., & Colquhoun, W. (1969). Signal Probability and Vigilance: A Reappraisal of the SignalRate Effect. British Journal of Psychology , 169-178. Bernston, G., Lozano, D., & Chen, Y. (2005). Filter properties of root mean square successive difference (RMSSD) for heart rate. Psychophysiology , 246-252. Castaldo, R., Melillo, P., Bracale, U., Caserta, M., Triassi, M., & Pecchia, L. (2015). Acute mental stress assessment via short term HRV analysis in healthy adults: A systematic review with meta-analysis. Biomedical Signal Processing and Control , 370-377. Delaney, J., & Brodie, D. (2000). Effects of Short-Term Psychological Stress on the Time and Frequency Domains of Heart Rate Variability. Perceptual and Motor Skills , 515-524. Garde, A., Laursen, B., Jorgensen, A., & Jensen, B. (2002). Effects of mental and physical demands on heart rate variability during computer work. European Journal of Applied Physiology , 456-461. Helton, W., & Russell, P. (2010). The Short Stress State Questionnaire and High Event Rate Target Detection. Human Factors and Ergonomics Society Annual Meeting, (pp. 798-801). Lamond, N., Dawson, D., & Roach, G. (2005). Fatigue Assessment in the Field: Validation of a HandHeld Electronic Pschomotor Vigilance Task. Aviation, Space, and Envrionmental Medicince , 486-498. Malik, M., Bigger, J., Camm, A., Kleiger, R., Malliani, A., Moss, A., et al. (1996). Heart rate variabilty: Standards of measurement, physiological interpretation, and clinical use. European Heart Journal , 354381. Melilo, P., Bracale, M., & Pecchia, L. (2011). Nonlinear Heart Rate Variability features for real-life stress detection. Case Study: students under stress due to university examination. Biomedical Engineering OnLine . Orsila, R., Virtanen, M., Luukkaala, T., Tarvainen, M., Karjainen, P., Viik, J., et al. (2008). Percieved Mental Stress and Reactions in Heart Rate Variability - A Pilot Study Among Employees of an Electronics Company. International Journal of Occupational Saftey and Ergonomics , 275-283. Putilov, A., & Donskaya, O. (2013). Construction and validation of the EEG analogues of the Karolinska sleepiness scale based on the Karolinska drowsiness test. Clinical Neurophysiology , 1346-1352.
6 Berezny et al.
Schubert, C., Lambertz, M., Nelesen, R., Bardwell, W., Choi, J.-B., & Dimsdale, J. (2009). Effects of stress on heart rate complexity - A comparison between short-term and chronic stress. Biological Psychology , 325-332. Shahid, A., Wilkinson, K., Marcu, S., & Shapiro, C. (2012). Karolinska Sleepiness Scale (KSS). In STOP, THAT and One Hundred Other Sleep Scales (pp. 209-210). Springer New York. Strang, T., Craig, G., Thomas, R., Soffer, R., Roy, S., Brown, A., et al. (2015). Evaluating Probability of Detection Using an Airborne Multi-Sensor System for Search and Rescue Operations: A Human Factors Flight Test Evaluation. Approved for publication. Suess, W., Alexander, A., Smith, D., Sweeney, H., & Marion, R. (1980). The Effects of Psychological Stress on Respiration: A Preliminary Study of Anxiety and Hyperventilation. Psychophysiology , 535540. Temple, J., Warm, J., Dember, W., Jones, K., LaGrange, C., & Matthews, G. (2000). The Effects of Signal Salience and Caffeine on Performance, Workload, and Stress in an Abbreviated Vigilance Task. Human Factors , 183-194. Warm, J., Matthews, G., & Finomore, V. (2008). Vigilance, Workload, and Stress. Performance Under Stress , 115-141.
7 Berezny et al.
