Human Performance Modeling of Synthetic Vision

Human Performance Model use of SVS San Jose State University

SJSU HAIL

Human Performance Modeling of Synthetic Vision System Use Koji Muraoka1, Savita Verma, Amit Jadhav, Kevin M. Corker and Brian F. Gore,

Industrial and Systems Engineering Department Human Automation Integration Laboratory San Jose State University San Jose, CA 95192-0180 ABSTRACT .................................................................................................................................................................2 ABBREVIATIONS ........................................................................................................................................................2 INTRODUCTION ..........................................................................................................................................................3 Synthetic Vision System(SVS)...............................................................................................................................3 Purpose ................................................................................................................................................................3 HUMAN MODEL ARCHITECTURE ...............................................................................................................................4 System Architecture .............................................................................................................................................5 Equipment Model .................................................................................................................................................6 PC Plane Architecture .........................................................................................................................................6 Cockpit Display....................................................................................................................................................6 Air MIDAS Symbolic Operator Model.................................................................................................................8 Scan Pattern Model..............................................................................................................................................9 Scan Pattern.........................................................................................................................................................9 Scan Pattern Policy............................................................................................................................................12 METHOD ...............................................................................................................................................................12 Scenario .............................................................................................................................................................12 Flight Area.........................................................................................................................................................12 Procedures .........................................................................................................................................................13 Participants........................................................................................................................................................14 Simulation Cases................................................................................................................................................14 RESULTS AND ANALYSIS .........................................................................................................................................15 Flight Profile and Task Sequences.....................................................................................................................15 Normal Approach ..............................................................................................................................................15 Go Around according to ATC command............................................................................................................16 Go Around by PF model's decision....................................................................................................................20 Average Workload .............................................................................................................................................20 Timing of "Runway-In-Sight" Callout................................................................................................................24 Flight Time Analysis ..........................................................................................................................................24 Go Around Performance....................................................................................................................................28 Go Around due to ATC command ......................................................................................................................28 With SVS v/s without SVS...................................................................................................................................29 Altitude where go around command was issued ................................................................................................29 Go Around due to pilot decision ........................................................................................................................32 Scan Pattern Analysis ........................................................................................................................................34 CONCLUSION ...........................................................................................................................................................42 ACKNOWLEDGEMENT ..............................................................................................................................................43 REFERENCES............................................................................................................................................................43 APPENDIX A. AIR MIDAS ACTIVITY DESIGN .........................................................................................................45 Cockpit Layout...................................................................................................................................................45 Human Performance Database..........................................................................................................................45 Task Time Calculation Methods ........................................................................................................................47 Air MIDAS Task Time........................................................................................................................................48 REFERENCE:....................................................................................................................................................49 1

Senior Researcher, Human Factors Team, Air Safety Technology Center, Institute of Space Technology and Aeronautics, Japan Aerospace Exploration Agency

SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

1


SJSU HAIL

Abstract The San Jose State University, Human Performance Human Automation Integration Laboratory (HAIL) under support from the NASA Aviation Safety Program2 has developed human models to predict the performance of operators using the NASA Synthetic Vision System (SVS). The standard Air MIDAS model of visual performance (Corker, 2000) was augmented to simulate pilot's monitoring of instrument and out-the-window scanning while on approach to landing. An aircraft dynamic simulation model PC Plane,© was integrated into the human-system model in order that Air MIDAS operators would be controlling aircraft performance under realistic temporal constraints. Test scenarios for the simulation were developed and procedures were established based on established cockpit procedures for a current aircraft. Simulation runs were performed under several conditions: approach & landing, and go around both with “current day” technologies or SVS cockpit configurations. Simulation results suggest that SVS use might cause small delays initiating several cockpit tasks, however, its use did not provide any issues with respect to flight safety during normal approach, landing and go around flight phase. The SVS does offer approach and landing support in all-weather conditions and support in approach to “non-instrumented” airports. Abbreviations AOI

Area of Interest

ATC

Air Traffic Control

DA

Decision Altitude

DLL

Dynamic Link Library

EICAS

Engine Indication and Crew Alerting System

FMS/CDU

Flight Management System/Control Display Unit

GA

Go Around

IMC

Instrument Meteorological Condition

MCP

Mode Control Panel

MIDAS

Man-machine Integration Design and Analysis System

ND

Navigation Display

OTW

Out The Window

PFD

Primary Flight Display

SOM

Symbolic Operator Model

SVS

Synthetic Vision System

UWR

Updatable World Representation

VMC

Visual Meteorological Condition

WM

Working Memory

2

This work was supported under NASA Grant NAG2-1563. Dr. David Foyle was the contracting officer’s technical representative. The SJSU team wishes to thank Dr. Foyle and his team for the unflagging support of this effort in both administrative and technical support. © PcPlane has been developed by NASA Langely Research Center and the NASA Ames Research Center. SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program 2


SJSU HAIL

Introduction NASA is developing a number of technologies designed to aid the flight crew in the safe operation of the aircraft under conditions that in the past have been shown to contribute to increased hazard in aviation operations. Those technologies have a common purpose in aiding the flight crew by providing information that has either been not available (e.g. improved traffic position information or rapid update of local meteorological conditions like turbulence) or has been obscured and degraded (e.g. visual acuity reduction in weather and at night). The advancements in computational techniques, sensor and communication technologies have resulted in an enviable design situation in which the amount and quality of information available is large and therefore must be carefully selected to avoid overwhelming the flight crew. Interesting issues of information selection, information integration requirements and display operation are open to investigation in the conceptual and early design stages of the systems development. Synthetic Vision System(SVS) Recently, NASA has been developing augmentative technologies for enhancing safety in flight deck operations. These developments include a synthetic vision system (SVS) for commercial aviation as well as for business jets, and general aviation operations. The system is designed to generate a texture-mapped (or wire-frame) display of the terrain in proximity to the aircraft. Text and other symbology is intended to be overlaid onto the terrain display to display, for instance, the aircraft itself, its velocity, a “follow-me” aircraft, a “tunnel-in-the-sky” indication of the route, and indications of other nearby aircraft. In addition, flight controls (air speed, attitude, pitch, etc.) are planned to be overlaid on the display. A more complete review of the several designs under development for the support and provision of synthetic vision can be found in Corker & Guneratne, (2002). In addition to these augmentations, the existing display elements of current aircraft will be maintained in an SVS equipped aircraft. Inclusion of both current presentation modes and SVS presentation modes offers a challenge in research into the operational concept of their joint usage. Specifically, providing both of these sources of information may be problematic. On one hand, they support cross checking of flight deck systems. On the other hand, two types of information that are similar in source and content, but different in presentation mode may cause transformation workload for the pilot. When systems such as that proposed for the SVS are being designed, we suggest that computational human performance models can be used to predict various performance effects of introducing such augmented technologies in early design phases. Purpose The purpose of this study was to generate predictions of human performance using the synthetic vision system under several conditions of approach and landing. These predictions are provided by a computational model of human-system performance called Air MIDAS (Air Man-machine Integration Design and Analysis System). To support these predictions in an accurate representation of the time-varying dynamics of approach to landing a high fidelity aircraft performance model, PC Plane model (Palmer et. al., 1997), was integrated into the human performance model’s knowledge-base and the aircraft model interacted with the SVS display generation models. The combined human and aircraft model and the flight’s evolution in time served as a forcing function, or triggering mechanism, SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

3


SJSU HAIL

for emergent human performance in interaction with the display and control systems. The visual perception model, previously used in Air MIDAS which has been developed and validated in the course of Air MIDAS SVS project, was further enhanced and used to simulate pilot's instruments and Out The Window (OTW) scan pattern. Approach/Landing and Go Around, which are the most critical part of a normal flight phases were selected for the simulation scenario and human performance was predicted in terms of a number of dependent variables that include aircraft control performance (e.g., cross track and vertical error, workload, decision making) and visual scan pattern change caused by SVS usage. Human Model Architecture Figure 1 depicts functional architecture of the entire Air MIDAS SVS simulation environment. The Air MIDAS software (a NASA Ames Research Center, SJSU development effort) is a performance prediction software system that uses models of human performance within an integrated computational framework to generate workload, and activity timelines in response to operational environments. The main components of the model exercised in this study were the simulated operator’s perceptual processes and the world representation in the symbolic operator model (SOM) representing perceptual and cognitive activities of an agent. In the SOM, the Updateable World Representation (UWR) contains information about the environment, crewstation, vehicle, physical constraints and the terrain. Updates of the states of these elements are provided through the perceptual and attention processes of the SOM. The world representation serves to trigger activities in the simulated operator to serve mission goals in nominal operations or respond to anomalies. The UWR also contains the Working Memory (WM) of the simulated operator, the domain knowledge, and a goal-based procedural activity structures. Activities to be performed are managed through a queuing process and scheduled according to priority and resource availability. Four resource pools (Visual, Auditory, Cognitive, and Psychomotor) are checked for resource availability in response to the demands for those resources by the required tasks (McCracken & Aldrich, 1984). PC Plane flight simulation model framework was integrated to the existing Air MIDAS architecture as a part of the operator's world representation. PC Plane is a NASA-developed PCbased flight simulation software mainly used for human-in-the loop part-task simulation of flight management system, cockpit display and future air traffic operation. PC Plane aircraft dynamics provide flight and system status to equipment components comprising of Primary Flight Display (PFD), Navigation Display (ND), Engine Indicating and Crew Alert System (EICAS), SVS and OTW and is controlled by inputs from Air MIDAS pilots. Visual scan pattern and flight crew's cockpit tasks were implemented into SOM for SVS application. Following sections describe system architecture of the simulation environment as well as detailed implementation of PC Plane and visual scan pattern model.


4


SJSU HAIL

Figure 1 Overall Air MIDAS SVS application Architecture System Architecture Figure 2 depicts the system architecture of Air MIDAS SVS simulation environment. Three modules including PC Plane aircraft dynamics, Flight Management System/Control Display Unit (FMS/CDU) and Air MIDAS were integrated into the simulation. A set of Dynamic Link Library (DLL) functions that generate cockpit control input and time synchronization control to PC Plane through socket connection were prepared. DLL functions were invoked by Air MIDAS module, which was written in LISP, through JAVA network interface architecture. Time synchronization control function realized precise synchronization of Air MIDAS and PC Plane during simulation and enabled dynamic closed loop simulation. Microsoft Office Access database architecture was used to share flight and aircraft system parameters between SOM and world representation. The database comprises PFD, ND, EICAS, SVS and OTW data sheets and each sheet includes parameter values displayed on it. PC Plane updates all of the parameters in the database as time proceeds. Air MIDAS visual scan pattern model reads data from a particular data sheet when the agent fixates on a corresponding display. PC Plane

Control Input

Control Input Interface

Control

JAVA Server Input DLL Functions (C++)

Flight & System Paramters Flight Plan & Nav Database

FMS/CDU

Air MIDAS LISP module

Flight & System Paramters

Microsoft Office Access Database

Flight & System Paramters

: LISP-JAVA Link : Socket Connection : File Access

Note) Magenta Color: Newly Implemented Architecture

Figure 2 System Architecture SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

5


SJSU HAIL

Equipment Model PC Plane Architecture The original PC Plane has the functions of an aircraft mathematical model, PFD, ND and a processor for human control input such as flight control from mouse and game stick devices. Also, it has the function of communicating with external modules such as FMS/CDU and Mode Control Panel (MCP). For the Air MIDAS world representation, FMS/CDU were used and PC Plane software was further enhanced by adding interface functions that process control input from Air MIDAS pilot agent (instead of a real human pilot). Also, a simulation time control function was added to PC Plane. This enabled fast time simulation synchronization with Air MIDAS instead of the real time human-in-the-loop simulation. Boeing B757 aerodynamic and engine model of PC Plane was used for this study. FMS/CDU module of Air MIDAS was connected to PC Plane through socket connection to provide navigation database and flight plan data. Since scenarios of this study did not require FMS operation, no input functions from Air MIDAS operator agent to replace the human input were added to the original module.

(A) PC Plane

(b) FMS/CDU

Figure 3 PC Plane Modules Cockpit Display The cockpit display model was developed to simulate the PC plane's flight and system status for Air MIDAS pilot agent. We assumed displays shown in Figure 4 were equipped on the aircraft. Air MIDAS does not have a vision function of depth perception or transformation mechanism from visual image perception (in a plan view display, for example) to recognition of the meaning of that information with respect to route of flight. Therefore, the display model was designed to provide the flight and system status by means of numerical values. Microsoft Office Access Database framework was used to share the parameter values between Air MIDAS SOM and PC Plane. The database is comprised of data sheets for PFD, ND, EICAS, SVS and OTW. Each sheet includes flight parameter values that would be shown on an equivalent display. Each sheet also includes the location of displayed area was specified for each parameter. Figure 5 summarizes specification of the data sheets. All of the parameter values in the data sheets were SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

6


SJSU HAIL

continuously updated by PC plane. However, the pilot’s internal representation was only updated when Air MIDAS vision model read part of them by "fixating" on an equivalent area of a display. GS

LOC

GS

150

5000

165 TAS 165 TRK 240 / 15

274

MAG

GVO DME

220 20

200

10

180

27

1,000

20

6 2

10

800

16 5 4

24

1

30

GOLET

(1000)

20

LOBER

700

140

10

120

20

600 10

R-33L

1

GVO 2

695

6

BARO 550

400

700

VOR L DME 5.2

VOR R DME 15.3

(a) PFD

(b) ND 1.44

20

10

10

1.8 20 30

30

1.4

1.44

1.11

1.11

1.8

1.0

1.4

1.0

EPR 34

33

32

7.4

7.4 FF 5 1

125 G 120

50 BA 0 VS

10

DOWN

GEAR

15

UP

20

FLAPS

25

30

(ｃ) SVS

(d) EICAS

Figure 4 Assumed Displays Images


7


SJSU HAIL

EICAS

PFD Parameter thedg phidg easkt selias altft selalt roc apth_e01 appt_e01 aprl_e01

Description

UNIT VALUE (ex) Pitch Angle (deg) 5.20 Bank Angle (deg) 10.1 IAS (kt) 213 Speed Command (kt) 200 Press. Altitude (ft) 3,235 Altitude Command (ft) 3,000 Rate of Climb (fpm) 500 Autothrottle Mode SPD Aitopilot Pitch Mode VNAV Autopilot Roll Mode LNAV

AREA ATT ATT SPDTAPE SPDTAPE ALTTAPE ALTTAPE ALTTAPE FMA FMA FMA

Parameter flap nsgear sbrk

Description

UNIT VALUE AREA (ex) Flap Angle (deg) 20.0 CONTROL Gear Position 1 CONTROL Speed Brake Angle (ratio) 0.1 CONTROL

OTW Parameter

Description

thedg phidg visibility rpos_tw_dme rpos_rw_brg

Pitch Angle Bank Angle Visibility DME to Runway Bearing to Runway

UNIT VALUE (ex) (deg) 5.20 (deg) 10.1 (smi) 5.0 (nm) 20.1 (deg) 32.0

AREA ATT ATT TRR NAV NAV

SVS Parameter

ND Parameter

Description

UNIT VALUE (ex) psidg Heading Angle (deg) 276.0 track Track Angle (deg) 269.0 selhdg Heading Command (deg) 300.0 to_wpt Name of To Waypoint GOLET rpos_to_dme DME to To WPT (nm) 11.2 rpos_to_brg Bearing to To WPT (deg) 125.0 rpos_tw_dme DME to Runway (nm) 20.1 rpos_rw_brg Bearing to Runway (deg) 32.0

AREA HDG HDG HDG MAP MAP MAP MAP MAP

Description

UNIT VALUE AREA (ex) thedg Pitch Angle (deg) 5.20 ATT phidg Bank Angle (deg) 10.1 ATT easkt IAS (kt) 213 SPDTAPE selias Speed Command (kt) 200 SPDTAPE altft Press. Altitude (ft) 3,235 ALTTAPE selalt Altitude Command (ft) 3,000 ALTTAPE roc Rate of Climb (fpm) 500 ALTTAPE rpos_tw_dme DME to Runway (nm) 20.1 OTW rpos_rw_brg Bearing to Runway (deg) 32.0 OTW Note) Altitude and Speed on SVS was not used for the trigger of procedural tasks.

Figure 5(a) Cockpit Display Data Sheets Specification GS FMA

150

GS 165 T AS UPPER 240 / LEFT

LOC 5000

220 20

20

200 SPDTAPE 180

10

140 120

6 ALTTAPE 2

10

800

20

MAG

1

24

GOLET

10

(a) PFD

30

1.44

1.4

400

1.4

LOBER R-33L

ATT

7.4 FF

MAP

5

LOWER VOR R DME 15.3 RIGHT

LOWER VOR L DME 5.2 LEFT

10

1.0

EPR ENGINE

GVO

700

1.11

1.8

1.0

(1000)

1

6

BARO 550

1.44

1.11

1.8

2

695

HDG

UPPER GVO RIGHT DME

7.4

20

700 600

10

274

HDG 27

1,000

ATT

16 5 4

TRK

(b) ND

DOWN

1 15 CONTROL UP 20 FLAPS

GEAR

30 30

(c) EICAS

25

SPDTAPE

TERRAIN

ALTTAPE BA

(d) SVS

Figure 5(b) Cockpit Display Area Definition Air MIDAS Symbolic Operator Model Flight and system information provided by cockpit display models was passed into the SOM through its visual perception models. Once this information was perceived through the scan pattern, it was passed into the UWR and salient values of the data were used to trigger cockpit activities. For this study, WM storage nodes to accommodate PFD, ND, EICAS, SVS and OTW were prepared. In WM domain knowledge and rules to invoke actions regarding cockpit procedures such as (1) Approach & Landing and Go Around procedure, (2) Standard callout, (3) Checklist, (4) ATC communication and (5) Landing/go-around decision were implemented. In this research effort, perceptual processes associated with the SVS system and/or OTW observation are critically important. Detailed structure is described below. SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

8


SJSU HAIL

Scan Pattern Model Air MIDAS visual perception model simulates pilot's information acquisition process from cockpit displays and OTW. This activity was assumed to be comprised of both periodical information sampling and directed information acquisition associated with demand from a particular cockpit task. During the flight, pilot continuously monitors flight and system status based on the periodical information sampling. If a certain demand comes from a cockpit task, for example pilot's confirmation of his/her own action of setting speed command, the pilot would intentionally focus on a particular information, in this example the speed command indication on PFD speed tape, by interrupting the normal periodic information sampling. Directed information acquisition of visual perception was implemented as a part of Air MIDAS cockpit procedural tasks and periodical information sampling was implemented as a scan pattern model is described below. A scan pattern model has been developed and validated in the course of ongoing Air MIDAS human performance modeling research efforts (Corker et al., 2003). For this study, the scan pattern policy was a new addition and the scan pattern's specification of display configuration and corresponding human performance database was slightly modified to fit the simulated cockpit configuration. SCAN PATTERN POLICY Select IMC Strategy or VMC Strategy

AirMIDAS Query

SCAN PATTERN Select Display to be Fixed

World Representation

Select Area to be Fixed Determine Fixation Duration

Probablic Density

Scan Pattern Human Performance Database (Table 2.1)

FAIL UWR Update

SUCCESS UWR Update Fail Average

Threshold

time Fixation Success/Failure Filter

Equipment PFD SVS

ND

UWR

Success

EICAS

Get Data & Update UWR

OTW

Note: Magenta Color implies Implimentation for SVS application.

Figure 6 Visual Perception Model Logic Scan Pattern Figure 6 depicts logic of Air MIDAS scan pattern model. The scan pattern selects display, selects the area to fixate its eyes, aggregates the values of displayed parameters and then updates UWR. Failure of data aggregation was also simulated, with a corresponding failure to update the UWR. SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

9


SJSU HAIL

The normal internal scan pattern and fixation time was based on NASA HPM team’s data collected for Human In The Loop simulation (Foyle & Hooey, 2002). The Phase-1 of this modeling effort had used NASA’s report on the Analysis of Pilot’s Monitoring and Performance on Highly Automated Flight Decks generated by Mumaw et al., (2000). Since their experiments were focused on VNAV descent flight phase without SVS, the data provided by Foyle & Hooey (2002) was used for the current phase of development. The modified scan pattern model shown in Table 1 was prepared based on following assumptions: 1) The pilot applies different scan pattern according to the availability of information on OTW. Two different scan pattern strategies, Instrument Meteorological Condition (IMC) strategy and Visual Meteorological Condition (VMC) strategy, were prepared for both with and without SVS configuration. 2) Off-Area of Interest (AOI) and overlapped AOI fixation percentages in the Foyle & Hooey (2002) data analysis were combined. Off-AOI scans signify inattention to the instruments, but the same logic might not be applied for overlapped AOI. Overlapped AOI simply means that the operator is not foveating, but his/her peripheral vision can detect warnings and similar changes in the instruments. However, Air MIDAS did implement wide area peripheral vision for this version. So, Off-AOI and overlapped fixation patterns were combined. 3) "Fixation on control setting" data in Foyle & Hooey (2002) data analysis report was used for EICAS fixation considering the difference of cockpit configuration. Human-in-the-loop simulation setup did not have EICAS but have PC screen based control input to get data to be shown on EICAS.


10


SJSU HAIL

(a) VMC Strategy (without SVS) Area of Interest (Display) Off + Overlapp OTW SVS PFD NAV MCP EICAS

Fixation Average SD (%) Duration (sec) (sec) 16.29 0.235 0.65 11.51 0.214 1.46 0 0 0 33.03 0.236 0.74 35.63 0.299 1.43 3.18 0.322 2.75 0.37 0.274 0.84

(c) IMC Strategy (without SVS) Area of Interest Fixation Average SD (Display) (%) Duration (sec) (sec) Off + Overlapp 3.03 0.200 0.21 OTW 2.41 0.222 0.60 SVS 0 0 0 PFD 38.89 0.437 1.22 NAV 47.12 0.421 1.29 MCP 3.28 0.365 2.78 EICAS 5.26 0.530 1.75

(b) VMC Strategy (with SVS) Area of Interest (Display) Off + Overlapp OTW SVS PFD NAV MCP EICAS

Fixation Average SD (%) Duration (sec) (sec) 16.10 0.235 0.65 11.38 0.214 1.46 0.14 0.180 0.06 32.65 0.236 0.74 35.22 0.299 1.43 3.14 0.322 2.75 0.37 0.274 0.84

(d) IMC Strategy (with SVS) Area of Interest Fixation Average SD (Display) (%) Duration (sec) (sec) Off + Overlapp 3.86 0.225 0.58 OTW 0.34 0.285 0.20 SVS 25.34 0.347 2.72 PFD 29.92 0.392 0.82 NAV 32.21 0.393 0.95 MCP 4.19 0.423 3.42 EICAS 4.14 0.392 1.80

(e) Fixation Rate rate for Display Area PFD ND EICAS SVS OTW Area Fixation Area Fixation Area Fixation Area Fixation Area Fixation ATT 34.0% HDG 40.0% ENGINE 80.0% ATT 25.0% Terrain 33.0% SPDTAPE 27.0% MAP 40.0% CONTROL 20.0% OTW 25.0% NAV 33.0% ALTTAPE 29.0% UPLEFT 5.0% SPDTAPE 25.0% ATT 34.0% FMA 6.0% UPRIGHT 5.0% ALTTAPE 25.0% HDG 4.0% LOWLEFT 5.0% LOWRIGHT 5.0%

Table 1 Scan Pattern Model Human Performance Database

Once the display, its area and duration to be fixated were determined, the fixation success/failure filter evaluated whether the fixation could successfully aggregate data or not. For this filter, we used a simplification of our method of partial information intake. In other simulations, as dwell time increased components of information are gradually provided to the appropriate slots in the UWR update. (This assumes that there are actions that can be taken based on partial information.) In this simulation we applied a threshold function such that it was assumed all the data included in the area can be successfully perceived when the fixation duration was long enough or not perceived at all if the threshold value was not met. Based on this assumption, Air MIDAS updated parameters in UWR when the fixation duration was longer than a specified threshold, but if shorter, it did not perform any UWR update. Since this assumption was associated with failure of perception and selection of a threshold would affect on the results of simulation, threshold setting was examined through simulation runs and determined to be set at (mean_duration-1.0standard_deviation) (sec) so that rate of scan failure was about 10% or less. Examination of threshold setting will be described in Result and Analysis section.


11


SJSU HAIL

Scan Pattern Policy Scan pattern policy selects scan pattern strategy to be applied for scan pattern. Algorithms of the scan pattern policy summarized in Table 2 were defined simulating both PF and PNFs' roles in system monitoring. These were defined according to "Scan Pattern Policy" in an aircraft operation manual. The procedures, as specified in the aircraft operation manual, (see Figure 8) were used to implement the scan pattern policy. Since only the PF's eye fixation data of humanin-the-loop simulation in both VMC and IMC condition was available, VMC strategy was applied for PNF's strategy including "outside scan" to detect the runway, and IMC strategy was applied for PNF's "instrument scan." Table 2 Scan Pattern Policy Algorithm Pilot PF

PNF

Condition

Strategy Documented Scan Pattern Policy (Simulated) Selection PF: -Outside view should be included into his/her scan pattern Before Runway Insight IMC after "Runway in Sight." After passing Approaching Minimum: After Runway Insight or VMC - Outside View should be included into his/her scan pattern. After DA PNF: - After passing Final approach fix, outside view should be Before Runway Insight VMC included in his/her scan pattern. After runway becomes insight, he/she should perform instrument scan. After passing Approaching Minimum: After Runway Insight IMC

METHOD A series of simulation runs were performed to evaluate SVS's impact on cockpit performances focusing on scan pattern change associated with SVS implementation. Normal approach and go around flight were simulated and two different decision altitudes (DA) were prepared. The two different go around triggers including ATC command and lack of visibility of runway at decision altitude (DA) were prepared for go around simulation. Scenario Flight Area Figure 7 depicts approach chart for the simulation. Based on the existing GPS approach procedure to Santa Barbara airport, GPS-VNAV/LNAV approach to runway 33L was assumed. Initial position was located 5 (nm) north west of GAVIOTA - initial approach fix, and flight using autopilot and auto throttle with VNAV and LNAV mode was assumed. Two decision altitudes were assigned in order to examine the impact of SVS usage on DA selection for approach procedures. DA 650 (ft) is as high as the usual non-precision approach decision altitude and DA 200 (ft) is as high as category I instrument approach decision height. A detailed missed approach pattern was not prepared as the scope of the go around simulation was focused on the phase of making the go around decision and initiating a go around climb. The go around simulation terminated when aircraft achieved positive climb with gear up and flaps 5 configurations. The landing simulation terminated when aircraft touched down. No other aircraft were assumed to be in the area.


12


10000 132 (5)

SJSU HAIL

(IAF) GAVIOTA 4920

SAN MATEUS 555 485 461

480 379

372

334

450

330 (3)

5000 163 (16)

1000 PHANTOM 007 (6)

Aircraft Initial Condition (5nm before GAVIOTA) Position (Lon) [deg] 34.599 Position (Lat) [deg] -120.133 Weight [lb] 18000 Altitude [ft] 10000 Airspeed [kt] 250 Heading [deg] 150 Gear Up Flap Up

(1800) GOLET (FAF)

3000 068 (11)

LOBER

GOLET Procedure Turn NA 1800

GOLET

PHANTOM

1.1NM to RW33L

007 GS 4.00 TCH53

1000 5.0 NM

2.0 NM

1.1

CATEGORY

LNAV/ DA VNAV

650/200

650-1 1/2 640 (800-2)

Figure 7 Flight Area Procedures Cockpit activities including (a) Approach/Landing/Go Around, (b) Standard Callout, (c) Checklist, (d) Scan Policy and Scan Pattern, (e) ATC Communication, (f) Go Around (GA) decision were implemented as a part of Air MIDAS activities. Among these activities, (a) through (d) are usually described in an aircraft operating manual and similar documents in figure 8 were used for Air MIDAS implementation. (d)Scan Pattern policy was installed as a part of scan pattern model and others were implemented as a part of Air MIDAS's domain knowledge and tasks. Activity - (e), which was ATC communication included approach clearance, landing clearance, or go around command. PNF model was assumed to be in charge of ATC communication tasks. Activity-(f), GA decision for this model was defined as a set of activities that decide whether to continue landing or perform go around based on the flight and system status such as visibility of the runway, tracking of nominal approach path, and the stability of aircraft etc. Go Around (GA) decision for Air MIDAS was designed so that it was taken immediately after passing DA. Detailed implementation of cockpit activities are described in Appendix 1.


13


SJSU HAIL

Standard Callout (Approach & Landing)

Scan Policy PF: -Outside view should be included into his/her scan pattern after PNF calls out "Runway in Sight." After passing Approaching Minimum: - Outside View should be included into his/her scan pattern. PNF: - After passing FAF, outside view should be included in his/her scan pattern. After runway or visual cue to identify the runway becomes insight and s/he callout it, he/she should perform instrument scan. After passing Approaching Minimum: - PNF should concentrate on instrument scan.

Flight Phase FAF Field Elv.+1,000 ft (BARO) Field Elev.+500ft (BARO) DA + 80 ft DA Runway In Sight MAP 100ft RA

PF **** (GOLET) (Roger) Stabilized Check Landing/Go-Around

PNF ****, **ft (GOLET, xx ft) (One Thousand) (Five Hundred) (Approaching Minimum) Minimum Runway In Sight Missed Approach Point (One Hundred)

Landing Checklist LANDING GEAR..................................DOWN SPEEDBRAKE......................................ARMED FLAPS....................................................XX

Approach & Landing PF

PNF

Order "Flaps XXX" according to Flap Extension Schedule Order "Gear Down" Order "Flaps 20"

Speedbrake Lever ARM Landing Flap "Flaps XX"

Readback "Flaps XXX" Set Flap Lever XXX Readback "Gear Down" Landing Gear DN Readback "Flaps 20" Set Flap Lever 20 Readback "Flaps XX" Set Flap Lever

Set Missed Approach Alt on MCP Perform "Landing Checklist" Order "Landing Checklist" Callout "Checklist Complete" Monitor Approach Progress

Go Around PF

PNF

Callout "Go-Around" Push TO/GA Switch Order "Flaps 20"

Readback "Flaps 20" Set Flap lever 20 Confirm Go-Around Attitude and Increasing Thrust Check appropriate GA-Thrust and correct Thrust Setting if necessary. Check Positive Rate of Climb Positive Rate of Climb Readback "Gear Up" "Gear Up" Gear Lever Up Followings were omitted for simulation setting

Figure 8 "Documented" cockpit procedures. Participants Three human agent models, pilot flying (PF Air MIDAS), pilot-not-flying (PNF Air MIDAS) and air traffic control (ATC controller Air MIDAS), were included in each simulation run. The ATC agent’s set of activities were mostly communication and included providing the clearance message. No cognitive process of traffic control tasks was assumed. The ATC activities were designed in this way so that the researcher could control the timing of message generation for PF and PNF Air MIDAS. Simulation Cases Table 3 summarizes simulation cases. Normal approach without SVS case was used as a baseline. Two go around conditions including go around following controller's command and go around based on PF's decision were also examined. Two different out the window visibility levels, which switched at a specified altitude, were used associated with cockpit activities "Runway-In-Sight" callout, and PF's go around decision. Visibility was set so that Runway became insight at 150 ft before DA, except in the PF's go around decision cases. Visibility in case 9, 10, 11 and 12 was set so that go around event is triggered due to an inability to see the runway at DA. In ATC's go around command cases, two sets of the timing were used so that the interruption of pilot activities took place both in busier and the less busy phase of flight. The command was issued about 100 ft before DA, which was the busier phase including the tasks of runway in sight callout and approaching minimum callout, in case 5, 6, 7 and 8, and it was issued at 250ft before DA, where no particular activity was not expected. SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

14


SJSU HAIL

In each simulation run, flight parameters such as altitude, airspeed, position etc, VACM workload of PF and PNF model, and the status of visual scan including location and success/failure of the scan were recorded. Table 3 Simulation Cases Case No. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Note)

Approach

SVS

DA (ft)

Normal Approach Normal Approach Normal Approach Normal Approach Go-Around Go-Around Go-Around Go-Around Go-Around Go-Around Go-Around Go-Around Go-Around Go-Around Go-Around Go-Around

Without With Without With Without With Without With Without With Without With Without With Without With

650 650 200 200 650 650 200 200 650 650 200 200 650 650 200 200

Weather vis_abv / alt / vis_blw (smi)/(ft)/(smi) 0.5/800/10.0 0.5/800/10.0 0.5/350/10.0 0.5/350/10.0 0.5/800/10.0 0.5/800/10.0 0.5/350/10.0 0.5/350/10.0 0.2/650/0.2 0.2/650/0.2 0.2/200/0.2 0.2/200/0.2 0.5/800/10.0 0.5/800/10.0 0.5/350/10.0 0.5/350/10.0

Events

ATC GA Com @750ft ATC GA Com @750ft ATC GA Com @300ft ATC GA Com @300ft

ATC GA Com @900 ATC GA Com @900 ATC GA Com @450 ATC GA Com @450

Description

Runs

Base Line Base Line DA@200 DA@200 GA by ATC GA by ATC GA by ATC GA by ATC GA by Pilot GA by Pilot GA by Pilot GA by Pilot ATC@HighWL ATC@HighWL ATC@HighWL ATC@HighWL

5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5

vis_abv: Visibility above (boundary) altitude. vis_blw: Visibility below (boundary) altitude

Results and Analysis Flight Profile and Task Sequences In all runs, landing or go around mission were safely completed by Air MIDAS pilots. Followings summarize simulation results of flight and task sequence in landing, go around by ATC command and by PF model's decision cases by selecting some of simulation runs for analyses. Normal Approach Figure 9 shows the flight path and task sequence of case 2 run 1, one of the normal approach cases. Speed command setting, flap lever position and gear position are the system parameters, which were manipulated by Air MIDAS pilots. Since the aircraft was flown with autopilot (VNAV and LNAV modes) and auto-throttle (VNAV mode), control surface parameters do not show any Air MIDAS manipulations. After the flight started, airspeed was reduced overriding the setting of the MCP speed knob. Airspeed was reduced by VNAV programmed airspeed command. Then the PF model set airspeed command to 200 (kt) and overrode the VNAV airspeed command, which is the activation of the speed intervention, by pushing speed knob. When the airspeed was reduced to 218 (kt) at 131.7(sec) PF model ordered flap deployment and PNF model set the flap lever to 5 (deg). At 134.5(sec), PF model decreased airspeed command to 160 (kt) for further flap extension. The timing of this action was before the aircraft had achieved the previous airspeed command since the starting condition of the task was specified by the completion of the flap deployment action and airspeed with a certain margins (between 210kt and 190kt). SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

15


SJSU HAIL

After passing 800 (ft), which is a boundary altitude of low and high visibility area, PNF model called out "Runway-In-Sight" at 772(ft), and 784.7(sec) at that moment PNF model fixated on OTW. At 796.3 (sec), after passing DA(= 650ft), PF decided landing and called out "Landing" then aircraft touched down on the runway at 845.4(sec). Figure 9 (c) and (d) are time history of the PF and PNF model's workload. (For visual clarity among procedures, scan pattern was omitted to plot, since this procedure was performed continuously during the simulation run with visual workload of 5.9). This procedure was always performed as a back ground task and was interrupted by other tasks performed by the PF and PNF agents. Both PF and PNF models had higher density of workload period before they completed the configuration landing flaps, airspeed and gear. PF model also had higher density of workload period from around DA to touch down. PF model/ agent had maximum visual workloads of 7.0 when it performs speed reduction and orders flap extension, and when it performs landing/go ground decision with maximum cognitive workload at 7.0. Maximum auditory and motor workload was 5 in the simulated flight. As for PNF model, it did not have the moment when the workload value reached 7.0. Go Around according to ATC command Flight and task sequence of case 8 run 1 is shown on figure 10 as an example of simulation results of go around due to ATC command scenario. At 8126(sec), go around command was issued by air traffic controller model. Both pilot models heard it and PF model called out "Go Around" at 812.9(sec). Then PF model pushed go lever and set pitch attitude to 10 degrees. PNF model set the flap lever to 5(deg) at 818.6 (sec) following the order from PF model. After confirming positive climb, PF model ordered gear up and PNF set the gear lever up position at 821.5(sec). Figure 10 (c) and (d) shows time history of PF and PNF model's workload. PF model had larger density of visual, cognitive and motor workload after hearing go around command with auditory workload at 5 compared with the density around and after DA in normal approach flight. This was caused by time-critical tasks required to perform go around.


16


HAIL

5.0

VNAV Speed Intervention

Airspeed Speed Command 34.7 34.6 Latitude (deg)

Attitude

Airspeed

Pressure Altitude

10,000 (ft) 8,000 6,000 4,000 2,000 0 (kt) 240 200 160 VNAV Speed 120 (deg) 10.0

SJSU

Pitch Angle Flight Path Angle

0.0 -5.0

EPR

1.2

34.3

1.0

Flap Angle Nose Gear Visual Workload Auditory Workload

-120.2

10.0 0.0 UP

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

Flap Angle Flap Lever Angle

-120

-119.9

-119.8

34.2 -119.7

(b) Horizontal Profile

100 200 300 400 500 600 700 800 900 Time (sec) (a) Flihgt Profile Set Missed Approach Altitude

Landing Clearance

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 0

0

-120.1

Londitude (deg)

20.0

DN 0

Cognitive Workload

34.4

1.1

0.9 30.0 (deg)

Motor Workload

34.5

100 200 300 400 500 600 700 800 900 Time (sec) (c) Workload (PF)

App & Ldg Checklist Callout ATC GA Decision

100 200 300 400 500 600 700 800 900 Time (sec) (d) Workload (PNF)

Figure 9 Flight Profile (Case 2, Run 1) - Normal Approach with SVS DA650 SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

17


HAIL

5.0

34.7 Airspeed Speed Command 34.6


0.0 -5.0

EPR

1.2


34.3

1.0 -120.2

10.0 0.0 UP

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0


-120

-119.9

-119.8

34.2 -119.7


100 200 300 400 500 600 700 800 900 Time (sec) (a) Flihgt Profile

ATC Go Around Command

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 0

0

-120.1

Londitude (deg)

20.0

DN 0

Cognitive Workload

34.4

1.1

0.9 30.0 (deg)

Motor Workload

34.5

Latitude (deg)

Attitude

Airspeed

Pressure Altitude

10,000 (ft) 8,000 6,000 4,000 2,000 0 (kt) 240 200 160 120 (deg) 10.0

SJSU




Figure 10 Flight Profile (Case 8, Run 1) - Go Around by ATC Command SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

18


HAIL

5.0

34.7 Airspeed Speed Command 34.6


0.0 -5.0

EPR

1.2


34.3

1.0 -120.2

10.0 0.0 UP


7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

-119.9

-119.8

34.2 -119.7


7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 GA Decision at DA

0

-120

100 200 300 400 500 600 700 800 900 Time (sec) (a) Flihgt Profile

7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0

-120.1

Londitude (deg)

20.0

DN 0

Cognitive Workload

34.4

1.1

0.9 30.0 (deg)

Motor Workload

34.5

Latitude (deg)

Attitude

Airspeed

Pressure Altitude

10,000 (ft) 8,000 6,000 4,000 2,000 0 (kt) 240 200 160 120 (deg) 10.0

SJSU



7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0 0


Figure 11 Flight Profile (Case 12, Run 1) - Go Around by PF model's decision SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

19


SJSU HAIL

Go Around by PF model's decision Figure 11 shows the flight and the task sequence of case 12 run 1 as an example of simulation results of go around by PF's decision scenario. At 827.6 (sec), after the aircraft passed the DA, PF model confirmed whether the runway had become visible, and whether the aircraft had been stabilized. OTW equipment model provided the visibility and the distance to the runway and the computed/perceived value was: visibility - distance_to_runway = 0.2 - 0.437 = -0.163 (nm) This value was less than zero, and the PF model found that runway was not visible and decided to perform go around. "Go Around" call out was taken at 827.9(sec) followed by a series of go around tasks. Figure 11 (c) and (d) shows the time history of PF and PNF model's workload. The PF model had one of maximum cognitive workload 7.0 around the DA which is caused by DA decision procedure. PF model had a larger density of visual, cognitive and motor workload after go around decision with maximum cognitive and visual workload at 7.0. Average Workload Figure 12 shows total average and every procedure's workload during each flight mission. Average workload, which was used to examine the contribution of each procedure to the overall workload, was defined by WL i |total =

jall nall

∑∑ WL j

WL i, j =

n

⋅ Δt i, j,n / t total

: Total Average workload

nall

∑ WL

i, j,n

n

i,n

⋅ Δt i,n / t total

: Average workload (each procedure)

Where i

: V(Visual), A(Auditory), C(Cognitive) or M(Motor)

j

: Procedures (Approach&Landing, GoAround, Scan Pattern, Checklist,

Standard Callout, ATC, GA Decision and Others) n WL i, j,n Δt i, j,n t total

: Number of accomplished activities : Workload of each task item : Duration of each task item (sec) : Flight Time (sec)


20


SJSU HAIL

Total Visual, Auditory and Cognitive average workloads of both PF and PNF were around 4.0, 0.2 and 1.0 respectively in both normal approach and go around cases. Average Motor workload was about 0.25 for PF and 0.16 for PNF in both normal approach and go around simulation. Major differences in activities across each scenario were only in the short final phase and they didn't have any significant impact on total average workload. Larger visual and cognitive workload than motor workload characterized the flight, which was performed using automatic flight systems. Scan pattern activities mainly contributed towards the Visual workload. Motor workload was experienced by the agent, not due to manual flight control tasks, but due to the autopilot commands, like pull down gears, extension of flaps etc., which were specified in Approach & Landing or Go Around procedure. Figure 12(a) and (b)-(b2) illustrates that more than 80% of Motor workload was caused by the flight procedures. Ratio of the Approach & Landing and Go Round’s auditory workload in case 5~8 and 13~15, that required verbal orders of manipulation such as "Gear Up" and "Set flap 20 degrees", was higher than in the cases 1~4 and 9~12, although the amount of auditory workload was almost same across the cases. This was because the difference in the scenario settings. In case 5~8 and 13~15, Act’s approach clearance was omitted to focus on Act’s communication task for the Go Around command, while cases 1~4 and 9~12 included approach clearance and landing clearance. The amount of auditory workload of ATC communication procedure was very low and negligible in all cases. Contributions of the checklist and go around decision towards workload were also small. These procedures contained fewer task items than Approach & Landing, Go Around, or callouts. It is important to note that lower average workload do not reduce the importance of the procedures. Check list and go around decisions are some of the most important procedures to ensure a safe flight. Scan pattern caused more than half of the Visual workload and almost 30% or more Cognitive workload both for the PF and PNF. No major difference in visual workload was observed


21


SJSU HAIL

between with and without SVS operation.

Figure 12 (a) Average Workload (PF)


22


SJSU HAIL

Figure 12 (b) Average Workload (PNF)


23


SJSU HAIL

Timing of "Runway-In-Sight" Callout Figure 13 summarizes timing of "Runway-In-Sight" callout analysis. Timing of "Runway-InSight" callout Δt RwVis was defined by the time from the moment when the runway became physically visible to the moment when PF realized that the runway was in sight and made the callout after fixating on OTW. Formally by

Δt RwVis = t Srwy − t Vis Where t Srwy

: Time when PNF started "Runway-In-Sight" callout

t Vis

: Time when runway became physically visible ( VIS > d rwy , VIS: Visibility, d rwy :

Distance from Aircraft to Runway) Average "Runway-In-Sight" timings in without SVS cases (1, 3, 5, and 7) were 0.02 to 1.38 (sec) faster than with SVS cases (2, 4, 6 and 8). It is considered to be caused by the difference of scan pattern. PNF's "Runway-In-Sight" activity was triggered by the PNF's internal evaluation of the status of visibility and the distance to runway, which were both provided by OTW. The earlier the fixation on OTW occurred after runway became physically visible, the earlier the callout was taken. The scan pattern with SVS reduced the chances of fixation on OTW and it caused the delay of the average "Runway-In-Sight" callout timing. We do not consider that these delays have a significant impact on the entire flight safety because the amount of delay was 0.02 to 1.38 (sec) and this time variation is task is not critical to safety of flight. Visibility check should be performed at DA by PF to make final decision of landing or go around. This flight phase is much more time critical but the fixation should be performed not by a part of scan pattern sequence but by PF's directed gaze. The delay of "Runway-In-Sight" callout timing should not happen in this phase because the visual search at the DA is not simply part of the standard scan pattern. Flight Time Analysis To analyze SVS's impact on overall pilotage tasks flight time in each run were analyzed. Since the approach flight mission requires Air MIDAS pilot to reduce airspeed gradually with the flaps deployment, timing of each procedural task could affect the change of flight time. Figure 14 summarizes flight time which is defined by the time from initial position to the point of DA or the altitude where visibility changed which was specified in the scenarios. Since the aircraft flies to track the nominal flight path with its VNAV and LNAV modes, it should pass DA and visibility change altitude at almost same position and these altitudes could be used as a measure of the flight time. Average flight time in all cases with SVS was shorter than those without SVS. Shorter flight time equals to higher speed during the flight, assuming equivalent flight paths. Generally shorter SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

24


SJSU HAIL

flight time is preferable for efficiency but these simulation results might mean degradation of pilot's procedural activities performance. VNAV speed intervention was applied for airspeed control and its command was set and changed manually by the Air MIDAS PF model. So, shorter flight time was caused directly by later timing of airspeed setting actions. Since a series of speed command setting tasks and flap setting tasks were specified to be triggered by airspeed status, which was perceived by visual perception model, smaller chances of fixation on each display (particularly the SVS Scan pattern) caused delay in initiating these control input tasks and that invariably led to shorter flight times. While the tendency of reduced flight time was observed in the cases with SVS in the scan pattern, no hazardous flight maneuver was observed and aircraft landed safely or made a go around. So, shorter flight time tendency in the SVS cases cannot be considered as having a great impact on flight safety Among 80 runs of simulation runs, there were four “extremely” shorter flight time runs were obtained; four runs of with SVS cases (case 10 run3, case 14 run5 and case 16 run4) and one run of without SVS case(case 13, run 5), which are written in red color in Figure 14. Two of them (case 10 run 3 and case 13 run 5) were caused by the characteristics of autopilot in conjunction with the timing of airspeed command action. Figure 15 shows comparison of airspeed and airspeed command history. Delay of first airspeed command setting and speed intervention action was not so large, however, smaller deceleration that was the resultant motion controlled by aircraft autopilot system was achieved and it caused further delay of airspeed reduction and subsequent task initiation. Usually almost same amount of deceleration is expected to be achieved by speed mode of autopilot system, however sometimes it could differ due to aircraft configuration, initial airspeed and thrust settings when the mode is engaged, and so on. Other two runs (case 14 run 5 and case 16 run 5) were caused by Air MIDAS PF's improper sequence of airspeed command setting actions. Setting airspeed command at 200 kt and pushing VNAV speed intervention switch was performed after a series of reduced airspeed command settings were performed although they should be performed in the beginning of reduced airspeed command setting tasks. While the condition of initiating these tasks was defined by aircraft altitude (ex. altitude becomes less than a particular altitude), the condition of other airspeed command settings was defined by airspeed (ex. airspeed becomes less than a particular airspeed). This setting was considered to cause improper sequence in two runs although it worked fine in other 78 runs. These four cases were not eliminated from the statistical analysis as they represented legitimate behavior by the model (despite their extreme value).


25


SJSU HAIL

(b) Flight Time (To VISALT) (Case 5-8 Summary) (sec) Case No. 5 No. 6 No. 7 No. 8

Run 1 Run 2 Run 3 Run 4 Run 5 AVRG


780.3 782.6 780.2 779.0 780.6 780.5

782.6 782.6 779.0 768.4 774.5 777.4

814.2 820.1 820.6 819.8 820.7 819.1

806.4 800.2 820.4 819.6 820.5 813.4

777.0 779.4 780.3 781.6 779.6 779.6

850

850

800

800

Flight Time (sec)

Flight Time (sec)

(a) Flight Time (To VisAlt) (Case 1-4 Summary) (sec) Case No. 1 No. 2 No. 3 No. 4

750 700 650

776.1 767.1 781.6 779.5 776.2 776.1

815.2 817.6 817.5 818.8 815.2 816.9

807.3 817.3 807.3 821.7 807.3 812.2

750 700 650

600

600 No. 1 No. 2 No. 3 No. 4

No. 5 No. 6 No. 7 No. 8

Case No. 9 No. 10 No. 11 No. 12

(d) Flight Time (To ATCalt) (Case 13-16 Summary)(sec) Case No. 13 No. 14 No. 15 No. 16



796.5 797.4 796.8 796.2 789.3 795.2

794.1 780.2 647.5 793.7 790.2 761.1

829.3 833.9 834.5 833.3 829.4 832.1

827.5 829.3 827.5 827.2 801.1 822.5

㪎㪌㪈㪅㪌 773.4 777.5 㪎㪎㪊㪅㪉 764.7 773.6 764.1

777.0

777.9 767.2 628.0 744.7

850

850

800

800

750

Flight Time (sec)

Flight Time (sec)

(c) Flight Time (To DA) (Case 9-12 Summary (sec)

Run 1 Run 2 Run 3 Run 4 Run 5 Average

700 650 600

751.3 751.2 628.0 729.3

773.4 749.3 772.1 768.4

759.3 590.0 770.0 732.2

750 700 650 600 550

No. 9 No. 10 No. 11 No. 12

No.13 No.14 No.15 No.16

Figure 13 Timing of "Runway-In-Sight" Callout


26


SJSU HAIL

(b) Flight Time (To VISALT) (Case 5-8 Summary) (sec) Case No. 5 No. 6 No. 7 No. 8



780.3 782.6 780.2 779.0 780.6 780.5

782.6 782.6 779.0 768.4 774.5 777.4

814.2 820.1 820.6 819.8 820.7 819.1

806.4 800.2 820.4 819.6 820.5 813.4

777.0 779.4 780.3 781.6 779.6 779.6

850

850

800

800

Flight Time (sec)

Flight Time (sec)

(a) Flight Time (To VisAlt) (Case 1-4 Summary) (sec) Case No. 1 No. 2 No. 3 No. 4

750 700 650

776.1 767.1 781.6 779.5 776.2 776.1

815.2 817.6 817.5 818.8 815.2 816.9

807.3 817.3 807.3 821.7 807.3 812.2

750 700 650

600

600 No. 1 No. 2 No. 3 No. 4

No. 5 No. 6 No. 7 No. 8

Case No. 9 No. 10 No. 11 No. 12

(d) Flight Time (To ATCalt) (Case 13-16 Summary)(sec) Case No. 13 No. 14 No. 15 No. 16



796.5 797.4 796.8 796.2 789.3 795.2

794.1 780.2 647.5 793.7 790.2 761.1

829.3 833.9 834.5 833.3 829.4 832.1

827.5 829.3 827.5 827.2 801.1 822.5

㪎㪌㪈㪅㪌 773.4 777.5 㪎㪎㪊㪅㪉 764.7 773.6 764.1

777.0

777.9 767.2 628.0 744.7

850

850

800

800

750

Flight Time (sec)

Flight Time (sec)

(c) Flight Time (To DA) (Case 9-12 Summary (sec)


700 650 600

751.3 751.2 628.0 729.3

773.4 749.3 772.1 768.4

759.3 590.0 770.0 732.2

750 700 650 600 550

No. 9 No. 10 No. 11 No. 12

No.13 No.14 No.15 No.16

Figure 14 Flight Time Analyses


27


HAIL

5.0

Pressure Altitude Pitch Angle Flight Path Angle

Airspeed

Airspeed Speed Command

0.0

-5.0

1.2

1.2

1.1

1.1

10.0


0.0 UP

Flap Angle

20.0

20.0 10.0


0.0 UP

Nose Gear

Flap Angle

1.0 0.9 30.0 (deg)

0.9 30.0 (deg)

DN 0



0.0

-5.0

1.0

Nose Gear

5.0

EPR

EPR

10,000 (ft) 8,000 6,000 4,000 2,000 0 (kt) 240 200 160 120 (deg) 10.0

Attitude

Attitude

Airspeed

Pressure Altitude

10,000 (ft) 8,000 6,000 4,000 2,000 0 (kt) 240 200 160 120 (deg) 10.0

SJSU

DN 0

100 200 300 400 500 600 700 800 900 Time (sec) (a) Case 10 run 3 (Shorter Flight Time)

100 200 300 400 500 600 700 800 900 Time (sec) (b) Case 10 run 2

Figure 15 Slower deceleration attained by the autopilot (Case 10 run 3 compared by case 10 run 2) 1.2

5.0

EPR

0.0 -5.0 0

1.0

30.0 Flap Angle


1.1

0.9


Nose Gear

Attitude

Airspeed

Pressure Altitude

10,000 (ft) 8,000 6,000 4,000 2,000 0 (kt) 240 200 160 120 (deg) 10.0

(deg)

20.0 Flap Angle Flap Lever Angle

10.0 0.0 UP DN

100 200 300 400 500 600 700 800 900 Time (sec)

0

100 200 300 400 500 600 700 800 900 Time (sec)

Figure 16 Improper Sequence of Airspeed Command Setting (Case 14 run 5)

Go Around Performance Go Around due to ATC command Figure 17 summarizes response time of go around action Δt GA ATC (sec) and altitude loss ΔH GA ATC (ft). Go around response time was defined by the time from completion of ATC controller's go around command communication t end ATC to PF's completion of setting go around SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

28


SJSU HAIL

pitch attitude. During that period, PF has to make "Go around" callout, maximum thrust setting and pitch up control to perform the go around procedure. These actions should be taken immediately after receiving the go around command. Altitude loss ΔH GAATC was defined by the difference between altitude at t end ATC and minimum aircraft altitude in a go around maneuver. The minimum altitude is usually achieved sometime after the go around actions that include maximum thrust setting and pitch up control, because of the inertia of the aircraft. To reduce altitude loss, immediate action is required by the pilots. Average response time of each case was from 3.8 to 4.7 (sec) and average altitude loss for each case ranged from 88.2 to 114.9 (ft). With SVS v/s without SVS Comparing case 5 with 6, case 7 with 8, case 13 with 14 and case 15 with 16, no major difference in average of Δt GA ATC (response time of go around action) were found between the with-SVS and without-SVS cases. Since ATC command was perceived not using the visual perception model but via the hearing perception model, this could only affect go around performance if there was competition for resources in the cognitive domain. Altitude where go around command was issued Although no major difference in response time was observed when comparing case 13 and 14 (Hatc=higher DA+250ft) with case 15 and 16 (Hatc=lower DA+250ft), response time in case 5 and 6 (Hatc=higher DA+100ft) seems longer than case 7 and 8 (Hatc=lower DA+100ft). Figure 17(b) shows duration of each task performed during the "response time." Air MIDAS's duration of each action is determined only by Markov process based on specific mean value and standard deviation and so, once initiated, duration is not affected by model's workload status or UWR status. Also, a series of go around tasks were defined as 'sequential activities', that should be serially performed. Therefore the timing of starting a series of go around tasks might be a parameter, which could be affected by the difference in altitude. From Figure 17(b)-(a), no such difference were observed in the timing of initiating "Go Around" callout. So we can conclude that ATC go around altitude does not impact the human and flight system characteristics. Timing of ATC Command Comparing case 5, 6, 7 and 8 with case 13, 14, 15, and 16 respectively, no significant relationship regarding the timing of go around command versus response time was observed. Timing of the go around command in cases 5, 6, 7, and 8 was more critical (closer to the ground and busier) than that in cases 13, 14, 15 and 16, and this was due to the 100ft difference between the visibility-change and decision altitude,. However the PF model responded to the ATC command as quickly as it could, and smoothly performed the go around tasks.


29


SJSU HAIL

Table 4 Go Around Performance 1 (Go Around due to ATC command) (a) GA Response Analysis (Case 5) Go Lever Pitch Angle HATC ATC CommandA/C Alt Callout (ft) (ft) Tstart Tend Tstart Tend Tstart Tend Tstart Tend

Hmin

t GA

HGA

Run 1 750.0 Run 2 750.0 Run 3 750.0 Run 4 750.0 Run 5 750.0 Average

781.1 783.6 784.4 785.8 783.8

782.1 784.6 785.4 786.8 784.8

737.8 737.2 737.6 737.5 737.0

785.0 786.5 787.8 789.0 787.4

787.5 788.6 790.4 790.8 789.9

(ft) 632.9 650.3 636.0 653.5 635.2


780.2 771.3 785.8 783.6 780.4

781.2 772.3 786.8 784.6 781.4

(b) GA Response Analysis (Case 6) 737.9 781.5 782.2 782.2 783.2 783.2 737.2 772.6 772.8 774.0 774.0 774.0 737.2 787.1 788.5 788.5 789.7 789.7 737.9 784.9 786.4 786.4 787.3 787.3 736.9 781.7 782.2 782.2 783.2 783.2

785.9 776.3 792.2 789.8 785.0

638.4 648.4 631.8 635.1 656.6

4.7 99.4 4.0 88.9 5.4 105.5 5.2 102.8 3.6 80.3 4.6 95.4


819.5 821.9 821.8 823.1 819.5

820.5 822.9 822.8 824.1 820.5

(c) GA Response Analysis (Case 7) 285.9 820.8 822.0 822.0 823.1 823.1 286.3 823.2 824.5 824.5 825.6 825.6 285.5 823.1 823.2 823.2 824.2 824.2 285.5 824.4 825.2 825.2 826.3 826.3 285.9 820.8 821.6 821.6 822.8 822.8

824.9 827.6 825.8 828.3 824.6

189.6 184.2 208.1 190.1 193.3

4.4 96.3 4.7 102.1 3.0 77.5 4.2 95.4 4.1 92.6 4.1 92.8


811.6 821.6 811.6 826.0 811.6

812.6 822.6 812.6 827.0 812.6

(d) GA Response Analysis (Case 8) 286.1 812.9 813.4 813.4 814.3 814.3 285.8 822.9 823.8 823.8 825.1 825.1 286.1 812.9 813.4 813.4 814.3 814.3 286.7 827.3 827.4 827.4 828.4 828.4 286.5 812.9 813.1 813.1 814.0 814.0

816.7 827.3 816.7 830.7 815.7

188.9 182.8 189.0 194.4 206.7

4.1 97.2 4.7 103.0 4.1 97.2 3.7 92.3 3.1 79.8 3.9 93.9

Go Lever Pitch Angle H ATC ATC CommandA/C Alt Callout (ft) (ft) Tstart Tend Tstart Tend Tstart Tend Tstart Tend

Hmin

t GA

HGA

(sec) 4.0 4.2 5.4 4.1 3.5 4.2

(ft) 107.3 112.8 148.9 108.1 97.5 114.9 105.5 118.4 97.4 96.1 106.6 104.8

782.4 784.9 785.7 787.1 785.1

783.8 785.3 786.8 788.0 786.1

783.8 785.3 786.8 788.0 786.1

785.0 786.5 787.8 789.0 787.4

(sec) (ft) 5.4 104.9 4.0 86.9 5.0 101.6 4.0 83.9 5.1 101.8 4.7 95.8

(e) GA Response Analysis (Case 13)


777.1 773.3 778.0 767.3 628.1

778.1 774.3 779.0 768.3 629.1

881.7 881.9 899.9 881.7 881.8

780.3 776.2 781.7 770.7 630.9

782.1 778.5 784.4 772.4 632.6

(ft) 774.4 769.1 751.0 773.6 784.3


751.6 764.9 751.4 751.3 628.1

752.6 765.9 752.4 752.3 629.1

(f) GA Response Analysis (Case 14) 882.6 752.9 753.1 753.1 754.0 754.0 880.0 766.2 767.2 767.2 768.2 768.2 881.8 752.7 752.9 752.9 754.2 754.2 882.5 752.6 752.8 752.8 754.0 754.0 881.9 629.4 630.2 630.2 631.4 631.4

756.3 770.5 755.8 755.6 633.2

777.1 761.6 784.4 786.3 775.3

3.7 4.6 3.4 3.3 4.1 3.8


809.6 809.8 809.7 785.5 808.3

810.6 810.8 810.7 786.5 809.3

(g) GA Response Analysis (Case 15) 436.1 810.9 811.3 811.3 812.2 812.2 436.4 811.1 812.6 812.6 813.8 813.8 434.6 811.0 811.7 811.7 813.1 813.1 448.4 786.8 787.5 787.5 788.8 788.8 435.9 809.6 810.0 810.0 810.9 810.9

814.5 815.3 814.7 791.2 813.2

343.7 343.9 346.9 334.1 343.5

3.9 92.4 4.5 92.5 4.0 87.7 4.7 114.3 3.9 92.4 4.2 95.8


813.7 800.3 795.5 614.8 806.2

814.7 801.3 796.5 615.8 807.2

(h) GA Response Analysis (Case 16) 436.0 815.0 815.9 815.9 817.3 817.3 436.2 801.6 802.0 802.0 803.2 803.2 437.0 796.8 797.4 797.4 798.5 798.5 429.7 616.1 616.7 616.7 617.9 617.9 435.8 807.5 807.9 807.9 808.9 808.9

818.9 805.8 800.6 619.9 810.7

346.0 334.8 343.2 357.6 352.1

4.2 90.0 4.5 101.4 4.1 93.8 4.1 72.1 3.5 83.8 4.1 88.2

778.4 774.6 779.3 768.6 629.4

779.1 775.0 780.4 769.3 629.8

779.1 775.0 780.4 769.3 629.8


780.3 776.2 781.7 770.7 630.9

30


HAIL

(i) Response Time (Summary) t GA (sec) No. 5 No. 6 No. 7 No. 8 5.4 4.7 4.4 4.1 4.0 4.0 4.7 4.7 5.0 5.4 3.0 4.1 4.0 5.2 4.2 3.7 5.1 3.6 4.1 3.1 4.7 4.6 4.1 3.9

6.0 5.0 4.0 3.0 2.0 1.0 0.0 No. 5

No. 6

No. 7

No. 8

(i) Response Time (k) Altitude Loss HGA (ft) (Summary) No. 5 No. 6 No. 7 No. 8 104.9 99.4 96.3 97.2 86.9 88.9 102.1 103.0 101.6 105.5 77.5 97.2 83.9 102.8 95.4 92.3 101.8 80.3 92.6 79.8 95.8 95.4 92.8 93.9

120 100 80 60 40 20 0 No. 5

No. 6

No. 7

No. 13No. 14No. 15No. 16 (j) Response Time


No. 8

(k) Altitude Loss

(j) Response Time (Summary) t GA (sec) No. 13 No. 14 No. 15 No. 16 4.0 3.7 3.9 4.2 4.2 4.6 4.5 4.5 5.4 3.4 4.0 4.1 4.1 3.3 4.7 4.1 3.5 4.1 3.9 3.5 4.2 3.8 4.2 4.1

6.0 5.0 4.0 3.0 2.0 1.0 0.0


Altitude Loss (ft)

Altitude Loss (ft)

Case Run 1 Run 2 Run 3 Run 4 Run 5 AVRG


time (sec)

time (sec)

Case Run 1 Run 2 Run 3 Run 4 Run 5 AVRG

SJSU

(l) Altitude Loss HGA (ft) (Summary) No. 13 No. 14 No. 15 No. 16 107.3 105.5 92.4 90.0 112.8 118.4 92.5 101.4 148.9 97.4 87.7 93.8 108.1 96.1 114.3 72.1 97.5 106.6 92.4 83.8 114.9 104.8 95.8 88.2

160 140 120 100 80 60 40 20 0 No. 13No. 14No. 15No. 16 (l) Altitude Loss

Figure 17(a) Go Around Performance 1 (Go Around due to ATC command)


31


SJSU HAIL

(m) Timing of Initiating GA actions (Summary) t GA (sec) Case 5 Case 6 Case 7 Case 8 Run 1 0.3 0.3 0.3 0.3 Run 2 0.3 0.3 0.3 0.3 Run 3 0.3 0.3 0.3 0.3 Run 4 0.3 0.3 0.3 0.3 Run 5 0.3 0.3 0.3 0.3 AVRG 0.3 0.3 0.3 0.3

(n) Duration of GA Call Out Action (Summary) t GA (sec) Case 5 Case 6 Case 7 Case 8 Run 1 1.4 0.7 1.2 0.5 Run 2 0.4 0.2 1.3 0.9 Run 3 1.1 1.4 0.1 0.5 Run 4 0.9 1.5 0.8 0.1 Run 5 1.0 0.5 0.8 0.2 AVRG 1.0 0.9 0.8 0.4

1.2 0.8 0.6 0.4

time (sec)

time (sec)

1.0

0.2 0.0

time (sec)

(o) Duration of Push Go Lever Action (Summary) t GA (sec) Case 5 Case 6 Case 7 Case 8 Run 1 1.2 1.0 1.1 0.9 Run 2 1.2 0.0 1.1 1.3 Run 3 1.0 1.2 1.0 0.9 Run 4 1.0 0.9 1.1 1.0 Run 5 1.3 1.0 1.2 0.9 AVRG 1.1 0.8 1.1 1.0 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0


Case 5 Case 6 Case 7 Case 8 (n) Duration of GA Call Out Action (p) Duration of Set Pitch Action (Summary) t GA (sec) Case 5 Case 6 Case 7 Case 8 Run 1 2.5 2.7 1.8 2.4 Run 2 2.1 2.3 2.0 2.2 Run 3 2.6 2.5 1.6 2.4 Run 4 1.8 2.5 2.0 2.3 Run 5 2.5 1.8 1.8 1.7 AVRG 2.3 2.4 1.8 2.2 3.0 2.5

time (sec)

Case 5 Case 6 Case 7 Case 8 (m) Timing of Initiating GA actions

1.6 1.4 1.2 1.0 0.8 0.6 0.4 0.2 0.0

Case 5 Case 6 Case 7 Case 8 (o) Duration of Push Go Lever Action

2.0 1.5 1.0 0.5 0.0 Case 5 Case 6 Case 7 Case 8 (p) Duration of Set Pitch Action

Figure 17(b) Go Around Performance 1 (Go Around due to ATC command Detailed Task Duration)

Go Around due to pilot decision Δt

Figure 18 summarizes response time of go around action GA PF (sec) and altitude loss go-around by PF model's decision cases. (Table 5 provides the performance data.)

ΔH GA PF

in

No major difference in response time and altitude loss was found among cases 9, 10, 11 and 12. In these scenarios, PF model needed to fixate on OTW to get the status of the runway visibility as soon as the aircraft passes the DA. The visibility scan in "Runway-In-Sight" Callout task was assumed to be performed intentionally, whenever aircraft passed DA. Whenever the pilot made the decision to go around, actions were performed at proper times in all the cases, and both SVS and decision altitude did not have any impact on the go around activities. SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

32


SJSU HAIL

Table 5 Go Around Performance 1 (Go Around due to Pilot Decision) (a) GA Analysis (Case 9) Callout Go Lever Pitch Angle

Hmin

t GA

HGA

802.0 801.7 801.3 800.1 794.1

(ft) 533.7 555.6 548.0 559.3 545.9

(sec) 5.5 4.3 4.5 3.9 4.8 4.6

(ft) 116.3 94.4 102.0 90.7 104.1 101.5

(b) GA Analysis (Case 10) 794.5 795.0 795.0 796.1 796.1 780.6 781.2 781.2 782.1 782.1 647.8 648.2 648.2 649.1 649.1 794.1 795.6 795.6 796.6 796.6 790.4 791.3 791.3 792.7 792.7

798.4 784.5 651.5 798.3 794.4

550.6 550.4 561.2 551.5 556.2

4.3 4.3 4.0 4.6 4.2 4.3

99.4 99.6 88.8 98.5 93.8 96.0

196.9 194.7 197.7 194.9 194.8

(c) GA Analysis (Case 11) 829.6 830.2 830.2 831.1 831.1 834.3 835.5 835.5 836.8 836.8 834.7 835.3 835.3 836.3 836.3 833.7 833.9 833.9 835.0 835.0 829.8 830.7 830.7 831.7 831.7

833.3 838.5 838.4 837.0 833.3

105.9 101.7 107.5 110.3 111.1

4.0 4.6 3.9 3.7 3.9 4.0

94.1 98.3 92.5 89.8 88.9 92.7

194.5 196.9 196.3 197.4 195.6

(d) GA Analysis (Case 12) 827.9 828.7 828.7 829.7 829.7 829.5 829.6 829.6 830.7 830.7 827.8 829.2 829.2 830.3 830.3 827.3 827.5 827.5 828.6 828.6 801.4 802.4 802.4 803.3 803.3

831.7 104.4 832.8 112.6 832.3 97.8 830.7 111.3 805.7 96.6

DA (ft) Run 1 650.0 Run 2 650.0 Run 3 650.0 Run 4 650.0 Run 5 650.0 Average

DA timeA/C Alt (sec) (ft) 796.5 644.4 797.4 646.7 796.8 647.1 796.2 645.7 789.3 645.3


794.1 780.2 647.5 793.7 790.2

Run 1 200.0 Run 2 200.0 Run 3 200.0 Run 4 200.0 Run 5 200.0 Average Run 1 200.0 Run 2 200.0 Run 3 200.0 Run 4 200.0 Run 5 200.0 Average

Tstart

Tend

797.0 797.7 797.0 796.6 789.7

798.2 798.7 797.9 797.2 790.8

Tstart

Tend

Tstart

Tend

799.3 800.0 799.0 798.2 792.0

799.3 800.0 799.0 798.2 792.0

645.1 645.5 649.5 645.6 647.4

829.3 833.9 834.5 833.3 829.4

827.5 829.3 827.5 827.2 801.1

798.2 798.7 797.9 797.2 790.8

(e) Timing of GA Decision (Case 9-12 Summary (sec)

(f) Altitude Loss (Case 9-12 Summary (sec) Case No. 9 No. 10 No. 11 No. 12

Case No. 9 No. 10 No. 11 No. 12 Run 1 5.5 4.3 4.0 4.2 Run 2 4.3 4.3 4.6 3.5 Run 3 4.5 4.0 3.9 4.8 Run 4 3.9 4.6 3.7 3.5 Run 5 4.8 4.2 3.9 4.6 AVRG 4.60 4.28 4.02 4.12


6.0

116.3 94.4 102.0 90.7 104.1 101.5

99.4 99.6 88.8 98.5 93.8 96.0

94.1 95.6 98.3 87.4 92.5 102.2 89.8 88.7 88.9 103.4 92.7 95.4

120 Altitude Loss (ft)

time (sec)

4.2 95.6 3.5 87.4 4.8 102.2 3.5 88.7 4.6 103.4 4.1 95.4

4.0

2.0

0.0

100 80 60 40 20 0

No. 9 No. 10 No. 11 No. 12

(g) Timing of GA Decision

No. 9 No. 10 No. 11 No. 12

(h) Altitude Loss

Figure 18 Go Around Performance 2 (Go Around due to PF model's decision) SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

33


SJSU HAIL

Scan Pattern Analysis This section is comprised of two analyses. The first one, called Scan Failure Rate, describes the development of the scan pattern to include the possibility of emergent error rates in perception. The second section on Scan Pattern Results details comparisons between different cases/ conditions. Scan Failure Rate The visual perception model of the Air MIDAS assumed failure of the scan when the duration of the fixation was not long enough to fetch displayed information. Formally, when duration_of_fixation < mean – x * SD

(x=constant),

Air MIDAS PF agent failed to fetch data from the display and did not update the UWR in the agent’s working memory. Selection of the threshold (x * SD) will affect the occurrence of failure of the scan pattern and it could be one of the important human performance parameters for prediction. Sensitivity of x was examined, to provide the guideline for threshold selection. All of the fixations in the scan pattern activities, in all the scenarios and runs were used for the analyses. In Air MIDAS' the duration of each fixation is determined by a Markov process, and it is not affected by external factors such as the status of workload. All the sample data have been integrated into a single table. Figure 19 summarizes the sensitivity analysis. Failure rate was defined by n fail n fix where n fail : the number of failure, n fix : total number of fixation,

No scan pattern failure happened for the thresholds less than -1.75SD. Failure rate increased linearly with the increase of threshold after threshold was larger than -1.50. For the series of simulation runs in this paper, we selected (mean-1.0SD) for the threshold of failure occurrence so that the error rate of scan perception is 10% or less.


34


SJSU HAIL

(a) Failure Rate

-3.00 -2.75 -2.50 -2.25 -2.00 -1.75 -1.50 -1.25 -1.00 -0.75 -0.50 -0.25 0 0.25 0.50 0.75

Success 39027 39027 39027 39027 39027 39027 39016 37157 36693 32971 32003 28962 25730 25363 23210 21226

Fail

Total

0 0 0 0 0 0 11 1870 2334 6056 7024 10065 13297 13664 15817 17801

39027 39027 39027 39027 39027 39027 39027 39027 39027 39027 39027 39027 39027 39027 39027 39027

Failure Rate 0 0 0 0 0 0 0.000 0.048 0.060 0.155 0.180 0.258 0.341 0.350 0.405 0.456

60% 50% Failure Rate (%)

x

40% 30% 20% 10%

-3.00

-2.00 -1.00 0.00 Threshold Setting (x)

0% 1.00

Figure 19 Scan Failure Rate Analyses Scan Pattern Results The scan pattern analyses will describe the results of the visual perception model in air-MIDAS. The analyses will include a series of comparisons that mostly focused on whether SVS was included or not in the scan pattern and they are as follows:

a) comparison between Normal Approach procedures with and without SVS included in the scan pattern b) comparison between Go-Around procedures with and without SVS in the scan pattern c) comparison of scan patterns in the go-around procedures with different Decision Altitudes d) comparison of scan patterns with different levels of visibility e) Comparison of PF model data with scan pattern by Mumaw et. al. (2000) a) Normal Approach (with v/s without SVS) Cases 1and 3 were combined to provide values for a scan pattern that included SVS and similarly cases 2 and 4 were combined for scan pattern results that did not include SVS. Fixation percentage, dwell duration and dwell percentages were analyzed for normal approach procedures, with and without SVS in their scan pattern.


35


SJSU HAIL

60 without SVS with SVS

Fixation Percentage

50 40 30 20 10 0 PFD

SV S

ND

EICA S

OTW

MCP

Overlapp

Display

Figure 20. Fixation Percentage for PF with and without SVS on normal approach

The Fixation data (Figure 20) shows that PF agent fixated a little more on PFD and Navigational Display when SVS was not available. This may be because SVS is designed as a display that overlays PFD. Figure 21 shows dwell durations and it is interesting to note that dwell durations are very long for SVS and MCP. This signifies that as the activity was designed in looking at SVS there is cognitive processing involved. This added cognitive process in looking at SVS elongates the fixation duration. Although the dwell durations are not too long for PFD and ND when compared to SVS patterns, overall more time is spent looking at PFD and ND (see Figure 22). This result corresponds to the fixation percentage data where the agent fixated more on PFD and ND. 4.5 without SVS with SVS

4.0

Duell Duration (sec)

3.5 3.0 2.5 2.0 1.5 1.0 0.5 0 PFD

SVS

ND

EICA S

OTW

MCP

Overlapp

Display

Figure 21. Dwell Duration with and without SVS (PF) on Normal Approach


36


SJSU HAIL

60 without SVS with SVS

Dwell Percentage

50 40 30 20 10 0 PFD

SVS

ND

EICAS Display

OTW

MCP

Overlapp

Figure 22. Dwell percentage with and without SVS (PF) on Normal Approach

b) Go-Around Procedures (with and without SVS) 60 without SVS with SVS

Fixation Percentage (%)

50 40 30 20 10 0 PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 23 Fixation percentages for Go-Around scenario with and without SVS (PF)


37


SJSU HAIL

4.0 without SVS with SVS


3.5 3.0 2.5 2.0 1.5 1.0 0.5 0 PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 24 Dwell Duration for Go around Scenario with and without SVS (PF) 18 without SVS with SVS

Dwell Percentage(%)

16 14 12 10 8 6 4 2 0 PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 25 Dwell Percentages Go-Around Scenario with and without SVS (PF)

Overall, dwell percentage reflects the dwell duration and fixation percentage. If both dwell duration and fixation percentage are the high dwell percentage goes up. The scan pattern shows dwell percentages are the same for all displays in the SVS and non-SVS cases, with a few differences. The PFD and ND have higher dwell per4centage in non0SVS cases. MCP and Overlap have higher dwell percentage in SVS cases.


38


SJSU HAIL

c) Go-Around with different Decision Altitudes


60 DH=650 DH=200

50 40 30 20 10 0

PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 26. Fixation Percentages for Go-around Scenario with different DA (650 and 200 ft)- PF 4.0 DH=650 DH=200

Dwell Duration (sec)

3.5 3.0 2.5 2.0 1.5 1.0 0.5 0 PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 27. Dwell Durations for Go-Around Scenario with different Decision Heights (650 and 200 ft)


39


SJSU HAIL

45 DH=650 DH=200

40

Dwell Percentage (%)

35 30 25 20 15 10 5 0 PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 28. Dwell Percentages G-Around for Different Decision Heights (650 and 200ft)

Figure 26, 27 and 28 show comparisons between scan patterns for different decision Heights (650 and 200ft). It is very evident that decision heights did not impact the scan patterns very significantly. The dwell percentages which are a good metric of the scan pattern have more or less the same values for the two decision height except for OTW fixations. When the decision height is lower i.e. at 200ft, the PF has longer dwell durations that contribute to the higher dwell percentage. d) Scan Pattern at different visibility 50

High Visibility Low Visibility

45


40 35 30 25 20 15 10 5 0

PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 29. Fixation Percentages for Go-Around High v/s Low visibility (PF)


40


SJSU HAIL

3.5 High Visibility Low Visibility


3.0 2.5 2.0 1.5 1.0 0.5 0

PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 30. Dwell Duration for Go-Around high v/s low visibility 45 40

High Visibility Low Visibility

Duell Percentage(%)

35 30 25 20 15 10 5 0

PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 31. Dwell Percentage High v/s low visibility (PF)

In this analysis, the PF’s scan pattern for low and high visibility were explored across normal or go-around approaches. High visibility was set at visibility at or above 10 mi, and low visibility was set at 0.2 mi. The scan pattern shows that PF fixated less on OTW under low visibility conditions and compensated the OTW with more fixations on SVS. Thus SVS was used as source of information about the external world, under low visibility conditions. e) MIDAS v/s Mumaw As a part of the validation effort, a comparison of the scan pattern of the PF-model and some empirical data on scan pattern collected by Mumaw et. al. (2000) was also done. Mumaw et. al. collected scan pattern data for pilots flying the in descent phase of flight using VNAV, and it did not have SVS on its flight deck. Thus the dwell duration and dwell percentage model data in the normal approach conditions were compared with the Mumaw data, and the results are shown in Figure 32 and Figure 33.


41


SJSU HAIL

2 Air MIDAS Mumaw

1.8

Dwell Duration (sec)

1.6 1.4 1.2 1 0.8 0.6 0.4 0.2 0

PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 32. Dwell Duration (sec) Air MIDAS v/s Mumaw 60

Dwell Percentage (%)

50

Air MIDAS Mumaw

40 30 20 10 0

PFD

SVS

ND

EICAS

OTW

MCP

Overlapp

Display

Figure 33. Dwell Duration Percentage Air MIDAS v/s Mumaw et. al.

The PF-model has higher dwell duration on PFD, EICAS and MCP than pilots studied by Mumaw et. al. The dwell percentage, which is calculated as the dwell duration on any one display over the total dwell duration on all the displays, has slightly different numbers. The PF agent’s dwell percentages are higher than the Mumaw pilot’s for PFD, ND and EICAS displays. Since we did not have fixation data available for Mumaw, we could not compare the same with the model data. However, other researchers (e.g. Bellenekes et. al, 1999) have found that the numbers of fixations are fewer when the dwell durations are long. It seems that the modeled agent is setup to look at more components than human pilots do, which is contributing to long dwell durations. The overall trend in dwell duration data between the model and Mumaw data is more or less the same. Conclusion

Prediction of human performance using the synthetic vision systems in approach, landing and go-around flight was performed by using Air MIDAS. PC plane aircraft simulation model was used for the world representation interacted by Air MIDAS pilot agents. Detailed scan pattern model was newly implemented and activities including approach, landing and go-around SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

42


SJSU HAIL

procedures, standard callout, checklist, ATC communication and landing/go-around decision were installed. Result and analysis of 80 simulation runs are summarized as follows: (1) SVS would not adversely affect the flight safety in approach, landing and go-around phase regardless of decision altitude and triggers of go-around including PF's intention at decision altitude and ATC's command, while it would allow approach and landing in conditions that would otherwise be unattainable. (2) Small delays of action initiation in flight control were observed in approach phase with SVS operation. This occurred because that the chances of fixation on each display was decreased by adding SVS to conventional display configuration, (3) No human performance degradation and no delay of task initiation were observed in landing and go around phase, though there were time shifts in the approach phase. (4) A scan pattern model which simulates pilot's instrument scan was validated by using the data of human-in-the-loop simulation. Sensitivity analysis on threshold setting for information acquisition failure model was performed and (mean-1.0SD) fixation duration was selected for the threshold of failure occurrence so that the error rate of scan perception was 10% or less. Analysis in this study was performed considering scan pattern change induced by the SVS and it did not include aspects of SVS's benefit which could potentially enhance pilot's Situation Awareness. For further enhancement of model capabilities, more detailed human internal information processing model including perception, comprehension and projection should be developed to predict SVS's features of enhancing situation awareness. Acknowledgement

The authors would like to thank Mr. Tom Marrow, Silicon Valley Systems Inc. for his technical help and support for this study. The study was undertaken for the NASA Aviation Safety Program under Grant NCC2-1305 to the San Jose State University, Dr. David Foyle. References

Corker, K.M. (2000). Cognitive Models & Control: Human & System Dynamics in Advanced Airspace Operations. (2000) in N. Sarter and R. Amalberti (Eds.) Cognitive Engineering in the Aviation Domain. Lawrence Earlbaum Associates, New Jersey. Corker, K.M., Gore, B. et al. (2003) Human Performance Modeling Predictions in Reduced Visibility Operation with and without the use of Synthetic Vision System Operations. HAIL Laboratory Report N). SVS-2-2002. San Jose State University, San Jose CA. Endsley, M.R. (2000) Situation models: An avenue to the modeling of mental models, Proceedings of the Human Factors and Ergonomic Society 44th Annual Meeting, Santa Monica, CA, 2002 Keller, J. and Leiden, K. (2002) Information to Support the Human Performance Modeling of a B757 Flight Crew during Approach and Landing, Micro Analysis and Design, Inc. June, 18 Mumaw, R., Sarter, N., Wickens, C., Kimball, S., Nikolic, M., Marsh, R., Xu, W., & Xu, X. (2000) Analysis of pilot monitoring and performance on highly automated flight decks (NASA Final Project Report: NAS2-99074). Moffett Field, CA: NASA Ames Research Center. McCracken, J.H. and Aldrich, T.B. (1984) Analysis of selected LHX mission functions: SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

43


SJSU HAIL

Implications for operator workload and system automation goals (Technical note ASI 479-02484(b)). Anacapa Sciences, Inc. Palmer, M. T., Abbott, T. A., Williams, D. H. (1997) Development of Workstation-Based Flight Management Simulation Capabilities within NASA Langley’s Flight Dynamics and Control Division, Ninth International Symposium on Aviation Psychology, Columbus, OH, April, 28-May, 1 Purcell, K., Corker, K., and Guneratne, E. (2002) Human Factors Issues and Evaluations for Commercial and Business Aircraft Synthetic Vision Systems, San Jose State University, February


44


SJSU HAIL

Appendix A. Air MIDAS Activity Design

Cockpit Layout Figure A.1 shows a cockpit layout prepared for task time calculation. It is as much size as those of large transport aircraft. Table A.1 shows coordinates of cockpit device point to be touched by pilot.

(a) Plan View

(b) Side View

Figure A.1 Cockpit Layout (Plan View) Table A.1 Cockpit Device Positions (mm) Device PF's Left Hand PF's Right Hand PNF's Left Hand PNF's Right Hand Left Wheel Left Grip Left Wheel Right Grip Right Wheel Left Grip Right Wheel Right Grip

x 930 930 930 930 930 930 930 930

y -660 -410 410 660 -695 -375 375 695

z 500 500 500 500 700 700 700 700

Device x Throttle Lever 790 Speed Brake 790 Flap Lever 790 Gear Lever 1040 Mode Control Panel 1000 Display Control Panel 1000 Front Pedestal (FMS/Right) 965 Check List 930

y 0 135 -120 310 0 450 100 760

z 600 550 500 700 980 980 450 400

Human Performance Database Hands Movement

Fit's law was used to calculate time required for moving hands. Size of each cockpit device was assumed as Table A.2. Fit's law: t t arg et = IM log 2 (D /S + .5) (msec) where IM = 100[70 ~ 120] (msec/bit) Standard deviation (SD) of hands movement was assumed as 25% of the average time ( IM =100). SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

45


SJSU HAIL

Table A.2 Cockpit Device Scales (Width mm) Device Wheel Push to Talk Switch Throttle Lever Speed Brake Flap Lever Gear Lever Mode Control Panel SW Mode Control Panel Knob Display Control Panel Knob Radio FRQ Set (FMS) Check List

w 30 5 80 30 30 30 20 20 10 30 50

Device Manipulation Time ( t dev )

Experimental data (Sundstrom et. al, 1980) of required time to manipulate cockpit devices was used for device manipulation time. Table A.3 Device Manipulation Time Control/Display Type Pushbutton Two-position toggle switch Three-position toggle switch Covered toggle switch Single rotary switch Rotary switch in an array Single thumbwheel Thumbwheel in an array Hand lever 5 deg to 10 deg movement Hand lever 10 deg to 20 deg movement Hand lever 20 deg to 40 deg movement Hand lever 40 deg to 60 deg movement Rotary knob Hand wheel Discrete indicator Analog indicator Digital indicator

Average Time (sec) 1.0400 1.1100 1.3500 1.5000 1.5800 1.6400 1.9500 2.0000 1.6500 1.8500 2.0500 2.2500 1.6900 2.3900 0.2500 2.0000 0.7500

Speech Rate( f speech ) and Cognitive Cycle( τ cgn )

Speech Rate(Sundstrom et. al, 1980) and cognitive cycle(Card et. al, 1983) data were used to calculate duration of speech and cognitive activities.


46


SJSU HAIL

Table A.5 Speech Rate( f speech ) and Cognitive Cycle( τ cgn )

f Speech Rate speech Cognitive Cycle

τ cgn

Unit (word/sec) (sec)

Average 166 0.0070

Task Time Calculation Methods Task time was calculated by the following procedures. Speech

t speech = words ⋅ f speech (msec) words: The number of words contained in a sentence. Standard deviation (SD) of speech time was assumed as 30% of the average time. Also, noise factor of 500 (msec) was added as variable delay between speech events.

SDspeech = 0.3⋅ t speech + 500 For radio communication tasks, time required to push push-to-talk switch, which is calculated in Manipulation section, was added to t speech . Hearing

Haring time was assumed to include processing time of sentence decoding in addition to speech time. Sentence was divided into chunk(s) in which have certain meanings. For example, a sentence "SBR tower, NASA 123, Over GOLET at 8000, GPS RNAV 33L approach, Information Z, Request Landing" was divided into 8chunks; "SBR tower(Receiver ID)," "NASA 123(Sender ID)," "Over GOLET(Position)," "8000(Altitude)," "GPS RNAV(Approach Procedure)," "33L approach(Expected Runway)," "Information Z(ATIS information)," and "Request Landing(Request)." Processing time was calculated by multiplying chunk(s) by cognitive cycle.

t hear = t speech + n chunks ⋅ τ cgn SD of processing time was assumed as 30% of average processing time. Also, noise factor of 500 [msec] was added as variable delay between hearing events.

SDhear = 0.3⋅ t speech + 0.3n chunks ⋅ τ cgn + 500 Manipulation

Manipulation time was calculated by adding required time for moving hands to a target and device manipulation time of a target device. All actions except those expected to follow preceding action immediately performed start from nominal hand position and end at completion of device manipulation. Actions expected to follow preceding action immediately starts from hands position on a device manipulated in the preceding action. Nominal positions of each pilot's hands were assumed on his/her knees (since we are simulating automatic flight). SJSU/NASA Coordination of Air MIDAS Safety Development Human Performance Modeling: NASA Aviation Safety Program

47


SJSU HAIL

t manip = t hand + t dev SD of hands movement time was assumed as 25% of its average time and SD of manipulation time was assumed as 30% of its average time. So, SDmanip = 0.25 ⋅ t hand + 0.3⋅ t manip Air MIDAS Task Time Based on the above methods, duration of each procedural task item were calculated. Table A.6 Speech and Hearing Tasks Sentence

Words

Speech (sec) Hearing (sec) Chunks Av. SD Av. SD

Moving Hand (sec) Av. SD

"NASA123, Make Go Around."

7

2.530

0.759

2

2.670

0.801

4.451

1.291

"Flap 5"

2

0.723

0.217

1

0.793

0.238

N/A

N/A

"Gear Down, Flap 20s, Speed plus 5"

7

2.530

0.759

3

2.740

0.822

N/A

N/A

"Gear Down"

2

0.723

0.217

1

0.793

0.238

N/A

N/A

"Flap 20"

2

0.723

0.217

1

0.793

0.238

N/A

N/A

"Landing Check List"

3

1.084

0.325

1

1.154

0.346

N/A

N/A

"Down"

1

0.361

0.108

1

0.431

0.129

N/A

N/A

"GOLET, Missed Approach 5000 ft"

5

1.807

0.542

2

1.947

0.584

N/A

N/A

"SBR tower, NASA 123, Over GOLET at SA.alftf, GPS RNAV 33L approach, Information Z, Request Landing"

21

7.590

2.277

8

8.150

2.445

9.512

2.809

"NASA 123, Clear to Land RWY 33L"

11

3.976

1.193

3

4.186

1.256

N/A

N/A

"Roger Cleared to Land RWY 33L, NASA123"

12

4.337

1.301

4

4.617

1.385

6.259

1.834

"Cleared to Land RWY 33L"

7

2.530

0.759

4

2.810

0.843

N/A

N/A

"Runway In-Sight"

3

1.084

0.325

1

1.154

0.346

N/A

N/A

"Stabilize"

1

0.361

0.108

1

0.431

0.129

N/A

N/A

"NASA 123, Going Around"

6

2.169

0.651

2

2.309

0.693

4.090

1.183


48


SJSU HAIL

Table A.7 Manipulating Tasks Manipulation

Pilot

Av.(Sec) SD(Sec)

Set MCP Knob

PF

2.290

0.657

Push MCP Mode Switch

PF

1.640

0.462

Set Flap Lever

PNF

2.292

0.665

Set Flap Lever After Gear Down

PNF

2.103

0.608

Set Gear

PNF

1.520

0.435

Push Talk Switch on Wheel

PNF

1.921

0.532

PF

2.097

0.607

PNF

0.524

0.131

PF

2.453

0.698

PNF

2.133

0.618

Disengage Autopilot on Wheel

PF

1.421

0.407

Push Go Lever

PF

1.040

0.312

GA Decision (=Cognitive Cycle)

PF

0.070

0.021

Set Speed Brake Lever Pick Up Check List Set DA on Display control Panel Set Radio Frequency

REFERENCE: Roskam, J; Airplane Design, PART III: Layout Design of Cockpit, Fuselage, Wing and Empennage: Cutaways and Inboard Profiles, Roskam Aviation and Engineering (1989) Card, S., Moran, T. and Newell, A.; The Psychology of Human Computer Interaction, Lawrence Erlbaum Associates (1983) Sundstrom, J. L., NASA TLA Workload Analysis Support Volume 1 Detailed Task Scenarios for General Aviation and Metering and Spacing Studies, NASA CR 3199 (1980)


49



SJSU HAIL

50