Human Factors Report: TMA Operational Evaluations 1996 & 1998 · 2016. 10. 13. · 2. INTRODUCTION The Traffic Management Advisor (TMA) is a component of the Center-TRACON Automation

NASA/TM–2000-209587

Human Factors Report:TMA Operational Evaluations 1996 & 1998

Katharine K. Lee, Cheryl M. Quinn, Ty Hoang, and Beverly D. Sanford

February 2000

Since its founding, NASA has been dedicated to theadvancement of aeronautics and space science. TheNASA Scientific and Technical Information (STI)Program Office plays a key part in helping NASAmaintain this important role.

The NASA STI Program Office is operated byLangley Research Center, the Lead Center forNASA’s scientific and technical information. TheNASA STI Program Office provides access to theNASA STI Database, the largest collection ofaeronautical and space science STI in the world.The Program Office is also NASA’s institutionalmechanism for disseminating the results of itsresearch and development activities. These resultsare published by NASA in the NASA STI ReportSeries, which includes the following report types:

• TECHNICAL PUBLICATION. Reports ofcompleted research or a major significant phaseof research that present the results of NASAprograms and include extensive data or theoreti-cal analysis. Includes compilations of significantscientific and technical data and informationdeemed to be of continuing reference value.NASA’s counterpart of peer-reviewed formalprofessional papers but has less stringentlimitations on manuscript length and extentof graphic presentations.

• TECHNICAL MEMORANDUM. Scientific andtechnical findings that are preliminary or ofspecialized interest, e.g., quick release reports,working papers, and bibliographies that containminimal annotation. Does not contain extensiveanalysis.

• CONTRACTOR REPORT. Scientific andtechnical findings by NASA-sponsoredcontractors and grantees.

The NASA STI Program Office . . . in Profile

• CONFERENCE PUBLICATION. Collectedpapers from scientific and technical confer-ences, symposia, seminars, or other meetingssponsored or cosponsored by NASA.

• SPECIAL PUBLICATION. Scientific, technical,or historical information from NASA programs,projects, and missions, often concerned withsubjects having substantial public interest.

• TECHNICAL TRANSLATION. English-language translations of foreign scientific andtechnical material pertinent to NASA’s mission.

Specialized services that complement the STIProgram Office’s diverse offerings include creatingcustom thesauri, building customized databases,organizing and publishing research results . . . evenproviding videos.

For more information about the NASA STIProgram Office, see the following:

• Access the NASA STI Program Home Page athttp://www.sti.nasa.gov

• E-mail your question via the Internet [email protected]

• Fax your question to the NASA Access HelpDesk at (301) 621-0134

• Telephone the NASA Access Help Desk at(301) 621-0390

• Write to:NASA Access Help DeskNASA Center for AeroSpace Information7121 Standard DriveHanover, MD 21076-1320

NASA/TM–2000-209587


Katharine K. Lee, Cheryl M. Quinn, and Ty HoangAmes Research Center, Moffett Field, California

Beverly D. SanfordSterling Software, Mountain View, California

February 2000

National Aeronautics andSpace Administration

Ames Research CenterMoffett Field, California 94035-1000

Available from:

NASA Center for AeroSpace Information National Technical Information Service7121 Standard Drive 5285 Port Royal RoadHanover, MD 21076-1320 Springfield, VA 22161(301) 621-0390 (703) 487-4650

AcknowledgmentsThe authors would like to thank the controllers, traffic management coordinators, and area supervisors ofthe Ft. Worth Center, DFW TRACON and Towers for their valuable input and feedback to the CTASdevelopment process, and for their patience and participation in the 1996 Operational Evaluation and inthe 1998 Daily Use Field Survey data collection efforts. We would also like to acknowledge the help ofthe NTX Engineering Support Team and the NTX Researchers; in particular, we would like to thankTom Turton for all his work on behalf of the 1998 Daily Use Field Survey. We would also like to expressour gratitude to Michelle Foster (Ft. Worth Center TMU) for providing helpful input and sharing herexpertise on ATC and traffic management operations.

iii

CONTENTS

Abstract........................................................................................................................................................................................1

1. Acronyms ................................................................................................................................................................................1

2. Introduction .............................................................................................................................................................................2

2.1 Background ...........................................................................................................................................................................3

2.1.1 Traffic Flow Management .................................................................................................................................................3

2.1.2 TMA Development and Testing........................................................................................................................................3

2.1.3 Human Factors Assessments .............................................................................................................................................4

3. 1996 Human Factors TMA Operational Evaluation..............................................................................................................4

3.1 Objectives and Methods........................................................................................................................................................4

3.2 Results and Discussion .........................................................................................................................................................6

3.2.1 Center TMC General Impressions.....................................................................................................................................6

3.2.2 Center Sector Controller Data ...........................................................................................................................................6

3.2.2.1 Data Collection................................................................................................................................................................6

3.2.2.2 CTAS Features Ratings ..................................................................................................................................................6

3.2.3 Center General Impressions...............................................................................................................................................8

3.3 1996 Operational Evaluation Summary ...............................................................................................................................9

3.4 Recommended Issues for Future Evaluations.....................................................................................................................9

4. 1998 Daily Use Field Survey................................................................................................................................................10

4.1 Background .........................................................................................................................................................................10

4.1.1 Airspace Changes.............................................................................................................................................................11

4.1.2 Dual Route Operations.....................................................................................................................................................11

4.2 1998 Daily Use Field Survey Objectives ...........................................................................................................................12

4.3 Methods ...............................................................................................................................................................................12

iv

4.4 Results and Discussion .......................................................................................................................................................13

4.4.1 Center TMC Summary Data............................................................................................................................................13

4.4.1.1 Equipment and CTAS Displays....................................................................................................................................13

4.4.1.2 ATC and CTAS Experience and Training ...................................................................................................................13

4.4.1.3 Metering Operations .....................................................................................................................................................14

4.4.1.4 Dual Route Operations..................................................................................................................................................14

4.4.1.5. Use of CTAS................................................................................................................................................................14

4.4.1.6 General Impressions......................................................................................................................................................17

4.4.2 Center Sector Controller Summary Data ........................................................................................................................17


4.4.2.2 Metering and Dual Routes Operations.........................................................................................................................17

4.4.2.3 Use of CTAS.................................................................................................................................................................18


4.4.3 TRACON TMC Data.......................................................................................................................................................21



4.4.3.3 Metering Operations .....................................................................................................................................................21

4.4.3.4 Dual Route Operations..................................................................................................................................................21

4.4.3.5 Use of CTAS.................................................................................................................................................................21


4.4.4 Tower TMCs ....................................................................................................................................................................22


4.4.4.2 Use of CTAS.................................................................................................................................................................22

4.5 1998 Daily Use Field Survey Summary.............................................................................................................................22

5. Recommendations.................................................................................................................................................................25

5.1 User Interface ......................................................................................................................................................................25

5.2 Functionality........................................................................................................................................................................25

5.3 Training ...............................................................................................................................................................................25

5.4 Documentation ....................................................................................................................................................................26

v

6. Concluding Remarks.............................................................................................................................................................26

7. References .............................................................................................................................................................................27

Appendix A. Examples of the TGUI and the PGUI ................................................................................................................28

Appendix B. Questionnaires Used in the 1998 Daily Use Field Survey.................................................................................31

Appendix C. Draft TMA Quick Reference Card for Center TMCs........................................................................................44

Appendix D. Draft TMA Quick Reference Card for Center Sector Controllers ...................................................................45


KATHARINE K. LEE, CHERYL M. QUINN, TY HOANG, BEVERLY D. SANFORD*

Ames Research Center

ABSTRACT

The Traffic Management Advisor (TMA) is a component of the Center-TRACON AutomationSystem (CTAS), a suite of decision-support tools for the air traffic control (ATC) environmentwhich is being developed at NASA Ames Research Center. TMA has been operational at the ATCfacilities in Dallas/Ft. Worth, Texas, since an operational field evaluation in 1996. The OperationalEvaluation demonstrated significant benefits, including an approximately 5% increase in airportcapacity. This report describes the human factors results from the 1996 Operational Evaluation andan investigation of TMA usage performed two years later, during the 1998 TMA Daily Use FieldSurvey. The results described are instructive for CTAS-focused development, and provide valuablelessons for future research in ATC decision-support tools where it is critical to merge a well-defined,complex work environment with advanced automation.

1. ACRONYMS

AAR Airport Acceptance Rate

ACID Aircraft ID

ACT Waco (Specialty): Southwest gate in thepre-Metroplex airspace

ARTCC Air Route Traffic Control Center, alsoreferred to as “Center”

ASD Aircraft Situation Display

ASP Arrival Sequencing Program

ATC Air Traffic Control

BYP Bonham (Specialty): Northeast gate in theMetroplex airspace

CM Communications Manager (module ofCTAS)

CQY Cedar Creek (Specialty): Southeast gate inthe Metroplex airspace

CTAS Center-TRACON Automation System

D-side Data Position Sector Controller

* Sterling Software, Mountain View, California.

DFW Dallas/Ft. Worth (typically refers toDFW TRACON)

DSR Display System Replacement

ETA Estimated Time of Arrival

FAA Federal Aviation Administration

FZT Frankston (Specialty): Southeast gate inthe pre-Metroplex airspace

GUI Graphical User Interface

JEN Glen Rose (Specialty): Southwest gate inthe Metroplex airspace

MF Meter Fix

MiT Miles-in-Trail

NOAA National Oceanic and AtmosphericAdministration

NTX North Texas Field Site

OJT On-the-Job (referring to training)

pFAST The Passive Final Approach Spacing Tool

PGUI Planview Graphical User Interface

PVD Planview Display

2

R-side Radar Position Sector Controller

Red-X Warning indicator that appears on the TGUIdisplay when there is an update loss

SFH Spatial Freeze Horizon

SGFF Single Gate Free Flow

SPS Wichita Falls (Specialty): Northwest gate inthe pre-Metroplex airspace

STA Scheduled Time of Arrival

TGUI Timeline Graphical User Interface

THD Threshold

TMA Traffic Management Advisor

TMC Traffic Management Coordinator

TMU Traffic Management Unit

TRACON Terminal Radar Approach Control, alsoreferred to as “Terminal Area”

TXK Texarkana (Specialty): Northeast gate in thepre-Metroplex airspace

UKW Bowie (Specialty): Northwest gate in theMetroplex airspace

ZFW Ft. Worth ARTCC (Center)

2. INTRODUCTION

The Traffic Management Advisor (TMA) is a componentof the Center-TRACON Automation System (CTAS), asuite of decision-support tools for the air traffic control(ATC) environment which is being developed at NASAAmes Research Center in cooperation with the U.S.Federal Aviation Administration (FAA). TMA has beenused operationally on a daily basis at the Ft. Worth AirRoute Traffic Control Center (ARTCC, or Center) andDallas/Ft. Worth Terminal Radar Approach Control(TRACON) since an operational field evaluation of TMAin 1996. This report summarizes the human factors resultsfrom the 1996 Operational Evaluation of TMA as well asa follow-on investigation (conducted in 1998) as part ofthe TMA Daily Use Field Survey.

Ft. Worth Center and Dallas/Ft. Worth TRACON haveserved as the test sites for TMA research and develop-ment activities since 1994. Traffic managementcoordinators (TMCs) and controllers from Ft. WorthCenter (ZFW) and Dallas/Ft. Worth TRACON (DFW)

have been continually involved in the design and develop-ment of CTAS tools, providing their expertise in thecreation and evaluation of new CTAS functionalities.CTAS development is human-centered (ref. 1), withhuman factors issues investigated and addressed through-out the development process. Iterative prototyping isemphasized in the CTAS development process,incorporating the feedback from the end-user as newconcepts are tested. Human factors engineers, integratedinto the tool development teams, help to ensure that end-user concerns are not overlooked. Human factorsengineers also help to determine requirements andprovide human factors-based assessments as the systemsare developed. The reader is referred to the CTASwebsite: http://www.ctas.arc.nasa.gov for more detailedinformation on CTAS development and the other CTAStools.

Both engineering and human factors data were collectedduring the Operational Evaluation of TMA, conducted inJune-July 1996 at ZFW. As reported in reference 2, theOperational Evaluation determined that there was anincrease in airport acceptance rate (AAR) of approxi-mately 5% and a reduction in delays of 2-3 minutes underTMA operations. Some of the preliminary human factorsfindings were reported in reference 2; Section 3 of thisreport contains the complete human factors results andconclusions from the 1996 Operational Evaluation, whichwere based on observations of TMC interactions withTMA and questionnaires and interviews with TMCs andCenter Sector controllers.

The 1996 results formed the basis for some of theoperational questions that were posed in the TMA DailyUse Field Survey, conducted at ZFW/DFW in July 1998.It was expected that there would be changes in how TMAwas used in an operational evaluation versus on a daily-use basis. The Daily Use Field Survey focused on boththe Center and the TRACON; observations, question-naire, and interview data were collected to examine TMAinteractions in a daily-use environment as well as tounderstand the impact of some significant airspace andprocedures modifications that were implemented inOctober 1996 (these changes are described in detail inSection 4.1.1). The TMA system was also made availableto the DFW Tower just prior to the July 1998 evaluation,and some limited observations were conducted in theTower. An internal trip report and training recommenda-tions were produced from this field survey. Section 4 ofthis report fully summarizes the human factors resultsfrom the 1998 Survey and contrasts them with the 1996Evaluation. The 1998 Daily Use Field Survey results canbe used as a baseline against which pending and futurechanges to TMA functionality will be evaluated. Periodic

3

follow-on studies are planned to examine how TMAusage continues to change over time, especially assignificant functionality is introduced.

This report contains specific references to ATCoperations at the Ft. Worth Center and DFW TRACONfacilities as they relate to TMA functionality and features.It is assumed that the reader has a general knowledge ofair traffic control operations as they pertain to traffic flowmanagement, and also has some familiarity with TMA.Specifics regarding the functionality of the TMA softwarecan be obtained from the TMA Reference Manual (ref. 3).

2.1 Background

2.1.1 TRAFFIC FLOW MANAGEMENT

Traffic flow management in the en route environment isrequired to regulate arrival traffic into a terminal areawhen the traffic demand exceeds capacity. The CenterTMCs, as part of the traffic management unit (TMU), areresponsible for the management of the traffic flow in theen route environment. TMCs evaluate the overall trafficdemand and determine how the Center will deliveraircraft safely and efficiently to the TRACON. Becausethe TRACON is the receiving facility, the TRACONdetermines the overall AAR and conveys other restric-tions, as needed, to the Center. If the amount of trafficthat the TRACON receives from the Center exceeds theTRACON’s capacity, the TRACON may elect to “shutthe door” on the Center, meaning they may elect to stopaccepting arrivals from the Center. Shutting off the Centerhas the operational effect of arrival aircraft being held inthe Center’s airspace until such time as the TRACON canonce again accept the aircraft. How traffic is held variesfrom facility to facility; there are some TRACONs thatcan hold aircraft in their airspace, but in many facilities,TRACONs do not have enough airspace to safely holdaircraft, and holding is accomplished in the Center’sairspace.

To prevent overloading the TRACON, the Center mayuse several procedures for managing the arrival flow oftraffic into a terminal area from the en route airspace.Two of the primary means of achieving traffic flowmanagement are Miles-in-Trail (MiT) or Time-basedMetering; MiT is the most commonly-used method fortraffic flow management (ref. 4). Under MiT operations,aircraft in the Center are delivered over a feeder fix with aparticular in-trail spacing between aircraft. Under meter-ing operations, aircraft are scheduled to cross a meteringfix at a particular time.

Under either procedure for traffic flow management, theCenter sector controllers typically vector or use some

speed control within their sectors to achieve the requiredin-trail spacing or to cross the aircraft over the meter fixesat the assigned time, “absorbing” the delay that isassigned to the aircraft. It is possible that even with suchmethods in place, some holding may still be required; forexample, at ZFW, when the delays exceed 6 minutes, thecontrollers often establish a holding pattern at a fix withintheir sector.

MiT operations are procedurally simpler than meteringoperations (ref. 4); metering operations require controllersto follow the schedule presented on a metering list, whichshows each flight with its corresponding fix crossing timeand an assigned amount of delay. In general, some formof automation is needed to generate the metering sched-ule. The Arrival Sequencing Program (ASP) was ametering scheduling system which was in operation atZFW prior to the Operational Evaluation of TMA in1996. Given the complexity of meeting the meteringschedule, the controller workload associated withmetering operations is generally higher for metering thanfor MiT operations (M. Foster, personal communication,1999). However, research has shown that time-basedmetering is more efficient than MiT spacing of aircraft,and results in lower overall delays (ref. 4).

2.1.2 TMA DEVELOPMENT AND TESTING

TMA uses flight plans and radar tracks of arrival traffic tocalculate Estimated Times of Arrival (ETAs) to variousreference points (the meter fix, the final approach fix, andthe runway threshold). TMA further calculates ScheduledTimes of Arrival (STAs) to these locations based on legalseparation requirements, the overall traffic conditions, theconstraints imposed by the TRACON (such as AAR) andaircraft performance models. The STAs form the basis forthe overall schedule and sequence of aircraft which areconveyed to the TMCs via two types of graphical userinterfaces, the Timeline Graphical User Interface (TGUI)and the Planview Graphical User Interface (PGUI).Through these two interfaces, TMCs view the currenttraffic and scheduled delays projected over an amount oftime into the future (depending upon the availability ofradar data). TMA enables TMCs to plan changes to thetraffic flow and to view the results of such changes. TheTMA schedules and delays may also be provided to theCenter Sector Controller. However, the TGUI and PGUIare not available for the sector controller to view at theradar position; the schedule information is presented tothe controllers via metering lists on their radar displays.Consequently, the presentation of the schedulinginformation is determined by the capabilities of theexisting radar displays and the associated software in the

4

Center facility. Figures depicting typical TGUI and PGUIdisplays can be found in Appendix A.

Although TMA is meant to be a tool used by CenterTMCs, it is also used to convey traffic flow and trafficmanagement information to the TRACON TMCs. TMAprovides a comprehensive picture of the traffic flow intothe terminal area, and as the Center conducts its flowmanagement based on TMA information, the TRACONTMU can better understand the overall traffic picture andthe Center’s traffic decisions. The basic TGUI and PGUIfeatures were, in large part, determined through previousTMA development activities and human factors assess-ments at Denver Center and Denver TRACON; theseearly recommendations for feature use and appearance aredocumented in reference 5.

Currently, the Denver ATC facilities, as well as ATCfacilities in Atlanta, Miami, and Los Angeles, haveoperational TMA systems that have limited schedulingfunctionality (the schedules are not presented to the sectorcontrollers, and TMA is not used for metering). At thetime of this writing, ZFW and DFW are the first of manysites that have the TMA functionality planned for nationaldeployment as part of Free Flight Phase I.1 While theresults described in this report are focused on the opera-tions at only the ZFW/DFW field sites, it is possible todraw inferences that would benefit the understanding ofhow TMA might function in other facilities with differentairspace and air traffic management requirements.

2.1.3 HUMAN FACTORS ASSESSMENTS

Human factors assessments are conducted throughout thedevelopment of the CTAS tools. These human factorsassessments focus on the tool’s usability, suitability, andacceptance (ref. 6). These categories form the basis forthe data that are described in this report and are defined asfollows:

Usability: perceptually-based aspects of the human-computer interface, such as keystrokes, detectabilityof colors and text, and equipment manipulation.

1 Free Flight Phase I is an FAA program under whichseveral air traffic management decision-support tools willbe deployed to ATC facilities nation-wide. Included in thecapabilities are two of the CTAS tools, TMA and thePassive Final Approach Spacing Tool (pFAST). Moreinformation about Free Flight Phase I can be found on thewebsite: http://ffp1.faa.gov.

Suitability: information content and representationfor the users’ tasks, including questions of workloadand the way in which the tool supports the desiredwork functions.

Acceptance: reflecting usability and suitability of thesystem, as well as job satisfaction, demonstrableperformance, and self-esteem (ref. 7).

TMA human factors issues were examined in 1993 at theDenver ATC facilities, and the findings were used toprovide feedback on the look and feel of the userinterface, as well as document how TMA was incorpo-rated into daily operations. As detailed in reference 5,TMA was shown to be a significant aid in determining thetraffic load, evaluating whether or not metering would berequired, and how long a metering period should last.TMA enabled the Center and TRACON to work from acommon picture of the traffic demand, and facilitatedtheir coordination regarding the airport acceptance rateand configuration changes. The TRACON additionallyused TMA information to help make staffing decisions.This assessment of TMA provided many insights intohow TMA was likely to be received at other ATCfacilities.

In contrast to the Denver assessment of TMA, whichfocused primarily on how TMA was used to enhanceTMC situational awareness, the human factors assessmentobjectives described in this report describe how the TMA-generated schedules were utilized for metering operationsat ZFW and DFW. In both the 1996 and 1998 evaluations,the TMCs and controllers were asked to make specificcomparisons between TMA and ASP schedules. Thehuman factors results described in this report areinstructive for CTAS-focused development, and highlightissues for future research in ATC decision-support toolswhere it is critical to merge a well-defined, complex workenvironment and advanced automation.

3. 1996 HUMAN FACTORS TMAOPERATIONAL EVALUATION

3.1 Objectives and Methods

The goal of the 1996 operational evaluation, conductedover 4 weeks in June-July at ZFW, was to examine theusefulness of the TMA functionality and the acceptanceof the user-interface according to the ZFW TMCs andsector controllers (ref. 8). The engineering results aredescribed in references 2 and 8 and describe the accuracy

5

of STAs2 and delay values. Sequencing efficiency, the useof available landing slots, and runway balancing werealso measured.

Human factors objectives concerned questions of how thescheduling information was displayed, and the workloadassociated with, and the usefulness of, the TMA-calculated schedules and sequences compared to ASP.Questionnaires were presented to TMCs and CenterSector Controllers to specifically collect ratings ofworkload and system acceptance, and suggestions forsystem improvement. Observations were also conductedat arrival sectors during test rushes, as staffing permitted.Although the TRACON and Tower were affected byZFW decisions based on TMA, there was no formalassessment made of the impact of TMA on TRACON orTower operations. Some anecdotal data from theTRACON TMU were collected.

The sector controller metering list displays were notaffected by CTAS information beyond the addition of asingle line to the list header to indicate that CTASmetering was in progress. However, changes were madeto the nature of the information provided in the meteringlists, and what the controller could do with this informa-tion. The three new features that were added with CTAS,the manual swap, delay countdown, and outer arccrossing times, are defined in table 1.

2 The accuracy of the STAs is a function of algorithmsresponsible for the ETA and STA calculations. ASPcalculations are based on simplified models of the airtraffic system. ASP schedules all aircraft to a singlereference point that represents all of the runwaythresholds and uses a general model of aircraft speeds(ref. 9); it creates schedules based on the AAR withoutconsidering separation requirements between aircrafttypes or specific runway constraints (refs. 2 and 9).CTAS, in contrast, uses much more accurate aircraft statemodels and precise routing to specific runway thresholds.As delays are derived from the difference between theETA and STA values, a delay can be considered accuratebased the accuracy of these ETA and STA calculations.In addition, as the aircraft state changes, the delay count-down information that was provided would increase ordecrease to reflect the current aircraft conditions, thusproviding a more dynamic measure of the delay to beabsorbed.

Table 1. Sector Controller Features.

Feature Description

Manualswap

Allows the sector controller to swap themeter times between two aircraft in themetering list. A number of keyboardentry inputs are required to achieve theswap. Swapping the meter timesbetween two aircraft effectivelychanges the sequence of the aircraft inthe meter list.

Delaycountdown

Delay values are provided on themetering list associated with eachaircraft. The countdown provided thecontroller with information on theamount of delay remaining to beabsorbed. (The ASP delay informationremains static, and does not change toindicate how much delay is left.)

Single outerarc meteringlist (outerarc crossingtimes)

For sectors in which there are aircraftcrossing more than one outer fix, thecrossing times were incorporated into asingle list referenced to an outermetering arc (an adaptable, fixed radialdistance from the meter fix) (ref. 2).

One of two TMCs was dedicated to working with TMAduring the Operational Evaluation. Observations andinterviews were conducted in the Center TMU. Theobservations consisted of noting TMC interactions withTMA, the information gathered from other data sourceswithin the TMU, TMC coordination with other TMCs andArea Supervisors, as well as coordination with otherfacilities. Following each test rush, the TMC working thetest rush was interviewed regarding TMA’s performanceand the impact on workload.

In 1996, the arrival airspace at ZFW was divided up into4 arrival area specialties responsible for the aircraftarriving from 4 geographical locations: Wichita Falls(SPS) in the Northwest, Texarkana (TXK) in theNortheast, Waco (ACT) in the Southwest and Frankston(FZT) in the Southeast. Observations were conducted atboth the high and low arrival sectors in an area, andfocused on manual swap entries, discussions about theTMA-generated sequences and crossing times, and anydeviations from the TMA sequences at the boundarycrossings. Questionnaires were distributed to sectorcontrollers at the conclusion of each metering period, andwhen possible, individual controllers were interviewed.

6

A total of 154 sector controller questionnaires werecollected. Due to limitations on available personnel fordata collection, and the desire to concentrate the observa-tions and questionnaire collection from the sectors mostimpacted by metering operations, most of the humanfactors data were gathered from the busier arrival sectors.Data from the TXK specialty were collected morefrequently than from the other sectors.

3.2 Results and Discussion

3.2.1 CENTER TMC GENERAL IMPRESSIONS

The human factors data collected in the TMU indicatedthat TMA use contributed to delay reduction, smoothertraffic flow and improved situational awareness. Specifi-cally, the ZFW TMCs subjectively estimated a 2-3 minutedelay reduction per aircraft, on average, across the vari-ous test periods. TMCs from both ZFW and DFWreported that the flow of traffic into the TRACON wassmoother when TMA was in operation.

The Center TMCs also reported that their situationalawareness was improved with the display of the singleouter arc metering list and the delay countdown on thesector plan view displays (PVDs). The added informationhelped the TMCs to more quickly grasp the trafficconditions as they monitored the individual sectors. TheTMCs did not report any significant impact upon theircoordination as a result of using TMA for metering.

3.2.2 CENTER SECTOR CONTROLLER DATA

3.2.2.1 Data Collection

Questionnaires were distributed to controllers in the4 arrival area specialties. Approximately 33% of thequestionnaire data were collected from TXK and 31%were collected from FZT (the East-side specialties), andthe remaining questionnaire data were collected from SPSand ACT. The questionnaire data were collected fromboth R-side and D-side controllers.3 Approximately 78%of the questionnaire data were collected from the R-sidecontrollers.

3 Center sector controllers can be assigned to severalpositions, including R-side (the radar position) and D-side(the data position). The R-side controller is primarilyresponsible for issuing control instructions to the aircraft(ref. 10). The D-side controller is generally responsiblefor assisting the R-side controller; when the D-sideposition is active, this controller typically takes care ofcoordination and managing the flight progress strips.

3.2.2.2 CTAS Features Ratings

After each metered test rush period, the Center SectorControllers rated the workload associated with using thenew features of CTAS (compared to ASP): the singleouter arc metering list, scheduled times, sequences,keyboard inputs for the manual swap feature, delayinformation, metering list ripples,4 and the overallworkload associated with using CTAS. The controllerswere also asked to rate the acceptability of these features.The sector controller workload results are plotted infigure 1, which depicts the mean ratings for the differentelements, along with the standard deviations indicated bythe vertical bars. As the workload ratings show, each ofthe TMA features provided some decrease in controllerworkload. Each of the scheduling, sequencing and delayfeatures was rated as slightly reducing controller work-load. While the amount of list ripples were rated betweenminimally increasing and not affecting workload, notethat the overall workload associated with CTAS was ratedas a minimal decrease in workload.

The sector controller mean acceptability ratings (with thestandard deviations represented by the vertical bars) areplotted in figure 2. The acceptability results show that allof the CTAS features were rated as being somewhatacceptable, with the exception of the list ripples, and theoverall acceptability of CTAS use was rated betweensomewhat and completely acceptable. The controllersrated the TMA scheduled times as more acceptable thanthe ASP scheduled times.

4 The controllers use the metering list to plan the controlof their aircraft. During metering, there may be conditions(such as change in AAR, airport configuration, andweather deviations) which may require the TMU tochange the sequence and delay times of the aircraft. Thisis known as a “ripple” of the schedule (and of the meter-ing list) which may initially cause confusion and increasecontroller workload. Consequently, the TMU tries toavoid rippling the metering list once a metering periodhas begun (M. Foster, personal communication, 1999).

7

1

2

3

4

5

6

7Greatly DecreasedWorkload

Somewhat DecreasedWorkload

Minimally DecreasedWorkload

Did not AffectWorkload

Minimally IncreasedWorkload

Somewhat IncreasedWorkload

Greatly IncreasedWorkload

Single Outer Arc Meter List

Scheduled Times

Sequences

Manual Swap Inputs

Delay Information

List Ripples

Overall Workload

Associated with CTAS

Sector ControllerWorkload Ratings

Figure 1. Center Sector Controller Mean Workload Ratings (standard deviations represented by vertical error bars)

Sector Controller Acceptability Ratings

Completely Acceptable

SomewhatAcceptable

Neither Acceptable

nor Unacceptable

Somewhat Unacceptable

Completely Unacceptable

Single Outer Arc Meter List

Scheduled Times

Sequences

Manual SwapInputs

DelayAccuracy

DelayDistribution

List Ripples

Overall Accep tab i l i t y

of CTAS

Figure 2. Center Sector Controller Mean Acceptability Ratings

8

Sector controllers also reported their satisfaction withCTAS. They indicated that they were slightly moresatisfied with CTAS overall, compared to ASP. Severalcontrollers said that they would prefer not to meter at all,but that the use of CTAS is preferable to the use of ASP.Additionally, many sector controllers commented posi-tively on the reduced delays during TMA test periods.

3.2.3 CENTER GENERAL IMPRESSIONS

General Comments: Controllers indicated that CTASmetering operations produced more accurate measures ofdelay compared to ASP metering. As a result, meteringappeared to begin later under CTAS operations, andholding would happen later as well. One controller notedthat when ASP was in operation, its use led to second-guessing the system because sometimes ASP wouldassign clumps of times to controller positions or wouldleave huge gaps in times when a sector was very busy.As a result, metering was seen as increasing the overalltraffic complexity, and controllers would tend todisregard the metering times and just work to separatethe aircraft.

Single Outer Arc Metering List: The single outer arcmetering list received the highest acceptability ratings,and of all the features that were individually assessed, itwas rated as reducing workload the most. Controllersreported that the single outer arc list reduced the amountof information integration required to meter in the highsectors. As mentioned before, TMCs also found thisfeature useful. They reported that the single list reducedthe amount of time needed to understand the sectorcontroller’s plan when they quick-look5 the sector.

Aircraft Sequences: With regard to the sequence ofaircraft displayed on their metering lists, the controllersfrequently reported that they did not refer to thesequences in the metering list (regardless of whetherCTAS or ASP was being used for metering) and itappeared that this was prompted by experience with ASPsequence inaccuracies. Some controllers who did refer tothe sequences often reported that the lists seemed to bemore accurate than those provided by ASP.

5 Quick-look is defined as the ability to display thedatablocks of the tracked aircraft from other controllerpositions (ref. 10). When a TMC quick-looks a con-troller’s sector, s/he is able to see the traffic that thecontroller is working, and also view exactly what infor-mation is displayed on sector radar scope, including thesector’s metering list.

Manual Swap Feature: Use of the manual swap featuredepended primarily on sector staffing and traffic levels.R-side controllers working without a D-side often did notuse this feature, especially on busy sectors. However, thisfeature was used often by sectors with D-side controllers,and it was not clear whether the R-side or the D-side waschiefly using the swap feature. Traffic levels dictatedwhether or not the D-side controller position was staffed,and whether the R-side controller would be able toconvey the intended sequence to the D-side controller.Depending upon the load on the R-side controller, theplanned aircraft sequence did not always match themetering list, as the D-side did not necessarily know thesequence that was planned until an aircraft was turned inby the R-side controller.

When the swap feature was used, three main benefits ofmanually swapping aircraft were identified. First, itprovided a more centralized source of information withinthe sector. Rather than having to reconcile the traffic flowon the radar scopes with the positioning of the flightprogress strips (which generally reflect the sequence thatthe controller was planning), it was possible to just referto the controller’s radar scope in order to view all of thisinformation. Second, entries made in the high sector werereflected at the lower sector, thus providing more accurateinformation for the next controller. Third, since the listbetter represented the radar controller’s plan for the trafficflow, it reduced the time needed by TMCs to understandthe traffic flow when quick-looking the sector.

One of the most frequent complaints about the manualswap feature was that only one pair of aircraft could bemoved in the metering list per entry. The controllers oftenwanted to be able to swap (or resequence) several aircraftin the list. With the functionality available during the1996 Operational Evaluation, that would require severalentries to specify the order of aircraft.

Delay Countdown: Initially, controller opinion about thedelay countdown feature seemed to be divided. However,through the course of the test, controller acceptance ofthis feature increased as the countdown information wasshown to be reliable. Controllers reported that thecountdown provided useful feedback about delayabsorption.

Metering List Ripples: Although the TMCs typically tryto avoid rippling the list once metering has begun, thereare conditions that will require a change in the scheduleand consequently a change in the controller’s meteringlist. According to controller ratings, the list ripples duringthe TMA test did not meaningfully contribute to theirworkload. The acceptability rating for list ripples nearedthe “somewhat acceptable” anchor. Controllers

9

commented that ripples seemed to occur less frequentlywith TMA than with ASP. They also commented thatwhen the list did ripple, it seemed to result in a moreaccurate, usable list.

3.3 1996 Operational Evaluation Summary

The comparison of TMA and ASP in the human factorsdata is based on anecdotal evidence and specific questionsaimed at this comparison; there was no opportunity tocollect baseline human factors data (such as ratings orsystematic observations during metering with ASP).However, the impressions of the TMCs and SectorControllers echo the overall improvements that werefound in the engineering analyses. The engineering datadetermined that under TMA metering, there were moreaccurate meter fix crossing times, per-aircraft delayreductions, and overall, better delay distribution (ref. 2).Additionally, DFW TMCs reported that the length of thefinals was more consistent through the rush during TMAtest periods. Samples from a particular rush during whichTMA was used demonstrated a 70 second average delayreduction with TMA versus ASP schedules (ref. 2).TMCs subjectively reported that TMA usage resulted insome per-aircraft delay reductions of about 2 minutes, andthe controllers reported a noticeable reduction in delays.Several controllers said that given the traffic conditions,they felt that the delays would have been higher withASP. The manual swap, delay countdown, and the singleouter arc metering list features contributed to theobserved improvements in accuracy of meter fix crossingtimes (ref. 2). The Sector Controllers reported that thescheduled times were, in general, more accurate than theASP scheduled times, even with list ripples. The con-trollers reported better spacing of aircraft arrival intervals.Both TMCs and Sector Controllers reported all of theTMA features to be acceptable and useful.

In terms of overall workload, the Sector Controllersreported that TMA provided a minimal decrease in theirworkload and there was no individual TMA feature thatwas reported to increase controller workload. In somecases, the controllers said that the use of TMA seems tohave prevented the need for metering and that they feltthis contributed significantly to reducing their workload.As expected, the list ripples were rated as minimallyincreasing their workload. While this might be seen as anegative finding, the controllers’ comments suggestedthat with TMA, the overall disruption caused by listripples was reported to be less than with ASP, and theTMA-based list ripples were not rated to be unacceptable.In addition, the increased workload from list ripples underTMA also underscored the basic problem that ASP listripples often resulted in generally unusable lists;controllers had become accustomed to ignoring the ASP

metering lists because they were often inaccurate. WithTMA providing better, more usable metering lists, thecontrollers had to start paying attention to meeting themetering times, thus creating more workload.

The discussion of the workload results demonstrates thecomplexity of metering. While implementing meteringdoes help to make the traffic more manageable from asystem-wide perspective, metering creates extra workloadfor the individual controllers (ref. 4). Thus, controllersand TMCs will consider any metering operation, bydefault, an increase in their workload. How much moreworkload is incurred from TMA versus ASP must beconsidered, and whether this increase in workload isacceptable, given the more efficient traffic conditions thatgenerally result from traffic management practices.

Traffic levels also interact with the interpretation ofworkload. The traffic levels change how the controllermanages the traffic situation; as traffic levels increase,the controller may not have opportunities for as muchstrategic planning as there is less time to coordinate withthe D-side controller or other controllers, and the con-troller may resort to reacting to the traffic. The manualswap feature, intended to improve overall system effi-ciency, may not have been utilized as much due to theextra effort required. The manual swap feature might havebeen used more frequently when the traffic was lighter;however, lighter traffic might also mean that controllerswould be more likely to make mental swaps (as it wouldbe easier to mentally keep track of fewer aircraft).

The evaluation of the manual swap feature alsohighlighted the difficulty in conveying its benefits to thecontrollers. While a metering list with an old schedule isnot necessarily detrimental within an individual sector,the lack of updated information has implications for thecontroller in the next, receiving sector, who would nothave the most accurate schedule on her/his metering list.As this inaccuracy propagated, controllers would continueto evaluate the slots rather than associating the specificaircraft assigned to each slot. It remained to be deter-mined if the controllers would eventually begin to use themanual swap feature over time, with more training orpractice, or if some change to the user interface might berequired.

3.4 Recommended Issues for FutureEvaluations

The human factors findings from the 1996 OperationalEvaluation resulted in a number of issues that warrantedfollow-up once TMA was in daily operation at ZFW.These issues focused on both improving the data collected

10

and examining how operations might have changed andaffected the use of TMA:

• A manual sequence feature was requested so thatmore than one aircraft could be moved in themetering list at a time. This feature was imple-mented and later examined in the 1998 DailyUse Field Survey.

• Most of the data from the TMU reflected theopinions of TMCs who were familiar with CTASand TMA development. Though these responseswere thorough, a wider range of respondents andopinions was desired for future assessments.

• Human factors observations were limited, andprimarily reflected observations at the heavysectors/meter fixes. Data from lighter sectorscould be less informative (as the controllerswould be less likely to use the CTAS features orevaluate the times) but lighter sectors should beassessed in case other issues were overlooked.

• The times of day during which the testing tookplace meant that the same crews provided inputto the questions. This created fatigue for thecrews, and may have created some bias in thequestionnaire responses.

• Some of the data collected might have beenuneven because of the distribution of responsi-bilities by the R-side and the D-side controllers.The use of the swap feature in busy sectorsdepends upon how well the R-side and D-sidecontrollers are able to communicate, and theworkload level of the D-side.

• Due to limited research personnel, no data weregathered from the DFW TRACON or Tower;this resulted in a lack of information on how thedownstream facilities were affected by TMAmetering operations.

• To detect any benefits from TMA metering, thetraffic levels must be adequate. Traffic levelsinfluence noticeable delays, holding, andwhether or not the controllers are able to payattention to the metering list.

4. 1998 DAILY USE FIELD SURVEY

4.1 Background

Since the 1996 Operational Evaluation, ZFW has had asingle TMA system that is used to make operationaltraffic management decisions. This operational systemalso feeds a “repeater” system at the DFW TRACONwhich allows the DFW TRACON TMU to view the sameinformation that ZFW has. The repeater is automaticallyupdated with the scheduling inputs from ZFW’s system.The DFW TMU also has two additional TMA systems,which do not interact with the Center. One is used for theTRACON to conduct its own scheduling evaluation; theTRACON TMU may update this system according to thechanges on the Center system, but may additionally testout different configurations or schedule modificationswithout impacting the Center’s system. The otherTRACON TMA system is used to view Passive FAST(pFAST) schedules. Beginning in April 1998, TMAdisplays (which are also repeaters of the Center’s data)were added to the three DFW Towers.

Currently at the DFW/ZFW facilities (also collectivelyknown as the North Texas Field Site, or NTX), the FAAand its contractors provide daily use operational supportthrough a team of engineers who work with the facilitiesto maintain and troubleshoot the CTAS tools. Also atNTX are a group of NASA scientists and contractors whoconduct research at the field site. Both groups workclosely together and with the users to provide feedbackon tool development and daily use issues.

Modifications have been made to the CTAS functionalityand user interface since 1996. Some of these changeshave been the result of informal evaluations conducted atDFW/ZFW, in which the TMCs have identified changesto help improve the system’s usability. For example, themanual sequence feature was implemented as a directresult of the 1996 Operational Evaluation. The manualsequence feature allows the sector controller to changethe existing sequence of aircraft to better match the timeslots. The controller may specify a desired sequence inthe metering list of up to 5 aircraft at a time. A number ofkeyboard inputs are required to enter the sequence in themetering list. To use this feature, the controller must havetrack control of the aircraft, and the sequencing can onlybe done for like aircraft types (all jets or all props).

11

A number of operational changes occurred following the1996 Operational Field Evaluation. A significant airspaceredesign was implemented, as well as the addition of adual route operation. These two significant changes aredescribed below, and provide a context for understandingthe changes in TMA use.

4.1.1 AIRSPACE CHANGES

The Ft. Worth Center/DFW TRACON airspace wasmodified in October 1996 for the purposes of accom-modating the projected increase in DFW Airport traffic(M. Foster, personal communication, 1999). The newairspace is commonly referred to as the Metroplex. Theold navaids were decommissioned and new Metroplexcornerpost fixes were created. The new arrival gates andfixes are depicted in figure 3. The DFW TRACON wasexpanded by a radius of approximately 10 miles and newarrival routes into DFW were added. A new parallelrunway was also opened at DFW Airport, enabling triplesimultaneous approaches. As a result of the airspacechange, CTAS software adaptation changes were requiredto incorporate the new arrival routes into the TRACON.

4.1.2 DUAL ROUTE OPERATIONS

Dual route operations were introduced into theZFW/DFW traffic management procedures following theairspace change. A dual route allows arrival traffic to berouted over a secondary metering fix at a particularcornerpost. Because these aircraft assigned to the dualroute are not counted toward the AAR, this enables theCenter to feed more aircraft into the TRACON thandesignated by the AAR. By taking these aircraft out of theschedule, the Center can also remove 3-4 minutes ofdelay from the system. The Center will request theoperation of a dual route from the TRACON, and ifgranted, the TRACON specifies the number of aircraftthat can be accommodated on the dual route. Only onedual route is granted at a time; the dual route may begranted over one corner, and then switched to anothercorner later in the rush. Aircraft on a dual route areseparated by in-trail restrictions (typically 10 miles-in-trail) and the TMU uses TMA to suspend the aircraft fromfurther CTAS scheduling (so that they are not counted inthe AAR).

Bowie(UKW)

Bonham(BYP)

Cedar Creek(CQY)

Glen Rose(JEN)

MASTY (parallel)

BAMBE (primary)

JONEZ (parallel)

KARLA(primary)

HOWDY

JAGGO (parallel)

FEVER(primary)

JUMBO (parallel)

DFW Airport

Figure 3. Metroplex Airspace Arrival Gates (in bold) and Primary and Parallel Arrival Fixes (figure not to scale).Satellite airport arrival fixes are not represented in this figure.

12

4.2 1998 Daily Use Field Survey Objectives

A number of issues motivated the 1998 Daily Use FieldSurvey effort. With the enlarged airspace and new dualroute operation, it was naturally anticipated that the use ofTMA might change and it was of interest to determinehow flexible TMA would be to such operational changes.In addition to documenting these effects upon TMAusage, there were six other general areas objectives forinvestigation:

1. Follow-up to 1996 Operational Evaluation Issues.

The 1996 Operational Evaluation, and subsequentfield reports, revealed some TMA use issues thatwarranted further investigation, including:

• The manual swap and manual sequencefeatures were not reported to be widely usedby the controllers, although the manualswap feature was favorably received duringthe 1996 Operational Evaluation.

• The impact of TMA operations on theTRACON was not documented in the 1996Operational Evaluation.

2. Transition to Full-Time Usage.

Since 1996, ZFW and DFW transitioned from merelytesting TMA functionality to using it full-time tomanage traffic. The incorporation of TMA into dailyoperations meant that the system was used morefrequently and more regularly by a larger pool ofusers than the small cadre of TMCs who had beenactively involved in its development. A formal fieldsurvey under daily use operations was needed tocreate a baseline for comparison with futureTMA/CTAS enhancements.

3. Anticipation of Evolving Traffic FlowManagement Strategies.

Previous studies suggested that the increasedexposure to TMA in the operational environmentwould lead to new strategies of traffic flowmanagement (ref. 5).

4. Anticipation of New TMA Use Strategies.

The use of the TMA features themselves wereexpected to evolve; previous reports suggested thatthe TMC’s strategy for using the timelines or thetraffic load graphs to make metering decisions wouldbe affected by training and increased exposure toTMA operations (refs. 5 and 11).

5. New Status and Scheduling Panel Information inthe TGUI.

The CTAS software release just prior to the FieldSurvey included significant changes to the appear-ance of the TMA Scheduling and Sequencing panels,which would likely have long-term effects on usepatterns, training, and understanding of TMA. Theseissues needed to be documented, and any problemswith the new interface needed to be captured.

6. Continued Follow-Up as Part of the CTASDevelopment Process.

Despite the fact that TMA development wassufficiently mature to be included in plans fornational deployment, continued follow-up andevaluation of TMA is a critical part of the CTASsoftware development process.

4.3 Methods

To meet the objectives outlined above, observations,interviews, and questionnaire data were collected fromFt. Worth Center, DFW TRACON, and the DFW Towers.The Daily Use Field Survey consisted of a pre-test phaseand a test phase. During the pre-test phase, questionnaireswere distributed to controllers and TMCs to get feedbackto determine major areas of concern regarding TMA/CTAS use. Engineering data gathered during the pre-testphase, which included recordings of adherence to theTMA sequences at the meter fix, were analyzed todetermine when sequence swapping occurred at eachcornerpost. The test phase consisted of observationsduring metering operations and interviews with TMCsand controllers regarding TMA use. Specific areas thatwere explored in the questionnaires are described intable 2. (The questionnaires used in the pre-testing phasecan be found in Appendix B.)

13

Table 2. Specific Issues for 1998 Daily Use Field Survey.

User Group Issues

CenterTMCs

• How TMA is used for metering,and how often certain features areused.

• Conditions under which the TMCsrevert to ASP for metering.

• What they like/dislike aboutCTAS/ASP for metering.

• How TMA is used under meteringand dual route operations.

CenterSectorControllers

• How often manual swap or manualsequence entries are made, anddifficulties encountered in usingthese features.

• If the manual swap or manualsequence entries are not used, whynot.

• The acceptability of the sequencesand schedules.

• The effect of CTAS metering(compared to ASP) uponworkload.

TRACONTMCs

• Comparison of the TRACON’sown TMA system and a TMAsystem that served as a “repeater”of the Center’s TMA system.

• How TMA is used under meteringand dual route operations.

• How helpful the TMA features arein terms of workload and inmaking staffing decisions.

• The acceptability of the TMAfeatures.

4.4 Results and Discussion

The results of the Daily Use Field survey and thediscussion of the findings are organized below accordingto the three main TMA user groups: Center TMC, CenterSector Controller, and TRACON TMC. The results arealso organized into the following topic areas:

1. Equipment and CTAS displays

2. ATC and CTAS experience and training

3. Metering

4. Dual route operations

5. Use of CTAS

6. General impressions

4.4.1 CENTER TMC SUMMARY DATA

4.4.1.1 Equipment and CTAS Displays

The ZFW TMU has CTAS information displayed onseveral monitors. At the time of the Daily Use FieldSurvey, two monitors were devoted to TGUI informationand two were devoted to PGUI information. One TGUIdisplayed timelines with all traffic to DFW Airport,except for the dual route. A second TGUI displayedtimelines of traffic over the dual route and Dallas-LoveField arrivals. A load graph for the traffic to DFW Airportwas also displayed on this monitor. The two PGUImonitors showed maps (which were configured differ-ently at different times, according to TMC preferences)and sequence list information. The PGUI was alsoconfigured to show timelines depicting aircraft on thedual route.

4.4.1.2 ATC and CTAS Experience and Training

The ZFW TMU is staffed by approximately 20 TMCs.The questionnaire data from the TMU were obtained fromseven TMCs. The TMCs who responded to the question-naires were generally a less-experienced group of users;the seven TMCs had an average of 2.7 years of TMCexperience, with a range of 4 months to 5 years. Theamount of CTAS training time that these TMCs reportedranged from 0 to 24 hours, with a mean of 10.8 hours(note that the amount of training time is generally relatedto the amount of TMC experience; more experiencedTMCs have potentially had more opportunities fortraining). Overall, the TMCs reported their comfort levelwith CTAS as 2.8 (on a 0 [very uncomfortable] to 4 [verycomfortable] scale). The range of comfort levels reportedwas 2 to 4.

The interviews were conducted with 13 TMCs. TheseTMCs had from 6 months to several years of TMCexperience. The interviews consisted of general questionsregarding the usage of TMA, as well as specific questionsthat were raised from the questionnaire results.

14

4.4.1.3 Metering Operations

From the questionnaire data, the TMCs reported that onaverage, the facility meters 2.5 times a day; they use thedual route procedure in addition to metering 2.1 times perday. They reported that they revert to ASP for metering2.3 times per month. When CTAS was used for metering,the TMCs reported rippling the list less frequently, onaverage, than when ASP was used: only 1.6 times permetering period under CTAS (with a range of 1 to2.5 times), and 4 times per metering period for ASP (witha range of 2 to 9 times).

In general, metering now occurs for shorter periods oftime (usually 20 minutes or less); it is stopped whendelays go below 4 minutes or once the sequence of all thetraffic is set. In the past, metering would last an entirerush (approximately 30 minutes). The reason for thischange is partially due to the change in the airspace, aswell as the implementation of the dual route procedure.The TMCs estimated that metering is unnecessary 95% ofthe time when the dual route procedure is used. Due topersonal preferences, some TMCs may feel that they needto do all they can to avoid metering. Consequently, somemay prefer to run dual route operations.

4.4.1.4 Dual Route Operations

As described in Section 4.1.2, use of the dual routeprocedure is negotiated between the Center and theTRACON. In general, the TRACON will grant up to8 aircraft per rush to fly over the dual fix. Of the 8 slots,6 slots are used on average. By moving aircraft over tothe dual route, as the system delay is reduced, metering issometimes eliminated for a particular rush.

The dual route procedures have evolved over time.According to the Center, the TRACON was initiallyreluctant to accept aircraft on the dual route and wouldlimit the total number of aircraft allowed on a dual routeto 2-3. Now, during some rushes, the TRACON willaccept a higher number of aircraft on the dual routes inorder to assist in the front-loading process.6

6 “Front-load” is a term typically associated with meteringfacilities. During a rush period, there is a gradual buildupof the traffic level. At the beginning of the rush, there isinsufficient pressure put on the airport by the number ofarriving aircraft. As a result, it is possible to relax con-straints into the TRACON so that the traffic is allowed toarrive without traffic management restrictions (no addi-tional delays) until the airport capacity is reached.Metering would not be implemented until after front-loading had occurred.

Initially, the Center only used the dual route if therewasn’t an available slot in the primary stream. Now theCenter tries to use all of the dual route slots granted fromthe TRACON, even if aircraft slots are available on theprimary stream. Six to 8 aircraft placed on the dual routemay reduce the need for metering; metering may still berequired if only 3 to 4 aircraft are placed on the dualroute.

There is generally a constant stream of traffic over theBonham cornerpost (BYP), and the TMCs reported that60% of the time that there is a rush over BYP the dualroute procedure is used. Forty percent of the time thatthere is a rush over the Bowie cornerpost (UKW) the dualroute procedure is used. The Center TMU will keep trackof which aircraft were first and last on the dual route, andhow many aircraft altogether used the dual route. Despitethis record-keeping, the TMU didn’t have a way todirectly measure how much the dual route operationsreduced their need for metering.

When metering and dual routes are being used together,TMCs will sometimes estimate which aircraft will bebound for the dual route and suspend those aircraft fromscheduling. Although the TMCs can generally guesswhich aircraft will be on a dual by looking for ties (wheretwo aircraft appear to be arriving at the same merge pointat approximately the same time), if aircraft assumed to bebound for the dual routes end up taking the primary route,or if controllers pick the aircraft for dual route at the lastminute, and the aircraft are inside the freeze horizon,7

there is no efficient way to take the dual route aircraft outof the schedule without rippling the list. It should benoted that at the time of the Daily Use Field Survey, therewere no firmly-established procedures for determininghow to use metering and the dual route proceduretogether; this was still an evolving process.

4.4.1.5. Use of CTAS

The TMCs were asked to report how frequently they usethe different CTAS features on a scale of 0 to 4, where“0” represented never and “4” represented often. Theplots of the mean ratings are divided into three groups:

7 The freeze horizon (or “freeze”), as described in thispaper, is also known as the STA Freeze, or the time afterwhich TMA generally stops performing regular sched-uling calculations for an aircraft. The freeze horizondiffers between stream classes and is site-dependent(ref. 9).

15

figure 4 shows the main TGUI features (such as timelines,delay information, and the information contained on thefunction panels), figure 5 shows the Scheduling Options(which are more specific scheduling features that are alsopart of the TGUI interface), and figure 6 shows the PGUIand Single Outer Arc Metering List results. The plotsdepict the mean rating values, with the standard devia-tions indicated in parentheses along the y-axis with the

feature’s legend. The TMCs who responded to thequestionnaires indicated that they never used the featureswhich are found on the F4 panel: Proposed Flights, Noton Timeline Flights, and Departure Time. These featuresare typically used for scheduling internal departures (fromairports within the Center’s airspace to the primaryairport), and have been found to be used more frequentlyat other sites.

Figure 4. TGUI Features: Mean Ratings of Feature Use (standard deviations in parentheses with legend).

Figure 5. TGUI Scheduling Menu Features: Mean Ratings of Feature Use (standard deviations in parentheseswith legend).

16

Figure 6. PGUI Features and Single Outer Arc Meter List: Mean Ratings of Feature Use (standard deviationsin parentheses with legend).

As illustrated in figure 5, many of the scheduling featuresavailable to the TMC through the scheduling menu arenot frequently utilized. The TMCs themselves observedthat they lacked training on these features, and this lack oftraining contributed to their infrequent usage.

The TMCs reported (in the questionnaire and interviewdata) that they use the CTAS displays to give them trafficinformation, delay information, and to keep track of dualroutes. They use a variety of strategies to determinewhether or not to meter. The TMCs will check the delayinformation on a traffic load graph; one traffic load graphwas frequently displayed (zoomed in), that showed all theexpected and planned traffic, and the number of minutesof delay. A load limit line was set at 4 minutes and theTMCs noted when the delay plot exceeded this limit toindicate that metering could be needed. Several TMCssaid that they use this information as the first cue, andthen they consult other sources of traffic information(such as the Aircraft Situation Display [ASD]). It ispossible to view front-loading with the traffic load graphs,and they are able to monitor the reduction in the trafficload due to metering.

The TGUI timelines are used to check the delays, andcomparisons are made between delays shown on meter fix(MF) and runway threshold (THD) timelines to determinethe actual delay that the Center has to absorb. Five to sixminutes of Center delay as shown on the THD timelinestypically indicates that metering is necessary. SomeTMCs reported that prior to the freeze horizon, theysometimes see large delays for the unfrozen aircraftwhich prompt them to reschedule several times, ratherthan to wait and see what schedule results after the freeze.

As indicated in responses to the open-ended questions, thedelay information represented on the MF versus THDtimelines is sometimes confusing and leads to questionsabout whether or not metering is needed. The TMCsnoted that there were specific prop flights each day thatare assigned excessive delay (11-12 minutes) because oftheir position; their schedules are frozen later, and theirslots are lost to jets. One or two of these flights areaffected each day in the “noon balloon” (the rush thatoccurs around the noon hour).

The TMCs were asked to describe the conditions underwhich they do not use CTAS. While metering is oftenunnecessary when the weather is clear, CTAS may alsonot be used when the weather conditions result in theclosure of one or more cornerposts. Usually, the TMUchanges to a MiT operation if a corner is shut down.Some of the TMCs may not be aware that TMA can beused in such conditions, by utilizing the blocked intervalfeature to block the unavailable arrival fixes during theclosure.

If CTAS is not working properly, and metering appears tobe necessary, the TMU may elect to revert to ASP. Thereasons they reported reverting to ASP included: theRed-X graphical warning message on the TGUI (or otherindication of CTAS software failure), shuffling of delaytimes, or the appearance of unequal delay distribution.

The PGUI sequence list is not used extensively, and thenewer TMCs, who have not had as much experience withASP, generally do not consult the sequence list. Thedifference between the information provided on thesequence list versus the timelines is that on the sequence

17

list (like the PVD linear list), the aircraft are displayedwith their exact numerical ETAs and STAs. On the TGUI,however, the aircraft times are presented as an aircraft tagon a moving timeline, and the exact time must be esti-mated from the timeline position. The TMCs did notindicate that making such estimates was a problem.

The TGUI and the PGUI are both used to monitor aircrafton the dual route. The TMCs suspend the aircraft fromscheduling via the PGUI’s sequence list or timelines. Theaircraft assigned to the dual route are displayed on asingle PGUI timeline (referenced to the THD) on a moni-tor adjacent to the main TGUI display. The controllersdecide which aircraft get assigned to the dual route; oftenaircraft naturally fit there based on their current routing. Itshould be noted that although there are timelines on theTGUI, there is still use for timelines as part of the PGUI.In fact, the PGUI timelines were rated as the most-frequently used of the PGUI features.

4.4.1.6 General Impressions

TMCs were asked what features they liked or dislikedabout using the ASP system versus the CTAS system formetering. In general, the TMCs responded that ASP wassimpler to use. They found that it was easier to determinewhen metering should begin and with which aircraft.With ASP, there was little need to interpret the data.However, the TMCs also noted that ASP was less flexi-ble, and that the metering times were less accurate andsometimes unrealistic (when two aircraft were assignedthe same crossing time).

TMCs indicated that CTAS was more complex, requiringmore training than ASP. Other TMCs indicated that theCTAS information was easy to read and apply, and thatthe presentation of the traffic load graphs was especiallyhelpful. One TMC noted that with CTAS operations,metering occurred less frequently. The delay informationthat was provided by TMA required some interpretationin order to determine when to begin metering.

From the questionnaires, it was clear that not all of theTMA functionality was being exercised. Overall, only7 of 20 TMCs responded to the questionnaire; those whodid fill out the questionnaire were less-experienced withTMA/CTAS, reporting only about 10 hours of training onaverage; however, these TMCs reported feeling “verycomfortable” using the CTAS system.

Interviews with TMCs revealed many problems thatfurther underscored the need for additional training. OneTMC said that CTAS does not efficiently assign runwaysto alleviate congestion on the East side. However, he wasnot aware of the TMA manual assign feature that could be

used to schedule aircraft to the West side as needed.Another issue brought up was that some TMCs who aremore familiar with the system may turn a feature like rushalert on and then new TMCs don’t know how to turn itoff. While there appears to be a fairly standard way inwhich TMA information is displayed in the TMU, thisdoes not necessarily account for which details of theTGUI displays may be enabled. Another effect on theassessment of TMA usage was that at the time of ourinterviews, the weather had been unusually clear and dryfor several months, with no thunderstorms or otherweather phenomena. The weather was characterized as“severe clear.” As a result, no one could recall recentlyexercising all of TMA’s functionality.

Some TMCs reported that they will never use all ofTMA’s capabilities because they only use TMA formetering. This is another issue that could be a function oftraining. It is clear that any added training should includethe opportunity to observe the system in use during actualtraffic rushes, under varying traffic conditions. The TMCsshould be given the chance to test the different features ina shadow-mode, or by shadowing TMCs who are veryfamiliar with TMA operations.

4.4.2 CENTER SECTOR CONTROLLER SUMMARYDATA


Questionnaire responses were received from 32 control-lers in 4 specialties, with the following distribution: 22%from Bowie, 19% from Bonham, 28% from Glen Rose(JEN), and 31% from Cedar Creek (CQY). Some of thecontroller data below were supplemented with additionalinterviews with the controllers regarding the manual swapand manual sequence features and TMA/CTAS meteringin general.

The respondents reported working approximately8.2 metering periods per month. On average, the respon-dents had 12 years of ATC experience, and had worked11.6 years at a level 3 facility, and 9.7 years at ZFW.

4.4.2.2 Metering and Dual Routes Operations

In addition to the effect of training, controller responsestowards CTAS were also influenced by their opinionsregarding metering. Some of the controllers clearly feltnegatively about metering and indicated that meteringincreased their workload. As a result, their opinions ofCTAS were negative because CTAS was used as ametering tool.

18

Dual route procedures are not clearly defined (as notedabove). A general operating practice is that controllerswill use dual routes instead of vectoring aircraft on theprimary route, when possible. The sector controllers maynot always use the dual routes, or they may not alwaysuse all the available dual route “slots.” Alternatively, thecontrollers may sometimes leave an aircraft on the dualroute if it works out better for the aircraft, even if there isa slot on the primary route.

4.4.2.3 Use of CTAS

Due to winds or aircraft performance, the controllers mayneed to change the sequence of the aircraft. The swapfeatures that were implemented allowed either the R-sideor D-side controller to manually swap a pair of aircraft(manual swap) or change the sequence of up to 5 aircraft(manual sequence). Seventy-two percent of the question-naire respondents had used the manual swap feature, 47%had used the manual sequence feature, and 60% reportedhaving used the delay countdown feature. The controllerquestionnaire posed some possible reasons for not usingthe manual swap or manual sequence features. From thesechoices, the controllers reported the following reasons fornot using the features (in order of frequency):

• Too busy.

• Easier to make manual swaps using the strips.

• Too many keyboard entries required.

• Some controllers indicated that when they wereworking as R-side controllers, the D-sidecontroller was responsible for making the swapentries.

• Prefer to use the strips to make the swaps(regardless of the ease of using the manual swapfeature).

Although some controllers noted that having the sequenceset up properly by the high sector would mean that swapswere not necessary for the low sector, many controllersreported that they were never trained on the use of themanual swap and manual sequence entries and they didnot know where to access the instructions on how to usethe features. In fact, there was no documentation at thepositions indicating the commands necessary to use themanual swap and manual sequence features. Somecontrollers did use the swap feature for props in lowaltitude. Others mentioned that they did not care aboutmatching the aircraft IDs (ACIDs) and the slots; theymatched whatever aircraft was convenient with anavailable slot.

Controllers who were very familiar with the swap featuresfelt that they were very helpful. It is likely that withpractice, the keyboard inputs will not seem to add toomuch workload, and eventually the swap features will begenerally regarded as helpful. Some controllers indicatedin the questionnaire data that they were too busy to usethe swap features, or that it was easier to have a list onpaper (though they acknowledged that if the metering listrippled, that makes the paper list unusable).

The mean ratings of how difficult it was to use the manualswap, manual sequence, and delay countdown features areplotted in figure 7 (the vertical bars represent the standarddeviations). The scale used was from 0 to 4, where “0”represented Very Difficult and “4” represented VeryEasy.

The delay countdown feature was rated towards the easyend of the scale. The swap features were rated near theneither easy nor difficult anchor, but towards the difficultend of the scale.

The controllers were asked to rate the acceptability of thefollowing CTAS features: scheduled times, sequences,delays at the sector, the delay countdown feature, thesingle outer arc metering list, and the overall acceptabilityof CTAS. The mean acceptability ratings are plotted infigure 8 (the vertical bars represent the standard devia-tions of the ratings). The scale used was from 0 to 4,where “0” represented Completely Unacceptable and “4”represented Completely Acceptable.

The acceptability of the CTAS features were nearly allrated between Neither Acceptable nor Unacceptable andCompletely Acceptable. The acceptability of the singleouter arc metering list was rated 3.1 on this scale;favorable comments were made indicating that the singleouter arc list was helpful in figuring metering times.Overall CTAS acceptability was rated 2.6.

The controllers were also asked to rate the workloadassociated with using the following CTAS features ascompared with ASP: the scheduled times at the sector, theaircraft sequences, the delays at the sector, and the delaycountdown feature; these results are plotted in figure 9(vertical bars representing standard deviations). (As thiswas a CTAS versus ASP comparison, the controllers werenot asked for a workload rating associated with the singleouter arc metering list, which is only available duringCTAS metering.) The scale used was from 0 to 4, where“0” represented Better with ASP and “4” representedBetter with CTAS.

19

Sector Controller Ratings of Difficulty of Feature Use

0

1

2

3

4

Very Diff icult

Neither Easynor Difficult

VeryEasy

Manual Swap

Manual Sequence

Delay Countdown

Figure 7. Center Sector Controller Mean Ratings of Difficulty of Feature Use (standard deviations representedby vertical bars).

Sector Controller Acceptability Ratings

0

1

2

3

4Completely Acceptable

Completely Unacceptable

Neither Acceptable nor Unacceptable

Scheduled Times

Sequences

Delays atSector

Delay Countdown

Single OuterArc Meter

List OverallAcceptability

Figure 8. Center Sector Controller Mean Acceptability Ratings (standard deviations represented by vertical bars).

20

Sector Controller Workload Ratings

0

1

2

3

4Better with CTAS

Better with ASP

ScheduledTimes

SequencesDelays at

Sector

DelayCountdown

Figure 9. Center Sector Controller Mean Workload Ratings (standard deviations represented by vertical bars).

All of the CTAS features were rated toward the Betterwith CTAS end of the scale. Some controllers commentedthat the delay countdown did not seem to take intoaccount the effect of wind variations between the sectors.It is possible that the CTAS wind input was sometimesold; at the time of the Field Survey, CTAS wind informa-tion was updated from National Oceanic and AtmosphericAdministration (NOAA) every hour, with forecasts validfor up to three hours. There may have been situations inwhich the wind information was not current. However, itis more likely that the delay countdown problem was dueto the freeze occurring too early at some sectors undercertain weather conditions; this problem will be investi-gated with the new Spatial Freeze Horizon (SFH)feature.8 Overall, the controller questionnaire data show

8 The Spatial Freeze Horizon (SFH) will be used toestablish a freeze horizon that is defined by spatial loca-tion, rather than using an estimated time to the meter fix(which is how TMA currently operates). This is expectedto be useful in conditions when large headwinds or tail-winds are encountered that create uncertainty in the deter-mination of the aircraft sequence. For example, under theexisting system, aircraft negotiating a headwind may havetheir schedules frozen deep in the sector airspace, effec-tively reducing the usable airspace and creating controllerworkload, as the controller has fewer options for vector-ing or sequencing this aircraft. By using a SFH in thissituation, the controller can establish a SFH at a spatiallocation relative to the meter fix, and can more easilypredict aircraft freeze order and reduce workload.

that CTAS had less of an impact on controller workloadthan ASP.


Controller opinions of CTAS were generally favorablecompared to their opinions of ASP, but some of thecontrollers’ opinions were influenced by their negativeopinion of metering in general. As was found in the 1996Operational Evaluation results, the metering system usedaffects how the controllers run their traffic. Some con-trollers observed that they would have liked to knowwhether to expect CTAS or ASP metering as theyperceived a difference in how reliable the metering listinformation was under the two systems. Other controllersalso mentioned wanting more advance notice whenmetering is likely to be implemented. It is possible thatthe addition of the SFH will aid with earlier trafficplanning in some sectors, including addressing theproblem of the prop aircraft that are assigned excessivedelay in the noon balloon. The SFH feature may help toaddress controller concerns that they should get informa-tion on some aircraft earlier, and SFH should also help insituations where controllers feel that they need to doseveral swaps due to winds or aircraft performance.

Metering operations have been reduced overall, which isattributed to the greater airport capacity at DFW. Whilesome controllers seemed to feel that the reduced metering

21

frequency was a positive trend, they also noted that theywere more proficient at metering operations when theymetered all the time.

The controllers recognized that their opinions of CTASwere influenced by their level of CTAS training. Onecontroller felt that CTAS didn’t work as well as adver-tised, but was unsure if this was due to unrealistic expec-tations, or if the CTAS features were not being usedcorrectly. Another example was the responses regardingthe usefulness of the delay countdown feature; somecontrollers reported it was very helpful, and others did notunderstand how it worked.

There was at least one enthusiastic controller who thoughtthe manual swap feature was easy to use. Many of thecontrollers did not use the swap features regularly or hadnot been aware of the features.

4.4.3 TRACON TMC DATA


As discussed in Section 4.1, the DFW TRACON has3 sets of CTAS displays: a Center TMA Repeater TGUIand PGUI, their own TMA PGUI and TGUI, and apFAST PGUI and TGUI. The Center Repeater systemprovides added situational awareness for the TRACON,so that they can view how the Center is managing thearrival flow. The TRACON’s own TMA system is usedfor creating what-if scenarios for traffic management. TheTMU keeps the TRACON and Center systems synchro-nized; the TMCs look for changes on the Center repeaterand enter these changes on their own system. TheTRACON TMCs reported that they always compare theCenter Repeater and TRACON systems.


Questionnaire responses were obtained from 4 out of the7 TMCs at DFW TRACON. These TMCs reported anaverage of 6.25 years of TMC experience, with a range of2 to 9 years.

The TMCs that were surveyed reported an average of30 hours of CTAS training, with a range of 4 to 80 hours.The comfort level they reported with TMA was 3.4, on ascale of 0 to 4, where “0” represented very uncomfortableand difficult to use, and “4” represented very comfortableand easy to use.

4.4.3.3 Metering Operations

When the TRACON TMU feels that the traffic levelsexceed capacity, they may elect to “shut off” the Center,and not allow any more traffic into the TRACON. The

TRACON reported that they rarely shut off the Centernow, and do so only for weather reasons. The TMCsattribute the reduced metering frequency to their “goodairport.” One TMC said that the equipment that they have(including CTAS) helps them to achieve smootheroperations than under ASP metering. Because of CTAS,the traffic information that they have is more accurate,and allows them to plan more effectively. The TMCsindicated that there was more uncertainty with ASP, butthat when the delays did not appear correct in CTAS, theCenter would revert to ASP.

4.4.3.4 Dual Route Operations

The TRACON reported allowing about 3-4 aircraft overone dual fix and then opening up a dual route overanother fix. They only run one dual at a time but recog-nized that the Center may wish to increase the number ofsimultaneous dual routes. The TMCs noted that if oneside of the airport was shut off due to weather that itwould be helpful to run dual routes over the other fixes.The decision whether to use the dual route proceduredepends on several considerations, including whether theaircraft have a place to go, and if there are departureconflicts (including Dallas-Love Field departure traffic).

4.4.3.5 Use of CTAS

The TMCs use the timelines a great deal. One TMC usesthe F4 panel to verify flight proposals to Dallas-LoveField. They also display a load graph to show traffic onthe dual route.

The TMCs were asked to rate how frequently they use theCTAS/TMA features. Overall, since TMA is not beingused to the same extent (nor for the same purposes) asTMA is used in the Center, many of the schedulingfeatures are not utilized. The TMCs reported that theymost often reference the timelines and delay information(delay numbers appended to the aircraft tags). As in theCenter, the TMCs reported using the PGUI timelines themost frequently of the PGUI features. The features theyreported using least frequently were the TGUI trafficcount, the TGUI F11 flight plan information and theTGUI F8 pop-up auxiliary information.


The TMCs reported a high level of comfort with TMA,and expressed confidence that with TMA, they can “counton what the system says it is going to deliver.” One TMCsuggested that under ASP operations, there was a “fudgefactor” that affected their planning. The TRACON spendsconsiderable time comparing the data from the Center and

22

TRACON TMA systems in order to understand the basisfor Center metering decisions.

4.4.4 TOWER TMCS

The DFW airport has three air traffic control towers, theEast Tower, the West Tower, and the center tower. TheEast and West Towers are staffed during the day and thecenter tower is staffed only at night during the 11:00 PMto 7:00 AM shift. If there is any problem with an outertower, the staff moves to the center tower.


TMA was installed in the East Tower in April 1998and the West Tower in June 1998. There are 4 trafficmanagers who work in both of the Towers. The areasupervisors use the TMA system more than the TMCs.TMA provides the first glimpse of arrival traffic thatthe Towers have ever had; their existing radar displaysystem, the BRITEs, only provide traffic information15-20 minutes into the future.

4.4.4.2 Use of CTAS

The West Tower is not as busy as the East Tower. TMAis not used very much in the West Tower, beyond helpingto determine staffing or the current location/STA ofaircraft. In general, the East Tower uses TMA to planstaffing, to determine the direction of the rushes, and todetermine when the rushes are going to start. TMA helpsthem to deal with unusual events, such as closing arunway for maintenance and delays due to departures.

During the observations conducted one afternoon, TMAaided in airport planning for the arrival of an emergencyaircraft. The Tower was able to access the flight planinformation and meter fix crossing time of the emergencyaircraft so that they knew when to alert the fire depart-ment. The Tower has found the timelines to be beneficial,but at the time of the Daily Use Field Survey, the runwayassignments were not always accurate (since the Centerradar tracks, which feed their TMA displays, are droppednear the airport). The Tower indicated that they wouldlike to get pFAST, or at least the TRACON radar, fed intotheir TMA display so that they have a better indication ofthe actual landing runway assignments.

4.5 1998 Daily Use Field Survey Summary

The 1998 Field Survey addressed many of the questionsand issues raised in the 1996 Operational Evaluation.Although more data were collected from a wider range ofTMCs and Center Sectors than in the 1996 OperationalEvaluation, it was still necessary to focus on the busiersectors in order to capture more information regarding

metering operations. For some less-busy sectors, themetering list did not really affect how the controllerwould control traffic. Data were also collected from theTRACON and the Tower, and this will serve as usefulbaseline information for future TMA assessments of thosefacilities. The results are summarized in terms of the sixobjectives outlined in Section 4.1. Where these issueshave changed since the 1998 Field Survey are alsodiscussed.

1. Investigation of Issues Raised in the 1996Operational Evaluation.

Manual Swap and Manual Sequence FeatureUsage:

The manual swap and manual sequence featureswere not widely used. This could be attributed toa number of issues: controller habits with regardto metering operations, insufficient training, andinsufficient opportunity.

Some Center Sector Controllers use the meteringlist as a list of times to hit (rather than strictlyassociating each aircraft with its respectivetime). The controllers may write down the timesand make sure one aircraft crosses the meter fixat each of the specified times. This serves thepurpose of metering the flow into the TRACON,but if the high sectors do not make entries intothe system to reflect the sequence, then the lowsector list does not correspond to the aircraftsequence, and the low sector cannot plan aseffectively. The implications of not using thesefeatures and the resulting effects on the sched-ules may not be apparent to the controllers.Some controllers who were aware of thesefeatures often said they were too busy to usethem. Some controllers reported that theypreferred to make swaps mentally. A fewcontrollers did report using the manual swapfeature all the time, and that they felt it washelpful. It is possible that with training, addeddocumentation or reminders, as well as practice,these features would be used more frequently.

The majority of the controllers we spoke to wereeither not aware of the manual swap and manualsequence features, or were not aware that theD-side as well as the R-side could make theseinputs, reflecting insufficient (or not recent-enough) training. Related to this training issue isthe opportunity to exercise the features them-selves. If metering occurs less frequently, con-trollers will generally have limited opportunities

23

to try these features and stay proficient at usingthem.

Since the 1998 Field Survey, ZFW has installedthe Display System Replacement (DSR) cons-oles in their facility. Controllers are now using anew user interface and a new radar display.Quick reference cards have been created andprovided to the controllers to remind them howto use the manual swap and manual sequencefeatures (see Section 5.4 and Appendix D). Itremains to be investigated in future surveys howthe DSR interface may or may not influence theuse of the swap features.

TRACON and Tower Data:

Data were gathered from the TRACON TMCs,who report that TMA is helpful to their opera-tions by providing reliable scheduling infor-mation. The TRACON also makes use ofcomparisons between their system and therepeater of the Center system to understand theCenter’s traffic management plans. Unlike theCenter, the TRACON has fewer TMCs and didnot experience as much turnover. There wereconsequently several TMCs who have beenworking with TMA for quite some time. As newTMCs are introduced into the TRACON, it willbe important to make sure they are quickly andthoroughly trained on TMA in order to avoidsome of the training-based problems that theCenter has experienced.

The Tower operations with TMA were not well-defined; as mentioned above, the Tower has alimited TMU staff, and at the time of the DailyUse Field Survey, only had experience withTMA information for a few months. As moreaccurate information is provided via pFAST orTRACON radar, it would be instructive toagain investigate if TMA is providing bettersituational awareness and planning for theTower operations.

2. Transition to Full-Time Usage.

The data collected from the 1998 Daily Use FieldSurvey provides a baseline for understandingMetroplex operations with TMA. TMA has beenadapted to the new airspace, and the Center hascreated the new dual route operations that they areable to plan and monitor using TMA. There areprocedures in place for addressing system problems,and the NTX engineering support team are on-site to

troubleshoot and monitor the system. TMA is theprimary metering system that is used under mostconditions.

Overall, the frequency of metering appears to havebeen reduced. This has been attributed to not only thechanges to procedures and airspace, but to changesthat the airlines have made to their schedules. Thefacilities reported that Delta Airlines has adjusted itsschedule so that its banks of aircraft do not competedirectly with American Airlines. As a result, therushes have gotten longer, and the peak levels oftraffic are not as high.

In the 1998 Daily Use Field Survey, Center TMCsreported reverting to ASP if TMA was not “workingproperly.” The definition of “working properly”varied among TMCs; some were more likely to revertto ASP than others, and this might have reflected thelevel of training. Sometimes if the delay valuesseemed unusual (such as if the TMA delay valuesseemed too high, or were higher than ASP), theywould revert to ASP. The level to which a delay wasconsidered unusual may have also reflected the levelof training. Even with the Red-X warning, the TMCsreported that they could not adequately judge theseverity of a problem with the system. Sometimes ifmetering was just about to begin, and there appearedto be problems with CTAS, the TMCs may havedecided to use ASP.

Since the 1998 Daily Use Field Survey, the TMCshave reported that the TMA system has proven to bemore reliable and experiences fewer failures that inthe past would have necessitated reverting to ASP.TMCs are now more likely to call the NTXengineering support team with questions regardingthe TMA schedules, than to report a system failure.In general, the ZFW TMCs prefer not to revert toASP if it can be avoided; they have noted that if ASPis utilized, the DFW TRACON will reduce its AARbecause of reduced confidence in the ASP system,and may not allow the use of the dual routeprocedure.

3. Anticipation of New Traffic Flow ManagementStrategies.

New strategies in traffic flow management haveevolved at ZFW since the 1996 Operational Evalua-tion. While we cannot speculate whether thesestrategies would have evolved even without TMA tohelp visualize the effect of such changes, it is clearthat TMA is able to support these new strategies.TMA provides the means to integrate new procedures

24

(such as the dual route procedure) with the existingsystem. Using a dual route procedure is seen toreduce workload. As dual route operations haveincreased, both TMCs and controllers recognize thatcontrollers may be showing some degradation oftheir sequencing skills under metering operations.This trend could continue and further reduce oppor-tunities to practice sequencing techniques. If con-trollers are perceived as becoming less proficient atsequencing, TMCs may opt to use the dual routeprocedure more frequently. However, as traffic levelsare likely to increase, the need for meteringoperations will increase again as well.

In response to the concerns (described in Section4.4.2.3) that aircraft schedules were frozen incon-sistently in some sectors due to weather, TMA wasbe modified to incorporate the Spatial FreezeHorizon. The ability to set a new freeze horizon wassomething that controllers requested; they were verypleased to know that this change was on the way.

Single Gate Free-Flow (SGFF) is a new feature thatis scheduled to begin operational testing in the springof 2000. SGFF will be used to remove meteringrestrictions from a busy corner during a metered rush.The busy corner will not receive meter times, andwill only delay aircraft as needed for in-trail spacing(sequencing) purposes. Delays that would have beenassigned to the aircraft arriving over the free-flowingcorner post will have to be redistributed equally toaircraft at the remaining gates. SGFF will allow forall the aircraft in the free-flowing corner to becounted in the AAR (unlike the dual route procedure,which meters the aircraft on the primary route anddoes not count the aircraft over the dual route in theAAR). TMA will be modified to incorporate thesechanges to the scheduling process, and the TMCswill be able to directly view the impact of suchchanges on the overall schedule.

The use of the dual route procedure may continue toevolve with the use of SGFF; there are some sugges-tions that a dual route will still be required with thefree-flowing corner, and how this will be accom-plished procedurally will need to be investigated. Itwill also be important to examine the coordinationimplications of SGFF. Both the Center and theTRACON will need to evaluate the possible effectsof using SGFF, and how TMA will support thoseoperations. SGFF operations may prove to be thetype of procedure that can be introduced at MiTCenters who will be using TMA as they transition tometering operations.

4. Anticipation of New TMA Use Strategies.

The Denver Center Assessment of TMA in 1993identified two distinct strategies in the use of TMA,focusing on either load graphs or timelines to deter-mine when to meter (ref. 5). These approachesreflected the TMCs using TMA without having hadthe benefit of extensive training. The TMCs whoparticipated in the 1996 Operational Evaluation,however, had had a great deal more exposure toTMA, and in fact had worked closely with thedevelopers to define the use strategies. They alsobenefited from the lessons learned in Denver. As aresult, the ZFW TMCs’ approach for using TMA wasto consult the traffic load graphs to understand theoverall load, and then refer to the timelines to decideon the metering time. This approach is consistentwith the “load graph” strategy defined in the 1993Assessment. Because it was not clear prior to the1998 Field Survey that there would be a variety oftraining and experience levels in the TMU, this usestrategy issue was not investigated. But during the1998 Field Survey, TMCs indicated using both loadgraphs and timelines in this manner. It is likely thatthe training the TMCs received reflected thisapproach as well.

The TRACON also relied on the load graphs andtimelines. PGUI timelines continue to be used,despite the fact that TGUI timelines are available forthe same purpose. It may be that since PGUI time-lines are available on a display separate from theTMA/TGUI, they are viewed differently at a con-ceptual level. The addition of pFAST to dailyTRACON operations further underscores theimportance of PGUI timelines used for comparisonwith TMA information.

It can be assumed that new strategies for TMA usewill emerge as the SGFF operations are tested andimplemented. Thus far, TMA has demonstrated greatflexibility in adapting to new procedures and opera-tions and continues to assist the TMCs to visualizethe traffic conditions and make effective operationaldecisions based on this information.

5. New Status and Scheduling Panel Information inthe TGUI.

The Status and Scheduling Panel (F1) in the TGUIwas designed with significant input from the ZFWTMCs. Combining the scheduling and configurationinformation into one place in the TGUI makes it easyto view current and future scheduling settings andallows the user to make changes and see the effects

25

of these changes. The new panels were generallyviewed as helpful and there were no negativecomments. The structure of the panels also includesinstructions on the proper sequence of steps to enacta particular configuration or scheduling change. Thisshould help with making the TGUI more under-standable to new users. As this was a new featureintroduced in 1998, it will be instructive to determinehow the F1 panel has helped in training since then.

6. Continued Follow-Up as Part of the CTASDevelopment Process.

TMA development has continued even as TMA isimplemented for daily use and a version of TMA ispart of the Free Flight Phase I national deploymentplan. While much of the basic functionality was well-defined at the end of the 1996 Operational Evalua-tion, there were significant user-interface changesmade to accommodate the daily use of the system.There were also many other human factors chal-lenges and many engineering challenges that arosefrom the daily use of TMA that were not anticipatedthroughout the development process.

How well a tool and its features are utilized must beinvestigated beyond the final proof-of-conceptevaluation that is conducted in the field. The long-term effects of staff turnover, lapses in training, andchanges to airspace and procedures are all variablesthat cannot be properly examined in the context of asingle operational evaluation. They are all variablesthat influence the utility of a tool, and the benefitsthat can be realized from the tool. From the experi-ence with TMA, there were clearly new researchareas to be explored that affect the development ofother tools for the Center and TRACON TMCs andcontrollers.

It is also critical to investigate such long-term issuesbecause they raise pertinent questions about thedevelopment of follow-on decision-support tools andenhancements. Continued investigation providesinsight into the evolving air traffic control environ-ment, and can highlight problems and new areas forresearch. For example, due to staff turnover, thetraining of the TMCs at the Center was uneven. As aresult, it was not a simple matter to draw conclusionson all of the potential benefits of TMA for trafficmanagement decision-making. While insufficienttraining makes assessment more difficult, it also canpoint to the need for improvements in the userinterface and the documentation so that less-experienced users are better able to understand thesystem. As traffic management is a complex process,

the TMA training may need to include discussion ofthe techniques and approaches used in traffic man-agement. Specific training recommendations aredescribed in Section 5.

5. RECOMMENDATIONS

Based on the 1998 Daily Use Field Survey, a number ofrecommendations are made to improve CTAS function-ality and enhance ATC operations. These recommenda-tions fall under the following categories: user interface,added functionality, training, and documentation.

5.1 User Interface

There were no significant interface changes requested forthe TGUI as an outcome of the Daily Use Field Survey.Some of the new functionality that is being defined (suchas SFH and SGFF) will require associated interfacechanges. The Center TMCs did make a recommendationto simplify the entry of the metering message to the Host,which is accomplished through the CM (or Communica-tions Manager) process of CTAS. This change is beingevaluated.

With the recent implementation of the DSR consoles,there may be changes in the usability of the manual swapor manual sequence features given the new user interface;this will be an issue to investigate in a future survey ofTMA daily use.

5.2 Functionality

Changes to the proposed SGFF functionality, provided bythe ZFW TMCs, are being researched. In addition tochanges to the scheduler, these proposed changes includeadditions to the TGUI F1 Status and Scheduling panel,and the representation of SGFF information on thetimelines and in the overlays.

5.3 Training

Training is probably the most significant issue thataffected the interpretation of TMA at ZFW. Whileoverall, CTAS/TMA has provided many benefits forZFW operations, not all the TMCs understood what TMAdoes, and how it does it. Some TMCs didn’t know how toset up TMA, and what options are available. There wasno regular mechanism, either through memos, briefings,or documentation, to alert the users to updates or theavailability of new features. While training sessions wereconducted following the 1996 Operational Evaluation,since then, the turnover rate in the TMU has become moreregular. At the time of the Daily Use Field Survey, therewere about 10 senior TMCs who knew ASP well, and theremainder of the TMC staff “grew up with” CTAS.

26

Most of the newer Center TMCs learned aboutCTAS/TMA through word-of-mouth, trial and error, andtalking to the NTX engineers and researchers. Some ofthe TMCs learned a great deal about CTAS because theywere willing to sit down and work through the manuals,but in general, it appeared that the manuals were notaccessed very frequently. One of the TMCs said that ittakes 6-12 months to get familiar with everything onCTAS. Another TMC estimated that he uses probably afourth of TMA’s total capabilities because he doesn’tcompletely understand the system.

Because the new TMCs rely on other TMCs for training,they are subject to the biases of whomever is conductingthe training. A TMC who doesn’t like CTAS, prefersASP, or prefers not to meter, will influence how a newTMC will learn about CTAS. The TMCs all agreed thatany new training effort should be hands-on/on-the-job(OJT). Shadowing an operational system where users canobserve the effects of traffic flow management decisionswould also provide training benefits. The trainingrecommendations based on the 1998 Daily Use FieldSurvey are as follows:

• Additional training for the Center TMCs is needed, inthe form of OJT, or other exercises with an opera-tional system. This training should be offered peri-odically to coincide with significant staff turnover inthe TMU. Additionally, periodic information onupdates to the software should be provided so thatTMCs are aware of new features. This periodicinformation could be incorporated into regular TMCbriefings. The NTX engineering support team couldprovide updates on the status of the CTAS system tothe TMCs on a regular basis so that upcomingchanges are anticipated.

• The TRACON TMU turnover rate was much lowerthan that observed at the Center TMU. Therefore,additional training for the TRACON is onlyrecommended for new personnel. New trainingshould also take the form of OJT or operationalexercises. Regular briefings on system updatesshould be provided.

• Refresher training is recommended for the Towerusers to provide an overview of the CTAS system.Regular briefings on system updates should beprovided.

• While the Center Sector controllers could also benefitfrom refresher training on the CTAS features, asnoted in Section 4.5, it could be a complicated issueto address as their opportunities to try out the CTASfeatures are available only when metering. Whether

the sector controllers view any benefit to updating theschedules according to manual swaps or sequencesremains to be investigated.

5.4 Documentation

The CTAS software is extensively documented in userreference manuals describing how features can beutilized. The documentation can be used as a tutorial forunderstanding all of the different functionality withinCTAS. However, to support daily use operations, addi-tional documentation, in the way of quick referenceguides for the TMCs and controllers, are recommended toprovide a brief synopsis of the information that TMCs andcontrollers use most frequently.

For the Center TMCs, a quick reference card has beendeveloped to provide guidance on interpreting data tags,delay information, and for quickly determining whatinformation is contained in the different set-up panels inthe TGUI and PGUI. It is helpful that, in general, theTMA displays are set up in a standard configuration thatis rarely changed. The quick reference card may beadditionally helpful for less-experienced users to under-stand the more subtle changes that are made to a TMAdisplay. For the sector controllers, a quick reference cardhas also been developed summarizing the keyboard inputsto use the manual swap and manual sequence features.Drafts of these cards are included in Appendices C and D.

6. CONCLUDING REMARKS

Overall, TMA is working well at all the Dallas/Ft. WorthATC facilities. The human factors results of the 1996Operational Evaluation showed that there were significantbenefits to using TMA, and that there were some antici-pated increases in workload with TMA usage, but thatoverall, traffic management operations were enhanced,making such workload increases acceptable. Since the1996 Operational Evaluation, many changes have takenplace in the form of new ATC procedures, airlinepractices, and airspace configuration. TMA has been indaily use throughout those changes, and has been shownto be adaptable to such changes. Even during periods ofclear weather, in which metering is not occurring asfrequently as expected, TMA still provides a useful meansof visualizing the traffic, determining delays, and aidingin decisions regarding dual route operations and staffing.As staff are introduced into the TMUs, TMA is becomingthe new standard for visualizing the traffic flow. It isanticipated that with more increases in traffic expectedover the next several years, TMA will still be able toprovide significant benefits to traffic flow management,and certainly if metering operations are increased, more

27

of TMA’s functionality will be exercised. Additionaltraining and documentation is needed to better familiarizenew TMCs and provide reminders to the TMU as a wholeabout the capabilities that can be exercised with TMA.

The single outer arc meter list continues to be a usefulCTAS feature. The manual swap feature appears to besomething that is useful but which requires controllers’familiarity with the keyboard inputs and with the purposeof this feature. Traffic levels and staffing (the availabilityof a D-side, and the coordination between the D-side andR-side controllers) influence the use of the manual swapfeature. Additional training is needed to familiarizecontrollers with the benefits that could be provided byusing the manual swap feature.

TMA has proven to be a robust system which has evolvedin its usage even as the facilities and the operationsthemselves have changed. As a result, the daily supportactivities provided by the (FAA-contracted) NTXengineering support team become more important inkeeping the users apprised of new system functionalityand identifying when training and additional documenta-tion are needed. The NTX engineering team has beenvery responsive to all the users in addressing problemswhen they occur.

The 1998 Daily Use Field Survey human factors resultsare instructive as baseline data for future CTAS/TMAenhancements (such as SFH and SGFF). The humanfactors results and recommendations in this report shouldalso provide guidance in the more wide-scale deploymentof CTAS and development of follow-on DSTs especiallywith regard to training, documentation, and long-termsystem use issues that may be encountered at otherfacilities.

7. REFERENCES

1. Erzberger, H.; Davis, T. J.; and Green, S. M.: Designof Center-TRACON Automation System. In:Proceedings of the 56th Symposium on MachineIntelligence in Air Traffic Management, Berlin,Germany, 1993, pp. 52-1 to 52-14.

2. Swenson, H. N.; Hoang, T.; Engelland, S.; Vincent,D.; Sanders, T.; Sanford, B.; and Heere, K.:Design and Operational Evaluation of the TrafficManagement Advisor at the Fort Worth Air RouteTraffic Control Center. Presented at the 1st

USA/Europe Air Traffic Management Researchand Development Seminar, Saclay, France,June 17-19, 1997.

3. Aviation Operation Systems Development Branch,NASA Ames Research Center. TrafficManagement Advisor: TMA Reference Manual,Release 5.4.3 ceft. Moffett Field, CA.

4. Sokkappa, B. G.: The Impact of Metering Methods onAirport Throughput. MITRE MP-89W00022.1989.

5. Harwood, K.; and Sanford, B. D.: Denver TMAAssessment. NASA CR 4554. 1993.

6. Harwood, K.: Defining Human-Centered SystemIssues for Verifying and Validating Air TrafficControl Systems. In: J. Wise, V. D. Hopkin, andP. Stager (eds.), Verification and Validation ofComplex and Integrated Human Machine Systems.Berlin: Springer-Verlag, 1993.

7. Harwood, K.; and Sanford, B. D.: Evaluation inContext: ATC Automation in the Field. In: J. Wise,V. D. Hopkin, and P. Stager (eds.), Human FactorsCertification of Advanced Aviation Technologies.Daytona Beach, FL: Embry-Riddle AeronauticalUniversity Press, 1994.

8. Hoang, T.; and Swenson, H. N.: The Challenges ofField Testing the Traffic Management Advisor inan Operational Air Traffic Control Facility.AIAA-97-3734. 1997.

9. Wong, G. L.: The Dynamic Planner: The Sequencer,Scheduler, and Runway Allocator for Air TrafficControl Automation. NASA TM-209586,Feb. 2000.

10. Federal Aviation Administration. FAA Order7110.65J. Air Traffic Control. July, 20, 1995.

11. Harwood, K.; Sanford, B. D.; and Lee, K. K.:Developing Automation in the Field: It Pays toGet Your Hands Dirty. Air Traffic ControlQuarterly. 6 (1). 1998.

28

APPENDIX A. EXAMPLES OF THE TGUI AND THE PGUI

The Timeline Graphical User Interface (TGUI) consists of a basic timeline display, a load graph display, variousoverlays of information, and a number of function panels from which the user may access the TMA features and makechanges to the scheduling constraints that govern the CTAS schedules.

Figure A.1. TGUI timelines. This figure shows the basic TMA display of timeline information. Each timeline isreferenced to meter fixes or runway threshold.

29

Figure A.2. TGUI traffic load graph. This is an example of the traffic load graphs available in TMA.Up to nine graphs can be displayed at a time.

30

The Planview Graphical User Interface (PGUI) consists of a basic map display, upon which other information such assequence list, timelines and other traffic information can be displayed. The PGUI also has various function panels uponwhich the user may specify the display parameters.

Figure A.3. PGUI Display. This PGUI displays a Center map of the Ft. Worth Center airspace, with a sequencelist displayed in the upper left corner.

31

APPENDIX B. QUESTIONNAIRES USED IN THE 1998 DAILY USE FIELD SURVEY

The following questionnaire will be used to gather information on how Ft. Worth Center TMCs are impactedby the use of the Center/TRACON Automation System (CTAS) for metering purposes. Your responses willhelp us to make improvements to CTAS or provide input into documentation or training. We will also use yourresponses to determine how to assess CTAS during a field study this summer. We appreciate your taking thetime to fill out this questionnaire, and welcome any comments you might have. If you would like to talk tosomeone about these concerns, please feel free to contact the following CTAS personnel:

Kathy Lee phone: 650-604-5051 e-mail: [email protected]

Cheryl Quinn phone: 650-604-5793 e-mail: [email protected]

Thanks for your participation! Please return this form to the marked box.

Center TMC Questionnaire

Demographics

1. How many years of ATC experience do you have?

2. How many years have you worked in a level 3 facility?

3. How many years have you worked at ZFW?

4. How many years of TMC experience do you have?

5. Approximately how much training have you had using CTAS? hours

6. Please rate (by marking on the scale below) how comfortable you feel working with the CTAS TMA system, in general:

0 1 2 3 4Very Uncomfortable; Very Comfortable;

System seems difficult System seems easyto use to use

General Traffic Management

7. Please rate (by marking on the scale below) the general traffic complexity of the facility during the past month:

0 1 2 3 4Very Simple Neither Simple nor Very Complex

Complex

8. Approximately how often does the facility meter? ______ times/day

9. Approximately how often do you use dual routes in addition to metering?______ times/day

10. How many times do you need to revert to ASP? ______ times/month

32

11. Please describe the conditions that cause you to revert to ASP for metering instead of CTAS:

12. What features do you like/dislike about using the ASP system for metering?

13. What features do you like/dislike about using the CTAS system for metering?

14. Under ASP metering, on average, how many times per rush do youneed to ripple the list? times/rush

15. Under CTAS metering, on average, how many times per rush do youneed to ripple the list? times/rush

Center CTAS Features

Please rate how often you use the following CTAS features, over the course of a typical month, using thescale below:

Frequency of Use of CTAS/TMA Features Scale

0 1 2 3 4never sometimes often

16. Timelines

17. Traffic Load Graphs

18. Traffic Count

19. Status and Scheduling Information Panel (F1)

20. Delay Numbers (attached to aircraft tags on the timelines)

21. Configuration Summary (F7)

33



22. Pop-up Aux Display Information (F8)

23. Flight Plan Information from TGUI (F11)

24. Blocked Intervals

25. Blocked Slots

26. Proposed Flights (F4)

27. Not on Timeline Flights (F4)

28. Departure Time Panel (for Proposed and Not on TimelineFlights) (F4)

29. Delay Reporting Status (F6)

30. Previously Recorded Delay Reports

31. Scheduling Options: STA Freeze Horizon Settings (shift-F10)

32. Scheduling Options: Scheduler Settings (shift-F10)

33. Scheduling Options: Data Sources (shift-F10)

34. Manual Rescheduling

35. Change Runway Assignment

36. Allocate Runway

37. Changing Meter Fix Assignment

38. Proposed Meter Fix Change

39. Suspending Aircraft

40. Reset Aircraft

41. Priority Aircraft

42. Find Slot for Aircraft

43. PGUI (Planview) Traffic Display

44. PGUI Timelines

34



45. PGUI Sequence List

46. PGUI Flight Plan Information

47. How frequently do you monitor the sectors by using theinformation in the single outer arc meter list?

Please provide any comments or suggestions in the space below.

35

The following questionnaire will be used to gather information on how Ft. Worth Center controllers areimpacted by the use of the Center/TRACON Automation System (CTAS) for metering purposes. Yourresponses will help us to make improvements to CTAS or provide input into documentation or training. Wewill also use your responses to determine how to assess CTAS during a field study this summer. Weappreciate your taking the time to fill out this questionnaire, and welcome any comments you might have. Ifyou would like to talk to someone about these concerns, please feel free to contact any of the following CTASpersonnel:




Center Controller Questionnaire

Demographics

Specialty: _______________________

1. Approximately how many CTAS metering periods have you worked, in thearrival sectors, in the past month?

2. How many total years of ATC experience do you have?

3. How many years have you worked in a level 3 facility?

4. How many years have you worked at ZFW?

For the following questions (#5-#24), please keep in mind the traffic operations you have worked/experiencedin the past month.

CTAS Effects

Please use the following scale in answering questions 5-8:

Frequency of List Effects Scale


5. How often do you notice ripples in the metering list?

6. How often do you make manual swap (X B) entries at your position?

7. How often do you make sequence swap (X C) entries at your position?

8. How often do list ripples increase your workload during a metering period?

9. How often do you notice sequence swaps generated at another sector thataffect your list?

36

CTAS Effects, continued:

Please use the following scale in answering questions 10-13:

Difficulty of Feature Use Scale

0 1 2 3 4

Very Difficult Neither Easy nor Very EasyDifficult

10. How difficult is it to work metered traffic in your area?

11. If you have made manual swap (X B) entries, how easy is it to usethis feature? (Leave blank if you have never used this.)

12. If you have made sequence swap (X C) entries, how easy is it touse this feature? (Leave blank if you have never used this.)

13. If you have used the delay countdown feature, how easy is it to usethis feature? (Leave blank if you have never used this.)

14. If you do not make any manual swap entries, why? (Please check all that apply)

not enough metered traffic to make manual swaps

too busy to make swap entries

D-side controller makes the swap entries

too many keyboard entries are required

easier to make the manual swap with the strips themselves

prefer to make the manual swaps with the strips

prefer to make swaps mentally

other (please describe):

37

Acceptability

Please rate the acceptability of the following CTAS features and events, as you have experienced them,using the scale below:

Acceptability Scale

X 0 1 2 3 4Not Applicable Completely Neither Completely

Unacceptable Acceptable nor AcceptableUnacceptable

15. Scheduled times at your sector

16. Aircraft sequences

17. Delays at your sector

18. Delay countdown feature

19. Single outer arc metering list

20. Overall acceptability of CTAS for metering purposes

Workload

Please rate how the following events and CTAS features compare to ASP under similar traffic conditions, as itinfluences your workload, using the scale below:

Workload Scale

X 0 1 2 3 4Not Applicable Better Better(have never used) with ASP with CTAS

21. Scheduled times at your sector


23. Delays at your sector

24. Delay countdown feature

Comments


38

The following questionnaire will be used to gather information on how DFW TRACON TMCs are impacted bythe use of the Center/TRACON Automation System (CTAS). Your responses will help us to makeimprovements to CTAS or provide input into documentation or training. We will also use your responses todetermine how to assess CTAS during a field study this summer. We appreciate your taking the time to fill outthis questionnaire, and welcome any comments you might have. If you would like to talk to someone aboutthese concerns, please feel free to contact the following CTAS personnel:




TMC Questionnaire: TRACON

Demographics

1. How many years of ATC experience do you have? years

2. How many years have you worked in a level 5 facility? years

3. How many years have you worked at DFW? years

4. How many years of TMC experience do you have? years

5. Approximately how much training have you had using TMA? hours

6. Marking on the scale below, please rate how comfortable you feel working with the CTAS TMA system, in general:

0 1 2 3 4Very Uncomfortable; Very Comfortable;

System seems difficult System seems easyto use to use

When answering the following questions, please keep in mind the traffic operations you have experienced inthe past month.

General Traffic Management

7. Please rate (by marking on the scale below) the general traffic complexity during the past month:

0 1 2 3 4Very Simple Neither Simple nor Very Complex

Complex

39

8. How often do you refer to the Center TMA repeater system, versus the TRACON TMA system?

0 1 2 3 4Never Sometimes Always

9. How often do you compare the information on the two systems?


10. How often do differences in the information that is presented in the two systems create problems or difficulties in your interpretation of information?


TRACON TMA Features

Please rate how often you use or reference the following CTAS features, over the course of a typical month,using the scale below:

Frequency of Use of CTAS/TMA Features

0 1 2 3 4Never Sometimes Often


12. Delay Information (numbers attached to aircraft tagson the timelines)


14. Traffic Count

15. Current and Future Configuration Information (F7)


17. Pop-up Aux Display information (F8)


19. PGUI Timelines



40

Metering

How helpful (in terms of workload) were the following CTAS/TMA features in the past month during meteringoperations (or in helping to decide whether to meter)? (Please use the scale below):

Workload Scale

X 0 1 2 3 4Not Applicable Greatly No Effect Greatly

(have never Increases Decreasesused) Workload Workload

(not very helpful) (very helpful)




25. Traffic Count





30. PGUI Timelines



41

Dual Route Operations

33. Approximately how often have you used dual routes in the past month? ______ times/week

34. Approximately how often do you use dual routes in addition to metering? ______ times/week

Please rate (using the scale below) how helpful, in terms of workload, CTAS/TMA information is during dualroutes operations (or in making the decision of whether to use dual routes):

Workload Scale







38. Traffic Count





43. PGUI Timelines



42

Staffing Decisions

Please rate (using the scale below) how helpful, in terms of workload, CTAS/TMA information is in makingstaffing decisions:

Workload Scale







49. Traffic Count





54. PGUI Timelines



43

CTAS Features

Keeping in mind what you need to do to achieve desired system performance, how acceptable are thefollowing events and CTAS features? (Please use the scale below)

Acceptability of CTAS in Achieving Desired System Performance

X 0 1 2 3 4Not Applicable Completely Neither Completely

Unacceptable Acceptable nor AcceptableUnacceptable

57. Physical manipulation required (keyboard entries,mouse movements, etc.)

58. Time spent working with the system (to manipulatefeatures, waiting for updates, etc.)

59. Required inter-facility communication

60. Required intra-facility communication

61. Delay distribution

62. Delay accuracy

63. STAs/Schedules


65. Summary of traffic information


67. Time required to feel comfortable using TMA

68. Overall acceptability of using TMA/CTAS

69. The most recent reference manuals (TMA andPassive FAST)

70. Quick Reference Guide (Passive FAST)


44

APPENDIX C. DRAFT TMA QUICK REFERENCE CARD FOR CENTER TMCS

TMA Timelines

Freeze Horizons:

Blocked Interval(indicates fixesaffected, interval,and start time)

Priority Aircraft

TMA Quick Reference Card

Load LimitLinevalue dependsupon settings inshift-F2 (TrafficLoad GraphSetup) panel

Reference Point

Plot Legend

Number ofAircraft

Time

Traffic Load Graphs (F2)

Peak Value

TGUI F-Panels DescriptionF1: Status and Scheduling...............Select parameters (TRACON & Gate)

and configuration (AAR & Rwy Flow)F2: Load Graph................................View load graphsshift-F2: Load Graph Setup............. Select load graph featuresF3: Current Display Format............. Load/save user-defined display filesF4: Aircraft Information.....................View/schedule proposed, not-

on- timeline, satellite departuresF5: Traffic CountF6: Delay Reporting StatusF7: Configuration Summary............. Current time, configuration statusF8: Popup Aux Data Overlay............Toggle overlay of auxiliary dataF9: Timeline Options........................Set timeline featuresshift-F10: Scheduling Options..........Set scheduling optionsF11: Flight Plan Readout..................Toggle overlay of aircraft flight plan

items in red: change actual schedules and sequencesitems in black: change display of information

PGUI F-Panels DescriptionF1: Map View................................... Select desired mapF2: General Setup Options.............. Set display featuresF3: Map Options.............................. Set map featuresF4: Defaults File Load Options.........Load/save display filesF5: Timeline Options (if enabled)......Set PGUI timelines featuresF6: Sequence Options......................Set sequence list featuresF7: HELP: Quick Keys......................List of quick key shortcutsF12: Refresh.....................................Refresh the display

Find flight on TGUI or PGUI:1. Type ÒaÓ2. Enter aircraft ID3. Press Return

Find waypoint on PGUI:1. Type ÒwÓ2. Enter waypoint ID3. Press Return

Available beginning with 5.4.1ceft release:Turn on PGUI Sequence List display: Scroll Lock key

PGUI Functions

TMA Quick Reference Card

TGUI Functions

For Jets

For Props

Size Symbolsmalllarge >heavy

B757 +

Reference Point(MF, THD, FAF) Current Time

Meter Fixes,Runways

Delay Valuegreen = 0 - 4 min.yellow = 5 - 10 min.orange = 11 - 15 min.red ³ 15 min.

ETAsgreen tags

STAsyellow tags = unfrozen

blue tags = frozen

45

APPENDIX D. DRAFT TMA QUICK REFERENCE CARD FORCENTER SECTOR CONTROLLERS

D-sideQuick Reference Card

Manual Swap:

¥ Enter ÒNEÓ

¥ Type in 2 CIDs (separated by a space).

¥ Press Enter.

Aircraft order will swap.

Sequence Swap:

¥ Enter ÒNJÓ

¥ Type in up to 5 CIDs in the sequence you want(each CID separated by a space).

¥ Press Enter.

Aircraft will be re-ordered in the list.

Purpose of these Features:Manual Swap: to swap the order of 2 aircraft in the meterlist.Sequence Swap (Ripple List): to sequence up to 5 aircraft inthe meter list.

Using these features will help match slots to the scheduledtimes. High sectors using the feature will help the lowsectors to know what order to expect the aircraft.

R-sideQuick Reference Card

¥ Trackball over Ripple List.¥ Press Enter on trackball, orselect F9 key.¥ Type in up to 5 CIDs in thesequence you want (eachCID separated by a space).¥ Press Enter.

Aircraft will be re-orderedin the list.

CRD Function Menu¥ Trackball over ManualSwap.¥ Press Enter on trackball, orselect F5 key.¥ Type in 2 CIDS (separatedby a space).¥ Press Enter.

Aircraft order will swap.

-or-

CTAS Problems orTroubleshooting:Please contact CTAS OpsSupport at x 7635

Notes:1. You must have track control ofthe aircraft you are planning toswap.2. You can only swap orsequence the same types ofaircraft (prop for prop, jet for jet,up to 5 props, up to 5 jets).

On/Off Lists F1Arpt Config F2Arpt Rate F3Manual Assign F4Manual Swap F5Arrival Delay F6Arpt Combine F7Manual Delete F8Ripple List F9

1. Trackball over Meter list button on ÒRÓ CRD.2. Press Enter on trackball.3. Use Manual Swap or Sequence Swap:

Purpose of these Features:Manual Swap: to swap the order of 2aircraft in the meter list.Sequence Swap (Ripple List): tosequence up to 5 aircraft in the meter list.

Using these features will help match slotsto the scheduled times. High sectorsusing the feature will help the low sectorsto know what order to expect the aircraft.

REPORT DOCUMENTATION PAGE

8. PERFORMING ORGANIZATION REPORT NUMBER

10. SPONSORING/MONITORING AGENCY REPORT NUMBER

Form Approved

OMB No. 0704-0188

12b. DISTRIBUTION CODE12a. DISTRIBUTION/AVAILABILITY STATEMENT

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES)

4. TITLE AND SUBTITLE 5. FUNDING NUMBERS

6. AUTHOR(S)

1. AGENCY USE ONLY (Leave blank)

9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)

2. REPORT DATE 3. REPORT TYPE AND DATES COVERED

15. NUMBER OF PAGES

16. PRICE CODE

20. LIMITATION OF ABSTRACT19. SECURITY CLASSIFICATION OF ABSTRACT

18. SECURITY CLASSIFICATION OF THIS PAGE

17. SECURITY CLASSIFICATION OF REPORT

14. SUBJECT TERMS

13. ABSTRACT (Maximum 200 words)

Standard Form 298 (Rev. 2-89)Prescribed by ANSI Std. Z39-18298-102

NSN 7540-01-280-5500

Public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources,gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of thiscollection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for information Operations and Reports, 1215 JeffersonDavis Highway, Suite 1204, Arlington, VA 22202-4302, and to the Office of Management and Budget, Paperwork Reduction Project (0704-0188), Washington, DC 20503.

11. SUPPLEMENTARY NOTES

Unclassified Unclassified

Unclassified — UnlimitedSubject Category 03 Distribution: StandardAvailability: NASA CASI (301) 621-0390

A-00V0008

NASA/TM–2000-209587

February 2000

Ames Research CenterMoffett Field, CA 94035-1000

National Aeronautics and Space AdministrationWashington, DC 20546-0001

576-01-24

53

A04


Katharine K. Lee, Cheryl M. Quinn, Ty Hoang, and Beverly D. Sanford*

The Traffic Management Advisor (TMA) is a component of the Center-TRACON Automation System(CTAS), a suite of decision-support tools for the air traffic control (ATC) environment which is being devel-oped at NASA Ames Research Center. TMA has been operational at the ATC facilities in Dallas/Ft. Worth,Texas, since an operational field evaluation in 1996. The Operational Evaluation demonstrated significantbenefits, including an approximately 5% increase in airport capacity. This report describes the human factorsresults from the 1996 Operational Evaluation and an investigation of TMA usage performed two years later,during the 1998 TMA Daily Use Field Survey. The results described are instructive for CTAS-focuseddevelopment, and provide valuable lessons for future research in ATC decision-support tools where it iscritical to merge a well-defined, complex work environment with advanced automation.

Traffic Management Advisor, TMA, Air traffic control, Traffic flow management,Center/TRACON Automation System, CTAS, Human factors, Decision-support tools

Technical Memorandum

Point of Contact: Katharine K. Lee, Ames Research Center, MS 210-6, Moffett Field, CA 94035-1000 (650) 604-5051

*Sterling Software, Mountain View, California

Documents

Human Factors Report: TMA Operational Evaluations 1996 & 1998 · 2016. 10. 13. · 2. INTRODUCTION The Traffic Management Advisor (TMA) is a component of the Center-TRACON Automation