ote ote technical note tech

Air Traffic Control Specialist Decision Making and Strategic Planning – A Field Survey
Jean-François D’Arcy, Ph.D., Titan SRC Pamela S. Della Rocco, Ph.D., ACT-530

March 2001 DOT/FAA/CT-TN01/05

Document is available to the public through the National Technical Information Service, Springfield, Virginia 22161

U.S. Department of Transportation Federal Aviation Administration William J. Hughes Technical Center Atlantic City International Airport, NJ 08405

NOTICE This document is disseminated under the sponsorship of the U.S. Department of Transportation in the interest of information exchange. The United States Government assumes no liability for the contents or use thereof. The United States Government does not endorse products or manufacturers. Trade or manufacturer's names appear herein solely because they are considered essential to the objective of this report. This document does not constitute FAA certification policy.

Technical Report Documentation Page
1. Report No. 2. Government Accession No. 3. Recipient’s Catalog No.

DOT/FAA/CT-TN01/05
4. Title and Subtitle 5. Report Date

March 2001 Air Traffic Control Specialist Decision Making and Strategic Planning – A Field Survey
7. Author(s) 6. Performing Organization Code

ACT-530
8. Performing Organization Report No.

Jean-François D’Arcy, Ph.D., Titan SRC and Pamela S. Della Rocco, Ph.D., ACT-530
9. Performing Organization Name and Address

DOT/FAA/CT-TN01/05
10. Work Unit No. (TRAIS)

Federal Aviation Administration William J. Hughes Technical Center, ACT-530 Atlantic City International Airport, NJ 08405
12. Sponsoring Agency Name and Address

Titan SRC 5218 Atlantic Ave., 3rd Floor Mays Landing, NJ 08330

11. Contract or Grant No. 13. Type of Report and Period Covered

Federal Aviation Administration Human Factors Division 800 Independence Ave., S.W. Washington, DC 20591
15. Supplementary Notes

Technical Note
14. Sponsoring Agency Code

AAR-100

16. Abstract

This study investigated Air Traffic Control Specialists' perspective regarding decision making and planning and related cognitive processes such as learning, memory, and situation awareness. The results of 100 semi-structured interviews indicated that controllers emphasize the importance of safety, situation awareness, planning skills, backup strategies, and the collective nature of their task. Participants reported that they plan their first actions and start building their mental picture prior to assuming control of their position. They indicated using flight progress strips to support their memory. Controllers described that they become more conservative when facing difficulties like high workload, fatigue, aging, and bad weather. Concerning the respective effects of experience and facility type, the more experienced participants were, the more likely they reported formulating backup plans. Terminal controllers were more likely than en route controllers to report using the first strategy that they develop instead of considering alternatives when a potential conflict is detected or when workload is high. Terminal controllers also indicated that they were less likely to wait and see when they are not sure if there is a conflict. Finally, respondents expressed a need for conflict probes, better weather information, data link communication, and better radars.

17. Key Words

18. Distribution Statement

Air Traffic Control Strategies Decision Making Planning

This report is approved for public release and is on file at the William J. Hughes Technical Center, Aviation Security Research and Development Library, Atlantic City International Airport, New Jersey 08405. This document is available to the public through the National Technical Information Service, Springfield, Virginia, 22161.

19. Security Classif. (of this report)

20. Security Classif. (of this page)

21. No. of Pages

22. Price

Unclassified Form DOT F 1700.7 (8-72)

Unclassified Reproduction of completed page authorized

109

ACKNOWLEDGMENT The authors wish to acknowledge the contribution of many individuals to this study. First and foremost, many thanks to Anthony Buie, Operational Supervisor (OS), Jacksonville Air Route Traffic Control Center (ARTCC), who served as subject matter expert to this project, coordinated the data collection visits to 20 facilities, and interviewed half of the participants. Special thanks are also due to Tressa Woodmancy, Titan SRC, for her relentless help with the data entry, data reduction, and statistical analyses. Jean Dunn, Federal Data Corporation, revised this document and improved its presentation in a timely and meticulous manner. Thanks are due to Leonard Williams, ATCS, Jacksonville ARTCC, John Goldman, Federal Data Corporation, Philip Bassett, OS, Jacksonville ARTCC, and Alice Hardison, ACT-510, for providing us with their expert advice during the elaboration of the questionnaire. We also gratefully acknowledge our multiple hosts from the visited facilities and the participants for their enthusiastic response to our survey. Thanks to Dr. Parimal Kopardekar, Titan SRC, for his availability, advice, and trusting supervision. The authors express their appreciation to Dr. Earl S. Stein, ACT-530, who patiently served as the technical monitor for this study. Dr. Stein offered the author the opportunity to conduct this study autonomously and to learn greatly.

iii

iv

TABLE OF CONTENTS Page Acknowledgement iii Executive Summary ix 1. Introduction ................................................................................................................................. 1 1.1 Background ........................................................................................................................ 1 1.2 Literature Review............................................................................................................... 1 1.2.1 Decision Making and Planning in Air Traffic Control .............................................. 1 1.2.2 Cognitive Model of the Controller’s Task ................................................................. 3 1.2.3 Factors Influencing Decision Making ........................................................................ 6 1.2.4 Theories, Models, and Approaches of Dynamic Decision Making ........................... 9 1.3 Purpose and Rationale...................................................................................................... 12 2. Method ...................................................................................................................................... 12 2.1 Participants....................................................................................................................... 12 2.2 Apparatus and Interview Protocol.................................................................................... 13 2.2.1 Audio Tape Recorders.............................................................................................. 13 2.2.2 Interview Questions Development ........................................................................... 14 2.3 Procedure ......................................................................................................................... 16 2.3.1 Interviewers.............................................................................................................. 16 2.3.2 Interviews ................................................................................................................. 16 2.4 Data Analysis ................................................................................................................... 18 2.4.1 Data Entry and Coding............................................................................................. 18 2.4.2 Content Analysis ...................................................................................................... 18 2.4.3 Statistical Analyses .................................................................................................. 18 3. Results ....................................................................................................................................... 18 3.1 Situation Awareness......................................................................................................... 19 3.2 Memory and Flight Progress Strips ................................................................................. 22 3.3 Expertise........................................................................................................................... 23 3.4 Decision Making and Planning ........................................................................................ 27 3.5 Pilots’ and Controllers’ Requests..................................................................................... 33 3.6 Decision-Making and Planning Difficulties .................................................................... 35 3.7 Aids to Decision Making and Planning ........................................................................... 42 4. Discussion ................................................................................................................................. 47 4.1 Study Sample ................................................................................................................... 47 4.2 Situation Awareness and Memory ................................................................................... 48 4.3 Controller Skills and Experience ..................................................................................... 49 4.4 Decision Making and Planning ........................................................................................ 49 4.5 Pilots and Controllers Requests ....................................................................................... 51 4.6 Decision-Making and Planning Difficulties .................................................................... 51 4.7 Decision Aids................................................................................................................... 53

v

Appendices A - Participants Recruitment Letter B - Summary Tables C - Institutional Review Board D - Interview Protocol E - Informed Consent Form List of Illustrations Figures Page

1. Illustration of Aircraft Restricted Airspace................................................................................ 2 2. Adaptation of Wickens et al.’s (1997) Cognitive Model of the Controller’s Task.................... 4 3. Number of Actions Planned by Controllers Before They Assume Control of Position .......... 27 Tables Page 1. Selected Demographics of Study Participants ......................................................................... 13 2. Air Traffic Control Facilities Visited ....................................................................................... 14 3. Sources of Information Used Before Assuming Control of Position....................................... 19 4. Types of Information Gathered Before Assuming Control of Position ................................... 19 5. Moment When Mental Pictures Are First Formed................................................................... 20 6. Comments Regarding Scanning Technique ............................................................................. 20 7. Comments Regarding Awareness of Other Sectors ................................................................. 21 8. Personal Memory Techniques Used by Controllers................................................................. 22 9. Memory Techniques Involving the Use of Flight Progress Strips ........................................... 23 10. Reasons to Use Flight Progress Strips ................................................................................... 23 11. Methods Used by Controllers to Keep Improving After Formal Training ............................ 24 12. Improvements Resulting From Greater Experience............................................................... 25 13. Personality Traits of Controllers Handling Large Volumes of Traffic With Ease................. 25 14. Skills and Techniques of Controllers Easily Handling Large Volumes of Traffic ................ 26 15. Opinions on Which Group of Controllers More Frequently Bet on the [Out]Come ............. 26 16. Conditions Requiring More Planning When Assuming Position .......................................... 27 17. Strategies Used When Conflict is Uncertain by Type of Facility.......................................... 28 18. ATC Experience and Strategy Used When Conflict is Uncertain ......................................... 28 19. Use of First Strategy in Conflict Situation by Type of Facility ............................................. 29 20. Strategies Considered Under High Workload........................................................................ 29 21. Use of Backup Strategies by Controllers ............................................................................... 30 22. ATC Experience According To Usage Of Backup Strategies ............................................... 30 23. Conditions in Which Controllers Formulate Backup Plans ................................................... 31 24. Conditions in Which Controllers Use a Larger Buffer .......................................................... 31 25. Factors Leading Controllers to Ask for Help ......................................................................... 32 26. Characteristics of Other Controllers Causing Participants to Adapt...................................... 33 27. Situations When Controllers Do Not Honor Pilots’ Requests ............................................... 33 28. Benefits of Direct Flights ....................................................................................................... 34 29. Disadvantages of Direct Flights ............................................................................................. 35 30. Conditions In Which Direct Flight Requests Are Honored ................................................... 36 31. Coping Strategies Used to Deal With Boredom..................................................................... 36 vi

List of Illustrations (Cont.) Tables 32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. Page

Influence of High Workload on Separation Decisions and Planning..................................... 37 Effects of Fatigue ................................................................................................................... 37 Coping Strategies Adopted When Fatigued ........................................................................... 38 Effects of Age on Controller Planning and Decision Making ............................................... 39 Adaptive Strategies to Mitigate Age-Related Effects ............................................................ 40 Most Difficult Situations in Which to Maintain Separation .................................................. 40 Effects of Bad Weather .......................................................................................................... 41 Strategies Adopted During Bad Weather ............................................................................... 42 Planning and Conflict Detection Aids Used by CPCs ........................................................... 42 Respondents’ Complaints Regarding Conflict Alert Tool ..................................................... 43 Radar-Based Tools Supporting Controller Decision Making and Planning .......................... 44 Other Tools Supporting Controller Decision Making and Planning...................................... 45 Types of Aids That Would Benefit Controller Decision Making and Planning .................... 45

vii

viii

EXECUTIVE SUMMARY The Panel on Human Factors in Air Traffic Control Automation proposed to increase the level of automation in Air Traffic Control (ATC) facilities to accommodate the growth in the number of flights projected over the next decades. They also recommended that automation efforts in the near future focus on the development of decision aids for conflict resolution and for maintaining separation. Developing efficient decision aids requires a good understanding of human decision making and planning. Human factors researchers from the Federal Aviation Administration William J. Hughes Technical Center conducted semi-structured interviews with 100 Air Traffic Control Specialists (ATCSs) to examine their perspective regarding controller decision making and planning and related cognitive processes such as learning, memory, and situation awareness (SA). ATCSs described a variety of decision-making and planning strategies. They reported that they plan their first actions and start building their mental picture prior to assuming control of their position. Most controllers indicated that they always try to formulate a backup plan. The more experienced participants were, the more likely they reported formulating backup plans. The strategies reported by participants sometimes varied according to their type of facility. Terminal controllers were more likely than en route controllers to report using the first strategy that they develop instead of considering alternatives when a potential conflict is detected or when workload is high. Terminal controllers also indicated that they were less likely to wait and see when they are not sure if there is a conflict. Some common themes pervaded participants' answers to many of the questions. First, participants often reported becoming more conservative or cautious (e.g., use a larger buffer) when confronted with difficulties like bad weather, high workload, fatigue, and aging. This reflects the main priority of ATC of ensuring safety. Participants' reports also emphasized the collective nature of ATC. Controllers must coordinate their actions and plans with many other actors, such as pilots and controllers working with and around them. Results suggest that controller SA generally includes knowledge of the skills and preferences of the other controllers. The importance of teamwork was also emphasized when participants reported fighting boredom by watching other sectors and protecting other controllers. Helping without a specific request corresponds to the highest level of team coordination. Finally, participants’ responses suggest that ATC is a service industry and that honoring pilots' requests is their duty. Participants indicated that they consider honoring requests based on their workload, on the potential impact on the traffic in their own sector, and on the impact on the controllers' workload in the next sectors. Some of the responses may facilitate the development and implementation of decision aids adapted to the needs of controllers. Decision support systems should consider the crucial role of SA in controller decision making and planning. For example, according to participants, experienced and skilled controllers would have a greater SA than novices and less skilled controllers. Future decision aids could assume that the level of SA would vary according to the experience and ability of controllers. Future support systems should also consider that controllers start forming their mental picture before assuming control of their position and provide them with the relevant information. Decision aids could also help controllers to maintain their SA of surrounding sectors and positions. Electronic flight strip systems may have ix

to provide users with ways or procedures that will replace the flight strip procedures that currently support controller memory. Participants' reports emphasized the difficulties bad weather creates and the need to develop systems that will support controllers in these conditions. This study should provide investigators with different targets of opportunity for future studies. One could determine the importance of the different types of information that controllers collect to establish their mental picture and to identify which ones are not usually covered in the position relief briefing. Another study could investigate the frequency that memory techniques and separation strategies reported by the participants are used and if usage varies according to controllers' experience and type of facility. Another investigation could help to assess how much controllers agree on what characterizes skilled controllers by asking them to rate the importance of the different factors identified in the present study. The participants most often requested conflict probe type decision aids. This coincides with the Panel on Human Factors in Air Traffic Control Automation's recommendation to develop automated decision aids for conflict resolution and maintaining separation. Moreover, many controllers reported that they have limited trust for existing systems. A future study could therefore concentrate on controllers' perceived needs regarding conflict probes to ensure that future automation will meet their expectations. Some controllers wished that data block presentation could be modified. An interesting question would be to determine if an automated system emphasizing different types of information according to the situation would help controllers. Similarly, other participants wished that data blocks be added to ground radar displays. Determining tower controllers' needs could facilitate the implementation of such a feature. The present study has provided a greater knowledge of controller decision making and planning. The results may guide designers of decision support systems and help them match these tools with users' perceived needs and facilitate user acceptance. The results will also help to identify targets of opportunity for more focused interviews in field facilities.

x

1. Introduction Air Traffic Control Specialists (ATCSs) are decision makers in a dynamic environment involving many actors, constant updating of relevant information, and, sometimes, conflicting goals. They often need to make difficult decisions with incomplete information, under time pressure and high workload. The Panel on Human Factors in Air Traffic Control Automation proposed to increase the level of automation in Air Traffic Control (ATC) facilities to accommodate the growth in the number of flights projected over the next decades (Hopkin, 1998; Wickens, Mavor, Parasuraman, & McGee, 1998). They also recommended that automation efforts in the near future focus on the development of decision aids for conflict resolution and for maintaining separation. To be effective, these decision support systems must rely on good models of human decision making (Mosier, 1997; Mosier & Skitka, 1996). 1.1 Background In FY 1999, the Research Development & Human Factors Laboratory at the William J. Hughes Technical Center initiated the first in a series of studies to investigate ATCS decision-making strategies. Human factors researchers from the National Airspace System (NAS) Human Factors Branch (ACT-530) conducted semi-structured interviews with 100 ATCSs to examine their perspective regarding controller decision making and planning. The goal was to explore controllers’ views of important issues related to the information they use, difficulties encountered, and potential improvements. ACT-530 designed the study to expand the knowledge base and serve as a foundation on which to build future research on decision support automation and training. 1.2 Literature Review 1.2.1 Decision Making and Planning in Air Traffic Control Controllers collaborate with pilots, technical staff, management, and other controllers to assure the safe, orderly, and expeditious flow of air traffic. They ensure safety by guaranteeing minimum separation between aircraft. To do so, they must reserve a block of airspace around each aircraft. This space is defined by altitude and lateral dimensions and is shaped like a “hockey puck” (see Figure 1). The size of the reserved block has different values in different regions of the airspace, as defined in the ATC Handbook (FAA, 2000a). For example, under Instrument Flight Rules (IFR), the minimal vertical separation is 1000 feet at or below Flight Level (FL) 290. Above, it becomes 2000 feet. Factors like the aircraft performance characteristics and navigation systems in use also determine the size of the restricted airspace. The role of the controller is to not let the reserved airspace of two aircraft overlap (Nolan, 1994). If they do overlap, a separation error occurs.

1

Figure 1. Illustration of aircraft restricted airspace. ATCSs use different techniques to ensure aircraft separation. Some of the most common ones are speed control, altitude change, radar vectors, and holding patterns (Ammerman, Becker, Jones, & Tobey, 1987). The frequency with which controllers use separation techniques differs greatly from one type of facility to another. For example, reducing speed in Air Route Traffic Control Centers (ARTCCs) may not be desirable because it would reduce air traffic efficiency. However, final approach controllers in terminal radar approach control (TRACON) facilities may use speed control extensively. Ensuring the safety of aircraft is a controller’s main priority, but another part of the Federal Aviation Administration (FAA) mission is to guarantee the efficient flow of traffic through the NAS. Provided that safety is not compromised, airline companies, pilots, and the traveling public have an interest in efficient traffic flow. Controllers must address the sometimesconflicting goals of safety and efficiency “through an intricate series of procedures, judgments, plans, decisions, communications, and coordinated activities” (Wickens, Mavor, & McGee, 1997, p. 21), in an environment in which errors may have dramatic consequences. Decision makers working in complex environments make errors (Reason, 1990). In the context of ATC, Wickens et al. (1997) proposed that there are two types of errors: operational errors and controller errors. An operational error is a formal designation and occurs when the reserved airspace of two aircraft overlap or when minimum separation criteria are not met between aircraft and terrain, obstacles, or obstructions (FAA, 1987). This type of error has more serious safety implications. Controller errors refer to “a much wider range of inappropriate behaviors that result from breakdowns in information processing” (Wickens et al., 1997, p. 103). These errors may have minor safety implications or severe ones. Most operational errors are made under conditions of moderate to light levels of workload, traffic complexity, and traffic volume, and when controllers are working under the combined radar/radar associate function (Redding, Ryder, Seamster, Purcell, & Cannon, 1991). Redding and his colleagues suggested that deficient Situation Awareness (SA) due to a lack of vigilance in monitoring caused many errors. Redding and Seamster (1994) confirmed the previous findings when observing that most operational errors occur with traffic levels of moderate complexity, with an average of only eight aircraft under control, and immediately following a shift break. They also proposed that failure to maintain adequate SA was a major cause of operational errors.

2

Faulty controller decision making may also result in operational errors or compromised safety. For example, in November 1975, an Eastern Airlines DC-10 and a Trans World Airlines L-1011 almost collided head-on while operating on the same airway at FL 350 (Danaher, 1980). The pilot of the DC-10 avoided the midair collision with an evasive maneuver that still resulted in 24 persons being injured. Investigation of the incident revealed that a Cleveland ARTCC radar controller had cleared the Eastern Airlines flight to climb through FL 350 to FL 370, while the L1011 was cruising at FL 350. The controller was aware of the potential conflict but decided to wait hoping that separation would be ensured when the two aircraft passed each other. This is referred to as “anticipated separation.” The controller assumed that he could keep monitoring the aircraft on his radar and determine in time if new clearances would need to be issued. However, the controller became absorbed with secondary tasks, and another controller relieved him 1 minute before the near-collision. The second controller detected the unresolved conflict 50 seconds after taking over the position and immediately instructed the DC-10 to descend. One second before the descent instruction was issued, the DC-10 captain sighted the other aircraft, which prepared him to execute the evasive maneuver promptly. Deficient decision making, the first controller’s decision not to take immediate positive action, almost caused a midair collision. Despite the challenges confronting ATCSs, the number of operational errors is still relatively low. However, the projected increase in air traffic will put more pressure on the system and emphasize the need to reduce the likelihood of errors. The Panel on Human Factors in Air Traffic Control Automation suggested “decision making may be improved by training and displays that are sensitive to strategies that do work in real-world environments” (Wickens et al., 1997, p. 108). The group subsequently recommended that automation efforts in the near future focus on the development of decision aids for conflict resolution and maintaining separation (Wickens et al., 1998). One concern of the panel is that automated decision aids relying on incorrect models of human decision making may result in systems that are less efficient than the human alone (Hopkin, 1988; Mosier, 1997; Mosier & Skitka, 1996). The development of decision support technologies should therefore benefit from an enhanced understanding of the decision-making and planning processes used in operational settings by ATCSs. Understanding what situations make the task of controllers difficult and impair their performance will help to design the most effective decision aids (Leroux, 1997). 1.2.2 Cognitive Model of the Controller’s Task Decision making is a complex process. An understanding of its mechanisms requires examining decision making in a larger framework in which it interacts with many other cognitive processes. Figure 2 illustrates a generic model of the cognitive processes involved in the ATCS’s task, proposed by Wickens et al. (1997). The model includes five cognitive stages or processes that intervene between events (on the left) and actions (on the right): selective attention, perception, SA, decision making and planning, and action execution. The model suggests that controllers selectively attend to and perceive events to build and maintain awareness of the situation. SA is the principal input to decision making and planning, which may result in the execution of actions like communications and keyboard use. The model also illustrates the contribution of memory in the controller’s task. Immediate memory supports computations and maintains an awareness of the dynamic aspects of the situation. Prospective memory allows remembering actions to be

3

External Events Aircraft weather

Selective Attention

Vocal Perception Phonetic working memory Predictions Mental Model Attentional resources Spatial working memory Intentions & Plans Manual

Action Execution

Situation Awareness

Decision making and planning Communications

Radio weather reports Display strips Perceived events

Keyboard

Tools

Immediate memory

Prospective memory

Airspace, aircraft, organization, individuals, equipment

Strategies for allocating attention, remembering, verifying and improving SA Long-term memory

Procedures Tasks Goals Constraints

Figure 2. Adaptation of Wickens et al.’s (1997) cognitive model of the controller’s task. performed in the near future. Controllers draw on the structures of long-term memory to access their static knowledge of the airspace, radar equipment, and weather (Redding et al., 1991). Controllers’ knowledge of ATC and strategies supporting processes like decision making and planning reside in long-term memory. Decision making and planning are highly dependent on the processes of attention, perception, memory, and SA. Controllers attend to and perceive events that come in many different forms. These events include visual changes on the radar display. Visual scanning of the radar display is a crucial activity for controllers. Scanning influences decision making and planning because it plays a major role in conflict detection and the acquisition and maintenance of SA. Soundly made decisions may still result in mistakes if they are based on information acquired through deficient scanning processes. Breakdowns in the serial process of scanning may result in a critical event being missed, making decision making more vulnerable (Stein, 1992). However, Redding et al. (1991) concluded that lack of attention and active processing of information appears to be largely responsible for the misuse or misidentification of data rather than decreased visual scanning.

4

SA greatly influences decision making (Endsley & Smolensky, 1998). For example, the selection of a problem-solving strategy is based on SA. Similarly, the goals of the decision maker influence SA. Endsley (1988) generally defined SA as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status in the near future” (p. 97). In a dynamic environment like ATC, a controller must remain aware of critical features that vary constantly. Endsley (1995) stressed the importance of SA for decision making by pointing out that inaccuracies or lapses in SA might lead to a disaster, even if the decision maker makes the right decision. However, Endsley added that poor SA does not always lead to poor performance. For example, a decision maker might realize his or her lack of SA and adopt strategies that will reduce the likelihood of poor performance. According to Endsley, an understanding of SA should allow design improvements to decision support systems that will provide decision makers with the information they need in an appropriate form. Several studies have investigated what information ATCSs seek to maintain aircraft separation and their awareness of the situation. The list of relevant pieces of information is very long (known winds, weather patterns, airspace considerations, aircraft turn rate, descent and ascent rate, etc.), but some seem to play a greater role than others. For example, Helbing and Eyferth (1995, as cited in Hutton, Olszewski, Thordsen, & Kaempf, 1997) observed that call sign, altitude, cleared altitude, and exit waypoint accounted for 93% of all demands of information in their study. Altitude and relative position were best memorized and used more frequently by Bisseret’s (1971) subjects, who had been told that the experiment was concerned with problemsolving time and not memory. After analyzing verbal protocols and interviews, Leplat and Bisseret (1966, as cited in Vingelis, Schaeffer, Stringer, Gromelski, & Ahmed, 1990) proposed that controllers are interested in the future states of pairs of aircraft, and they found that the following six attributes are compared in this order: a. b. c. d. e. f. Altitude Flight paths Longitudinal separation Relative speeds Direction of flights after reporting points Lateral separation

Determining what information is used under different strategies in different conditions would certainly lead to the design of decision-making aids that would be compatible with the decision style and actions of ATCSs. Knowing what type of information is prioritized in different situations should lead to the development of displays that are sensitive to the strategies used by controllers in their operational settings (Wickens et al., 1997). Controllers rely heavily on their memory to execute their tasks. The FAA recognized the need to investigate the impact of memory on ATC operations (Stein, 1991). A handbook providing information and helpful hints on human memory was developed for controllers (Stein & Bailey, 1989).

5

To help controllers reduce the frequency of operational errors caused by memory lapses, the Technical Center launched a 3-year research program to develop practical and effective memory aids (Vingelis et al., 1990). Gromelski, Davison, and Stein (1992) observed that controllers perceive memory aids as crutches for unskilled controllers and that the three memory aids they used the most involved strips: strip management, tilted strips, and strip marking. Based on these observations and some of the other data collected during the 3-year research program, Stein and Bailey (1994) published a new controller memory guide and made it available in ATC facilities. Used as a memory aid, flight strips facilitate strategic planning and prospective memory. Flight strips also support controller decision making by providing vital flight plan information and allowing the detection of conflicts. However, use of flight strips vary enormously between facilities. 1.2.3 Factors Influencing Decision Making Wickens et al.’s (1997) cognitive model of the controller’s task illustrates that decision making and planning are highly influenced by their interaction with cognitive processes like SA and memory. Decisions and plans are also determined by the characteristics of the decision maker, task, and context. 1.2.3.1 Decision Maker Related Factors Investigating how experts make efficient decisions and plans will certainly benefit the development of automated decision aids. However, understanding what differentiates novices from experts, or, in other words, how expertise develops, might be even more crucial. One concern of the Panel on Human Factors in Air Traffic Control Automation is that novices who use decision support expert systems do not perform as well as experts (Mosier, 1997). Novices using automated aids seem to achieve the most satisfactory results when “the task is routine and covered by standard procedures” (Mosier & Skitka, 1996, p. 210). This suggests that automated decision support systems will be potentially more efficient if they integrate user models that reflect different levels of expertise. Many studies have examined the effects of expertise on decision making and use of strategies. According to Brehmer (1992), experienced subjects have learned that, to perform well in a complex dynamic system, they have to adopt “grandmother rules.” More specifically, compared to less experienced subjects, they will make fewer decisions, collect more information before making a decision, and check the results of their decisions before making new decisions. Dreyfus and Dreyfus (1984) suggested that novices tend to make decisions in a careful, analytical fashion, whereas experts appear to make decisions quickly rather than making serial and exhaustive searches. Similarly, Klein (1989) proposed that, in real-world situations, experienced decision makers learn a large set of patterns and associated responses and that, in general, they do not compare a set of alternatives based on their predicted outcomes but, instead, recognize a situation and retrieve an appropriate response.

6

In their extensive cognitive task analysis of en route ATC, Redding et al. (1991) observed how novice, intermediate, and experienced controllers use strategies. When compared to novice controllers, experienced controllers tend to use a smaller number of strategies, which include more control actions and aircraft. Experienced controllers also use a greater variety of different strategies, which indicates that they possess a wider repertoire of strategies. According to Redding et al. (1991), experts use more workload management strategies than novices. They especially use strategies allowing them to identify aircraft that can be expedited through their sector and reduce the number of aircraft to which they need to attend. Intermediates also used more workload management strategies than beginners did, suggesting that the use of these strategies increases with experience. The authors also concluded that the greater the number of strategies used overall, particularly monitoring strategies, the fewer the errors. More specifically, three workload management strategies are closely associated with a reduced number of errors: determining what to do to eliminate a factor, identifying aircraft that are not a factor, and determining how to expedite aircraft through your sector. Another controller-related factor that is often highly correlated with experience is age. Few studies investigated the effects of aging on performance. Becker and Milke (1998) suggested that “the ability to handle simultaneous visual and auditory input or to return to a task after a break to complete another task is critical to success and is the sort of cognitive function most affected by age” (p. 944). The authors also pointed out that many of the controllers forming the current ATC workforce were hired after the Professional Air Traffic Controller Association strike and subsequent 1981 firing by then President Reagan. They stressed the importance of determining the nature and extent of the effects of aging because they believe that “a high proportion of the ATC workforce will be at risk for displaying age-related changes in job performance efficiency over the next 10 years” (p. 944). Many studies have investigated the effects of aging on cognition (Fisk & Rogers, 1997). Agerelated decrements in decision-making processes have been observed in tightly controlled laboratory experiments, but studies conducted in more natural settings or in the workplace have shown more similar performance levels among older adults and younger ones (Walker, Fain, Fisk, & McGuire, 1997). Many of these studies have argued that, in numerous working environments, individuals can use varying task strategies and control the scheduling of different tasks, allowing older adults to keep performing normally by using different decision heuristics (Davies, Taylor, & Dorn, 1992; Johnson, 1990). It is also believed that domain-relevant experience or skill maintenance might help older individuals to maintain their performance level (Morrow, Leirer, Altieri, & Fitzsimmons, 1994). Controllers may change the way that they approach traffic separation problems or may bring to bear cognitive processes that are not affected by aging (Becker & Milke, 1998). Gromelski et al. (1992) reported that 9 out of 10 controllers contend that they experience boredom on the job. Few have identified ways to avoid this situation. Boredom may promote overconfidence and lack of attention, which would make decision making vulnerable. Low workload episodes could represent an opportunity for controllers to adopt strategies that are less cognitively economical or to employ infrequently used strategies.

7

Many other controller-related factors might influence ATCS decision making and planning. Stress may promote problem solving rigidity (Cowen, 1952). Fatigued subjects tend to choose riskier strategies (Holding, 1974). High trait anxiety subjects appear to adopt strategies that would result in more control over time-constrained tasks (Leon & Revelle, 1985). Finally, depressed individuals may lack the same levels of motivation and willingness as the less depressed and make an ineffective use of strategies (Williams, Watts, MacLeod, & Mathews, 1988). 1.2.3.2 Task-Related Factors Decisions made by decision makers are contingent on many task-related factors. In ATC operations, for example, the complexity of the sector for which they are responsible, the volume and complexity of the traffic, and time pressures may influence controllers. In a simulation presented to approach controllers, Sperandio (1971) observed that, under low traffic loads, controllers used more direct routings, which required that they consult aircraft performance information more frequently. Under heavy traffic loads, controllers tended to use standardized routings and more holding patterns, which required less performance data. According to Sperandio, controllers maintained their performance level by using the standardized routings, which reduced the number of variables they needed to process. Under the low workload condition, using direct routes was more work for the controllers because they had to process more variables, but it fulfilled their need to maintain a certain level of activity. As described by Sperandio (1978), controllers regulate their increasing workload (or maintain it at an appropriate level) by using successively more economical strategies. As traffic increases, controllers might progressively use more standardized routings to allow them to process a smaller number of variables for each aircraft and help them treat “each aircraft as one link in a chain whose characteristics remain stable and not as an independent body moving in free space among other independent moving bodies” (p. 196). Sperandio (1978) proposed that workload also influences decision-making processes by determining which objectives controllers will prioritize. Although ATC objectives may sometimes conflict, Sperandio suggested that they are hierarchically organized. The fundamental objective for the ATCS is to maintain safety by observing separation standards, immediately followed by the goal of maintaining a high rate of progress of aircraft through the system. The secondary objectives would relate to providing ATC service and increasing efficiency such as assigning requested altitudes and routes to maximize fuel efficiency. As their workload increases, ATCSs often take secondary objectives less and less into account to concentrate on the primary ones. 1.2.3.3 Contextual Factors Controllers have to make their decisions and plan their separation strategies with the collaboration of pilots, technical staff, management, and other controllers. The controllers working with them and around them, the type of management leadership, and the requests of pilots may influence how controllers make their decisions and plan their strategies.

8

Redding et al. (1991) reported that many ATC errors are made when controllers are working under the combined radar/radar associate function. They suggested that this situation probably promotes overconfidence and a lack of vigilance, which in turn jeopardizes the quality of decision making. Sperandio (1978) also suggested that, as the task load increases, the tasks of the associate become increasingly dependent on the tasks of the radar operator and consequently tends to overload the principal operator even more. ATC is a service industry and pilot and airline requests heavily influence controllers’ separation strategies and decisions. For example, they make their requests for different routes or to fly at aircraft optimal altitudes to allow time savings, fuel economy, and greater comfort for passengers. An important goal for ATCSs is to satisfy users’ requests as long as safety and the efficiency of the airspace is not compromised. By giving airspace users more flexibility in determining their own flight routes, the implementation of Free Flight proposals (Planzer & Jenny, 1995; RTCA 1995a, 1995b) might also increase the number of pilot requests. It is currently unknown how this will impact controller decision strategies. Training also has an influence on controller decision making. ATCSs receive their training in several phases. The Air Traffic Control Academy in Oklahoma City offers initial qualification and basic training (e.g., TRACON controllers take the Academy basic radar course). However, the assigned facility provides most of the advanced training. The ATCS training program has often changed over the years. For example, when the ATCS Nonradar Screen program was operational, the emphasis was on screening candidate controllers instead of training them (Fisher & Kulick, 1998). In 1992, the ATCS/Pre-Training Screen (PTS) replaced the previous program, and the Academy implemented a train-to-succeed curriculum. However, due to technical considerations, PTS did not last very long and is currently on hold. An important part of the facility training consists of on-the-job training (OJT), where developmental controllers work the different positions of the facility under the close supervision of an instructor. Previously, instructors were told not to teach their personal strategies or techniques (Fisher & Kulick, 1998). It was believed that trainees should be allowed to develop their own preferences. The extent to which different controllers will have learned from the instructors might vary. Controllers may also have learned or perfected their skills outside of formal training. For example, Sperandio (1978) pointed out that, although controllers might be exposed to the entire repertoire of operational strategies during formal training, it is through personal experience that they really learn how to alternate from one strategy to another. 1.2.4 Theories, Models, and Approaches of Dynamic Decision Making The development of decision support systems will benefit from a better understanding of the factors influencing controller decision making and planning. A long history of decision-making research will also contribute to that development. The study of decision making has generated many theories and formal models of decision making (for reviews of the literature, see Doherty, 1993, Edwards, 1987; Letho, 1997, and Lipshitz, 1993), which all serve one or more of the three following purposes (Sarma, 1994): a. Normative models aim to characterize optimal or most efficient decision-making processes.

9

b. Prescriptive models attempt to describe how decision makers should be trained or how decision aids should interact with them. c. Descriptive models try to identify the psychological processes used by decision makers.

Models of decision making also differ along many other dimensions. For example, theories may aim to explain individual versus group decision making. Most of the initial efforts in decisionmaking research resulted in normative and prescriptive models, developed in the fields of economics and statistics, quantitatively representing a rational and optimal decision maker. Many have suggested that classical decision theory is too rigid and static to provide an adequate representation of decision making in real-world environments (Beach & Lipshitz, 1996) and that approaches emphasizing the dynamic nature of decision making must be adopted (Brehmer, 1992). Edwards (1962, as cited in Brehmer, 1992), in his classic description, identified the characteristics of a dynamic decision-making environment: a. b. It requires a series of decisions. The decisions are not independent.

c. The state of the problem changes, both autonomously and as a function of the decision maker’s actions. Brehmer later added a fourth item to the list: d. The decisions have to be made in real time.

Edwards (1962, as cited in Brehmer, 1992) and Toda (1962, as cited in Brehmer, 1992) made the first efforts to understand dynamic decision making by applying the subjectively expected utility theory (von Neumann & Morgenstern, 1947), a classical decision theory, to dynamic problems instead of static ones. They adopted a normative-descriptive approach in which the behaviors of real decision makers were compared to an ideal decision maker. Discrepancies would have suggested that limitations are imposed on decision makers. Their approach suffered from at least two problems identified by Rapaport (1975). First, as dynamic problems become complex, it quickly becomes impossible to find analytical solutions to solve them. Second, even when decision makers adopt strategies largely different from the ideal ones, the outcomes are often the same. The “flat maximum problem,” as it is designated, makes identifying the limitations imposed on the subjects difficult. Cognitive Continuum Theory (Hammond, 1980) is a more recent approach to dynamic decision making. It suggests that decision-making activities are located on a cognitive continuum varying from highly intuitive decisions to very analytical ones. In a review of previous research, Hammond (1993) showed that the decision-making tendency to rely on analysis instead of intuition augments when: a. b. c. d. The number of cues increases. Cues are measured objectively instead of subjectively. Cues are of low redundancy. Decomposition of the task is high. 10

e. f. g. h. i. j.

Certainty is high. Cues are weighted unequally in the environmental model. Relations are nonlinear. An organizing principle is available. Cues are displayed sequentially instead of simultaneously. The time period for evaluation is long.

Payne, Bettman, and Johnson (1988, 1993) contributed to the development of a similar approach, the theory of contingent decision making. This theory adopts a cost-benefit framework in which decision makers compare the cognitive effort against the accuracy of different decision strategies. The characteristics of the task and its context determine cognitive effort and accuracy. Decision makers will switch strategies to reduce the cognitive effort, increase accuracy, or respond to time pressures. The theory of contingent decision making is in agreement with Sperandio’s (1978) description of controllers regulating their increasing workload by adopting strategies that are more economical. Naturalistic decision making, a recent strain in decision-making research, has focused on the critical aspects of operational settings and more natural and dynamic environments (for reviews see Klein, Orasanu, Calderwood, & Zsambok, 1993 and Zsambok & Klein, 1996). This approach criticizes traditional models of decision making for their emphasis on laboratory studies and for having no direct relevance to real-world decisions (Klein, 1989). The Recognition-Primed Decision Model (RPDM) is one representative of naturalistic decisionmaking models. Klein (1989) developed the RPDM after observing the decisions made by firefighters and experts from other fields in their naturalistic environment. He concluded that experts make most of their decisions without comparing different alternatives, contrary to what traditional models postulate. Instead, experts are involved in a situation recognition process in which, based on their experience, they classify the situation and immediately consider the typical way to handle it. After evaluating the feasibility of the option, they implement it if they foresee no problems. If something might go wrong, the decision maker will modify the option or simply reject it and consider another typical solution. Experts from domains like fire fighting, paramedics, and other time-pressured environments use the RPDM to represent the decision-making activities (Klein et al., 1993). Some studies have also applied this model to ATCS decision making (Hutton et al., 1997; Mogford, Allendoerfer, Snyder, & Hutton, 1997; Mogford, Murphy, Roske-Hofstrand, Yastrop, & Guttman, 1994). The model has certainly gained some popularity among researchers investigating dynamic and timepressured domains like ATC, but it also received some criticisms (Doherty, 1993). One shortcoming of the RPDM is that, by focusing on expert decision making, it might fail to represent the evolution of a novice becoming an expert. By representing only the processes of expert decision makers, the models might fail to serve prescriptive purposes such as indicating how decision aids should interact with less experienced controllers. Beach (1993) provided us with an interesting summary by describing four revolutions in the development of behavioral decision theory. The first one occurred when it became clear that 11

decision makers rarely examine all the alternatives to a decision, that they use heuristics (Tversky & Kahneman, 1974) or that they adopt a satisficing rule (i.e., settle for the first choice that is “good enough”) instead of optimizing (Simon, 1955). The second one consisted of realizing that decision makers choose between strategies to make decisions, as illustrated by the contingency theory (Payne et al., 1993) and the cognitive continuum theory (Hammond, 1980). According to Beach, the third one is presently occurring because we are recognizing that decision makers rarely make choices and, instead, rely on prelearned procedures, as suggested by the RPDM (Klein, 1989). Beach reveals that the last one is just beginning. Decision research is adopting a multidisciplinary perspective drawing not only on economy but also on cognitive psychology, organizational behavior, and systems theory. 1.3 Purpose and Rationale The purpose of the present study was to enhance our current knowledge of controller decision making and planning. Such knowledge may improve the design and implementation of new decision support tools. It also investigated controllers’ opinions, preferences, and beliefs regarding their decision-making and planning operational practices and assessed their concurrence with theories such as those presented in the introduction. Reducing the potential mismatch between controllers’ perceived needs and future decision aids was another goal. Finally, this study investigated if the opinions, perceived needs, and operational practices of controllers differ between individuals according to some variables such as their age, the type of facility in which they work, and their level of experience in ATC. 2. Method 2.1 Participants The researchers interviewed 103 ATCSs who participated in the study on a voluntary basis (recruitment letter is presented in Appendix A). At the request of a local union representative, two interviews were not completed. Another one was not completed because the participant had to return to the operation. We therefore discarded the results of the three participants. Table 1 presents selected demographics of the 100 ATCSs who completed their interview (more extensive demographics are presented in Appendix B in Tables B1 to B3). Study participants included 7 females and 93 males. Eight participants were staff members maintaining their operational currency. The age1 of participants ranged from 27 to 57 years and averaged 41.4 (SD=5.71). Their ATC experience varied from 4.5 to 36 years with a mean of 17.6 (SD=5.93). The Technical Center local Institutional Review Board (IRB) reviewed and approved the study protocol. The IRB Application Form is presented in Appendix C.

1

Age statistics exclude one missing value.

12

TABLE 1. SELECTED DEMOGRAPHICS OF STUDY PARTICIPANTS (N=100) Demographics Gender Females Males Facility Type Air Traffic Control Tower (ATCT) TRACON Combined ATCT/TRACON ARTCC Average Age (Years) Average ATC Experience (Years) 13 6 13 68 41.4 (SD=5.7) 17.6 (SD=5.9) 7 93 Participants

Participants included controllers from ATCT, TRACON, combined ATCT/TRACON, and ARTCC facilities. A majority of the respondents, 68 out of 100, worked in ARTCC facilities. Participating controllers spent between 8 months and 31 years at their current facility (M=13.0, SD=6.1). These results suggest that controllers have spent most of their career at the same facility (13 years out of 17.6 average). Results show that 30% of participants have had some experience in facility types different than the one in which they work. The proportion of participants who have worked in other types of facilities is smaller in ARTCCs (15%) than in the three other types of facility (38%). Although 30% have had some experience in other facility types, results indicate that this type of experience was limited to an average of 1.6 years for all the participants (SD=2.9). We visited high traffic level facilities (Table 2) from six different FAA regions2. We selected one ATCT, one TRACON, and one ARTCC from each region, and these facilities were closely located geographically to minimize travel costs. 2.2 Apparatus and Interview Protocol 2.2.1 Audio Tape Recorders With the consent of respondents, the researchers collected audio recordings of the interviews. We viewed the tapes only as a means to backup the collected information. Sudman, Bradburn, and Schwarz (1996) reported that there is no evidence that the use of a tape recorder in an interview affects responses.

2

The research team visited only one TRACON and one ARTCC in the Western-Pacific Region.

13

TABLE 2. AIR TRAFFIC CONTROL FACILITIES VISITED FAA Region Facility Denver ARTCC Northwest Mountain Denver TRACON Denver ATCT Fort Worth/Dallas ARTCC Southwest Fort Worth/Dallas TRACON Fort Worth/Dallas ATCT Washington ARTCC Eastern Dulles TRACON/ATCT Reagan National ATCT Atlanta ARTCC Southern Atlanta TRACON Atlanta ATCT Chicago ARTCC Great Lakes O’Hare TRACON O’Hare ATCT Western Pacific Los Angeles ARTCC Santa Barbara TRACON

2.2.2 Interview Questions Development The questionnaire consisted of two parts: the demographic questions and the interview questions (Appendix D). Demographic questions included a. b. c. d. e. f. g. h. gender, type of facility, facility level, status as ATCSs or staff maintaining operational currency, number of years of ATC experience, number of years in their current facility, number of years in different types of facility, and age. 14

The second part of the questionnaire contained 26 open-ended questions. Recent research has shown that respondents tend to limit their answers to close-ended questions to the choices offered to them and will neglect to volunteer an opinion not included in the response choices, even if the researcher does not wish them to do so (Bishop, Hippler, Schwarz, & Strack, 1988; Presser, 1990). According to Krosnick (1999), open-ended questions should be considered a viable tool of research, and many criticisms regarding their usage have proved to be unfounded. For example, Geer (1988) has shown that open-ended questions work well even with people who are not very articulate. Contrary to a common belief, respondents do not tend to answer openended questions with the most salient possible response instead of the most appropriate one (Schuman, Ludwig, & Krosnick, 1986). Finally, Krosnick also mentioned that some older studies had shown the superior reliability and validity of open-ended questions over close-ended ones (Hurd, 1932; Remmers, Marschal, Brown, & Chapman, 1923). Therefore, we selected the open-ended format for the questions so that the ATCSs’ responses would not be restricted. We first determined the issues addressed in the interview and the construction of the questions after a review of the relevant literature on decision making and planning. A first set included questions adapted from Gromelski et al. (1992) and some questions developed by the researchers and a subject matter expert (SME). Five SMEs reviewed the questions and offered their advice regarding their relevance, understandability, and interest. The researchers selected a subset of questions based on the advice of the SMEs and submitted them to a pretest. We conducted the pretest of the questionnaire with five non-bargaining unit ATC staff from the Atlantic City International Airport Terminal and from the Philadelphia International Airport Terminal. We used conventional methods of questionnaire pretesting (Bischoping, 1989; Nelson, 1985) to identify questions that respondents had difficulty understanding or that they interpreted differently than we intended. While one researcher conducted the interview, the other sat in the same room and observed. We alternated roles after each interview. After interviewing a total of five respondents, we discussed their experience in a debriefing session in which they identified problematic questions (requiring further explanation, with confusing wording, difficult to read, which respondents refuse to answer, etc.) and made the necessary adjustments. The resulting semi-structured interview included sets of open-ended questions about SA, ATC expertise, decision making and planning in specific contexts, interacting with pilots and other controllers, and decision aids. SA questions requested that respondents describe their scanning techniques, how they establish their mental picture, and how much they are aware of the surrounding sectors or controllers. Participants revealed what memory techniques they use to maintain the picture and remember actions that they want to execute later. Another set of questions examined participants’ opinions regarding ATC expertise. Participants described how they improve their skills after becoming ATCSs, how their decision making and planning changed with experience, and what characteristics experienced and expert ATCSs possess. Many questions addressed the different strategies controllers use in specific contexts. Participants described how much planning they do before assuming control of their position, if it is sometimes better to wait when they are not sure if there is a conflict, if the first strategy they

15

think of is sufficiently good when they identify a conflict, if they consider alternative strategies even in high workload situations, if they use backup plans, when they use a buffer, and when they ask for help. Other questions addressed the strategies controllers use when confronted with potentially difficult situations. Participants explained how they or other controllers deal with boredom, high workload, fatigue, and aging. They also identified which situations make decision making and planning the most difficult. Controllers continually interact with other controllers and aircraft pilots. Participants had to describe how controllers working with or around them influence their planning and separation strategies. Other questions investigated how they decide whether to honor pilot requests and what influence direct routes have on their decision making and planning. Finally, we asked the participants about the decision and planning support tools that they use and which tools are available to them. Interviewers also asked participating controllers to describe which and how decision making and planning aids could benefit them. 2.3 Procedure 2.3.1 Interviewers A subject matter expert and a human factors researcher separately conducted half of the interviews each. A Technical Center Psychologist had instructed the interviewers in interview techniques. 2.3.2 Interviews We conducted semi-structured interviews to investigate the decision-making processes and separation strategies used by controllers in their operational settings. Seashore (1987) suggested that interviews are “highly efficient in accepting unanticipated responses, clarifying ambiguous meanings, and adapting the interview somewhat to the particular case” (p. 319). The interviews focused on controller decision making and planning, but they also examined processes such as SA and memory, which Wickens et al. (1997) depicted as critical components in their model. Researchers traveled to the participants’ workplaces and spent approximately one day at each facility. The duration of each interview was approximately 45 minutes. Interviewers conducted the interviews in a private setting to help ensure confidentiality and minimize organizational disturbance. Before conducting each interview, the interviewer a. b. c. thanked the controllers for their cooperation; described the goals of the study; emphasized that confidentiality and anonymity would be ensured, that the names of the respondents would not be written on the questionnaire, and that no background information would allow the respondents to be identified;

16

d. mentioned that the researcher would take notes but that the participants could consult the notes any time they might wish to do so; e. asked permission to use an audio tape recorder to allow us to complete the handwritten notes if necessary and mentioned that participants usually forget about the audio tape recorder after the interview starts; and asked if the participants had any questions.

f.

To ensure confidentiality and anonymity, we did not identify participants with their name on the questionnaire, and the protocol interview did not contain questions that could conceivably be used to identify the participants. We secured completed forms and audiotapes at all time. After receiving the preceding instructions, each respondent received a copy of the study consent form (presented in Appendix E). Every participant read the form, acknowledged understanding its content, and indicated willingness to participate in the study by signing the form. After signing the form, the researcher kept the signed copy and offered a copy to the respondent. We first completed the controller background information section. Each interview began with the first question in the interview protocol. We then proceeded sequentially through the list of questions. When answering a question, controllers may have addressed issues covered in other parts of the questionnaire. In this situation, we let the participant elaborate and recorded the additional comments under the respective section on the response sheet. When we asked a question for which a partial answer had already been given, we reminded the controller of the initial comments and asked for more elaboration. In order to clarify the meaning of the answers and to gather more comprehensive data, we invited participants to comment or elaborate on their answers. We were also instructed to feel free to clarify the meanings of questions and response choices if participants expressed uncertainty or asked for help. Many researchers have suggested that rigid interviewing, a prevailing principle in survey research, which instructs interviewers to avoid interfering in this manner, might compromise data quality (Briggs, 1986; Mishler, 1986; Suchman & Jordan, 1990). Schober and Conrad (1997) even demonstrated that conversational interviewing could enhance the validity of reports. Gromelski et al. (1992) suggested that the major advantages of conducting an in-depth (conversational) interview are “that the interviewer can a. b. ask for examples to clarify a point; explore the meanings of various phrases that respondent use;

c. probe, that is, ask a question in a variety of ways, to ensure that he or she understands the point that the respondent is making; d. observe body language of the respondent; and e. pursue new topics that the respondent may rise, thereby adding to the comprehensiveness of the data gathered” (p. 7). The last question asked the participant if there was anything that should have been asked about strategic planning but was overlooked. After the last question, we thanked the participant again for his or her cooperation, reemphasized that confidentiality would be preserved, and asked if there were any questions. 17

2.4 Data Analysis 2.4.1 Data Entry and Coding Three data entry clerks performed the data entry, which consisted of transcribing all of the handwritten answers onto spreadsheet files. Researchers assigned a list of numerical codes to each question. A different code represented every different answer to a question. When an answer included more than one statement (e.g., “I use a larger buffer when there is weather or high workload”), data entry clerks assigned every statement a different numerical code and another code represented the multiple-statement answer. They also reserved codes like “99” for missing data and “88” for “don’t know” responses. 2.4.2 Content Analysis A human factors researcher performed the content analyses by analyzing every question independently. This task consisted mainly of regrouping answers or statements into meaningful categories. Then, we assessed the direction (e.g., raising SA versus hindering it) or the frequencies they implied (e.g., never, sometimes, or always). Some answers did not lend themselves to inclusion in a larger category but did provide unique, interesting insights. We included these low-frequency items in expanded tables in Appendix B. We regrouped questions according to the themes to which they referred (e.g., SA, workload). A second human factors researcher and a SME also examined the resulting groupings. Both concurred with the content analysis performed by the first human factors researcher. 2.4.3 Statistical Analyses The statistical analyses consisted almost entirely of descriptive analyses leading to the presentation of raw frequency and contingency tables. Frequency tables in the Results section indicate how many participants reported each item. Total frequencies will not necessarily add up to 100 (the number of participants in the present study) because the same participant could report more than one item when answering a question or may not have commented at all on the items included in a particular table. Frequency tables present the items that participants thought of when answering open-ended questions in the semi-structured interview format. Therefore, participants’ reports are not exhaustive, and they do not necessarily indicate how frequently the different items were used or their importance. Thus, only raw frequencies were presented. Neither percentages nor proportions were appropriate and neither was computed. The openended question format allowed us to identify issues within categories for future research. The questions were not designed, in most cases, for inferential statistical analyses between groups. Inferential analyses were performed as appropriate with nonparametric statistical tests: chisquare, Mann-Whitney U, and the Jonckheere-Terpstra test for ordered alternatives (Rossini, 1997). 3. Results Researchers present the participants’ answers in the following sections: SA, memory and flight progress strips, expertise, decision making and planning, pilots and controllers’ requests, decision-making and planning difficulties, and aids to decision making and planning. 18

3.1 Situation Awareness Controller decision making, planning, and strategies depend highly on SA (Endsley & Smolensky, 1998). When they assume control of their position, ATCSs must maintain a continuously changing mental picture of the airspace. Respondents described how they establish their mental picture prior to assuming control of the position (question 1a). Answers to question 1a indicated that controllers form their mental picture from many different types of information, which they gather from different sources. Table 3 shows that the two sources of information reported the most often were the radar display and the flight progress strips. TABLE 3. SOURCES OF INFORMATION USED BEFORE ASSUMING CONTROL OF POSITION Source of Information Radar display Flight progress strips Status board Data blocks Observe relieved controller Number of Participants 50 26 17 10 6

Participants reported using 27 different types of information when forming their mental picture. Table 4 shows that the answers reported most often were that controllers form their mental picture by looking for conflicts and checking the status of the weather. TABLE 4. TYPES OF INFORMATION GATHERED BEFORE ASSUMING CONTROL OF POSITION Type of Information Conflicts Weather Flow of traffic Equipment status Number of Participants 19 16 8 7

Note. Expanded list of answers is presented in Table B4, Appendix B.

Thirty-five participants also specified when they usually start forming their mental picture. Table 5 reveals that 34 of these controllers reported starting to form their mental picture before or during the relief briefing. Only one controller declared establishing a mental picture after the briefing, by “tuning out” what the relieved controller said.

19

TABLE 5. MOMENT WHEN MENTAL PICTURES ARE FIRST FORMED Moment Before the briefing During the briefing After the briefing Number of Participants 7 27 1

Scanning skills are critical for SA. Question 3 investigated if and when controllers modify the way they scan their radar display. Table 6 presents some of the comments made by the participants regarding their scanning techniques. It indicates that 38 respondents declared that their scanning changes depended on the context and that 30 reported that they always try to scan the same way, using the same pattern consistently. Twenty-two participants mentioned that the way they scan is specific to the sector in which they work. For example, one controller reported using a V-like scan when working approach control, instead of scanning in a circular, aroundthe-clock fashion. According to 22 participants, their scanning was not always uniform because they tend to pay more attention to hot spots or, in other words, regions where conflicts occur frequently or where they seem likely to happen. Other participants added that controllers are sometimes vulnerable to tunnel vision, when all their attention becomes focused on a subpart of their sector and they become unaware of the activity in the rest of the airspace. TABLE 6. COMMENTS REGARDING SCANNING TECHNIQUE Comment Scanning technique changes according to context Always try to use the same scanning technique More time is spent scanning the hot spots Scanning technique depends on sector Number of Participants 38 30 22 22

Twelve participants identified some of the factors that result in controllers changing the way they scan the radar display. Such factors included traffic patterns, bad or foggy weather, automation, runway configuration, traffic complexity, and volume. Some participants indicated that the type of information they seek is always the same when they scan. More precisely, 17 controllers reported always using the same type of information, whereas 7 indicated that the type of information they sought varied according to the context. Question 14 asked tower controllers how much they are aware of what is going on at the other positions and radar controllers how much they are aware of what is going on in adjacent sectors. Thirteen controllers reported that they monitor the activity in other sectors by listening to the other controllers in the room. Participants indicated that their awareness of other sectors depended on a few different factors. For instance, 24 controllers reported that their workload

20

determined how much they were aware of what was happening in other sectors. The busier they are, the more they focus on their own sector and, therefore, become less aware of what happens in other sectors. According to some participants, experience also influenced their level of awareness. They suggested that, with experience, controllers became more aware of other sectors or positions. Experience increased awareness by making controllers more aware of traffic flows (e.g., know when rush is coming), more familiar with the sectors they work in, and familiar with a greater number of sectors and positions. How much controllers are aware of what is going on in other sectors depends on the type of sector and which sector they were working. Some controllers commented on the influence of sector characteristics on awareness. Table 7 presents some of these comments, indicating that controllers have a greater awareness of sectors in their own area and when they are sitting next to each other. TABLE 7. COMMENTS REGARDING AWARENESS OF OTHER SECTORS Comment Aware of sectors that impact your flow of traffic Greater awareness of feeding sectors Greater awareness of sectors in own area Very aware, especially when sitting beside adjacent sector ARTCC controllers do not see the other sectors on the radar scope as much as TRACON controllers Depends on configuration of airspace Greater awareness of arrival sectors that I feed (need to know if the “door will slam”) Greater need to know when high altitude and low altitude sectors in ARTCC feed each other In TRACON, less with sectors well below Local controller knows what the other local controller does and ground controller knows what the other ground controller does More aware when familiar with sector Must keep up with satellite and departure sectors TRACON departure sector is aware of arrival sectors

21

3.2 Memory and Flight Progress Strips Wickens et al.’s (1997) cognitive model implies that memory is the foundation on which SA, decision making, and planning stand. Question 4 asked controllers to identify the personal techniques that they use to help them maintain the picture and remember plans that they want to execute later. Respondents offered a large number of techniques. The most popular answers are presented in Table 8. These results reveal that memory techniques involving the use of flight progress strips are, by far, the ones that were mentioned the most often. Nineteen respondents declared having no need for such techniques. TABLE 8. PERSONAL MEMORY TECHNIQUES USED BY CONTROLLERS Personal Technique Flight progress strips J-Ring [ARTCC only] Data block management [TRACON and ARTCC only] No need/none Writing on notepad Avoid having to remember Help from others (“D-side”/radar associate or pilots) Fix things immediately this way I will not forget (Think of it, you do it)
Note. Expanded list of answers is presented in Table B5, Appendix B

Number of Participants 59 22 20 19 13 12 7 7

Table 9 presents controller memory techniques involving the use of flight progress strips. The two most often reported techniques were to offset the strips in the holding bay and to mark the strips.

22

TABLE 9. MEMORY TECHNIQUES INVOLVING THE USE OF FLIGHT PROGRESS STRIPS Technique Offsetting strips Strip marking Pulling out or holding strip Sequencing strips/Positioning strip Consulting/reviewing strips Use the strip to indicate closed runways Planning Pointing to or touching strips to reinforce memory Number of Participants 28 25 5 3 2 2 2 2

In question 2, participants reported using flight strips for different reasons. We can see in Table 10, which lists the 6 most popular reasons, that 55 participants reported using them for quick reference, 42 as a memory aid, 33 as a backup, 23 to maintain the picture, 17 to detect conflicts, and, finally, 9 to plan. TABLE 10. REASONS TO USE FLIGHT PROGRESS STRIPS Reason Quick reference Memory aid Backup Maintain picture Detect potential conflicts Planning 3.3 Expertise Some of the questions in the present study investigated participants’ opinions regarding how ATCSs develop their expertise and what constitutes ATC expertise. Question 6 asked how controllers keep improving their planning and separation skills after formal training, or more specifically, after their OJT was completed and they become ATCSs. Table 11 summarizes the answers to question 6. As many as 49 of the respondents mentioned that controllers keep improving their skills just by going to work on a daily basis and performing their normal duties. Number of Participants 55 42 33 23 17 9

23

TABLE 11. METHODS USED BY CONTROLLERS TO KEEP IMPROVING AFTER FORMAL TRAINING Method Through experience (daily practice and repetition) Watching other controllers Learning from difficult situations Desire to improve and professional attitude Trying new and different techniques
Note. Expanded list of answers is presented in Table B6, Appendix B

Number of Participants 49 20 18 16 15

According to 20 interviewees, controllers improve their skills by observing other controllers performing their duties. They also mentioned shadowing controllers in other departments (quality assurance, airspace and procedures, and Traffic Management Unit [TMU]) as a beneficial activity. Six respondents also reported that receiving other controllers’ input helped them improve. Eighteen respondents reported that controllers improve when they encounter difficult situations and learn their way out or how to avoid such situations. Sixteen participants specified that controllers improve only if they have the desire to learn and maintain a professional attitude at work. Besides describing how controllers can keep improving their separation and planning skills, a few participants also mentioned factors making continuous learning more difficult. They mentioned that the environment is not conducive to improvement, that refresher training and computer-based instruction are useless, that there are no more simulation practices of emergency situations, and that conflict alert prevents learning. We asked participants to describe how their approach to planning and separating aircraft changed with experience (question 5). Table 12 depicts the most common answers. More specifically, controllers reported most often that, with more experience, their SA, comfort and confidence, and planning have improved. In question 22, we asked participants to describe what special skills, attributes, or techniques allow some controllers to handle large volumes of traffic with ease. Controllers provided a long and diverse list of skills, attributes, and techniques. Only two participants declared that these controllers have no common traits.

24

TABLE 12. IMPROVEMENTS RESULTING FROM GREATER EXPERIENCE Improvement Greater SA More comfortable and confident Better planning Increased familiarity with sectors and controllers Better knowledge of aircraft type performance Act earlier Developed automatisms Less conservative More conservative

Note. Expanded list of answers is presented in Table B7, Appendix B

Number of Participants 32 27 23 14 12 10 8 8 6

Participants defined controllers who easily handle large volumes of traffic with 32 personality traits. Table 13 presents the three traits mentioned the most often. According to the most commonly suggested traits, “jet jocks,” as one participant designated them, would be selfconfident, calm, and intelligent. TABLE 13. PERSONALITY TRAITS OF CONTROLLERS HANDLING LARGE VOLUMES OF TRAFFIC WITH EASE Personality trait Self confident Calm Intelligent (common sense, logical, etc.)
Note. Expanded list of answers is presented in Table B8, Appendix B

Number of Participants 18 17 13

Respondents have also identified a large number of skills and attributes that they think allow some controllers to easily handle large volumes of traffic. Table 14 indicates that the most frequently mentioned attributes are a superior SA, the capacity to think and act rapidly, good planning and prioritization skills, and experience.

25

TABLE 14. SKILLS AND TECHNIQUES OF CONTROLLERS WHO EASILY HANDLE LARGE VOLUMES OF TRAFFIC Skill Superior SA Think, decide, and act quickly Good at planning and prioritizing Good communicators Experienced Number of Participants 34 21 15 12 10

Note. Expanded list of answers is presented in Table B9, Appendix B

Question 17 asked respondents to define what “bet on the [out]come” means to them and if they thought that experienced controllers and novices “bet on the [out]come” as often. The question allowed interviewers to ensure that respondents defined betting on the come as something close to not ensuring positive separation. Table 15 indicates that 15 respondents judged that experienced controllers “bet on the [out]come” more often than novices, that 20 thought that novices do it more often than experienced controllers, and finally, that 31 answered that the two groups do it to the same extent. The same table also shows that participants who chose the experienced controllers had 14.4 years of experience on average. Participants who said that there was no difference were more experienced than the previous group, with 17.9 years, but they were less experienced than those who chose the novices, who had 20.1 years of experience. These results suggest that the more experienced the participants, the more likely they were to believe that novices “bet on the [out]come” more often than experienced controllers. Conversely, it suggests that the less experienced controllers were, the more likely they were to report that experienced controllers “bet on the [out]come” more often. A Jonckheere-Terpstra test for ordered alternatives (Rossini, 1997) verified this trend by showing that the averages for these three groups occurred in that specific order (τ=.28, p=.004).3 TABLE 15. EXPERIENCE OF PARTICIPANTS ACCORDING TO THEIR OPINION ON WHICH GROUP OF CONTROLLERS “BET ON THE [OUT]COME” THE MOST OFTEN Experience (years) Group Experienced Novices Same (No difference) N 15 20 31 Mean 14.4 20.1 17.9 SD 5.3 5.9 5.8

3

The p-value for the Kendall statistic is equivalent to the two-tailed p-value for the Jonckheere statistic.

26

3.4 Decision Making and Planning When asked how much planning they have already done once the briefing is over and they assume their position (question 1b), many respondents suggested that the number of planned actions or “moves” depended on a variety of factors. For instance, some controllers indicated that more planning was required when one of the conditions listed in Table 16 was in effect. TABLE 16. CONDITIONS REQUIRING MORE PLANNING WHEN ASSUMING POSITION Condition Sector is busy (high volume) Pending conflicts or urgency IFR day Complex traffic or sector Fast sector Do not respect the relieved controller much Briefing is not easy Do not know the relieved controller Sequencing sectors Position other than local controller Many controllers have reported that, once they assume control of the position after the relief briefing, they already know what their first few clearances or “moves” will be. Figure 3 presents the number of planned actions reported by the 58 respondents who provided such estimates. Controllers suggested that they have already identified an average of 3.5 actions when they assume control of a position.
16 14 12 10 8 6 4 2 0 0 1 2 3 4 5 6 7 8 9 10 Number of moves

Figure 3. Number of actions planned by controllers before they assume control of position.

Number of participants

27

Interviewers also investigated what strategies controllers adopt when they are not sure if there will be a conflict (question 20). More specifically, participants had to indicate if they thought that it was sometimes better to wait and see how the situation developed or if they thought that it was always better to intervene immediately and resolve the issue. We see in Table 17 that 62 participants answered that it is sometimes preferable to wait and see, whereas 34 thought that it was always better to act immediately. The same table also indicates that only 8 terminal4 controllers out of 30 thought that it was sometimes better to wait and see, whereas a majority of ARTCC controllers, 54 out of 66, believed the same. Not surprisingly, a chi-square test confirmed that ARTCC controllers were more likely than terminal controllers to wait and see how the situation developed when they were not sure if there was a conflict (χ2=27.43, p