Airmanship in Australian Aviation

Christine Ellen Carrick, BSc (Hons) MSc

Thesis submitted for the degree of Doctor of Philosophy

(Aviation)

December 2015

i

Declaration

I hereby certify that the work embodied in this thesis is contains no

material which has been accepted for the award of any other degree or

diploma in any university or other tertiary institution and, to the best of my

knowledge and belief, contains no material previously published or written

by another person, except where due reference has been made in the text. I

give consent to the final version of my thesis being made available

worldwide when deposited in the University’s Digital Repository, subject to

the provisions of the Copyright Act 1968.

Christine Carrick

ii

Acknowledgements

I owe many thanks to my supervisor Dr Kerry Chalmers for her

guidance, patience and tenacity. My thanks also go to the organisations

that allowed me access to their personnel and to the many friends and

acquaintances who participated in or promoted my research.

I am also very grateful to my children, Luke and Ruth, and my

mother, Sheila (1925-2015), for their support and encouragement.

iii

Publication Notes

Excerpts and summaries from two chapters of this thesis have been presented

as conference papers, as shown below.

Carrick, K. (2009, June). Human factors as a component of airmanship. Paper presented at the 8th Industrial and Organisational Psychology Conference, Sydney, Australia.

Carrick, K. and Chalmers, K. (2010, April). Airmanship: Voices from an on-line survey. Paper presented at the 9th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.

Carrick, K. and Chalmers, K. (2010, April). Development of airmanship. Paper presented at the 9th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.

Carrick, K., Wahlberg, J., Wright, K. and Chalmers, K. (2014, August). Australian aviators’ conception of airmanship. Paper presented at the Massey Academic Forum of the 2014 PACDEFF, Auckland, New Zealand.

iv

v

Table of Contents

Declaration

Acknowledgements

Publication Notes

Table of Contents

List of Tables

List of Figures

Abstract

i

ii

iii

v

vii

ix

x

Chapter 1. Airmanship in Australian Aviation 1

Models of Airmanship 2

Training and Development of Airmanship in Australia 25

Current Research 29

Chapter 2. Airmanship: Definition and Development 33

Methodological considerations 34

Method 40

Results and Discussion 41

Conclusion 55

Chapter 3. Airmanship Across Industry Sectors 59

Method 62

Results - Definition of Airmanship 67

Conclusion – Definition of Airmanship 91

Results - Training and Development of Airmanship 93

Conclusion – Training and Development of Airmanship 128

Chapter 4. Current Airmanship Training and Assessment 131

Method 132

Results – Analysis of Demographics 135

Results – Definition of Airmanship 137

Discussion - Definition of Airmanship 156

Results - Training and Assessment of Airmanship 162

Discussion: Training of Airmanship 185

vi

Chapter 5. The Structure of Airmanship 193

Comparison with Previous Research 193

Model of Airmanship 195

Conclusion 200

Chapter 6. Airmanship Concept Domains 201

Methodological Considerations 201

Pilot Study 203

Method 203

Results 205

Discussion 207

Main Study 207

Method 208

Results 210

Discussion 218

Chapter 7. Airmanship Conclusions 221

Survey 1 222

Survey 2 224

Interviews with Current Instructors 229

Interviews: Training and Development of Airmanship 233

The Structure of Airmanship 237

Airmanship Concept Domain Testing 237

Definition of Airmanship 238

Training of Airmanship 240

Limitations of the Current Research 242

Future Research 242

Conclusion 243

References 247

Appendices 255

vii

List of Tables

Table Title Page Table 2.1 Number of Respondents (of a Total of 40) who Mentioned Each Concept in

Questions 1, 2 and Overall (Qs 1 to 11). 42

Table 2.2 Number of Respondents who Mentioned Each Concept in Questions 6, 9 and 10 (of N = 40).

47

Table 2.3 Number of Respondents who Mentioned Listed Scenarios or Phases of Flight in Questions 7, 9 or 10

49

Table 2.4 Number of Respondents Answering Question 3 49 Table 2.5 Number of Respondents (of total N=40) who Mentioned the Listed Concepts

in Response to Question 4b 51

Table 2.6 Number of Respondents (of a total N=40) who Mentioned the Listed Concepts in Response to Question 5

53

Table 2.7 Number of Respondents (of a total N=40) who Mentioned the Listed Concepts in Response to Question 8

54

Table 3.1 Comparison of Word Count across Industry Sectors 65 Table 3.2 Comparison of Hours of Experience and Years of Experience across

Industry Sector 66

Table 3.3 Percentage of Respondents who Mentioned Listed Concepts from Airmanship Models in Response to Combined Definition of Airmanship Questions

69

Table 3.4 Percentage of Respondents who Mentioned Listed Additional Concepts in Response to Combined Definition of Airmanship Questions

70

Table 3.5 Percentage of Respondents who Mentioned Listed Events or Phases of Flight in Response to Combined Definition of Airmanship Questions, by sector

71

Table 3.6 Percentage of Respondents who Mentioned Listed Concepts from Airmanship Models in Response to Combined Definition of Airmanship Questions

73

Table 3.7 Percentage of Respondents who Mentioned Listed Additional Concepts in Response to Combined Definition of Airmanship Questions

79

Table 3.8 Percentage of Respondents who Mentioned Listed Events or Phases of Flight in Response to Combined Definition of Airmanship Questions

83

Table 3.9 Number of Respondents in each Sector of Background and of Current Operation who Responded to Question 14 and Question 15

84

Table 3.10 Airmanship Concepts Ordered by Frequency of Mention in the Definition Questions, Rating in Question 14 and Ranking in Question 15

90

Table 3.11 Percentage of Civilian or Military Sectors Respondents by era of training (before 1991 or 1991 and after) Who Responded to Question 3(a)

94

Table 3.12 Percentage of Civilian or Military Sector Respondents by era of training (before 1991 or 1991 and after) who Mentioned the Listed Concepts From Airmanship Models in Response to Question 3(b)

95

Table 3.13 Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned the Listed Additional Concepts in Response to Question 3(b)

99

Table 3.14 Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Responded to Question 4

101

Table 3.15 Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned Concepts From the Airmanship Models in Response to Question 4

102

Table 3.16 Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned the Listed Additional Concepts in Response to Question 4

104

viii

Table Title Page Table 3.17 Percentage of Respondents in Each Sector who Mentioned the Listed

Factors in Response to Question 5 108

Table 3.18 Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 6

114

Table 3.19 Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 9

118

Table 3.20 Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 10

124

Table 4.1 Numbers of Instructors in Each Group, Experience in Aviation and Tertiary Degree Status

135

Table 4.2 Percentage of Participants in Each Group, and of Total Participants, who Commented on the Process of Definition

137

Table 4.3 Percentage of Participants in Each Group, and of Total Participants, who Commented on the Process of Definition

138

Table 4.4 Percentage of Participants in Each Group who Spoke of Listed Concepts in Their Initial Definition of Airmanship

140

Table 4.5 Percentage of Participants in Each Group who Mentioned Specific Definitions or Listed Concepts from the Airmanship Models During Interview

142

Table 4.6 Percentage of Participants in Each Group who Mentioned Listed Additional Concepts Relevant to the Definition of Airmanship

147

Table 4.7 Comparison of Means of Years of Experience by Mention of Common Sense 150 Table 4.8 Percentage of Participants in Each Group who Mentioned Listed Actions

and Considerations Relevant to the Definition of Airmanship 154

Table 4.9 Concepts Constituting Airmanship Ordered by Percentage of Participants who Mentioned each, from the Interview Data and from the Online Survey

160

Table 4.10 Percentage of Participants who Commented on Aspects of the Training Program

163

Table 4.11 Percentage of Participants in Each Group and Overall who Mentioned Inclusion of Listed Airmanship Items in Initial Flight Training

171

Table 4.12 Percentage of Participants who Commented on Listed Aspects of the Training Process

177

Table 6.1 Domains and representative concepts 204 Table 6.2 Mean relatedness between pairs of concepts in the pilot data grouped by

domain 206

Table 6.3 Mean relatedness for forward and reverse worded pairs and outcome of Wilcoxen Signed Ranks (WSR) tests and Pearson correlation for each pair

212

Table 6.4 Mean relatedness between pairs of concepts grouped by domain 204

ix

List of Figures

Figure Title Page

Figure 1.1 The Kern model of airmanship. An adaptation from Redefining Airmanship T. Kern, 1996, McGraw-Hill p. 22

3

Figure 1.2 The Kern model of airmanship. An adaptation from a diagram in Kern (2009a) CASA threat and error management roadshow (wrap-up power-point presentation, slide 11).

4

Figure 1.3 Schematic representation of the elements of airmanship in the Ebbage and Spencer model. From "Airmanship training for modern aircrew", by L Ebbage and P. D. Spencer, 2003

9

Figure 1.4 Threat and Error Management Model from "Error management training: Defining best practice” by M.J.W. Thomas (2005). ATSB

15

Figure 1.5 Mismatch emergency model from "Error management training: Defining best practice” by M.J.W. Thomas (2005). ATSB

16

Figure 3.1 The rank of mean ratings for the 26 concepts rated in Question 14

85

Figure 3.2 The mean rankings for the 26 concepts ranked in Question 15

87

Figure 5.1 A revised airmanship model 196

Figure 6.1 Scree plot of stress values for one to seven dimension solutions

216

Figure 6.2 MDS 2-dimensional plot 216

Figure 6.3 MDS 3-dimensional plot at 20 degrees of rotation 217

Figure 6.4 MDS 3-dimensional plot at 60 degrees of rotation 217

Figure 6.5 Revised model of airmanship 220

x

Abstract

The current project involved investigation into the understanding of airmanship among

Australian aviators and the way in which airmanship is trained in Australia. The starting

point was Kern’s model of airmanship (1996, 2009a), developed from research with aviators

in the USA. The current research revealed two insufficiencies in the Kern model. First that it

includes few of the non-technical skills that become important as an aviation career

progresses, and secondly it emphasises knowledge over application. The Ebbage and

Spencer (2003) model of airmanship, developed in the UK, includes non-technical skills

more overtly and some additional concepts not included in the Kern model. Both Kern and

Ebbage and Spencer recommend that training of airmanship should include three phases: 1)

instil the importance of airmanship; 2) overtly teach and model airmanship; and 3) assess

and provide feedback.

In the current project, an initial on-line survey reached mainly general aviation pilots.

The participants mentioned many of the concepts included in the existing models but also

some additional concepts. A second survey reached more participants and enabled

comparison of differences in views between military and civilian background aviators. It also

established the relative importance placed by the participants on the component concepts of

airmanship. A third study involved the interviewing of a cross-section of flight instructors

and trainers from civilian and military sectors, at ab-initio and advanced levels of training. It

appears that the training of airmanship generally meets the final two phases of the suggested

training process, but there seemed to be a lack of formal introduction to airmanship and no

use of a structure to facilitate development of airmanship. A model of airmanship was

developed and its structure tested empirically. A revised model is presented, which provides

a more balanced approach to the importance of the component concepts than earlier models

and also recognises the influence of context on the development and expression of

airmanship. The use of the revised model to expressly embed airmanship in training

programs may alleviate the paradox of the use of the term ‘airmanship’ both as a global

expression of safe and efficient flight and also to describe only the non-technical skills

component, in some training environments.

1

Chapter 1. Airmanship in Australian Aviation

In the last hundred years aviation has moved from being an experimental

adventure to being an accepted, even expected, form of transportation, and an integral

part of military capability. In the process, aviation has also become a closely regulated

activity. The International Civil Aviation Organisation (ICAO), a branch of the United

Nations, coordinates the aviation regulations of signatory nations. In each nation, the

Air Navigation Act (or similar legislation) determines what aviators can and cannot do.

The process of gaining a pilot’s licence requires passing several exams on

aerodynamics, navigation, meteorology, aircraft systems and human factors, while the

practical side of learning to fly is also governed by a regulator-set syllabus and

assessment procedure. Once qualified, the pilot’s flying activity is subject to standard

operating procedures (SOPs) set by the manufacturer of the aircraft flown or by the

operating organisation, and regular training and assessment activities are required to

maintain a licence.

It is the human pilot who is ultimately responsible for flying the aircraft safely,

and despite the regimentation and regulation, there is still room for variation in the

operation of the aircraft. The literature concerned with aviation accident analysis

frequently refers to the idea that anywhere upwards from 70% of aircraft accidents

involve some human factor among the causes (Helmreich & Foushee, 1993). On the

other hand, there has been more recent recognition that of all the flights on which

something unexpected has happened, of those that land safely or are survivable, 100%

are due to the intervention of a human on the flight deck (Reason, 2008). Generally the

media will tell us that good airmanship has saved the day. What is good airmanship?

There are many definitions. James Reason considers airmanship to be professionalism,

and defines it further.

Let me begin by teasing out what I understand by “professionalism’. It is a fuzzy-edged term embracing a number of different qualities. Aviators call it airmanship; mariners call it seamanship. These labels describe abilities that go well beyond the competent deployment of technical skills. They imply the capacity to see the broader picture, to think ahead and to draw upon a wide range of knowledge and experience so as to perform demanding work safely, elegantly and effectively. It means having a deep understanding of all the various factors that can impact upon task performance for good or ill. It also

2

entails a willingness to engage in all aspects of the job – tedious or otherwise – to the best of one’s ability. (Reason, 2008, p. 161)

Airmanship is defined by ICAO as “the consistent use of good judgement and

well developed skills to accomplish flight objectives” (ICAO, 2006, cited in CASA,

2008). Australian military pilot trainees are taught that it is “the safe and efficient

handling of the aircraft on the ground and in the air” (BAE Systems, 2006, slide 8).

Edwards (2013) condenses a variety of ideas to “the (technical, operational and non-

technical) knowledge, skills and attitudes aircrew employ to operate an aircraft

effectively, efficiently and safely” (p. 13). Certainly good airmanship is considered an

essential quality for aviators (Ebbage & Spencer, 2003), however it can be a rather

elusive concept to pin down (Kern, 1996). The most extensive attempt to bring order to

the definition has been the model of airmanship developed by Kern (1996).

Models of Airmanship

While there are many definitions, opinions and discussions of airmanship on

public forums such as aviation magazines and various web-pages, there are just three

published models presenting a solid formulation of what constitutes airmanship. These

are the books by Kern (1996, 1997 and 1999); a paper presented at a military flight

training conference by Ebbage and Spencer (2003) and the book by Edwards (2013). A

brief description of each of these models follows. There are also several models of

human performance in aviation environments that reflect some aspects of airmanship,

which will be discussed in a further section, below.

Kern (1996, 2009a) Model of Airmanship

Overview. Kern (1996) presented a model of airmanship represented as a building

façade, based on the foundation or bedrock principles of “skill, proficiency and the

discipline to apply them in a safe and efficient manner” (p. 21). Kern (1996)

summarised his model as:

Airmanship is the consistent use of good judgement and well-developed skills to accomplish flight objectives. This consistency is founded on a cornerstone of uncompromising flight discipline and developed through systematic skill acquisition and proficiency. A high state of situational awareness completes the airmanship picture and is obtained through knowledge of one’s self, aircraft, team, environment and risk (p. 22).

3

These foundation or bedrock principles of discipline, skill and proficiency, with

the pillars of knowledge, support the capstone outcomes of situational awareness and

good judgment in decision making, leading to superior airmanship. A diagrammatic

representation of the model is presented Figure 1.1 (from Kern, 1996). The goals of

airmanship are stated as “safety, mission effectiveness and efficiency” (Kern, 1996, p.

254). Rather than thinking of these components of the model as separate, Kern (1999)

presents the idea that they should be a seamless whole that informs and directs the

conduct of the aviator at all times.

Figure 1.1 The Kern model of airmanship. An adaptation from Redefining Airmanship T. Kern, 1996, McGraw-Hill p. 22.

Bedrock Principles. Flight discipline is seen as the support or backbone for

airmanship (Kern, 1997). Kern (1996) describes discipline as the ability and willpower

to fly safely “within operational, regulatory, organisational and common-sense

guidelines” (p. 29). Kern (1997) devotes an entire book to the importance of flight

discipline, and upgraded discipline to a separate sub-heading, “cornerstone” more

recently (2009a). In writing for a more general audience (rather than just aviation) Kern

(2009b) suggests that there is an individual responsibility for preventing personal error,

predominantly through self-discipline, or empowered accountability.

Skill and proficiency are shown as separate foundations in Kern’s (1996) model,

but are discussed together. Skill is considered to be the ability to handle the aircraft

generally. According to Kern, skill should build up with experience from the minimum

safe qualification through the levels of effectiveness and efficiency, culminating in an

4

attitude of precision and improvement wherein the aviator is aiming to improve with

each flight. Kern (1996) also points out that many aviators are happy to sit at the

effective or efficient levels and never progress, thus demonstrating a lower level of

airmanship. Proficiency is presented as personal assessment of the currency and

recency of flying skills, with good airmanship including a personal responsibility to

ensure that these are not degraded by lack of practice. Kern mentions in passing that as

well as the obvious flying skill, skills in communication, decision making, teamwork

and self-assessment are also needed for good airmanship, however these are not

elaborated upon in the discussion of skill and proficiency.

Pillars of Knowledge. Kern’s (1996) original model included five pillars of

knowledge, knowledge of one’s self, the team, the aircraft, the environment and risk

associated with the flight. An additional pillar was added more recently (Kern 2009a)

representing knowledge of the mission (see Figure 1.2).

Figure 1.2 The Kern model of airmanship. An adaptation from a diagram in Kern (2009a) CASA threat and error management roadshow (wrap-up power-point presentation, slide 11).

Discussion of knowledge of one’s self includes physiological airworthiness, such

as the effects of fatigue, illness, stress and nutrition, and also considers hazardous

attitudes. The five main hazardous attitudes are anti-authority, impulsivity,

invulnerability, machismo (or showing-off) and resignation. A strong desire to get to the

planned destination despite developing problems, known as “get-there-itis”, is also

5

considered a hazardous state (Trollip & Jensen, 1991). Kern adds some further

hazardous attitudes: complacency, emotional jet-lag (or over-perfectionism and

dwelling on errors), excessive professional deference and passenger/co-pilot syndrome

(the idea that the other pilot will pick up any errors).

Kern (1996) returns to the theme of individual responsibility in discussing

knowledge of the team: he considers that one should be sufficiently competent to

contribute effectively to the team. Crew resource management (CRM) is discussed, with

the comment that “airmanship is the individual structure on which CRM builds” (Kern,

1996, p. 139). Kern notes that many of the topics included in CRM training programs

are elements in the airmanship model, with a list of eight basic inclusions in CRM

training being situational awareness, group dynamics, communication, risk

management, decision making, stress awareness and management, mission planning,

review and critique, and physiology and human performance. Leadership and

followership are also considered as part of this “team” pillar.

Under knowledge of the aircraft, Kern (1996) states that there should be a

thorough understanding of the aircraft type being flown, including its subsystems,

limitations and emergency procedures. It is emphasised that this knowledge should be

continuously rehearsed and updated. Kern notes that discussions with other pilots of the

same aircraft type will bring to light idiosyncrasies that may not appear in the flight

manuals.

Knowledge of the environment includes the physical, regulatory and

organisational environments. The physical environment includes terrain, meteorology,

light, and airspace. The regulatory environment includes everything in the regulations,

including air traffic control regulation. The organisation’s environment includes an

awareness of corporate culture and challenges to airmanship that may arise with

conflicting demands (e.g. depart on time vs. checking a minor malfunction). Kern

(1996) does not draw attention to Standard Operating Procedures and other

organisational policies, procedures or practices that are part of the organisational

environment.

Kern (1996) places importance upon the knowledge of risk and the effect of risk

upon the various components of the model. Kern defines risk as “the probability and

severity of a loss linked to a hazard” (p. 202) and a hazard is any event, or item, that

6

affects a flight with the potential for damage, injury or delay. There is no flying without

some risk, but the risks should be recognised and managed in a display of good

airmanship.

A more recent addition to the model (Kern, 2009a) is the idea that the knowledge

of the mission forms a further support pillar for good airmanship. This seems logical,

knowing what each flight will involve, including preparation, flight planning and

obtaining weather forecasts, is clearly going to assist in developing and maintaining

situational awareness and safety, and thus is good airmanship. Figure 1.2 shows the

Kern model of airmanship presented in 2009 (Kern 2009a) including the increased

emphasis on discipline and the additional pillar for knowledge of the mission.

Capstone Outcomes. Kern (1996) places situational awareness feeding into good

judgement at the apex of his model, as “capstone outcomes”. A simple definition of

situational awareness (SA) is “What happened previously, what is happening now, and

what will happen in the future” (Sarter & Woods, 1991). A more comprehensive

definition is “the perception of the elements in the environments within a volume of

time and space, the comprehension of their meaning and the projection of their status in

the near future” (Endsley, 1988, cited in Endsley, 1999, p. 258). Kern interprets and

discusses these three levels of SA, being perception, comprehension, and projection. He

also reports a survey of F15 fighter pilots (Wagg, 1993, cited in Kern, 1996) who

reported seven elements crucial to good SA: building the big picture; integration of

information from multiple sources; keeping track of spatial position and relationship

with other aircraft; keeping up with changes in the situation; prioritizing tasks and data;

making good, timely decisions; and prediction into the future.

Kern (1996) emphasises the importance of recognising when SA has been lost and

of taking steps to remain safe while trying to recover SA. In aviation, good SA is

sometimes referred to as “staying ahead of the aircraft” while lack of SA is considered

“being behind the aircraft”. Once it is recognised that one is being “left behind”,

catching up again becomes a priority.

Keeping track of multiple variables in a dynamic environment, the essential task

of SA, is challenging but essential for making decisions, hence its central importance in

airmanship. Judgement – or the ability to make sound decisions – is ultimately how

flights remain safe, effective and efficient.

7

Kern (1996) takes the view that judgement is an integral part of every action an

aviator takes, in the air and on the ground, and relies upon the interaction and

interdependence of all the aspects of airmanship discussed thus far. Judgement and

decision making are treated as synonyms and both of the terms are used in his

discussion. A concern is expressed by Kern (1996) that increased use of checklists and

procedures while leading to more predictable decisions and safer flight, may actually

mean that pilots are overly reliant upon automation and manuals, reducing their ability

to take independent and timely action when it is called for. Contrary to what is

expressed as common opinion (that one intrinsically has good judgement or one

doesn’t) good judgement is presented as an acquired skill that can be systematically

developed. Kern’s discussion of the decision making process appears to be based on the

models of naturalistic decision making and concentrates on assessing the situation and

determining alternatives (Kampf & Klein, 1994).

Integration of model components. Kern (1996) considers airmanship as an

integrated holistic notion that encompasses the elements described above. To be

effective pilots must be ready to utilise their airmanship whenever the opportunity to

demonstrate it occurs, whether that is handling a regular routine flight or coping with

disaster and pulling off what Reason (2008) terms a “heroic rescue”. Kern presents

Captain Al Haynes’ leadership of the crew of United Flight 232 and the crash landing at

Sioux City in 1989 as one such heroic rescue. Flight deck crew coordination and careful

aircraft handling overcame loss of all control hydraulics and, despite the aircraft

breaking up on landing, most of the passengers and crew survived. A more recent

example of such an integration of skill, knowledge, situational awareness and

judgement is Captain “Sully” Sullenberger’s ditching of US Airways Flight 1549 in the

Hudson River on 15 January, 2009, with no casualties, after multiple bird-strikes led to

the loss of both engines just after take-off. In April, 2009 Sullenberger was awarded the

Jabara Award for Airmanship by his alma mater the USAF Academy. He later wrote

On January 14, 2009, my life had been a series of thoughtful opportunities to be the best pilot, leader, and teammate I could be. I was an anonymous regular guy... On January 15, circumstances changed everything, a reminder that none of us ever knows what tomorrow will bring. I flew thousands of flights in the last forty-two years, but my entire career is now being judged by how I performed on one of them. This has been a reminder to me: We need to try to do the right thing every time, to perform at our best, because we never know which

8

moment in our lives we’ll be judged on (Sullenberger & Zaslow, 2009, p. 313-314).

Captain Richard de Crespigny also commented on teamwork and excellence in

crew performance. He led the crew aboard the “Nancy-Bird Walton”, Qantas Flight 32

out of Singapore on 4 November, 2010. An uncontained engine explosion on climb

destroyed one engine, put holes through the wing and fuel tanks leading to fuel leaks,

severed many control and information conduits leaving the remaining engines degraded,

and peppered the fuselage with shrapnel. After the event, he wrote:

So from my perspective and having made it through the toughest challenge of my professional life with no injuries or deaths, I can honestly say that everyone on board was lucky that they had a crew full of knowledgeable, case-hardened, experienced pilots who worked effectively as a team as we wrestled to keep Nancy-Bird Walton in the sky and then bring her back to earth... You get a first officer like [name] because the airline recognises his role as a professional, as a “safety and profit” centre not a “cost centre”, and therefore actively develops his excellence... The QF32 was a great flight, but we were only one of thousands of flights that take off every day, each of which are manned with excellent crews (de Crespigny, 2012, p. 332-3).

It appears that both Captains agree with Kern, that the integration of skill,

knowledge and judgement – good airmanship – brought off the almost impossible

“heroic rescue” that an unusual circumstance had presented them with. They also imply

that the combination of factors, airmanship, did not come about by chance. Both

extracts indicate a process of development and experience that resulted in a crew ready

to handle the emergency. Kern (1996) after defining and describing airmanship, goes on

to present a method for continuous development towards excellence in airmanship,

discussed below under the heading training and development of airmanship.

Ebbage and Spencer Model of Airmanship

Ebbage and Spencer (2003) present a model of airmanship based on their work

with military ab-initio training in Britain, and acknowledge the influence of the Kern

(1996) model in the structure they present. They define airmanship as:

A personal state that enables aircrew to exercise sound judgement, display uncompromising flight discipline and demonstrate skilful control of an aircraft and a situation. It is maintained by continuous self-improvement and a desire to perform optimally at all times (p. 2).

A schematic representation of the model is presented in Figure 1.3. In this model,

the three central facets of good airmanship are judgement, control and discipline.

9

Judgement is presented as a product of situational awareness, problem solving, mental

workload and foresight. Control of the aircraft is required for safe flight and is shown as

developing from flying skills, automation and information management, navigation and

communication skills. Discipline is indicated as arising from self-improvement,

vigilance, cooperation and confidence. Ebbage and Spencer (2003) state that discipline

is needed to detect errors early and ensure that judgements are considered and actions

controlled.

Ebbage and Spencer (2003) suggest that airmanship is built on a foundation of

knowledge, skills and attitudes. They present these foundation requirements as a table

of syllabus items for the training of airmanship.

Knowledge. Ebbage and Spencer (2003) refer to three types of knowledge, that of

the aircraft, the environment and risk. Aircraft knowledge is specified as including

thorough familiarity with the aircraft systems, approved procedures, flight

characteristics and limitations of the aircraft, and with automation. Knowledge of the

environment includes the same three sub-categories as Kern (1996), the physical,

regulatory and organisational environments. Physical environmental features, such as

weather and terrain, are considered in terms of their effects on aircraft control. The

Figure 1.3 Schematic representation of the elements of airmanship in the Ebbage and Spencer model. From "Airmanship training for modern aircrew", by L. Ebbage and P. D. Spencer, 2003.

10

regulatory and organisational environments are presented as possibly posing challenges

to airmanship. An understanding of the risks that may occur is the third part of the

knowledge foundation.

Skills. Ebbage and Spencer (2003) list a variety of skills that they consider central

to good airmanship and that should be developed in military flight training courses.

They group these skills into five categories: physical skills; cockpit management;

communication; cognitive skills and team skills. Flying skills, including instrument

flying and emergency handling and recovery, navigation skills and combat survival are

listed as required physical skills. Cockpit management includes information

management and specific mention of “avoiding the pitfalls of automation (over-reliance,

complacency, bias)” (Ebbage & Spencer, 2003, p. 6). Communication skills are spelt

out to include vigilant monitoring, active listening, appropriate, clear and concise

phraseology and inquiry. Unlike Kern (1996), Ebbage and Spencer include situational

awareness and decision making as cognitive skills, rather than as outcomes. They also

include problem solving, workload handling and self-assessment as cognitive skills.

Finally, team skills cover performance monitoring, leadership and initiative,

interpersonal skills, coordination and decision making, and team communication and

SA. There is some overlap between cognitive and team skills, perhaps in order to

emphasise that these are as important in single-pilot operations as multi-crew, and that

in multi-crew (team) environments additional attention is needed to ensure that all team-

members are “in the loop” and aware of what is happening.

Attitudes. Within the consideration of the development of appropriate attitudes,

Ebbage and Spencer (2003) give prominence to an understanding of hazardous attitudes.

These were mentioned briefly in Kern (1996) as part of “knowledge of self” and are

based on the hazardous attitudes discussed by Trollip & Jensen(1991). The idea is that if

trainee pilots are exposed to the sorts of attitudes that can lead to dangerous behaviour,

and their antidotes (sayings used to defuse a hazardous thought), then they are less

likely to indulge in or be led astray by them. Thus, awareness of the impact of such

hazardous attitudes on airmanship and safety is a key part of the training regime that

Ebbage and Spencer (2003) propose. On a more positive note, they include the

development of a sense of professionalism, as being “understanding the values and

principles embodied in airmanship” (p. 6), and an attitude of self-improvement,

11

including the motivation for life-long learning, understanding of the need for self-

assessment and the will to develop excellence.

Finally, Ebbage and Spencer (2003) consider discipline to be an attitude that

should be fostered and exercised in terms of ongoing maintenance of knowledge and

skills, and of self-discipline (seen as managing one’s own attitudes and stress). They

also consider discipline in terms of various aspects of flying activity such as

preparation, vigilance and lookout, maintenance of situational awareness, and keeping

within operational and regulatory policy. Post-flight evaluation is included as an area of

application of discipline, perhaps in recognition that once flight training is completed

(with its formal post-sortie debriefings) such evaluation might be skimped and yet can

be a very useful reflection.

Edwards’ Model of Airmanship

Edwards (2013) defines airmanship as the technical, operational and nontechnical

knowledge, skills and attitudes required to fly safely, effectively and efficiently. He

presents a discussion of airmanship (as opposed to a formal model) in four main

sections: technical elements, operational elements, non-technical elements and other

factors. Each of the elements is described in terms of the knowledge, skills and attitudes

required to exhibit good airmanship.

Technical elements. Edwards (2013) considers these to be knowledge about the

aircraft flown, the skills to fly accurately and smoothly and an attitude of study, practice

and self-discipline.

Operational elements. Knowledge of the people, systems and processes related

to the job are the first part of the operational elements. Operational skills include

forming and executing a plan of action, time management, and awareness of what part

of the plan requires implementation at which time. Operational attitudes embrace a

balance of safety and getting the job done, self-discipline, preparation, thinking ahead,

and not making assumptions.

Non-technical elements. Edwards notes that this section includes human factors,

of which crew resource management is a subset. Non-technical knowledge covers the

psychological and physiological operation of humans in the aviation environment. Non-

technical skills are discussed as a set of standards of performance covering

12

communication, leadership and teamwork, workload management, situational

awareness, and problem solving and decision making. Non-technical attitudes are noted

as overlapping those of the other sections, but including in addition: self-respect and

confidence; respect for others and tolerance; willingness to learn from experience;

integrity and honesty; and being positive and ensuring success rather than looking for

excuses for failure.

Other factors. These are aspects that are not included in the three main elements

but awareness of which Edwards (2013) notes as being important for good airmanship.

The factors discussed include stress; fatigue; hazardous attitudes; culture; automation;

monitoring; and threat and error management. Most of these could be considered

covered by the ‘knowledge’ sections in each element of airmanship.

Comparison of the Airmanship Models

Ebbage and Spencer (2003) acknowledge the influence of Kern (1996) on the

formulation of their ideas about airmanship and therefore there are many similarities

between the two models, but there are also some differences. Both the Kern and the

Ebbage and Spencer models of airmanship described above have their roots in the

military environment, with Kern using many military case studies as well as civilian,

and Ebbage and Spencer specifically addressing military flight training. This may have

influenced the emphasis on discipline in both these models. Ebbage and Spencer list

areas in which discipline should be applied, while Kern develops the concept of flight

discipline much further, and expands upon it with the discussion of rogue pilots who

eschew discipline and the consequences of such behaviour (Kern, 1999). Edwards

(2013) acknowledges his own background in military aviation but uses exclusively civil

aviation examples in his discussion of airmanship. Self-discipline is mentioned as a

desirable attitude but is not over-emphasised.

Compared with the Kern (1996, 2009a) model, the Ebbage and Spencer (2003)

and the Edwards (2013) models have a more comprehensive list of the skills required

for good airmanship, drawing more attention to communication and team skills than

does Kern. Edwards (2013) and Ebbage and Spencer (2003) include situational

awareness and decision making as skills, while Kern (1996) discusses them as outcomes

of the exercise of skill and use of knowledge.

13

Kern (1996), Ebbage and Spencer (2003) and Edwards (2013) agree that

airmanship includes a thorough understanding of the aircraft type flown, its subsystems,

limitations, emergency procedures and so on. Both the Kern and Ebbage and Spencer

models include knowledge of the physical, regulatory and organisational environments,

however neither of them draw attention specifically to Standard Operating Procedures

(SOPs) and other organisational policies, procedures or practices. In contrast, Edwards

discusses operational elements of aviation in a general sense, without mention SOPs

specifically, except in terms of exercising self-discipline to conform to procedures and

checklists “even when there doesn’t seem to be the need” (p 33).

Kern (1996) includes in the knowledge pillars, knowledge of one’s self and of the

team, while Ebbage and Spencer cover these areas with their comments on team skills

and with self-assessment included as part of cognitive skills, and Edwards (2013)

discusses human factors and teamwork quite specifically under non-technical elements.

All three models consider awareness of the same hazardous attitudes, Ebbage and

Spencer include these more prominently as part of an attitudes grouping, while Kern

includes these and some others within the pillar for knowledge of the self, and Edwards

discusses these among his ‘other factors’. Both Kern and Ebbage and Spencer consider

the knowledge of risk and the effect it has upon the various components of their models,

while Edwards does not, except insofar as threat and error management is covered. The

Kern model (2009a) includes knowledge of the mission, but with no elaboration, while

the Ebbage and Spencer model simply includes “preparation” as an area in which

discipline should be applied. In Edwards this is covered among the operational skills, in

developing, organising and implementing a plan for the flight.

The Kern and the Ebbage and Spencer models place importance on situational

awareness as a precursor to good judgement. Kern places these at the apex of his model,

as capstone outcomes, while Ebbage and Spencer place judgement as one of the three

main elements of airmanship, depending upon situational awareness, problem solving,

mental workload and foresight. Edwards includes situational awareness and decision

making as two of his six non-technical standards.

Compared with Kern (1996), both Ebbage and Spencer (2003) and Edwards

(2013) put more emphasis on attitudes, including hazardous attitudes. They also list

more prominently skills of communication, crew co-ordination, interpersonal skills,

management within the flight-deck or cockpit, self-assessment and team leadership. In

14

addition, Ebbage and Spencer emphasise professionalism and self-improvement as part

of airmanship, which the other two approaches do not, but do seem to assume.

None of these models present any empirical evidence for the structure of the

model or the concepts included within them. Nor has there been any formal evaluation

of the models against the views of current pilots.

Other Models Related to Airmanship

In addition to the models of airmanship discussed above, there are a number of

other models of aviation related human performance that are relevant to the general

discussion of airmanship.

The end goal of airmanship is to maintain safety at all times. Operating safely can

be seen either as part of airmanship or as the outcome of good airmanship. In the

announcement of the release of the Guild of Air Pilots and Air Navigators (GAPAN)

threat and error management (TEM) program for General Aviation and regional airlines

it was suggested that the TEM model is a framework for airmanship, and that TEM

training “brings airmanship training into the 21st century” (Elms, 2007, p. 1). TEM

training addresses the safe conduct of flight and thus follows the intent of the broadest

definitions of airmanship. TEM was first proposed by Helmreich, Klinect and Wilhelm

(1999) as part of the revision of crew resource management (CRM) training for airline

pilots, and is a return to the original intent of CRM, this being error handling and crew

interaction in order to avoid accidents (Helmreich, Merritt & Wilhelm, 1999). However,

TEM is also relevant to single pilot operations and this was the basis for the GAPAN

program. The TEM model provides a framework that demonstrates that management

actions by the flight crew in response to external threats and crew errors can render the

hazards arising from those threats and errors inconsequential, however, mis-

management can lead to an incident or accident. The relationship between these

elements is shown in Fig 1.4. The model has been found to be useful for aircrew in

presenting a way of thinking about flight-deck management, such that ICAO has

mandated its inclusion in ongoing flight crew training. (CASA, 2008).

15

Figure 1.4 Threat and Error Management Model from "Error management training: Defining best practice” by M. J. W. Thomas (2005). Australian Transportation Safety Bureau. Copyright 2005 ATSB.

The model can also be used for post hoc analysis of the kinds of threats or errors

that occurred in a flight and to draw attention to the skills and actions that led (or would

have led) to a safe conclusion to the flight. Helmreich (2002) lists the core CRM

behaviours for addressing TEM as being: “active Captain leadership; briefing known

threats; asking questions, speaking up; decisions made and reviewed; operational plans

clearly communicated; preparing/planning for threats; distributing workload and tasks;

and vigilance through monitoring and challenging” (p. 8).

The model itself does not address specifically the recognition of threats and

errors, nor the actions that need to be taken to manage them. This lack is addressed in

Thomas’s (2005) “mismatch emergence model” (see Fig 1.5). This model indicates the

balance between situational awareness of the actual state of the system and the mental

model of the desired state of the system. A mismatch between the two starts from a

feeling that “something is wrong” or recognition of an abnormal state, threat or error. It

is suggested that an aviator who is ready to deal with threats and errors has good

situational awareness as well as an accurate mental model of the desired state of the

system (based on a good knowledge of the aircraft and the operating environment). In

addition, the pilot must be able to develop a dynamic plan to achieve a safe outcome.

The “mismatch emergence model” is presented as an approach to the training of threat

16

and error management, thus providing for threat or error recognition, missing from the

TEM model.

Figure 1.5 Mismatch emergency model from "Error management training: Defining best practice” by M. J. W. Thomas (2005). Australian Transportation Safety Bureau. Copyright 2005 ATSB.

Thomas (2005) summarises the non-technical skills and generic error-

management strategies that are required to detect and effectively manage threats and

errors as being cognitive skills such as situation awareness, vigilance and monitoring,

self-regulation and problem identification, and interpersonal skills including

communication, negotiation, assertiveness and task management. The parallel with

airmanship is seen in the central requirement for situational awareness, and an

assumption that skill and proficiency is present in order to effectively correct and

recover, and self-regulation could be considered a form of discipline. The inter-personal

skills of communication and task-management were also emphasised in Ebbage and

Spencer (2003) and Edwards (2013), although assertiveness and negotiation are not

mentioned in the two main models of airmanship.

Mavin (2010) proposes a model for assessment of pilot performance (MAPP) that

has very similar components to the Kern (1996, 2009a) Edwards (2013) and Ebbage and

Spencer (2003) models of airmanship. Mavin reports asking simulator instructors what

qualities they were most concerned about in the captaincy upgrade training of airline

pilots. In other words, what is it that makes a pilot competent to move from First Officer

to the rank of Captain, with its associated responsibility and authority. The outcome of

this research was the MAPP model which presents three essential skills that pilots must

have: (1) situational awareness; (2) decision making and consideration of risk; and (3)

the ability to maintain the aircraft within acceptable tolerances. These essential skills are

supported by the enabling skills of aviation knowledge, management (of workload,

threat and error), and communication. Mavin and Roth (2014) represent these elements

17

as ‘six discursive repertoires’ (p. 215) for the discussion of pilot performance without

dividing them into essential versus enabling skills. Mavin’s model, like Ebbage and

Spencer (2003), places more emphasis on cockpit management and communication,

than does the Kern model of airmanship. Mavin (2010) does not specify in detail the

areas of knowledge required. Discipline is not mentioned, nor other personal qualities or

attitudes that the other models include, perhaps because these are not as relevant to

simulator instructors making decisions about upgrade training. The central importance

of situational awareness, decision making (or judgement) and aircraft handling skills

parallel the airmanship models very closely.

Airmanship Research

There has been little direct research on airmanship as a holistic concept, although

there is a great deal of research about many of the individual components, such as

situational awareness, decision making or CRM issues. However, there is some research

on what might be considered a “good pilot”, which directly relates to good airmanship.

Swauger (2011) reported on a survey conducted among South West Airlines First

Officers concerning what they expected of good Captains. When the results of this

survey were distilled down to a single descriptive sentence, it can be said that First

Officers value a Captain who is technically competent, psychologically confident, and

promotes an effective CRM environment. In a more detailed report of the study,

(Swauger, June, 2006) it was reported that 180 First Officers returned the surveys

(about a 7% response rate). Sixteen main categories of comment were reported in three

tiers based on how frequently the comments were made. The top three areas of

comment were: establishment of a good CRM environment (empowering and allowing

contribution); personality (friendly, relaxed, balanced) and technical expertise (good

book and aircraft knowledge, briefings and standardization). Second most frequently

mentioned characteristics were: instructional abilities (good teachers); flying skills

(precise, smooth and accurate); easy-going nature (unflappable, able to defuse stressful

events); and professional (without further definition). The final tier of characteristics

included elements that were mentioned less often, but were still considered important:

team building; open communication; company asset (considered to have a selfless

approach towards promoting company success, good ambassadors); flight management

(organising the flow of the flight); sense of humour; mentoring (passing on accumulated

18

wisdom, rather than just technical training); situational awareness; decision making; and

experience.

There are obvious commonalities between the characteristics reported in Swauger

(2006) and the airmanship models of Kern (1996, 2009a), Ebbage and Spencer (2003)

and Edwards (2013), although clearly the characteristics are not ranked in the same

order of importance as implied in these models. In Swauger, situational awareness and

decision making were mentioned least often of those characteristics listed while

knowledge (or at least aircraft knowledge) was considered of primary importance, along

with active promotion of CRM, team building and communication. Flying skills were

also reported as part of a good Captain, at the second tier of reported frequency.

Discipline was not directly mentioned by Swauger’s survey participants, although this

could have been implied in the mention of professionalism. Additional concepts,

mentioned in this survey but not directly included in the airmanship models, are the

mention of personality and nature, sense of humour, teaching and mentoring.

A slightly different selection of concepts were reported by Nergård, Hatlevik,

Martinussen and Lervåg (2011), following questioning of pilots’ views of desirable

personal attitudes in a series of interviews (15 group and 30 individual interviews for a

total of 250 pilots) held during or just after CRM courses at two Scandinavian airlines.

The pilots were asked to discuss and define a good pilot. All groups used the term

‘airmanship’ and commented on ‘the attitudes of a good airman’. They were then asked

to define airmanship and attitude. Nergård et al report that there was no single definition

for airmanship, for a good pilot or for the abilities or qualities of a good pilot. Skills and

abilities were seen by the participants to depend upon the kind of operation in which the

pilots were engaged. Good communication skills were also seen as important. A good

attitude was reported as being reflected in acceptance within the pilot community,

acting and socialising appropriately within an unwritten code of conduct. Being a good

crew-member was mentioned, and included knowing how to work with others and

exhibiting cooperation and interaction. Humility was discussed in terms of recognising

one’s own errors and helping others with problems. Self-awareness, self-efficacy and

self-esteem were also included as features of a good pilot. Finally, learning to fly the

aircraft using CRM was regarded as essential. Nergård (2014) reported on an additional

20 interviews, as well as those reported earlier, and included additional comment on the

ability to fly the aircraft being central to airmanship. The ‘rationale’ for airmanship was

19

reported as a strong awareness that the pilot’s actions create safety, and that one can die

from making mistakes.

Nergård et al (2011) report a second study that involved 174 participants

completing a questionnaire developed from the statements evoked in the first study. A

principle components analysis extracted four factors (explaining 60% of the variance):

principles in airmanship knowledge, self-awareness, flying skills and CRM. The

airmanship factor included thinking ahead of one’s actions, awareness of aircraft

limitations, initiative to improve, monitoring for fatigue and stress, being a good role

model, applying good airmanship, knowing rules and regulations, awareness of own

errors and limitation and helping others. The latter two also loaded on the CRM factor,

which included statements about cooperation and interaction, communication and

coordination. The self-awareness factor included awareness of own errors and

limitations, communication, asking for help, and interaction. Flying skills covered

knowledge of rules and regulations, mastery of procedures, manual flying skills and

operating ahead of the aircraft.

The pilots involved in the Nergård et al (2011) and Nergård (2014) studies

mentioned the skills and abilities to fly the aircraft, agreeing with the basic airmanship

requirements of the Kern (1996), Ebbage and Spencer (2002) and Edwards (2013)

models of airmanship. However, like Swauger’s (2006) sample, these pilots also

omitted mention of discipline. CRM competence was considered essential and while

good crewmember skills such as communication, cooperation and interaction were

included, there appears to have been no mention of situational awareness (as such,

although thinking ahead and operating ahead could be indicative of it) nor of judgement

or decision making. Among this group, judgement and situational awareness do not

appear to have the same central focus as that accorded them by Kern (1996). In

Nergard’s study, the only knowledge mentioned was that of rules and regulations,

although awareness of aircraft limitations implies knowledge, again not reflecting the

central, supportive role that Kern considers for all aspects of knowledge.

Other research specifically claiming to address airmanship includes Ali,

Gluckenberger, Rossi and Williams (2000) who carried out an evaluation of training in

a time-speeded environment and self-instructional techniques for what they referred to

as “airmanship tasks on a flight simulator”. It turned out that these were simply basic

flight manoeuvres (straight and level, climbing and turning) assessed relative to

20

tolerances in altitude, heading and airspeed. The term “airmanship”, while in the title,

was not mentioned within the paper itself. Clearly they see airmanship as being hands-

on stick and rudder skills.

A similar approach is seen in the 8th Edition of “Van Sickle’s Modern

Airmanship” (Welch, Bjork & Bjork, 1999; first edition 1957) which, interestingly,

does not use the term “airmanship” anywhere in the 950 pages of text. One is left to

assume that a sound grasp of the material included in the text will somehow translate

into “airmanship’. The chapter headings include all the technical aspects of operating an

aircraft from basic aerodynamics, through propulsion to avionics and meteorology, with

a brief discussion of aeromedicine. There is no discussion of group dynamics,

communication, or personal limitations and nothing remotely resembling crew resource

management or threat and error management. The section on accidents mentions pilot

error, inexperience, overconfidence, and poor judgement, merely concluding that they

will improve with experience (p. 941). In this context, airmanship appears to be equated

with a good working knowledge of aircraft systems and environment, and the

aeromedical aspects of self-knowledge, and glosses over what would be considered

human factors or non-technical skills in modern aviation training, and not mentioning

situational awareness or decision making directly, nor discipline at all.

Airmanship and National Culture

The model of airmanship presented in Kern (1996) is based on historical aviators

and interviews and a survey carried out in the USA. The Ebbage and Spencer model and

the Edwards model were developed in the UK, although utilising information from

around the globe. Swauger (2008) surveyed American pilots, Nergård (2011, 2014)

interviewed and surveyed Scandinavian pilots and Mavin (2010, 2014) interviewed

Australian pilots. It is possible that the differences between the models and between

findings reported in the research papers, are due to the national culture in which

participants were sourced. However, the pilot cultures in these countries are more like

each other than they are like other nationalities for example, Asian or African countries.

(Helmreich & Merritt, 1998).

Helmreich and Merritt (1998) presented a survey of pilot groups around the

world. Their 15,454 pilot participants came from 23 different countries and 33 airline

groups. They reported that there was clear recognition across all nationalities that

21

communication and coordination on the flight deck are important, and secondly that all

pilots appeared to want to talk freely with their peers and superiors. There were some

items on which there was strong difference of opinion across the different nationalities;

however generally the Anglo and Scandinavian countries were at the same end of each

distribution of nationality scores, and often British Hong Kong, Australia and the USA

were adjacent or quite close in the distribution of countries. The only exception to this

was a set of questions that related to realistic appraisal of personal performance under

stress. The British Hong Kong pilots were at the top of the distribution (most realistic)

while the USA and Australia were about half way down. The Scandinavian countries

fell between these, in the upper half of the distribution.

A cluster analysis of responses to questions about work values grouped the

‘Anglo’ countries together (Australia, USA, NZ, Ireland and the USA) and the

Scandinavian countries together (Norway, Sweden and Denmark); other clusters were a

Germanic cluster, South America plus some Asian countries, and a cluster of just Korea

and Taiwan. Italy was unique. The Anglo, Scandinavian and Germanic clusters were

more like each other than they were like the other clusters in their higher ranking of

financial reward and advancement and of questions relating to independence and lower

ranking of questions relating to order within the job.

Views on airmanship, or what makes the best pilot, are quite likely to vary across

different national cultures, given the different values placed on inter-personal

interactions in different cultures. However, the USA, Australia and UK appear to be the

least likely to differ given their common cultural background and the findings of Merritt

and Helmreich (1998).

Airmanship, Crew Resource Management and Selection

An historical view of what constituted airmanship, at various times, can be gained

by looking at what was selected for (presumably to identify good pilots or potentially

good pilots) and trained for (presumably to improve pilot performance), on the

assumption that a good pilot exhibits good airmanship.

Historically (up until 50 years ago) selection of pilots was dominated by

assessment of flying skills or the ability to develop such skills based on performance on

ability and cognitive tests (Hunter and Burke, 1994). Based on this selection activity

and on Van Sickles’ Modern Airmanship editions from 1957 to 1999 focussing on

22

technical knowledge alone (Welch, Bjorg & Bjorg, 1999), airmanship appears to have

been about flying the aircraft skilfully and precisely. There were some early comments

about the kind of person who made a good pilot, unfortunately somewhat contradictory:

‘high spirited and happy-go-lucky sportsman’ (Rippon & Manuel, 1918, cited by

Nergård et al, 2011) and ‘quiet, methodical men’ (Dockeray & Isaacs, 1921, cited by

Nergård et al, 2011).

Concern for the human-machine interface and pilot performance were frequent

research topics reported in the years after the Second World War. In addition there was

progress in relating psychological research in attention, perception, learning, decision

making and psycho-motor skills to aviation (Bond, Bryan, Rigney & Warren, 1968;

Jensen, 1997). This includes a history of aviation psychology conferences dating back

to the 1960’s (Maurino, 1994). Awareness of human factors as they effect pilot

performance was becoming more widespread, and the first human factors course for

airline pilots was initiated in 1977 by KLM (Hawkins, 1987).

This perhaps reinforced the perception of airmanship as an individual quality, but

gradually widened the scope of airmanship from just technical skills to include other

aspects of personal behaviour, and on occasion left technical skills behind:

Airmanship is used to describe a level of performance, but it is more often used to describe a personality trait. As performance, it describes excellence in the decision making and judgement of the pilot, as a trait, it is the ability and tendency to define the relevant aspects of an aerial problem or opportunity. (Mané, 1981, p. 161)

In the 1970s a series of aircraft accidents occurred in the USA that involved ‘pilot

error’ or poor performance of the flight crew in handling unusual circumstances. The

main aspects of such poor performance were identified as failures of communication,

decision making and leadership, rather than anything lacking in ‘stick and rudder’

aircraft handling skills. A workshop on the issue in 1979, sponsored by NASA, led

many of the attending airlines to develop new training programs to address this lack of

inter-personal skills. The label ‘Cockpit Resource Management’ was applied to such

programs, although this later evolved to Crew Resource Management (Helmreich

Merritt & Wilhelm, 1999). The initial program in Australia was run by Trans Australian

Airlines (TAA) in the mid-1980s, as Aircrew Team Management (Margerison, McCann

23

& Davies, 1986), although as other Australian airlines developed their own courses and

international interaction progressed, the term CRM was also applied in Australia.

The focus and content of CRM courses has evolved over the decades since

(Helmreich, Merritt & Wilhelm, 1999), but the essence of the training remains as (or

has returned to) the development of inter-personal skills to manage errors and threats as

they occur during flight (Helmreich, Klinect & Wilhelm, 1999). In recent years, CRM

has expanded its application to other team environments in aviation, including air traffic

control and maintenance crews, and has also been introduced to other teamwork

environments, for example marine bridge crew, railway crews and surgical teams (Flin,

O’Connor & Crichton, 2008; Hayward & Lowe, 2010).

Topics covered under the CRM umbrella relating to inter-personal behaviour on

the flight deck include situational awareness, decision making, communication,

teamwork and leadership (Helmreich & Foushee, 2010, Jensen, 1997). Such skills are

also termed non-technical skills, which were defined by the European Joint Aviation

Authorities as “the cognitive and social skills of flight crew members in the cockpit, not

directly related to aircraft control, system management and standard operating

procedures” (Flin, et al, 2003, p. 96). As CRM training became accepted, so the skills

that it represents have become absorbed into the overall definition of airmanship.

Inclusion of both human factors and CRM topics in the syllabus for pilot licence

training followed the ICAO 1986 Resolution on Flight Safety and Human Factors

(ICAO, 1989). In Australia, Human Performance and Limitations (HPL) was introduced

into the theory syllabus in 1991. HPL became part of the examination system in 1992

for the Airline Transport Pilot Licence and in 1995 for the Private Pilot and Commercial

Pilot Licences. Mandating the study of HPL as part of basic pilot licencing reinforces its

importance as part of good airmanship.

The accident record and training response has drawn attention to the inter-

personal behaviours required for effective multi-crew performance, however concern

that CRM training alone may not be ‘enough’ to counter ingrained non-CRM

behaviours has led to the increasing inclusion of such skills in selection procedures

(Bartram & Baxter, 1994; Damitz, Manzey, Kleinmann & Severin, 2003; Salas, 2014).

The challenge for selection has been how to assess and predict such behaviours,

especially in the selection of ab-initio student or cadet pilots. One response was the

24

development of assessment centres to examine ‘live’ social interaction (Damitz, et al,

2003), however the expense of such centres and problems with inter-rater reliability and

construct validity has led to recent consideration of whether social competence

questionnaires may replace such centres (Hoerman & Goerke, 2014). The importance of

such behaviours for aircraft operation has certainly percolated to the workforce. An

assessment of the knowledge, skills, abilities and other factors required for effective

operation on advanced flight-decks found that while cognitive and psycho-motor

abilities were seen as remaining relevant, it was inter-personal skills that gained the

highest relevance rating from pilots themselves (Goerters, Mashchke and Eiβfeldt,

2004, cited by Damos,, 2014).

Personality has figured in selection activity from the early days of flight, however

the use of measures of personality has showed only modest relationships to training

outcomes (Campbell, Castenada & Pulos, 2010; Martinussen, 1996). Development of

the Big Five factors of personality organisation (Costa & McCrae, 1989) led to further

attempts to relate personality to flight performance, with various degrees of success

(King, 2014; Rose, Barron, Carretta, Arnold & Howse, 2014). Openness to experience,

conscientiousness and extroversion show some potential, however application depends

on specific job analysis and assessment of predictive validity (Campbell et al, 2010;

King, 2014; Rose et al, 2014; Tvarynas, 2014). In addition, neuroticism/emotional

stability has been suggested as a screening variable (King, 2014). Personality is seen as

a potential source of predictive power for selection purposes, and has been part of some

discussions of airmanship. Kern (1996, 2009a) does not include any personality

variables, unless discipline is counted as such, in his model of airmanship. Discipline is

also included by Ebbage and Spencer (2003) and Edwards (2013). Ebbage and Spencer

mention confidence and self-improvement, while Edwards adds self-respect, integrity

and honesty, all of which could be considered at least personal qualities, although not

formal personality traits.

The history and development of aviation selection and training shows the

increasing emphasis on human factors and psychological aspects of the flight task.

These include decision making and situational awareness, and inter-personal skills like

communication and teamwork. It would therefore be expected that these concepts have

made their way into pilot’s generally held ideas of airmanship. While such concepts

feature in the models of airmanship, these models have not been tested empirically.

25

Conclusion

Kern’s (1996) model of airmanship is based on the study of historical heroes of

aviation, accident analysis and interviews with military aviators in the USAF. The

responses of 300 USA aviators, including civilians, to a survey were also drawn upon

(T. Kern, personal communication, June 10, 2009). This research activity was not

presented in a scientific, peer reviewed format, but rather as an instructional textbook

aimed at ab-initio student pilots. Similarly, Edwards (2013) book is presented as a

guide, apparently based on personal experience and analysis of aircraft accidents. The

model of airmanship proposed by Ebbage and Spencer (2003) was presented as a

theoretical paper at a military training conference with little discussion of any research

basis on which the model was founded, although both of the authors were employed by

the training provider to the RAF. Thus, none of the major models of airmanship have

been published in a peer reviewed format, nor does it appear that there are peer

reviewed reports of validation by application in a real world setting. It is worth

considering whether any of these models are relevant to and descriptive of how aviators

actually see airmanship. The research of both Swauger (2006), Nergård et al (2011) and

Nergård (2014) suggests that there may be some differences between Kern and Ebbage

and Spencer’s ideas and the opinions of airline pilots. Some of the differences may be a

result of the military opposed to civilian flying environments, with discipline being a

more military concept and personality and CRM issues being more relevant in civilian

environments.

The existing models of airmanship are not based on systematic analysis of pilots’

views, nor have they been empirically tested. The current research aims to investigate

the views of Australian aviators about airmanship.

Training and Development of Airmanship in Australia

Both Kern (1996) and Ebbage and Spencer (2003) suggest that airmanship should

be an active and integrated part of overall flight and ground training rather than being

merely a by-product of conventional training. Kern (1996) promotes the integration of

airmanship within ongoing aircraft handling skills training, and suggests three phases of

airmanship training: (1) for the instructor to explain the importance of airmanship and

instil motivation for improvement; (2) to model and teach airmanship; and (3) to

evaluate airmanship actively and aggressively. Kern emphasises the importance of the

26

instructor as a mentor to introduce a sound aviation culture, model appropriate

behaviours and enforce standards and expectations. Kern encourages the use of his

model of airmanship as a template for the structuring of pre-flight briefings and post-

flight debrief and reflection, an activity that should, he writes, continue throughout

one’s flying career, and not just take place during initial training. He was writing for

both civilian and military audiences, with sections of the text for instructors as well as a

section that could be used for self-guided development by student pilots.

Ebbage and Spencer (2003) outline a similar three phase approach to training

airmanship, proposing:(1) explanation of the concept and its importance, by discussion

of case studies; (2) instruction covering the required knowledge, skills and attitudes; and

(3) use of objective assessment with provision of feedback. Given the presentation at a

military training conference, presumably the paper was directed mainly at military flight

training.

Edwards (2013) includes a section on ‘How is this learnt?’ in each chapter on his

three main aspects of airmanship: technical, operational and non-technical elements. He

does not set out an overall training strategy for airmanship, but consistently returns to

study, practice and focussed training for each part of each element. Edwards emphasises

the importance of the debrief activity, whether after a training session or crew or self-

debrief after a flight.

The type of initial training differs between military and civilian sectors of the

aviation industry, certainly in Australia. Military flight training caters for highly

selected cadets who have to meet strict performance expectations in a very high tempo

course. There are intensive civilian training programs leading to a commercial pilot

licence with the expectation for graduates of proceeding on to an airline career.

However, these programs are not quite as high-tempo or inflexible as the military

model. In contrast, most civilian training in flying clubs may be in response to interest

or enthusiasm rather than a career move, and tends to be self-funded and self-paced.

Airmanship is included in current military ab initio training, in both ground

school and flight training in Australia (Carrick, Graham, Healey & Pickard, 2008, see

Appendix A). Airmanship instruction takes the form of classes covering safety and non-

technical skills, which are examinable. Airmanship is demonstrated during flight

training and is assessed, along with preparation and technical aircraft handling, at

27

intervals throughout the flight training syllabus. Thus “airmanship” in this environment

presumably does not include preparation or aircraft handling skills, unlike the two

models described above.

In the Australian civilian aviation industry airmanship, as such, is not part of the

theory syllabus for flight licenses, although the study of human factors is included,

forming the Human Performance and Limitations part of the VFR Syllabus and ATPL

Syllabus. Airmanship is supposed to be included in pre-flight briefings during flight

training (CASA, 2006) and Licence flight tests have included a subjective assessment of

airmanship carried out by the testing officer. From 2009, the human factors (HF) and

threat and error management (TEM) issues covered in the human performance and

limitations part of the theory syllabus have been explicitly tested in pilot licence flight

tests. The criteria to assess competency in HF and TEM in flight tests were introduced

by the Civil Aviation Safety Authority (CASA) on the basis of being part of “old-

fashioned good airmanship” (CASA, 2008) and the suggestion is that for “CASA to link

airmanship to human factors, is in effect, tantamount to bringing science to the often

nebulous concept of airmanship” (CASA, 2008, p. 8). CASA requires the following

five elements to be assessed during the flight test: maintain effective lookout; maintain

situational awareness; assess situations and make decisions; set priorities and manage

tasks; and maintain effective communications and interpersonal relationships (CASA,

2008, p. 7). In addition, there is also assessment of recognition and management of

threats, errors and undesired aircraft states, as assessment of the threat and error

management part of the syllabus (Carrick, 2009).

The CASA publication “Flight Instructor Manual – Aeroplane” does include brief

comment on airmanship in most of the air exercises covered, generally relating to

lookout and smooth handling. The introduction to the manual provides the following

requirements for airmanship: “The instructor must possess and outwardly demonstrate a

high standard of discipline, common sense and initiative, which are the cornerstones of

airmanship. The instructor must also display leadership qualities, without which the

above qualities will be lacking” (CASA, 2006, p. iii). This brief description does

mention discipline and leadership, but not any other elements of the various models of

airmanship.

Ongoing training differs somewhat between industry sectors in aviation. Ongoing

training in the military includes conversion training onto specific aircraft types, post-

28

conversion training and development in operations and tactics, refresher and currency

training. For some, career development may include advanced training, instructor or test

pilot courses, or exchange duty with air forces of allied nations. In the airline sector,

there are several types of pilot training including: induction and procedural training for

newly hired pilots; upgrade training to change aircrew status (such as first officer to

captain); conversion training onto a specific aircraft type within an airline; and

mandatory recurrent training (usually six-monthly) which ensures that aircrew maintain

the required skills, proficiency and knowledge to operate a particular aircraft (Orlady,

2010). Under the current structure of the industry in Australia, “air work” covers

operations that are commercial but not airline, including flying doctor services,

helicopter operations, agricultural operations and charter flying, as part of the broad

general aviation sector. Ongoing training for “air work” pilots varies between operators,

and may parallel the processes described above for airline pilots, or may be simply

regular proficiency checks. In the private side of general aviation, business, club and

recreational flying, post licence training can be infrequent, and while all pilots are

expected to keep current on their aircraft type, this may be no more than an occasional

check ride with a flying instructor.

Crew resource management (CRM) is a formal part of pilot training in airlines,

mandated by ICAO and in Australia, CASA, with the intent to improve team

coordination and monitoring by utilising all available resources - people, equipment,

and information (Salas, Burke, Bowers, & Wilson, 2001). Airlines around the world

have introduced and maintained CRM from the mid 1980’s as adjunct courses dealing

with interpersonal communication, decision making, leadership, situational awareness,

conflict resolution, and group processes, though course content and duration has varied

(Flin & Martin, 2001; MacLeod, 2005). More recently threat and error management

(TEM) has been incorporated into CRM training, such that the two terms are often used

together or synonymously (CRM/TEM). The term “non-technical training” has also

been used to describe human factors and CRM/TEM training, to distinguish it from

technical training (hands-on, stick and rudder and procedures training). Military training

in Australia has included CRM from the late 1990’s (Shepherd, 2005). The themes

covered in the CRM courses are closely aligned with the concepts that are included in

definitions of airmanship, such as situational awareness, decision making,

29

communication and team building, although not taught using airmanship as part of the

terminology.

The question remains, whether airmanship itself is considered part of the training

of Australian aviators, or is it something that “hopefully” develops without formal

instruction, almost as a side-product of technical and non-technical training.

Current Research

Airmanship is a term that continues to be used in the aviation industry and media,

despite suffering from a lack of specific definition, and plethora of ideas about what that

definition should be (Marchbank, 2008). Kern (1996, 2009a) is the most widely quoted

source and his model of airmanship includes discipline as the cornerstone of

airmanship, skill and proficiency as the bedrock principles on which the six pillars of

knowledge rest. Knowledge of one’s self, the team, the aircraft, the environment, risk

and the mission are essential to support good situational awareness resulting in good

judgement (or decision making) to achieve the goal of safety, mission effectiveness and

efficiency. The model developed by Ebbage and Spencer (2003), specifically addressing

military training, is based on Kern’s model but draws more attention to attitudes and

self-assessment, and to skills required in multi-crew flight such as communication, crew

co-ordination, interpersonal skills, leadership and flight-deck management. Edwards

(2013) considers the knowledge, skills and attitudes across technical, non-technical and

operational elements of aviation that lead to safe, efficient and effective flight.

None of the existing models of airmanship have been objectively, empirically

assessed in a peer reviewed format. Both the Kern and Ebbage and Spencer models

have a military influence. The different emphasis seen in Swauger’s (2006) research and

that of Nergård et al. (2011) and Nergård (2014) may be a reflection of a military vs

airline contrast.

Both Kern (1996) and Ebbage and Spencer (2003) recommend the integration of

airmanship training into initial flight training activities, and specify the phases that

should be included in airmanship training. This does occur in ab initio military training

in Australia (Carrick et al., 2008) albeit with an apparently narrower definition. Civilian

ab-initio training includes some of the elements of airmanship as defined by the models,

while not labelling them as such. It appears at first glance that “airmanship” may be a

by-product of civilian training rather than a deliberate part of the process, in Australia.

30

However, this is a conclusion drawn from the training publications, rather than from

direct observation of training activity. There has been no specific research into the

training of airmanship, per se, in civilian ab-initio training environments in Australia.

The current project aims to redress the lack of empirical evidence regarding

aviators’ views of airmanship and to develop an overview of what it is that Australian

aviators consider “airmanship” to be, whether airmanship is specifically included in

pilot training in Australia, and if so, how. Differences in views between aviators in

military and civilian environments and differences in approach to training in these

environments and at the different levels of training are also considerations.

There are four main parts to this research. Firstly, two surveys of Australian

aviators were conducted to find out what they consider “airmanship” to be. The initial

survey covered the views of mainly civilian general aviation pilots while the more

extensive second survey was able to include a comparison of responses between

aviators from military and civilian background and different current sectors of aviation

activity. In both surveys a comparison was made between the participants’ views and

the main models of airmanship. Secondly, the same aviators were asked what their

experience was in ab initio training with regard to the development of airmanship, and

its inclusion in the current training that they undertake. Thirdly, a selection of military

and civilian flight instructors at ab-initio and advanced levels of training were

interviewed about their definition of airmanship and how they include these concepts in

their training of students. A comparison of these responses between sectors and levels

was conducted, and considered in terms of the training recommendations of Kern

(1996) and Ebbage and Spencer (2003). Fourthly, the structure of a proposed new

model of airmanship was tested through a survey of Australian pilots.

An analysis of a first survey of Australian pilots forms Chapter Two of this thesis.

This chapter explores the results of an initial on-line survey that reached mainly pilots

from the general aviation sector and addresses their definition of airmanship, training

experiences and the development of airmanship during their flying experience.

Comparison with the Kern (1996, 2009a), Ebbage and Spencer (2003) and Edwards

(2013) models is also reported.

Results from a more extensive and enlarged online survey are presented in

Chapter Three. A greater number of respondents allowed a comparison between civilian

31

and military background pilots. The match between the concepts raised by the

respondents and the Kern (1996, 2009a), Ebbage and Spencer (2003) and Edwards

(2013) models is explored. The training experiences reported and current training

activity is analysed with reference to the recommendations made by Kern (1996) and

Ebbage and Spencer (2003).

The next chapter, Chapter Four, examines the responses of flight instructors in

military and civilian environments, at ab initio and more advanced levels, to questions

presented in individual interviews about the definition of airmanship, its inclusion in

flight training and the techniques used to assess airmanship. The responses of ab-initio

and advanced instructors in each sector are compared, and matched with the models of

airmanship. The responses between the military and civilian instructors are compared

and discussed in terms of the match with the recommendations for instruction made by

Kern (1996) and Ebbage and Spencer (2003).

The first three studies triangulate onto the representation of airmanship among

Australian pilots, and the fifth chapter summarises the conclusions regarding the

definition of airmanship and presents a new model of airmanship. This model proposes

five domains of airmanship concepts and includes consideration of the context in which

airmanship occurs.

The sixth chapter reports a further survey, in which Australian pilots were asked

to rate the relationship between the main concepts making up airmanship, to confirm the

domain structure of concepts in the airmanship model presented in Chapter 5. This

resulted in some revision of the model.

The final chapter summarises the conclusions reached in the foregoing chapters.

Limitations of the current research are discussed and directions for future research are

explored.

32

Chapter 2 Page | 33

Chapter 2. Airmanship: Definition and Development

among Australian Pilots

The term “airmanship” is used throughout the aviation industry. It appears in

the civilian “Flight Instructor Manual – Aeroplane” (CASA, 2006) as a general aim

for aviators, is part of the flight tests for licensing and is a part of the assessment of

trainee military aviators (Carrick, Graham, Healey & Pickard, 2008). There is some

debate as to the meaning of the term in general usage, Marchbank (2008) suggesting

that we need something better than the common statement about airmanship: “I know

it when I see it” (p. 11).

This chapter addresses whether Australian aviators consider airmanship to

include similar concepts to those described in the Kern (1996; 2009), Ebbage and

Spencer (2003) and Edwards (2013) models of airmanship, and whether there are any

additional themes brought up by members of the Australian aviation community. The

concepts included by Kern rest upon discipline as a key cornerstone for airmanship

as well as the bedrock principles of skill and proficiency. Knowledge of self, the

team, the aircraft, the environment (including physical, organisational and legislative

environments), risk and mission also support the “capstone outcomes” of situational

awareness and judgement (or decision making). In addition to these concepts,

Ebbage and Spencer (2003) suggest that the pilot should display suitable attitudes,

including professionalism and self-improvement, and the avoidance of hazardous

attitudes such as invulnerability and impulsivity. Further skills are included in this

model, specifically communication, crew coordination, interpersonal skills,

management of the flight deck, self-assessment and team leadership. Edwards (2013)

while organising his model differently, does not include anything that the above two

models do not. Together, these models provide a fairly wide range of concepts

covered by “airmanship”, but it is possible that Australian aviators would come up

with some alternatives in addition to, or instead of, these concepts. Rather than

provide a tick-list of possibilities, allowing for recognition and selection of concepts

for inclusion in a definition of airmanship, open ended questions requiring

spontaneous production of responses were preferred as a more useful method of

investigation.

34

Both Kern (1996) and Ebbage and Spencer (2003) recommend the integration

of airmanship training into initial flight training, thus an additional aim for the

current investigation is to identify the experiences of Australian pilots with regard to

instruction in airmanship during their ab-initio training. Kern (1996) suggests that

airmanship should be a career-spanning process of reflection and improvement and

Edwards (2013) also advocates life-long learning through study, training and

debriefing. Any further influences on the development of airmanship reported by

Australian pilots would be of interest to determine whether their experiences

conform to this ideal.

The aims for this study are to explore what Australian pilots think airmanship

is, and to discover what part airmanship played in their training. These aims were

addressed in the development of a short on-line survey of open-ended questions.

Specifically the intent of the survey was to identify the concepts that Australian

pilots associate with the term “airmanship”, to assess whether they think that

airmanship was part of their initial training and to determine what these pilots

consider led to them developing a sense of airmanship.

Methodological considerations

Collecting definitions of airmanship and views about airmanship training from

a selection of aviators has the potential to provide a rich source of information and

opinion. There is some challenge in determining a suitable method of analysis that

preserves the individuality of participants’ contributions while also ensuring that

comparison with existing models of airmanship can be conducted. This section

locates the research perspective within the range of qualitative epistemological and

methodological frameworks and outlines the approach taken to the analysis of the

qualitative data in the survey responses.

According to Gray (2009), there are two main philosophical perspectives

through which qualitative analysis is undertaken: naturalistic and progressive. A

naturalistic paradigm assumes that there is an objective reality that can be

approached by extensive collection of data in real settings. Researchers with this

outlook tend to be somewhat detached from the setting and objective in outlook,

collect data from a number of representative sources, and aim for minimal

interference and bias in the collection and coding of verbatim accounts. Within this

35

perspective common methodological frameworks include grounded theory, thematic

analysis and content analysis.

The naturalistic approach contrasts with progressive approaches such as post-

modernism and constructivism, which are based on the idea that reality and science

are socially constructed. In these approaches, researchers form part of the research

setting and it is considered that researchers should engage in reflexive and self-

critical dialog which forms part of the analysis. The aim is to reveal hidden realities,

lived experiences and/or structures of consciousness among the participants.

Methodological frameworks include phenomenology, ethnography and hermeneutics

(Liamputtong, 2013).

The current research is aligned with the naturalistic perspective, stemming

from an objectivist epistemology. The plan is to ask direct questions about aviators’

thoughts and experiences relating to airmanship and conduct an objective assessment

of their contributions. The most appropriate methodological framework is that of

content analysis. Grounded theory and thematic analysis assume separation from

existing ideas or models related to the topic area. The expectation in these

approaches is that themes and theories will be developed independently and are

based in the data. In the current research, existing models are well known, and a

deductive or directed approach to analysis of content is preferable and more

appropriate.

Content Analysis

Content analysis adheres to the naturalist paradigm in analysing text materials

(Hsieh & Shannon, 2005), but includes a range of approaches from intuitive and

interpretive to systematic, strict and formulaic (Krippendorf, 2013). While some

approaches to content analysis look only for ‘manifest’ or obvious surface content of

communication, there is also room for more impressionistic interpretation. Hsieh and

Shannon describe three main applications of content analysis: conventional, directed

and summative. Summative content analysis is concerned with contextual usage, and

as such is language focused rather than content focused. Conventional content

analysis is useful when theory and previous research is scarce, as researchers look for

categories arising from the data and avoid preconceived ideas. This approach shares

a similar initial analytical approach to grounded theory (Corbin & Strauss, 2008) in

36

which there is no guidance from existing theory to develop coding, but content itself

leads to generalisations and thence to theory development (Potter & Levine-

Donnerstein, 1999). It does need to be recognised, as Braun and Clarke (2006)

comment, that researchers cannot free themselves completely from their theoretical

background and data are not coded in an epistemological vacuum. Directive or

deductive content analysis recognises that pre-existing theory and research is known

to the researcher. Thus existing theory and research are used overtly in the

determination of coding frameworks (Mayring, 2000). Researchers begin by

identifying key concepts or variables that have been presented in the relevant

literature. Established theoretical positions are used to create initial coding

categories. A first run through the data helps produce operational definitions

(Krippendorff, 2013). As further ideas or concepts that are not covered by the coding

framework are encountered in the data, new categories or sub-categories can be

created and defined (Hsieh & Shannon, 2005). Findings can offer both supporting

and non-supporting evidence for existing theory, because both are searched for in the

coding process. As Hsieh and Shannon note “a directed approach makes explicit the

reality that researchers are unlikely to be working from the naïve perspective that is

often viewed as the hallmark of naturalistic designs” (p. 1283).

The intent in all forms of content analysis is to reduce text into categories. A

category may consist of one or more words, or may represent an idea or concept

expressed in various ways (Weber, 1990). Content analysis should not be seen as

simple word count: interpretation is required to identify synonyms, explanations or

euphemisms within the material under analysis (Stemler, 2001). Nonetheless, content

analysis can result in the production of frequency data (Wilkinson, 2000) and thus

allow some quantitative analysis of what started off as qualitative data (Ryan &

Bernard, 2000). In fact Weber (1990) notes that perhaps the best content analysis

uses both qualitative and quantitative operations on textual material, thus combining

what are usually thought of as antithetical modes of analysis.

Reliability and Validity in Content Analysis

Consideration of reliability and validity is of considerable importance in any

qualitative research methodology. Potter and Levine-Donnerstein (1999) address the

reliability and validity of the coding process in content analysis. They consider that

37

reliability and validity should be conceptualised differently across various forms of

content analysis. They consider that two foundational issues must be determined, the

nature of the data content and the role of theory, before considering the extent of

coding checks required. Content can be manifest, latent or projective. That is, either

the coder is coding the obvious surface expression of the participants (manifest

content); or looking beyond the surface to latent meaning. Latent meaning could

include looking for patterns in the data or may require coder interpretation of deeper

meanings. Projective content is revealed as coders look at content as symbols and use

their own mental schema to judge the meaning within the content. The role of theory

has been discussed above, deductive content analysis uses existing research or theory

to guide the development of coding schemes. The alternatives presented by Potter

and Levine-Donnerstein are not using theory at all, or inductive development of

theory from data after analysis.

Validity is a two-step process: firstly, the development of a coding scheme that

details the codes and code definitions; and secondly, comparison of coder decisions

against an expert standard. Validity can be maximised by ensuring a strong coding

scheme that reduces the need for coders to develop their own interpretations. Using a

coding scheme ensures that coding is uniform and can be considered systematic and

thus scientific. Reproducibility of coding output is the strongest realistic method of

determining reliability in content analysis (Potter & Levine-Donnerstein, 1999). This

quantitative approach to reliability in qualitative analysis involves the use of multiple

coders over common data, counting the areas of agreement and disagreement and

then calculating inter-rater reliabilities (Krippendorff, 2013).

Mayring (2000) reports increasing criticism of ‘classical’ quality criteria for

reliability applied in content analytic research. Alternatives that Mayring presents

include documentation of method, interpretation safeguards, and triangulation, that

is, using multiple sources, methods, investigators and theories (Lincoln & Guba,

1985). Braun and Clarke (2006, 2014) agree with this critique, charging that iterative

processes of checking and discussing coding to reach consensus is sufficient check

on reliability without calculating a number artificially.

38

Challenges for Reliability and Validity in Manifest Deductive Content Analysis

Potter and Levine-Donnerstein (1999) consider that the primary threat to

reliability in the analysis of manifest data is coder fatigue, due to the high levels of

concentration required. Ensuring that coding sessions are relatively short and spaced

out, coupled with several runs over the data, address this problem. Hsieh and

Shannon (2005) suggest that in a deductive content analysis, the researcher may

approach the data well informed but biased, and be more likely to find supportive

than non-supportive evidence of the theoretical framework. Discussion between

coders about uncategorised content, update of the coding guidelines and repeated

runs over the data address this challenge. In addition, Hsieh and Shannon indicate

that a desire to please the researcher may lead to participants agreeing with

suggestions or hints made by the researcher related to the theoretical framework.

Audit of the survey and interview questions and training of interviewers counter this

possibility. The third challenge that Hsieh and Shannon raise, is that over-emphasis

on the theory can lead to researchers ignoring context within which the phenomenon

occurs. One way to address this issue may be to specifically ask about context, or

include context references in the coding framework.

Procedures for the Current Research

In the current research, content is primarily manifest; that is, attention is paid to

the overt content of the data, with some interpretation in dealing with synonyms or

descriptions. The approach is that of deductive content analysis with existing models

of airmanship providing the initial coding categories (or concepts).

Hsieh and Shannon (2005) state that all content analysis should use the ‘seven

classic steps’, with variations depending on the approach to content analysis being

used. These steps are: formulate the research question; select the sample to be

analysed; define the categories to be applied; outline the coding process and conduct

coder training; implement the coding process; determine the reliability of the

analysis; and analyse the results of the coding process (Kaid, 1989, cited by Hsieh &

Shannon, 2005, p 1285). Mayring (2000) is more prescriptive regarding the approach

used, specifying that the sequence for deductive content analysis should include

specification of the initial, theory based categories, and revision of the categories as

coding progresses.

39

In the current research, the process used followed the basic classic steps, with

attention to the need to carefully construct the initial coding categories and revise

them as new categories (or concepts) appear in the data. The process included a final

return to the coded concepts in order to sort through them to establish possible

combinations of concepts and identification of any themes within the data. The

process also attempted to address reliability and validity requirements and counter

the difficulties and challenges for deductive content analysis raised by Potter and

Levine-Donnerstein (1999) and Hsieh and Shannon (2005). Coding sessions were not

prolonged, the coders frequently discussed the coding process, guidelines and any

difficult cases until consensus was reached, and there were several runs over the data.

The process can be outlined as follows:

1. Determination of the research question and identification of relevant

models

2. Development of data-capture strategies and decisions regarding participant

groups

3. Definition of the category system (concepts) from the models

4. Definition of coding guidelines (definitions, examples and coding rules)

5. Training of coders, discussion of coding guidelines

6. Initial run-through, preliminary coding and identification of additional

concepts with definitions and anchor examples

7. Revision of category system with additional concepts, discussion between

coders to ensure reliability of coding.

8. Further work through of material (iterate with step 6)

9. Final establishment of concepts, concept combinations and themes

10. Analysis and interpretation

This general process was applied in each qualitative study in the current

research, any deviations from it will be addressed in the report on each study.

40

Method

Participants

There were 40 participants in the survey. The respondents varied in level of

licence held from a student pilot through to an airline transport pilot licence (ATPL)

holder; in hours of experience (28 hours to over 20,000 hours, M = 3776 hours SD =

5,518 hours); and in length of time involved in aviation (from less than one year to

55 years in the industry, M = 17.8 years, SD = 13 years). Sixteen respondents held

Instructor ratings.

Measure

The full questionnaire is presented in Appendix B. The first part of the

questionnaire asked demographic questions about kind of licence held and flying

experience. The second part consisted of 11 open ended questions about: the

respondents’ definition of airmanship; the inclusion of airmanship in initial training;

development of airmanship in respondents; development of airmanship in students;

awareness of airmanship during current aviation activities; and examples of good and

poor airmanship. The final question asked for any other comments.

Procedure

Ethics approval was granted through ethics application, H-647-1207.

The on-line survey was promoted at CASA safety seminars from May to

August 2008 and in the May/June 2008 issue of Flight Safety magazine. This

magazine is produced by CASA and, at the time, distributed to all Australian pilot

licence holders. There was an option for participants to contact the researcher to

obtain a paper copy of the survey, instead of completing it on-line. Only one

participant took this option. There were 40 responses by September 2008.

Analysis

The survey responses were analysed according to the approach detailed in

‘Methodological Considerations’, above. Two coders worked on the data, both had

training in the analysis of qualitative data and the use of NVivo. NVivo9 (QSR,

2011) was used as the coding platform for determining concepts raised in the

responses. Response length ranged from 42 words to 781 words, with a mean of 308

41

words. Existing models of airmanship were used to create initial coding categories

(tree nodes) covering the basic concepts of those models. Initially, 12 tree nodes

were established to reflect the concepts included in Kern’s model and nine to cover

the additional Ebbage and Spencer (and Edwards) themes. Further categories (free

nodes) were created as other concepts were discovered. A total of 44 nodes were

developed, each representing concepts that respondents related to airmanship or

issues that they raised. The coders worked through the survey responses until they

were satisfied that a reliably coded data set had been achieved.

Initially, it was intended to conduct comparisons by experience, but a median

split would have been at 1000 hours or 20 years of experience, which would give a

“less experienced” group that was actually quite experienced. In addition, there was

some concern that hours had been mis-recorded for several respondents (over 30

years but only 700 to 1400 hours) although this may reflect a group dominated by

recreational aviation.

Results and Discussion

Definition of Airmanship

What is airmanship? The questions of most interest for the definition of

airmanship are Question 1: “What do you think airmanship is?” and Question 2,

which asked the respondent to elaborate on the answer given in Question 1. Other

questions relevant to the definition of airmanship include questions that asked about

the most important aspects of airmanship (Question 6), the situations in which

respondents were most aware of airmanship (Question 7) and about examples of

good and poor airmanship (Questions 9 and 10). Questions 3 to 5 and 8 are discussed

in a later section about the development of airmanship.

Table 2.1 shows the number of respondents who mentioned each of the

concepts found in the responses to Questions 1 and 2, and the total number of

respondents who mentioned each concept in any of the questions (i.e. questions 1 to

11). In the discussion of the concepts mentioned by the participants, any quotations

are identified by a three-digit participant number.

42

Table 2.1

Number of Respondents (of a Total of 40) who Mentioned Each Concept in Questions 1, 2 and Overall (Qs 1 to 11).

Concepts Q1 Q2 Overall Nil responses 0 10 0 N of respondents 40 30 40 Kern concepts Judgement or decision making 2 5 10 Situational awareness 10 9 22 Knowledge 6 5 9 Self 2 3 5 Aircraft 2 1 11 Team 0 0 1 Environment 0 0 1 Rules, regulations 9 4 20 Checks 1 5 14 Risk 3 5 10 Mission: Planning 9 10 16 Skill 9 2 12 Proficiency 1 1 1 Discipline 4 4 5 Concepts specific to Ebbage & Spencer/Edwards Attitudes: Hazardous 0 0 0 Professionalism 0 2 4 Self-improvement 1 1 6 Skills: Communication 4 4 15 Crew co-ordination 0 0 0 Interpersonal 1 1 2 Management of flight-deck 3 0 5 Self-assessment 3 1 4 Team leadership 0 0 0 Additional concepts (not in existing models) Safety 13 14 25 Courtesy/ethics/respect 12 12 12 Attitude / personal qualities 10 0 22 Common sense 9 5 9 No shortcuts 2 3 3 Lookout 2 1 19 Responsibility 2 0 5 Knowledge - medical 1 0 1

As shown in Table 2.1, the main themes of the Kern model were reflected in

some of the respondents’ comments. No respondents listed all the Kern concepts, nor

all of the concepts included in the Ebbage and Spencer or the Edwards models. There

were additional concepts that were expressed by respondents but are not included in

43

any of the existing models. The mean number of concepts raised per respondent was

2.95 in Question 1 (N=40). The mean for respondents who answered Question 2 was

2.90 concepts per respondent (N=30). Additional concepts were mentioned by more

of the respondents over the entire set of questions, compared with just Question 1 or

Question 2, as further probes led to more concepts being generated.

A chi-square goodness of fit (with α = .05) was used to assess whether the

Kern model concepts were equally likely to be mentioned across all the survey

questions (the Overall column in Table 2.1). The chi-square test was statistically

significant, χ² (13, N = 137) = 60.737, p < .001, indicating that some concepts were

mentioned more frequently than others. As an index of effect size, Cohen’s w was

0.67, which can be considered large.

Safety was the concept with the most mentions by respondents in Question 1,

Question 2 and across all questions. Kern did not include safety in his model,

possibly considering it an outcome of good airmanship rather than part of it. The

survey respondents mentioned safety as central to airmanship, with comments like:

“Airmanship is a way of being safe” (010), “An aviator's own attitude towards his /

her own safety and the safety of others” (016) and “Sensible aviation practices that

take into consideration the safety of yourself and others” (019). It should also be

noted that ‘safety’ was used as an example in the text of Question 2 (“For example, if

you wrote ‘being safe’, write in more detail what ‘being safe’ would involve”). This

may have influenced participants to include comment on safety, however fewer than

half the participants wrote about safety in either Question 1 or Question 2.

Judgement or decision making is the major capstone outcome according to

Kern, but was not considered by respondents to be particularly central to airmanship,

in that only two mentioned it in response to Question 1 and five in their elaboration

in Question 2. In contrast, situational awareness (SA) clearly is seen as a significant

concept relating to airmanship, with 10 respondents mentioning it in Question 1, 9 in

Question 2, and 22 respondents including the concept over all questions. Some

examples of comments coded in this theme are: “awareness of the operating

environment” (020), “being aware of the situation around you i.e. the weather, other

traffic” (012) and “I believe airmanship to be an awareness and understanding of an

aviator of their surroundings and what is taking place” (015). Lookout is an activity

upon which SA is dependent, in visual flight, and is also frequently mentioned

44

although not so much in response to the first two questions (being mentioned three

times) with 19 respondents including it over all questions.

Thirteen respondents referred to knowledge: six made mention of knowledge

generally; others specifically related to self (2), the aircraft (2) or to risk (3); with one

person commenting on knowledge about medical aid (something none of the existing

models included). There were no references to knowledge of the team in responses to

the definition questions, but this may have been a function of the mostly general and

recreational aviation sample, flying mostly in single pilot operations. Teamwork is

more important in the multi-crew cockpits of commercial aviation or military multi-

crew operations.

There were few specific references to the environment. However, when

elaborating upon the model, Kern divides environment into the physical, regulatory

and organisational environments. When considered at this level, nine respondents

mentioned rules, regulations and operating procedures in their definition (Q1 and 2),

and 20 respondents did so overall questions.

Those respondents who included flight planning, planning generally,

preparation and thinking about what one was doing, in their responses are shown as

“mission” in Table 2.1. Clearly there is consideration among respondents that

planning of what will be going on in flight is part of airmanship.

Skill and proficiency are separate items in Kern’s model. Twelve respondents

included skill in their definitions of airmanship, or stated “ability to control the

aircraft” (004); “aircraft handling” (037); or “operate an aircraft” (029). One of these

stated that airmanship “is usually referring to judgement and decision making more

so than manipulative skill” (022). Only one respondent specifically mentioned

proficiency: “as a pilot without a lot of hours I need to fly regularly enough to remain

competent” (009). One respondent gave as an example of good airmanship: “a flying

competition that required pilots to touchdown as close to a spot as possible on the

runway” (017) which indicates a skill based component within their conception of

airmanship.

Discipline is the cornerstone of airmanship, according to Kern (1996), and it

was included in their definitions by four of the respondents, with added comment in

response to Question 2 for example: “airmanship is the discipline that you apply to

45

your flying” (011) and “Discipline, implying self-discipline should be exercised at all

times as an essential element of airmanship” (034). Three other respondents wrote of

“not cutting corners” or “no short-cuts” (009, 033, 039) which could be interpreted to

indicate the application of discipline to complete each task in full.

The concepts included in the Ebbage and Spencer (2003) and Edwards (2013)

models that were not also included by Kern are shown in the middle section of Table

2.1. While Kern mentions good communication when writing about the

characteristics of teamwork, as part of his “Knowledge” structure, communication is

rather more active than just a “knowledge of” item would imply, and is included as a

skill set by Ebbage and Spencer and Edwards. Good communication is seen as

important to maintaining situational awareness and thus safety. This is recognised by

the four respondents who mentioned communication in their initial definitions or

elaboration, and by 15 respondents overall. Examples of such responses to Question

1 include: “Ability to communicate with other users of aircraft to achieve safe

reliable outcomes” (001) and “keeping in contact with other aircraft” (021).

Management of the flight-deck and self-assessment were each mentioned by

three respondents as part of their initial definition of airmanship. An example of this

is “The desired behaviours are such that I am effectively managing flight in a way

that both achieves the intended mission of the flight and manages the risks to both

my aircraft and to others” (040).

In addition to the concepts included in the existing models, the respondents

included a number of other concepts in their definition and elaboration of

airmanship. There were many mentions of a complex of ideas that can be represented

as courtesy, as well as mention of courtesy itself. These included consideration of

other airspace users, helping others, manners, doing the right thing, and ethical

behaviour, elaborated as “honest in all dealings and treat others as you would wish to

be treated” (001). Courtesy was always mentioned as part of a grouping of

characteristics that makes up airmanship, rather than alone, such as in the definitions:

“A combination of flying skills, common sense and courtesy” (002) and “Being alert

and abiding by all the rules and being courteous to other pilots” (008).

Common sense is a concept that was included by nine of the 40 respondents.

Three of the nine elaborated on common sense in Question 2: “Know your abilities

46

and operate within that area of competence” (001) and “the ability to think a situation

through logically” (037). The third response was just another mention of it:

“Recognition of the need to apply common sense” (036).

Some respondents mentioned attitude and various personal qualities as forming

part of airmanship, for example: “a combination of knowledge, discipline and

attitude toward flying” (025) and “attitude to safety and fellow aviators” (004). This

category also included other personal characteristics, for example: “Furthermore

good airmanship is a reflection of the quick thinking never panicking pilot, who

seems to have a dogged determination to keep his plane and contents in a safe

configuration” (003). There was also mention of a desire to do as good a job as

possible and having a sense of responsibility. None of the respondents specifically

mentioned the hazardous attitudes that are mentioned in the existing models and

listed as something to avoid, in their formulation of airmanship.

Several people mentioned the difficulty of defining airmanship, and also that it

is an integral or integrated part of flying.

Finally, the responses included the idea of setting priorities and managing the

flight, one of the human factors competencies for flight tests being introduced by

CASA (CASA, 2008), although only mentioned by five respondents overall. The

other CASA human factors competencies are lookout, situational awareness,

decisions and communication, and were all mentioned by respondents as being part

of airmanship as noted above, although none of the respondents listed all five.

Most important aspects of airmanship. Table 2.2 indicates the main concepts

raised by respondents when asked: “In your current aviation activities, are you

normally aware of airmanship as part of your operation? What would be the most

important things that come to mind?” (Question 6). Table 2.2 also indicates the

concepts mentioned in the examples of good and poor airmanship (Questions 9 and

10), as these are indicative of the airmanship related concepts that are relevant in

“critical incidents”, although this term was not specifically used.

47

Table 2.2

Number of Respondents who Mentioned Each Concept in Questions 6, 9 and 10 (of N =40). Concepts Q6 Q9 Q10 Nil Response to Question 3 9 13 Number of Respondents 37 31 27 Kern concepts Judgement or decision making 1 2 0 Situational awareness 5 8 5 Knowledge 0 2 0 Self 1 0 0 Aircraft 2 0 0 Team 0 0 0 Environment 0 0 0 Rules, regulations 0 1 0 Checks / SOPS 9 0 0 Risk 1 0 0 Mission: Planning 8 1 3 Skill & Proficiency 0 1 1 Discipline 0 1 0 Concepts specific to Ebbage and Spencer/Edwards Attitudes: Hazardous 0 0 0 Professionalism 0 0 0 Self-improvement 0 1 0 Skills: Communication 3 8 6 Crew co-ordination 0 0 0 Interpersonal 0 0 0 Management flight deck 1 0 0 Self-assessment 0 0 0 Team leadership 0 0 0 Additional concepts (not in models) Lookout 9 1 0 Safety 1 0 0 Courtesy/ethics/respect 2 3 0 Attitude / personal qualities 2 1 3 Common sense 1 1 0 No shortcuts 1 0 0 Everything 4 3 0

Situational awareness and lookout are related concepts that were frequently

mentioned in responses to these questions, supporting the emphasis given to the

concepts in the earlier questions. Similarly there were frequent mentions of planning

as being important to airmanship. Standard operating procedures (SOPs) and checks

were also mentioned frequently. SOPs are somewhat related to rules and regulations

48

as a knowledge item but were listed as the organisational side of environmental

knowledge. Rules and regulations were mentioned only once. Again, there were few

explicit mentions of skill or discipline in response to these questions.

Communication, or lack of it, was a major issue raised in examples of good and poor

airmanship, raised at least as often as situational awareness. Four respondents refused

to specify any most important aspect, stating that all aspects are important, or that

everything should be included.

There were no mentions of teams or the skills of crew co-ordination, inter-

personal skills or team leadership, perhaps a reflection of the predominately single-

pilot GA background of most of the respondents. There were also no mentions of the

physical aspects of the environment, hazardous attitudes or professionalism.

When is airmanship important? The scenarios shown in Table 2.3 were

mentioned by respondents in answer to Question 7 (“In your current aviation

activities, during what phases of flight or events would you be most actively aware of

airmanship issues?”) and in the examples of good and poor airmanship (Questions 9

and 10). There was frequent mention of weather and fading light, especially in the

examples. The most frequently mentioned scenario was of flying in the circuit.

Landing and take-off is a similar phase of flight to circuits, also mentioned

frequently in response to Question 7. Mention of high traffic situations, most likely

to occur in the circuit and round aerodromes, adds to the impression that constant

awareness of where other aircraft are in relation to one’s own, and one’s own in

relation to the ground, is a major part of airmanship. Twenty-five percent of the

respondents stated that they were aware of airmanship at all times. Some of the

respondents, while giving an example of good or poor airmanship, did not specify

any event or phase of flight, these are shown in Table 2.3 as “non-specific”

responses.

49

Table 2.3

Number of Respondents who Mentioned Listed Scenarios or Phases of Flight in Questions 7, 9 or 10 (of total N=40). Scenario or Phase Q7(When) Q9(Good) Q10(Poor) Nil Response 1 10 13 Responses to Qs 39 30 27 Weather/light 2 9 6 Pre-flight/planning 5 0 3 Taxi/runway use 2 0 5 Circuit / CTAF 8 9 10 Traffic 5 2 1 Aircraft problem 0 3 1 Approach/landing 5 1 1 Take-off / landing 8 0 0 All the time 10 0 0 Agricultural flying 2 1 0 Non-specific 0 4 3 Other 4 2 0

Development of Airmanship

Inclusion of airmanship in initial training. The majority of respondents

stated that airmanship formed some part of their initial flight training. The questions

were designed to determine what happened in the air, as part of flying instruction

(Question 3), as distinct from what was included in briefs or ground instruction

(Question 4). Responses to Question 3 (“Was the development of airmanship a

specific part of the initial flight training you did? Please explain what this involved”)

are shown in Table 2.4. Responses to Question 4 are summarised in Table 2.5.

Table 2.4

Number of Respondents Answering Question 3 (Inclusion of airmanship in initial training; N=40) Answer N Yes 18 Not really, but... 6 Related activities 3 Skill related 2 No 8 Can’t remember 2 No answer 1

Eighteen respondents stated “yes”, some with added explanation, to Question 3

as shown in Table 2.4. Three respondents listed airmanship related activities such as

50

checks, lookout and procedures. A further six responses were along the lines of “not

really, but…” and included concepts to do with airmanship. Two respondents

indicated that airmanship was part of their initial instruction but that this was only

skill related or did not include human factor related concepts. This gives a total for

inclusion of airmanship (however defined) as 29, or 73% of the respondents. Eight

respondents indicated that airmanship was not addressed (20%). Of the remaining

three respondents, two indicated that it was so long ago that they could not

remember, one stating: “I cannot recall that airmanship was specifically discussed

when I was learning to fly in 1954. I sometimes wonder if common sense was more

expected in those days” (034). One respondent did not answer the question.

Some examples of positive responses are: “Yes. From the start my instructors

emphasised that airmanship was what defined a good pilot and they demonstrated

that right through my training: lookout, ground checks, aircraft handling and so on

were all part of each flight” (037) and from another respondent “in each brief,

airmanship was mentioned in regard to lookout, communications and aircraft

awareness” (013). Several responses to this question included mention of briefs, so

the distinction between Questions 3 and 4 was not necessarily picked up by the

respondents.

Question 4 had two parts. Question 4a asked “In your initial flight training,

was airmanship, or concepts related to airmanship, discussed in ground training or

pre- or post- flight briefings?” and allowed only a Yes or No response. Question 4b

asked for elaboration, “If so which concepts and what sort of discussions?”

There was a response of “Yes” from 67.5% of respondents (N=27) and “No”

from 25% (N=10) in answer to Question 4a. Three respondents did not answer the

question.

Nine respondents left Question 4b blank as they had replied “No” to Question

4a or did not answer. A further four wrote that airmanship was not a major item, or

only discussed in a general sense and another two respondents simply mentioned

discussion in the bar or “hanger talk” as the source of information about airmanship.

Table 2.5 indicates the concepts that were raised in response to Question 4b.

Most respondents included more than one idea in replying to Question 4b

(mean of 2.2 concepts per respondent). The most frequent concept mentioned was

51

lookout, listed by 14 respondents. Related responses were: traffic, communication or

radio use, and listen-out. Checks, procedures, planning and safety issues were also

mentioned. There were various other considerations raised as shown in Table 2.5.

Table 2.5

Number of Respondents (of total N=40) who Mentioned the Listed Concepts in Response to Question 4b Concept N Nil response 9 Respondents to Q4b 31 Lookout 14 Aircraft knowledge 7 Communication/radio/listen 6 Checks 6 Traffic 4 Skills: general aircraft flying 3 Safety Issues 3 Planning 3 Specific to lesson/changed 3 Procedures 2 Hand over/ take over 2 Courtesy 1 Common sense 1 Developing a flow of activity 1 Relationship with instructor 1 Appropriate decisions 1 Bar Talk 2 General sense, not a major item 4

There were ten aircraft or skill related comments including general handling (3

mentions) and aircraft knowledge included mention of: understanding of systems (2

mentions), using airworthy aircraft; aircraft awareness; taxi speeds and run up into

wind; aircraft performance; temperatures and pressures; and energy and fuel state.

Five of these respondents included only aircraft handling concepts in their response

to this question.

It appears that most respondents had some discussion of airmanship in their

initial training, but for some of the respondents airmanship was not discussed, or was

limited to aircraft handling or aircraft state issues alone. While these are part of

airmanship, a broader view of airmanship includes the human factors aspects of

52

situational awareness and judgement. Of the five respondents who only listed aircraft

or handling issues in response to these questions, only one really had the broader

view when the definitions given in the first questions defining airmanship were

reviewed. Two wrote a little about issues that could be interpreted as situational

awareness and the other two gave quite narrow, aircraft handling related definitions.

Two of these respondents had over 20 years of experience, but the other three were

instructed fairly recently, with 2.5, 3 and 6 years of experience as PPL holders.

Given that human factors has been part of the theory syllabus for the PPL, as human

performance and limitations, for approximately 20 years, it is concerning that there

are pilots who have not incorporated these concepts into their understanding of

airmanship. On the other hand, it may be that their instructors had not discussed

human factors in terms of airmanship, or airmanship in terms of human factors.

Looking at these responses from another angle, there were at least 12

respondents who made no mention of aircraft handling skills when writing about the

discussion of airmanship in their initial training. This may be a reflection of the

military training system of assessing technical skills and airmanship as separate

aspects of flight proficiency (Carrick, Graham, Healey & Pickard, 2008). Although

only three respondents indicated a background in the military, it may be that the

instructors of the non-military respondents had a military background, and so

perpetuate the separation of technical skill and airmanship.

Influences on the development of airmanship. When asked what had led to

the development of their own sense of airmanship, (Question 5: “Thinking about

your progress to date as a pilot, what do you feel led to the development of your

sense of airmanship?”) respondents mentioned experience, reading, developing a

sense of responsibility and the influence of the people around them. Table 2.6 shows

the numbers of respondents who mentioned various influences on their development

of airmanship.

53

Table 2.6

Number of Respondents (of a total N=40) who Mentioned the Listed Concepts in Response to Question 5 Concept N Nil Response 1 Respondents to Q 5 39 Experience 13 Desire to “Get it right’ 10 Self-preservation / fear 5 Teaching others 3 Common sense 3 Luck 1 Specific episode 1 Influence of other people (28) Training 6 Talking to others 6 Reading 6 Mistakes/experiences 5 Loss due to an accident 3 Club environment 2

As it was an open-ended question, several respondents listed more than one

influence on their development. Only one person did not provide a response. Some

examples are: “Fear of having an accident because of a minor mistake, and a desire

to be [as] prepared and professional as possible” (025); “Watching and talking with

other pilots to develop my own ideas of airmanship from examples around me I

identified with [and] being scared and not wanting to go through a similar experience

again” (022); and “Common sense and experience. Be ready for the unexpected”

(001). Another simply wrote “leaving a fuel cap off” (030).

Ongoing experience in aviation is seen as important in developing a sense of

airmanship, but also the observation of what others do, whether these others are

initial instructors, fellow aviators, or observation through reading. This supports the

comments of both Kern (1993) and Ebbage and Spencer (2003) that early instructors

provide a significant role model, but extends further to imply that learning from

others continues as the aviator develops post ab-initio training. There is also a clear

internal motivation to improve or excel, to be as professional as possible, among

some of the respondents. This is included by Ebbage and Spencer as one of the

54

attitudes that form the basis of airmanship and that should be encouraged in initial

training.

Practice of current instructors. Question 8 asked: “If you are currently an

instructor, what do you do to ensure that your students develop good airmanship?”

There were 16 instructors among the sample but 17 responses to this question, with

23 not applicable or nil answers. One instructor did not answer the question, and two

respondents who were not instructors did answer, and stated that they intended

becoming instructors and wrote about what they would do when instructing; it was

decided to include their responses. Responses are shown in Table 2.7.

Table 2.7

Number of Respondents (of a total N=40) who Mentioned the Listed Concepts in Response to Question 8 Concept N Nil response 23 Respondents to Q8 17 Provide a good example 8 Briefs, discussion, in-flight emphasis 5 Lookout 2 Get students to read 2 System for managing flight 1 Attitude and understanding 1 Encourage them to think 1 Instil points mentioned 1 Post-flight evaluation 1

Half of the responding instructors reported that they ensure the development of

good airmanship among their students by modelling good airmanship or providing a

good example themselves.

While the formula of instilling motivation, teaching the skills and providing

feedback may be part of the instructor training system and be recommended by both

Kern (1996) and Ebbage and Spencer (2003) as a structure for instruction, there were

only five respondents who wrote about briefs and/or discussion, and one who

specifically mentioned post-flight self-evaluation. Of the two instructors who

mentioned getting their students to read, one specifically mentioned the book,

Redefining Airmanship (Kern, 1996). The two mentions of lookout were to “keep

yourself alive and keep a good lookout” (032) and “I hit them (not very hard) when

55

they turn without looking (they all get hit at least once)” (025). Only five of the

respondents mentioned more than one concept.

Conclusion

While the concepts included by Kern (1996) form a useful structure for

consideration of airmanship, based on the responses to this survey it appears that

airmanship in Australian aviation is slightly differently defined than the Kern,

Ebbage and Spencer or Edwards model. There is little emphasis on discipline among

the current respondents, despite its central role in the Kern and Ebbage and Spencer

models of airmanship. Similarly, discipline is not mentioned by pilots in research

into what makes a good pilot (Nergård, 2011; Swauger, 2006). This may reflect the

primarily civilian samples compared with the military influences on both Kern’s and

Ebbage and Spencer’s models. By contrast, communication appeared as an important

concept, mentioned quite frequently. Communication is included in the Ebbage and

Spencer and Edwards models as part of the required skill set, but is only listed as a

knowledge item (under “Team”) by Kern. Communication is a central skill for

TEM/CRM (Helmreich, 2002; Kanki, 2010).

Just over half of the respondents wrote about situational awareness, and far

fewer respondents mentioned decision making (no-one used the term ‘judgement’).

Both are central concepts in all three of the existing models of airmanship, and

generally seen as important non-technical skills (Flin & Martin, 2001; Helmreich &

Foushee, 2010; Mavin, 2014; Orasanu, 2010; Thomas, 2004).

The occurrence of the concept of courtesy among responses to the definition of

airmanship questions shows an awareness of the importance of good interpersonal

interaction in this participant group. The frequency of mention of personal qualities

and attitudes as part of the overall airmanship construct is also interesting. It implies

that airmanship is seen as integral to an aviator’s personality and lifestyle, rather than

just something that one does, or has, when flying. Use of the term “common sense”

also reflects this idea of integration within the individual, and also the use of the term

within the CASA Flight Instructors Manual (2006). These are not concepts that have

attracted much attention in the research literature, except insofar as personality is

seen as a potentially useful predictor in selection of pilots (Campbell, Castenada &

Pulos, 2010; King, 2014).

56

All three existing models place importance on relevant knowledge in various

domains. Respondents to this survey also listed various areas of knowledge or simply

mentioned “knowledge” as a general reference. The emphasis among participants

was more on rules, regulations and operating procedures, and on planning, than on

the other areas of the models, with very little mention of knowledge of “Self” or

“Team”. Knowledge of one’s aircraft was mentioned more in later questions in the

survey than in the initial definition section, so while this aspect may not have come

immediately to mind when asked about airmanship, respondents did bring it into

responses as a part of airmanship later in the survey.

Based on the responses to this survey, it appears that airmanship is generally

included in student instruction in Australian aviation, but perhaps not consistently

across all training establishments, given that 20% of respondents stated airmanship

was not part of their initial training. In addition, individual experiences and the

quality of airmanship displayed by the people with whom the student pilot associates

is clearly influential in the development of airmanship, and continues to be important

after initial training.

The close examination of responses to Questions 3 and 4 indicates that where

the initial training did include airmanship concepts, it did not always reflect a truly

integrated view of airmanship. There appear to be three groups of respondents: those

for whom airmanship was presented as an integration of flying skills and human

factors concepts; those for whom airmanship referred only to flying skills and

aircraft knowledge related items; and those for whom airmanship referred to only the

human factors related concepts, not including technical flying skills. This group

appears to be the largest, based on consideration of the initial definition questions.

Responses to the initial question on the survey asking for a definition of airmanship

included only 10 mentions of skill or technical proficiency (see Table 2.1), thus 30

respondents (75% of the respondents) did not include these aspects in their initial

definition of airmanship and so probably consider airmanship to be predominantly

about non-technical or human factors aspects of flying.

A factor that may limit any generalisation from these results is the very small

sample. In addition, respondents were recruited from among those interested enough

in safety to attend CASA Safety Seminars, or to read Flight Safety Australia, and

thus may not reflect the views of the wider aviation community. Also it is possible

57

that the respondents were writing only the most important aspects of their reactions

to each question, and were not taking the time to explain their activities or views in

depth. Most respondents wrote no more than a few words or a single sentence in

response to each of the questions asked.

A further limitation of the survey is the lack of both military and airline pilots

among the respondents, and the possibility that the resulting view of airmanship is

not representative of all sectors of the aviation community. It was also evident that

the respondents did their initial training in different eras, and this might have some

bearing on their training in airmanship. With the median number of years of

experience at 20, half of the respondents did their ab-initio training before human

factors was part of the theory syllabus. Although extensive exposure through

magazine and CASA seminars, among other methods, have introduced such issues to

pilots generally over the years since then, airmanship may well not have included

consideration of human factors at the time of their training, before 1991.

Examinations in human factors were introduced from 1992 in the airline licence

(ATPL) exams and 1995 in the private and commercial level exams (PPL and CPL).

Human factors questions were included in the Helicopter ATPL examinations from

1998. Respondents who were trained to fly after 1991 should include more human

factors, or non-technical, related issues in their definition of airmanship, and mention

such aspects as included in their initial training more frequently, than those trained in

the earlier era, who may have experienced a more technically oriented, aircraft

handling approach to airmanship. This aspect of group comparisons is addressed in

Chapter 3, where a larger respondent group and more detailed demographic

questions in an extended questionnaire made such divisions more practical for

statistical analysis. The comparison of responses from military and civilian

background respondents as well as era of training are reported in Chapter 3.

58

59

Chapter 3. Airmanship Across Industry Sectors

The survey responses described in Chapter 2 matched some of the major

aspects of the Kern (1996), Ebbage and Spencer (2003) and Edwards (2013) models

of airmanship, while differing on others, and mentioned some concepts that the three

models did not include. Overall, situational awareness was frequently mentioned,

with judgement/decision-making appearing in responses about half as often.

Australian aviators included various knowledge items, with an emphasis on

knowledge of rules and regulations and planning rather than on other areas of

knowledge listed by Kern or Ebbage and Spencer.

Discipline is a central tenet of both the Kern (1996) and Ebbage and Spencer

(2003) models of airmanship, with Kern (1997, 1999) continuing to emphasise the

idea of discipline as the cornerstone of airmanship. The survey described in Chapter

2 found that Australian aviators did not place nearly as much emphasis on discipline,

in fact it was rarely mentioned by respondents. In contrast, communication was

mentioned fairly frequently by respondents to the survey, confirming Ebbage and

Spencer’s and Edwards’ inclusion of it as a skill, while Kern made only passing

reference to communication as part of the discussion of the components of

knowledge of team. Common sense, courtesy, attitude and various personal qualities

appeared among the responses to the survey, but were not considered in the models

of airmanship, except insofar as Ebbage and Spencer included interpersonal relations

among their skill set and Edwards included some personal attributes as “attitudes”.

The training of airmanship is a central concern in both Kern (1996) and Ebbage

and Spencer (2003). Of the respondents to the survey reported in Chapter 2, 20%

indicated that airmanship was not included in their initial training, with a few others

indicating that it was picked up more from “hanger talk” and meeting at the bar. The

flight instructors who responded, indicated that they are careful to set a good

example for their students, and many respondents mentioned that the example of

others had influenced their development of airmanship.

The small number of respondents and the lack of military and airline pilots

among them were identified as limitations of the survey described in Chapter 2.The

respondents were mostly from general aviation (GA) and while 16 were instructors,

many of the remainder appeared to be recreational fliers. There is the possibility that

60

the resulting view of airmanship is not representative, with such a narrow group of

respondents. On the other hand, given the difference in training between military and

civilian pilots, there may not be a convergence of ideas, and thus no wholly

representative view.

It was decided to undertake a second survey of Australian aviators, in an

attempt to gain a larger and wider sample of pilots, specifically aiming to include

military and airline pilots as well as GA pilots. Changes to the survey itself consisted

of more specific demographic questions to allow separation of pilots into a number

of career-related sub-groups (described below) and the inclusion of further questions

building from the findings of the first survey.

The broad aims of this survey were (1) to determine the definition of

airmanship by Australian pilots and (2) to examine whether, and how, airmanship is

included in the training of Australian pilots. In more detail, the first aim includes the

definition of airmanship by a wider cross section of Australian aviators and whether

there are any consistent differences between military and civilian pilots as well as

whether their views on airmanship have changed over the course of their careers in

flying. How closely the Australian definitions align with the models of Kern (1996),

Ebbage and Spencer (2003) and Edwards (2013) is also of interest. The second aim

addresses the inclusion of airmanship in ab-initio and on-going training and also

considers how pilots think that they developed airmanship. This information can be

assessed against the recommendations for airmanship training presented by Kern

(1996) and Ebbage and Spencer (2003).

Depending upon the number of respondents to the survey, a comparison could

be made between the pre- and post- introduction of human factors to the training

syllabus for pilots. It is possible that undertaking initial training in different eras

might have some bearing on training in and understanding of airmanship and the

inclusion of human factors aspects. In Australia, the Human Performance and

Limitations (HPL) part of the syllabus was only introduced into the theory syllabus

in 1991. HPL became part of the examination system in 1992 for the Airline

Transport Pilot Licence (ATPL) and in 1995 for the Private Pilot and Commercial

Pilot Licences (PPL and CPL). Human factors competencies were specifically

included in flight testing only from July, 2010, but a general “airmanship”

assessment has always been in the flight test. Thus, there is a basis for dividing

61

survey participants into training eras of before and after the introduction of human

factors (in the HPL syllabus) in 1991. An additional demographic question identified

when the respondents began their training.

There may also be differences in ideas about airmanship dependent upon

pilots’ initial training in military or civilian sectors. Views on airmanship may also

be influenced by the sector in which the pilot has been flying, as a military pilot, or

in civilian industry broadly divided into general aviation (GA) or airline flying.

There are also pilots who began their careers in the military then moved into the

airlines or into GA thereafter. This results in five sectors reflecting current and past

involvement in aviation: civilian airline, civilian GA, military followed by airline

flying, military followed by GA, and pilots who have only flown in the military.

Several additional questions were intended to identify whether initial training was

civilian or military, current sector of activity, and experience (years and hours) in

civilian and/or military flying.

The open ended questions remained almost the same as in the first survey, with

one change in order and two new questions. The questions about the definition of

airmanship remained, the questions about airmanship included in ground training and

in “in air” flight training were reordered so that the ground training question came

first. An additional question about continuing training was added, to gain insight into

the ongoing training of pilots in airmanship across their careers. To compliment this

question, another was added asking if the respondents’ ideas about airmanship had

changed over time, and if so in what way. This question was intended to capture any

change in views with increased exposure to human factors issues since 1991, and

also add to the information gained from the question about development of

airmanship.

In addition to the open-ended questions, two sets of rating and ranking

activities were added to the questionnaire. Based on the findings of the previous

survey, a list of 26 concepts was constructed. The respondents to the second survey

were asked to rate each concept for level of importance in airmanship and to rank the

five they felt to be the most important.

In this second survey, comments on the definition of airmanship were expected

to differ by the sector of background and current activity. Comments on initial

62

airmanship training were expected to differ according to the sector in which initial

training was undertaken and the era of training, and comments about ongoing

training by sector of current activity.

Method

Participants

Eighty-three pilots participated in this study. Of these participants, 51 had

completed their initial flying training through civilian operators, 11 of these

participants had then moved on into the military and therefore participated in military

ab-initio training as well, because civilian training is not considered equivalent to

military training. Thirty-two participants had completed their initial flying training

through the military sector, 17 of these moved on to civilian flying, but they did not

repeat ab initio training in the civilian environment as military training is recognised

and equivalent licencing is issued. Sixty-six of the participants reported an

instructional or training background, of these 35 had civilian, 9 had military, and 22

reported both military and civilian instructional or training experience. Seventy-

seven participants received their initial training in Australia, and six had taken their

initial training overseas (three in the UK, two in Canada, and one in New Zealand)

but were currently flying in Australia. There was a wide variation in both years of

experience (range: 2 to 50 years) and total flying hours accrued (range: 120 to 25 000

hours) among participants.

Aviation sector classification was determined for each participant by

examining their licence type, flying history (proportion of flying hours and years of

experience in civilian and military operations), and how the participant self-identified

when asked about their most recent aviation activity. Participants were divided into

two overarching groups: those with civilian experience only (40 participants), and

those who had a military background, but may currently be working either in military

or civilian environments (43 participants). Civilian pilots were classed as those who

fly in regular public transport airlines (airline sector) and those in general and

recreational aviation (GA sector) including charter and helicopter operations. Among

the participants who reported military flying experience, three sub-categories were

created: military+airline to cater for those respondents with a military background

who identified as working in the airlines sector; military+GA for those with a

63

military background who identified as operating in the general aviation sector; and

military-only. The latter group included participants who identified as military pilots

and whose civilian hours totalled less than 20% of their military hours. The reverse,

predominately civilian flying with military experience consisting of less than 20% of

their total, were included in the military+airline or military+GA sectors, depending

on the nature of their civilian experience. The distribution of participants across

aviation sectors indicated 21 respondents in airlines, 19 in general aviation, 21 in

military+airlines, 7 in military+GA, and 15 respondents in the military-only sector.

Measure

The on-line survey reported in Chapter 2 was modified as follows. Questions

were added to the demographic section of the survey to determine sector of aviation

operation. In the second section, consisting of open ended airmanship related

questions, additional questions about ongoing training resulted in a total of 13

questions including a last “any other comment” question. A new third part of the

survey included two questions. In the first of these, participants were asked to rate

each of 26 concepts (selected from the Survey 1 findings) with regard to their

interpretation of airmanship, on a five-point Likert type scale from “not important”,

through “marginal”, “moderate”, “quite” to “extremely important”. The second

question asked respondents to pick the five most important concepts from among the

same 26, and number them “1” for the most important down to “5” for the fifth most

important. A final question asked the participants if they had any further comments.

The modified questionnaire can be found in Appendix C.

Procedure

Three methods were used to recruit participants. The first strategy employed a

snowballing recruitment method, in which all pilots known to the researcher and

colleagues were emailed and asked to complete the on-line survey. Their assistance

was sought to forward the email to other pilots whom they knew. Secondly, the

researcher attended several aviation conferences and sought participation by

conference attendees and further distribution of the invitation through their

organisations. Lastly, assistance was sought from CASA to include the survey details

in CASA safety seminars (as was done for Survey 1). However, this did not occur, or

not widely, possibly due to changes in management and personnel since the first

64

study. Participation in the survey was voluntary. There were 198 hits on the survey

website of which 37 were disregarded as there were no responses recorded for the

survey. Data from 80 participants were excluded as they completed the demographic

details only. Two respondents completed the survey via a word document, off-line.

Analysis

The analysis process followed the sequence described in Chapter 2. Three

coders were involved, all had undertaken training in qualitative analysis and the use

of NVivo and all communicated frequently concerning the coding process.

Ultimately a coded data set was developed that all coders agreed was reliably and

validly coded. The data set was small enough that one of the coders was able to

check code all the data. PASW20 (IBM, 2012; Allen & Bennett, 2010) was used to

conduct statistical analyses on the demographic details from Section 1 (demographic)

and Section 3 (Questions 14 and 15) of the survey. Responses to the open ended

questions in Section 2 were entered into NVivo9 (QSR, 2011) used as the platform to

record concepts and themes raised in these responses. As in the previous study, 12

tree nodes were established to reflect the main concepts included in the Kern (1996,

2009) model. A further 9 tree nodes represented minor concepts from Kern and

aspects of Ebbage and Spencer’s (2003) and Edwards’ (2013) model not covered by

Kern. Additional free nodes were created as other concepts emerged in the responses.

A total of 91 free nodes were developed. PASW20 was then used to make

comparisons between node frequencies across the demographic groupings. A Chi

Square test of contingencies was used to compare the responses of pilots in civilian

flying with those of the pilots with any military background. Fisher’s Exact test was

used where there were one or more cells with an expected frequency less than 5,

unless otherwise noted.

Amount written by respondents in each industry sector. A word count was

conducted on each response. It is possible that any interpretation of concepts

mentioned by participants could be confounded by the amount written by participants

in each sector. The mean word counts for each sector were compared to address this

possibility. The civilian sector word counts were skewed, so a Mann-Whitney U test

was appropriate. This indicated that there was no significant difference between the

mean ranked word counts (U = 667.5, p = .079) for the civilian and the military

background participants (see Table 3.1). Given the possible division of the

65

participants into five sectors, word counts were checked at this level also. The

distribution of word counts in the airline sector violated assumptions of normality, so

a Kruskal-Wallis one-way ANOVA was conducted, which indicated that there were

no differences in mean word counts between sectors (H (corrected for ties) = 3.626,

df = 4, N = 81, p = .459, see Table 3.1). On the whole, the military-only background

respondents tended to write more in their answers, and the civilian GA participants

wrote the least, but the differences were not statistically significant.

Table 3.1

Comparison of Word Count Across Industry Sectors Civilian Military

Mean

409.1 507.4 Std Dev

293.0 286.6

Airline GA

Military +Airline

Military +GA Military only

Mean

449.1 364.9 504.4 495.0 517.5 Std Dev

337.2 236.0 303 142.8 326.1

Level of participants’ experience by industry sector. Another possible

confounding factor is the level of experience among respondents. It is generally

accepted in aviation that military pilots fly fewer hours than airline pilots. It costs a

lot of money to fly a fast military jet and the Defence budget does not allow for

anything near the maximum allowable hours for civilian pilots (800 hours per year,

with some exceptions) and military pilots move into desk jobs as their rank increases,

reducing the amount of flying undertaken. A comparison of hours of experience for

military and civilian groups by means of a Mann-Whitney U test (used because the

distributions are skewed) indicated that there is no significant difference in hours of

experience between the two groups (U = 663.5, p =.139; see Table 3.2 for means and

standard deviations). Comparison of years of experience was conducted by a Mann-

Whitney U test (the military group distribution was skewed,) and indicated a

significant difference between the two groups (U = 435.5, p < .001, see Table 3.2).

Although there is no difference in hours of flying experience, the military group was

significantly more experienced in terms of years of involvement with aviation.

66

The two groups are made up of participants in different sectors of aviation and

who are likely to fly different patterns of hours across a career. A military pilot who

moves on into airline flying will probably fly more hours than a military-only pilot

who moves into a desk job, and airline pilots will generally fly far more hours than a

GA hobby pilot. A five-sector examination of experience was carried out to

determine if these differences existed between participants in the different industry

sectors. The distribution of hours of flying experience was skewed for airline and GA

respondents, therefore a Kruskal-Wallis one way ANOVA was conducted. This

indicated significant differences (H(corrected for ties) = 36.937, df = 4, N = 81, p <

.001, large effect size (Cohen’s f = 0.926)) among the mean ranks of hours of

experience for the participants in the 5 sectors (see Table 3.2 for means and standard

deviations).The hours of experience of participants in the airline, military+airline and

military+GA sectors are not very different, but mean hours of participants in the GA

and military-only sectors are much lower.

Table 3.2

Comparison of Hours of Experience and Years of Experience Across Industry Sector

Civilian Military p value Hours Experience

Mean

8878.8

10371.2

.058 Std dev

7924.5

6233.5

Years Experience Mean

21.5

31.8

.001

Std dev

12.4

10.9

Airline GA Military +Airline

Military +GA

Military only p value

Hours Experience Mean

13307.5 3983.9 14512.6 10294.3 5161.3 .001

Std dev

7232.7 5455.5 4555.5 7616.1 2667.8 Years Experience

Mean

26.6 15.9 34.6 32.3 26.7 .001 Std dev

12.1 10.5 7.4 14.6 12.5

An examination of years of flying experience across the five sectors used a

Kruskal-Wallis one way ANOVA because, although distributions were not skewed,

the Levene Statistic (3.302, df = 4,76, p = .015) indicated that the assumption of

homogeneity of variances had been violated, and since the group sizes differ widely,

a non-parametric test should be performed. The Kruskal-Wallis test indicated

67

significant differences between the mean ranks of years of flying experience (H

(corrected for ties) = 21.474, df = 4, p < .001, Cohen’s f = .605, see lower rows of

Table 3.2 for means and standard deviations). Participants in the GA group had the

fewest years of experience; military only and airline only participants had about the

same, with military + GA and military + airline participants having far more years of

aviation experience. The latter two groups represent pilots who have engaged in two

careers, in military and in civilian flying. Therefore it is not altogether surprising that

they also have more years of experience, but not many more hours of experience as

military flying does not involve as many hours, as noted above. The GA group of

respondents has fewer years of experience and fewer hours – so is simply a less

experienced group than the others. GA flying generally does not result in as many

hours per year as airline flying.

Results - Definition of Airmanship

Six of the survey questions were designed to elicit the individual’s ideas about

the definition and components of airmanship. The first two questions in the survey

addressed this directly (Q1. What do you think airmanship is? Q2. Thinking of your

response to the first question, can you elaborate on each of the key words in your

response? For example, if you wrote ‘being safe’, write in more detail what ‘being

safe’ would involve). Questions 7 asked about awareness of airmanship and what

might be most important about it, while Question 8 directed attention to the phases of

flight and events that might influence airmanship (Q7. In your current aviation

activities, are you normally actively aware of airmanship as part of your operation?

What would be the most important things that come to mind? If you are no longer

flying, were you normally actively aware of airmanship as part of your operation?

Q8. During what phases of flight or events would you be most actively aware of

airmanship issues?). Questions 11 and 12 elicited examples of good and poor

airmanship, indirectly addressing what respondents considered constitutes

airmanship. (Q11. Please describe a flight, that you observed or know about, in

which good airmanship was displayed, including why you feel it was particularly

good airmanship. Q12. Please describe a flight, that you observed or know about, in

which poor airmanship was displayed, including why you feel it was particularly

poor airmanship).

68

The content analysis resulted in frequency tables of the number of participants

who mentioned particular themes: from Kern (16 nodes); from Ebbage and Spencer

or Edwards (but not in Kern, 9 nodes); or free nodes (not included in either model).

Re-analysis of the free nodes for these questions led to the combination of similar

concepts and a reduction of nodes to 28 from all 6 questions, and identification of 18

phases of flight or events mentioned in relation to airmanship. In the following

discussion, participants are cited by participant number when comments are quoted.

Differences between Sectors.

Initially, differences between participants grouped into the two overarching

sectors of civilian background (40 participants) and military background (43

participants) were considered.

Due to the low cell frequencies for many concepts, the frequency counts for

each node were combined across all six questions relating to the definition of

airmanship. Care was taken to ensure that each respondent was still only counted

once for each concept, so that repeated mention of the same concept by individual

participants in response to different questions did not weight the analysis. Table 3.3

shows the percentage of respondents who mention each of the concepts included in

the Kern model or additional concepts included in the Ebbage and Spencer or

Edwards models of airmanship, and the results of the comparison tests between the

two groups (Civilian or Military background) for each concept. Due to the large

number of comparisons, a Bonferroni adjusted critical value of p < .001 was used.

While an examination of the data reveals some apparent differences between

numbers of pilots in each sector who mentioned certain concepts, on examination

none of these were significant.

Reference to Table 3.3 shows that about twice as many military background

respondents (54%) as civilian (27.5%) mentioned aircraft handling skills as being

part of airmanship and three times as many (23.3% vs 7.5%) mentioned proficiency,

however these differences are not significant. None-the-less this is interesting given

that airmanship and technical skills are assessed separately in early military training,

implying that airmanship is something other than technical handling skill (Carrick,

Graham, Healey & Pickard, 2008). It appears that in the intervening years since their

69

training more than half of the military respondents have developed a more inclusive

view of airmanship.

Table 3.3

Percentage of Respondents who Mentioned Listed Concepts from Airmanship Models in Response to Combined Definition of Airmanship Questions.

Civilian Military

Significance values

Respondents 40 43

% %

p

Kern model 7.5 4.7 Judgement or decision making 52.5 58.1 Situational awareness 57.5 60.5 Knowledge: General 22.5 27.9 Self 22.5 25.6 Aircraft 35.0 58.1

Team 10.0 11.6

Environment: General 20.0 11.6

Legislative 32.5 55.8

Organisational 27.5 53.5

Physical 15.0 20.9

Risk 27.5 39.5

Mission: Planning 10.0 18.6

Skill 27.5 51.2

Proficiency 7.5 23.3

Discipline 15.0 23.3

Additional Ebbage & Spencer or Edwards concepts Attitudes: Hazardous 10.0 18.6

Professionalism 20.0 11.6 Self-improvement 5.0 0.0 Skills: Communication 37.5 30.2 Crew co-ordination 27.5 25.6 Interpersonal 7.5 2.3 Management flight-deck 12.5 16.3 Self-assessment 7.5 11.6 Team leadership 15.0 7.0

Table 3.4 shows the mentions of additional concepts by respondents. This table

includes a count of the use of either of two common phrases that occurred in

responses. There is a standard definition of airmanship used in the military: “safe and

efficient operation of a flight, both in the air and on the ground”. Another comment

that was made several times was to refer to “the saying that airmanship is the

70

exercise of superior judgement to avoid the use of superior skills”. These statements

were only made by military sector pilots, and appear at the top of Table 3.4.

Table 3.4

Percentage of Respondents who Mentioned Listed Additional Concepts in Response to Combined Definition of Airmanship Questions.

Civilian Military Significance values

Number of Respondents 40 43

% %

p

Military definition 0.0 23.3

.001 Superior judgement/skill 0.0 9.3

Other concepts: Safety 72.5 76.7 Personal qualities 52.5 41.9

Efficiency/effective 32.5 39.5 Experience 22.5 37.2 Common sense 12.5 37.2

Courtesy/consideration 40.0 11.6

CRM 7.5 20.9 Care for aircraft 12.5 14.0 Responsibility 15.0 11.6 Foresight 5.0 18.6 Thinking 17.5 7.0 Briefing/discussion 5.0 16.3 Act proactively 7.5 11.6 Time management 10.0 9.3 TEM 12.5 7.0 Cognitive capacity 5.0 9.3 Normal ops not aware 5.0 7.0 Gut feeling/ 6th sense 2.5 9.3 Human factors 7.5 4.7 Organisational culture 10.0 2.3 Prioritising 5.0 7.0 Checks 5.0 4.7 Management of

automation 7.5 0.0 Mistakes 5.0 2.3 Documentation 2.5 2.3 Examination 2.5 2.3

71

The only significant difference between the two groups of respondents was the

use of the military definition (χ² (1, N = 83) = 10.577, p < .001, effect size φ = .37 a

medium effect), which was only reported by military background respondents.

Table 3.5 indicates the phases of flight or events mentioned by respondents.

Three significant differences were detected between responses of civilian and

military background participants.

Table 3.5

Abnormal or emergency events were mentioned by very few civilian

background respondents (7.5%) while military background respondents brought

these up much more frequently (44.2%), the effect size being medium to large (χ² (1,

N = 83) = 14.32, p < .001, w = .42). Abnormal or emergency events were only

mentioned in response to questions 7 and 8, which asked about what phases of flight

Percentage of Respondents who Mentioned Listed Events or Phases of Flight in Response to Combined Definition of Airmanship Questions, by Sector.

Civilian Military Significance

values Number of Respondents 40 43

% % p Phase of flight or event

All the time 47.5 51.2 Weather/visibility 27.5 53.5 Departure/arrival 32.5 30.2 Traffic 32.5 27.9 Abnormal or Emergency 7.5 44.2 .001

Pre-flight/preparation 17.5 30.2 Altered plans 12.5 20.9 Fuel 15.0 16.3 Ground operations 20.0 9.3 Pressure 2.5 25.6 Circuit 12.5 7.0 Fatigue 10.0 9.3 Landing (and approach) 7.5 7.0 Training 0.0 9.3 Workload 7.5 2.3 Bird strike 5.0 2.3 ATC 0.0 4.7 Automation 5.0 0.0 Stress 5.0 0.0 Cruise 2.5 0.0

72

pilots’ were most aware of airmanship and in response to the questions asking about

good and bad examples of airmanship, and responses were evenly split between the

two questions..

Overall, there were very few significant differences between the civilian

background and military background respondents. This may be because there are not

very many pilots in each group or in fact civilian and military pilots do not raise

different concepts. There is also some confounding between the groups, as some of

the military background respondents have also worked as airline pilots (after their

military career). There were not enough pilots in each sub-grouping to consider

comparisons at that level

Comparisons with Models

As there were so few significant differences in the mention of themes between

the two groups (civilian vs military background) in response to the definition

questions, the content of the definition responses can be discussed for all the

respondents together.

Kern model. The 15 concepts representing the Kern (1996, 2009a) model of

airmanship were mentioned by many respondents, although no-one listed all the

themes. Five respondents indicated knowledge of the Kern model itself, referring to

it in their responses. Table 3.6 shows the total percentages of participants who

mentioned each of the Kern, Ebbage and Spencer (2003) or Edwards (2013) model

concepts in the definition questions.

A Chi square goodness of fit on the total frequencies across Kern themes tested

whether there were significant differences in the frequencies of mention. This was

statistically significant (χ² (14, N = 81) = 95.039, p <.001). The index of effect size,

Cohen’s w was 1.08, which can be considered large. The test confirms that the

themes relating to the Kern model of airmanship were mentioned by significantly

different numbers of respondents, rather than being mentioned equally.

Situation awareness and judgement/decision-making were mentioned most

frequently, by more than 50% of participants. Knowledge of aircraft, legislation and

organisation, and risk were mentioned quite frequently, by between 30 and 50% of

73

participants. Of the Ebbage and Spencer/Edwards additional concepts,

communication was most frequently mentioned, by 33.7% of the participants. With

the exception of knowledge of self (mentioned by 24.1% of participants) the least

frequently mentioned concepts seem to be those that relate to the self or the team,

including knowledge of the team and team leadership (each 10.8%), self-assessment

(9.6%), interpersonal skills (4.8%) and self-improvement (2.4%).

Table 3.6

Percentage of all Respondents who Mentioned Listed Concepts from Airmanship Models in Response to Combined Definition of Airmanship Questions.

Total

Number of Respondents

83

%

Kern model

6.0 Judgement/decision

making

55.4 Situational awareness

59.0

Knowledge: General

25.3 Self

24.1

Aircraft

47.0 Team

10.8

Environment: General

15.7

Legislative

44.6 Organisational

41.0

Physical

18.1 Risk

33.7

Mission: Planning

14.5 Skill

39.8

Proficiency

15.7 Discipline

19.3

Ebbage and Spencer or Edwards Additional Concepts

Attitudes: Hazardous

14.5 Professionalism

15.7

Self-improvement

2.4 Skills: Communication

33.7

Crew co-ordination

26.5 Interpersonal

4.8

Management flight-deck

14.5 Self-assessment

9.6

Team leadership

10.8

74

Situational awareness was the most commonly mentioned theme, closely

followed by judgement or decision making, these being Kern’s capstone outcomes.

Frequently, the respondents simply listed “situational awareness”, while others gave

a definition such as: “Situational awareness: Having enough brain space to fly the

aircraft and consider all external aspects of the aviation environment enabling

yourself to predict the aircrafts position and state with enough time to avoid 99% of

hazards” and “Situational awareness is the ability of the pilot to be able to know:

where his aircraft is in relation to airports, other air traffic, mountains, adverse

weather and range from the remaining fuel available” (military respondents, 038 &

090); “Situational awareness - appreciating what is going on around you, what risks

and threats exist, what effect your actions can have” and “Situational Awareness = In

simplest terms where are we now and where are we going to be at some time in the

future, this process starts before flight planning continues throughout the flight and

concludes when the last post flight issue has been dealt with” (from airline

respondents, 032 & 040). One GA respondents remarked “the ability to see changes

in the operational environment, know how the changes will impact your flight

progress and then take actions to adjust aspects of the flight to maintain expected

operational margins” (024). Those respondents who gave a definition of situational

awareness seemed to have a good grasp of the meaning according to one of the

standard definitions “the perception of the elements in the environments within a

volume of time and space, the comprehension of their meaning and the projection of

their status in the near future” (Endsley, 1988, cited in Endsley, 1999, p258).

Kern’s model refers to judgement, and while some respondents used that term,

in general respondents reported that decision making was part of airmanship. Some

made direct reference to judgement e.g. “I regard the outcomes of airmanship as

increased situational awareness, the development of sound judgement, and the

correct employment of operational risk management” while others were more

indirect “The captain's job is to choose the best solution when faced with lots of

options about weather, maintenance, options before and during flight” (both military

respondents, 080 & 102). Other comments on this theme include: “The ability to

consider all factors involved with operating the aeroplane and make positive

decisions to maintain safe operations at all times” (156, an airline respondent) and

“making effective decisions regarding risks and hazards” (178, a GA respondent).

75

The “Pillars of Knowledge” did not get equal mention by respondents.

Knowledge of aircraft was most frequently mentioned of the “Pillars”, with

significantly more military respondents placing more emphasis on this aspect of

knowledge. Some examples of comments made include: “essential knowledge

includes aircraft”, “the serviceability state of the multitude of systems in the

aircraft”, “knowledge of aircraft capabilities and systems” (all from military pilots,

025, 090, 126) and “having a complete and thorough knowledge of the aircraft

technical systems, performance capabilities and its handling characteristics in the

operational environment” (009, an airline respondent), “knowledge of the aircraft

performance envelope, aircraft systems...” (024, a GA respondent).

Knowledge of the legislative and organisational environments was also

frequently mentioned, and again more often by military than civilian sector

respondents. Comments along these lines were generally listed clearly, such as:

“knowledge of rules (SOPs)”, “compliance with regulations, rules, procedures”,

“Knowledge of orders and instructions” (from military respondents, 025, 090 &

126); “Operating an aircraft in a professional manner having regard for rules of the

air and Company Standard Operating Procedures”, “Considering rules, procedures”

(from airline respondents, 058 & 039) and “good knowledge of the rules”,

“Respecting and adhering to rules and regulations” and “knowledge of air law and

rules to conduct a safe flight from start to stop” (from GA respondents, 054 & 113).

Respondent’s comments about knowing their own capabilities or being aware

of their own limitations were coded as “knowledge of self”. These were fairly evenly

distributed between civilian and military background respondents.

Risk was mentioned somewhat frequently by 33.7% of respondents overall.

Mention was made of knowledge of risks, risk assessment or application of risk

management. Sometimes this was elaborated, for example: “An aviator has to

understand risk. An aviator has to understand severity, frequency, likelihood and

consequence. An aviator has to understand the finality of death” (an airline

respondent, 009) and from a military respondent: “Risk Management. This is the

what-if side of airmanship. We continually, both formally and informally, assess the

outcomes of any hazard we encounter. We compare the outcome with the likelihood

of the hazard will manifest itself and we make a decision based upon the risk of the

occurrence and the possible, or expected, outcome” (165).

76

The physical environment also mentioned relatively infrequently, by 18.1% of

respondents. These were usually references to terrain or meteorology in terms of

situational awareness or in examples of good or poor airmanship. “Mission”

(planning) was also mentioned infrequently (14.5%), with comments on preparation

and flight planning included in the concept node when coding.

Least often mentioned of the Knowledge items, was “Team” (10.8%). One

respondent (151), who referred to Kern, included “knowledge of the team (other

crew, ATC, engineering, cabin crew) in so far as what their roles and responsibilities

are” in his comments on airmanship. Otherwise, mention of “team” or “crew”

tended to be part of list of considerations.

Of the “bedrock foundations”, skill (aircraft handling) was mentioned most

frequently overall (39.8% of respondents), and more often by military (51.2%) than

civilian (27.5%) respondents. This is interesting considering that in their training,

military cadets are marked on preparation, technical skills and airmanship as separate

items (Carrick, Graham, Healey and Pickard, 2008). However the respondents to the

survey were generally very experienced, and had perhaps developed a more

integrated view of airmanship. That civilian respondents did not mention technical

flying skills very frequently may reflect the separation of technical skills and human

factors in the theory and testing for pilots licences and emphasis on human factors in

aviation publications, and their knowledge that the survey was being run from a

School of Psychology (stated in the introduction to the survey). Proficiency was also

differentially mentioned, more by military respondents, but overall by only 15.7% of

respondents. Discipline was mentioned by 19.3% of respondents, quite low given the

emphasis placed on discipline in Kern’s writing (Kern, 1997). Examples of

respondents’ comments coded as discipline related include: “everyone has different

abilities and comfort levels, and we must have the ‘self-discipline’ to adhere to our

‘personal limits’”, “Discipline - adherence to rules and procedures; self-discipline”,

and “Obeying the rules” (046, 086, 138).

Ebbage and Spencer and Edwards models. The additional concepts found in

the Ebbage and Spencer and Edwards models were not mentioned with equal

frequency by respondents, but there were no significant differences between the

civilian and military background respondents. Communication was mentioned most

frequently of the Ebbage and Spencer concepts, by 33.7% of respondents. Comments

77

included: “the ability to effectively communicate such that messages are clear,

concise and related”; “being prepared to share information” (airline pilots, 176, 032);

“Good communication – clear communication; English language proficiency;

readback; for multi-crew ops – seeking feedback, creating an environment where

superiors encourage input from their subordinates” (a GA pilot, 035); “listen to the

rest of your crew” (military+airline, 191); “airmanship involves taking action or

speaking up when something is observed to be out of place...” (military+GA, 200);

“good communication among the crew was essential” (military-only, 110).

Crew co-ordination was raised as part of airmanship by 26.5% of the

respondents, including mention of crew resource management, teamwork and

working well with colleagues (airline pilots); “working together with other crew

members, non-dictatorial management style” (a GA pilot, 035); “timely and

appropriate guidance to the crew” (military+airline, 091) and “management of the

crew resource” and “the Captain’s ability to employ all crew members and stake

holders” (military-only, 090, 155)

Professionalism was mentioned as part of a list of desirable qualities by 15.7%

of respondents, also including: “airmanship is the professional application of

everything that you have learned...” (military-only, 046) and “operate in a

professional manner” (airline, 058).

Management of the flight deck was mentioned directly or indirectly, by 14.5%

of respondents including comments like “organisation in terms of the flight” (GA,

063) and “flight management – approach planning to minimise crew workload during

the approach” (military+airline, 013).

Hazardous attitudes were included in descriptions of airmanship by 14.5% of

respondents, sometimes specifically (‘avoiding hazardous attitudes” by a GA pilot,

178) or by the use of examples, such as: “it seems helicopter pilots have very

cowboy attitudes’(GA pilot, 037);‘cowboy and cavalier practices are unacceptable”

(military+airline, 190); “owner trying to impress potential buyer” (military+GA,

076); and “beware of “continue-on-itis”’(military-only, 046).

Team leadership was listed among airmanship attributes by 10.8% of

respondents, such as “very aware of team skills and leadership” (military+airline,

034) with few elaborations however there was one example of good airmanship that

78

noted good leadership in that having been asked to conduct a flight in marginal

weather, the flight leader decided not to fly and under pressure to change his mind,

“stood his ground and displayed the kind of leadership that is second to none”

(military-only, 046)

Self-assessment was not frequently mentioned, only 9.6% mentioned this, and

only 2.5% specifically wrote about self-improvement. Examples are “Experience

allows a pilot to realise his own as well as other people’s limitations” (military+

airline, 089); “airmanship is what causes us to continually look for improvement”

(military+GA, 055); “post flight personal review of performance” (military-only,

025).

Other concepts. Table 3.7 shows other concepts raised that were not included

in any of the models and the percentage responses among all respondents. Safety was

raised as part of airmanship by the majority of respondents (74.7%) and to a rather

greater extent than in the earlier survey (62.5%, Table 2.1). While safety is not a

formal part of the airmanship models, it is stated as the aim of airmanship, and it is in

this context that it was mentioned by the respondents.

Personal qualities were frequently mentioned (by 47% of respondents). This

concept included such comments as “being neither a fool nor a coward” (009),

patience, selfless attitude, conscious of others limitations, integrity, honesty,

“underlying values and attitudes of the aircrew” (111) and “to develop good

airmanship you have to be passionate about the industry” (140).

Running an efficient and/or effective flight was mentioned as being part of

airmanship by 36.1% of respondents. One respondent defined this as: “Efficiently

conduct a flight: pilot the aircraft from A to B in the minimum amount of time, using

the least amount of fuel, and ensuring the minimum amount of stress on the aircraft”

(100). Another explained this as “Operating the aircraft so as to achieve the required

outcomes using the least amount of resources that could reasonably be expected”

(111).

79

Table 3.7

Experience was mentioned in the definition of airmanship by 30.1% of

respondents. Examples of mentions of this concept include: being aware of the

experience or lack of it among other airspace users; application of knowledge and

experience to a given situation; and “store the experience away so it becomes part of

the memory bank of airmanship” (166).

Common sense was mentioned by 25.3% of respondents overall. Some

examples from military background respondents include: “common sense application

Percentage of Respondents who Mentioned Listed Additional Concepts in Response to Combined Definition of Airmanship Questions. Total Number of Respondents 83

%

Other concepts: Safety 74.7

Personal qualities 47.0 Efficiency/effective 36.1 Experience 30.1 Common sense 25.3 Courtesy/considerate 25.3 CRM 14.5 Care for aircraft 13.3 Responsibility 13.3 Foresight 12.0 Thinking 12.0 Briefing/discussion 10.8 Act proactively 9.6 Time management 9.6 TEM 9.6 Cognitive capacity 7.2 Normal ops not aware 6.0 Gut feeling/ 6th sense 6.0 Human factors 6.0 Organisational culture 6.0 Prioritising 6.0 Checks 4.8 Management of automation 3.6 Mistakes 3.6 Documentation 2.4 Examination 2.4

80

of hazard analysis / risk management”; “Common sense compliance with regulations,

rules, procedures” (086); “In many respects, I think Airmanship is little more than

formalized common sense, but in an aviation context. During my own training I

recall the phrase ‘Common Sense and Airmanship’ being used repeatedly. In

hindsight, I think it was a redundant phrase!!” (099); and “An active form of

‘common sense’ which is a mix of experience, training, attitude and perspective

which informs decisions and affects the interpretation of incoming information”

(124).

A longer explanation came from a military pilot:

Aviation is a very regulated and rule based environment, however there are situations that arise that will require those rules and regulations to be broken. This by its nature is an exceptional violation and would require post flight de-brief with all key players, ie the regulator, the AOC holder etc. Example: Arrive at isolated destination with adequate fuel reserves, weather is below ILS minima and lasts for longer than your fuel reserves allow. Rule says you can't descend below DA however once you burn fuel to minimum it is now time to fly the ILS to the ground in control vice running out of fuel and crashing out of control (038).

An airline respondent wrote “A combination of situational awareness, common

sense and common courtesy. Common sense - appreciating that the safest way to

complete a task will ultimately be the most efficient” (032). A GA respondent

included: “Respecting and adhering to rules and regulations whilst realising common

sense can sometimes have a significant influence over an operation” (113). In general

then, common sense is used to modify the idea of conformity and being overly rule-

bound, and to inform the decision-making process.

Courtesy and consideration for others was also mentioned by 25.3% of

respondents overall, and was slightly (not significantly) more frequently mentioned

by civilian than military background pilots and rather more frequently by GA

(47.4%) than airline pilots (33.3%) (again, not significantly) if the detail is examined.

This is appropriate given the environment in which GA pilots operate: in and out of

small airfields and having to interact with other operators directly, more-so than

airline pilots, who interact mainly through ATC using formalised phrases. The

differences are understandable given the environment in which GA pilots operate: in

and out of small airfields and having to interact with other operators directly, more-

81

so than airline pilots, who interact mainly through ATC using formalised phrases.

Some examples of the comments made are: “common courtesy – appreciating that

other airspace and airport users are also wanting similar outcomes and being

prepared to share information and facilities” (airline, 032); “politeness – considering

others and how you like to be treated” (airline, 039), “considerate of other airspace

users [but] you would not be able to be considerate if your flight safety was in

question” (airline, 184); “considerate – ensuring all parties can achieve their own

objectives without outside influence” (GA, 116); “treat others the way you like to be

treated” (military+ airline, 191). Courtesy is thus an expression of how the

relationship with other pilots should be conducted. A more detailed comment is:

Courtesy - Acting in a manner which creates a good outcome for all users of the air, not only yourself, no matter how to the rule it is. Definitely not an excuse to break a rule, rather think of another way in which to legally and efficiently make a situation better for all concerned. E. g. Extend downwind for another 30 seconds or so, to enable the guy on the ground who's been waiting to take off for some time, to get away safely (GA, 054);

Crew resource management (CRM) was included in responses by 14.5% of

respondents, overall. Rather more military than civilian pilots mentioned it, but this

was not a significant difference. Some examples are: “airmanship (for me) was

brought clearly into focused summary with the introduction of CRM training”

(military+airline, 034); “I consider CRM principles to be an element of airmanship”

(military, 080); “modern training refers to airmanship as cockpit resource

management” (military+ airline, 096). A military participant (169) commented

“Well I am of the opinion that I have never observed a significant lapse in

airmanship. CRM on the other hand...”. Thus he is implying that CRM is not part of

airmanship.

Comment about caring for the aircraft was included by 13.6% of respondents

as being part of airmanship. This included comment such as “operating the aircraft so

that no undue stress is placed on the airframe or engines” (military+GA, 111) and

“don’t bend or break the aircraft” (military+airline, 099). The same proportion of

respondents indicated that responsibility is important, some stating individual

responsibility such as “accountability for your actions” (military-only,165) or

“individual responsibility” (airline, 051), and another suggesting “ultimate

responsibility falls on the supervisors at Squadron level” (military-only, 046), while a

82

final person saw it as a shared obligation “owning up to your actions and if need be

taking responsibility for others actions to ensure the outcome is safe” (airline, 176).

Foresight was brought up as a separate concept by 12.3% of respondents. This

included comment such as “rigorous forward thinking” (military-only, 165), and

“ability to foresee problems” (military+airline, 088). Comment about thinking was

mentioned as frequently, some examples are: “a constantly evolving or changing

environment through which you intend to fly requires constant adaptation,

consideration, adjustments and decisions” (military+airline, 088) “their decision...

demonstrated great thinking “outside the box” (military+airline, 099) and “poor

airmanship is really about not thinking efficiently” (airline, 026).

Briefing or discussion among crew was mentioned as part of airmanship. A

military+GA respondent wrote “briefing other members of the crew and passengers,

to clarify their duties and encourage them to have a positive input” (101), while

others simply mentioned the concept.

The remaining concepts were brought up by fewer than 10% of respondents.

The range of ideas that were included in the descriptions of airmanship indicates the

lack of an overarching consistent definition or interpretation of the term. However,

the underlying theme of ensuring safe flight was certainly present, even if not clearly

verbalised.

Phases of flight or events that occur in flight were brought up in relation to

direct questions or in relation to the examples of good and bad airmanship, and these

are listed in Table 3.8. The interesting outcome from asking respondents when they

were most aware of airmanship was that nearly half (49.4%) challenged the question

by responding that they were aware of airmanship at all times during flights. Some

respondents expanded the scope of the question to indicate that pre-flight preparation

(24.1%) and ground operations (14.5%) are also times during which they are aware

of airmanship. Weather and visibility were frequently mentioned in response to the

question about examples of good and poor airmanship, 41% of respondents included

some comment about weather or used examples that included a weather related

event. Respondents listed several phases or circumstances of flight that are known to

involve more risk or higher workload: when there is increased traffic (30.1%), in the

circuit (9.6%), during approach and departure (31.3%) and landing and approach

83

(7.2%). Concern regarding the management of fuel featured as well, another critical

feature of flight. Individual factors of feeling under pressure, high workload or stress

as well as fatigue were mentioned by some respondents.

Table 3.8

Percentage of Respondents who Mentioned Listed Events or Phases of Flight in Response to Combined Definition of Airmanship Questions. Total Number of Respondents 83

%

Phase of flight or event All the time 49.4

Weather/visibility 41.0 Departure/arrival 31.3 Traffic 30.1 Abnormal or Emergency 26.5 Pre-flight/preparation 24.1 Altered plans 16.9 Fuel 15.7 Ground operations 14.5 Pressure 14.5 Circuit 9.6 Fatigue 9.6 Landing (and approach) 7.2 Training 4.8 Workload 4.8 Bird strike 3.6 ATC 2.4 Automation 2.4 Stress 2.4 Cruise 1.2

Concept Rating and Ranking

The final two questions of the survey asked participants to rate (Question 14)

and to rank (Question 15) the 26 concepts related to airmanship selected from the

Kern model and from frequent responses found in the earlier survey (reported in

Chapter 2). The concepts can be seen in the questionnaire at Appendix B.

Three participants did not complete either question. A further three omitted

Q15 and another 5 completed it incorrectly, generally by using only a few numbers

repeatedly instead of just 1 to 5. Thus there were 80 responses to Q14 and 72 to Q15,

84

distributed over the sector of background and of current operation as shown in Table

3.9.

Table 3.9

Number of Respondents in Each Sector of Background and of Current Operation who Responded to Question 14 and Question 15

Sector Q14 Q15 Background: Civilian 38 33 Military 42 39 Current operation: Airline 21 17 General Aviation 17 16 Airline and Military 20 17 GA and Military 6 6 Military 16 16 Total 80 72

Rating Question. Question 14 asked the participants to rate the importance of

the 26 concepts, with regard to their interpretation of airmanship, from “not

important”, “marginal”, “moderate” and “quite” to “extremely important’. One

participant used only “extremely important” for all except one of the 26 concepts.

Seventeen participants used only the top two ratings and only five participants rated

anything using the lowest rating. There appears to be an upward bias, rather than a

central one.

A comparison of ratings between the five sectors of background and current

activity, using a Kruskal-Wallis ANOVA, indicated no significant differences with a

Bonferroni adjusted critical value of p < .002.

A comparison of ratings (a Mann-Whitney U analysis) between the civilian

background and military background participants (combining the five sectors into

two) indicated that there was one significant difference, with a Bonferroni adjusted

critical value of p < .002. The significant difference in rating occurred for

communication with other aircraft (U = 502.0; Z = -3.104; p = .002, r = .347, a

medium effect). The civilian background participants rated the concept higher than

did the military background participants.

85

As there was only one significant difference, the sectors were combined to

allow an overall consideration of the ratings for the 26 concepts. A Friedman two-

way ANOVA indicated that there is significant variation in ratings over the concepts

(χ² = 461.731, df = 25, N = 80, p < .001). Figure 3.1 shows the distribution of the

ranking of the mean ratings, as calculated by this analysis.

Figure 3.1 The rank of mean ratings for the 26 concepts rated in Question 14

Situational awareness was rated highest, overall. There followed a group of

five concepts with approximately equal ratings, responsibility, discipline, judgement-

and-decision-making, and preparation. It is interesting to see responsibility and

discipline rated so highly, given that they were not mentioned very frequently in the

earlier “free recall” open ended questions. After fairly steep steps down from

common sense to medical fitness, the ranks decrease more gradually from crew

interaction to knowledge-of-the-regulatory/organisational-environment, and then

there is a tail of lower rated concepts from ability-to-operate-automation to courtesy.

86

Follow-up pair-wise comparisons with the Wilcoxen Signed Rank test and a

Bonferroni adjusted alpha of .0125 were made between situation-awareness and

responsibility, responsibility and risk assessment (in the upper slope), risk

assessment and communication-with-other-aircraft (in the lower slope) and between

communication-with-other-aircraft and ability-to-operate-automation (in the “tail’).

All four comparisons were significant.

Situation awareness (Mean Rank =18.93) was rated significantly more

important than responsibility (Mean Rank =17.5), T = 112, z = -2.524, (corrected for

ties), N-Ties = 14, p = .012. This effect can be described as large, r =.675.

Responsibility (Mean Rank =17.5) was rated significantly more important than risk

assessment (Mean Rank=15.4), T = 45, z =-2.803, (corrected for ties), N-Ties = 22, p

= .005, r =.598, a large effect. Risk assessment (Mean Rank =15.4) was rated

significantly more important than communication-with-other-aircraft (Mean rank =

12.18), T = 759.5, z = -2.956, (corrected for ties), N-ties = 45, p = .003; r = .441, a

medium to large effect. Finally, Communication-with-other-aircraft (Mean rank =

12.18) was rated significantly more important than ability-to-operate-automation

(Mean rank = 9.58), T = 300.00, z = -2.611, (corrected for ties), N-Ties = 45, p =

.009; r =.389, a medium effect. The overall significant variation in the ratings of

concepts is most likely because of differences between mean ratings across the range,

with some concepts rated alike, but others stepped apart, where several steps leads to

a significant difference in mean rating.

Ranking Question. Question 15 asked “If you had to pick the five most

important concepts, which would you pick?” Participants were asked to number their

top five from 1 (most important) to 5 (fifth most important), and to not number any

other concepts. As mentioned above, five participants felt that ranking only five was

too few and ranked more than five concepts; these participants were removed from

this analysis.

The resulting data was reversed scored, so that the most important concept now

had a rank of 5, down to the least important rank of 1 and the concepts not ranked

were given a 0. This resulted in a more meaningful ordinal scale.

87

A comparison of ratings between the five sectors of background and current

activity, using a Kruskal-Wallis ANOVA, indicated no significant differences, with a

Bonferroni adjusted critical value of p < .002.

A comparison of ratings (a Mann-Whitney U analysis) between the civilian

background and military background participants also indicated that there were no

significant differences with a Bonferroni adjusted critical value of p < .002.

With no significant differences, the sectors were combined to allow an overall

consideration of the ratings for the 26 concepts. A Friedman two-way ANOVA

indicated that there is significant variation in ratings over the concepts (N = 71, χ² =

402.549, df = 25, p < .001). Figure 3.2 shows the distribution of the mean rankings,

as calculated by the analysis.

Figure 3.2 The mean rankings for the 26 concepts ranked in Question 15

88

Follow-up pair-wise comparisons with the Wilcoxen Signed Rank test and a

Bonferroni adjusted α of .008 checked the differences between the three top-ranked

concepts and between concepts stepped along the distribution shown in Figure 3.2.

The latter comparisons were between risk analysis and discipline, risk analysis and

responsibility and discipline and crew interrelations. Of the six comparisons, four

were significant.

Situation awareness was ranked significantly higher than judgement/decision-

making with Z = -3.009 (corrected for Ties), N-Ties = 64, p = .003, 2-tailed, r = .376

(a medium effect). Forty-six respondents ranked situation awareness higher than

judgement/decision-making; 18 ranked judgement/decision-making higher than

situation awareness, and there were 8 ties (did not rank either concept). Situation

awareness was also ranked significantly higher than risk assessment: Z = -4.459

(corrected for ties), N-Ties = 62; p = .001, 2-tailed, r = .566 (a large effect). Forty-

five respondents ranked situation awareness higher than risk assessment; 17 ranked

risk assessment higher than situation awareness and there were 10 ties (did not rank

either concept).

The rankings of judgement/decision-making and risk assessment were not

significantly different, nor were the rankings of risk assessment and discipline. The

rankings of risk assessment and responsibility were significantly different, Z = -

2.673 (corrected for ties), N-Ties = 34; p = .008, 2-tailed, r = .458 (a large effect).

Twenty-six respondents ranked risk assessment higher than responsibility, 8 rated

responsibility higher than risk assessment, and there were 38 ties (did not rank either

concept). The rankings of discipline and crew interaction were also significantly

different, Z = -3.618 (corrected for ties), N-Ties = 22; p < .001, 2-tailed, r = .771, a

large effect. Nineteen respondents ranked discipline higher than crew interrelations,

3 ranked it lower and there were 50 ties (did not rank either concept).

The significant variation of rankings across the 26 concepts was partly due to

the high ranking of situational awareness and judgement/decision-making by many

of the respondents, and partly due to the differences between concepts at various

steps through the distribution shown in Figure 3.2.

Following Question 15, the final question asked for any other comment and

eight participants made some comment about the final two questions. They all

89

pointed out that it was difficult to separate the top five, several at some length. For

example: “How can you expect anyone to insert anything other than VERY

IMPORTANT in any of the boxes?” (042), “It is difficult as I would count a few of

these as equally important” (043) and “The foregoing concepts are inter-related, and

differing situational contexts may alter the assignment of importance. I would

include consistency, balance, established specialisation, a broad knowledge base, and

an enthusiasm for excellence.” (080). All but one of those that commented completed

the question.

Comparison among question types. There are some differences in the order

of importance ascribed to the concepts based on the three different styles of question:

open ended, rating and ranking. These can be seen most clearly in Table 3.10, where

the concepts raised in response to the combined definition questions and that were

either included in the Question 14 and Question 15 list or are similar to those

concepts, are ordered by frequency of mention in the left of the table, and the

Question 14 and Question 15 concepts are ordered by the Friedman analysis means,

in the centre and right of the table.

It is clear that situation awareness is seen as the most important aspect of

airmanship, with judgement/decision-making a close second, overall. Risk

assessment (or knowledge of risk) also maintained a fairly high profile across all

three question types. While discipline was not mentioned overly frequently in the

definition questions (half-way down the list), it is close to the top of the list when the

concepts were rated, and a bit lower for the ranking exercise but still near the top.

Similarly, common sense and preparation (included as mission planning in the

analysis of the open-ended definition questions) both moved further up the listing in

the last two questions. Knowledge of the environment was rated and ranked fairly

highly (in the top 10 of both Q14 and Q15) but was a bit lower down in the ranks in

the open-ended definition questions. Specific mention of the physical environment

and mention of the environment without specification were separately counted, and

combining the count would bring the open-ended definition questions ranking nearer

to the top.

90

Table 3.10

Airmanship Concepts Ordered by Frequency of Mention in the Definition Questions, Rating in Question 14 and Ranking in Question 15 Definition Questions N Question 14 Rate Question 15 Rank Situation Awareness 49 Sit Awareness 18.93 Sit Awareness 21.21

Judgement/DM 46 Responsibility 17.5 Judgement /DM 18.9

Know Aircraft 39 Discipline 17.42 Risk assessment 16.34

Personal qualities 39 Judgement/DM 17.38 Common sense 15.94

Know Legis (Regs) 37 Preparation 17.08 Preparation 15.63

Skill 33 Common sense 16.79 Discipline 14.69

Know Risk 28 Know Env 15.98 Know Env 14.48

Communication 28 Risk assessment 15.4 Know Aircraft 14.3

Crew co-ordination 22 Lookout 15.23 Proficiency 14.18

Common sense 21 Psych fitness 14.71 Flying skill 14

Courtesy 21 Know Aircraft 14.08 Team management 13.31

Know Self 20 Medical fitness 13.76 Responsibility 13.03

Discipline 16 Crew interrelations 12.97 KnowReg/Org 13

Know Env (Phys) 15 Team management 12.83 Personal attitude 12.68

Know Env(General) 13 Self-improvement 12.68 Comm Aircraft 12.46

Proficiency 13 Personal attitude 12.64 Psych fitness 12.07

Mission: Planning 12 Cooperation 12.27 Courtesy 12.06

Manage flight deck 12 Comm Aircraft 12.18 Lookout 11.75

Responsibility 11 Comm ATC 12.12 Crew interrelations 11.69

Know Team 9 Proficiency 11.86 Assertiveness 11.65

Team leadership 9 Know Reg/Org 11.66 Comm ATC 11.46

Self-assessment 8 OperateAutomation 9.58 Cooperation 11.46

Interpersonal 6 Comm Maint 9.48 Medical fitness 11.32

Manage automation 3 Flying skill 9.07 OperateAutomation 11.2

Self-improvement 2 Assertiveness 9.03 Self-improvement 11.18

Lookout 1 Courtesy 8.41 Comm Maint 11 Note: Know = Knowledge of; Legis = legislation; Regs = regulations, Org = organisation; Env = environment; Comm = communication with; Maint = maintenance

Flying skill, “Skill” in the definition questions, was mentioned frequently, in

the top 10, as it was in the Question 15 rankings. However in Question 14, flying

skill was not so highly placed, being down near the end of the list. Courtesy followed

a similar pattern, in the upper half of ordered concepts in the definition questions,

courtesy was at the end of the list in Question 14, but climbed to be in the middle

third of the Question 15 rankings. Communication was separated into three in Q14

and Q15, perhaps reducing its impact, as it dropped somewhat in the latter two

91

columns. Crew co-ordination (as coded in the definition questions) was nearest in

connotation to crew interrelations, which rated and ranked a bit lower than crew co-

ordination. Team leadership (in the definition questions) and team management (in

Q14 and 15) showed the opposite pattern, improving in rating and rankings.

Responsibility was not mentioned often in response to the definition questions, but

was near the top of the ratings, and about half way up the rankings.

The free recall in the definition questions led to a variety of different concepts

being brought up. Questions 14 and 15 listed only 26 concepts. The rating question

allowed every one of these 26 concept to be rated, and some respondents used more

of the available scale than others which may have meant that their negative (or less

positive) opinion of some concepts weighted the ratings disproportionately.

Meanwhile, in Question 15 only five concepts could be ranked, and perhaps a bit

more thought went into exactly which five concepts could be considered most

important. The graph shows flattening because all unselected concepts were assigned

“0”, thus being equally weighted in the analysis.

Conclusion – Definition of Airmanship

One of the aims of this study was to determine if there were differences in how

civilian and military pilots define airmanship. There were only two significant

differences between the civilian and military background participants, in terms of the

number of respondents mentioning each concept. Military background pilots

mentioned a specific military definition of airmanship while civilian pilots did not.

Significantly more military background pilots mentioned abnormal or emergency

events than did civilian background pilots. It is possible that more military

background pilots have experienced abnormal or emergency events, because military

flying pushes the envelope rather more than normal civilian flying. Overall, it cannot

be said, on the basis of this study, that there are significant differences in the

definition of airmanship between civilian background and military background

pilots.

The concepts included in the Kern (1996, 2009a) model of airmanship were

generally well represented in the free recall definition questions, although the

number of mentions differed significantly across the concepts. Situational awareness

and judgement/decision-making were most frequently written about by respondents.

92

Knowledge of the aircraft was the most frequently mentioned knowledge element,

although knowledge of the legislative and of organisational environments were also

frequently included in responses. Knowledge of the team was least often mentioned.

Knowledge was written about more as an application of knowledge than simply

possession of knowledge. The additional Ebbage and Spencer (2003) and Edwards

(2013) concepts were not as well represented, apart from communication and crew

co-ordination, with self-improvement and interpersonal skills least often mentioned.

Most frequently mentioned further concepts included safety, personal qualities,

experience, common sense and courtesy. Common sense tended to be used to

moderate the application of knowledge and decision-making, while courtesy was

associated with interaction with other pilots.

The rating and ranking questions supported situational awareness and

judgement/decision-making as the most important airmanship concepts. While

discipline was not mentioned very frequently in the free recall definition responses,

when it was available for rating or ranking it appeared high in the order of

importance. Preparation was similarly infrequently mentioned in the free recall

definition responses (coded as the Knowledge element: mission planning), but was

highly rated and ranked in the final two questions. Overall, the Kern model elements

were ranked as more important than any of the other concepts included in the

available options, with only common sense and responsibility intruding in the top

half of the rankings. The Kern elements were rated highly as well, again

responsibility and common sense were inclusions in the upper ratings, while

proficiency, knowledge of regulations/organisation and flying skill were low in the

order of ratings.

The concepts mentioned by Australian pilots and the importance placed on a

variety of airmanship related concepts, clearly supports those included in Kern’s

model of airmanship (2009a). However, the Kern model lacks reference to many of

the crew resource management concepts that were included by Ebbage and Spencer

(2003) and Edwards (2013) and that were mentioned by the respondents in this

study. Clearer reference to communication, the framing of knowledge items as

actively applied rather than passively held and inclusion of crew co-ordination and

team management should be part of any revised model of airmanship.

93

These results also support the importance placed on situational awareness,

decision making, risk assessment and flying skill (‘maintaining aircraft within

tolerances’) that are central to Mavin’s (2010) model for the assessment of pilots.

That these comments were so frequently mentioned supports the importance placed

on them by many researchers in the area of pilot performance (Flin & Martin, 2001;

Helmreich & Foushee, 2010).

Results - Training and Development of Airmanship

The survey included five questions that addressed the training and development

of airmanship as experienced by the respondents, and one question that asked any

respondents who were flight instructors or had a check/training role what they did to

ensure airmanship developed in their students. These questions were included to

address the second aim of this study, to examine how airmanship is included in flight

training and what contributes to the development of airmanship.

Initial Training

Questions 3 and 4 asked about the initial training of the participants. In order to

assess the responses to Questions 3 and 4, the participants were categorised by their

answer to the demographic question about the sector of their initial training and any

who were initially trained somewhere other than Australia (N = 6) were omitted from

this analysis. There were 37 participants whose initial training was in the civilian

sector, 28 whose initial training was with the military. A further 7 respondents who

were initially trained in the civilian sector but who then moved into the military (and

therefore started again in military ab initio training) clearly identified in Questions 3

and 4 that they were writing about their military experience and so were added to the

military-trained group, bringing that group up to 35 respondents. An additional 5

respondents who were initially trained in the civilian sector but then moved into the

military were removed from the analysis because it was not possible to tell which

sector they were actually referring to when asked about their initial training.

The participants were divided into “era of training” groups based upon when

they started their initial training (a question in the demographics section of the

questionnaire). They were placed in “before 1991” or “1991 and after” groups,

94

referring to the year in which human factors training was introduced to the civilian

Day VFR syllabus. It is possible that the introduction of human factors as a syllabus

area in the theory examinations system may have raised awareness of topics linked

with airmanship, such as situation awareness, decision making, communication and

teamwork, and that this awareness flowed on into flight training.

There is no equivalent change that occurred in military training, except perhaps

the move of ab initio training from Point Cook, Victoria, to Tamworth, NSW, and

the introduction of a civilian contractor to run the program (BAE Systems), however

part of the contract was to conform to the existing military training system. From

1986, all military cadets were required to take a degree at the Australian Defence

Force Academy before proceeding to flying training and this might be an appropriate

division point, but again, there was no official change in training activity at this date.

Ground Instruction – Question 3. Question 3 (a) of the questionnaire asked

“In your initial flight training, was airmanship, or concepts related to airmanship,

discussed in ground training or pre- or post- flight briefings?” and the survey was

set up so that they had to answer either “yes” or “no’. Table 3.11 indicates the

responses made by each group. Nearly all the participants indicated that airmanship

or airmanship related concepts were included in their early training, with a slightly

stronger response from the military trained groups. There was no significant

difference between eras within the civilian sector, nor within military sector, nor

when the sectors were combined. Combining eras, there was no difference between

sectors.

Table 3.11

Percentage of Civilian or Military Sectors Respondents by era of training (before 1991 or 1991 and after) who Responded to Question 3(a)

Civilian Military

Before After Before After

1991 1991 1991 1991

Number of Respondents 22 15 29 6

% % % %

Yes 86.4 86.7 93.1 100.0 No 13.6 13.3 6.9 0.0

95

Question 3 (b) asked “If so, which concepts, and what sorts of discussion?”

Table 3.12 summarises the concepts mentioned in the responses that match the

existing airmanship models.

Table 3.12

Percentage of Civilian or Military Sector Respondents by era of training (before 1991 or 1991 and after) who Mentioned the Listed Concepts From Airmanship Models in Response to Question 3(b)

Civilian Military

Before 1991

After 1991

Before 1991

After 1991

Number of Participants 22 15 29 6 Missing answers 4 2 4 0 Number of Respondents 18 13 26 6

% % % %

Kern concepts: Judgement or decision making 5.6 0 26.9 33.3 Situational awareness 33.3 7.7 19.2 50 Knowledge: General 5.6 0 7.7 0 Self 0 7.7 7.7 0 Aircraft 0 7.7 11.5 16.7 Team 0 15.4 0 0 Environment: General 0 7.7 7.7 0 Legislative 5.6 15.4 23.1 0 Organisational 0 0 34.6 0 Physical 0 0 3.8 0 Risk 5.6 7.7 3.8 0 Mission: Planning 5.6 7.7 15.4 0 Skill 5.6 0 15.4 0 Proficiency 5.6 0 3.8 0 Discipline 16.7 0 3.8 0

Additional Ebbage and Spencer or Edwards concepts: Attitudes: Hazardous 5.6 0 0 0

Professionalism 0 0 0 0 Self-improvement 0 0 0 0 Skills: Communication 22.2 15.4 3.8 0 Crew co-ordination 5.6 0 3.8 0 Interpersonal 0 0 0 0 Management flight-deck 5.6 0 3.8 0 Self-assessment 0 0 0 0 Team leadership 5.6 0 0 0

96

When considering ground training (the theory part of learning to fly) and the

pre and post flight briefings, most participants recalled that there was discussion of

the things that they consider to be airmanship. The details of what was mentioned

varied considerably, and cell numbers were generally too low for formal comparison

except in the case of situation awareness, judgement, safety, courtesy and lookout,

when frequencies were combined across eras of training to allow comparison across

sector, using a Pearson’s chi-square test of contingencies (with a Bonferroni adjusted

α such that p < .01).

Situation awareness was the second-most often mentioned concept, and there

was no significant difference between the sectors with regard to frequency of

mention. Mention of safety, courtesy and lookout did not differ significantly between

sectors. However, mention of judgement or decision making was nearing

significance, with more military-trained respondents mentioning than civilian-trained

respondents (χ² (1, N = 63) = 7.310, p = .013, w = .341, a medium effect).

The only civilian-trained respondent’s comment about judgement was the

comment “Mainly revolved around the concept of having the necessary skills to

make good decisions” (151). Some of the military-trained respondents simply listed

“making decisions” (197) or “applying sound judgement” (126) or similar, or else

expanded on the theme, like the comment “Post flight usually focussed on

performance on those aspects as well as any wrong decisions being made, or just the

lack of decisions which I believe is a more important skill to learn” (110).

Situational awareness was generally clearly stated by respondents, although

there were some who stated something else but the comment was coded as situation

awareness, such as “traffic monitoring” (035, civilian-trained), “interpretation of

incoming information” (124, military-trained) and “pre-flight tends to be listen and

lookout, traffic separation and a good understanding of where other aircraft are when

approaching an uncontrolled airfield” (200, military-trained).

Comments counted as general references to knowledge included the mention of

“knowledge” in a list of topics covered during briefings by one civilian trained

respondent and one military-trained respondent. Another military-trained person’s

response seemed to be separating knowledge from airmanship, writing “In training,

97

we were assessed on preparation, knowledge, the proficiency of our execution, and

airmanship considerations that applied to our prep and subsequent execution” (080).

The concept “knowledge-of-self” included comments about knowing one’s

own limitations and capabilities. References to knowledge-of-aircraft included

“aircraft limitations” (155) and also “knowledge of orders and instructions; aircraft

capability and systems” (126) and “judicious fuel management and monitoring on-

board indications to ensure the safe operation of all aircraft systems including power-

plant and life support” (187).

Knowledge-of-environment comments tended to be just references amid a list

of requirements, as were references to knowledge-of-team, for example “Knowledge

about one's aircraft, environment, crew, and operation” (162, military-trained).

Comments from respondents coded as knowledge of the legislative

environment and knowledge of the organisational environment were often linked, for

example “adherence to rules and procedures” (119, military-trained). Comments that

mentioned rules or regulations were coded in “knowledge-legislative”. Comments

that mentioned procedures, SOP’s or instructions were coded in “knowledge-

organisational”. The knowledge-of-the-physical-environment comment was a

reference to terrain. Knowledge about risk included comments about risk

management or risk assessment, or simply awareness of risks and threats. Comments

about preparation or planning or knowing about the operation were included in

knowledge of the mission.

Flying skill was included in the answers to Question 3b, for example “All

concepts of airmanship listed above were constantly discussed and reviewed in most

Mass Briefs and in all individual instructor flight briefs and debriefs. It's a

combination of several factors all working together [including] the ability to safely

and competently fly and navigate an aircraft.” (089, military trained). There were two

references to proficiency or “highly refined flying standards” (051, civilian trained).

Discipline was coded from mention of discipline, obedience and “not taking

shortcuts” (142, civilian trained).

From the additional Ebbage and Spencer/Edwards concepts, only

communication was frequently mentioned, especially among the civilian trained

respondents. Comments coded here included mention of making radio calls or radio

98

procedures as well as the use of the word communication. There was only one

reference that could be interpreted as relating to hazardous attitudes, which was “it

was a kind of ‘do unto others’ approach, as in not being a ‘cowboy’” (142, civilian

trained). Crew co-ordination included the listing of “crew support” (176, civilian

trained) and “timely and appropriate guidance to the crew” (091, military trained)

which was also coded as management in cockpit, as was “management skills” (040,

civilian-trained). Team leadership was represented by mention of “leadership skills”

(040, civilian-trained). There were no mentions of professionalism, self-

improvement, interpersonal skills or self-assessment.

Table 3.13 summarises the additional concepts and other comments made by

the participants in response to Question 3 (b).

Safety was the most frequently mentioned concept raised as discussed during

ground training, or in pre or post flight briefings. It was either included in a list of

things discussed, or specific attention was drawn, such as with the comment

“Airmanship was usually discussed pre-flight with respect to safety considerations”

(119, military-trained) and “the safety related aspects of airmanship were particularly

emphasised during training” (100, civilian-trained). Lookout was also frequently

mentioned, for example “Keep a good look out. See and be seen was always stressed

from day one” (026, civilian trained).

Courtesy was mentioned by respondents from both civilian and military

training backgrounds. Comments coded included mention of courtesy, “do unto

others”, being polite, showing consideration or having a pleasant manner, all

mentioned as something that was considered part of airmanship and discussed during

training.

Prioritising was mentioned more often by military-trained respondents as

something specifically taught. The saying “aviate, navigate, communicate” was

quoted as the order in which to prioritise. The remaining concepts listed in Table

3.13 were each mentioned by fewer than five respondents, but the variety shows the

range of different concepts remembered as being part of airmanship training.

99

Table 3.13

Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned the Listed Additional Concepts in Response to Question 3(b)

Civilian Military

Before 1991

After 1990

Before 1991

After 1990

Number of Participants 22 15 29 6 Missing answers 4 2 4 0 Number of Respondents 18 13 26 6

% % % %

Other concepts: Safety 27.8 15.4 38.5 16.7

Lookout 22.2 7.7 19.2 33.3 Courtesy 22.2 15.4 11.5 16.7 Prioritising 0 7.7 11.5 16.7 Caring for the a/c 11.1 0 7.7 0 Taxi 16.7 0 3.8 0 Common sense 5.6 0 7.7 0 Efficiency 5.6 7.7 3.8 0 Professionalism 5.6 7.7 0 16.7 CRM/TEM 0 15.4 3.8 0 Checks 0 0 7.7 0 Fuel 5.6 0 3.8 0 Weather 5.6 0 3.8 0 Abnormal/emergency 0 0 0 16.7 Documentation 5.6 0 0 0

Other Comments: Non-specific use 16.7 0 15.4 33.3

Can't remember 0 15.4 15.4 0 Pre-HF/TEM 16.7 0 3.8 0 Experience 0 7.7 11.5 0 Examination 0 0 11.5 0

Other comments that were made by respondents included complaint about the

non-specific use of the term “airmanship”. This included such comments as

“Airmanship was always used as a ‘catch all’ phrase with no specific definition”

(042, civilian trained), “Airmanship was a vague concept but in general related to

100

behaviours” (173, civilian trained), “Yes, but the concept was vague” (201, military

trained) and “Most lectures struggled to find an accurate definition. We were given a

few different definitions of airmanship from various writers, however there was not

one singled out” (185, military trained). A longer comment was:

In my military training, the concept of airmanship was a formal feature in both the sortie briefing and debriefing process (not so much ‘airborne’). On reflection, it was mainly used as a tool to define and/or reinforce those behaviours/outcomes during the flight which were acceptable or needed improvement or were unacceptable (189, military trained).

There were six respondents who knew that airmanship was discussed (replied

“yes” to question 3a) but could not remember the details. Four respondents made the

point that their training pre-dated the introduction of human factors or Threat and

Error Management training, for example, “Self-assessment and the management of

potential threats were not mentioned in those days (1960's)” (051, civilian trained)

and “The term threat and error management was not used back then, but Airmanship

was taught in the same sort of terms” (201, military trained). A few respondents

implied that later experience was required to firm up the concept of airmanship and

several military-trained respondents made mention of the fact that they were

examined on or assessed on airmanship as well as everything else.

101

In-air training – Question 4. When asked whether airmanship was part of

instruction in the air (Question 4 “Was the development of airmanship a specific part

of the initial ‘in the air’ flight training you did? If so, please explain what this

involved.”), some participants were equivocal, and did not give a definite yes or no

answer, and this question did not force the choice. Table 3.14 shows the pattern of

yes/no responses to Question 4.

Table 3.143

Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Responded to Question 4

Civilian Military

Before After Before After

1991 1991 1991 1991 Number of participants 22 15 29 6 Missing Answer 2 2 2 1 Number of respondents 20 13 27 5

% % % %

Yes 30.0 53.8 77.8 80.0 No 50.0 23.1 7.4 0.0 Other 20.0 23.1 14.8 20.0

There were no significant differences in the yes/no responses between eras

within each sector, although the Civilian sector looks as though it may have neared

significance (however p < .107). There was no significant difference between eras if

the sectors are combined. However, a comparison between sectors (ignoring era)

indicated that the military-trained respondents were more likely than the civilian-

trained respondents, to indicate that airmanship was discussed during their in-air

training (χ² (1, N = 53) = 11.841, p < .001, w = .473, a large effect).

102

Table 3.15 indicates the concepts that were mentioned by respondents in

answer to Question 4. Numbers of mentions were too low to carry out any statistical

tests, except for situational awareness, which was non-significant.

Table 3.15

Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned Concepts From the Airmanship Models in Response to Question 4.

Civilian Military

Before 1991

After 1991

Before 1991

After 1991

Number of Participants 22 15 29 6 Missing answers 6 3 2 1 Number of Respondents 16 12 27 5

% % % %

Kern Concepts: Judgement or decision making 0 0 7.4 20.0

Situational awareness 18.8 25.0 14.8 40.0 Knowledge: General 0 0 0 0 Self 18.8 0 0 0 Aircraft 6.3 0 7.4 20.0 Team 0 0 0 0 Environment: General 6.3 0 3.7 0 Legislative 0 0 7.4 20.0 Organisational 0 0 11.1 0 Physical 0 0 3.7 0 Risk 0 0 0 0 Mission: Planning 0 0 3.7 0 Skill 0 0 0 60.0 Proficiency 0 0 7.4 0 Discipline 0 0 11.1 0

Additional Ebbage and Spencer or Edwards concepts: Attitudes: Hazardous 0 0 0 0

Professionalism 0 0 0 0 Self-improvement 0 0 0 0 Skills: Communication 18.8 0 14.8 0 Crew co-ordination 0 0 0 0 Interpersonal 0 0 0 0 Management flight-deck 0 0 0 0 Self-assessment 0 0 0 0 Team leadership 0 0 0 0

103

Respondents from all groups mentioned that situational awareness was part of

their in-air training. Mostly this was clear inclusion of the term in their response, but

sometimes other words were used, for example “Having due respect and awareness

for other aircraft and their safety” (181, civilian-trained) or “Knowing and

developing an ‘air picture’ of what was going on around” (155, military-trained)

were included as situational awareness. Judgement, actually decision-making, was

mentioned only by military-trained respondents. One said that as students, decision-

making was forced upon them and that “how we addressed these problems was a

factor in determining our level of airmanship awareness” (165). The other two

mentions of decision-making were similar comments, that “airmanship was ...

positively reinforced when a good decision was made and negatively reinforced if a

poor decision was made” (187) and “when good decisions were made, this was

complemented as good airmanship and that when poor decisions were made, that this

was labelled as poor airmanship” (200).

The concept of knowledge-of-self included comments about personal abilities

and limitations. Knowledge-of-the-aircraft comments referred to limitations of the

machine and the flight envelope. There was a reiteration of comment on rules and

procedures, coded in knowledge of the environment, legislative and organisational.

The reference to the physical environment was a comment regarding awareness of

geographical position.

Skill was mentioned by one respondent who wrote that “things like lookout and

correct control of the aircraft was integral” (155, military trained) while two others

noted that preparation, technique and airmanship were assessed separately, thus

distinguishing between flying skill (technique) and airmanship. Proficiency was

mentioned by two additional military trained respondents as an expectation of

reaching a certain standard of flying skill. Discipline was a simple inclusion of

“Mental control of one self. (Discipline)” (162, civilian trained). Comments classed

as communication included reference to listening, listen-out, radio use, read-back,

and “maintenance of audio communication in relation to the position of other

aircraft” (090, military-trained).

Table 3.16 lists the additional concepts mentioned by respondents in answer to

Question 4, these covered what was included in the in-air training and also comments

about how the training was conducted

104

Table 3.16

Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned the Listed Additional Concepts in Response to Question 4.

Civilian Military

Before 1991

After 1991

Before 1991

After 1991

Number of Participants 22 15 29 6 Missing answers 6 3 2 1 Number of Respondents 16 12 27 5

% % % %

Additional Concepts: What Lookout 18.8 0 33.3 20.0

Courtesy 12.5 16.7 0 0 Abnormal/Emergency 0 0 7.4 20.0 Safety 6.3 16.7 0 0 TEM 0 8.3 7.4 0 Prioritise 0 8.3 3.7 0 Care for aircraft 0 0 3.7 0 Common sense 6.3 0 0 0 Survival 6.3 0 0 0 Taxi 6.3 0 0 0 Untrusting 0 8.3 0 0 Weather 6.3 0 3.7 0 Additional Concepts: How

Briefs 6.3 16.7 18.5 40.0 Demonstration 0 0 18.5 0 Examination 0 0 7.4 60.0 Correction 6.3 8.3 7.4 0 Training process 0 0 3.7 20.0 Good example 0 8.3 0 0 Other Comments:

Can't Remember 0 8.3 0 0 Informal 0 0 3.7 0 Pre-CRM/HF 6.3 0 3.7 0 Experience 12.5 8.3 7.4 0 Non-specific use 31.3 16.7 3.7 20.0

Again numbers were too low to carry out statistical tests, except for the

mention of lookout. Combined across eras, analysis of lookout comments indicated

that the military-trained respondents mentioning lookout as a topic for instruction

rather more than civilian-trained respondents but the difference was not significant.

Courtesy was only mentioned by civilian-trained respondents and performance

105

during abnormal or emergency events only by military-trained respondents. Safety

was commented upon only by a few civilian trained respondents.

There was some mention of the recognition and management of threats,

prioritisation and single mentions of several other relevant issues, including never

trusting anyone and keeping a close eye on the weather.

Some respondents included an explanation of the methods used in conducting

training, and these are listed as the “how” of training in Table 3.16. The inclusion of

airmanship in briefings was reiterated, despite the question being about in-air

training. Some of the military-trained respondents noted that airmanship was

demonstrated, although a few of these comments were actually about the

demonstration of flying sequences (and so technique related). Examination and

assessment was also revisited. There was mention of being corrected or reminded of

airmanship points during flights. A couple of respondents noted that they were

provided a consistently good example through their flight instructor.

One respondent noted that the training process included allowing him to

“progress down a ‘non-optimal’ course of action as long as safe in order to provide a

teaching point and as a chance to recognise a dangerous situation next time it

occurred” (124, military-trained). A further respondent gave an extensive explanation

of the training process:

A "building block" approach was used. A new student was not expected to handle complex emergencies from day one. But, once appropriate lectures, exams, experience and training had occurred (e.g. radio failure procedures) the student was generally exposed to a simulated exercise to “immerse” or “cement” familiarity with that aspect or discipline. Subsequent reinforcement might occur with no notice and an acceptable standard of proficiency was expected. By the end of the course syllabus students were expected to be able to use airmanship to diagnose and manage complex, and often compound (i.e. multiple concurrent) emergency scenarios to a successful outcome. (197, military-trained)

There were a few comments that were neither about what was included nor

how training was carried out – one such comment being that the respondent did not

remember the details. One person stated about airmanship training: “Not formally.

Flight training inherently critiques and develops airmanship” (119, military-trained)

106

and two noted that their training in airmanship pre-dated introduction of crew

resource management and human factors training.

There were also some comments coded as experience, but that reflected the

lack of regard for airmanship in their training, “No. It was like seamanship you

picked it up as you went along.” (009, civilian-trained), “Airmanship is something

that is gained with experience” (026, civilian-trained). “Not really. Well not for me

anyway. I think it's something that develops more with experience and the more

comfortable you are with the operation” (064, civilian-trained), “It was spoken of as

an evolving ethos, as it were, that came with experience and practice. Good

airmanship would allow you to apply some limited knowledge to the new/unexpected

circumstances” (088, military-trained) and “When you start training you as a student

are not aware of the word airman-ship. It is only after you survive the experience

that you realize that you have developed airman-ship” (096, military-trained).

Finally, there were some respondents who, as part of their response, indicated

that airmanship was integrated throughout the training rather than being a specific

area of instruction, but that it was not necessarily well defined. Some examples are:

“If you did something wrong or poorly, it was criticised as poor airmanship. When

you pushed the instructor for a definition of airmanship, they could only give

examples based on their recent experience” (042, civilian-trained), “It varied from

trainer to trainer. As far as I recall airmanship was a commonly used term but no one

could really explain what it meant.” (151,civilian-trained), “Not so much a ‘specific’

part, but it was just something which was constantly being taught throughout all my

training.” (054, civilian-trained) and “I don’t recall that it was specifically included

in training” (200, military-trained).

Discussion - initial training. Most of the respondents indicated that

airmanship was included in their ground training and/or pre and post flight briefs

(Q3), while more military than civilian trained respondents reported that airmanship

formed part of their in-air flight training (Q4). There were no significant differences

between eras of training. Airmanship (however defined) was reported to have been

included by respondents from both eras. Tracing any differences between the eras in

concepts raised by respondents was made difficult by the low numbers once the

respondents were divided into four groups, especially the military post-1991 group.

107

Safety was the most frequently mentioned concept raised as having been

discussed in ground-training or in briefs, followed by situational awareness, lookout,

judgement/decision-making, courtesy and then knowledge of regulations and

procedures. Most frequently mentioned concepts in the responses about in-air

training were lookout, situational awareness and communication, followed by

knowledge of the aircraft and courtesy. There is consistent inclusion of Kern’s

capstone outcomes of airmanship: situational awareness and judgement, as well as

the overall aim of airmanship – safety. Lookout is one activity that leads to

development of situational awareness, communication being another, and these were

both quite frequently mentioned. There was a range of other concepts included in

responses, covering many of the other Kern model elements as well as some Ebbage

and Spencer/Edwards elements.

Consistently omitted from responses were reference to self-improvement, self-

assessment, interpersonal skills (unless courtesy is interpreted as interpersonal skills)

and team related comments. The lack of referral to teamwork in ab-initio training is

understandable, as initial flying is not a team activity. However, it is interesting to

note that the respondents were not encouraged to consider self-assessment, self-

improvement or much in terms of knowledge of themselves, during their initial

training. A few comments on being aware of one’s own limitations were all that was

noted.

It is also note-worthy that complaint about airmanship being assessed without

provision of a clear definition was frequently brought up in responses to both

questions.

Development of Airmanship

Participants were asked, in Question 5, “Thinking about your progress as a

pilot, what do you feel led to the development of your sense of airmanship?” For

analysis of this question, participants were divided by sector as they were for the

earlier definition questions, civilian background (both airline and GA) and military

background including people who moved on to fly in airlines or in general aviation

after a period in the military. Table 3.17 shows the development factors included by

the respondents.

108

Table 3.17

Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 5

Civilian Military

Participants 40 43 Missing answers 4 2 Respondents 36 41

% %

Internal development factors: Experience 27.8 56.1

Personal qualities 30.6 19.5 Common sense 0 12.2 Responsibility 8.3 2.4 Thinking/reflection 8.3 2.4 Continual learning 2.8 4.9 Proficiency 8.3 0 Knowledge of Aircraft 0 2.4

External development factors: Good/poor examples 27.8 29.3

Training 16.7 22.0 Reading 5.6 24.4 Instructors 11.1 12.2 Talking with others 13.9 7.3 Mistakes 8.3 9.8 Safety 8.3 8.3 Organisational culture 5.6 12.2 Discussions 2.8 9.8 HF training/CRM 5.6 4.9 Supervision 0 9.8 Survival/threat 2.8 7.3 Death 0 7.3

The same question was presented in the previous survey and responses

separated into personal reasons and influences of others (see Table 2.6). The current

data can be arranged to reflect a similar pattern. Personal or internal forces include

one’s own experiences, motivation, common sense, responsibility, thinking /

reflection and continual learning. This selection could also include safety,

proficiency and aircraft handling. The influence of others over the development of

airmanship can be seen in the examples provided by others and their mistakes,

training including HF/CRM, instructors, supervision, organisational climate, reading,

109

talking to others and discussions. The death of a workmate could also be seen as an

external influence.

Formal comparison between the numbers of respondents from each sector was

conducted for the seven concepts mentioned most frequently. The Bonferrroni

adjusted critical value was p < .007. None of the comparisons were significant.

The most commonly included factor that influenced the development of

airmanship was experience. Mentioned more often by military background than by

civilian background participants (but not significantly so) generally this factor was

mentioned in combination with other factors. An example is “experience and

common sense as well as studying and reading books and magazines” (089, a

military + airline participant) or “I believe experience was probably the biggest

factor in developing a sense of airmanship” (100, airline participant).

Various personal qualities were mentioned as factors in developing airmanship,

slightly (but not significantly) more often by civilian than military background

participants. Mostly these were comments reflecting an internal motivation. Some

examples from civilian respondents include: “a personal desire to be better” (040),

“self-development” (151), “continual self-improvement” (179) and “the personal

rewards that come from keeping high standards” (051). Some military background

respondents wrote: “My own desire to apply and gain the appropriate skills to

produce acceptable results” (119), “I also wanted to be ‘the best’ at my job, bar

none” (046) and “the drive to fly well” (126). There were some other kinds of

comments. Two military participants mentioned fear of failure as a motivator to

develop airmanship, and one mentioned “attitudes of peers” (124), without indicating

whether these were positive or negative.

Only military participants mentioned common sense in response to this

question, but without much elaboration for example “Experience and common sense

as well as studying” (084), “common sense and good instructors” (025) and “using

common sense” (126). A lengthier response was “Repetition!! I'm not sure that

[people] have a particularly strong innate level of common sense. I suspect that

many aviation accidents may well be attributable to either a lack of airmanship (read

common sense in the air), or at best a poorly developed level of it” (099). Here it is

not so much that common sense was a driver for developing airmanship, as that

110

repetition develops common sense, seen as equivalent to airmanship by this

participant.

Some respondents mentioned that the increase in responsibility as they took

over running a business or reached command level was significant. A few

participants commented that reflection or thinking about their experiences led to

learning and development of airmanship, while some pointed out that the

development of airmanship is continual “I learnt about airmanship on every flight

that I flew - airmanship and its development never ends” (110, military). There were

a few civilian participants who indicated that their airmanship was improved because

of having to maintain their flying skills at a high level of proficiency. One military

background participant indicated that changing aircraft type is a key factor, saying

that “the most significant influence on the development of airmanship [is] the

changes in aircraft handling and performance [between types]” (013).

Observation and learning from the good (and poor) examples of others was the

most frequently mentioned external factor significant to the development of

airmanship, included by both groups of participants. Some examples of comments

from civilian background participants include: “Watch those who you considered to

be good pilots” (008), and “observing other pilots’ behaviour, good or bad” (178).

Some comments from military background participants are “the example set by

senior pilots” (091), “the initial training set the scene but seeing other pilots display

poor airmanship or good airmanship reinforced these principles in me” (111) and

“experience, and observing the correct and incorrect execution by more (and less)

experienced aircrew” (080).

Training or instruction was mentioned by both groups as a positive influence

on development of airmanship. Some examples from civilian background

respondents are “my initial training” (178) and “a consistently high standard of

training” (042). Military background examples are “excellent instruction” (088) and

“quality on-going check and training” (119). One dissenting voice stated “There was

nothing in my formal training – this was all self-development” (151, civilian).

Reading was significantly more often mentioned by military than by civilian

background participants (χ² (1, N = 77) = 5.169, p = .023, w = .259). Most of these

participants specifically indicated the reading of accident reports and/or flight safety

111

magazines. Several mentioned study or research, including one civilian participant

who indicated that his University degree was a factor in developing his sense of

airmanship.

Instructors or mentors were listed as forces behind the development of

airmanship by both civilian and military participants. As well as simply listing

instructors as influences, there were other comments, for example from two military

background participants: “the availability of (informal) mentors from time to time”

(089) and “mentors (ie pilots whose professionalism and skills I wanted to emulate)”

(197).

Talking to other pilots was more often listed by civilian than military

background participants (but not at statistically significant levels) and included

comments like “I ‘hung on every word’ of what I considered good advice /

knowledge imparted by those more experienced than myself” (166) and “Experience

of chatting with others, I have learnt much about ‘what not to do’ after chatting with

more experienced aviators. This has really developed airmanship” (037). Two

military participants made much the same comments “studied aviation from ‘top to

bottom’ and ‘picked the brains’ of anyone who I thought could help” (046) and

“while flying with more experienced people [took] the opportunity to seek their

knowledge or ideas on how to best do things” (165). One of each group mentioned

“war stories” as an influence, this being in reference to stories about flying

experiences, not necessarily wartime.

Learning from one’s own mistakes, or those of others, was another influence

listed by a few participants from each group, as was a sense of developing and

maintaining safety. Organisational culture was mentioned as an influence by the

military background participants, rather more than by civilian participants, perhaps a

reflection of pride in their military background. One comment from a civilian

participant was “In my first job... the company had what I consider to be a

professional approach to business” (058). Military background participants

commented: “operational culture on the squadron (very strong peer pressure and

inevitable disciplinary-based motivators!!!)” (189), “great training in a disciplined

and supportive organisation” (192) and “flying in a large flying organisation that

lived and breathed flying and allowed us to learn from the experiences (sometimes

mistakes) of others” (102).

112

Some respondents mentioned discussion, this was not quite the same as an

active attempt to gain knowledge from talking to others, but appeared to be more

training related. Mention of briefings was included in this category. Several

participants specifically mentioned training in human factors, crew resource

management or aviation psychology as having led to improvement in airmanship. A

few military background participants drew attention to the importance of good

supervision. The idea that one has to survive in a threat-filled environment

encouraged some participants in their development of airmanship, closely related to

this concept are the few who listed the death of a workmate or friend as something

that influenced them, for example “I attended more than ‘Forty’ funerals of fallen

comrades.....and I did not want to be one of the next” (046, military background

participant).

Discussion – development of airmanship. Overall, it is experience that is

mentioned most often as an influence in the development of airmanship, whether it is

personal experience or using the experiences and mistakes of others as salutary or

inspiring examples. Internal motivation is recognised as an important personal

quality that is engaged in the development of airmanship, which supports the position

of Ebbage and Spence (2003) that attitudes of professionalism and self-improvement

are important foundations for airmanship that should be developed in early training.

Kern (1996) also considers that the motivation for improvement should be instilled

early in training, and should be continued life-long as “empowered responsibility”

(Kern, 2009b). This is in contrast to the responses to Questions 3 and 4, in which the

ideas of self-improvement and self-assessment were barely mentioned.

The further development of airmanship with increased responsibility and the

continued reflection and thinking about experiences confirm that this aspect of

internal motivation continues beyond the initial training stages. The frequent mention

of early training, instructors and mentors, organisational culture and supervision,

recognises the importance of these roles in development of airmanship, supporting

the emphasis that both Kern (1996) and Ebbage and Spencer (2003) place on the

inclusion of airmanship in ab initio training, and the importance of positive role

models. However, as with the earlier findings presented in Chapter 2, it is not just the

ab initio instructor who provides this important role model, but later career

influences in other pilots as mentors that is also seen as important in developing

113

airmanship. There is also some recognition that airmanship is a case of life-long

learning, among a few of the respondents. It is also interesting to note some more

negative influences, in seeing aviation as something to be survived, and for a few, the

influence of a death forcing confrontation with the need to improve airmanship.

Current Training

Question 6 asked “Do you participate in ongoing training? Does this include

airmanship and if so, what does this entail?” and Table 3.18 provides a summary of

responses to this question. Because this question asked about the participants’ current

activity, their sector of current, or most recent, activity was used to examine the

responses, regardless of background. The Airline group consists of people from

either civilian or military background, currently or most recently flying for an airline.

The general aviation (GA) group is those participants from either civilian or military

background, currently or most recently flying in general aviation, including

recreation, flying training clubs or in charter or helicopter operations. The Military

group covers any participants who are currently, or were most recently, flying with

the military.

There was no statistically significant difference between the three sectors with

regard to participation in training. Where there were sufficient responses, chi-square

analysis was carried out, but there were no significant differences in frequency of

mention of any concept between the sectors.

Several respondents appeared to have misunderstood the questions and

answered with comments about what they did as trainers or instructors, rather than

the type of training that they engaged in themselves, these are shown as ‘instructor’

in the table.

Many respondents indicated that they participated in training, but that this did

not specifically include airmanship. Some comments from respondents in the GA

sector include “but it doesn't really include much airmanship” (064), “Once again it

is an implied notion rather than a specific detail” (073), and “It's more something we

talk about over a debrief/beer with the rest of the crew or other crews. Very informal

but lessons are learnt” (064). Some comment from airline participants includes

“Airmanship training, not specifically” (006), “airmanship is only acquired with

experience, not something that can be taught” (026), “but airmanship seems to be an

114

unspoken concept in modern training programs” (032) and “as it [airmanship] is

assumed knowledge at my level of operation” (156).

Table 3.18

Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 6

Airline GA Military

Number of Participants 42 26 15 Missing answers 3 4 1 Number of Respondents 39 22 14

% % %

Yes 84.6 72.7 64.3 No 12.8 18.2 14.3 Other 2.6 9.1 14.3

Concepts: Instructor 5.1 13.6 28.6

Yes, but not airmanship 28.2 36.4 21.4 Centrality 20.5 22.7 14.3 Inclusion with CRM/TEM/HF 23.1 9.1 28.6 Safety 2.6 0.0 21.4 Common sense 7.7 0.0 0.0 Lack of formal definition 10.3 0.0 0.0 Formality/requirement 0.0 4.5 0.0 Kind of Training 28.2 31.8 28.6 What:

emergencies 5.1 0.0 0.0 night ops 2.6 0.0 0.0 decisions 0.0 4.5 7.1 safe, legal, efficient 0.0 4.5 0.0 reading, talking, courtesy 0.0 4.5 0.0 Aircraft handling 2.6 0.0 0.0 SOPs 7.7 0.0 0.0 Note. ‘Other’ includes statements that did not indicate whether or not the respondent engaged in training.

Some respondents indicated that airmanship is central to all flight operations,

including training, and thus is not a separate item. For example, from airline

respondents “Organisationally, airmanship is at the core of all training and checking

activity. Rarely would it be considered a stand-alone subject” (051), “Each and every

day your management and airmanship skills are being assessed. It may be by ATC,

115

your crew or customers. It is a skill that needs to practiced and constantly

developed.” (176) “All flights and training entail the application and subsequent

assessment of airmanship principles” (119) and “[Airline] is pro-active in the

development of airmanship and it is reinforced by many monitoring and training

functions” (192).

Several respondents drew the parallel between airmanship and the content of

Crew Resource Management, Threat and Error Management and/or Human Factors

training, indicating that it is in these non-technical training areas that airmanship is

addressed. Examples of comments from the airline sector are “but now [airmanship]

is called different things CRM, Security, OH&S training, New Technology Training,

Technical training modules + Highly proscriptive operating procedures” (040), “It

does entail airmanship in a different guise; not airmanship per se but the study and

application of Human Factors and Crew Resource Management where we are taught

how to work together to achieve safe and effective operation and study the cause and

effect of how poor airmanship can contribute to accidents.” (166) and “CRM training

will often involve discussion on recent events and how we might handle them” (190).

From a GA respondent, “Crew resource management is a part of training nowadays,

which is an important aspect of airmanship” (101) and from military participants:

“Included, principally under CRM, during formal triennial refresher training” (086)

and “Crew Resource Management and Aviation Risk Management training is

refreshed annually” (097).

One airline participant referred to a cabin safety briefing “where safety aspects

of our operation are discussed” (058). The military respondents drew attention to

safety culture, safety ‘stand-downs’ and the responsibility for safe operations, as

being part of the airmanship training in which they participate.

There were some references to common sense, all from airline respondents, for

example “One could possibly summarise it as the effective application of good old

common sense!!” (166) and “I teach safety management systems, flight operations

management and aviation quality assurance. I stress common sense which in the final

analysis is airmanship” (009).

There was some complaint about the lack of definition of airmanship, from

airline respondents, and the consequent subjectivity of assessments. Some examples

116

are “Airmanship is an item required in our line check system, while not defined

anywhere, it concerns the general conduct of the flight” (043), “Airmanship is

always an assessable component, and is generally assigned a somewhat subjective

score by the check captain” (100), and “Airmanship seems to be a very subjective

thing to assess and it is entirely up to the assessor as to what their opinion might be”

(184).

About 30% of respondents overall gave a description of the type of ongoing

training that they are involved in, without necessarily indicating how airmanship was

included in these activities. Some airline examples are: “CRM and Recurrent training

simulators” (053), “always undertaking simulator check rides as part of licence

renewal requirement” (096), “ongoing training in the form of check flights and

simulator training at regular intervals” (100), “recurrent simulator checks” (179) and

“Training programs associated with my work as an ATPL pilot” (173). Among GA

respondents comment included: “I undergo 3 checks a year as part of my

employment” (054), “We conduct check and training every six months as part of our

companies ops manual” (113), and “If you can count instructor renewals as training”

(116). Military respondents included “Yes, annual check flights” (165) and “formal

triennial refresher training” (086) as well as reference to safety stand-downs.

There were few actual descriptions of what was included in training, whether

airmanship or not. One description came from an airline participant and included

“Day proficiency consists of a session of in-flight emergencies and refresher training

then the remainder of the two day shift observing normal operations with particular

emphasis on SOP's. Night proficiency is night sortie usually to a difficult bush strip

where night operations including some emergencies are checked.” (058). Standard

operating procedures (SOPs) were mentioned by other airline respondents, for

example one wrote “My organisation has a very strong SOP-based culture and

assessment is conducted at both technical and non-technical levels” (189).

Two respondents wrote about decision making. From one airline respondent

this was a negative comment: “Airmanship is rarely mentioned in ongoing training

and if mentioned is generally as a negative reinforcement to a poor decision” (187),

while a military instructor wrote that he takes over some tasks to allow the students

“a little more free brain space to work with, in problem solving and decision making”

(165). One GA respondent listed “reading as much as I can in the industry, asking

117

my peers about changes, respecting other fliers” (138) in response to the question

and another indicated keeping safe, legal and efficient.

Discussion – current training. In summary, most participants engage in some

form of on-going training, if they are still active pilots, and this does not generally

have any formal inclusion of airmanship as such, but there is recognition that

airmanship underpins all aviation activity. Quite a few respondents recognised the

application of CRM and HF training in developing airmanship. In addition, the lack

of formal definition and subjective assessment of airmanship was noted among some

airline pilots. There are some minority views expressed, such as airmanship being all

about aircraft handling, or that it is only common sense anyway, and that it cannot be

taught but depends upon experience.

Instructors’ Practices

In Question 9, participants were asked about their practices as instructors or

trainers with regard to airmanship: “If you are currently an instructor or hold a

check/training role, what do you do to ensure that your students develop good

airmanship? (or if you were an instructor, what did you do?). Responses to this

question are summarised in Table 3.19. The participants were divided into three

groups as they were for the analysis of Question 6, based on current sector of

activity, being airline, general aviation (GA) or military.

There was much less detail in these responses compared to previous questions,

possibly some respondents felt that they had already written enough. The general

aviation (GA) group, especially, gave very concise answers. Three respondents

referred to previous questions, content of which was not included in the analysis for

Question 9, but they also wrote additional comment, which was included. The kinds

of responses made fell into two main types: 1) indications about how training was

carried out and 2) discussion of what sorts of things students were taught. The

frequency of mention of each topic was low, and only briefings and examples were

tested for differences between sectors, in both cases, the differences were not

significant.

118

Table 3.19

Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 9

Airline GA Military

Participants 42 26 15 Missing answers 18 14 4 Respondents 24 12 11

% % %

Critical Comment 12.5 0 0 How:

Briefing/discussion 41.7 33.3 18.2 Examples 29.2 33.3 45.5 Scenarios 16.7 0.0 27.3 Observe 16.7 0.0 9.1 Correction 4.2 0.0 9.1 Information load 4.2 0.0 9.1 Experience 4.2 0.0 9.1 What:

Safety 4.2 33.3 27.3 Knowledge 8.3 25.0 27.3 Think/Introspection 20.8 0.0 18.2 Decisions 16.7 0.0 18.2 Situation awareness 16.7 0.0 9.1 Attitude 0.0 33.3 9.1 Preparation 4.2 8.3 9.1 Work cycle/routine 8.3 0.0 0.0 Communication 4.2 0.0 0.0 CRM 4.2 0.0 0.0 Crew coordination 4.2 0.0 0.0 Comparing types 4.2 0.0 0.0 Consideration 0.0 8.3 0.0 Skill 0.0 0.0 9.1

Three airline participants made critical comment about the content of the

question. One of these was that “All aspects of training input include airmanship as

intrinsic” (051) and thus no specific practices were required, one assumes, as no

further comment was made. An opposing view was presented as: “I NEVER use the

un-defined word Airmanship. I NEVER try to develop an undefined skill” (042) and

again no further comments were made. A third participant noted that “Yes, however

the due to the experience level of trainee already has a high level of airmanship

119

instilled.” (013), but did go on to give further comment. Airline participants are

likely to be training pilots who are undergoing type or upgrade training, rather than

initial training, handled by the GA instructors. The Military instructors could have

been doing either kind of training.

Briefings or discussion were the most frequent type of comment. One airline

respondent mentioned using “Kern's airmanship model frequently as a template for

discussion” (151). Others emphasised the importance of briefings and discussions for

highlighting issues to do with airmanship including “Briefings before and after flight

are extremely important in getting the right mind-set for safe and efficient flight”

(034), and “debrief airmanship issues ... analyse errors in order to help students avoid

repeating the same mistake in future” (197). Respondents from the GA sector wrote

much the same sort of comments, for example “discussion prior and post flight are

the best times to highlight airmanship.” (073) and “emphasize the importance of

these points during all stages of briefing and flight and post flight, even in general

conversation around a beer” (118). The military respondents wrote that briefs should

be thorough and consistent.

Being a good example as an instructor was brought up by many respondents.

An airline respondent stated: “First of all as an instructor, airmanship is taught by

example” (100) and another: “LEAD BY EXAMPLE!!!... The role of the mentor

cannot be overlooked in developing Pilot's Airmanship” (192). A GA respondent

wrote: “try to model appropriate attitudes and provide an example” (124). Some

military respondents had identical messages, including: “Demonstrate how to

develop good airmanship by example on each and every flight.” (090) and “Lead by

example. Ensure I do practice what I preach” (183).

Respondents from the airlines and military wrote about the use of different

scenarios in the training environment, so that what is learnt covers a “broad range of

operations” (025) and might be “as simple as an in-flight malfunction, through to a

major divert or critical malfunction... it also might include an unusual flight regime

or requirement.” (165), both are comments from military respondents. However, one

airline respondent warns, “adhere to the building-block approach (ie don't give an

impossible scenario for the student's level of experience)” (197) and another that “it

may be necessary to create a situation or allow a situation to develop to a stage where

the student ‘stares death in the face’ in order to get the message across” (166).

120

The latter approach is echoed by a few comments about simply observing how

the student manages events. Airline examples are “allow circumstances to develop

without comment or action and observe the crew(s) performance” (119), and “I

would let them manage the situation as much as was appropriate and their

authority/limits allow” (190), however a military respondent points out “another

noteworthy dilemma is how far you let a student go ‘safely’ to learn a valuable

lesson for life at the risk of letting him or her go too far and cause an accident” (102).

The final few comments concerning the ‘how’ of instruction included comment

on correcting students, “if they do something that would be considered poor

airmanship” (043), on not overloading them with too much information, and on the

role of experience. Two people commented that while they can provide the basics,

“finesse comes with experience and is an extension of the basics” (110) and “for the

most part pilots “evolve”, from student to Co-pilot to Captain the ability to see

problems and avoid or react increases with experience” (192).

One airline respondent made an additional statement in the next question that is

relevant here: “I actually provide my students with more than a comment along the

lines of ‘airmanship is really important’. I give them a framework of why it is

important, what is required, and how to get there!” (151), this being a reference to

the Kern model.

The ‘what’ of flight training looks quite familiar, the main components of the

two airmanship models are represented in the list in Table 3.18. Safety (the expected

outcome of the models) was the most frequently mentioned concept, as something

that the instructors were conscious of developing in their students. One airline

respondent wrote “I try to develop good, safe habits.” (042), echoed by GA

respondents in comments like “My aim is to instil a good attitude and good practices

that promote good airmanship and safe operation” (111) and “emphasizing the

importance of regulations and flight manual limitations to the safety of flight” (124).

A military instructor wrote “Teach them, and insist on TOTAL ADHERANCE to the

safe principles of flight....’Professional airmanship’......if they don't get the

picture....FAIL THEM” (046).

Knowledge was also a frequently mentioned concept among the ‘content’

concepts. This appeared in terms of imparting knowledge or checking it, and in some

121

cases knowledge was left unspecified. Specific mention was made of rules,

regulations, SOPs, and aircraft systems and limitations. One longer example is

Make sure they know the rules, SOPs etc, but that they don't get bogged down by them. Rules are for the guidance of the wise, and the blind obedience of fools - the trick is knowing when it's appropriate to bend/break the rules!! (099).

Both airline and military trainers explained that they try to get their students to

think about what they are doing and the consequences of their actions (or inactions),

and one added “encourage introspection” (173) as part of developing airmanship. It

was clear that the instructors among the participants were concerned that their

students develop good decision making skills. An airline participant noted that “it is

also essential that students be developed by allowing them to make decisions...”

(192), and military respondents added “decision making with the basics always being

used were the main areas for the students to have under control.” (110) and “force

issues upon students where a decision needs to be made. The making of this decision

will generally involve most of the airmanship considerations already discussed.”

(165). Situation awareness is also key aspect of airmanship, and was noted by both

airline and military instructors as something to be developed, for example, “I try to

develop constant, accurate situational awareness.” (042) and “Direction of attention

to ensure the student is picking up the correct clues (traffic etc.) whilst airborne to

update their situational awareness” (185).

There was some mention of developing the right attitude to aviation, which is

included in the Ebbage and Spencer (2003) model of airmanship and mentioned by

Kern (1996) as the first step in developing airmanship. This was most clearly stated

by a GA respondent as:

The most important thing you can do as an instructor is have the student obtain a state of mind that makes him or her an aviator... To develop good airmanship you have to be passionate about the industry you are entering and be wanting and willing to absorb and apply the knowledge and experience you gain (140).

A military respondent stated something similar, almost as a warning, “Be

aware of the likelihood of the development of habit patterns and attitudes in students

who are at a formative stage” (097).

122

Respondents from the GA and military trainers mentioned preparation as part

of the expectation they had of students. A GA respondent noted that he “Promoted

and tested preparation (knowledge and thinking) for flight” (024) and a military

respondent expected “pre-flight preparation (including positive mental preparation

by knowing how the sortie was going to be flown before getting into the briefing

room)” (110).

Creating a routine or ‘work cycle’ in the cockpit was mentioned by two airline

respondents, and communication, Crew Resource Management, and crew co-

ordination were each mentioned once by airline respondents. One airline respondent

stated that “as an instructor, the highlighting of traps and pitfalls and comparing to

previous aircraft types highlights airmanship issues” (013). One GA respondent

noted an expectation of courtesy in stating that “I demand that the pilot consider

others at all times” (111). Finally, a military respondent mentioned the requirement

for flying skill, stating “demonstrate of the discipline, requirement for skill and

knowledge across broad range of operations” (025). This was also the only mention

of discipline.

Some comment about instruction appeared in response to the following

question, but is relevant to Question 9. One of these comments was a serious

criticism of the competence of current instructors:

If there is one criticism of the aviation industry I have, it is the lack of intuitive airmanship skills in many young pilots. I put this down to the lack of “career instructors” now in the industry. Too often pilots are being trained by inexperienced instructors who are not particularly interested in their jobs, as all they want to do is progress to an airline position. So you have the situation where an inexperienced pilot who has not developed good airmanship knowledge / skills, being unable to pass those skills on to the student. (166)

123

Discussion – instructor’s practices. The three stages of development of

airmanship as noted by Kern (1996) and by Ebbage and Spencer (2003) are:

1) explanation of the importance of airmanship (Kern adds to instil motivation

to improve, while Ebbage and Spencer add discussion of case studies as a

method)

2) model and teach good airmanship (Ebbage and Spencer specifying skills,

knowledge and attitudes)

3) evaluate and provide feedback

None of the respondents, individually, clearly articulated the three stages as

presented in the two major models of airmanship. However, the components are

there in many of the responses. There were frequent mentions of discussion and

briefing of airmanship, and of the necessity to be a model of good airmanship. The

“what” of airmanship instruction covered many of the skills, knowledge and attitudes

specified in each model. There were not quite so many mentions of evaluation or

feedback, although some comment was made by a few respondents.

Change in Airmanship over Time

Question 10 asked the participants “Have your ideas about airmanship

changed over time? If so, in what way?” The response to this question may be

influenced by both the sector of training and background (civilian vs military) and by

the time spent flying in the different sectors (current activity). Table 3.20 shows the

responses to the question across the five sectors.

124

Table 3.20

Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 10 Background Civilian Military Current Activity Airline GA Airline GA Military Participants 21 19 21 7 15 Missing answers 2 5 3 0 1 Respondents 19 14 18 7 14

% % % % %

Yes 57.9 57.1 50.0 28.6 64.3 No 26.3 35.7 38.9 57.1 28.6 Other 15.8 7.1 11.1 14.3 7.1 Factors:

Experience 21.1 35.7 11.1 42.9 42.9 General development 10.5 14.3 11.1 14.3 28.6 CRM/Human Factors 10.5 7.1 11.1 14.3 0.0 Criticism of modernity 5.3 14.3 22.2 28.6 0.0 Awareness/content 10.5 0.0 27.8 0.0 0.0 Examples 15.8 7.1 0.0 28.6 0.0 Importance 15.8 0.0 5.6 0.0 7.1 For modernity 21.1 0.0 0.0 0.0 0.0 Training 10.5 14.3 5.6 0.0 0.0 Basics remain the same 5.3 0.0 5.6 0.0 0.0 Eastern Philosophy 0.0 7.1 0.0 0.0 0.0

In general, more than half of the respondents indicated that they felt that their

ideas about airmanship had changed over time, however, the respondents with a

military background but currently active in general aviation differed in that more

than half stated that their ideas had not changed. A chi-square analysis indicated that

there was no significant differences across sectors in the Yes/No responses to the

question (χ² (4, N = 64) = 2.856, p = .582). There were a number of respondents who

made some sort of general comment about airmanship but did not state a clear yes/no

response to the question. These responses are shown as ‘other’. In this question,

unlike Question 3a, the yes or no response was not set up with a tick box to force

response.

Several of those who said that their ideas had not changed, then qualified that

with a general observation such as “Not really changed, just developed more to

125

include other things” (054) or “No, just reinforced” (055) or “Matured but not

essentially changed” (086) .

The lower part of Table 3.20 shows the factors that were mentioned by

respondents in their answers. Responses included reference to experience, most

frequently across all sectors, except for military background airline pilots. A chi

square analysis indicated that any differences between the sectors were not

significant (χ² (4, N = 64) = 5.740, p = .219). Some examples of the use of

experience include: “Initially when I was inexperienced airmanship seemed like a

series of dot points (taxi speed, lookout, etc.). Over time, more experience and

knowledge of aviation in general has helped to define the concept.” (100, airline),

“No, I don't believe my ideas have changed over time, however airmanship is an

evolutionary process as one tends to build on it with experience.” (166, airline), “My

ideas about airmanship have not so much changed, but have evolved and developed

with increasing experience” (021, GA), “Yes. The balance of my importance on the

many aspects involved in airmanship has changed according with my experience.”

(038, military only), and “I have probably become more conscious of the application

of the principles of good airmanship as I accumulated experience and grew older”

(126, military-only).

There were eleven comments overall, that reflected an acknowledgement of

some general development over time. Numbers were too low to run any formal tests,

but the comments were fairly evenly spread across the sectors, with a few extra from

the military-only respondents. Some examples are: “I think I have always been in

pursuit of good airmanship in myself and others. Having the benefit of working with

some fine organizations and some excellent people I think my understanding of

airmanship has matured and become more inclusive” (058, airline), “Yes, as I have

matured more into a flight and instructional role I have found myself being more

hard on myself to set a higher standard” (118, GA). A military+GA participant wrote

“Yes I think I may have observed something in the past and not taken action or

thought ‘someone else will take care of that’. These days I tend to speak up straight

away” (200). Two military-only respondents wrote “Matured but not essentially

changed” (086) and “I guess it is a continual refining of what you knew before and

applying it to your latest situation” (185).

126

Six respondents made mention of CRM, TEM, non-technical behaviours or

human factors. Some comments were positive for example “Not only I have I grown

in that time but the industry has really changed regarding airmanship, safety and

CRM over the past 10 years and in a positive way. Training is improving and the

more we become educated about things like human error and accident prevention the

safer the skies will be. Airmanship is perhaps what links all this together” (043,

airline). Another was “Yes, initially my understanding of airmanship was simply

about being a safe pilot, now I view airmanship as being an array of pilot skills

involving technical and non-technical skills.” (178, military+GA). One comment was

not quite so positive: “As described, it has become de-identified by human factors,

CRM and TEM.” (013, military+airline).

There was some criticism of ‘modern trends’ or new pilots, mostly among

military + airline respondents, but with one comment from an airline respondent,

which was quoted above, regarding the lack of career-instructors and a prevalence of

young inexperienced instructors. This conclusion was echoed by a GA respondent:

“My ideas about airmanship have not changed only that I am concerned about a

growing lack of airmanship and awareness among many recreational pilots” (113). A

Military+Airline participant wrote: “Very critical of the current trend in risk-analysis

as it attempts to quantify and categorise a natural process that is inherent in those

who possess good airmanship. The ongoing process of experienced operators

coaching junior aircrew and/or ostracizing those who are aberrant is now seen to be

old fashioned and not PC” (119). Another noted: “The modern flight deck

environment and extensive use of automation and computerisation has clouded the

airmanship concepts eg. crew programming complex arrival procedures when a more

simple technique would suffice especially when time constrained” (013).

In contrast, there were four airline participants who were more positive about

changes in modern aviation. Two reflected on positive influence of CRM and human

factors research: “Yes. As human factors research has provided great insights into

behaviour during the last 20 years” (173). Another more general approving comment

was “As I get older and I see more new technology at play I realise that Airmanship

today is just as important as always, due to the fact that aircraft are managed around

the sky and airmanship is a required part of the management process.” (040), and

finally, with reference to another more recent initiative, “Operational safety audits

127

have done more than any other initiative to provide pilots with both good and bad

examples of airmanship among their peers. Ultimately, the organisation that provides

the individual pilot with tools to SELF MANAGE, particularly in terms of individual

strengths and susceptibilities, will achieve the highest order of airmanship.” (051).

Several respondents commented that their understanding of airmanship had

improved in terms of awareness of wider content. For example an airline respondent

wrote: “Yes, it was once considered the polite thing to do. Since then, airmanship for

me is much more diverse with many aspects” (043). Among military+airline

respondents three comments are: “Yes. I became more aware of how vague, or

general, the topic can be; but also more aware of the various elements of the subject.”

(034), “As one gets older and is no longer immortal then safety is something one

strives more actively to achieve rather than seen as the inevitable result of sound

aircraft operation” (071) and “I think the whole understanding of airmanship has

changed over the years. We have changed with it. It's not so magical any longer but

we are working at making it much more tangible” (081).

There were five comments that reflected increased awareness of the importance

of airmanship for example “Yes, become much more important in my daily

operations” (053, airline), “Yes. I have become more aware of the principles of

airmanship and the importance of its application.” (097, military) and “It's like

growing up! The older you get, the more important and involved you realize it

is.’(190, military+airline).

Examples drawn from other pilots were raised as a significant factor in

developing their airmanship by six respondents, some examples are: “They have

probably strengthened as a result of seeing how a lack of airmanship can lead to

poor, or in worst case, tragic, outcomes.” (032, airline), “Having the benefit of

working with some fine organizations and some excellent people” (058, airline) and

“My perceptions on airmanship have changed through flying/speaking with more

experienced aviators. Their mistakes have been there to help me to remember not

what to do!” (037, GA).

There were five mentions of training or instruction, one positive: “Training is

improving and the more we become educated about things like human error and

accident prevention the safer the skies will be.” (043, airline) and the others more

128

negative, for example: “yes it is not as good as it was I feel the training is not there”

(138, GA), “We are now expected to ‘teach’ airmanship as a subject, some of this is

appropriate but much about airmanship is how you live.” (140, GA), “Yes, there is

too much checking and not enough training” (191, military+airline).

Finally two respondents noted that the basics have remained the same, and one

GA respondent noted “Yes - I see eastern philosophy such as mediation as having a

role in performance enhancement” (143). It is possible that ‘meditation’ was the

word intended.

Discussion – development of airmanship. There were no significant

differences across the groups of respondents in the initial yes/no response to the

question. There were quite a variety of views expressed, with some people taking

advantage of the survey to voice some cherished complaints about the industry, while

others were more positive. On the whole, most people recognised that their views of

airmanship had changed, developed or matured during their time in aviation. Mostly

this was put down to experience, to the introduction of CRM and HF training or to

the examples of others. Many also just commented on general development,

increased awareness or developing a sense of the importance of airmanship, without

referring to any mechanism through which this might have occurred. The numbers of

comments were too low to carry out any extensive statistical comparisons.

Conclusion – Training and Development of Airmanship

One aim of this project was to investigate whether any differences in the

training of airmanship had occurred with the introduction of human factors and crew

resource management to the theory syllabus. It became apparent that the very

experienced participants were mostly trained before that 1991 introduction,

especially the military background participants. Nonetheless, there did not appear to

be any differences between sectors or eras when asked about inclusion of the

concepts of airmanship in ground training and briefing. The military participants

were more likely to report inclusion of airmanship in in-air training, but differences

between eras was not significant. Decision-making in briefings was mentioned by

more military than civilian participants, but not significantly so. Situational

awareness and lookout were among the most frequently mentioned concepts. The

lack of mention of any self-awareness or self-improvement concepts in response to

129

the questions about initial training is interesting, especially compared with the

number of referrals to these sorts of personal qualities that were mentioned in

response to the question about the development of airmanship

The second aim was to determine whether airmanship training conforms to the

three part formula that Kern (1996) and Ebbage and Spencer (2003) recommended.

As discussed above, none of the respondents, individually, clearly articulated these

three stages. The main components are there in many of the responses, with mention

of briefing, debriefing and having taken note of examples of airmanship or providing

a good example.

Assessment of airmanship without reference to a strict definition appeared as a

fairly common complaint in response to the initial training questions and (less

commonly) in response to the question about current training.

The Instructors among the participants were asked about their practices to

ensure airmanship is developed in their students, and generally referred to the

importance of briefings and of providing a good example. There was an emphasis on

safety and development of knowledge among GA and military respondents, while

airline respondents were more concerned with thinking, decisions and situational

awareness among their students. This relates well to the findings of Mavin (2010)

and Mavin and Roth (2014) regarding assessment in airline training.

The final question in this section, about any change in ideas over time, resulted

in a number of comments about the role of experience. There was also at least some

mention of the influence of CRM and/or human factors training on their development

of airmanship. Criticism of modern techniques and young, inexperienced instructors

by some of the more experienced respondents was balanced by others who

recognized the positive influence of the introduction of human factors training and

other safety related initiatives.

The majority of the participants were trained quite some time ago (62% before

1991) so asking about their ab-initio training provides information that has to be

considered time related. Of those that are current instructors, not all answered the

question about current training practices, nor can it be ascertained whether they were

referring to ab-initio or more advanced training in that question. The inclusion of

airmanship in current training and the way in which this is carried out, may be better

130

considered by a more direct questioning approach, addressed in the next part of the

project, the interviewing of current instructors in military and civilian sectors, at ab-

initio and advanced levels.

131

Chapter 4. Current Airmanship Training and Assessment

The conduct of online surveys, as described in the preceding chapters, allowed

for the questioning of a cross-section of participants in the aviation industry about

their views on airmanship. The questions used were limited in the extent to which

one can be sure that the responses referred to current practice or reminiscence from

past experience. Respondents to these surveys included some current aviation

instructors but the length of the survey and the position of the questions about

training towards the end of the survey resulted in lack of response by some

instructors. It was not clear whether instructors who did respond were referring to ab-

initio or more advanced training, or were reporting ideas that were based on a

combination of all types of instruction they had been involved with. Further directed

enquiry would allow a more detailed examination of the role of airmanship in flight

training.

The initial stage of learning to fly is referred to as ab-initio training. This is

generally carried out in flying clubs on a casual basis, in flying schools catering for

those who wish to make commercial aviation their career, or for flying cadets in the

military, at the Basic Flight Training School (BFTS) in Tamworth. After passing this

course, Air Force, Army and Navy cadets progress separately to further training on

their service’s specific aircraft types.

As a flying career progresses, training becomes more specific. In the civilian

airlines, it is a requirement that pilots undergo training and checking every six

months. In the larger airlines, this check and training process generally occurs in a

simulator environment, although line checks also occur during commercial flights.

Airline pilots can be selected to transfer to different types of aircraft within the same

company, requiring a conversion course. Command training occurs when a pilot at

First Officer level is ready to move up to a Captain position. This training is

generally carried out by more senior Captains who have received training as trainers

within the company. In the RAAF, pilots also have additional training to convert

onto specific types of aircraft. It is also possible to be selected to undergo training to

become a Qualified Flight Instructor at Central Flying School (CFS, East Sale), and

then instruct at BFTS or at other training squadrons within the RAAF.

132

The purpose of this section of the research project was to discover what current

flight instructors in the military and civilian environments both at ab-initio and at

more advanced levels of training, think airmanship is. A further interest is in how

(and if) they include airmanship considerations in the training and assessment that

they carry out. To this end, several civilian flight training schools were approached

for instructors to participate and one of the major Australian airlines were asked for

access to instructors in their training sections. Two additional former airline trainers

(from a different airline) known to the researcher also agreed to participate.

Permission was gained from the RAAF to approach BFTS, the Central Flying School

and a training squadron at Williamtown for participants. The training program at

BFTS is administered by a civilian contractor, and some of the instructors are

civilians, while others are military. Both sets of instructors were sampled, with

permission from the civilian organisation. Interviews were carried out allowing more

depth of discussion than was achieved in the on-line surveys.

Method

Participants

A total of 39 participants from the organisations outlined above were

interviewed. The combinations of civilian and military activities and level of training

undertaken resulted in five groups of instructors: two groups representing the ab-

initio level of military training at BFTS with the instructors divided by whether they

were military or civilian; instructors at the more advanced level within the military,

some of whom train trainers at CFS and others carry out jet fighter conversions at a

Williamtown based Squadron; ab-initio instructors from two civilian flying schools

in regional NSW; and Check and Training Captains from two major airlines, the

advanced level of civilian training. All the participants were male and all had been

active in Australian aviation, whether military or civilian, for at least six years. The

range of aviation experience was from 6 years to 38 years. Experience as an

instructor ranged from 4 months to 17 years.

Measure

The interviews were semi-structured, with some set questions that varied a

little between groups. The interviewer probed and asked for elaboration as necessary,

without leading the respondent. The participants were asked about their years of

133

experience and any tertiary qualifications. Tertiary qualifications were questioned as

the Griffith University undergraduate aviation program has a specific “Airmanship”

course, whereas the other Australian university aviation courses do not, but do have

human factors courses. The first question was simply along the lines of “what do you

think airmanship is?” The interviewer asked for clarification if the participant simply

listed concepts or made general statements.

Once the participant seemed to have covered all their initial points, they were

asked what other skills, knowledge or attitudes might also be a part of airmanship.

Then they were asked to provide an example of a situation in which good airmanship

was displayed, and then an example in which a pilot displayed poor airmanship. The

participants were asked if they specifically mention airmanship in the pre-flight

briefing, and what sorts of things are mentioned. If they were an ab-initio instructor

they were also asked what aspects of airmanship were included in the initial lessons.

They were then asked about the assessment of airmanship and what sorts of

behaviours they look for in making any such assessment. Finally participants were

told about the findings of a survey of instructors (Carrick, Graham, Healey &

Pickard, 2008) which indicated that decision making, safety and situational

awareness were most often mentioned in regard to assessment of airmanship and that

written reports of feedback generally referred to checks, situational awareness and

radio calls in regard to airmanship. The participants were asked if they agreed with

these findings, and would they add anything or not include anything. The interview

schedules are presented in Appendix D. In general each interview ran for about 40 to

60 minutes.

Procedure

Organisations were contacted with a request for volunteers. A convenient date

for interviews was then established. The researcher conducted interviews at the place

of employment in vacant offices, conference rooms, or the participants own office, as

available. The interviews were electronically recorded using an iRiver MP3 player

with a small tape recorder used as a backup. The resulting voice files were

transcribed to word files by a transcription agency, and then checked by the

researcher to ensure that the transcript matched the audio file. The researcher

conducting the interviews had attended training in interview techniques, had

conducted many selection interviews (for CASA and ATSB) and lectures in

134

assessment techniques, including interviews, in undergraduate and post-graduate

psychology programs.

Analysis

The transcribed interview files were loaded into NVivo10 (QSR, 2013) as five

separate projects, one for each group. The analysis process followed the sequence

described in Chapter 2. Two coders were involved, both had undertaken training in

qualitative analysis and the use of NVivo and communicated frequently concerning

the coding process. In the same fashion as the survey analyses reported in earlier

chapters, tree nodes were established representing the main concepts included in

Kern (1996, 2009a) and the aspects of the Ebbage and Spencer (2003) and Edwards

(2013) model not covered by Kern. Additional free nodes were established to

represent further concepts discovered during coding. Cross-checking between the

project files ensured that the free nodes covered the same concepts in each group of

interviews. Once coding was completed, nodes were compared and combined where

there was overlap in concepts. Concepts that were unique to each group or were not

widely mentioned were still represented in the final node set. A total of 67 free nodes

were coded, relating to the main themes of the definition of airmanship (38), general

comments and overviews about airmanship (17) and relating to the training and

assessment process (12). The final coded data set was agreed by both coders to be

reliably and accurately coded. Each participant file was identified by a three letter

code and these codes are used to identify participants when quoting their comments

below.

135

Results – Analysis of Demographics

Experience, education and transcript length were examined to ensure that no

systematic bias was present in the data sets.

Experience and Education

Table 4.1 shows the numbers of instructors in each group and their years of

experience, in aviation in total and as an instructor. The number of participants in

each group with tertiary degrees is also presented.

Table 4.1

Numbers of Instructors in Each Group, Experience in Aviation and Tertiary Degree Status. Civilian

Ab-initio trainers

Civilian Advanced

trainers

Civilian at BFTS

Military at BFTS

Military Advanced

trainers N 7 7 9 6 10

Total experience (years) Range 7 - 17 25 - 38 10 - 32 6 - 18 8 - 23 Mean 11.29 30.28 22.22 11.0 15.5 Std.dev 3.545 4.231 7.496 5.019 6.381

Instruction experience (years) Range .33 - 14 4 - 17 1.5 - 14 .3 - 15 .15 - 15 Mean 7.76 10.0 7.94 4.03 6.37 Std.dev 5.453 4.041 5.0 5.769 5.219

Tertiary Education Bachelor degree 3 0 0 4 4 Masters degree 1 2 0 1 1

The distribution of years of experience as an instructor was non-normal for the

Military BFTS group and the test of homogeneity of variance was significant for

total experience, thus the conservative approach is to use a non-parametric test to

check the differences between the means for both variables. Kruskall-Wallis

ANOVAs indicated that there were no significant differences among the groups

means for number of years of experience instructing (H (corrected for ties) = 4.942,

df = 4, N = 39, p = .293 and Cohen’s f = 0.387). The effect size f could be considered

large, so there is a difference among the means, possible not significant because of

the low number of participants. There was a significant difference among the groups

in terms of mean total experience, H (corrected for ties) = 22.397, df = 4, N = 39, p <

.001 and Cohen’s f = 1.198. The effect size is very large.

136

The civilian advanced trainers were the most experienced group and had the

most time in the job, and the military ab-initio participants were the least

experienced. This reflects the differing expectations in the two sectors: in airlines,

experienced Captains are deliberately chosen for Check and Training roles; in the

military, the training role is only undertaken after one or more operational postings,

but this can occur fairly early on in a military career and as it happened there were

several relatively new instructors among the participants. In contrast, civilian ab-

initio training roles tend to be taken up by low-time pilots building experience

towards their ATPL, although this is not always the case, and the participants

included some long term ab-initio instructors.

The participants were asked about any higher education undertaken. Two of the

civilian ab-initio instructors had completed the BSc (Aviation) and one had the

Master of Aviation Management from the University of Newcastle, one other

instructor had a B Computer Science. Two of the Civilian Advanced group of

instructors held the Master of Aviation Management from Griffith University. None

of the Civilian background instructors at BFTS had completed degrees. Among the

Military BFTS instructors, four had bachelor degrees from the Australian Defence

Force Academy (ADFA) of which only one was related to aviation, being in

aeronautical engineering. One additional instructor in this group held a Master in

Science (Aviation) from University of NSW (UNSW). Four of the Military

Advanced instructors held Bachelor level degrees, none related to aviation. One

instructor held a Master of Aviation Management from University of Western

Sydney. It is not a requirement to have taken a degree program to enter the aviation

industry, however it is becoming more common. Since 1986, military pilots have

been required to take a degree program at ADFA before moving on to the flight

training school at Tamworth, unless they are direct entry recruits or have taken a

degree elsewhere before recruitment. None of the degree programs taken by

participants had a specific airmanship course.

Transcript length.

An analysis of the number of words in each transcript was undertaken, to

determine if there were any differences between the groups. Table 4.2 shows the

relevant data for each group. The tests for normality and skew were acceptable and

Levene’s statistic was non-significant, F(4,34) = 1.750, p = .162, so the assumption

137

of homogeneity of variance was not violated. An ANOVA was statistically

significant indicating difference among the group means for transcript length, F (4,

34) = 2.701, p = .047. The effect size was f = .564 which can be considered a large

effect. Post hoc tests, using Gabriel’s procedure due to uneven group sizes, showed

that the only significant difference was between the civilian ab-initio group and the

civilian advanced group (p = .026). The civilian ab-initio instructors did not talk as

much as the civilian advanced trainers,

Table 4.2

Mean Transcript Word Length for each Participant Group, Standard Deviation and Range. CAB CAD CBF MBF MAD Total N 7 7 9 6 10 39 Mean words 5730.29 9361.14 7180.33 7393.83 7724.00 7483.74 Std deviation 2535.63 2517.04 1233.13 1436.11 2378.08 2280.43 Minimum 2753 7270 5383 4934 3788 2753 Maximum 10021 13675 9120 9265 11594 13675 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced

Results – Definition of Airmanship

Process comments.

Comments were made about the process of defining airmanship, by some of

the participants. Some mentioned the difficulty of defining airmanship, others made

the point that airmanship is all-encompassing and/or made statements that it is very

informal and non-specific. Table 4.3 shows the distribution of these comments across

the groups of instructors. In the discussion of the concepts mentioned by the

participants, any quotations are identified by a three letter participant code.

138

Table 4.3

Percentage of Participants in Each Group, and of Total Participants, who Commented on the Process of Definition CAB CAD CBF MBF MAD Total

N 7 7 9 6 10 39

Comments % % % % % %

Difficult to define 14.3 42.9 0.0 16.7 60.0 28.2 All-encompassing 14.3 71.4 100.0 100.0 60.0 69.2 Subjective 42.9 71.4 11.0 0.0 10.0 25.6 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced

More of the advanced military instructors said that airmanship is difficult to

define than instructors in the other groups. One military instructor stated “it is

broadly talked about all the time, but is almost impossible to define” (MEG) while

another instructor was quite adamant, “airmanship is subjective, I will not define

good airmanship” (MEA). Instructors from the other groups simply stated that it is

difficult to define.

Many of the participants noted that airmanship is an “all-encompassing”

concept, including all of the instructors interviewed at BFTS. Comments from the

civilian BFTS instructors included “we score preparation, technique and airmanship,

to me they all go hand-in-hand” (BCA) “they are all interwoven” (BCB), “it is an

overarching umbrella to your performance” (BCC) and “I don’t think that you can

separate it… it is a combination of everything … how you handle everything”

(BCE). The military BFTS instructors often stated that the boundaries are blurred,

and that preparation, technique and airmanship are all closely linked and explained as

“it is a general feel of how he is operating the whole aircraft” (BME). Comments

from the advanced civilian instructors included that it is a global view and that it is

an integration of many skills. This longer statement from a civilian advanced

instructor illustrates the idea that airmanship is everything put together:

Because sometimes we know we’ll use that, you know, “that was fabulous airmanship” and that just covers a lot of things because the whole operation whether it be technical or soft side of things was well done and put together and the result was what we wanted to see as far as the aeroplane arriving where it’s supposed to arrive safely. (CCE)

139

About a quarter of the participants also noted that airmanship is fairly

subjective and broad brush (CAC, CAD, BCC) and used adjectives like “intangible”

(CCB) “holistic” (CCC) and “unquantifiable” (CCE) or as one advanced military

instructor said: “it is all esoteric… you can’t write it down” (MEA). This instructor

summarised the interview with the comment:

I really can’t go beyond it’s the way that you go about doing your job, and even if the job is exactly the same day after day after day, the environment that you’re working in, the people that you’re working with change on a daily basis and that affects airmanship. So even though a certain proportion of the way that you do your job doesn’t change on a daily basis, the airmanship side of it does and it’s very, very subjective (MEA).

Despite some saying that airmanship is difficult to define, all-encompassing

and unspecifiable, all the participants were willing to talk extensively about what

they thought was involved in airmanship.

Initial definitions.

The participants’ initial reaction to the question “what do you think airmanship

is?”, after explaining the difficulties in defining it, generally resulted in some attempt

at a definition or mention of important aspects. Among the 39 participants, there

were 25 different combinations of concepts mentioned in the first reaction to the

question. Overall, 12 of the participants mentioned situational awareness, in

combination with various other concepts, eight included following of rules and

regulations, and seven talked about decision making. Nine of the responses included

the basic military definition of safe and efficient flight, some with additional

concepts and two included courtesy. Nine of the respondents mentioned common

sense, two of them only included common sense and the other seven talked about it

in combination with other concepts as well. Table 4.4 shows the various

combinations of concepts, and how they were represented across the groups of

instructors. Apart from the numbers of instructors who used the military definition as

a base for their own definition, there were few other patterns, but some common

themes such as frequent mention of situation awareness, decision making and

common sense. The military definition of airmanship, as mentioned in Chapter 3, is

the phrase: “the safe and efficient operation of a flight, both in the air and on the

ground”.

140

Table 4.4

Percentage of Participants in Each Group who Spoke of Listed Concepts in Their Initial Definition of Airmanship CAB CAD CBF MBF MAD Total Number of Respondents 7 7 9 6 10 39

% % % % % %

Concept Combinations Safe and efficient 0 0 9.1 66.7 20 17.9 Safe &efficient, courtesy 28.6 0 0 0 0 5.1 SA (but vague) 0 14.3 0 0 10 5.1 SA and safety 14.3 0 0 0 0 2.6 Care for aircraft, SA, ongoing study 14.3 0 0 0 0 2.6 SA, knowledge, perception, handle self, manage risk 0 0 9.1 0 0 2.6 SA, care for aircraft, safety 0 0 9.1 0 0 2.6 Professional pride, CS and SA 0 0 9.1 0 0 2.6 CS, SA and Safety 0 0 9.1 0 0 2.6 CS 0 0 9.1 0 10 5.1 CS and thinking 0 14.3 0 0 0 2.6 CS and experience 0 14.3 0 0 10 5.1 Responsible conduct CS, courtesy 14.3 0 0 0 0 2.6 Thinking and DM 0 14.3 0 0 0 2.6 SA, DM, rules 0 0 0 16.7 20 7.7 DM and prioritizing 0 0 0 0 10 2.6 Knowledge, attitudes, TEM 0 14.3 0 0 0 2.6 Non-technical skills 0 14.3 0 0 0 2.6 Thinking behaviour and interaction for safe flight 0 14.3 0 0 0 2.6 Preparation and mental agility, ability 0 0 9.1 0 0 2.6 Discipline, rules and regulations, courtesy 0 0 0 16.7 0 2.6 Discipline, knowledge, safety 0 0 9.1 0 0 2.6 Checks, lookout, safety 0 0 9.1 0 0 2.6 Follow the rules 28.6 0 0 0 0 5.1 Reducing risk 0 0 0 0 10 2.6 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced; SA = situation awareness; CS = common sense; DM = decision making; TEM = threat and error management

Once a basic definition was stated, the participants were asked what made up

the components of that definition, or were asked to elaborate on some of the points.

141

The next section deconstructs the full interview in terms of the concepts mentioned

and their distribution over the various instructor groups.

Model related concepts.

The Kern model was mentioned specifically by two of the civilian advanced

group of participants, almost as soon as they had been asked to define airmanship

and by one of the military advanced group, towards the end of his interview. In all

three cases, the participant did not step through the Kern model, but rather, having

noted that they had read Kern (1996) went on to talk about their own ideas about

airmanship. One of the civilian advanced instructors went on to talk about common

sense, knowledge and experience. The other civilian advanced instructor talked about

the use of threat and error management as a structure around which to build

airmanship. The military advanced participant who had read Kern (1996) stated that

he did not find the book particularly helpful and did not discuss any details of the

model as such.

Most of the concepts included in the models of airmanship, Kern (1996, 2009b)

Ebbage and Spencer (2003 ) and Edwards (2013), were spoken about by participants,

but not all of them by any one participant. Table 4.5 shows the percentage of

participants who raised each concept during the interview.

The majority of participants in each group mentioned the capstone outcomes of

the Kern model, situational awareness and decision making (the term “judgement”

was not generally used) at some point in the interviews. Eight participants were

prompted by the interviewer about situational awareness and/or decision making

once the question based on previous research was reached, all but one then agreed

that these concepts were an important part of airmanship and spoke about their

relevance to airmanship. The one exception was a participant in the civilian BFTS

group, who deflected discussion into the idea of thinking through problems and

coming up with solutions, but did not actually mention decision making, as such.

142

Table 4.5

Percentage of Participants in Each Group who Mentioned Specific Definitions or Listed Concepts from the Airmanship Models During Interview

CAB CAD CBF MBF MAD Total

Number of Participants 7 7 9 6 10 39 Concepts – Kern model % % % % % % Kern mentioned by name 0.0 28.6 0.0 0.0 10.0 7.7 Judgement/DM 100.0 100.0 88.9 100.0 100.0 97.4 Situational Awareness 100.0 100.0 100.0 100.0 100.0 100 Knowledge: General 28.6 100.0 44.4 50.0 90.0 64.1 Self 28.6 28.6 66.7 16.7 30.0 35.9 Aircraft 71.4 42.9 66.7 83.3 70.0 66.7 Team 0.0 0.0 0.0 0.0 0.0 0.0 Environment: General 0.0 14.3 33.3 0.0 10.0 12.8 Legislative 100.0 71.4 88.9 83.3 80.0 84.6 Organisational 57.1 71.4 77.8 66.7 90.0 74.4 Physical 14.3 28.6 66.7 66.7 40.0 43.6 Risk 14.3 42.9 22.2 33.3 20.0 25.6 Mission: Planning 100.0 14.3 56.0 33.3 20.0 43.6 Knowledge: Application 14.3 42.9 88.9 100.0 30.0 53.8 Skill 71.4 100.0 88.9 66.7 100.0 87.2 Included in Airmanship 57.1 57.1 22.2 33.3 60.0 46.2 Proficiency 28.6 28.6 11.1 16.7 10.0 17.9 Discipline 28.6 42.9 55.6 16.7 30.0 35.9 Discipline + implied 57.1 71.4 77.8 83.3 50.0 66.7 Additional Ebbage and Spencer or Edwards Concepts Attitudes: Hazardous 14.3 14.3 33.3 0.0 0.0 12.8 Professionalism 28.6 14.3 22.2 33.3 40.0 28.2 Self-improvement 28.6 14.3 0.0 16.7 0.0 10.3 Skills: Communication 71.4 100.0 56.0 50.0 40.0 61.5 Crew co-ordination 0.0 42.9 22.2 16.7 20.0 20.5 Interpersonal 0.0 71.4 0.0 0.0 10.0 15.4 Mgt of flight-deck 28.6 85.7 22.2 16.7 30.0 35.9 Self-assessment 0.0 28.6 22.2 33.3 10.0 17.9 Team leadership 0.0 71.4 11.1 0.0 0.0 15.4 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced

The pillars of knowledge were not all mentioned. No one mentioned

‘knowledge of the team’ specifically – although if one considers this as including

crew resource management and crew related issues, then there was some mention

especially among the civilian advanced trainers, but such comments were coded as

Ebbage and Spencer/Edwards additional concepts or among the free nodes. Just over

143

half of the participants overall (53.8%) made comments that showed a concern with

the application of knowledge during flight, at the right time or appropriately. Nearly

all of the civilian and military instructors at BFTS made such comments. Over half

the participants (64.1%) made a general comment about knowledge. In some cases

this was prompted by the question “What other skills, knowledge or attitudes do you

think might be included in airmanship?” which was used to generate further

discussion.

Knowledge of one’s self was represented by comments about knowing and

applying one’s personal limits (CAG, BCB, BMD, MWB), recognising one’s health

and fitness (BCH, CCD) or when one is sick, seeing the medic (BCA). There was

also one reference to the “I’m safe” mnemonic (CAD) which suggests consideration

of illness, medication, stress, alcohol, fatigue and eating habits before going flying.

Knowledge about the aircraft was frequently mentioned, by 66.7% of the

participants overall. Generally this was expressed as abiding by the aircraft speeds

and limitations, the implication being that the student should know what these are

(CAA, BCA, BCE, BMB). There was also comment about knowing the aircraft

systems (BMA, BME, CCB), knowing the manuals (CCC) and general comments

about knowledge of the aircraft (MEA, MEB).

There were not many comments about the environment in general terms, an

example of this sort of comment is “they’re gonna need to be pretty well educated

about the technical aspects of the equipment they’re using and the environment in

which they use it” (MEC). The most mentioned aspect of the environment was the

legislative environment usually specific mention of following rules and regulations.

Overall, 86.4% of the participants noted this aspect, including all of the civilian ab-

initio instructors, who frequently paired comment about knowledge with comment

about abiding by them, or simply talked of abiding by the rules, implying that

therefore these need to be known. For example, “a good broad knowledge of the

rules and regulations is very important obviously to abide by them” (CAB) and

“you’ve got rules, you must abide by them, don’t go breaking them” (CAC). Finally,

one participant took it a little further, saying “not only a knowledge of the rules, but a

desire, you know, that ethical, moral desire to go out and learn the rules, knowing

that there’s likely to be a rule governing this” (MEG).

144

Knowledge of standard operating procedures (SOPs), and in the military, also

of standing orders and rules of engagement, were coded as knowledge of the

organisation, as well as knowing what an organisation expects, for example, “taking

the time to learn how a particular company operates their aircraft in respect of power

settings, fuel flows…” (CAG). Other comments included “knowing your SOP’s”

(CCF), “knowledge of all the procedures, which would definitely include all

emergency procedures” (BCG), “obviously you need to know the rules of

engagement” (BMA), “he conducts the aircraft in accordance with the student air

training guides, limitations and the standing instructions” (MED).

Knowledge concerning the physical environment was not mentioned as much

as the legislative and organisational environments. It was more frequently mentioned

by the instructors at BFTS, both military and civilian, where it was usually expressed

as awareness of airspace, and also geographic awareness, in terms of where would

one land if suddenly faced with an emergency forced landing, and one instructor

added “knowledge of weather” (BCE).

Risk was not mentioned by many participants, and when it was mentioned, the

comments tended to be about the assessment or management of risk, rather than

knowledge of risk. Most such comments were made by the civilian advanced group

of instructors. One example, which spells out the application aspect, is “the

application of practical risk management on the run, rather than reviewing it as a

desk-top exercise” (CCB).

Knowledge of mission included references to planning of flights with rather

more detail than just a comment about preparation. All of the civilian ab-initio

instructors commented on this aspect of knowledge, just over half of the civilian

BFTS instructors but not many of the military instructors or the advanced civilian

instructors. A typical comment is the following:

their general preparation of even their navigation equipment, maps, charts, the obtaining of appropriate briefing for weather and NOTAMS and making sure that those things have been interpreted and assessed conservatively rather than a “well come on lets go” type attitude (CAA).

Skill was specifically addressed, as part of the question: “What other skills,

knowledge of attitudes does airmanship include”. Although not all interviewees took

145

up the discussion of technical flying skill in this context, overall 87.2% of the

participants made some mention of technical flying skill. The line after ‘skill’ in

Table 4.5 indicates whether the participant included flying skill within their concept

of airmanship. There is a clear difference in frequency of inclusion of flying skill

within airmanship: most of the BFTS instructors, both military and civilian,

specifically excluded flying skill from airmanship – only 2 from each group (33%

and 22% respectively) suggested that flying skill was part of airmanship – despite

saying that the components are all interrelated. This relates to the practice at BFTS of

separately assessing technical flying skill, airmanship and preparation (Carrick,

Graham, Healey and Pickard, 2008). The other three groups included technical flying

skill as part of airmanship about half the time. Proficiency was mentioned

infrequently, only by one or two interviewees in each group.

Discipline, the keystone foundation concept for the Kern model, was

mentioned by 35.9% of respondents across all the five groups. Interestingly,

discipline was least frequently mentioned by the military instructors at BFTS and

most frequently mentioned by the civilian BFTS instructors. Examples are “It’s the

discipline to be well educated in your craft and the discipline to apply your

knowledge to keep the operation safe” (BCC) and “there’s a certain level of ethics

required and self-discipline” (MEG). There is an implication that discipline is

required to adhere to rules and procedures, and not cut-corners, and if comments

which mentioned these aspect are included in the count for discipline, then about

66.7% of participants either mentioned discipline or implied its presence, and the

military BFTS instructors are then shown as implying a need for discipline most

frequently.

Hazardous attitudes were raised by some of the civilian BFTS instructors and

one each of the civilian ab-initio and advanced instructors. They did not use the

specific terminology of hazardous attitudes, necessarily, but used similar wording,

for example, rather than “invulnerability” BCA used “it won’t happen to me”.

Professionalism was mentioned most by the military participants, generally as an

adjective: “a professional desire to fly ... in a safe and efficient manner” (BMB) and

“it comes down to a professional attitude for airmanship” (MEB). Self-improvement

was not frequently mentioned, three out of the four mentions were by civilian

instructors, and included reference to keeping up to date with technology (CAE) and

146

continually striving for improvement (CAG, CCD). Self-assessment is a similar

concept, listed in the Ebbage and Spencer/Edwards skill sets, and was mentioned by

one or two instructors in all but the civilian ab-initio group. This was phrased as

“having a good honest level of self-critique” (BMD) or “a good ability to self-

criticise or self-analyse what they do” (CCA).

The civilian advanced level trainers most frequently mentioned the remaining

skill sets: communication, crew coordination, interpersonal skills, management of the

flight-deck and team leadership. These are all skills closely related to the ideas of

crew resource management and highly relevant for multi-crew flight decks, for

which environment this group of participants are training crew. In contrast, the ab-

initio trainers and the military advanced trainers are more concerned with single pilot

operations, and mention of the above concepts was less frequent.

Of these skill sets, communication was most frequently mentioned across the

groups, overall 61.5% of participants raised communication in relation to airmanship,

and 100% of the civilian advanced group did so. The civilian ab-initio instructors

also raised communication frequently, 71.4% included it in discussion. Comments

from the civilian advanced instructors were generally about sharing information

within the flight deck crew, and comment was also made about some reluctance to

communicate openly in the check and training environment:

They feel if they were to talk openly that they might reveal, incriminate themselves, so to speak. Which we normally try to point out to them that we’re sitting in the back with nothing much to do other than watch what they’re doing so there’s not much that they can tell us that we haven’t already seen but they still have a reluctance to do it. I think … it’s a society thing I suppose. Our society, especially men, are not particularly open to talking about their mistakes. (CCF)

Most of the comments from the ab-initio instructors were concerned with

communication with the Tower and other aircraft, and the use of standard

phraseology on the radio. There were also some comments about encouraging

student pilots to speak up if they feel uncomfortable about what is happening in the

air. One of the military advanced instructors related the importance of

communication in terms of rank and hierarchy, noting that this is not generally an

issue but that junior officers should have the communication skills to be able to

express their views (MEB).

147

Additional concepts.

There were many other concepts that were brought up in the interviews that did

not fit within the framework presented by the models of airmanship. Table 4.6

presents the concepts that were relevant to the definition of airmanship and related to

behaviours or qualities.

Table 4.6

Percentage of Participants in Each Group who Mentioned Listed Additional Concepts Relevant to the Definition of Airmanship CAB CAD CBF MBF MAD Total Number of Respondents 7 7 9 6 10 39

% % % % % %

Military definition 28.6 14.3 22.2 83.3 50.0 38.5 Personal qualities 85.7 85.7 100.0 100.0 70.0 87.2 Safety 85.7 57.1 100.0 100.0 50.0 76.9 Preparation 57.1 42.9 88.9 83.3 90.0 74.4 Experience 57.1 57.1 22.2 83.3 90.0 61.5 Attitude 57.1 85.7 33.3 50.0 50.0 53.8 Prioritising 14.3 71.4 44.4 66.7 60.0 51.3 Common sense 42.9 85.7 44.4 33.3 50.0 51.3 Thinking Cognition 14.3 85.7 55.6 16.7 60.0 48.7 Captaincy/Command 42.9 28.6 55.6 66.7 10.0 38.5 Foresight 42.9 28.6 66.7 50.0 0.0 38.3 Courtesy 100.0 28.6 11.1 33.3 10.0 33.3 TEM/error management 28.6 71.4 44.4 16.7 0.0 30.8 Efficiency 28.6 0.0 22.2 83.3 0.0 23.1 C R M 0.0 28.6 0.0 33.3 30.0 17.9 Responsibility 42.9 28.6 0.0 16.7 0.0 15.4 Information gathering 0.0 28.6 0.0 0.0 40.0 15.4 Time management 0.0 0.0 44.4 0.0 10.0 12.8 Human factors 28.6 0.0 22.2 0.0 0.0 10.3 Authority gradient 0.0 42.9 0.0 0.0 10.0 10.3 Automation 0.0 57.1 0.0 0.0 0.0 10.3 Non-Tech Skills 0.0 42.9 0.0 0.0 0.0 7.7 Behaviour 0.0 28.6 0.0 0.0 0.0 5.1 Multi-tasking 0.0 14.3 0.0 0.0 10.0 5.1 Cooperation 0.0 14.3 0.0 0.0 0.0 2.6 Situation management 0.0 0.0 0.0 16.7 0.0 2.6 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced

148

The formal military definition of airmanship is “safe and efficient flight both in

the air and on the ground”, which is more a statement of the outcome of good

airmanship. This definition was quoted across all the groups, but most frequently by

the military BFTS group. Two civilian ab-initio instructors added courtesy, phrasing

it as “safe, efficient and courteous flight” as their initial response (CAA, CAG). One

participant from the civilian advanced group and a few from each of the military

groups did not use this definition in their initial response but brought it up later in the

interview, so the figures in Table 4.6 differ from those in Table 4.4. The definition of

airmanship as being safe and efficient flight was used more frequently by the military

than civilian instructors, as would be expected.

Many of the participants spoke about the personal qualities which they felt

were relevant to airmanship, both in the positive and negative sense. Concepts coded

as personal qualities included honesty, integrity, commitment, diligence, intelligence

and motivation. On the negative side, distractibility, complacency, over-confidence

and fear of failure were mentioned as not conducive to good airmanship. Overall,

87.2% of the participants raised one or more such personal qualities.

Safety was the concept next most frequently mentioned, overall 76.9% of

participants mentioned safety as part of or as an outcome of airmanship. It is

interesting that safety was more frequently mentioned by the ab-initio instructor

groups (86 to 100%), rather than the instructors at the more advanced level (57% and

50%). Possibly safety is more of a concern for instructors and the ab-initio level,

where inexperienced student pilots may be more likely to become unsafe than the

more advanced pilots.

Preparation seemed to be more of concern to the military (including civilian

instructors at BFTS) than to the civilian instructors. Preparation is something that is

assessed at the ab-initio level in military training, explaining its frequent mention.

Generally preparation was raised in reply to the questions about good or poor

examples of airmanship. These responses generally began with a comment about the

level of preparation evident in the student. Civilian instructors talked more about

flight planning which was coded as “Mission”, about which the BFTS instructors did

not talk as much.

149

Experience was included in the initial definition of airmanship by two

advanced level instructors (one civilian, one military) and was mentioned later in the

interview by a further twenty-two participants across all the groups. Nearly all of the

military instructors spoke about experience, and about half of the civilian instructors.

In general, experience was considered to be important in the development of

airmanship and that thus training provides a variety of experiences. Instructors stated

that situation awareness and decision making improve when the pilot has more

experience to help make sense of situations and inform decisions.

Attitude was included in the early probe question “what other skills, knowledge

or attitudes do you think are included in airmanship?” Not all participants talked

about attitude, and some of the comments coded overlapped with personal qualities,

like arrogance (or lack of it) or willingness to take correction. Having a good attitude

or an attitude of trying to be the best that you can possibly be, were mentioned by

civilian instructors (CAE, CCD). Other comments included that good airmanship

involves a professional attitude, a positive attitude and/or general attitude towards

flying.

The idea that prioritising is part of airmanship came up frequently with all

except the civilian ab-initio group of instructors. A commonly used phrase was to list

the terms “aviate, navigate, communicate” as the ordering of priority, with some

adding “administrate” as the final, lowest priority activity.

Common sense was included by instructors across all groups, most frequently

among the civilian advanced level participants. In general common sense was seen as

being responsible, logical and aiming for safe outcomes. It boiled down to acting for

self-preservation with concern for safety. One civilian advanced instructor

summarised common sense as the process of asking “is it safe, is it smart, is it

legal?” (CCG). Based on the speculation suggested in earlier analysis of survey data

that more experienced aviators may be more likely to equate airmanship with

common sense, an additional analysis was conducted to compare the mean

experience (both total and as an instructor) between those who mentioned common

sense and those who did not, combined over groups. Neither comparison was

significant, but there was a small to medium effect size, slightly greater for total

experience. The relevant values are presented in Table 4.7

150

Table 4.7

Comparison of Means of Years of Experience by Mention of Common Sense Mention of

common sense

N Mean (years)

Std Dev

t df p Effect size d

Total experience

No 19 16.158 8.092 1.456 37 154 .4664

Yes 20 20.250 9.375

Instruction experience

No 19 6.332 4.927 1.110 37 274 .3556

Yes 20 8.167 5.369

Comments about thinking, cognition or processing of information were made

by nearly half of the participants. The civilian advanced trainers made the most

mention of thinking. Their emphasis was that airmanship is thinking about

consequences of actions and having a plan ready for whatever might happen. The

military advanced trainers also considered airmanship to be more about thinking and

the thought processes behind decisions than about skill on the controls. It is not

necessarily about making quick decisions every time, but rather recognising that

there may be time to think things through more thoroughly in some circumstances.

To several of the ab-initio instructors, it was important that students think about what

they are doing, and they included this in airmanship. One instructor went as far as to

say that he saw his role mostly as teaching the students to think in certain ways, and

that if he can achieve that then everything else gets sorted out easily (Civilian BFTS,

BCG).

The idea of captaincy or command was raised by some of the ab-initio

instructors, although two of the civilian advanced trainers also spoke about command

abilities. Captaincy is part of the assessment of airmanship at BFTS (Carrick,

Graham, Healey and Pickard, 2008) and tends to cover decision making or the

decision making process, but also includes the integration of the demands of the

exercise with knowledge of rules and regulations and thinking ahead to end up well

positioned for the next stage of the sortie. One military BFTS instructor stated “It’s

not just decision making, again you know, you’re in charge of the aircraft, and whilst

you’re flying you’re making decisions all the time… which may encompass a

situation and what you are going to do” (BMA). The civilian ab-initio instructors had

a similar idea of command, one saying “I need to have people developing command

151

ability and command attitude, which involves significantly their ability to make safe

and prompt decisions” (CAA). There is an obvious overlap here with the outcome

level of Kern’s model of airmanship, the idea of good judgment or decision making,

but it is interesting that is given a different title or label by some participants: rather

than just talking of making decisions, this is “Captaincy” with an obvious initial

capital. Also significant is the idea that this is a skill that is developed early on. Not

all instructors who talked about decision making called it Captaincy, but all

Captaincy discussions involved mention of decision making (and the comments were

also coded as such).

Courtesy was predominantly mentioned by civilian ab-initio instructors. All of

this group mentioned courtesy in one form or another, including, as mentioned

earlier, as part of a formal definition of airmanship, or simply mention of

consideration for other pilots operating in the airspace. A couple of the civilian

advanced instructors mentioned courtesy, one saying that it was a reflection of the

history of the gentlemanly art of flying but really has nothing to do with real

airmanship (CCB) and the other commented that being courteous and polite is way to

ensure good relations and exchange of information with others (CCC). Two of the

military BFTS instructors also mentioned courtesy, as good manners and politeness

in operating among other pilots, and that it is part of airmanship (BMA, BMC).

Similarly, the one participant mentioning this concept from among the civilian BFTS

instructors suggested that manners are part of good airmanship (BCA). The one

military advanced instructor who mentioned courtesy, also commented that it should

not get in the way of getting the job done and said “I guess there’s no nice way to say

yes put those bombs down now, how do you be nice to someone you’re about to

shoot?” (MEB).

Foresight was coded among the free nodes as something rather more than the

part of situation awareness that suggests pilots should be predicting current events

into the future. This node included comments that covered the concept of thinking

ahead to the next part of the sortie or flight, planning what might happen next or

predicting the movements of other aircraft, mentioned independently of definitions of

situational awareness. More of the civilian BFTS instructors, than other instructor

groups, mentioned some aspect of foresight. They emphasised the idea that the

trainee should always be thinking ahead (BCA, BCB), should always be ready and

152

have an ‘escape route’ if things went wrong (BCE). The military BFTS instructors

expressed this as thinking ahead of the aircraft (BMC, BMF). One civilian advanced

instructor spoke about pulling trainees up for not thinking far enough ahead nor

thinking about the ramifications of their actions further along in the flight (CCG).

Foresight was not coded among the responses of the military advanced instructors,

while they did speak about situational awareness and a few mentioned the aspect of

prediction into the future, none made a point of thinking ahead or exhibiting

foresight as part of airmanship.

Threat and error management (TEM) was mentioned as such by civilian ab

initio and civilian advanced instructors. The military trainers also talked about

management of error, but without using the TEM label, and this has been recorded

alongside TEM in Table 4.6 above. The use of the TEM model was introduced into

the ab-initio “Day VFR” syllabus in 2010, and has been part of the regular crew

resource management training in airlines for some time. The concept has clearly been

incorporated into ideas of airmanship among the civilian instructors.

Crew resource management (CRM) was not mentioned by many of the

instructors. It was mentioned by some advanced civilian instructors as may be

expected since it is a required part of airline training. However, only two civilian

advanced instructors mentioned CRM, as such, despite many of them including the

skill sets that make up CRM in the discussion. Five of the military instructors

mentioned CRM. One military BFTS instructor used the term in passing while

discussing situational awareness, saying “it’s a standard crew resource management

type scenario where you are using all your sources to update your model and change

anything if needs be” (BME). Another military BFTS instructor qualified some

comments as “in the training environment”, prompting the researcher to ask about

the real world which led to a discussion of multi-crew flying and the comment

I’m sure you’re aware with the whole crew resource management thing. I think that’s linked to a degree with airmanship as well because again it comes down to the good manners and the ethics of aviating. …I’ve heard of some real classic breakdowns of CRM that lead to, you know, possible breaches of safety for the aircraft (BMC).

One of the military advanced instructors gave a more positive example, saying

that the outcome of decision making is more acceptable if:

153

you’ve taken the time to make a logical, thoughtful decision that you’ve run past the crew, you know, the CRM and all that sort of thing, you don’t always have to have a consensus but you’ve got a general feeling that yes there’s a reason to do this (MEG).

A further two military advanced instructors just commented that CRM is a part

of good airmanship.

Responsible conduct was included in the definition of airmanship by one of the

civilian ab-initio instructors, and mentioned by another as important a bit later in the

interview. A third said that employees have a responsibility “to ensure that a

particular person is trained and mentored properly” (CAG). Two of the civilian

advanced instructors stated that responsibility was part of airmanship, with one of

them relating this to discipline and not cutting corners. One of the military BFTS

instructors mentioned responsible conduct among all aircrew, not just pilots.

Information gathering was mentioned specifically only by instructors from the

military and civilian advanced training groups. Two of the civilian trainers

mentioned collecting information preparatory to making a decision, and four of the

advanced military instructors included the taking in and assimilating of information

in a reasonable time-frame as part of their definition of airmanship, and separately

from the general idea of perception and recognition of input that is part of situational

awareness.

There are a number of concepts that were mentioned only by a few civilian

advanced instructors towards the end of Table 4.6. These are authority gradient,

automation, non-technical skills, behaviour (that is, specifically identifying

airmanship as a form of behaviour), multi-tasking and co-operation. These are all

skills or concepts that are more relevant in multi-crew environments and in

commercial aircraft, or are talked about more in that context, so it is unsurprising that

these were only mentioned by instructors in that environment.

Mention of human factors was made by the civilian ab-initio and civilian BFTS

instructors only, as being relevant to or equivalent to airmanship. Human

performance and limitations has been part of the Day VFR syllabus since 1993, so

the civilian instructors should be well aware of the importance of this aspect of

knowledge.

154

External considerations.

In discussion of the nature of airmanship, a number of activities, actions and

considerations were mentioned by participants that would need to be executed or

taken into account in demonstrating good airmanship. These are shown in Table 4.8.

Table 4.8

Percentage of Participants in Each Group who Mentioned Listed Actions and Considerations Relevant to the Definition of Airmanship

CAB CAD CBF MBF MAD Total

Number of Respondents 7 7 9 6 10 39

% % % % % %

Checks 85.7 57.1 100.0 100.0 30.0 71.8 Lookout 100.0 14.3 100.0 100.0 30.0 66.7 Radio 85.7 57.1 100.0 66.7 30.0 66.7 Weather 71.0 71.0 67.0 17.0 40.0 53.8 Pre-flight Inspection 100.0 0.0 55.6 50.0 30.0 46.2 Go-round criteria 28.6 14.3 66.7 66.7 30.0 41.0 Care for aircraft 85.7 14.3 44.4 50.0 10.0 38.5 Admin paperwork 42.9 0.0 44.4 16.7 40.0 30.8 Traffic 28.6 0.0 44.0 83.0 10.0 30.8 Fuel 28.6 57.0 22.2 16.7 20.0 28.2 Pressure 14.3 57.1 33.0 17.0 20.0 28.2 Taxi 42.9 0.0 44.4 50.0 0.0 25.6 Work cycles 0.0 14.3 55.6 66.7 0.0 25.6 Passengers 28.6 0.0 22.2 0.0 0.0 10.3 Hand over 14.3 0.0 11.1 0.0 0.0 5.1 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced

Carrying out checks and checklists was more relevant at the ab-initio level, as

were lookout and use of and listening to the radio. These are concerns that need to be

actively taught at the ab-initio level and perhaps are assumed at the more advanced

levels of training, and this is reflected in the frequency of mention in regard to

airmanship. In fact, one civilian BFTS instructor initially defined airmanship simply

as doing checks and keeping a good lookout.

Weather, traffic and fuel were considerations raised by many of the

participants, though not evenly across the groups of instructors, in their descriptions

155

of good or poor examples of airmanship. The monitoring of fuel was raised by a few

participants in each group. Comments included “not running out of fuel” (BMA,

CAB) as part of a list of good airmanship practices, managing fuel (BCB, BCG),

being aware of remaining fuel (CAA, MWB, MWC) and balancing fuel load against

expected weather (CCB and CCG).

The next most frequently mentioned aspect of aviation mentioned in discussion

of airmanship was the carrying out of pre-flight inspections, that is, the checking of

oil and fuel and general serviceability of the aircraft before flight. Civilian ab-initio

instructors all mentioned this aspect of airmanship. The civilian advanced trainers

did not mention this at all, and this may be because they were simulator trainers, so

pre-flight checks would not be relevant in that situation. About half of the instructors

at BFTS mentioned this, and fewer than a third of the military advanced instructors.

‘Go-round criteria’ was listed among the free nodes as part of airmanship.

These are the criteria by which the pilot decides whether it is save to land, given how

well (or how badly) the approach has gone. There is a formal set of criteria used in

military instruction, and thus the compliance with these criteria was far more

frequently mentioned by the BFTS instructors.

Caring for the aircraft was an interesting inclusion in airmanship,

predominantly among the civilian ab-initio instructors, but cropping up fairly

frequently among the BFTS instructors as well, more-so than among the trainers at

the more advanced level. This included things like not starting the aircraft on gravel

(which damages the propeller), not taxi-ing over boggy ground, looking after the

aircraft generally, and being aware of temperatures and pressures and other

limitations of the aircraft and engine. The consequences of not caring for the aircraft

may be expensive repairs or more frequent maintenance, which can eat into the profit

of a flying school or small company so the instructors may be instilling a commercial

consideration at an early stage of training. Severe overstressing of the aircraft in

flight and ignoring extreme engine indicators can lead to airframe or engine failure,

with possibly fatal consequences, so there is certainly a safety aspect to this concern.

Administration and paperwork were regarded and mentioned as part of

airmanship by a few pilots in each group, except the civilian advanced group. These

156

comments referred to the necessary paperwork that needs to be done at the end of a

flight or sortie: the filling in of logs, maintenance reports and student records.

Taxi speed and considerations whilst taxiing were mentioned by some

participants in all the ab-initio groups, but not the military and civilian advanced

instructors. This may be because it is something that needs to be actively taught at

the ab-initio level, and assumed at the more advanced levels.

‘Work cycles’ is a term used by the military to refer to the structure of

practices in the cockpit, thus most mentions were from civilian and military BFTS

instructors. There was one additional mention from a civilian advanced instructor,

who happened to be ex-military. He stated that “flying tends to be made up of

routines and perhaps work flows, patterns, cycles of activity” (CCB) and the

suggestion that this assists with cognitive capacity.

Time pressure and commercial pressure were mentioned by four of the civilian

advanced instructors as something that has to be coped with while on the job, and

thus part of one’s airmanship. One ab-initio instructor spoke about some students

putting pressure on themselves to get things done, whereas he felt that simply putting

off the flight rather than rushing makes more sense (CAG). There was also mention

of stress during flight tests for ab-initio students.

The passenger brief was mentioned by two of the civilian ab-initio instructors,

as being something that should be carried out, and thus part of good airmanship

(CAA, CAB). The two mentions by the civilian BFTS instructors differed, one

including the safety of passengers as part of the definition of airmanship (BCG), the

other suggesting that thinking ahead and flying smoothly gives a more enjoyable

flight for the passengers (BCE). The “handing-over, taking over” exchange was

mentioned as one of the first airmanship lessons in the context of always being aware

of who has control of the aircraft with the formal exchange taking place in the

instructional environment (CAC, BCF).

Discussion - Definition of Airmanship

Similarities and Differences among Groups

There was agreement between the different groups of instructors about the

capstone outcomes of airmanship: situational awareness and decision making. All the

157

instructors included these in their discussion of airmanship. Of these 33 included

these concepts in their discussion of the definition of airmanship. The remaining six

(15%) included them after being prompted by a later question. There was also

general agreement that airmanship involves the knowledge and application of the

rules and regulations that form the legislative environment and the operating

procedures that are part of the organisational environment. Personal qualities were

brought up by many of the participants, across all groups. While the specific qualities

mentioned varied, it is interesting that so many felt that airmanship also included

aspects of the individual’s personality. There was also consistency across the groups

in the lack of specific mention of ‘knowledge of the team’ as such.

Differences between the groups could not be tested statistically, due to the low

numbers of participants in each group. However, the frequency of comments can be

contrasted informally by looking at which groups mentioned particular concepts

most often.

Instructors from the civilian ab-initio group made more mentions of mission

planning, self-improvement, and courtesy than did instructors from other groups.

They made next to highest mention of communication (after the civilian advanced

group) and safety (after the BFTS instructors). They made fewer mentions of the

physical environment, organisational procedures, knowledge of risk and the

application of knowledge than the other groups of instructors. Working in the volatile

GA environment preparing student pilots for independent flying may have influenced

the emphasis on adequate flight planning and the concern with communication and

courtesy in dealing with others.

The civilian advanced instructors made more mentions of multi-crew related

concepts, as these relate closely to their activities training within multi-crew

operations. This group made most mentions of communication, crew coordination,

interpersonal skills, management of the flight deck, team leadership and threat and

error management, and the only mentions of authority gradient, automation

management, non-technical skills, behaviour, multi-tasking and cooperation. They

also made most comments about thinking or cognition, attitudes and common sense.

They made least mention of aircraft knowledge and mission planning, and were next

to lowest on mention of knowledge of the physical environment. These later points

may be because of the simulator environment in which this group works.

158

The civilian instructors, both ab-initio and advanced, included mention of

proficiency, communication and responsibility more frequently than the other

instructors: and they were in the middle grouping in mentions of experience.

Civilian instructors at BFTS made the least mention of the role of experience

within airmanship, but made the most comments about knowledge of self, foresight,

time management and hazardous attitudes. They mentioned thinking about as often

as the military advanced instructors, less often than the civilian advanced group but

more than the civilian ab-initio and military BFTS groups. The military BFTS

instructors made the most mention of efficiency and the only mention of situation

management. Together the BFTS instructors made most mention of safety,

knowledge of the physical environment, captaincy and the application of knowledge,

and with the civilian advanced group, made most mention of discipline or implied a

disciplined approach. They did not tend to include flying skill in airmanship. The two

BFTS instructor groups, with the military advanced group, made most comment

about preparation. The role within BFTS of these instructors would have influenced

their comments about Captaincy, emphasis on preparation and separation of flying

skill from airmanship as these are marking criteria for this organisation. Other

concepts which they mentioned most are aspects that perhaps are emphasised more

in the military training environment than elsewhere. Given the contrast between the

initial statements of BFTS instructors that airmanship is an overarching concept and

that the three areas that are assessed are intertwined, it might be appropriate to re-

label what is currently assessed as airmanship, as “non-technical skills” and abstract

“airmanship” to represent the integration of the three assessable areas in terms of the

overall safe and efficient conduct of the flight.

The military advanced instructors mentioned information gathering more than

other instructors, and with the military BFTS instructors made most mention of

experience playing a part in airmanship. The two military groups, along with the

civilian advanced group, commented most on prioritising. The backgrounds of these

instructors either in combat or military service roles may have ensured prompt

processing of data and appreciation of the role of experience. Operational activity

would have led to the understanding of the importance of prioritizing tasks on a high

demand flight-deck, something also raised by the advanced civilian instructors,

veterans of busy airline operations.

159

Comparison with Survey Results

The relative importance (in terms of frequency of mention) of the concepts that

are part of airmanship raised in the interviews is very similar to the outcome of the

survey reported in the previous chapter. Table 4.8 compares the results combined

across groups, with the outcome from the on-line survey.

The interviews led to a higher percentage mention on most of the concepts than

did the survey, due to the different method with the interview allowing more probe

questions and questions about specific aspects of airmanship than did the survey, and

the participants in the interviews talking much more than the survey respondents

were willing to write down, so that there were more areas covered in the interviews.

The consistency across the two data collection formats and sets of questions

used indicates the robust nature of the idea of airmanship among Australian aviators.

Nineteen of the top 25 concepts are the same, although the ordering of concepts is

not exactly the same, it is clear that situational awareness, decision making and

safety are the central concepts in the formulation of airmanship. Personal qualities

are high on the lists, which may reflect a concern for the kind of person an aviator is.

It should also be noted that there was not much consistency among the qualities that

were coded, just that there tended to be a feeling that some qualities promoted good

airmanship and less positive qualities detracted from it.

Knowledge concepts, especially of rules and regulations (legislative),

procedures (organisational) and of the aircraft, were all high on the lists.

Communication came in eleventh on each list, just ahead of experience, with

common sense not far behind. Of the six concepts that were in the top 25 from the

interview data but not from the survey, three did appear further down the list in the

survey data – prioritising, thinking and foresight. Captaincy, preparation, and attitude

did not appear in the survey data, although hazardous attitudes did. The six survey

concepts that were not in the interview top 25 concepts, did all appear lower in the

list from the interview results: knowledge of risk, crew coordination, proficiency,

professionalism, hazardous attitudes and general comments about the environment.

160

Table 4.9

Concepts Constituting Airmanship Ordered by Percentage of Participants who Mentioned Each, from the Interview Data and from the Online Survey (top 25) Interview data N=39

Survey data N=83

% % Situational Awareness 100.0

Safety 74.7

Judgement/Decisions 97.4

Situational Awareness 59.0 Personal qualities 87.2

Judgement/Decisions 55.4

Knowledge Legislative 84.6

Knowledge Aircraft 47.0 Safety 76.9

Personal qualities 47.0

Knowledge Organisational 74.4

Knowledge Legislative 44.6 Preparation 74.4

Knowledge Organisational 41.0

Discipline 66.7

Skill (technical) 39.8 Knowledge Aircraft 66.7

Efficiency/effective 36.1

Knowledge – general 64.1

Knowledge: Risk 33.7 Communication 61.5

Communication 33.7

Experience 61.5

Experience 30.1 Attitude 53.8

Crew co-ordination 26.5

Common sense 51.3

Knowledge - general 25.3 Prioritising 51.3

Common sense 25.3

Thinking Cognition 48.7

Courtesy 25.3 Skill (technical, included) 46.2

Knowledge: Self 24.1

Knowledge Physical Env 43.6

Discipline 19.3 Knowledge Mission 43.6

Knowledge Phys Env 18.1

Captaincy/Command 38.5

Knowledge Env General 15.7 Safe and efficient 38.5

Proficiency 15.7

Foresight 35.9

Professionalism 15.7 Knowledge Self 35.9

Knowledge Mission: 14.5

Management of flight-deck 35.9

Hazardous Attitudes 14.5 Courtesy 33.3

Management of flight-deck 14.5

Note. Env = environment; Phys = physical.

The ubiquitous appearance of common sense is interesting. It is not something

that is included in either model of airmanship, nor perhaps should it be. “Common

sense” seems to be a short cut way to express using one’s acquired knowledge

sensibly and appropriately and not taking unnecessary risks. There is also an

implication that this knowledge has been so well integrated that it is what the

interviewee sees as something any normal aviator would abide by. That there is an

near medium effect size in relation with experience supports the idea that the more

experienced aviators are more likely to speak about common sense, as they have had

more time to develop that integration of knowledge and practical application.

161

Comparison with the Models of Airmanship

The analysis of the interviews is consistent with the basic structure that the

Kern model indicates. However, the results also draw attention to the areas that were

not at the forefront of that model – the aspect of operations in multi-crew

environments, and support the Ebbage and Spencer and Edwards models which add

emphasis in this area. Also, it was evident that simply listing areas in which

knowledge is required is not sufficient in the active training environment, where

there was emphasis on the application of knowledge, the ability to know when and

how to use acquired knowledge. Discipline was viewed rather as compliance with

rules, regulations and procedures. There is a contrast here with the survey data

reported in the previous chapter, where there were several comments that rules had to

be applied with a certain amount of common sense – the implication being that there

are occasions in which breaking a rule may be the best course of action.

Communication was again brought up in all areas, as part of good airmanship, and

probably requires more conspicuous placement in models of airmanship.

162

Results - Training and Assessment of Airmanship

The interview schedule included some specific questions about the place of

airmanship in the training programs at the various organisations from which

instructors were interviewed. These included questions about inclusion of airmanship

in the pre-flight briefing as well as the aspects of airmanship included in the initial

flight lessons (or sorties). The participants were also asked about the assessment of

airmanship during training in their organisations. There was also a lot of free range

discussion of training generally, including topics such as post-flight debrief, dealing

with students, handling of student errors and the training of instructors.

Airmanship in the Training Program Structure

There were a number of consistent types of comment made about the inclusion

of airmanship within a general structure of training programs at the relevant training

levels. The comments included that at advanced training levels, airmanship is

assumed to be adequate. The inclusion (or not) of airmanship in the pre-flight brief

and post-flight debrief was discussed and some participants commented on the

learning rate expected in ab-initio training. The combination of the idea that student

pilots will learn airmanship from the behaviour of others and the requirement for

instructors to provide a good example or airmanship, as well as the influence of the

organisational culture in which the training occurs were also raised. There were some

related comments on the progressive introduction of new content within the program

and the instructors reported observation of the development of airmanship over time.

Finally, there was some discussion of curriculum issues associated with training

programs, student record keeping and the relevance of ground school instruction.

Table 4.10 summarises the distribution of such comments across instructor groups. In

the discussion that follows, participants are identified by a three letter code (e.g.

CAA).

163

Table 4.10

Percentage of Participants who Commented on Aspects of the Training Program

CAB CAD CBF MBF MAD Total

Number of Participants 7 7 9 6 10 39

Comments % % % % % %

Assumed at this level 14.3 42.9 0.0 0.0 60.0 25.6 Pre-flight brief/talk 100.0 14.3 100.0 100.0 70.0 76.9 Post debrief 42.9 42.9 66.7 50.0 80.0 59.0 Learning rate 14.3 0.0 22.2 33.3 0.0 12.8 Learn from others 14.3 57.1 11.1 16.7 10.0 20.5 Provide an example 14.3 57.1 33.3 50.0 40.0 38.5 Organisational culture 14.3 71.0 0.0 33.3 40.0 41.0 Learn+example+culture 42.8 85.7 33.3 66.7 50.0 53.8 Progression 0.0 14.3 55.5 83.3 40.0 38.5 Development with time 0.0 42.9 22.2 83.3 30.0 33.3 Curriculum structures 0.0 0.0 66.7 16.7 0.0 23.1 Student records 14.3 0.0 44.4 66.7 0.0 23.1 Ground school 0.0 0.0 44.4 66.7 10.0 20.5 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced

Airmanship assumed. When airmanship training began to be addressed with

the participants, there was an interesting pattern of response among the participants.

Many of the advanced trainers (52.9%), especially in the military, made comments

about the experience that had been acquired by their students by this time in their

careers and that adequate airmanship is assumed at their level of training. Three of

the advanced civilian instructors noted that by the time pilots got to fly the sort of

aircraft they instructed on (large public transport aircraft) they have reasonable

airmanship (CCA), are assumed to have it (CCB) or are all good pilots (CCF). The

military advanced instructors made the point that the pilots they are dealing with

have all had squadron experience (MEG), “have above average airmanship” (MEC,

MED) and “there is nothing new to teach them about airmanship” (MEA). The one

ab-initio instructor who made this type of comment was referring to a higher level of

training, indicating that airmanship is not briefed or taught as much at that higher

level (CAB).

164

Pre-flight brief. The ab-initio trainers and Williamtown advanced military

trainers were asked if airmanship was included in the pre-flight briefs, the military

advanced trainers at East Sale were asked if they talked to the flight instructor

candidates (FICs) about airmanship, however the advanced civilian instructors were

not all asked an equivalent question.

All the civilian ab-initio instructors noted that airmanship issues were covered

in the pre-flight briefing from the first flight. Briefings were standardised within the

organisations and there would be four to seven airmanship points at the end of each

brief. The airmanship content of the briefs would depend on what was being covered

in the flight, for example safety considerations, situational awareness, weather

effects, the rules and regulations relating to the specific flight (such as right of way

entering the circuit), clearing turns and checklists.

The military and civilian BFTS instructors were consistent in all saying that

airmanship is always covered in their briefs. The pre-flight briefs are standardised

starting with the aim of the sortie, then the airmanship points relevant to the sortie

and finally any additional considerations (BMB, BMC, BMD, BME). These

participants noted that the airmanship points vary according to the sortie and how far

through the training program the student has progressed (BCA, BCD, BCI, BMA).

They stated that lookout and situational awareness are included throughout the

program (BCB, BCC, BCG, BCH, BMF) and other topics initially might include

safety, pre-flight checks and ten-minute checks (BCA, BCE). The “considerations”

are things that are not so vitally important to the exercise but would be good to see

demonstrated. Items in the “considerations” section in the early part of the flying

program move up into the “airmanship” section in later sorties, although not

everything is listed every time (BCF, BMD). One participant noted that the students

also had mass briefs in the classroom about the flying program before fronting for

their individual briefs with their instructors (BME).

Among the civilian advanced instructors, one participant responded to the

question about pre-flight briefs including mention of airmanship with a firm “no”,

and the comment that “if they are still in the system their airmanship is not too bad”

(CCA). Another participant in this group indicated that their organisation provides a

list of discussion points to be covered with each trainee and that this list includes

many points that cover airmanship, including “check the big picture”, as well as

165

technical knowledge (CCG). There may be a problem with terminology, as one

civilian advanced instructor noted, “for example on a check assessment form you’re

unlikely to see the heading airmanship. These days you’re more likely to see the term

non-technical skills” (CCB) or non-technical markers (CCE) that cover the sorts of

behaviours that were included in the definition of airmanship. The pre and post flight

briefs are more likely to address non-technical behaviours, rather than use the term

airmanship in the civilian advanced environment. The implication then, is that

technical skills are not considered part of airmanship.

The responses of the advanced military instructors at Williamtown, who teach

conversion to jet fighters, were consistent with the BFTS instructors. Their briefs

follow the same ‘aim, airmanship and considerations’ framework and it was also

noted that there are mass briefs as well as individual briefs (MWB, MWC).

The advanced military instructors at East Sale, who train future instructors,

were asked if they discussed airmanship with the flight instructor candidates (FICs).

The responses varied considerably, from “No, they should already have good

airmanship” (MED) via “it gets discussed with an instructional flavour” (MEC) to

“Yes, that is pretty much all I talk about” (MEF). One participant said that he gives

two briefs for each sortie, one directed at the hypothetical student (which he later role

plays) and one for the FIC, emphasising how to maintain their own airmanship and

situational awareness while delivering the training package (MEC). Given how

standardised the military is, this may be the general rule but the interview question

did not reliably elicit this information. The remaining four participants said that they

talk about airmanship in a general way but do not spend much time on it. One

participant noted that there is a series of lectures for the FICs about how to teach and

assess, but while it touched on airmanship it is mostly about the technical aspects of

flying. The situation was summarised by the statement:

I think we rather rely on the airmanship that they’ve demonstrated to the point that has got them here as reasonably good pilots, and we hope that implicitly they realise that the behaviour that has got them here is the behaviour that they’re now going to try and encourage of their students when they get where they’re going. So I don’t think we formalise too much instruction of airmanship. (MEC)

This situation is confirmed by some of the comments made by the military

BFTS instructors, when talking about their training at East Sale. They reported that

166

most of instruction was on how to get across the technical skills, the “hands and feet”

skills, and that airmanship was considered to be about providing the right example

and enforcing that (BMB, BMD, BMF).

Post-flight debrief. The post-flight debrief was not consistently mentioned by

the ab-initio groups of instructors, but there was no specific question in the interview

schedule that addressed the post-flight de-brief. In several cases, such as among the

advanced military East Sale group and advanced civilian instructors, in saying that

airmanship was not covered in pre-flight briefs, it was added that any deficiencies

were addressed post-flight. In the military environment, there is a required post-flight

debrief in which the student’s performance is appraised and indication given to

which areas need further work (MEA, MEE, MEF). A post-sortie debrief is also

expected in normal flying activities in the military, a self-debrief if necessary, or a

debrief with other pilots who had taken part in the flight (MEB, MEC). In the

training environment, rather more of the military advanced and military and civilian

BFTS instructors spoke about the debrief than did the civilian ab-initio and civilian

advanced instructors. Among these civilian instructors who mentioned the debrief,

comments indicated that the whole flight was debriefed and that would include

airmanship behaviours (CAB). There was comment from the civilian advanced

instructors that if the crew can come up with their own explanation of what might

have gone wrong, or not exactly as planned, then they are happier with the outcome

(CCA, CCF). In contrast, the ab-initio instructors phrased their comments to indicate

that they provide the feedback for the student, rather than facilitate the student’s own

self-analysis (CAA, CAC).

Training rate. The military ab-initio course at BFTS is a set length and

students are expected to keep up with the training program, with some allowance for

remedial training if required. There were a few comments that this can be a challenge

for the instructor as well as the student (BCA, BMB). One of the civilian ab-initio

instructors mentioned the challenge of the opposite situation, with private students

coming in for lessons infrequently making it difficult to reinforce desirable

behaviours, in contrast to the culture change that is achievable in a formal intense

course (CAA).

Learning from others and providing an example. Overall 20% of the

participants talked about their own or their students’ airmanship developing by

167

learning from others (CAD, BCF), watching what others do in the same organisation

(CCD, CCG) or from reading crash comics (reports of aircraft accidents) (CCA,

CCB), with most comment coming from the civilian advanced group. A parallel

concern is with providing a good example for students, and others, to learn from,

brought up by 38.5% of participants overall. This appears to be a central assumption

in the military instructor training process, as discussed above. The civilian advanced

instructors made the most mentions of providing a good example or having been

provided with a good example early in a career (CCB, CCD).

Organisational culture. Comments about good examples were also reflected

in the mentions of the importance of the promotion of a positive organisational

culture. Participants discussed the idea of a no-blame approach and concern with

why a mistake or error happened, paired with non-acceptance of deliberate violation

of rules (CCD, MED), generally known as a just culture, although that terminology

was not used. The lack of authority gradient, allowing the crew to speak up, was

given as another example of a positive culture (CCE, CCF). There was criticism of

cultures that normalise deviance or threat, and that hide rather than admit to

problems (CAG, CCA, CCB, BMB). This was related back to the training

environment being required to present this sort of culture in the first place (CAG,

BMC, MEE), and to reinforce it consistently across the different areas of the

organisation, different squadrons in the military (MEF) and different fleets in airlines

(CCD).

Learning from others, providing a good example and organisational culture are

related in that all are comments about the way in which airmanship is inculcated into

student pilots. Combining over the three concepts, without counting any participant

twice, results in an indication that 53.8% of the participants mentioned one or more

of these three concepts, and the civilian advanced group had the highest mention rate

(85.7%).

Progression. The BFTS instructors made the point that the flying training

program is a staged process (BCF, BMA, BMC, BMF), and that there is a graduation

to allow experiences to build up and ease transition to new tasks (BCB). There was

also comment made that after a while, the student is expected to take responsibility

for the pre-flight aircraft checks and start-up, but now and again the instructor will

check that it is being done correctly (BCA, BME). One of the civilian advanced

168

instructors made a similar comment, that the training is staged so that everything is

not happening at once and to allow the student to get the feel of the aircraft (CCA).

Only one of the advanced military instructors made a comment along these lines,

noting the gradual introduction and exposure to situations with a demonstration of

the appropriate response (MEC), and he may have been referring to the BFTS

process. Another advanced military instructor noted that as the FIC course progresses

there is a change from attention on the FIC’s airmanship to the impact of instructor’s

actions on the airmanship of the (future) student (MEE). Other military advanced

instructors commented that they try to extend the FIC by loading him up with tasks

(MEB, MED) as the course develops. The civilian ab-initio instructors did not

comment on the way in which the early lessons progress, but the advice from the

regulator sets out a progression of lessons (CASA, 2006).

Development. Some of the civilian advanced instructors talked about how they

may see pilots who work in the organisation infrequently, and notice how they have

developed as their career has progressed, in responsibility, decision making, or

simply being better at the job (CCD, CCE). The BFTS instructors see the same

development on a shorter time scale as the trainees move through basic flight

training, and commented on the instructors” expectations for the trainee to exhibit

increasing levels of airmanship (BCC, BMA, BME). The advanced military

instructors made much the same comments about basic flight training (MEA) and

across a full career (MEC).

Curriculum structures. The instructors in the military system made some

comments about the curriculum structures in place to ensure consistency. The

curriculum itself is determined by the Central Flying School at East Sale and this

provides a structure, syllabus and guidance material (BCB, BCH). There is

recognition that different instructors do things a little differently with regard to

airmanship (BCC, BCG) and that can cause difficulties for students who move

between instructors (BME). However, there are meetings to standardise the approach

to assessment and to exchange ideas (BCG, BCH, BCI). One participant noted that

with increases in workload, it is not as easy as it once was to get together and

compare notes (BCF).

Student records. The majority of comments about student records came from

the BFTS instructors, because the BFTS has a formal system of reporting on each

169

sortie. The one other comment came from a civilian ab-initio instructor who

expressed the desire to see much more formal record of student progress, “not only

talking about the, the tolerances and standards of the flight but also the elements of

airmanship that we observed” (CAA).

At BFTS, the student record is written into a structured document including

tick boxes for the aspects of flight covered in that lesson and a space for written

narrative (BCC). The written commentary is based on the post-flight debrief of the

student and tends to be a summary of the flaws observed in flight and the required

corrective action (BCA, BCD). It was also noted that the emphasis tends to be on

technical skills, although airmanship errors are also reported (BCC). Several of the

military BFTS instructors indicated their belief that the report is not primarily for the

student, but rather to inform another instructor, should one have to take over the

student (BMB, BME, BMF). One of the civilian BFTS instructors went as far as to

indicate that the student record is not there to give positive feedback to the student, it

is to provide documentation to enable removal of poorly performing students from

the training program (BCF). One of the military BFTS instructors reported that the

training undertaken to become a QFI did not include formal instruction on how to

write the reports, but rather feedback on practice write-ups (BMD). A typical

comment is

[The student record] is not there to give the student good feelings about himself, it’s there to pick up on every mistake that he makes so that the next instructor can try and correct those. The unfortunate reality is now we write them with the view that the student will read them and as such, you know, they become a little less, well a little bit less harsh I guess (BME).

Ground school. Ground school aspects of flying training programs were only

discussed by some of the BFTS participants and one military advanced participant.

Essentially the comments indicated that the students at BFTS have a fairly intensive

ground theory component, in which they are lectured in airmanship, medical aspects

of flying, limitations of the aircraft, rules and regulations and begin to learn their

basic techniques, work cycles and checks (BCB, BCH, BMB, BMC). They also

attend mass briefings which introduce each sortie and form part of their preparation

for flying (BCA, BCC, BMA, BME). Similarly, in more advanced training such as at

Williamtown, the students have tutorials, an air training guide and mass briefs, as

well as the individual briefs, before sorties are flown (MWB).

170

Initial Training of Airmanship

The participants who work with ab-initio student pilots were asked what they

started with in terms of including airmanship in initial flight training. Table 4.11 lists

the kinds of things that were mentioned across the three ab-initio instructor groups,

loosely ordered to reflect Kern’s model of airmanship.

After noting that airmanship is introduced from the first pre-flight briefs, the

different ab-initio instructor groups came up with a variety of different aspects of

airmanship that are introduced in the first few lessons, but most noticeably they all

talked about lookout being the most important initial airmanship related skill to be

taught. Two civilian ab-initio instructors specifically mentioned teaching the clock

code for identifying where other aircraft are located in the sky (CAB, CAE). Now

that the use of digital watches and clocks is very widespread, this may not be so

intuitive a method as it once was.

Weather awareness was noted by some of the instructors as something that they

drew attention to or required students to have found out about before a lesson (CAB,

CAG, BCC, BCG, BCH). Interestingly, only one instructor (CAB) indicated that the

flying school he was associated with had set up a procedure for getting weather and

doing pre-flight planning and fuel calculations so that the habit of doing so is

established early on. Five BFTS instructors mentioned that they expected students to

arrive prepared for the sortie, which may include such pre-planning, as well as being

aware of the activities they would be conducting during the session (BCC, BCE,

BCF, BCH, BMF).

In addition to unanimous mention of lookout, listen-out to the radio and

making appropriate radio calls was included by more than half the instructors (54.5%

overall) in initial lessons. Listening is associated with lookout in that the combination

allows the development of good situational awareness. Also in relation to developing

situational awareness, six of the BFTS instructors spoke about getting their students

to think ahead of the aircraft and about what would be happening next, from the early

stages of training (BCC, BCE, BCF, BMA, BMC, BMF) and two noted a concern

with ensuring fuel awareness on the part of the student (BMA, BMF).

171

Table 4.11

Percentage of Participants in Each Group and Overall who Mentioned Inclusion of Listed Airmanship Items in Initial Flight Training

Civilian Ab-initio

Civilian BFTS

Military BFTS Total

N 7 9 6 22 Item % % % % Pre-sortie Weather 28.6 33.3 0.0 22.7 Flight planning 14.3 0.0 0.0 4.5 Preparation 0.0 44.4 16.7 22.7 Situational Awareness Lookout 100.0 100.0 100.0 100.0 Listen-out and radio use 42.9 55.6 66.7 54.5 Thinking ahead 0.0 33.3 50.0 27.3 Fuel awareness 0.0 0.0 33.3 9.1 Judgement/Decisions Prioritising 0.0 11.1 33.3 13.6 Decisions/ Captaincy 0.0 0.0 33.3 9.1 Knowledge How to apply knowledge 0.0 11.1 50.0 18.2 Safety 14.3 0.0 0.0 4.5 Knowledge of rules 14.3 0.0 33.3 13.6 Aircraft limits/parameters 28.6 22.2 33.3 27.3 10 minute checks 0.0 33.3 83.3 36.4 Checks/checklists 42.9 55.6 83.3 59.1 Go round criteria 0.0 11.1 50.0 18.2 Skills Start-up 14.3 22.2 0.0 13.6 Taxi 42.9 33.3 33.3 36.4 Hand-over take-over 14.3 0.0 0.0 4.5 Smooth control 14.3 0.0 0.0 4.5 Procedures 14.3 0.0 0.0 4.5 Work cycles 0.0 33.3 50.0 27.3 Correct altitude 0.0 0.0 33.3 9.1 Emergency procedures 0.0 0.0 33.3 9.1 Discipline Pre-flight Inspection 14.3 0.0 16.7 9.1 Paperwork 0.0 0.0 16.7 4.5

A couple of the military BFTS instructors mentioned the expectation that

students would begin to exhibit decision making and exercising captaincy, a bit later

in the sequence of early flights (BMA, BMF). There was also an expectancy that

172

students would begin to start prioritising tasks, especially in busy phases of flight

(BCE, BMC, BMF).

Half of the military BFTS instructors, and one of the civilian BFTS instructors,

made a point of their role being to teach the students how to apply their knowledge in

the air. The ground school component of the military training program presents what

the students need to know, but recognising how to use that information requires some

coaching (BCC, BMC, BME, BMF)

One of the civilian ab-initio instructors noted that one of the first things that he

does with new students is ensure that they understand safety aspects of being near

aircraft on the ground (CAF). Several instructors expected students to know about

the rules and regulations, and standing instructions in the military case, appropriate

to the sortie (CAC, BMC, BMD). There was also mention of knowledge of the

aircraft limitations and parameters, for example the appropriate speeds, temperatures

and pressures against which to compare operational readings (CAC, CAE, BCF,

BCH, BMB, BMD).

Knowing and carrying out 10 minute checks were included by many of the

BFTS instructors. These checks are to ensure that the aircraft systems are operating

correctly and are inculcated early on. Knowing their checks and appropriate use of

checklists was also frequently mentioned across all three groups, overall by 59.1% of

the instructors. Four of the BFTS instructors spoke about the students having to

know and apply the go-round criteria specified for student flights (BCF, BMA,

BMD, BME).

In terms of flying skills that were included in the listing of airmanship points,

three instructors mentioned aspects of the aircraft start-up (CAE, BCF, BCG) and

eight included taxiing, generally in terms of appropriate speed and separation from

other aircraft. In the air, the ritual of the formal “handing-over – taking-over” routine

and knowing who is in control of the aircraft was mentioned by just one of the

civilian instructors (CAC). Another civilian instructor made the point of wanting to

see the student develop a relaxed and smooth handling of the aircraft (CAA). One

civilian instructor talked about the student learning their procedures (CAC), while six

of the BFTS instructors spoke about the student following their work cycles (BCA,

BCC, BCI, BMC, BME, BMF), these referring to correctly carrying out specific in-

173

air aircraft handling sequences. Flying at the correct altitude and being ready for

emergency procedures as they were introduced were both mentioned by two military

BFTS instructors (BMD, BMF).

There was mention of the requirement for carrying out a pre-flight inspection

(CAB). One military instructor said that after the first few sorties, the student was

expected to carry this out unsupervised (BME). Finally there was mention that the

flight was not over until any necessary paperwork was completed, such as noting any

problems with the aircraft in the maintenance log (BMF).

Clearly this is a snapshot of what the instructors came up with in an interview

setting, and in answer to one question, not an exhaustive list of everything each

organisation requires to be included in early flight training. There are commonalities

across the groups, and differences seem to be mainly in terminology: flight planning

vs preparation and procedures vs work cycles. The items that are most frequently

mentioned can be seen as those that are considered most important: lookout, listen-

out, and checks.

Assessment of Airmanship

The participants were asked what they look for in assessing airmanship, which

led to discussion of whether airmanship was assessed at all, at least with the civilian

instructors. The BFTS instructors were clearer about what they were assessing, and

how to do so, although there was not complete consistency. The advanced military

instructors were divided, in that the instructors at Williamtown responded in a similar

fashion to the BFTS instructors, while the East Sale instructors of instructors were

more inclined to say that there is a more subjective approach.

In the civilian ab-initio context, airmanship is assessed in an informal sense, in

that the instructor is expecting the student to have picked up and used the airmanship

points raised during the pre-flight briefing (CAC, CAD, CAG). There is not really a

formal assessment as such: generally the instructors de-brief the students on where

they see problems or a need for improvement, which would include airmanship

points (CAA, CAB, CAC, CAD, CAF). At one of the schools, there is a report form

on which the instructor has a tick box “airmanship has been maintained”, which the

participant said was not detailed enough, and a further break-down of what to look

for was required (CAE).The flight test pro-forma (which the regulator, CASA,

174

supplies) requires an assessment of the elements of airmanship, including

maintenance of situational awareness, and the comment was made that therefore the

instructors don’t put people up for the flight test unless they show these elements of

good airmanship (CAB, CAE).

Among the civilian advanced instructors there was general agreement that the

term “airmanship” is not used in assessment. Rather, assessment is divided into

technical and non-technical skills, where non-technical skills include situational

awareness, decision making, communication, leadership and automation

management, and technical skills include aircraft handling skills, such as flying an

ILS approach or managing an engine failure on take-off. As one instructor said,

“there isn’t an airmanship box … if you looked at the big umbrella it’s in there

encompassing the whole thing. What we try and do is sort of break it down a little bit

and look at specific areas” (CCE). However, while they are assessed, pilots are not

failed (on a simulator check) unless a problem in the non-technical (or airmanship)

factors is associated with an error or problem in technical skills (CCG). In contrast,

when assessing pilots for Captaincy upgrades, comment was made that non-technical

skills “are about all they look at” (CCF).

When asked about assessment, several of the civilian BFTS instructors

mentioned the “word pictures” that they use, or in some cases, no longer feel they

need to use. These are a set of rubrics for assigning marks for preparation, technique

and airmanship, on the basis that the behaviours exhibited by the student can be

matched to the levels described in the rubric. Thus, assessment consists of

determining whether the student is meeting the required standard for that stage of

training, or is making errors or simply not doing very well. The basic inclusions in

the rubric are the same as were expressed in the definition of airmanship: lookout,

listen-out, awareness of other aircraft, where they are, where the weather is, doing

checks and following work cycles, planning ahead and being aware of consequences,

getting appropriate clearances, applying go-round criteria and so on (Carrick,

Graham, Healey & Pickard, 2008), with more expected of students as they progress

through the program (BCA, BCI).

Comment was made that when the instructor was not sure if the student was

actually on top of things, they would “set them up”. For example demonstrate a

manoeuvre and hand over with the aircraft pointing outside the training area and see

175

if the student notices; another being to spring a practice engine failure and observe

whether the student was ready for it and knew which way to turn for an appropriate

forced landing area (BCC, BCG). There was also comment that there is more

emphasis on the technical handling of the aircraft (BCC) and on what the student has

done wrong (BCE, BCI) in the write up of the sortie.

Military ab-initio BFTS instructors made similar comments to those made by

the civilian BFTS instructors. The word pictures were mentioned (BMC) with the

comment that these are a guide and individuals develop their own interpretation of

them (BMD, BME). There was also comment that expectation of performance

changes as the student progresses (BMA, BMB, BME). More than half of the

instructors in this group noted that the three components that they mark (preparation,

technique and airmanship) are interlinked and fairly blurred as to where any problem

originated or where to allocate marks (BMA, BMB, BMC, BMF). Two of this group

of instructors indicated that they start off assuming everything will go well, then

mark down according to what went wrong or was incorrectly handled (BMA, BMD).

One mentioned setting the student up to make an error, so that they remember it

(BMB). Only two BFTS instructors (one civilian and one military) commented that

what they are looking for in airmanship is fairly well defined (BMB, BCD).

In the context of assessment, the Williamtown based military advanced

instructor participants made comments more similar to the BFTS instructors, than to

the East Sale CFS instructors. Their work in tactical instruction for pilots converting

to jet fighters is more like to the basic instruction at BFTS than to the training of

instructors that occurs at East Sale. Two of the three instructors at the Williamtown

squadron spoke about the use of word-pictures to guide the assessment of all the

different categories to be marked for a mission, one of which is airmanship (MWA,

MWB). One noted that assessment of airmanship tends to be on what went wrong or

non-optimally (MWA) while another noted commenting on and reinforcing good

airmanship as well (MWC).

The CFS participants train flight instructor candidates (FICs), who, if they

qualify, then progress as qualified flight instructors (QFIs) to the various training

squadrons or to basic flight training schools at Tamworth (New England tablelands,

NSW) or Pearce, near Perth in WA. These instructors, when asked about assessing

airmanship, did not mention word pictures at all. In the answer to the question about

176

assessment of airmanship, three of the seven East Sale instructors said that the FICs

were expected to have high to above average airmanship when they were sent for

QFI training, most would have flown in combat and have a lot of experience (MEC,

MED, MEE). However, two of these also said that during the initial phase of

training, conversion to the training aircraft, airmanship is assessed because of the

change-over to flying a different aircraft type and because “people still make

mistakes” (MED, MEE). There was also a comment about the extent to which

airmanship is defined by context, what is appropriate airmanship in one context (like

an operational squadron) may not be appropriate in another (such as a training

school) (MEF). So at this level of instruction, it appears that initially airmanship is

assessed and attention drawn to any areas that may need improvement (MEB), later

in the training it is more a case of drawing attention to issues of airmanship that

should be raised with their future trainee pilots (MEE, MEG). One participant was

adamant that airmanship is subjective and that it should not be reduced to a set

checklist, saying “airmanship isn’t about what you do, and as soon as it’s not about

what you do you can’t objectively assess against a checklist, and if people do

airmanship from a checklist well that isn’t airmanship” (MEA) and added that

assessment of airmanship is rather about evaluation of thought processes and the

resultant decisions that are made.

In summary, assessment of airmanship is apparently quite informal among the

civilian ab-initio instructors, only one mentioned formal recording of airmanship

behaviours. The civilian advanced instructors recognise that what they are assessing

is airmanship, but the term is not used, rather non-technical skills are assessed along

with technical handling, the combination of which forms the overarching idea of

integrated airmanship. In contrast, the military and civilian instructors at BFTS, and

the advanced trainers at Williamtown, while talking about airmanship as an

integration of technical and non-technical skills, are quite clear that they assess

preparation, technical skills and airmanship separately, where airmanship is

described as consisting of what the civilian advanced trainers would call non-

technical skills. Finally, the military advanced trainers at CFS were reluctant to say

that they assess the airmanship of the FICs, noting their experience and existing high

levels of airmanship. It was clear that they separated the technical handling skills

from airmanship in that the FICs have to relearn the specific technical handling

177

requirements that CFS include in the basic flying syllabus, but that airmanship

(excluding flying skills) is assumed to be fine and discussion relates more to how the

FIC will imbue their future students with this aspect of airmanship.

Training Process

In the course of the interviews the instructors made a number of comments

about the way in which they go about the training process. Some voiced a philosophy

of training or comment on their perception of their role as an instructor and what they

see as necessary instructor qualities. Other comments included dealing with the

students themselves and the advice they give to students, how they use errors made

by students to elaborate on the training program, and the use of queries and scenarios

in their instruction. There were also comments on the cognitive capacity of the

students. There was some explanation of instructor training, mostly by the BFTS and

advanced military instructors. There were also a number of comments about training

generally that were not further categorised. Table 4.12 shows the distribution of these

comments across the five instructor groups.

Table 4.12

Percentage of Participants who Commented on Listed Aspects of the Training Process

CAB CAD CBF MBF MAD Total Number of Participants 7 7 9 6 10 39

% % % % % %

Philosophy 42.9 42.9 44.4 16.7 50.0 41.0 Instructor qualities 42.9 42.9 33.3 33.3 20.0 33.3 Dealing with Students 28.6 14.3 55.6 50.0 40.0 38.5 Student advice 14.3 57.1 55.6 33.3 20.0 35.9 Dealing with errors 42.9 28.6 22.2 50.0 40.0 35.9 Queries and scenarios 42.9 71.4 11.1 16.7 10.0 28.2 Cognitive Capacity 0.0 42.9 88.9 66.7 50.0 51.3 Training general 28.6 0.0 33.3 50.0 50.0 33.3 Instructor training 28.6 0.0 55.6 100.0 60.0 48.7 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced

Philosophy. One of the civilian ab-initio instructors made clear that he tried to

develop the idea in the student that he (or she) was expected to learn and memorise,

certain things, which then, he felt, led to a feeling of competence on the part of the

student (CAA). Another noted that if the student appeared to be handling things well,

178

he would just push them a little to see if they can handle a bit more (CAC). A third

participant in this group talked about the importance that the training organisation

teaches within the syllabus and that instructors are responsible for making a real

effort to develop students’ “technical and airmanship skills” (CAG).

The civilian advanced participants presented a consistent position that trainers

need to recognise what is wrong with a student’s performance and know how to fix

it, and ensure that the teaching is about practical application (CCA, CCC, CCD). One

participant in this group noted that it is harder to teach behaviours, so many trainers

just stick to teaching skills, but more often it is thinking through problems and

consequent behaviours that bring the student at this level unstuck (CCF).

The civilian BFTS instructors made some similar comment, that the instructor

should pick up what the student is doing wrong and advise how to fix it (BCC, BCG)

and show the student how to apply what they have learnt in theory (BCB, BCC). One

instructor in this group indicated his opinion that the job is to get the student through

the course at as high a level as possible (BCA), and one of the military BFTS

instructors made much the same comment (BME). In discussing a well-known

accident case study, a further civilian BFTS instructor indicated that taking the

training seriously at all levels can pay off when it counts (BCF).

One of the advanced military instructors commented that the point is to mould

the students into a consistent level of acceptable behaviour reflecting good

airmanship (MEF). Further, they do not train co-pilots (all candidates are expected to

eventually captain aircraft), so the captaincy part of airmanship is important (MEG).

The idea of fixing problems was mentioned here, with the comment that the tricky bit

is to “make it sound like the application of common sense, and make it sound like it

is something the student could realistically seek to achieve themselves” (MEC) and

thus the FIC has to learn to do this. There was also the comment that the FIC’s have

already got the skills and the airmanship, the job of the instructor (at this level) is to

get them to work out how to pass these on to their (future) students (MED). Finally,

there was the comment that airmanship is an art form not a science, and that example

and discussion lead to improvement (MEA).

179

The common theme across the instructor groups is that problems need to be

identified and fixed in a practical fashion, and the student coached in both technical

skills and airmanship behaviours in order for them to pass the course.

Instructor qualities. Providing a good example of airmanship for the students

has been discussed already, however, several instructors indicated that the level of

airmanship displayed needs to be “over the top” (CAA, BCC, MEA). Another

common comment about the qualities that must be displayed was that instructors

have to be alert, as things can change fast and get out of hand quickly, so they have

to know when to step in (CAC, CCG, BCF, BMC). There was also comment that

instructors need to be sure that their own skills are adequate (BMA) and that they

change the way they explain if the student does not understand (BCC). At the higher

levels, instructors should be willing to learn from the ways that other people operate

(CCD). There were a couple of comments on the perceived lack of experience and

knowledge among new young instructors in the civilian ab-initio training

environment (CAG, CCA).

Dealing with students. Some of the instructors, 38.5% of participants overall,

talked about how they interact with their students, not in response to specific

questions but as part of the general discussion. Among the civilian ab-initio

instructors, one spoke of ensuring that the student was at ease and comfortable in the

aircraft and instruction environment (CAA), while another spoke of the difficulties

he had found in instructing overseas students (CAC). Only one of the civilian

advanced instructors talked about interactions with students, saying that the student

needed to be comfortable, and also physically well, and noted the need to avoid

demoralising the student if errors were made (CCA).

The instructors at BFTS made rather more comments about their interaction

with students than the civilian instructors and military advanced instructors. Three of

them commented that they try to keep up their students’ confidence and not shatter it

(BCC, BMA, BMB), although one noted that fear tends to be a good motivator for

avoiding errors. Two of the instructors stated that they sometimes have to deal with

the students’ personal problems affecting their ability or concentration (BCD, BMA).

There is an expectation that students will speak up if they do not understand (BCA)

and will act in an adult manner, for example attend the medic if unwell (BCB). The

aim is to get the student thinking about what they are doing in the air (BCE) and to

180

recognise that correction is for their improvement (BME) although it was noted that

sometimes a fail is required to get the student to concentrate on improving their

progress (BCD).

The military advanced instructors commented more on what they tried to teach

the FIC’s with regard to handling their future students. One noted the need to set the

tone of the sortie so that the student will speak up if they think something is wrong or

if they are not comfortable (MEB). They noted that they give hints as to where

students may have problems in certain sorties (MEB) and strategies for dealing with

things the students may do (MEE) including surprise errors from otherwise good

students (MED). There was also a comment about trying to relate the things that the

FIC has done in operational squadrons to what they are trying to teach the student

and trickle in ideas about airmanship along the way (MEG).

Advice to students. What the instructors give as advice to their students was

mentioned by 35.9% of the participants overall, and most comment came from the

civilian advanced group and the civilian BFTS instructors. One of the civilian ab-

initio instructors indicated that he advised students to abide by the “I’m safe”

mnemonic (CAD). This is a strategy for self-assessment of the question “Am I fit to

fly?” It covers illness, medication, stress, alcohol, fatigue and eating habits, the idea

being to ensure that the pilot is free of adverse indicators. Among the civilian

advanced instructors, advice included that everyone makes mistakes (CCA), that

“you’ve got to really keep an eye on the small details and the big picture at the same

time” (CCD) and the need to be good at time-sharing (CCB). It was also pointed out

that standard operating procedures (SOPs) should always be followed, because if

something goes wrong and they were not adhered to, “you’re done for” (CCG).

Two of the civilian BFTS instructors mentioned an emphasis on preparation

(BCC, BCE). Other advice included an expectation that the student will speak up if

uncomfortable (BCB), keep fit and healthy (BCC), always expect an engine failure

on take-off (BCF), keep ahead of the aircraft (BCE), have a thorough knowledge of

emergency procedures (BCG) and know that the job is not done until the paperwork

is completed (BCG). Further advice was to follow the rules because they are written

in blood (BCF). Advice from the military BFTS instructors was to make a decision,

and remake it later if necessary (BMB) and to pay more attention to the problems

that are raised about a sortie in the debrief, than to the marks given (BMF).

181

Some advice from the military advanced instructors echoed other instructors in

advising that respect for and adherence to the rules be instilled and to always expect

the unexpected from students (MEC). Further advice to the FICs was to always

provide a good example of airmanship, because the students are always watching

(MEG) and to keep even tempered with the students (MED).

Dealing with errors. The participants made comment about how they deal

with the errors that students make. A consistent discussion point across all groups

was that they allow an error to develop so that the student has a chance to recognise

and correct it, so that the student really learns from it, but to not allow it to go outside

their own comfort zone, nearing which point they would intervene (CAA, CAB,

CAC, CCG, BCC, BCF, BMC, MED, MWA, MWC). Related comment included

setting up the aircraft improperly and seeing if the student notices or put them in a

situation where they are sure to miss something (BCC, BMB). The training process

for FICs includes discussion of how to intervene when an error is about to be made

(MEE). There was also comment from a civilian advanced participant that their

organisation is hard on errors, and that more attention should be paid to error

recovery (CCF).

Queries and scenarios. It appeared to be common to use queries and scenarios

to encourage situational awareness and effective decision making (CAG, CCA, CCC,

CCG) as well as to gauge recognition of problems (CAB, CAD, CCB) and aircraft

state (BCB). Questions are also considered useful to assess knowledge (MEE) and to

determine why certain decisions were made (BMC).

Cognitive capacity. There were a number of comments from participants

(mostly from the BFTS and military advanced participants) regarding the cognitive

capacity of their students - essentially whether they were able to keep up with all the

different tasks they are required to do in flying the aircraft and then have the ability

to cope with any emergencies. Several instructors noted that students” capacity was

limited at first, but that as they progressed they were able to take more in, handle the

tasks and broaden their situational awareness (CAB, BCD, BCH, BMB). It was

reported that it is noticeable that when students become overloaded they lose

situational awareness, stop listening, looking, monitoring and thinking ahead (BCA,

BCB, BCE, BCG, BMA, MEF).

182

It is possible to teach how to build up capacity, as one of the civilian advanced

instructors said, “you can teach people timesharing and work allocation, workload

priority, workload management in order to generate spare capacity” (CCB). A

civilian BFTS instructor agreed, “If the student has studied their content and

practiced their drills then they will have the brain space to be able to do the job and

still have that ability to reach out and be situationally aware” (BCF) and similar

comments were made by other instructors (BMA, BMC). The implication among the

comments was that the students at BFTS should know checks and procedures from

ground school and mass briefs, so that the instructor just demonstrates how to apply

them and to develop effective techniques which then helps free up brain space to

expand their situational awareness. Instructors from BFTS noted that they are

constantly assessing capacity of the students, their role being to recognise when the

student is overloaded and take over or reduce task loading for the student, and then

advise on remedial action (BCH, BCI, BMA).

The military advanced group of instructors made more general comments about

capacity, noting that better airmanship means more “ brain capacity” to think and

make decisions (MEE, MWA) and that poor situational awareness occurs when one’s

mental capacity gets overloaded (MEF) something to be alert for when flying alone

(MWC). There was also comment that while experience frees up “brain space” for

faster decisions, overload occurs at different rates for different people and in

different situations (MWB). As one of the civilian advanced instructors noted: “you

can always load people up to the point where they lose situational awareness”

(CCC). However, it was also noted that this is inappropriate in a training

environment. A large modern airliner cannot be flown single pilot, an individual does

not “have the mental capacity or the physical capacity to do it (alone), so you rely

heavily on the people around you” (CCF) sharing information with other crew

members to ensure good situational awareness and airmanship.

Training comments. There were some general comments made about training

that were not consistent with the other categories. There were several comments

about having to step in and take over, when operating in controlled airspace or if the

student is about to head into cloud (CAB), if the student is not using their feet

properly and the instructor has to take over to maintain a straight path on take-off

(BCC) or complete a radio call for the student if something unusual is said by ATC

183

(BMB). One of the civilian ab initio instructors stated that he punishes students for

poor airmanship, such as taxiing too fast or starting over stones, generally by making

them wash an aircraft, or run beside a taxiing aircraft to experience the correct taxi

speed (CAE).

There was comment about the sequence of events in an instructional sortie at

BFTS: the student should have the thought processes in place, then the technique is

demonstrated, they can think about it before the next sortie, when it is demonstrated

again and they have a go themselves (BCB). If an airmanship item is not performed

as it should be, the student is corrected and observed to see how he progresses during

that sortie and so marked. If it continues to be a problem then he will be marked

down on airmanship (BME).One of the BFTS trainers summarised good airmanship

as the student utilising the techniques that he had been taught (BMA). At BFTS, the

instructors generally have students from the same course, so at roughly the same

stage, but may have a mix and so possibly instruct students at very different stages of

the training program. The instructors can revise sortie content via the computer

system before briefings (BCI).

At East Sale there are two programs, A Flight takes on the new FICs and B

Flight runs abbreviated courses for former QFI’s who have been doing something

else and are now returning to an instructional position. B Flight is also the Roulettes

display team (MEC). During sorties, the instructor will talk through their thought

processes round how they are going to go about something, so that the FIC learns the

thinking behind decisions (MEA) or they will talk about why an airmanship breach

occurred, how that may have developed (for example from lack of checks or

inadequate lookout) and how that will impact on the airmanship score (MEB). The

comment was made that “we are very specific and standardised with how we want

the aircraft to be flown” and that sorties are carefully designed to achieve a specific

outcome (MED). A Williamtown instructor noted that because they fly in a tandem

aircraft, while there is a certain amount of observation of the student, it is difficult to

see what is going on, so the instructor will get the student to talk through what they

are doing (MWB).

Instructor training. During discussion about airmanship and training in

general, there were some comments made about training of instructors, however not

by the civilian advanced participants. One of the civilian ab-initio instructors

184

commented that when training instructors (which he also does), he makes the point

that what characterises bad drivers is much the same as poor airmanship, so good

airmanship is similarly being a considerate and responsible pilot (CAA). As was

pointed out by another civilian ab-initio instructor:

there’s no actual component of airmanship in an instructor rating … However part of the testing criteria at the end is that the pilot has shown sound airmanship during the course of the flight so that includes the decision-making, the situational awareness, the radio calls and all that stuff. But there’s no actual learning component or theory component behind it in the instructor rating anyway (CAD)

In contrast, another civilian ab-initio trainer who also trains instructors said that

there is a component of airmanship in the instructor rating, but that it is more focused

on such things as the principle methods of instruction and effective communication

(CAE). He also noted that instructor candidates should already know about

airmanship issues, but that frequently they did not.

Three of the civilian BFTS instructors commented that although they were

previously instructors in civil aviation, when they started at BFTS they went through

an on-the-job process of training to ensure that they flew as required by the Central

Flying School (CFS) (BCF, BCG), but that there was not much specifically on

airmanship, just what the briefings covered (BCH). One of the instructors noted the

use of the word pictures but felt that training in its use could be improved and the

rubric itself expanded (BCC). It was also pointed out that there is internal

continuation training for the instructors, although the more experienced did not

require much, and an annual CFS check (BCI). As far as airmanship goes, the

instructors talk about standardisation and testing and what is being looked for in a

student and there is an Instructor Guide that covers all the sorties including the

airmanship points that should be made (BCI).

All of the military BFTS instructors had been through the CFS course at East

Sale, and all but one commented that there was not a great deal included on

airmanship in that course. The course was said to be more about teaching how to

teach and assess students (BMB, BMD, BME). The process at CFS was remembered

as being like on-the-job training, with the instructor playing the role of the student

and later critiquing the FIC’s demonstration, debrief and write-up (BMD, BMF).

Again there was comment that the incoming FICs already had a good standard of

185

airmanship (BMB, BMD, BMF) although one participant noted that they would be

“pinged” if they did something in breach of good airmanship (BMC). Once at BFTS,

staff continuation training ensures that instructors are up to standard for

demonstration of techniques and aircraft handling (BMA).

The military advanced trainers from East Sale went into a great deal of detail

about the training of FICs. Their statements aligned with those of the military BFTS

participants, in saying that airmanship is not presented as a separate module, but

rather is an integral part of what they need to get across in each sortie and debrief

after it (MEA, MEG), and that airmanship is discussed in tutorials (MED).

Airmanship is briefed in relation to the sortie that is being practised and the FICs

attention is drawn to where their future students might have problems and also to

situations where the instructor might have difficulty maintaining good airmanship

while simultaneously observing what the student is doing (MEB, MEG). One of the

participants explained that the primary objective in the instructor course is to train

FICs to get across a physical skill, the secondary objective is to evaluate how well

the student understood and was able to apply that skill, and therefore how their

airmanship is developing. There is only a finite time to achieve this and so there is a

reliance on the FICs existing airmanship and not much formalisation of instruction in

airmanship (MEC). Some of the time in the air is spent for the FIC to refine their

skills and become expert in handling the training aircraft, so that they can

demonstrate exactly what the student is supposed to learn (MED). In addition, the

FICs are taught the most appropriate way to deal with students, and to use various

techniques to develop airmanship in their students (MEE). The course is “not really

focussing the light on their [the FIC’s] airmanship but focussing their attention on

their requirement to enhance or to educate or to bring on the airmanship of their

student” (MEC).

Discussion: Training of Airmanship

Main Findings

The major difference between the ab-initio instructors and the advanced

instructors is that just over half of the advanced trainers (civilian and military) stated

that airmanship is assumed at their level of training. The advanced military trainers at

CFS were fairly consistent in maintaining that they did not teach airmanship at their

186

level, just showed how to teach it to future students. The civilian advanced trainers

went on to talk about including airmanship (as non-technical skills) in the training

process. The ab-initio instructors all talked about the inclusion of airmanship in the

training that they do.

Learning from others, providing a good example and comments on

organisational culture indicate a concern with embedding airmanship –mention of

each concept was made by 20% to 40% of the participants in any one group, with the

majority of comments coming from the civilian advanced group of instructors. Six of

the seven civilian advanced instructors (85.7%) mentioned one or more of these

concepts and combined, 53.8% of the participants overall made a comment about one

or more of these concepts.

The BFTS instructors more often mentioned that the flight training program is

staged in that there is a gradual build-up of content during the program. Some

comment was made by the military advanced instructors, but this was not mentioned

by the civilian ab-initio instructors and by only one of the civilian advanced

instructors.

All of the ab-initio trainers said that airmanship was included in their pre-flight

brief. The military advanced trainers at Williamtown stated that it is included and the

comments from the military advanced trainers at CFS imply that it is part of the pre-

flight brief there as well. Whether the civilian advanced group include airmanship in

the brief is uncertain, but possibly under the label “non-technical skills’. Those

participants who mentioned the debrief either included airmanship or stated that the

debrief covered the whole flight. There were more mentions of debriefing from the

civilian and military BFTS instructors and the military advanced instructors than

from the civilian ab-initio or advanced groups.

The BFTS instructors talked about curriculum structures in place to ensure

consistency, student records and the relationship between the ground school and

flight training. While these aspects of a training program may exist in the other

training levels and types, they were not mentioned during the interviews, apart from

one civilian ab-initio instructor desiring more formal records and one military

advanced instructor mentioning mass briefs.

187

The responses to questions about the airmanship inclusions in the initial

lessons (or sorties) were consistent in unanimous mention of lookout and frequent

(more than half the participants) mention of listening and radio use. Other consistent

mentions across groups were checks and checklists, aircraft limits or parameters and

taxi speed. There could be some terminology differences as the civilian ab-initio

instructors mentioned flight planning while the BFTS instructors talked of

preparation; and the civilians mentioned procedures while BFTS instructors

commented on work cycles. The civilian ab-initio instructors included safety on the

ground, handover/takeover, and smooth control. The BFTS instructors added much

more detail about inclusions and there were some variations between the civilian and

military instructors about what was included.

Assessment of airmanship was most clearly articulated by the BFTS instructors

(both civilian and military) as being distinct from the assessment of preparation and

technical handling skill. However, several of the instructors noted that there was

some blurring across the lines, saying it is not easy to separate the three. Their

comments on assessment also contrasted with the earlier statements by these same

instructors that airmanship is global and all inclusive, although most of them then

specifically excluded technical skills from the definition of airmanship. Assessment

of airmanship was not so clear among the civilian ab-initio instructors, but there is

awareness of the flight test requirements for situational awareness and threat and

error management competencies to be evident. There does not appear to be a formal

process of assessment during the training program, at least in the organisations

sampled. The civilian advanced instructors noted that the terminology more

commonly used is non-technical skill (and thus excluding technical skills) which is

assessed but is not a fail item unless paired with a problem in technical skills. The

military advanced instructors at Williamtown had similar views to the BFTS

instructors while the CFS instructors appeared reluctant to say that airmanship was

assessed, only two or three made reference to any assessment of the FICs.

The discussion of assessment raised the question of whether technical skill is

part of airmanship – in general discussion about the definition of airmanship

technical flying skill appears to be included. However, in specific questions about

assessment airmanship appears to reduce to non-technical or behavioural aspects of

flight tasks – everything except technical skills. It seems, especially among the

188

military instructors, that there are two levels of thinking about airmanship: the

general, the umbrella concept that airmanship is everything that goes to make a safe

and efficient flight; and the specific, airmanship as assessed at BFTS/CFS that does

not include the technical skills or preparation. Among the civilian instructors that

two-level way of thinking does appear as well, although not as starkly. There were

comments that airmanship is the non-technical, threat and error management sorts of

activities and not the aircraft handling actions, among the ab-initio instructors, when

considering the inclusion of airmanship items in licence flight testing among the

civilian advanced instructors when considering simulator checks.

There was discussion of the training process including comment on training

philosophy, with a view that the instructor’s role is to correct students and suggest

fixes for any problems they have in the air, at least among the civilian advanced and

civilian BFTS instructors. Getting the students through the course and moulding the

students into a consistent level of good airmanship were other approaches among

military instructors.

Instructor qualities mentioned by the participants included the over-the-top

display of airmanship and being alert to take over if things got too far out of hand.

Participants also spoke about dealing with students – generally ensuring students are

at ease and maintaining their confidence. There were many and various spontaneous

comments about the advice given to students. There was discussion of dealing with

student errors and this mainly raised the “comfort zone” issue, across all groups,

concerning when to intervene if the student is progressing down an erroneous course

of action. Questions and probes were regularly mentioned as ways to keep track of

students’ level of situational awareness and knowledge. Cognitive capacity was

discussed in depth by mostly BFTS instructors (possibly because it requires a tick-off

on the student record form) but also by military advanced and some civilian

advanced instructors. General training comments varied with little consistent

comment.

Questions about instructor training led to comment that airmanship is not

explicitly included in the civilian instructor rating. The military BFTS instructors had

little memory of any extensive inclusion of airmanship in the CFS instructor training

program and the military advanced instructors at CFS agreed. Their position is that

FICs already have good airmanship and to instruct in it is unnecessary especially as

189

there is limited time available in the program, so they concentrate on how to teach

the technical skills, with airmanship being covered by hints on how to encourage

airmanship in the students and advice to provide a good example to them. Again this

discussion seemed to exclude technical skills and how to teach them from the

discussion of airmanship.

Comparison with recommendations.

The three step process that Kern (1996) proposed for the integrated instruction

of airmanship consists of initial indoctrination concerning the importance of

airmanship; the specific teaching and modelling of airmanship; and the enforcement

of airmanship standards. Ebbage and Spencer (2003) echo the first two steps and as

the third step, recommend assessment and provision of feedback. This process was to

some extent supported. The military BFTS program includes lectures on airmanship

in the ground school and at the Williamtown conversion training squadron. There

was mention of tutorials on airmanship at CFS. However, these comments need to be

considered in the light of their definition of airmanship as excluding technical skills.

There did not appear to be an equivalent introduction to airmanship, per se, in the

civilian ab-initio training organisations visited. The CASA set theory syllabus for the

licence exams includes human performance and limitations, and this and the

technical subjects would be covered in ground schools or technical college courses as

appropriate. Similarly, the civilian advanced level instructors tended to talk about

non-technical skills being included within the study materials but did not address

airmanship as such.

Overt inclusion of airmanship in ab-initio flight training in terms of the pre-

flight briefs and initial lessons was common to both the civilian and military

environments, covering both basic aircraft handling skills and lookout, checks and

other non-technical skills. Civilian advanced training includes both aircraft handling

and non-technical skills (although it was the non-technical skills that were equated to

airmanship). In the more advanced military training at CFS, airmanship was

considered assumed, and while there was some instruction it seemed focussed on

how to instruct future students rather than focussing on the FICs themselves

(regarded as largely unnecessary), and again was considered separately to aircraft

handling skills.

190

Assessment of airmanship forms the third part of the training process: to be

evaluated actively and aggressively as Kern (1996) put it. This may be more the case

in the BFTS environment, where there is overt assessment and feedback on the three

components of preparation, airmanship and technical skill. The lowest score is taken

as the overall mark and low marks can lead to a fail on the sortie. Persistent problems

and low scores can lead to ejection from the program, so there is equal attention to

the technical and non-technical skills being developed by the students. In the civilian

ab-initio environment, there does not appear to be overt assessment of airmanship (at

least in the organisations sampled) however comment was made that the instructors

know that their students ultimately have to meet the requirements of the flight test,

which CASA specifies as including aircraft handling skills and competency in

various aspects of non-technical skills and threat and error management, and thus

train to that standard. In the civilian advanced training environment, technical skill

appears to take precedence over non-technical skills in that poor performance in non-

technical skills can only cause a fail on the check if it is paired with a technical skill

problem. Thus, while non-technical skills may be assessed and debriefed, poor

performance on non-technical skills alone is not a failure. In the more advanced

military settings, assessment of performance in aircraft type conversion at

Williamtown appeared to parallel BFTS to some extent, with preparation, technical

handling and airmanship (as non-technical skill) being separately considered. At

CFS, assessment was not elaborated on as much, except that airmanship, it appears,

is not assessed. The FIC’s performance as an instructor was under scrutiny and led to

feedback at debrief, but there did not appear to be a formal assessment process.

Conclusion: Airmanship Training and Assessment

Airmanship as a term is not used specifically in the civilian training

environment, it seems, with the emphasis rather being on technical handling skills

and non-technical behaviours. However, these can be seen to combine to form the

integrated airmanship concepts that Kern (1996, 2009a), Ebbage and Spencer (2003)

and Edwards (2013) promote. The military system refers to airmanship as being

everything except technical handling skills when it comes to assessment, but the term

is used to embrace both in more general conversation, reflecting the integrated idea.

The military training programs each appear to consist of a set syllabus with defined

briefs and progression of lessons that integrate technical and non-technical skills

191

(called airmanship), with assessment and enforcement at each level. The civilian ab-

initio process, while guided by CASA publications, is more open to interpretation by

individual training establishments and may not have the same level of formal

assessment and enforcement of airmanship behaviours as is evident in military

training. Organisations at the more advanced levels of civilian training seem to be

somewhat restricted by the agreements with pilot organisations regarding the

evaluation of the non-technical side of the airmanship equation. This results in the

impression that non-technical skills are considered as somehow secondary to the

physical handling of the aircraft, which given the preponderance of human factors

issues in aircraft crashes, and spectacular saves, may be a mistaken viewpoint.

The interviews in this study resulted in a rich and detailed tapestry of

information about airmanship in a variety of instructional environments. The most

salient feature of the discussion is that the instructors seem able to use the term

‘airmanship’ both as a global, all-encompassing term and also a specific reference to

only non-technical skills when considering assessment. This must surely be

confusing for their students. The following chapter discusses the structure of

airmanship and the relationship of the findings in the present research to the existing

models and pilot performance literature.

192

193

Chapter 5. The Structure of Airmanship

The outcome of the research presented in Chapters 2, 3 and 4 is that in the

general sense ‘airmanship’ is a fairly loosely defined term that refers to the ability to

combine the qualities, skills and knowledge areas required to fly an aircraft safely,

efficiently and effectively. These three investigations into the definition and training

of airmanship have shown quite consistent results and broad agreement on the

concepts that make up airmanship, such as situational awareness, decision making

and communication. When writing or talking about airmanship in a general sense,

most participants included technical flying skills within airmanship, noting the all-

inclusive nature of airmanship. It was only when becoming more specific that the

trainers in military environments distinguished between airmanship, aircraft handling

skills and preparation; the three criteria which they assess. When discussing

assessment, the advanced civilian trainers also tended separate the technical handling

of the aircraft from other skills, these being labelled “non-technical” skills.

Furthermore, these trainers defined specific skills within the range of non-technical

skills that they look for during assessment. These non-technical skills were consistent

with the concepts discussed as part of the definition of airmanship.

There appears to be a tension between the general case of airmanship as “safe

and efficient flight” that requires technical skills as well as non-technical skills, and

the specific case of assessment of airmanship. The assessment of “safe and efficient

flight” requires a separation into component skills in order to allow for reasonably

objective appraisal. One approach is to separate technical handling and non-technical

skills, with further categorisation possible within each set of skills. The military has

labelled the non-technical skills as “airmanship”, resulting in some inconsistency

when discussing component concepts of airmanship in the general case and then in

terms of assessment. In the following discussion, airmanship will be considered in

the general case.

Comparison with Previous Research

Many of the findings of the research reported in the preceding chapters are

consistent with findings of previous research conducted in Australia and elsewhere

that has taken an overview of pilot performance. The most frequently mentioned and

194

highest rated and ranked concepts in the current research, including situational

awareness, decision making, knowledge, flying skills, communication, crew

coordination and preparation – have all been noted in such research. Situational

awareness and decision making have been presented as central concepts for good

TEM and for good captains (Mavin & Roth, 2014; Swauger, 2006; Thomas, 2005),

although were not specifically mentioned by Nergård (2011). Mavin (2010) notes

aviation knowledge as an enabling skill for good Captains. Book and aircraft

knowledge were also among the aspects of a good Captain reported by Swauger

(2006) and Nergård’s participants discussed knowledge of rules and regulations and

aircraft limitations. Mavin (2010) includes accurate flying as an essential skill,

Swauger’s participants also included flying skills and Nergård reports extensive

discussion about flying skills. Helmreich (2002) and Thomas (2004) list

communication as essential for TEM, Mavin includes it as an enabling skill and it is

included as a skill of a good pilot as reported in both Swauger (2006) and Nergård

(2011). Crew co-ordination is reported as another skill needed for good pilots in

Nergård (2011); Swauger (2006) reported team building and management of the

flight deck, and this wording is also used by Helmreich (2002) in listing essentials

for TEM. Preparation represented by briefing and planning are also considered of

great importance (Cahill, McDonald & Losa, 2013).

Some of the aspects of airmanship raised by participants (and reported in

Chapters 3 and 4) were less frequently noted in the existing literature. For example,

discipline was mentioned by some survey participants and was rated and ranked

quite high, and overall 36% of interviewees talked about some form of self-

discipline. The term was not reported in any of the studies mentioned above,

although Thomas (2005) included ‘self-regulation’ as a requisite for TEM, which

may relate to self-discipline. Personal qualities mentioned by participants in the

current research included responsibility, patience and honesty, and a desire to excel

or to improve. These are not reproduced in the existing research, although different

personal qualities were mentioned. Thomas (2005) included negotiation skills and

assertiveness as TEM requirements. Swauger (2006) noted mention of some personal

qualities such as having a friendly and relaxed personality and an easy-going nature.

Nergård et al (2011) reported ‘good attitude’, this being acting and socialising

appropriately, and also humility, explained as recognising one’s own errors and

195

asking for help as well as helping other. Self-awareness, self-efficacy and self-esteem

were also reported in that study.

Reference to crew resource management (CRM), as such, was not frequent in

the current research, participants tended more towards reporting components of CRM

(e.g. decision making and communication). In contrast, Swauger (2006) found that

‘establishing a good CRM environment’ was the most frequently mentioned aspect

of a good captain. Similarly, Nergård et al (2011) noted that participants indicated

that the ability to fly the aircraft using CRM was essential.

Model of Airmanship

The concepts raised by participants in writing or talking about airmanship can

be categorised into five main domains, as shown in Figure 5.1. The participants also

mentioned the context in which airmanship is developed or later occurs, and this is

also included in Figure 5.1.

Knowledge

Knowledge is essential to good airmanship. Knowledge is acquired during the

learning involved in obtaining a pilot licence and is represented by the theory

syllabus specified by the regulator, or the ground school learning in the military

sector. Knowledge includes an understanding of the operational parameters of

particular types of aircraft. It also involves updating one’s knowledge of rules and

regulations and procedures as changes occur. It is important to note that the

participants in the current research did not just mention the necessity for acquiring

relevant knowledge, they also indicated that the correct application of knowledge is

required for good airmanship.

Knowledge is specified in both the Kern (1996, 2009a) and Ebbage and

Spencer (2003) models, and is considered a major part of each element in Edwards’

(2013) discussion of airmanship. In addition the Swauger (2006) study indicated the

importance of knowledge, with book and aircraft knowledge being part of one of the

three most frequently mentioned categories of a ‘best captain’. Nergård (2014)

reported that the pilots in his study considered the gaining and application of

practical knowledge to be embedded in the concept of airmanship.

196

Figure 5.1 A new airmanship model

Preparation

Preparation and flight planning were mentioned in both the surveys and

interviews. The concept of preparation was important in both the rating and ranking

questions reported in Chapter 3. The details mentioned by participants when talking

about preparation included the noting of current and likely weather, what the flight

would involve that day, correct maps and charts and any notices to airmen

(NOTAMS) that might affect the flight. These things are covered by mission

knowledge in Kern (2009a). Ebbage and Spencer (2003) included flight preparation

as an example or sub-set of discipline and Edwards (2014) refers to preparation as

part of the operational element of airmanship. Other preparation items that were

mentioned in the current research are an assessment of one’s own limitations and

issues that may affect performance of other crew members. There were also noted by

Nergård (2011). The preparation and delivery of briefings, or attending them, and

pre-flight inspection of the aircraft were mentioned by participants. Risk assessment

197

is a preparation item, although awareness of risks and threats could also be

considered an ongoing activity during the flight.

Non-technical skills

The most common concepts referred to by the participants when defining

airmanship were the skills and abilities that are generally referred to as non-technical

skills: the human factors of flight and the more cognitive activities of the pilot. These

include situation awareness, the next most frequently mentioned concept (after

safety) throughout the current research, and rated and ranked most highly in the

rating and ranking questions reported in Chapter 3. Good situation awareness

requires lookout and listen-out, the earliest items that instructors introduce in

teaching airmanship, as noted by the instructors interviewed (reported in Chapter 4).

Update of the aircraft state by regular scan of the instruments (coded within ‘checks’)

is part of situational awareness, as mentioned by almost all of the instructors.

Decision making was mentioned almost as frequently as situation awareness

and was also ranked and rated highly as important to airmanship. Decision making

(as judgement) is the capstone outcome in the Kern model (1996), and is a major part

of both the Ebbage and Spencer (2003) model and the non-technical standards

discussed by Edwards (2013).

Communication was frequently mentioned as an important part of airmanship

in the surveys and the interviews in the current research. However, the appearance of

communication in the rating and ranking questions was split up into four phrases:

crew interrelations, communication with other aircraft, communication with ATC

and communication with maintenance. Possibly as a result of this dilution of

communication, each phrase ended up in the lower half of the ratings and rankings.

As noted above, communication is seen as an important component of CRM and

good piloting in the literature.

Crew-related and teamwork skills, including crew co-ordination, management

of the flight deck and leadership, were mentioned in the surveys in the current

research. These concepts were most frequently included as part of airmanship by the

advanced civilian instructors during the interviews (reported in Chapter 4). Such

skills are seen as essential to TEM (Helmreich, 200) and for good pilots (Swauger,

2006; Nergård et al, 2011).

198

Non-technical skills, apart from situational awareness and judgement, were

subsumed under ‘knowledge of the team’ in the Kern (1996) model, and formed

‘team skills’ in the Ebbage and Spencer (2003) model. Edwards (2014) considers

non-technical skills to be a major element of airmanship.

Technical Skills

The overall concept of airmanship includes aircraft handling or flying skills as

part of safe and efficient flight. A few participants in the first on-line survey,

(reported in Chapter 2) implied that airmanship is only about control of the aircraft.

In the second survey (reported in Chapter 3), significantly more military background

respondents mentioned aircraft handling skill as part of the definition of airmanship

than did civilian background respondents. By overall number of mentions, flying

skill was the sixth most frequently mentioned concept in the second survey. In

addition flying skill made it into the top ten concepts in the ranking question, but was

at the low end of the distribution of mean ratings. In the interviews (reported in

Chapter 4) flying skill was one of the most frequently mentioned concepts with

regard to airmanship. However, not all participants were prepared to include flying

skill within their definition of airmanship; most of the military ab-initio instructors

excluded technical skills. Nonetheless, when considering airmanship as a global

concept of safe and efficient flight, the skill to effectively control the aircraft has to

be included.

Kern (1996) included ‘skill’ as one of the bedrock principles of airmanship,

along with proficiency. Ebbage and Spencer (2003) included physical flying skills as

part of their model of airmanship and Edwards (2013) presented technical skills as

the first of his elements of airmanship. Swauger (2006) found that flying skill was

one of the middle tier elements (in terms of frequency of mention) of a “best pilot”,

and Nergård (2011) reported extensive discussion of technical abilities as part of

‘what makes a good pilot’.

Personal Attributes

There were many participants who listed various personal qualities or attributes

they viewed as part of good airmanship. These included discipline, self-

improvement, responsibility, common sense and diligence.

199

Discipline was mentioned by participants in all three studies. In the second on-

line survey, discipline was rated third-highest concept and ranked sixth, so it was

considered important by those participants. Self-discipline was also implied in the

comments about complying with rules and regulations and not cutting corners. Kern

(1996, 1997) sees discipline as the cornerstone of good airmanship, essential for

maintaining standards and safety. Rather than showing it as a separate element of the

model, as Kern does, it is included as a personal attribute in Figure 5.1.

Another personal attribute mentioned by participants in the current research

was the idea that the aviator should be internally motivated to improve, to be the best

they can and/or work towards improvement. This is consistent with the Ebbage and

Spencer inclusion of self-assessment and self-development in their model. Kern

(1996) also discusses these concepts in ‘knowledge of the self’. Edwards (2013)

included a number of personal attributes in his discussion of the non-technical

elements of airmanship (e.g., enthusiasm, motivation, confidence, honesty and

perseverance).

Common sense was frequently mentioned and rated and ranked highly in the

relevant questions in the second survey. Common sense is not particularly well

defined, but seems to be used to express the sensible, logical use of skills and

knowledge to achieve the safest outcome, especially where rules and procedures do

not apply. It is included in personal attributes rather than non-technical skills because

it is poorly defined, somewhat idiosyncratic, and not a skill, as such.

The current research also supports the idea that there is a place for individual

responsibility in airmanship, with many participants mentioning responsibility.

Responsibility was rated the second highest concept and was ranked twelfth in the

second survey (reported in Chapter 3), and was mentioned by many of the civilian

ab-initio instructors in their interviews (reported in Chapter 4).

Context

The influence of the context on airmanship is indicated by the sixth block in

Figure 5.1, presented in a different colour to contrast with the internal aspects of

airmanship (the blue blocks).

200

Kern (2009a, 2009b) suggests that constant striving for perfection and firm

discipline should be central to the idea of airmanship and can defeat any tendency to

make errors. However, there are occasions when the situation stacks up against the

individual, reflected in the position that errors, incidents and accidents can be

ascribed to systemic faults and training problems within organisations, and not

necessarily primarily related to fault on the part of an individual (Reason, 1990). The

individual pilot cannot be considered in isolation, but must be seen as part of a

system. Thus airmanship is also a product of the environment in which pilots were

trained, the examples they have had, and is modified by their current organisational

culture. These factors were all brought up by various participants in the current

research. The ideal situation is one in which the individual pilot accepts

responsibility for maintaining their own competence at the highest possible level (i.e.

good airmanship), supported by a positive organisational culture, effective training,

well maintained aircraft and operating within a well-regulated system. In a less ideal

world, a pilot may feel they have to rise above the inadequacies of a poor

organisational culture, inadequate training, defective aircraft and/or a dysfunctional

system. In this situation, maintaining individual standards of good airmanship

regardless of the context may make a significant difference to the safety of their

flight.

Conclusion

The model of airmanship proposed here is intended to provide a more fluid

impression than the rigid temple façade of the Kern (1996, 2009a) model and avoids

the hierarchy of the components of airmanship in that model. The domains in the

structure of the model presented in Figure 5.1 reflect grouping of compatible

concepts, raised by participants in the research reported here. Furthermore, this

model acknowledges the influence of the context in which airmanship is displayed.

201

Chapter 6. Airmanship Concept Domains

The airmanship model developed from the findings of the surveys and

interviews, reported in earlier chapters, and presented in Chapter 5, is a logical and

semantic organisation of the airmanship-related concepts mentioned by participants.

The model suggests five domains of concepts: knowledge, preparation, technical

skills, non-technical skills, and personal attributes. It also recognises that the exercise

of airmanship takes place within a context that includes the training environment,

mentorship, experience, organisational culture and the regulatory system.

A further study was required to confirm that the grouping of concepts into

domains in the proposed model of airmanship reflects the conceptualisation of

airmanship among Australian aviators. The study consisted of two parts: a pilot

study, in which a questionnaire was developed and tested for suitability and the main

study, which was an on-line survey using the refined questionnaire. The pilot study is

reported first, followed by the main on-line survey.

The aim of this study was to investigate whether Australian aviators consider

the concepts that were raised in discussion of airmanship in the earlier studies and

formed the basis of the model presented in Chapter 5, are appropriately grouped in

the five domains indicated in the model. Support for the domains and the model of

airmanship would be provided if the concepts paired with other concepts from the

same domain (within-domain pairs) are regarded as more closely related than pairs of

concepts from different domains (between-domain pairs).

A set of 15 concepts were drawn from the proposed model of airmanship.

Participants were asked to rate how closely related they thought pairs of these

concepts were, in the context of their interpretation of airmanship.

Methodological Considerations

A form of paired comparisons was considered more likely to produce useable

results than creating statements and asking participants how strongly they agreed

with them, or asking how important participants thought each of a selection of

concepts was to airmanship. A questionnaire asking about the relevance, or

202

importance, of the concepts in the model to airmanship is likely to suffer from an

upward bias, as was found in the rating section of the survey reported in Chapter 3,

wherein most concepts were rated as important or very important. This kind of

upward bias was also found by Nergård (2011). In a study of ‘what makes a good

pilot’, Nergård asked Scandinavian pilots to rate their agreement on a 10 point

Likert-type scale to a set of 18 statements; 15 of the mean scores for the statements

were between 8.20 and 9.49, very close to the top of the scale. Such upward bias

would reduce the differentiation among concepts and possibly rule out factor analysis

due to violation of assumptions of normality. The intent for the current study was

therefore to ask about the association between various concepts already known to be

relevant, from responses made by participants in the earlier studies, and use multi-

dimensional scaling to examine the clustering of concepts.

Multi-dimensional scaling (MDS) is a multi-variate statistical technique in

which quantitative comparisons between pairs of concepts can be used to determine

the structure of a data set (Bartholomew, Steele, Moustaki & Galbraith, 2002). MDS

has advantages over factor analysis in that MDS does not require that the underlying

data distribution be normal, nor that the relationships be linear, and generally

produces more parsimonious descriptions of underlying data structures (Brazill &

Grofman, 2002). One aim of MDS analysis, then, is to discover whether the data set

can be represented in a small number of dimensions. Co-location of data points in the

appropriate dimensional representation should reflect underlying groupings of data,

or the data may show variability along dimensions inferred from the analysis

(Steyvers, 2002).

In MDS, the goodness of fit of the data set to the dimensional representation is

referred to as the ‘stress’; values of stress that are close to zero indicate that the MDS

solution is a good fit. The standard given is that a stress of 0.20 is poor, 0.05 is good

and 0.00 is perfect (Kruskal, 1964, cited by Bartholomew, Steele, Moustaki &

Galbraith, 2002). Borg and Groenen (2005) note that the index of stress is closely

related to the amount of error (or variability) in the data: as this rises, so does the

stress.

There is a trade-off between the number of dimensions required to improve fit,

and the interpretability of the resulting multi-dimensional plot (Borg & Groenen,

203

2005). Too few dimensions may distort the MDS structure due to over-compression,

while too many dimensions may blur the structure due to over-fitting of noise

components. Thus, a data set with some variability in the estimates of distance may

lead to increased stress values, resulting in a decision to use a plot of higher

dimensionality than is supportive of the underlying data structure (Borg & Groenen,

2005).

Pilot Study

A pilot study was conducted to ensure that the survey instructions and the

format and content of the questions were clear. It also allowed estimation of the time

required to answer the questions.

Method

Participants

Thirty-five pilots known to the researcher were contacted and asked to try out

the survey. Of these nine returned the survey within the required time-frame (a 25%

response rate). These were all Australian males. Six of the participants trained

initially as civilians, two had initial civilian training but moved on to the military and

one did not answer this question. Two participants held a CPL, six held an ATPL and

one had military qualifications only. Five participants flew for airlines, two in the

military and one was a civilian instructor and one was in charter operations.

Experience ranged from 1500 hours to 18,000 hours (M = 4372.22, SD = 3707.18)

and from 3 years to 30 years (M = 14.14, SD = 8.69).

Measure

A survey was developed that asked pilots to indicate how closely related pairs

of concepts appeared to them. Three representative concepts were selected from each

of the five domains of airmanship shown in the model in Figure 5.1: knowledge,

preparation, technical skills, non-technical skills and personal attributes. In the cases

of knowledge, preparation and non-technical skills, these were the most frequently

mentioned concepts in each domain, consistently most frequently mentioned across

the data sets reported in Chapters 3 and 4. For the technical skills concepts that were

rated or ranked highly in the second survey were used and for the personal attributes

204

domain, concepts were extracted from those included in the personal qualities

composite node in the second survey.

Three concepts were chosen from each domain. Adding more concepts would

have made the number of comparisons too large for participants to complete in a

reasonable time frame. Each of the 15 concepts was paired with every other concept

(105 pairs). There were also 10 additional pairings of concepts from within the same

group and eight pairs of concepts from different groups, presented in reverse order,

for a total of 123 pairs. Reverse pairs were included as a check on the reliability of

the ratings. The domains and concepts are shown in Table 6.1.

Participants were first asked to think about their “ideas about ‘airmanship’ and

the components that make it up, the things that you need in order to display good

airmanship”. They were then presented with each concept in turn and asked to rate

the ‘relatedness’ of the concept with each of eight other concepts (two concepts had

9 others), some from the same domain and some from other domains. The concepts

were presented in random order, and the comparison concepts were randomised

within the set. The five point Likert-type rating scale was from “very low

relatedness”, through “low”, “medium relatedness”, “high” to “very high

relatedness”. The pilot survey was presented in an excel file in which the participants

indicated responses by adding a cross in the relevant column for the Likert scale.

Table 6.1 Domains and representative concepts

Domain Concepts

Knowledge aircraft type knowledge; knowing SOPs (standard

operating procedures); knowledge of rules and regulations

Preparation flight planning; pre-flight briefing; preparation

Technical skills aircraft handling; control of the aircraft; proficiency

Non-technical skills situational awareness; decision making; communication

Personal attributes discipline; motivation to excel; responsibility

205

Procedure

Ethics approval was granted through a variation to the ‘parent’ ethics

application, H-647-1207.

The researcher emailed 35 pilots known to her asking them if they would be

willing to complete the questionnaire (attached as an excel file) which was being

developed for her research. The participants were asked to return the completed

questionnaire, by email, together with any comments they wished to make. They

were also asked to provide an estimate of the time taken to complete the

questionnaire.

Analysis

The data from the returned excel sheets were converted into SPSS (IBM, 2015)

for analysis of mean relatedness.

Results

Mean relatedness between pairs of concepts is shown in Table 6.2. Within-

domain relatedness is shown in bold. For clarity, any data located in the upper

diagonal half of the table were transposed to the lower half, thus leaving only the

‘reverse pairs’ in the upper section. Two issues emerged from observation of the

pilot data. First, due to an oversight, one pair of concepts (pre-flight briefing with

discipline) was omitted from the pilot questionnaire. Second, the repeat measure

pairings were concentrated on pairs with situational awareness and preparation,

rather than more evenly distributed across the table.

206

Table 6.2 Mean relatedness between pairs of concepts in the pilot data grouped by domain

Domains Concepts

Knowledge Non-technical skills Preparation Personal attributes Technical skills

Aircraft type

knowledge

Know

ing SOPs

Know

ledge of rules &

regs

Com

munication

Decision

Making

Situational aw

areness

Flight planning

Pre-flight briefing

Preparation

Discipline

Motivation to

excel

Responsibility

Aircraft

handling skills

Control of aircraft

Proficiency

Know

ledge

A/c type knowledge 2.75 Knowing SOPs 3.00 4.50 3.00 Knowl of rules & regs 3.25 4.88 3.38

Non-

technical skills

Communication 2.00 2.40 3.00 4.38 4.13 Decision Making 3.80 4.10 4.00 4.25 4.63 3.75 4.25 Situational awareness 2.88 3.13 3.25 4.25 4.38 3.38 Prepara-

tion

Flight planning 3.10 3.63 4.00 3.00 4.25 3.10 4.75 3.25 Pre-flight briefing 3.75 4.13 4.38 3.50 3.75 3.75 4.75 4.75 Preparation 3.63 4.13 4.10 3.40 4.10 3.88 4.75 4.13 Personal

attributes

Discipline 3.63 3.80 4.38 2.80 3.63 3.63 3.63 4.38 3.75 Motivation to excel 3.40 3.40 4.00 2.75 3.50 2.30 2.75 3.00 3.10 4.50 3.88 Responsibility 4.00 3.60 4.10 3.00 3.80 3.88 3.50 3.30 4.38 3.88 4.00 3.00 Technical

skills

A/c handling skills 3.30 2.80 1.75 2.38 3.00 2.30 1.40 1.75 2.75 3.25 2.80 2.75 4.38 Control of a/c 4.13 3.00 1.80 1.50 2.30 3.38 1.38 1.80 2.00 3.30 2.50 3.40 5.00 Proficiency 4.40 4.75 4.38 3.30 3.88 4.38 3.30 3.13 3.60 3.88 3.30 3.13 4.25 4.50

Note. Within-domain means are shown in bold. A/c = aircraft, regs = regulations.

207

The pilot data, while from a very small sample, show that the within-domain

pairings were generally rated as more closely related than the between-domain

pairings. An exception was the mean relatedness of ‘aircraft type knowledge’ with

other concepts, nearly all of which were higher than the relatedness with the within-

domain knowledge concepts. More importantly the repeated pairs (shown in the

upper diagonal half of Table 6.2) showed similar relatedness ratings to the same

pairs in the reverse order in the lower diagonal half of the table. This indicates that

the relatedness did not depend upon the order in which the concepts within a pair

were presented and that relatedness is measured reliably.

The participants reported taking between 15 and 20 minutes to complete the

survey. Comments were that the survey was easy to understand and quite clear. One

participant mentioned that ‘relatedness’ was a clumsy term, without offering an

alternative.

Discussion

The pilot study drew attention to some problems with the distribution of

questions across the range of concepts. This was corrected by ensuring equal

representation of comparisons between each concept domain in the survey proper. It

appeared the length of the survey was reasonable and was not considered too time

consuming by the participants. The data followed the expected patterns, although

there was some indication that knowledge of aircraft type was judged differently

from the other knowledge concepts. Further clarification of this was expected to

emerge after a larger sample was surveyed.

Main Study

The aim of the study was to examine whether airmanship concepts are seen by

Australian aviators to be related to each other in terms of the five domains indicated

in the model presented in Chapter 5.

208

If this model of airmanship accurately represents the views of Australian

pilots, then it is expected that concepts paired with other concepts from the same

domain in the model (i.e. within-domain pairs) will be regarded as more closely

related than pairs of concepts from different domains (i.e. between-domain pairs)

when pairs of concepts are presented for comparison. It was also expected that multi-

dimensional scaling of the resulting data set would show clusters of concepts that

reflect the domains shown in the airmanship model.

Method

Participants

The criteria for inclusion were: 1) a pilot who had trained in Australia; or 2) a

pilot trained elsewhere but who was currently based in or flying in Australia. Of the

232 hits on the on-line survey site, 87 met one of the two criteria. The majority of

participants were pilots trained in Australia (n= 79). A further eight were pilots

trained elsewhere, but flying in, or based in Australia. Of the remaining 145 hits on

the site, 19 pilots who were neither Australian trained nor Australian based, and 126

did not complete the survey.

Ten participants were female and 76 were male, with one participant choosing

not to answer this question. Participants were asked in which sector they undertook

their initial training. Twenty-one participants reported initial training in the military,

ten reported initial civilian training and then a move to the military. The remaining

56 participants trained in the civilian sector. Experience ranged from 235 to 25,000

hours (M = 7037.78; SD = 5810.95), and from one year to 46 years in aviation (M =

18.97; SD = 11.02).

As reported above, the majority of the participants learned to fly in Australia (n

= 79). The remaining eight participants were currently flying in or were based in

Australia, but were trained in New Zealand (N = 3), UK (N = 3) or the USA (N = 2).

In total, 73 participants were currently flying in or were based in Australia; nine in

Hong Kong; three in Saudi Arabia; one in China; and one in Singapore. Participants

also reported their level of flying qualification: only two participants held a Private

Pilot’s Licence (PPL) or lesser qualification; 19 reported having a Commercial Pilot

209

Licence (CPL); 57 reported an Airline Transportation Pilot’s Licence (ATPL) and

nine reported Military qualifications.

Participants were also asked “what sort of flying do you mostly do, currently?”

Three participants chose not to answer this question. Forty-five participants reported

airline regular public transport activity, seven reported private or recreational flying,

five reported general aviation/airwork activity, three reported charter flying, twelve

reported civilian instruction and nine military instruction, and finally, three

participants reported military flying (not instruction).

Measure

An online survey was developed from the pilot survey. The same frequently

mentioned concepts were used as in the pilot study, three from each of the five

domains of concepts that make up the model of airmanship proposed in Chapter 5

(listed in Table 6.1). Assessment of the reliability of the relatedness ratings was

examined by adding 20 reverse pairs, 10 within-domain and 10 between-domain

pairs, balanced such that there were two pairs from each within-domain set and a

sample pair from each possible between-domain pairing, ensuring that the same

concepts were not over-represented. This gave a total of 125 pairs for participants to

rate.

Participants were first asked to think about their “ideas about ‘airmanship’ and

the components that make it up, the things that you need in order to display good

airmanship”, to establish the context of airmanship. They were then presented with

each concept in turn and asked to rate the relatedness of the concept to each of eight

or nine of the other concepts, some within-domain and some between-domain. The

15 concepts were presented in random order and each set of eight or nine comparison

concepts were also randomised. The five point Likert-type rating scale was from

“very low relatedness”, through “low”, “medium relatedness”, “high” to “very high

relatedness.” Some demographic questions concluded the survey, these included

questions about gender, initial training, country of initial training and current base,

level of pilot licence, current type of flying and amount of experience. A copy of the

questionnaire can be found in Appendix F.

210

Procedure

A snowballing recruitment strategy was employed, in which all 120 pilots

known to the researcher and colleagues were emailed and invited to complete the on-

line survey. Their assistance was sought to forward the email to other pilots whom

they knew and were comfortable contacting. Several contacts reported notifying their

airline training section and there was generally further distribution of the invitation

through these organisations.

The recruitment email included the URL for the survey site. The initial page of

the survey contained information about the survey and contact details for the

researchers and ethics committee. Continuation to the survey questions was

considered to indicate consent to participate. The closing page thanked the

participant for their time and indicated that if the participant wanted a report on the

results of the survey they could email the researcher and request a copy. There were

several requests from pilots not known to the researcher, indicating that some

snowballing had occurred.

Analysis

SPSS23 (IBM, 2015) was used to conduct statistical analysis of reliability and

relative relatedness of within-domain and between-domain concepts. Matlab (2015)

was used to perform multi-dimensional scaling to determine the explanatory

clustering within the data.

Results

Reliability

Two approaches to estimating reliability were used: first whether the mean

relatedness differed for the forward-worded concept comparisons and their reverse-

worded pairs; and second whether the ratings of these forward-worded concept

comparisons and their reverse-worded pairs were correlated. If the survey method is

reliable, the mean relatedness of the forward-worded and reverse-worded pairs

should not differ significantly and the ratings for the forward-worded concepts and

their reverse worded pairs should be highly correlated. Table 6.3 presents the

forward/reverse worded pairs, their mean relatedness and the outcomes of the

211

reliability checks. Due to the number of comparisons being made, the Bonferroni

adjusted critical value of p < .001 was used to control the family-wise false positive

error rate.

Because the five-point scale used to rate relatedness is essentially an ordinal

(rather than interval or ratio) scale, comparison of the mean relatedness of the

duplicated pairs was performed using the Wilcoxon signed ranks test. The Wilcoxon

tests on each set of pairs showed significant differences (at p < .001) between three

pairs: knowledge of rules and regulations with aircraft type knowledge; motivation

to excel with aircraft type knowledge; and aircraft handling skills with flight

planning. The remaining 17 duplicated pairs did not show significant differences in

mean relatedness as a function of forward vs reverse pairing of concepts.

Table 6.3 also shows the outcome of the calculation of a Pearson product

moment correlation coefficient for each of the forward/reverse worded pairs. The

majority of the pairs were significantly correlated, with correlation coefficients

ranging from .32 to .69. The between-domain pairs tended to have higher

correlations than the within-domain pairs.

Overall, few (three) of the sets of pairs showed significant differences in mean

rated relatedness when the order of the concepts was reversed. The majority (17) of

the forward/reverse-worded pairs’ ratings were significantly correlated. Although it

is not absolutely certain that reported relatedness would be the same regardless of the

order in which the concepts appeared in the pairings in the survey, it is reasonable to

say that the reported relatedness is sufficiently reliable to permit further analysis.

212

Table 6.3 Mean relatedness for forward and reverse worded pairs, and outcome of Wilcoxon Signed Ranks (WSR) tests and Pearson correlations for each pair

Relatedness means

WSR Pearson correlations

Concept Pairs forward reverse p r p

Within domain

Knowledge of rules & regs – Aircraft type knowledge 3.33 2.90

* .51 *

Knowledge of rules & regs - Knowing SOPs 4.37 4.18

.65 *

Situational awareness - Communication 4.13 4.33

.35 *

Situational awareness - Decision making 4.59 4.53

.22

Preparation - Flight planning 4.31 4.48

.32

Preparation - Pre-flight briefing 4.39 4.46

.43 *

Responsibility - Discipline 4.28 4.24

.55 *

Responsibility - Motivation to excel 3.84 4.09

.46 *

Control of aircraft - Aircraft handling skills 4.71 4.72

.39 *

Proficiency - Aircraft handling skills 4.30 4.28

.29

Between domains

Pre-flight briefing - Aircraft type knowledge 3.26 3.16

.58 *

Motivation to excel - Aircraft type knowledge 4.08 3.24

* .52 *

Control of aircraft - Knowing SOPs 3.29 3.08

.46 *

Situational awareness - Knowledge of rules & regs 3.52 3.64

.59 *

Discipline - Communication 3.67 3.51

.60 *

Proficiency - Decision making 4.18 4.06

.40 *

Preparation - Situational awareness 4.09 3.91

.52 *

Motivation to excel - Flight planning 3.40 3.11

.69 *

Aircraft handling skills - Flight planning 2.58 2.23

* .64 *

Aircraft handling skills - Responsibility 3.25 3.46

.60 *

Note: * p < .001; regs = regulations

213

Within-Domain and Between-Domain Differences

Table 6.4 presents the mean relatedness for the comparisons between each of

the 15 concepts and every other concept. The reverse worded pairs have been

omitted from the table and from any further analysis. Any pairs positioned in the

upper diagonal half of the table due to the concept order of the paired comparison

have been transposed to the lower diagonal half, so that all relatedness means fall in

the lower section. The concept pairs from within the same domain are shown in bold

in the table, along the diagonal.

An expectation of the data was that the ‘within-domain’ relatedness would be

greater than the relatedness of the relevant pairings with concepts from other

domains (between-domain). Construction of an overall within-domains relatedness

variable (combining the ratings that made up the means along the diagonal in Table

6.4) and an overall between-domains variable (combining the ratings that made up

the non-diagonal means) allows testing of this expectation.

A repeated measures t-test indicated that overall within-domain relatedness (M

= 4.11, SD = .412) is significantly greater than overall between-domain relatedness

(M = 3.63, SD = .490), t (69) = 13.06, p < .001, and this is a large effect (d = 1.07).

Within-domain and between-domain differences can also be examined at the

domain level and at the concept level. These are presented in Appendix G.

Using calculated overall relatedness means (as has been done above and in

Appendix G) supported the expectation that there will be systematic differences

between within-domain and between-domain means. However using such summary

means results in a loss of the information contained in the variations among

individual concept-pair ratings. An MDS analysis using the full set of concept pairs

takes into account such variation among individual comparisons.

214

Table 6.4 Mean relatedness of pairs of concepts grouped by domain

Domain Concepts

Knowledge Non-technical skills Preparation Personal Attributes Technical skills

Aircraft type

knowledge

Know

ing SOPs

Know

ledge of rules &

regs

Com

munication

Decision

Making

Situational aw

areness

Flight planning

Pre-flight briefing

Preparation

Discipline

Motivation to

excel

Responsibility

Aircraft

handling skills

Control of aircraft

Proficiency

Know

ledge

A/c type knowledge Knowing SOPs 3.41 Knowledge of rules & regulations 3.33 4.37

Non-

technical skills

Communication 2.71 3.47 3.32 Decision Making 3.77 4.08 4.16 4.14 Situational awareness 3.65 3.60 3.52 4.13 4.56 Prepara-

tion

Flight planning 3.33 3.55 3.90 3.13 3.98 3.73 Pre-flight briefing 3.26 3.84 3.97 3.90 3.83 4.00 4.13 Preparation 4.29 4.16 4.05 3.34 4.15 4.09 4.31 4.39 Personal

attribute

Discipline 3.81 4.46 4.17 3.67 3.97 3.84 3.71 3.95 4.09 Motivation to excel 4.08 3.76 3.71 3.39 3.36 3.04 3.40 3.49 3.69 4.30 Responsibility 4.02 4.11 4.24 3.93 4.23 3.82 3.76 3.82 4.14 4.28 3.85 Technical

skills

A/c handling skills 3.85 3.19 2.68 2.69 3.33 3.35 2.58 2.65 3.34 3.74 3.69 3.25 Control of aircraft 4.26 3.29 2.78 2.86 3.71 3.76 2.61 2.48 3.94 3.60 3.03 3.69 4.71 Proficiency 3.83 4.20 3.94 3.55 4.18 4.22 2.87 3.45 3.38 4.03 3.87 3.72 4.30 4.18

Note. Within-group means are shown in bold. A/c = aircraft; regs = regulations

215

Multi-dimensional scaling

The data presented in Table 6.4 were used in an MDS analysis using Matlab. In

order to do this, values for the diagonal cells were required (i.e. the comparison of

each concept with itself). These were set at ‘5’, based on the assumption that each

concept would be considered identical to itself and ‘5’ represents closest relatedness.

A second assumption was that the data could be reflected about the diagonal. The

checks of reliability reported above support this assumption; the order of concepts

within the pair comparisons does not lead to different reported relatedness for the

majority of pairs. The final assumption is that distance apart is represented by the

complement of the relatedness of the concepts (i.e. distance = 5 minus relatedness).

A non-metric analysis was used due to the concepts being a psychological estimate,

made on an essentially ordinal scale, rather than a physical measurement (Borg &

Groenen, 2005; Steyvers, 2002).

The scree plot of the MDS solutions is shown in Figure 6.1. As noted in the

discussion of methodology earlier in the chapter, a stress value of .05 is considered

good, but would require a 5-dimensional solution in this analysis. The ‘elbow’ in the

stress curve occurs at two dimensions and the 2-dimensional plot (Figure 6.2) is

easier to interpret, but the stress value is approximately .15. The 3-dimensional

solution has somewhat lower stress, at .10, and expresses the relationships between

the concepts more exactly. Matlab is able to show a rotation of the 3-dimensional

plot, making the relative positions of concept points more explicit. Figures 6.3 and

6.4 are two views of the 3-dimensional solution.

The MDS 2-dimensional solution is a good reflection of the expected

relatedness of the concepts within three of the five domains. The non-technical skill

concepts, the technical skills concepts and the preparation concepts are clearly

distinct from each other and from the knowledge and personal attribute concepts.

216

Figure 6.1 Scree plot of stress values for one to seven dimension solutions.

Figure 6.2 MDS 2-dimensional plot. Note that NT1 = communication, NT2 = decision making, NT3 = situation awareness, P1 = flight planning, P2 = pre-flight briefing, P3 = preparation, T1 = aircraft handling skills, T2 = control of the aircraft, T3 = proficiency, K1 = aircraft type knowledge, K2 = knowing SOPs, K3 = knowledge of rules and regulations, PA1 = discipline, PA2 = motivation to excel, PA3 = responsibility.

217

Figure 6.3 MDS 3-dimensional plot at 20 degrees of rotation. Note that NT1 = communication, NT2 = decision making, NT3 = situation awareness, P1 = flight planning, P2 = flight planning, P3 = preparation, T1 = aircraft handling skills, T2 = control of the aircraft, T3 = proficiency, K1 = aircraft type knowledge, K2 = knowing SOPs, K3 = knowledge of rules and regulations, PA1 = discipline, PA2 = motivation to excel, PA3 = responsibility.

Figure 6.4 MDS 3-dimensional plot at 60 degrees of rotation. Note that NT1 = communication, NT2 = decision making, NT3 = situation awareness, P1 = flight planning, P2 = flight planning, P3 = preparation, T1 = aircraft handling skills, T2 = control of the aircraft, T3 = proficiency, K1 = aircraft type knowledge, K2 = knowing SOPs, K3 = knowledge of rules and regulations, PA1 = discipline, PA2 = motivation to excel, PA3 = responsibility.

218

Figure 6.3 shows that knowledge of the aircraft type (K1) is closely associated

with control of the aircraft (T2); aircraft handling skills (T1) is relatively close by;

and all three concepts are quite distinct from the other concepts. Proficiency (T3) is

far closer to the personal attributes (PA) concepts, which form their own grouping

between the other knowledge items (K2, rules and regulations, and K3, SOP’s) and

the technical skills. The preparation and non-technical skills concepts appear mixed

together in Figure 6.3, but in the rotated view in Figure 6.4 it is clearer that they are

actually separated.

Discussion

The intent of this study was to determine if the airmanship model presented in

Chapter 5 accurately reflects how Australian pilots view airmanship. To do so,

concepts already shown to be relevant to airmanship through previous surveys and

interviews (discussed in Chapters 2, 3 and 4) were selected to represent the domains

of the airmanship model and participants were asked to rate the relatedness of pairs

of these concepts in an on-line survey.

The initial expectation was that the within-domain relatedness would be greater

than that of the relevant between-domain concept pairs. The significant difference

between the overall within-domain and between-domain relatedness variables

supports this expectation.

It was also expected that the multi-dimensional scaling of the concept

relatedness data would reflect the five concept domains as presented in the

airmanship model proposed in Chapter 5. The MDS solutions support the idea that

there at least three separate groups in the data, corresponding to the technical skills,

non-technical skills and preparation domains. Personal attributes seem to be fairly

closely associated with two of the knowledge concepts (knowing SOPs and

knowledge of rules and regulations) while the knowledge of aircraft type concept is

more closely related to aircraft handling and control of the aircraft from the technical

skills group. The remaining technical skill concept, proficiency, seems more closely

linked to personal attributes than to the other technical skill concepts.

219

It appears that the participants were considering the knowledge concepts in a

functional way, in terms of how each pair of terms related to the tasks at hand. Thus,

knowledge of the aircraft type is most highly related to (and co-located with) control

of the aircraft and, to a slightly lesser extent, aircraft handling skills for which such

knowledge is needed. Both knowing SOPs and knowledge of rules and regulations

are located fairly close the personal attribute concept of discipline, perhaps reflecting

what is required to maintain such knowledge. Similarly, proficiency is close to

discipline and motivation to excel, reflecting attributes required to attain proficiency.

The interleaving of knowledge with aviation activity is reflective of the

comment made in earlier surveys and interviews (reported in Chapters 3 and 4) that it

is not just having relevant knowledge that is important, but also the ability to apply it

appropriately. Similarly, personal attributes are not simply required of themselves,

but are important for the effect that they have on other domains within airmanship.

Possibly, without responsibility, preparation is not adequately completed; without

discipline, knowledge is not maintained; and without the motivation to excel,

proficiency is not achieved.

The model of airmanship presented in Chapter 5 has been adapted to take into

account the relationships that have become apparent in the MDS analysis. This

model reflects how Australian aviators report the relationships between concepts that

are part of airmanship. The model illustrates airmanship as having three main

domains of technical skills, non-technical skills and preparation activities, each of

which is informed by the appropriate knowledge and reinforced by relevant personal

attributes. Figure 6.5 illustrates the reworking of the model.

A limitation of the current study is that there were only 15 concepts included in

the survey. The inclusion of additional concepts may have unnecessarily complicated

the resulting plots, but may also have added further explanation. It would be

interesting to observe how the inclusion of additional concepts would interact with

those presented in the current study. However additional concepts would add

considerably to the time taken by participants to compete the survey.

220

Figure 6.5. Revised model of airmanship

In general, the concepts selected for use in the survey were the most frequently

mentioned in each domain. As noted earlier, the exceptions were the technical skills

and personal attributes concepts. It is possible that slightly different choices

regarding the included concepts may have led to different findings for relatedness

and thus a different pattern in the MDS plots.

A further study could use one representative concept used in this study from

each domain (as anchor points for comparison) with a different selection of other

concepts from each domain to see if the basic structure determined in this study is

replicable and not dependent only on the specific concepts selected here.

221

Chapter 7. Airmanship Conclusions

The current project has involved four different investigations into the

understanding of airmanship among Australian aviators and the way in which

airmanship is trained in Australia. The starting point for the investigation was formed

by the models of airmanship found in the aviation literature. These are the Kern

(1996, 2009a), the Ebbage and Spencer (2003) and Edwards (2013) models of

airmanship.

The Kern model of airmanship (1996, 2009a) appears frequently in discussion

of airmanship among aviators. This model was developed from research with

aviators in the USA and biographies of noted aviators. Kern (1996) presented his

model of airmanship as an aide for learning in a textbook for ab-initio trainees, but

also outlined some ideas about the training of airmanship and made the suggestion

that ongoing self-assessment and striving for improved performance is all part of the

armoury of a good aviator. One short-fall of the Kern model is that it leaves out, or

hides amid knowledge items, some of the non-technical skills that become important

as an aviation career progresses. In addition, while noting the influence of context on

development and exhibition of airmanship, context is not included within the model

itself. The Ebbage and Spencer (2003) model of airmanship includes the non-

technical skills more overtly; some that Kern had included in the ‘knowledge of

team’ part of his model, plus some additional concepts relating to crew coordination

and self-assessment and improvement. The Ebbage and Spencer model was

developed in the UK based on the training of military pilots in Britain. The Edwards

(2013) definition of airmanship is concerned with the knowledge, skills and attitudes

associated with the three areas of technical, operational and non-technical elements

of aviation activity, and also discusses some other factors, including stress, fatigue

and managing automation. Presented as a guide book for pilots, this presentation

appears to be based on Edwards’ own experience and on a series of case studies of

aircraft accidents.

None of the above models presents any empirical evidence for the structure of

the models nor have they been formally evaluated. While there has been considerable

research into the components of airmanship, there has been little published research

222

on the overall concept of airmanship, nor any investigation into the definition and

training of airmanship in the Australian aviation industry.

In the current project, an initial on-line survey reached mainly general aviation

(GA) pilots. This provided a starting point for considering the match between the

existing models and the views of Australian aviators. Recognition that the survey had

only managed to reach a narrow sector of currently operating pilots led to the second

survey. The wider distribution of this expanded survey allowed comparison of

differences in views between military and civilian background aviators, and a better

idea of the relative importance placed by the participants on various concepts that

make up airmanship. However, the numbers were low when divided across any

further differentiation of sub-sectors within the industry. The third study involved the

interviewing of a cross-section of flight instructors and trainers from civilian and

military sectors, at ab-initio and advanced levels of training. The interview method

allowed greater detail to be garnered in responses to questions about the definition

and training of airmanship. Together the first three studies triangulated on the views

of airmanship held by Australian pilots. A model of airmanship was developed from

the results of these studies, and was tested in a fourth study. In this study,

participants rated the ‘relatedness’ of pairs of concepts from the model. These data

were then used in a multi-dimensional scaling exercise, which confirmed the basic

structure of the model and indicated the influence of the domains of knowledge and

personal attributes on the other domains of the model.

This chapter summarises the main findings of each study and outlines some

directions for future investigation.

Survey 1

The results of the first survey were discussed in Chapter 2. The questionnaire

included some demographic questions about the kind of licence held and flying

experience (in years and in hours) in both civilian and military sectors. The main part

of the questionnaire consisted of open ended questions about the participants’

definition of airmanship and about their experience of airmanship during their

training and during subsequent flying activities. The 40 participants made mention of

many of the components of the Kern model, the most frequently mentioned being

situational awareness and rules and regulations. The latter are part of Kern’s

223

‘Knowledge of the environment’ component. It was interesting that only a few

participants made specific mention of discipline, Kern’s cornerstone to good

airmanship. However some participants did note that there should be no short cuts or

cutting corners – implying a discipline to conform to procedures and tasks in full.

Communication was the most frequently mentioned of the Ebbage and

Spencer/Edwards additional concepts. Safety was included in responses by more than

half the participants and was the most frequently mentioned concept overall. The

existing airmanship models mention safety as an outcome of good airmanship but

safety is not specifically included in the models. Additional concepts raised by

participants that did not appear in any of the models included various attitudes and

personal qualities, lookout, courtesy and common sense.

With regard to the training of airmanship, most of the participants indicated

that there was some form of inclusion of airmanship in their initial training, and

many stated that airmanship was discussed in their ground briefings during initial

training. Conversely, this can also be seen as more than a quarter of participants

indicating that airmanship was not included in their initial training. Nearly half of the

participants mentioned both aircraft handling and non-technical skills as being part of

airmanship instruction. However some participants omitted any mention of aircraft

handling skills as part of airmanship (30%) while a few participants (12.5%) did the

reverse, and mentioned only aircraft handling and not human factors or non-technical

skills, when indicating what was included in initial instruction in airmanship.

Many participants indicated that the predominant influence on the development

of their airmanship was the influence of others (44% of total mentions). This

conclusion was echoed by the instructors among the respondents, half of whom

reported that they try to provide a good example of airmanship, in order to instil

airmanship in their students.

The main limitations of this study were the low participation rate (with only 40

respondents), the predominance of General Aviation pilots and lack of many airline

or military participants. However, it provided information that was used to expand

the questionnaire for a second survey.

224

Survey 2

The details of the second survey were reported in Chapter Three. The

questionnaire developed in Survey 1 was expanded to include additional

demographic questions to identify participants’ main area of activity within the

aviation environment. A rating question and a ranking question were also added.

Each of these questions listed the concepts included in the airmanship models as well

as some of the additional concepts raised in the responses to the first survey.

Participants were asked to rate each concept and then to rank the five most important

concepts. Participants included both military and civilian trained pilots, currently

flying in airlines, in general aviation and in the military. They formed two main

groups, one of participants who had only civilian experience (40 participants), and

the other those who had a background in the military but were currently flying in

either military or civilian environments (43 participants).

Definition of Airmanship.

Analysis showed that when asked about the definition of airmanship there were

significant differences in the frequency of mention of some concepts between

civilian and military background participants. Among the Kern model concepts,

military background participants mentioned knowledge of the aircraft, rules and

regulations, organisational factors (such as procedures and orders), aircraft handling

skill and proficiency significantly more frequently than did civilian background

participants. Military background participants mentioned common sense significantly

more frequently than did civilian background participants, but civilian background

participants mentioned courtesy or consideration for others significantly more

frequently than did military background participants. There were other concepts on

which the two groups differed, but the actual number of mentions of each concept

was too low to calculate significance.

Overall, in the open ended questions about the definition of airmanship, the

most frequently mentioned concept was safety (by 74.7% of participants), followed

by situational awareness and judgement/decision making, personal qualities,

knowledge of the aircraft and rules and regulations. The responses to the rating and

ranking questions reflected much the same ordering of concepts, however with some

interesting differences. Situational awareness was rated and ranked as most

225

important, and significantly ahead of the next closest rated or ranked concepts. There

were no statistically significant differences among the top-most rated, or ranked,

concepts after situational awareness. Judgement/decision making was ranked second,

and the mean rating of the concept placed it fourth. Discipline was 13th most

frequently mentioned in the open ended questions, but was third in the mean ratings

and ranked sixth. Common sense was 10th most frequently mentioned, but rose to 6th

in the ratings and 4th in the rankings. Courtesy was mentioned as frequently as

common sense overall, but was more frequently mentioned by civilian respondents.

Courtesy had the lowest rating and was ranked quite low (coming in 17th). Courtesy

was included in the “top 5” by only six respondents. Comments about preparation,

coded as mission planning in the open ended questions, were 17th in order of

frequency of mention, but preparation was rated and ranked higher up at 5th.

Communication was 8th in order of frequency of mention, but dropped to a mean

rating of 18th and mean ranking was 15th.

The changes in ordering may simply reflect the difference between free recall

and recognition, in that while some participants may not have mentioned some of the

concepts, once prompted by their presence in the supplied list they worked with the

additional concepts and ascribed importance to them regardless of whether they had

been mentioned earlier. While the difference in rating and rankings was statistically

significant across the range of 26 concepts, the degree of difference between closely

rated or ranked concepts is not significant.

The elements of the various models were well represented in the open ended

questions and performed well in the rating and ranking questions, with some aspects

being more highly rated and ranked than the frequency of mention in the open ended

questions may have appeared to predict, for example, discipline and

preparation/mission knowledge. Situational awareness is clearly seen as the most

important feature of good airmanship, being mentioned most frequently (after safety)

and rated and ranked most highly.

Training and Development of Airmanship.

In the analysis of the questions about training and development of airmanship,

the participants were separated into two groups based on their sector of initial

training (civilian or military). For those who were trained first as a civilian and then

226

in the military, if it was obvious they were writing about their military training they

were added to that group. If it was ambiguous, they were omitted from the analysis.

Participants were also separated by the era of their training (‘before 1991’ or ‘1991

and after’) to reflect the introduction of human performance and limitations to the

theory syllabus. This resulted in rather uneven groups, as there were only six military

participants who began their initial training in 1991or later.

The majority of participants indicated that airmanship or airmanship-related

concepts were included in their ground training or briefings, with no significant

difference between eras or training background. They then listed the concepts that

were discussed, with safety being most frequently mentioned, followed by situational

awareness, with no significant difference in frequency of mention between groups.

Decision making was significantly more frequently mentioned by military trained

participants. There was mention of knowledge, most often to do with rules and

regulations or procedures. Communication was more frequently mentioned by

civilian trained participants, however the number of mentions by military trained

participants was too low to allow a statistical comparison. Lookout and courtesy

were among the concepts most frequently mentioned by both civilian and military

trained participants.

When asked about inclusion of airmanship in their in-air flight training, slightly

more of the civilian participants trained before 1991 noted that airmanship was not

included, than did participants in the other groups, and the military participants

reported inclusion of airmanship significantly more frequently than did the civilian

participants. A variety of concepts were listed, most frequently situational awareness,

but not as many concepts as were mentioned in the previous question. Lookout was

more frequently mentioned by the military trained participants, the difference was

not quite significant, but had a low-medium effect size. Courtesy and safety were

only mentioned by civilian participants, and performance during emergency or

abnormal events and examinations were only mentioned by military participants.

Five participants related that airmanship was not taught, that it was gained by

experience or picked up as one progressed. A further four participants noted that

airmanship was not well defined or not a specific part of the training.

There was little mention of team related concepts or inter-personal skills

(except for the mention of courtesy). These are more related to multi-crew flying, so

227

their omission from mention when discussing ab-initio training is logical. The

participants did not mention self-improvement or self-assessment, nor was there

frequent mention of “knowledge of self” concepts apart from some comment on

being aware of one’s own limitations.

The participants were asked about the development of airmanship as they

progressed as pilots, and it is in response to this question that concepts related to self-

development arose more frequently than previously. Kern (1996) argues that ongoing

reflection on one’s own performance is an important part of maintaining airmanship,

and it appears that some of the participants in this survey agree. Experience was most

frequently mentioned in response to this question, and mentioned significantly more

frequently by military background participants than by civilian background

participants. Experience was generally mentioned in combination with other

concepts. Civilian participants slightly (not significantly) more frequently mentioned

various personal qualities reflecting an internal motivation such as self-development

or wanting to be “the best’. Third most frequently cited as a factor in the

development of airmanship was the example provided by others (whether good or

bad).

Current training. A question about current training participation showed no

significant differences in frequency of mention of specific concepts, between the

three groupings used for analysis of this question (current or most recent activity in

airlines, general aviation or military). Most participants reported engagement in on-

going training, and that airmanship as such is not generally included. However, some

noted that there is recognition that “airmanship” is an integral part of aviation

activity and is thus not separately addressed. Many of the respondents in airlines and

the military noted the relationship between crew resource management and human

factors training and airmanship. In addition, the lack of formal definition and

subjective assessment of airmanship was commented on by some airline pilots.

There were some minority views expressed, such as airmanship being all about

aircraft handling, or that it is equivalent to common sense, or that it cannot be taught

but depends upon experience.

Instructor practices. Respondents to this question were grouped by area of

current activity: airline, general aviation (GA) or military. Responses included

comment on how they impart airmanship and also about what it is that they are

228

concerned with developing. The airline and GA instructors most frequently

mentioned briefings and discussions as the mode of instruction, while the military

instructors mentioned provision of a good example most frequently, followed by

mention of the use of scenarios and then briefings; however there were no significant

differences between the groups. Airline instructors also wrote about the use of

scenarios in the training environment to cover a broad range of operations. There

were some comments about observing how the student handles events and when to

intervene should the situation deteriorate.

What is included in training of airmanship covered many of the familiar

concepts represented in the airmanship models, and some additional concepts. Safety

was most frequently mentioned, along with getting students to think, develop

decisions, and maintain situational awareness, as well as various aspects of

knowledge.

None of the respondents clearly articulated all three stages of development of

airmanship as presented in Kern (1996) or Ebbage and Spencer (2003) however the

components are there in many of the responses. The three stages are to: (a) explain

the importance of airmanship; (b) model and teach good airmanship; and (c) evaluate

performance and provide feedback. Few mentions were made about evaluation and

feedback, although there was a notable exception among the military instructors who

advised failing the students who did not grasp the importance of adherence to the

safe principles of flight.

Changes over time. The majority of respondents agreed that their ideas about

airmanship had changed over time or made some other comment that indicated a

maturing or development of airmanship. Only about a third of respondents indicated

that their ideas had not changed, and there were no significant differences across

respondent groupings. There was a variety of views expressed in response to the

questions, with some participants taking advantage of the survey to voice some

cherished complaints about the industry (regarding modern techniques and young

inexperienced instructors), while others were more positive. Change was mostly

ascribed to experience, to the introduction of CRM and HF training or to the

examples of others. Many respondents simply commented on general development,

increased awareness or developing a sense of the importance of airmanship, without

referring to any mechanism through which this might have occurred.

229

Concluding Comment

The majority of the participants undertook their ab-initio training before 1991,

so asking about their early training provides information that has to be considered

time-related. Of those respondents who were current instructors, not all answered the

question about current training practices, nor was it easy to ascertain whether they

were referring to ab-initio or more advanced training in that question. The inclusion

of airmanship in current training and the way in which this is carried out, may be

better considered by a more direct questioning approach, which was addressed in the

next part of the project: the interviewing of current instructors in military and civilian

sectors, at ab-initio and advanced levels of training.

Interviews with Current Instructors

Definition of Airmanship

The third study within the current project aimed to directly address issues

regarding the definition and training of airmanship with the people who are most

concerned with the initial training and ongoing development of Australia’s pilots: the

instructors employed in ab-initio and advanced training in civilian and military areas

of operation. While the participants in the two on-line surveys included instructors,

the questions about training were towards the end of the survey and resulted in a lack

of response or very brief response. Face-to-face interviews can elicit more detailed

responses to targeted questions about current practice. A series of interviews were

carried out with flight instructors at two civilian flying schools, at the basic flight

training school (BFTS) for the military with both military and civilian background

instructors, at a jet training conversion unit at Williamtown and at the military

Central Flying School (CFS) at East Sale, as well as among training captains

recruited from major Australian airlines. Thus the interviewees formed five groups:

civilian ab-initio trainers; civilian advanced trainers; a group with civilian

background currently employed to train in the military (civilian at BFTS); military

trainers at BFTS and military advanced trainers (at CFS and Williamtown).

Process comments. There was comment about the process of defining

airmanship, with many of the military and civilian advanced instructors saying that

airmanship is hard to define and all of the BFTS instructors and more than half of the

advanced instructors (military and civilian) indicating that airmanship is all-

230

encompassing or an integration of many skills. About a quarter of the participants

added that airmanship is unspecifiable in some way, being subjective, holistic,

esoteric or unquantifiable.

Initial definition. Among the 39 interviews, there were 25 different

combinations of concepts included in the initial definition of airmanship. There were

12 mentions of situational awareness among the various combinations, nine that were

based on the formal military definition (of “safe and efficient flight both in the air

and on the ground’), nine mentions of common sense and seven of decision making.

Model related concepts. The Kern model was mentioned by two participants

(both civilian advanced instructors) in response to the question about the definition

of airmanship, but neither based their definition on that model. A third participant

mentioned the model later in the interview, as not being very helpful. Despite this

lack of recognition, most of the concepts included in the Kern (1996, 2009a), Ebbage

and Spencer (2003) and Edwards (2013) models were included in the discussions of

airmanship by participants, although not all of them by any one participant.

Essentially, all participants agreed that situational awareness and decision making

were important to airmanship. Knowledge was also considered important, with

general comments made about knowledge requirements and knowledge of rules and

regulations, legislative environment, being the most frequently mentioned knowledge

related concept. Aspects of teamwork and crew related issues were also discussed.

Of these concepts, communication was most frequently mentioned within each

group.

Skill was a term included in one of the questions and the majority of

participants made some mention of technical flying skill in relation to airmanship.

Most of the instructors at BFTS (both civilian and military) specifically noted that

technical flying skill and airmanship were distinct, despite earlier noting that all the

concepts were interrelated. About half of the remaining participants included flying

skill as part of airmanship. Discipline was not widely mentioned as such, with overall

just over a third of participants making mention of discipline. However, if comments

that implied a need for discipline are included in the count, then two thirds of

participants mentioned or implied a need for discipline.

231

Additional concepts. The formal military definition of airmanship as “safe and

efficient flight, both in the air and on the ground” was used by the majority of the

military BFTS instructors and by half of the military advanced instructors. There

were also a few mentions of this definition by instructors in the other groups. It is a

useful short and snappy definition but one still needs to ask what makes a flight safe

and efficient. Safety was mentioned as part of airmanship by three-quarters of the

participants, most often by the ab-intio instructors. Personal qualities were also

frequently mentioned but this may be somewhat misleading as many different

qualities were coded within this node, such as honesty, integrity and commitment,

and also distractibility, complacency and over-confidence were mentioned as not

being conducive to good airmanship.

Preparation is assessed in the military training context and thus was mentioned

more frequently by military instructors (including the civilian BFTS group).

Experience was raised as important in airmanship by 61.5% of participants overall,

but much more frequently by military than civilian instructors. Attitudes were

mentioned, often in response to a specific question about “other skills, knowledge or

attitudes” that might be included in airmanship. Prioritising was also mentioned

across the instructor groups. Common sense was most frequently mentioned by

civilian advanced level instructors, with fairly evenly spread of mentions across the

other instructor groups.

Other concepts that were mentioned included: thinking, captaincy or command,

courtesy, foresight, threat and error management (TEM) and CRM, responsibility,

and information gathering. Participants also mentioned a number of actions or

activities that demonstrate good airmanship. These included: checks, lookout, radio

usage, weather awareness, and pre-flight inspections, amongst other things.

Group similarities and differences. There was consistency among the

participants concerning the importance of situational awareness and decision making,

and also knowledge of rules and regulations and operating procedures. Many

participants also referred to aspects of personality, which were coded as personal

qualities. There was a lack of mention of “knowledge of the team” across all groups

of participants, although skills relating to team-work were mentioned. The civilian

ab-initio group of instructors made more mentions of mission planning, self-

improvement and courtesy than did instructors from other groups. The civilian

232

advanced instructors made more comments about skills related to multi-crew

operations including communication, crew coordination, interpersonal skills,

management of the flight deck, team leadership and threat and error management.

They also commented most on thinking, attitudes and common sense. They made

fewest mentions of aircraft knowledge and mission planning, perhaps because of the

simulator environment in which they work.

Each of the instructor groups tended to comment most about aspects of

airmanship most relevant to their operating environment, thus the civilian advanced

group spoke about multi-crew skills, the BFTS groups about preparation and

discipline, and the civilian ab-initio group about planning and courtesy.

Comparison with survey results. The ordering of concepts by frequency of

mention for the interview data and the results of the second survey led to very similar

lists (see Table 4.8). There are some differences in ordering, but only by a few places

either way. Nineteen of the top 25 concepts appeared on both lists, those concepts

appearing in the “top 25” of only one list, appeared lower down the rankings in the

other list. Situational awareness, decision making and safety were at the top of both

lists. The “personal qualities” node was high on both lists, the node included many

kinds of qualities and the number of comments may indicate a concern about the type

of person an aviator is. Near the top of both were comments about the legislative and

organisational environment (rules and regulations and operating procedures).

The consistency across the two data collection methods and sets of questions

used indicates the robust nature of the idea of airmanship among Australian aviators.

Comparison with the models of airmanship. The main concepts included in

the Kern model of airmanship were mentioned frequently by the participants. The

interview data also suggests that skills relevant to multi-crew environments are

important aspects of airmanship for aviators in that more advanced environment, but

these skills were not at the forefront of the Kern model. Communication, especially,

was mentioned frequently by participants from all groups, but in the Kern model is

simply a part of “knowledge of the team’. There was also a sense that the application

of knowledge is important, knowing when and how to use relevant knowledge was

emphasised by some participants. Discipline was expressed as compliance with rules,

regulations and procedures, and was brought up rather more frequently than in the

233

open-ended survey questions. There was some confusion over whether technical

flying skills are part of airmanship. In terms of an overall discussion, most

participants included flying skills as part of a global integrated view of airmanship.

However, military instructors (and civilians working in the military BFTS

environment) later excluded technical flying skills from airmanship, in the sense that

they assess airmanship and flying skills separately, and from that position, define

airmanship as everything else except flying the aircraft.

Conclusion – definition of airmanship. The interviews indicated that while

there are many differences in initial definition of airmanship, after some discussion,

the same main themes emerged as important components. Not all of these main

themes are well represented by the Kern model alone.

Interviews: Training and Development of Airmanship

Having covered the definition of airmanship, the interviews then addressed the

way in which airmanship is trained and developed at the different levels of

instruction represented by the participants.

Airmanship in the training program structure.

The most interesting finding was that just over half of the advanced civilian

and advanced military instructors stated that at that level, airmanship is assumed to

be adequate and so little additional attention is required. The majority of instructors

at the ab-initio level stated that they talk about airmanship in the pre-flight briefing,

and that the briefing is a standard format. In civilian advanced training, it was noted

that the term airmanship is not used, rather “non-technical skills” is used. The

military advanced instructors were divided: the jet-conversion instructors responded

in a similar fashion to the ab-initio instructors, while among the instructors at CFS

the general opinion seemed to be that not much time was spent on development of

airmanship, rather emphasis was placed on how to present a good example to future

students.

Learning from others, providing a good example to others and ensuring there is

a positive supportive organisational culture was most often mentioned by the civilian

advanced participants, and also by the BFTS instructors and military advanced

participants, and not so often by the civilian ab-initio group.

234

Initial airmanship training. After noting that airmanship is introduced from

the first pre-flight briefs, the participants from the different ab-initio instructor

groups all talked about lookout being the most important initial airmanship related

skill to be taught. They also mentioned a variety of different aspects of airmanship

that are introduced in the first few lessons. Most frequently and fairly consistently

mentioned across groups were: listen out, or radio usage; checks and checklists; how

to taxi; and knowing aircraft parameters and limitations. Some of the differences

between the groups seem to be mainly in the terminology used by civilian and

military groups, such as: flight planning versus preparation and procedures versus

work cycles.

Assessment of airmanship. In the civilian ab-initio context, airmanship is only

informally assessed until the formal flight test for a licence, and only insofar as

elements of airmanship are included in the competencies assessed. Several of the

civilian advanced-level instructors stated that they do not use the term airmanship,

rather assessment is divided into technical and non-technical skills. Non-technical

skills include situational awareness, decision making, communication, leadership and

automation management, and technical skills cover aircraft handling and automation

management skills. However it was explained that pilots are not failed on non-

technical skills alone, but only if such problems are associated with a technical

handling skill problem.

By contrast, the military related groups (BFTS civilian and military

participants and the non-CFS military advanced participants) were clearer about

assessment of airmanship as a separate fail-able item and mentioned the word-

pictures that form a rubric for assessment. However, it should again be noted here

that the military definition of airmanship for assessment purposes is everything

except preparation and aircraft handling – so essentially consisting of non-technical

skills. The military advanced trainers at CFS were reluctant to say that they assess

the airmanship of their flight instructor candidates (FICs), noting their experience

and existing high levels of airmanship. It was clear that they separated the technical

handling skills from “airmanship” in that the FICs have to relearn the specific

technical handling requirements that CFS include in the basic flying syllabus, but

that airmanship (excluding flying skills) is assumed to be fine and discussion relates

more to how the FIC will imbue their future students with this aspect of airmanship.

235

Training process. Among the civilian advanced and civilian BFTS instructors

comment was made that that the instructor’s role is to correct students and suggest

fixes for any problems they have in the air. Many of the participants indicated that

instructors should demonstrate exemplary airmanship themselves and must be ready

to take over from their students if necessary. There was concern expressed across all

groups of instructors about how long to allow students to continue along an error

path before stepping in and taking over.

Instructor training. Participants mentioned that airmanship is not explicitly

included in the civilian instructor rating, candidates simply have to exhibit good

airmanship during the instructor rating flight test. The military BFTS instructors did

not recall that airmanship was a major part of the CFS instructor training program

and the military advanced instructors at CFS agreed with this impression. Their

position is that flight instructor candidates already have good airmanship and to have

it as part of the training is unnecessary especially as there is limited time available in

the program. Again this is taking the view that airmanship excludes technical flying

skills, which are focussed on in the CFS program.

Comparison with model recommendations. The components of instruction

in airmanship revealed by the participants do not entirely reflect the three phase

process for instruction of airmanship presented in the Kern (1996) and Ebbage and

Spencer (2003) models of airmanship. The first step in each model is to instil the

importance of airmanship. It is not clear whether this is addressed as such either at

ab-initio or advanced levels of instruction in civilian or military environments.

The second phase of the suggested process of training airmanship specifies the

teaching and modelling of airmanship. The ab-initio civilian and military programs

include this stage, with airmanship (and technical skills) being included in briefings

and debriefs, and with many of the instructors noting their intent to provide a good

example. The comments by at least half the advanced level instructors that

airmanship is “assumed” at their level of training indicates that perhaps the active

inculcation of airmanship does not occur at this level. Other civilian instructors at

this level stated that technical aircraft handling and non-technical skills are instructed

and assessed, while the military advanced CFS instructors focussed more on how to

instruct future students rather than improvement of the FIC’s airmanship.

236

The third phase of the process is assessment and feedback. The military BFTS

and advanced conversion programs (non-CFS) provide feedback in debriefs and

include formal assessment of preparation, technical flying skill and what they call

airmanship (everything else, or non-technical skills) which can be failed, thus

meeting Kern’s (1996) requirement of aggressive evaluation. However the CFS

program for development of future instructors did not appear to have quite so overt

an assessment process for airmanship, being concerned more with instructional

abilities. Civilian ab-initio programs, while they may not have an internal pass/fail

evaluation process, do ensure that their students meet CASA requirements for

technical and non-technical skill, which are assessed in licence flight tests. At the

advanced civilian level, it appears that airmanship is assessed, but that poor non-

technical skills are only critical (for failure) if paired with a technical skill problem.

Conclusion – training and development.

It was interesting, because unexpected, that when asked about training of

airmanship the initial reaction of about half of the advanced level instructors was to

state that airmanship is assumed to already have been acquired, or already exists at a

high standard. Further discussion indicated that civilian advanced training covers

technical and non-technical skills, but that neither is referred to as “airmanship”.

There were some similarities in the process of instilling airmanship discussed

by the ab-initio instructors in military and civilian environments. Commonalities

included the use of standardised briefings and the skills that are introduced in initial

lessons. Structured ground school instruction and formal assessment of flying

performance were features of the military program that were not reflected in the

civilian programs (of the flying schools sampled). Participants from all instructor

groups were concerned about providing a good example to students and about when

to intervene when errors occurred.

The three phase training structures proposed by Kern (1996) and Ebbage and

Spencer (2003) were not fully reflected in the comments made by participants. The

military BFTS program is closest, having specific training and evaluative assessment

of both technical and non-technical skills. It does not appear common to use

airmanship as a structure to relate knowledge acquisition, skill development and

237

personal attributes to ongoing training and flying efficiently and safely, as Kern

(1996) proposed.

The Structure of Airmanship

A discussion of the similarities and differences between the findings in the first

three studies and the research reported in the literature that takes an over-view of

pilot performance is presented in Chapter 5. The concepts raised by the participants

were categorised into five principle domains of airmanship: knowledge, preparation,

non-technical skills, technical skills and personal attributes. These form a model of

airmanship that also acknowledges that airmanship occurs in a context of experience,

organisation and regulation.

Airmanship Concept Domain Testing

An examination of whether Australian aviators consider airmanship concepts

relate to each other in terms of the airmanship model is reported in Chapter 6. Pairs

of representative airmanship concepts formed a 125 item questionnaire, in which

participants were asked to rate the relatedness of each pair of concepts on a 5-point

Likert scale. Participants were 87 Australian (or Australian based) pilots. Overall,

pairs of concepts from the same domain (within-domain pairs) were rated as more

closely related than pairs of concepts from different domains (between-domain

pairs). The pairs formed with the knowledge domain concepts did not fit this pattern.

A multi-dimensional scaling analysis using the relatedness data indicated, in a 3-

dimensional solution, that concepts from the domains of preparation, non-technical

skills, technical skills, and personal qualities clustered together. The knowledge

concepts were separated, with aircraft type knowledge falling close to the technical

skills concepts and the other knowledge concepts (knowing SOPs and knowledge of

rules and regulations) appearing closest to the personal attribute concept of

discipline.

The relatedness of the concepts was interpreted as showing the functional

approach taken by the participants, looking at how each pair of concepts might relate

to flying tasks. The model presented in the previous chapter was reworked slightly,

not to change the domains themselves, but to indicate that the three main ‘activity-

238

related’ domains of airmanship, preparation, non-technical skills and technical skills,

are influenced by the underlying knowledge and personal attributes of the pilot. The

rearranged model was presented in Figure 6.5.

Definition of Airmanship

The first aim of the present research was to redress the lack of empirical

evidence regarding aviators’ views of airmanship and to develop an overview of

what it is that Australian aviators consider “airmanship” to be. The surveys and

interviews carried out during this research have provided considerable evidence of

Australian pilots’ ideas about airmanship.

The outcome of the present research is that in the general sense airmanship is a

fairly loosely defined term that refers to the ability to combine together all the skills,

attributes and knowledge areas required to fly an aircraft safely, efficiently and

effectively. The present research has demonstrated that Australian aviators include a

diverse selection of skills and attributes when asked to define airmanship, but that

there are some strong common concepts that were raised repeatedly. The findings of

the current research were used to support the development of a model of airmanship.

This model was then tested and found to be a reasonable representation of how

Australian aviators consider component concepts of airmanship to be related. The

model includes three main domains of airmanship activity, namely preparation, non-

technical skills and technical skills. These domains are influenced by knowledge and

personal attributes. The model also allows for the exercise of airmanship to be

considered within a context, aspects of which include personal experience and

organisational and regulatory influences.

The model of airmanship presented in Chapter 6 (the ‘Carrick Model’ for ease

of reference) groups concepts into representative domains, rather than trying to

specify everything. The Kern model (1996, 2009a) tries to define and include every

aspect of knowledge, leading to knowledge visually dominating the model design.

The ‘Carrick model’ groups personal attributes into one domain, in contrast to both

Kern and Ebbage and Spencer (2003) who both concentrate on discipline to the

exclusion of other attributes. This research did not find such an emphasis on

discipline, although it was considered important when suggested to participants in

239

the survey rating and ranking questions. Personal attributes have been included by

Edwards (2013), although called ‘attitudes’, in each of the elements of his model of

airmanship.

The ‘Carrick Model’ tries to form a balance between the major domains of

preparation, technical and non-technical skills without separately specifying certain

skills at the expense of others. For example, the Kern model places situational

awareness and judgement at the peak of the model as capstone outcomes, and other

non-technical skills such as communication and crew coordination are relegated to a

knowledge column. The Edwards (2013) model also has three main elements

including technical and non-technical skills. Edwards’ third element of ‘operational

elements’ is similar in content to the ‘preparation’ domain of the ‘Carrick model’, in

covering planning, but suggests that knowledge of other areas of operations, such as

ATC, should be acquired.

Mavin and Roth (2014) separate out the non-technical skills of situational

awareness, decision making, communication and management as separate elements

of their model, and also include as elements technical skills (as flying within

tolerances) and aviation knowledge. They do not include personal attributes as part

of their model, nor do they appear concerned with preparation and planning. Their

model is intended for specific use in the assessment of pilots within airlines

undertaking upgrade training (Mavin, 2010).

Nergård et al (2011) does not formulate a model of airmanship, but does

present the outcome of a principal components analysis based on responses to the

questions that his participants in an earlier study helped to frame. Some of these are

phrased as having to do with a ‘good pilot’ and others relate to ‘crew’. The four

factors are labelled: ‘principles in airmanship knowledge’ (although this includes

items relating to initiative, role modelling and monitoring for stress and fatigue);

self-awareness (which includes further personal attribute items); flying skills; and

CRM (covering non-technical skills). Nergård essentially covers four of the domains

that the current research found, (although his ‘knowledge’ factor included many non-

knowledge items), however does not consider preparation, and neither is context

considered.

240

The concepts that were raised by the participants in the current research were

generally the same concepts that have been considered in research and theory

throughout the literature, especially the non-technical skills that have become well

established in the industry through CRM courses and the HPL syllabus and listed as

essential for good threat and error management (Helmreich, 2002; Thomas, 2005). In

addition to these concepts, two ‘new’ concepts not mentioned in other research were

found: common sense and courtesy. Courtesy was mostly raised by pilots who need

to interact with other airspace users in unstructured environments (i.e. general

aviation) and these pilots may have more need to moderate their communication and

maintain working relationships outside their immediate crew or organisation.

Courtesy was generally mentioned in the context of communication or helping

others. Common-sense tended to be raised by more experienced pilots, not always in

a context that led to easy interpretation. This researcher’s interpretation is that these

pilots have been operating in aviation for so long that airmanship has become

ingrained, they no longer remember having to learn it all, and thus it becomes

‘common sense’ to them. It is not, however, a helpful term to use with less

experienced pilots, (neither is saying that airmanship depends on experience) as such

terms may be interpreted to mean that airmanship being something that will develop

naturally with time rather than something that can be actively learned.

The current research has demonstrated the awareness of airmanship among

Australian aviators and the variety of concepts that they feel are related to it. A

model of airmanship that better represents the concepts of airmanship and their

interrelatedness has been developed, presented and tested.

Training of Airmanship

The second aim of the current research was to investigate whether and how

airmanship is taught. Both Kern (1996) and Ebbage and Spencer (2003) suggest a

three phase approach to ensuring that good airmanship develops in pilots under

instruction. First, explain and promote the concept of airmanship; secondly, instruct

and model airmanship; and thirdly, evaluate and assess airmanship, and provide

feedback to the student or trainee.

241

Some of the participants in the survey indicated that airmanship was not part of

their initial instruction, but the majority indicated that it was. All of the ab-initio

instructors interviewed indicated that they did include airmanship in their briefings

before flights, instructed and demonstrated airmanship concepts in the air and

expected airmanship to be displayed by their students. It was interesting to repeatedly

hear from aviators at more advanced levels of training, that airmanship itself is

“assumed at this level”. That is, if the aviator does not have good airmanship he or

she would not have made it (somehow) to advanced levels of training. Despite that

statement, both non-technical skills and technical handling aspects of flying make up

the training process at advanced levels. At the ab-initio level, the instructors stated

that they ensure that airmanship is briefed, instructed and is well practiced, and also

they are actively aware of providing a good example. Most the components of

airmanship appear to be addressed in current training programs (apart from, perhaps,

the personal qualities raised by some participants), and what is introduced is

consistent across military and civilian ab-initio training. Thus the second phase of the

recommended sequence is currently being met.

Assessment of airmanship varied in intensity between the military and civilian

instructors at the ab-initio level. The military programs allow for regular assessment

and the possibility of failing, whereas this appeared to occur only at the stage of the

licence flight test for civilian trainees. In both sectors and at both ab-initio and

advanced levels, feedback is provided in post flight debriefing. This goes some way

towards meeting the third stage of the suggested sequence, although the civilian

process may not meet the suggestion that assessment be evaluated aggressively

(Kern, 1996).

The first stage in the instructional process is the presentation of the importance

of airmanship. This may well occur, but there does not appear to be any widespread

overt approach to using a model or structure that attempts to explain airmanship

conceptually as well as practically. The use of such a structure would provide the

trainee with a framework on which to build skills, knowledge and practices as his or

her career evolves. Only one respondent mentioned doing this, using the Kern model

for the purpose. The model of airmanship presented in Chapter 6 may provide an

alternative structure for this purpose.

242

Limitations of the Current Research

The first survey was limited by the low numbers of participants and their

homogeneity, as most were general aviation pilots. The second survey redressed this

issue in that there were more participants and this included general aviation, airline

and military pilots. However most of the participants began their training before

1991, especially those with a military background. This was an advantage in that the

participants were all very experienced, but does not give an up to date picture of

airmanship training, or how more recent training might influence the definition of

airmanship. The nature of the survey meant that it was difficult to undertake

statistical comparisons between groups, because the frequency of mention of some

concepts was too low. However, qualitative evaluation of the comments made by the

participants did allow a picture of the general structure of airmanship to be

determined. As with all such surveys, there is a possibility that the people who chose

to respond are not necessarily representative of the population of pilots, in this case

being perhaps more open to participating in research run from a School of

Psychology.

The third study in the present research was to interview a number of current

flight instructors and check and training captains. Again, data did not allow for

quantitative analysis, but there was more than sufficient information for a qualitative

assessment of the research questions. The civilian participants may not have been

representative of the entire civilian industry as it was difficult to access participants

from a wide range of organisations. In hindsight, it would have been interesting to

have administered the rating and ranking questions from the second survey, to the

interviewees in the third study. This would have allowed a more quantitative

comparison to be made between the groups within the interview study and between

the interview and survey participants.

The final survey to test the structure of the ‘Carrick Model’ of airmanship also

had a somewhat limited number of participants. However, this was adequate for the

statistical analyses employed.

Future Research

The revised model of airmanship developed from the present research could be

further tested for validity, one method of doing this would be to consult a selection of

243

expert aviators about the relevance and applicability of the revised model. In addition

the acceptability of the revised model among aviators in general could be assessed.

This would take the form of asking pilots about how appropriate the revised model is

as a representation of airmanship and about their preference between the revised

model and existing models. Questions could also be asked about how useful they

think the revised model might be for self-development and for use in flight training.

Assuming that the response to the revised model of airmanship is positive, it

could then be promoted for use in flight instruction. The model provides a structure

that would encourage the conceptual development and integration of airmanship

among flight students as they develop their flying skills and study for theory

examinations.

One possible line of future research could be a cross-cultural comparison of

concepts of airmanship. The revised model presented here is based on comment from

Australian pilots. Existing models were developed in the USA and the UK. Pilots

from other cultures may have different views on airmanship and it would be useful to

compare the results of surveys run in other countries with the Australian findings.

Future research could use a similar approach using on-line surveys with a

combination of open-ended and rating and ranking questions, to determine concept

structures around professionalism in other high-risk/high-reliability environments.

Concepts similar to those that make up airmanship are being introduced into aircraft

maintenance as “maintenance professionalism”, but apparently without any

theoretical or conceptual structure. An investigation of aircraft maintainers’ thoughts

regarding maintenance professionalism could result in a useful model. Airmanship

parallels the idea of seamanship, although there are obvious differences between the

two environments. Determining the main concepts that define seamanship would

highlight similarities and differences in the expression of professionalism in a marine

environment compared with aviation.

Conclusion

The aim of this research was to examine what Australian aviators consider to

be the definition of airmanship and whether there are any differences between

military and civilian pilots, or between military and civilian instructors at ab-initio

and advanced levels of training. There were few differences in frequency of mention

244

of concepts between military and civilian pilot participants in the survey study. There

were some differences in the frequency of concepts mentioned, between the flight

instructors who were interviewed, but numbers were too low to conduct statistical

tests. The civilian advanced level instructors made more mention than did other

instructors, of concepts relating to multi-crew environments, such as communication,

crew coordination, leadership and management of the flight-deck. The ab-initio

instructors at BFTS (both military and civilian) made more mentions of safety,

captaincy and the application of knowledge, than did the other instructors.

Differences are related more to sphere of usual operation than to military or civilian

background.

The first three studies led to the proposal of a model of airmanship based on

the concepts that were raised by participants when asked about the definition of

airmanship. The model presents concepts grouped into domains of knowledge,

preparation, non-technical skills, technical skills and personal attributes, and also

makes note of the effect of the context in which airmanship is developed and

expressed. A further study confirmed the domains and also indicated the influence of

knowledge and personal attributes on the other three domains. The model was

revised slightly to reflect this influence. The use of the revised model to expressly

embed airmanship in training programs may alleviate the paradox of the use of the

term ‘airmanship’ both as a global expression of safe and efficient flight and also to

describe only the non-technical skills component, in some training environments.

A second intent of this research was to investigate the extent to which

airmanship is included in civilian and military aviation training in Australia, at ab-

initio and at more advanced levels of training. At ab-initio level, the kinds of

airmanship skills that are first introduced were very similar between civilian and

military instructors, and both groups noted the use of standard briefings that include

airmanship pointers. The military training program considers the assessment of

airmanship to cover non-technical skills, as distinct from preparation and technical,

aircraft handling skills. This was not so clear cut among the civilian instructors, and

it appeared that little formal assessment is made, except insofar as the student is

prepared for a flight test. The main difference between the two levels of training is

that about half of the advanced level instructors stated that airmanship is assumed at

that level. The advanced level instructors went on to describe training that, among

245

the civilians included both non-technical and technical skills, and among the military

instructors at the Central Flying School, concentrated on technical skills. At the jet

conversion unit, both non-technical (airmanship) and technical skills were addressed.

Only one of the instructors interviewed in the third study noted the use of a model to

explain the structure of airmanship and how their training related to the development

of such skills. It is suggested therefore that it may be worthwhile to introduce the

model of airmanship developed from the present research to flying instructors and

trainers as a useful tool for that purpose. As eluded to above, this might also solve

the problem of military instructors use of airmanship both as a global all-inclusive

term and also to refer to non-technical skills when assessing performance.

There are a variety of alternatives for the continuation of this line of research.

The revised model of airmanship requires further investigation to establish that it is

representative and useful. Looking at professionalism in other sectors of the aviation

industry or in other environments may draw useful parallels, as well as providing

potentially useful models of professionalism for specific industries.

246

247

References

Ali, S. F., Gluckenberger, D., Rossi, M. & Williams, M. (2000). Evaluation of above

real-time training and self-instructional strategies for airmanship tasks on a

flight simulator. USAF Research Laboratory Report ADA 391561.

Downloaded from:

http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=A

DA391561

Allen, P. & Bennett, K. (2010). PASW statistics by SPSS, a practical guide.

Melbourne Aus: Cengage Learning.

BAE Systems. (2006). Airmanship ASH 01 Intro. Power point training aid,

Tamworth, Australia: BAE Systems.

Bartholomew, D.J., Steele, F., Moustaki, I. & Galbraith, J.I. (2002). The analysis and

interpretation of multivariate data for social scientists. Boca Raton, FL:

Chapman & Hall/CRC.

Bartram, D. & Baxter, P. (1996). Validation of the Cathay Pacific Airways selection

program. The International Journal of Aviation Psychology, 6(2), 149-169.

Bond, N.A., Bryan, G.L., Rigney, J.W. & Warren, N.D. (1968). Aviation psychology.

Los Angeles, CA: USC Press.

Borg, I & Groenen, P.J.F. (2005). Modern multidimensional scaling: Theory and

applications (2nd Ed). NY: Springer.

Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative

Research in Psychology, 3(2), 77 – 101.

Braun, V., Clarke, V. & Terry, G. (2015). Thematic analysis in P. Rohleder & A.C.

Lyons (Eds) (2015). Qualitative Research in Clinical and Health Psychology.

(pp. 95 – 113) NY: Palgrave McMillan.

Brazill & Grofman (2002). Factor analysis versus multi-dimensional scaling: binary

choice roll-call voting and the US Supreme Court. Social Networks, 24, 201-

229.

248

Cahill, J. McDonald, N. & Losa, G. (2013). Understanding and improving flight

crew performance of the preflight, flight planning and briefing task. The

International Journal of Aviation Psychology, 23(1), 27-48.

Campbell, J.S., Castenada, M. & Pulos, S. (2010). Meta-analysis of personality

assessments as predictors of military aviation training success. The

International Journal of Aviation Psychology, 20(1), 92-109.

Carretta, T. R. (2011). Pilot Candidate Selection Method: Still an effective predictor

of US Air Force pilot training performance. Aviation Psychology and Applied

Human Factors, 1, 3–8.

Carretta, T.R., Teachout, M.S., Ree, M.J., Barto, E.L. King & Michaels, C.F. (2014).

Consistency of the relations of cognitive ability and personality traits to pilot

training performance. The International Journal of Aviation Psychology, 24(4),

247-264.

Carrick, K. (2009, June). Human factors as a component of airmanship. Paper

presented at the 8th Industrial and Organisational Psychology Conference,

Sydney, Australia.

Carrick, K., Graham, S., Healey, A. & Pickard, S. (2008, April). An assessment of

feedback and evaluation in a flight training school. Paper presented at the 8th

International Symposium of the Australian Aviation Psychology Association,

Novotel Brighton, Sydney, Australia.

CASA (2003). Human factors and crew resource management training (AC121A-

09). Canberra, Aus: CASA.

CASA (2006). Flight instructor manual aeroplane. Canberra, Aus: CASA.

CASA (2008). Teaching and assessing single-pilot human factors and threat and

error management. (CAAP 5.59-1(0)). Downloaded

from: http://www.casa.gov.au/wcmswr/_assets/main/download/caaps/ops/5_59

_1.pdf

Corbin, J. & Strauss, A. (2008). Basics of qualitative research: Techniques and

procedures for developing grounded theory (3rd Ed). Thousand Oaks, CA:

Sage.

249

Costa, P.T. Jnr. & McCrae, R.R. (1989). The NEO Personality Inventory Manual.

Odessa, Fl: Psychological Assessment Resources.

Damitz, M., Manzey, D., Kleinmann, M. & Severin, K. (2003). Assessment center

for pilot selection: Construct and criterion validity and the impact of assessor

type. Applied Psychology: An International Review, 52(2), 193-212.

Damos, D. (2014). Editors preface to the special issue on pilot selection. The

International Journal of Aviation Psychology, 24(1), 1-5.

de Crespigny, R. (2012). QF32. Sydney, Aus: Macmillan.

Ebbage, L. & Spencer, P.D. (2003, October). Airmanship training for modern

aircrew. Paper presented at the RTO HFM Symposium on Advanced

Technologies for Military Training, Genoa, Italy, RTO-MP-HFM-101.

Downloaded from http://www.skybrary.aero/bookshelf/books/1131.pdf

Edwards, C. (2013). Airmanship (2nd Ed). Blacker Ltd.

Elms, G. (2007). GAPAN Press release. Downloaded from

http://www.atsb.gov.au/media/12581/gapan_press_release_20_08_07.pdf

Endsley, M.R. (1999). Situation awareness in aviation systems. In D.J. Garland, J.A.

Wise, & V.D. Hopkin (Eds) Handbook of Aviation Human Factors. Mahwah,

N.J.: Erlbaum.

Flin, R & Martin, L. (2001). Behavioural markers for crew resource management: A

review of current practice. International Journal of Aviation Psychology, 11,

95-118.

Flin, R. Martin, L., Goerters, K., Hoermann, J., Amalberti, R., Valot, C. & Nijhuis,

H. (2003). Development of the NOTECHS (non-technical skills) system for

assessing pilots’ CRM skills. Human Factors and Aerospace Safety, 3, 95-117.

Flin, R., O’Connor, P. & Crichton, M. (2008). Safety at the Sharp End: A guide to

non-technical skills. Aldershot, UK: Ashgate.

Flin, R., O’Connor, P. & Mearns, K. (2002). Crew resource management: Improving

team work in high reliability industries. Team Performance and Management

8(3/4), 68-78.

250

Gray, D.E. (2009). Doing research in the real world (2nd Ed). Thousand Oaks, CA:

Sage.

Hawkins, F.H. (1987). Human factors in flight. Aldershot, UK: Ashgate.

Hayward, B.J. & Lowe, A.R. (2010). The migration of crew resource management.

In B.G. Kanki, R.L. Helmreich & J. Anca (Eds.), Crew Resource Management.

(pp. 137-342). San Diego, CA: Academic Press.

Helmreich, R.L. (2002). Culture, threat and error: Assessing system safety. In Safety

in aviation: the management commitment: Proceedings of a Conference.

London, UK: Royal Aeronautical Society.

Helmreich, R. L., & Foushee, H. C. (1993). Why Crew Resource Management?

Empirical and theoretical bases of human factors training in aviation. In E.

Wiener, B. Kanki, & R. Helmreich (Eds.), Cockpit Resource Management (pp.

3-45). San Diego, CA: Academic Press.

Helmreich, R. L. & Foushee, H. C. (2010). Why CRM? Empirical and theoretical

bases of human factors training. In B.G. Kanki, R.L. Helmreich & J. Anca

(Eds.), Crew Resource Management. (pp. 3-58). San Diego, CA: Academic

Press.

Helmreich, R.L., Klinect, J.R. & Wilhelm, J.A. (1999). Models of threat, error and

CRM in flight operations. In Proceedings of the Tenth International

Symposium on Aviation Psychology. Columbus, OH: The Ohio State

University.

Helmreich, R.L. & Merritt, A.C. (1998). Culture at work in aviation and medicine:

National, organisational and professional influences. Aldershot, UK: Ashgate.

Helmreich, R.L., Merritt, A.C. & Wilhelm, J.A. (1999). The evolution of crew

resource management training in commercial aviation. International Journal of

Aviation Psychology, 9, 19-32.

Hoerman, H-J. & Goerke, P. (2014). Assessment of social competence for pilot

selection. The International Journal of Aviation Psychology, 24(1), 6-28.

Hsieh, H-F. & Shannon, S.E. (2005). Three approaches to qualitative content

analysis. Qualitative Health Research, 15(9), 1277 – 1288.

251

Hunter, D. & Burke, E.F. (1994). Predicting aircraft pilot-training success: A meta-

analysis of published research. The International Journal of Aviation

Psychology, 4(4), 297-313.

IBM (2012). PASW20 Statistics package for the social sciences.

ICAO, (1989). Human factors digest No. 1: Fundamental Human Factors Concepts.

(Circular 216). Montreal, CA: Author.

Jones, D.G. & Endsley, M.R. (2004). Use of real-time probes for measuring situation

awareness. International Journal of Aviation Psychology, 14, 343-367.

Kaempf, G.L. & Klein, G. (1994). Aeronautical decision making: The next

generation. In Johnston, N., McDonald, N. & Fuller, R.

(Eds), Aviation psychology in practice. (pp. 223-254). Aldershot, UK: Avebury

Technical

Kanki, B.G. (2010). Communication and crew resource management. In B.G. Kanki,

R.L. Helmreich & J. Anca (Eds.), Crew Resource Management. (pp. pp. 111-

146). San Diego, CA: Academic Press.

Kern, T. (1996). Redefining airmanship. New York; McGraw-Hill.

Kern, T. (1997). Flight discipline. New York; McGraw-Hill.

Kern, T. (1999). Darker shades of blue: The rogue pilot. New York; McGraw-Hil.l

Kern, T. (2009a, June). CASA threat and error management roadshow. Brisbane,

Australia.

Kern, T. (2009b). Blue threat: Why to err is inhuman. Monument, Co: Pygmy Books.

King, R.E. (2014). Personality (and psychopathology) assessment in the selection of

pilots. The International Journal of Aviation Psychology, 24(1,) 61-73.

Krippendorf, K. (2013). Content Analysis: An introduction to its methodology (3rd

Ed). Thousand Oaks, CA: Sage.

Liamputtong, P. (2013). Qualitative research methods. Melbourne, Aust: Oxford.

Lincoln Y.S. & Guba, E.G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.

252

MacLeod, N. (2005). Building safe systems in aviation: A CRM developer’s

handbook. Aldershot, UK: Ashgate.

Mané, A. (1981). Airmanship: an introduction. In Proceedings of the First

International Symposium on Aviation Psychology. Columbus, OH: The Ohio

State University.

Marchbank, M. (Nov-Dec 2008). Is airmanship dead? Flight Safety Australia.

CASA.

Margerison, C.J., McCann, D.J. & Davies, R. (1986). Human resource management

for TAA pilots and flight engineers. Asia-Pacific Journal of Human Resources,

24(2), 32-37.

Martinussen, M. (1996) Psychological measures as predictors of pilot performance:

A meta-analysis. The International Journal of Aviation Psychology, 1(1), 1-20.

Maurino, D. (1994). Foreword. In N. Johnston, N. McDonald & R. Fuller (Eds)

Aviation psychology in practice. (pp. xv-xx). Aldershot, UK: Ashgate.

Mavin, T. J. (2010, Nov). Challenges for training and checking. Paper presented at

the Pacific and Asia CRM Developers, Evaluators and Facilitators Forum,

PACDEFF-2010, Adelaide, Australia.

Mavin, T.J. & Roth, W-M. (2014). A holistic view of cockpit performance: An

analysis of the assessment discourse of flight examiners. The International

Journal of Aviation Psychology, 24(3), 210-227.

Mayring, P. (2000). Qualitative content analysis: Theoretical foundation, basic

procedures and software solution. Qualitative Social Research, 1(2), 1 – 142.

Retrieved from http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173

Nergård, V. (2014). Airmanship – a qualitative approach. Aviation, 18(3), 147-156.

Nergård, V., Hatlevik, O.E., Martinussen, M. & Lervag, A. (2011). An airman’s

personal attitude: Pilots’ point of view. Aviation, 15(4), 101-111.

Orasanu, J.M. (2010). Flight crew decision making. In B.G. Kanki, R.L. Helmreich

& J. Anca (Eds.), Crew Resource Management. (pp. 147-180). San Diego, CA:

Academic Press.

253

Orlady, L.M. (2010). Airline pilot training today and tomorrow. In B. Kanki, R.

Helmreich, & J. Anca (Eds.), Crew resource management, 2nd ed.. (pp. 469-

491). San Diego, CA: Academic Press.

Potter, W. J. & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in

content analysis. Journal of Communication Research, 27(3), 258 – 284

QSR (2011). NVivo data analysis software. Downloaded from

www.qsrinternational.com.

Reason J. (2008). The human contribution: Unsafe acts, accidents and heroic

recoveries. Aldershot, UK: Ashgate.

Rose, M.R., Barron, L.G., Carretta, T.R., Arnold, R.D. & Howse, W.R. (2014). Early

identification of unmanned aircraft pilots using measures of personality and

aptitude. The International Journal of Aviation Psychology, 24(1), 36-52.

Ryan, G.W. & Bernard, H.R. (2000). Data management and analysis methods. In

N.K. Denzin and Y.S. Lincoln (Eds) Handbook of qualitative research (2nd Ed)

Thousand Oaks, CA: Sage.

Salas, E. (2014). Commentary on the article by Hoermann and Goerke: In search of

collective orientated members in the flight deck and beyond – A need a

challenge. The International Journal of Aviation Psychology, 24(1), 29-31.

Salas, E., Burke, C.S., Bowers, C.A., & Wilson, K.A. (2001). Team training in the

skies: Does crew resource management work? Human Factors, 43, 641-674.

Sarter, N.B. & Woods, D.D. (1991). Situation awareness: A critical but ill-defined

phenomenon. The International Journal of Aviation Psychology, 1, 45-57.

Shepherd, G. (2005). Foreword. In DFS-ADF (Ed.), Focus on human factors in

aviation. Canberra, Australia: Directorate of Flying Safety-ADF.

Stemler, S. (2001). An overview of content analysis. Practical Assessment, Research

and Evaluation. 7 (17). Retrieved from

http://PAREonline.net/getvn.asp?v=7&n=17

Steyvers, M. (2002) Multidimensional scaling. In L. Nadel (Ed) Encyclopedia of

Cognitive Science. (pp. 1-5). NY, NY:Wiley.

254

Sullenberger, C.B. & Zaslow, J. (2009). Highest duty: My search for what really

matters. NY: Harper Collins.

Swauger, S. (June, 2006). The human factor: Best captains – what FOs want in a

captain. Reporting Point Express/Crossfeed. Southwest Airlines Pilot

Association.

Swauger, S. (2011, November). Identifying the qualities of the ideal captain. Paper

presented at the Pacific and Asia CRM Developers, Evaluators and Facilitators

Forum PACDEFF-2011, Queenstown, NZ.

Tabachnik, B.G. & Fidell, L.S. (2007). Using multivariate statistics (5th Ed.). Boston,

MA: Pearson/Allyn & Bacon.

Thomas, M.J.W. (2004). Predictors of TEM: Identification of core non-technical

skills and implications for training system design. The International Journal of

Aviation Psychology, 14, 207 – 231.

Thomas, M.J.W. (2005). Error management training: Defining best practice.

Canberra, Australia: ATSB.

Trollip, S.R. & Jensen, R.S. (1991). Human factors in general aviation. Englewood,

Co: Jeppesen Sanderson.

Tvaryanus, A.P. (2014). Commentary on the article by Rose et al.: Search for the

right stuff. The International Journal of Aviation Psychology, 24(1), 57-60.

Weber, R.P. (1990). Basic content analysis. Thousand Oaks, CA: Sage

Welch, J.F., Bjork, L. & Bjork, L. (1999). Van Sickle’s modern airmanship – 8th

edition. New York: McGraw-Hill.

255

Appendices

Appendix A: Conference Paper referred to in Chapter 1 257

Carrick, K., Graham, S., Healey, A. & Pickard, S. (2008, April). An assessment of feedback and evaluation at a flight training school. Paper presented at the 8th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.

Appendix B: Airmanship Questionnaire reported in Chapter 2. 269

Appendix C: Airmanship Questionnaire reported in Chapter 3. 273

Appendix D: Interview Schedules reported in Chapter 4. 279

Appendix E: Conference papers based on Chapter 2 287

Carrick, K. (2009, June). Human factors as a component of airmanship. Paper presented at the 8th Industrial and Organisational Psychology Conference, Sydney, Australia.

Carrick, K. and Chalmers, K. (2010a, April). Airmanship: Voices from an on-line survey. Paper presented at the 9th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.

Carrick, K. and Chalmers, K. (2010b, April). Development of airmanship. Paper presented at the 9th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.

Appendix F: Airmanship Questionnaire reported in Chapter 6. 305

Appendix G: Domain Differences from Chapter 6 313

Appendix H: Copyright Permissions 320

256

Page left deliberately blank

257

Appendix A: Carrick, Graham, Healey and Pickard (2008)

An assessment of feedback and evaluation in a flight training school.

Kirstie Carrick, Suzanne Graham, Alison Healey

University of Newcastle

Stephen Pickard

BAE Systems

Background

An Australian flight training provider requested an independent examination of feedback and evaluation of flying activity in their flight training school, with a view to identifying any areas that might require improvement.

Learning to fly an aircraft is essentially the learning of motor skills, related to the physical handling of the aircraft, and acquiring the judgement, decision making and situational awareness skills that can be referred to as ‘airmanship’. Acquiring the essential theoretical knowledge of such things as aerodynamics, systems, engines and legislation, underpins successful learning and execution of flying skills. Thus the process of learning to fly will involve ground based study and the learning and practice of relevant skills in the aircraft. Part of the process is the provision of feedback from an instructor as to the effectiveness of the learner’s performance. There may be two different aims here: to provide information about the accuracy and appropriateness of actions in order that the trainee acquire and improve his/her skills; and secondly, to assess (and let the trainee know) whether he/she is meeting the standard of proficiency and rate of progress expected or required by the training organisation. Educational literature refers to these two intentions as formative and summative feedback. Formative feedback is that conducted during a task to provide information about how accurately the trainee is carrying out the tasks and how the trainee can improve their performance. Summative feedback or evaluation is generally conducted after the task performance and sums up how well the task was performed relative to some standard (Telfer and Biggs, 1985).

The education and flight training literature suggests that evaluation of performance should be based on objective criteria or competencies (Reich, Bye & Henley, 2003). Roscoe and Childs (1980) were early advocates of effective assessment and feedback to trainee pilots using an objective, criterion referenced approach. However, one important challenge facing assessment methods in aviation is to avoid overloading the instructor with so many indices of performance that he (or she) cannot observe them all during in-flight assessment, particularly while maintaining aircraft safety and instructional benefits to the student.

258

Most recent expression of the concept of criterion referenced evaluation has been adopted in the wider field of skill training as the setting of ‘competencies’, the level of satisfactory performance of job tasks (Tovey and Lawlor, 2004). This is well known in workplace and trade training, and has been introduced to civilian flight training in the development by CASA of National Competency Standards for pilots. There has been some debate regarding the applicability of competency-based assessment when assessing higher level competencies such as critical thinking, judgment, and decision making (all of which relate to the concept of airmanship) due to the difficulty in describing these more subjective outcomes in terms that are directly observable and assessable (Reich, Bye, & Henley, 2003).

The provision of formative feedback about task performance to the trainee is a crucial step in the learning process in any training and assessment program, not just in aviation (Kunchinke, 2000; Reich et al, 2003; Weaver, 2006). Feedback is an essential aspect of the communication that takes place between an instructor and a trainee and, as with any communication, the trainee needs to receive and comprehend the feedback they are given in order for it to be successful (Wood et al, 2004). Feedback about both strengths and weaknesses in task performance will allow trainees to maximise their potential and continually improve, if they are willing to accept the feedback (Weaver, 2006). In a study of summative feedback related to assessment of text (in a written assignment) Fritz, Morris, Bjork, Gelman & Wickens, (2000) found that feedback is more effective if it is not just an overall numeric score, but rather a qualitative account of several aspects of the task. In a further study of feedback on written work, Weaver (2006) found that feedback is perceived as helpful by students if it contains information on both strengths and weakness, is specific but related to general improvements that can be made in the next task, is understandable at the receiver’s level and related to assessment criteria.

In the management and organisational psychology literature, it is advised that constructive feedback should be based on trust and be direct. It should provide specific examples and be given when the receiver is ready to receive and accept it. Constructive feedback should also be practical and contain an appropriate amount of information. Most importantly feedback should be valid and be aimed to benefit the person who is receiving it, so that development occurs (Wood et al, 2004). Feedback about the effectiveness of workplace performance is linked to motivation and job satisfaction (Furnham, 2005).

Combining these ideas suggests that good post-flight debrief feedback should include:

• Clear, understandable communication • Evaluation related to criteria, and seen to be valid • Not just a total score, include sub-scores and qualitative evaluation • Strengths and weaknesses • Specific examples of each • Improvement tips

259

Assessors should be aware that evaluation must be consistent, reliable and unbiased as possible especially where a number of assessors are evaluating a range of students. The nature of the instructor-student relationship and the values and beliefs of the instructor can impact on the assessment of individual performance (Williams, Klamen & McGaghiel, 2003). Macleod (2005) notes the effect of ‘observer bias’ on evaluation in aviation contexts, and discusses standards, recency and primacy, halo and horn effects, leniency and severity, central tendency and ‘scale abuse’, the latter referring to those who feel that perfection can never be achieved and so never award the top grade. It is important for accurate evaluation, that the specific details of the assessment requirement are described and conditions of assessment are standardised as much as is possible. In addition the assessors should be aware of the possibility of the above-mentioned external influences and actively minimise them, so that trainees are assessed as having met or not met the performance criteria as objectively as possible.

The aim of this study was to evaluate feedback provided to trainees and ask the trainees and instructors about their views of feedback and evaluation in the current system.

Method

Materials

Observation of actual post-flight debriefs was considered to be too time consuming and intrusive. The instructors at this organisation, however, do produce written feedback from evaluation flights. This feedback is in addition to the post-flight debrief, provided as a record of assessment and is sighted and signed off as accurate by the trainee. Thus it was felt to be a fairly accurate record of content of the actual debrief. Each feedback sheet included three evaluation scores, one each for preparation, technique and airmanship, the lowest of the three being considered the final score for the flight. A sample of 98 feedback sheets was collected for analysis. Sheets were selected from across the range of flights, from different students and instructors. All identifying information was removed before the material was provided to the researchers.

A survey of instructor and trainee opinions about current evaluation and feedback practices was also conducted. The survey questionnaires differed slightly for each group and included a few demographic questions. The main part of the questionnaires consisted of a mix of open-ended questions, allowing respondents to write down their ideas or opinions, and some statements to which respondents were asked to indicate their degree of agreement. This was indicated by placing a cross on a line which was anchored at each end with the statements ‘not at all’ and ‘completely’, and had intermediate indications of ‘somewhat’, ‘moderately’ and ‘mostly’. In analysis, responses were scored between zero, for ‘not at all’ and 8 for ‘completely’, with estimation to the nearest to 0.5.

Procedure

260

The researchers attended the flight training establishment and presented a brief talk on the literature relating to feedback and evaluation to the assembled instructors. The aim of the research was outlined and the expert opinion of the instructors requested. They were told that participation in the survey was voluntary and anonymous, and that questionnaires could be returned via a box left in reception or by post. Questionnaire packs containing an information sheet, the questionnaires and a reply paid return envelope were then distributed. Trainee questionnaire packs were left with management for distribution to trainees.

Participants

The response rates among the two populations were similar, with 11 of the 20 instructor questionnaires being returned (55%), and 20 of the 40 trainee questionnaires (50%). The instructors ranged in experience from 0 - 2 years as an instructor, to over 16 years experience. The mean fell in the ‘3 to 5 years’ group. The trainee participants ranged in age from 19 to 31, mean 22.62 years. The trainee participants were from two instruction groups, one (9 participants) was at a stage of 25 hours training or less, the other group (11 participants) had a spread of 40 to 70 hours training.

Results

Analysis of feedback sheets

Evaluation scores: In the sample of 98 feedback sheets, three were missing assessment scores. In the remaining 95 sheets, it was found that preparation was rarely the lowest score. The range of preparation scores was from 1 to 4, with a mean score of 3.58. The most common score was 4. For both Technical skill and Airmanship, the range was 0 to 4 and the means were similar (Technique = 2.81, Airmanship = 2.80). It appears that the scores awarded are quite low and a score of 5 was never awarded in the sample examined. Statistical comparison of the means using a Kendell’s W (non-parametric equivalent of an ANOVA) revealed that there was no difference between the rank mean scores for airmanship and technique, but that the rank mean score for preparation was significantly higher (Kendall’s W=.432, p<.001). There is also a strong significant correlation between the airmanship and technical skills scores (r = .819, p<.001) indicating that trainees generally are awarded similar scores for each. The same score was awarded for preparation, airmanship and technique on 21 (22%) of the feedback sheets, and airmanship and technique were awarded the same score (with preparation higher) on 47 sheets (49%). There were also high significant correlations between each of technique and airmanship scores and the final score, as would be expected, given that the final score is the lower of the other two and they were so often the same. Correlation between final score and technique score was r= .881 (p<.001) and final score and airmanship r=.909 (p<.001).

Feedback comments: The number of positive and negative comments written on the feedback sheets was counted. A positive comment occurred where there was an indication that something was done well. A negative comment included an indication of

261

poor performance. Neutral comments, usually simply a record of what was flown, were not counted. An assessment was made of the number of learning comments on the feedback sheets, where a learning comment is defined as advice regarding future improvement, or how to do things better, or what not to do in future. Examples of these are: ‘Do not fixate on runway but maintain an appropriate workcycle’ or ‘Avoid swapping hands to set frequencies’.

Statistical comparison of the mean number of positive, negative and learning comments per feedback sheet indicated a significant difference among them (Kendall’s W=.664, p<.000). The rank mean for learning comments was significantly lower than the rank means for positive and negative comments, and there was no difference between positive and negative rank means.

Content analysis of the positive, negative and learning comments allowed subtotals to be calculated for the three assessment categories, preparation, airmanship and technical skills. Comments that mentioned preparation were classed as preparation, Comments were classed as airmanship if they occurred under a general heading of airmanship, or if they corresponded to airmanship criteria included in the assessment word pictures. Anything else was considered to be related to technical skills. Table 1 indicates the frequency count for type of comment (positive, negative or learning) by aspect of assessment (preparation, technique or airmanship).

Table 1:

Frequency of type of comment by aspect of assessment

Type of Comment

Positive Negative Learning Totals

Preparation 57 15 3 75

Technique 403 579 32 1014

Airmanship 131 66 9 206

Totals 591 660 44 1295

There were relatively few comments about preparation on the feedback sheets, and most of these were positive. There were more negative than positive comments about technique and most feedback sheets included both negative and positive comments. The majority of learning comments related to technical skills. There were fewer comments made about airmanship than aircraft control technique. Comments on airmanship were more frequently positive than negative, and nine learning comments related to airmanship.

A statistical comparison of the mean numbers of positive and negative comments per feedback sheet in each of the three categories of assessment was carried out. Learning

262

comments were not included because of the low numbers. There were significantly more comments regarding the technical aspects of flight than preparation or airmanship, and there were more positive than negative comments on preparation and airmanship, but more negative than positive comments about technical skills (Kendall’s W = .541, p<.000). Figure 1 shows the rank means for this analysis.

0

1

2

3

4

5

6

Prep Air Tech

Aspect of Assessment

Mea

n Ra

nk (K

enda

ll)

PosNeg

Figure 1: Type of comment by aspect of assessment: rank means

Survey analysis

Verbal/written brief consistency: Instructors were asked ‘how much does the content of your verbal debrief reflect the content of the written feedback sheet’. Response was by placing a cross on a scale from ‘not at all’ (scored as zero) to ‘completely’ (scored as 8). For this question the mean was 7.1, with all but one response above ‘mostly’ (scored as 6), indicating an ‘almost completely’ response. Trainees were asked a similar question, the mean of the responses was 6.3, just above ‘mostly’. The difference between the means was tested (using non-parametric tests) and showed a significant difference between the two groups (Wilcoxen, z=2.58, p<.01). This indicates that instructors were more confident that the content of the feedback sheets reflected the verbal debrief than were the trainees.

Objectivity of feedback: Trainees were asked how much they thought the written feedback reflects their flight performance. The mean of the responses was 5.7 (just below ‘mostly’), with a range of 4 (moderately) to 7 (between ‘mostly’ and ‘completely’), as illustrated in Figure 2.

263

7.006.005.004.00

reflectperformance

12

10

8

6

4

2

0

Freq

uenc

y

Mean =5.6905Std. Dev. =0.73274

N =21

reflectperformance

Figure 2: Trainee response regarding how much written feedback reflects actual performance.

Trainees were then asked separately about whether the current assessment of airmanship and aircraft control techniques, accurately reflects their actual performance. Response to the question about airmanship resulted in a mean score of 4.9, (SD = 1.20, range 2.5 to 7) and the technical skills in a mean of 5.9, (SD = 0.89, range 4 to 7). These two means are significantly different (Wilcoxen, z=2.573, p<.01). This indicates that the trainees are more confident that assessment of technical skills reflects their actual performance than they are that the assessment of airmanship reflects their actual performance.

The instructors were asked how much they felt that the current assessment is an objective process. Response to the assessment of airmanship question resulted in a mean of 3.6 (SD = 1.38, range 2 to 5.5). The mean response to the question about assessment of technical skills was 4.9 (SD = 1.72, range 1 to 6.5). These two means are significantly different (Wilcoxen, z=2.187, p<.029). This result indicates that in general the instructors feel that their assessment of technical skill is more objective than their assessment of airmanship.

Statistical comparison of trainee and instructor scores indicates that there is a statistically significant difference between the mean responses for each group for the question about airmanship (Wilcoxen, z=2.398, p<.016). Trainees appeared more confident that the assessment reflects their true ability than the instructors were confident in their own objectivity when assessing airmanship.

There was no significant difference between the two groups of respondents for the responses to the same question about aircraft control techniques. The trainees and instructors were equally confident that technical skill assessment (between moderately and mostly) reflects ability and is objective.

Inter-rater reliability: Instructors were asked how consistent the system allowed them to be when assessing different trainees. The mean response to this question was 5.2, (SD

264

= 1.87, range 1 to 8), indicating an average position between ‘moderately’ and ‘mostly’. The distribution illustrated in Figure 3 indicates that six of the eleven instructor respondents see themselves as between mostly and completely consistent.

8.007.006.005.004.003.002.00

diffstudconsistent

4

3

2

1

0

Freq

uenc

y

Mean =5.1818Std. Dev. =1.87447

N =11

diffstudconsistent

Figure 3: Instructor opinion of their consistency across trainees

Trainees were asked two questions about instructor consistency, first, ‘how consistent do you think instructors are when assessing different students’ and then ‘if two instructors were to assess the same student how consistent do you think their assessment would be’, an attempt to estimate ‘reliability’.

The response to the consistency question resulted in a mean of 4.3 (SD = 1.78, range 1 to 7), indicating ‘moderately’, a position just over half way along the scale. Response to the question about reliability among instructors assessing the same trainee resulted in a mean of 3.8, (SD = 1.72, range 1 to 6), suggesting that trainees see instructors as being a bit less than moderately reliable, slightly below the middle of the scale. These means were compared statistically, but were found not to be significantly different (Wilcoxen z=1.483, p=.138).

The means representing instructor consistency and trainee view of consistency were compared and found not to be significantly different. It seems that instructors are considered to be (and consider themselves) generally internally consistent. Trainees report that different instructors assess the same trainees moderately consistently and there is no significant difference with their opinion regarding individual consistency across trainees. The range of scores indicates a general feeling of ‘moderately’ consistent, but with some spread of opinion.

System problems and improvements: Instructors were asked about problems with the current system of assessment for airmanship and aircraft control technique, and also to make suggestions for improvement. All the respondents wrote something about these. There were a range of views including suggestions for improving the consistency of instructor evaluation, by changes to the assessment guidelines and/or training for instructors in debriefing, assessment and standardisation. Two responses drew attention to the workload resulting from too many students and not enough instructors, resulting in adverse effect on quality of training/debriefing. One instructor (out of 11) stated that

265

no changes need to be made. There were almost twice as many responses (9) to the questions about problems with assessment of airmanship and suggested improvements, than there were to the same questions about technique (5).

Trainees were asked similar questions and also raised the issue of consistency and standardisation. Eight of the twenty responses to the question about the current system were positive, stating such things as ‘comments are a good indication of where to improve’. Five responses mentioned instructor differences and inconsistencies, and one specifically noted variation in the amount of feedback given. Another commented on the narrow range of scores awarded, noting it was thus hard to show improvement over time. Three responses commented that the course is short, challenging and high workload. Asked about improvements to the system, seven responses (out of a total of 13) were concerned with standardisation and consistency across instructors. Three further comments had to do with instructor issues, having the same instructor regularly, personality clashes and assessment of instructors. The other three responses were more general comments on the assessment system, averaging of scores, making it pass/fail rather than having scores, and providing more detail on the assessment criteria to trainees.

Discussion

Distribution of scores: There were no top scores, ‘5’s, in the sample of feedback sheets despite there being allowance in the assessment guidelines for some errors to occur in a ‘5’ evaluation. This non-use of the full range of possible scores is something that Macleod (2005) would term ‘scale abuse’. It was also noted that scores for technique and airmanship were often the same (71% of sample sheets), which may indicate that instructors are awarding a global score rather than making truly separate assessments.

Feedback content: Analysis of the feedback sheets indicated clear imbalance in the types of comments made and in the aspects of assessment commented upon. Few learning comments were recorded, preparation was not often mentioned, there were some comments on airmanship and the majority of comments related to technique.

The literature suggests that good feedback should include suggestions about how the student can improve their performance (Weaver, 2006). These were identified as learning comments in the analysis of the sample of feedback sheets. Only 44 learning comments were identified on 34 of the 98 sheets. Although negative comment by implication includes some learning intent, constructive advice is generally considered to be of more use to students. This may indeed occur in the formative feedback in air and/or in the post-flight debriefing, and not make its way into the feedback sheets. However, instructors reported that the sheets are a close reflection of the post-flight debrief. Inclusion of some indication of what should be done to attain a ‘5’ would help the trainee improve.

The majority of written feedback comments were concerned with technique. It could be that the instructors are more comfortable commenting on skill development within

266

the context of the technical criteria defined for the sortie, rather than the less easily observed aspects of airmanship (decision making, situational awareness, etc). There are also many different technical flying activities in each sortie which would require comment, while airmanship criteria may be considered across the whole flight. Comments on airmanship sometimes appeared to be in the form of a mechanical record of whether radio, checks and/or situational awareness were satisfactory, or not, rather than learning comments regarding airmanship. Little comment on preparation is made on the feedback sheets, with 75 comments of various types, made across the sample of 98 sheets. This may be because the feedback of a high score (‘4’) may be considered sufficient comment, and it would save time not to add anything else.

Instructor consistency: There was no statistical difference between the mean responses of instructors and trainees on the question addressing consistency across trainees, although the instructors’ responses were, in general, higher on the scale than the trainees’. On the whole the responses to these questions suggest that there is a perception of moderate to mostly consistent assessment. The ideal, of course, would be to have ‘completely’ consistent assessment.

In response to the final open questions, five of the trainees raised issues of instructors’ lack of consistency as concerns about the current system (out of 20 comments), and ten responses about future improvements (out of thirteen) were concerned with instructor issues. The content of the questions they had answered may have primed the respondents to some extent to report such issues, but clearly some dissatisfaction has been tapped among some of the trainees. One went as far as to suggest that there should be more “scrutiny of instructors and the use of feedback from trainees to assess instructors”. Research suggests that credibility of assessors is important when presenting negative feedback for it to be taken on board by the student (Steelman & Rutowski, 2004).

It is also clear that few of the instructors felt that they (or their colleagues) were able to be completely consistent across trainees. There were some suggestions among those made to the final questions asking about problems and improvements, which addressed issues of consistency. A couple of instructors raised a concern that standards might vary with instructors’ experience. One suggestion was to have regular meetings of instructors to ensure consistency and standardisation of assessment, another that some training in assessment and debriefing would be useful as these are not emphasised in instructor training.

Conclusions:

Feedback to flight trainees included both positive and negative comments, but lacked many learning comments, although these may be made during debriefs and not written into the feedback sheets.

267

Consistency in evaluation across different instructors appears to be of some concern, to instructors and trainees, and several suggestions were made to address the problem however it should also be noted that the indicators of consistency were towards the top of the scale rather than the bottom, so while consistency may not be seen as perfect, it is not seen as drastically lacking.

The relative lack of comment on airmanship in written feedback, the indication that airmanship is slightly less objectively assessed and concerns raised among the survey open responses may indicate that while there are clear criteria for demonstration of technical skills these may not matched by as definitive criteria for demonstration of airmanship. Definition and evaluation of airmanship is an area that is being explored further.

References

Fritz, C., Morris, P., Bjork, R. A., Gelman, R. & Wickens, T. D. (2000). When further learning fails: Stability and change following repeated presentation of text. British Journal of Psychology, 91, 493-511.

Furnham, A. (2005). The psychology of behaviour at work: The individual in the organisation. Hove, UK: Psychology Press.

Kuchinke, P. K. (2000). The role of feedback in management training settings. Human Resource Development Quarterly, 11(4), 381-401.

Macleod, N. (2005). Building safe systems in aviation: a CRM developer’s handbook. Aldershot, UK: Ashgate.

Steelman, L. A. & Rutkowski, K. A. (2004). Moderators of employee reactions to negative feedback. Journal of Managerial Psychology, 19(1), 6-18.

Reich, A., Bye, J., & Henley, I. (2003). Assessment and Evaluation. In I.M.A. Henley (Ed.), Aviation education and training. Aldershot, UK: Ashgate.

Roscoe, S. N., & Childs, J. M. (1980). Reliable, objective flight checks. In S. N. Roscoe (Ed.), Aviation Psychology. Ames, Iowa: Iowa State University.

Telfer, R. & Biggs, J. (1985). The Psychology of Flight Training. Cessnock, Australia: Aircrew Training Centre.

Tovey, M.D. & Lawlor, D.R. (2004). Training in Australia. Sydney, Australia: Pearson Prentice Hall.

Weaver, M. R. (2006). Do students value feedback? Student perceptions of tutors’ written responses. Assessment and Evaluation in Higher Education, 31(3), 379-394.

Williams, R. G., Klamen, D. A. & McGaghie, W. C. (2003). Cognitive, social and environmental sources of bias in clinical performance ratings. Teaching and Learning in Medicine 15(4), 270-292.

268

Wood, J., Chapman, J., Fromholtz, M., Morrison, V., Wallace, J., Zeffane, R. M., Schermerhorn, J. R., Hunt, J. G. & Osborn, R. N. (2004). Organisational Behaviour: A Global Perspective (3rd ed). Sydney, Australia: Wiley.

269

Appendix B: Airmanship Questionnaire reported in Chapter 2

Airmanship Survey

Thank you for deciding to complete this questionnaire. Please consider that what we want is your personal view, not that of an employer or organisation, or what you might have read, but what you, yourself, think.

Experience – these questions are factual. Please complete truthfully!

Copy and paste as required

1. Licence level: (tick one)

Unlicensed

Student

GFPT

PPL

CPL

ATPL

Military

2. Instructor rating and grade (if held) Rating

Grade 1

Grade 2

Grade 3

CFI

3. Where did you take your initial flight training (tick one)?

Military or

Civilian

4. Military experience (if any): Years of service: ____

Hours flown: ________

5. Civilian experience: Years of experience: _______

270

Hours flown: _______

6. Have you any Aviation related studies or qualifications (besides licensing requirements), if so, what? ____

7. Please indicate your general educational level:

Left before Yr 12

HSC, Matric, or equivalent toYr12

Trade or TAFE qualifications

University degree

8. Any Comments? ______

Questions about airmanship – please type in your responses as if you were speaking to someone. Just start typing on the line below the question.

1. What do you think airmanship is? (Take as much space as you want)

2. Think of your response to the first question, can you elaborate on each of the key words in your response? For example, if you wrote ‘being safe’, write in more detail what ‘being safe’ would involve. (Take as much space as you want)

3. Was the development of airmanship a specific part of the initial flight training you did? Please explain what this involved.

4. (a) In your initial flight training, was airmanship, or concepts related to airmanship, discussed in ground training or pre- or post- flight briefings?

271

4 (b) If so, which concepts, and what sorts of discussion?

5. Thinking about your progress to date as a pilot, what do you feel led to the development of your sense of airmanship?

6. In your current aviation activities, are you normally actively aware of airmanship as part of your operation? What would be the most important things that come to mind?

7. In your current aviation activities, during what phases of flight or events would you be most actively aware of airmanship issues?

8. If you are currently an instructor, what do you do to ensure that your students develop good airmanship?

9. Please describe a flight, that you observed or know about, in which good airmanship was displayed, including why you feel it was particularly good airmanship.

10. Please describe a flight, that you observed or know about, in which poor airmanship was displayed, including why you feel it was particularly poor airmanship.

11. Any other comments?

272

Page left deliberately blank

273

Appendix C: Airmanship Questionnaire reported in Chapter 3

Airmanship Survey

Thank you for deciding to complete this questionnaire. Please consider that what we want is your personal view, not that of an employer or organisation, or based on what you might have read, but what you, yourself, think.

When you have finished, save the survey and send as an attachment to kirstie.carrick@newcastle.edu.au

The survey will be separated from any identifying material (like email address) before analysis.

Demographics: we would like to know a bit about you so we can compare different groups of pilots.

Copy and paste as required

1. Most recent Licence level: (tick one)

Unlicensed

Student

GFPT

PPL

CPL

ATPL

Military

Other _________

2. Are you an instructor or trainer (or have you ever been)?

Instructor rating Yes No

Military QFI/FCI Yes No

Check or training role Yes No

If you are no longer in this role, in what year did you stop? ________

274

3. Where did you do your initial flight training?

Australia

Other

4. Was your initial flight training (tick one)?

Military or

Civilian

5. What year did you start your flying training? ________

6. Military experience (if any): Years of service: ____

Hours flown: ________

7. Civilian experience: Years of experience: _______

Hours flown: _______

8. If you no longer fly, in what year did you stop? ______

9. How would you describe your current aviation activity (or most recent if no longer flying)?

Recreational

Instruction

Aerial work

Charter

Regional airline

Major airline

Military instruction

Military

10. Any Comments? ______

275

Part 2

Questions about airmanship – please type in your responses as if you were speaking to someone. Just start typing on the line below the question.

1. What do you think airmanship is? (Go into as much detail as you want)

2. Thinking of your response to the first question, can you elaborate on each of the key words in your response? For example, if you wrote ‘being safe’, write in more detail what ‘being safe’ would involve. (Go into as much detail as you want)

3. (a) In your initial flight training, was airmanship, or concepts related to airmanship, discussed in ground training or pre- or post- flight briefings?

3 (b) If so, which concepts, and what sorts of discussion?

4. Was the development of airmanship a specific part of the initial ‘in the air’ flight training you did? If so, please explain what this involved.

5. Thinking about your progress as a pilot, what do you feel led to the development of your sense of airmanship?

6. Do you participate in ongoing training? Does this include airmanship and if so, what does this entail?

7. In your current aviation activities, are you normally actively aware of airmanship as part of your operation? What would be the most important things that come to mind? If you are no longer flying, were you normally actively aware of airmanship as part of your operation?

276

8. During what phases of flight or events would you be most actively aware of airmanship issues?

9. If you are currently an instructor or hold a check/training role, what do you do to ensure that your students develop good airmanship? (or if you were an instructor, what did you do?)

10. Have your ideas about airmanship changed over time? If so, in what way?

11. Please describe a flight, that you observed or know about, in which good airmanship was displayed, including why you feel it was particularly good airmanship.

12. Please describe a flight, that you observed or know about, in which poor airmanship was displayed, including why you feel it was particularly poor airmanship.

13. Have you any other comments?

Scroll down for Part 3

277

Part 3

14. Please rate the importance of each of the following concepts or ideas with regard to your interpretation of airmanship.

Please enter an X in one of the boxes for each concept (one per line).

Not important

Marginal Moderate Quite Extremely important

Awareness of own medical fitness

Awareness of own psychological fitness – e.g. personal stress

Flying Skill (‘stick and rudder’) Ability to operate automation Proficiency (currency and expertise)

Knowledge of your aircraft type Knowledge of environment (terrain, weather)

Courtesy Knowledge of regulatory/ organisational environment

Risk assessment Self-improvement Team management Communication with other aircraft

Situational awareness Discipline Judgement and decision making Crew interrelationships Personal attitude/qualities Communication with aircraft maintenance personnel

Communication with ATC Assertiveness Common sense Cooperation Maintain lookout Preparation/ planning Responsibility

278

15. If you had to pick the five most important concepts, which would you pick? Please number from 1 (most important) to 5 (fifth most important). Just number 5, don’t number the rest!

Rank 1-5

Awareness of own medical fitness Awareness of own psychological fitness – e.g. personal stress Flying Skill (‘stick and rudder’) Ability to operate automation Proficiency (currency and expertise) Knowledge of your aircraft type Knowledge of environment (terrain, weather) Courtesy Knowledge of regulatory/ organisational environment Risk assessment Self-improvement Team management Communication with other aircraft Situational awareness Discipline Judgement and decision making Crew interrelationships Personal attitude/qualities Communication with aircraft maintenance personnel Communication with ATC Assertiveness Common sense Cooperation Maintain lookout Preparation/ planning Responsibility

16. Any final comments?

Thank you for participating in this research! Results will be published when analysis is completed and a summary sent for publication in Flight Safety Australia.

279

Appendix D: Interview Schedules reported in Chapter 4

Interview Questions – Military Ab-initio Participants Case no: _____

Press Record!

Demographics:

For how long have you been an Instructor? Mil or Civ?

Have you any academic qualifications in aviation, BAv, MAvMg?

What & from which Uni?

1. Can you describe your concept of airmanship?

2. What do you think are the main components of airmanship? Can you say what each component is (eg what is SA?)

3. What skills are included?

4. Does it encompass knowledge as well?

5. What about attitudes?

6. Is there anything else that you can add to those components?

7. Please give an example of a situation in which good airmanship was displayed.

8. Please give an example of a situation where the pilot displayed poor airmanship.

9. Do you talk about ‘airmanship’ specifically during the pre-flight briefing, if so what sorts of thing do you mention?

10. What aspects of airmanship do you think develop first in trainees?

11. What aspects of airmanship do you think develop next?

12. What is it about (each airmanship component) that you are concerned with when assessing trainees?

280

13. What sorts of behaviours do you look for in trainees when evaluating airmanship? Eg what evidence or behaviour do you look for that the trainee is demonstrating good situational awareness?

14. The findings of the 2006 survey at FTT indicated that decision making, safety and situational awareness were most often mentioned in relation to airmanship assessment – would you change the order or add anything else?

15. The analysis of PT51 forms in 2006 showed that mention was made mostly of checks, situational awareness and radio calls when giving feedback on airmanship, would you agree that these are representative of airmanship? What else do you think could be used to evaluate airmanship?

16. Do you think that your assessment of and feedback to trainees has altered since the feedback given in late 2006?

281

Interview Questions – Sector Military Advanced Participants Case no: _____

Demographics:

For how long have you been an Instructor?

Military only, or any civilian?

Have you any academic qualifications in aviation, BAv, MAvMgt?

What & from which Uni?

1. What do you think airmanship is?

2. What do you think are the main components of airmanship?

3. What skills are included?

4. Does it encompass knowledge as well?

5. What about attitudes?

6. Is there anything else that you can add to those components?

7. Can you walk through a flight telling me what you would expect to see in terms of good airmanship?

8. Please give an example of a situation where a pilot displayed poor airmanship.

9. Do you talk about ‘airmanship’ specifically during the pre-flight briefing, if so what sorts of thing do you mention?

10. What aspects of airmanship do you think develop first in trainees?

11. What aspects of airmanship do you think develop next?

12 Do you assess airmanship when instructing trainees?

13. What is it about (each airmanship component) that you are concerned with when assessing trainees?

14. What sorts of behaviours do you look for in trainees when evaluating airmanship? Eg what evidence or behaviour do you look for that the trainee is demonstrating good situational awareness?

282

15. The findings of a survey of instructors who are required to assess airmanship indicated that decision making, safety and situational awareness were most often mentioned in relation to airmanship assessment – do you agree or would you add something else?

16. An analysis of feedback given to such trainees showed that mention was made mostly of checks, situational awareness and radio calls, would you agree that these are representative of airmanship? What else do you think could be used to evaluate airmanship?

283

Interview Questions – Civilian Ab-initio Participants Case no: _____

Demographics:

For how long have you been an Instructor? 1 to 5, 5 to 10 or 10 to 15 yrs, more?

Have you any academic qualifications in aviation, BAv, MAvMgt?

What & from which Uni?

1. What do you think airmanship is?

2. What do you think are the main components of airmanship?

3. What skills are included?

4. Does it encompass knowledge as well?

5. What about attitudes?

6. Is there anything else that you can add to those components?

7. Can you step me through a flight, telling me what you would expect to see in terms of good airmanship.

8. Please give an example of a situation where a pilot displayed poor airmanship.

9. Do you talk about ‘airmanship’ specifically during the pre-flight briefing, if so what sorts of thing do you mention?

10. What aspects of airmanship do you think develop first in trainees?

11. What aspects of airmanship do you think develop next?

12 Do you assess airmanship when instructing trainees?

13. What is it about (each airmanship component) that you are concerned with when assessing trainees?

14. What sorts of behaviours do you look for in trainees when evaluating airmanship? Eg what evidence or behaviour do you look for that the trainee is demonstrating good situational awareness?

15. The findings of a survey of instructors who are required to assess airmanship indicated that decision making, situational awareness and safety were most often

284

mentioned in relation to airmanship assessment – do you agree or would you add something else?

16. An analysis of feedback given to such trainees showed that mention was made mostly of checks, situational awareness and radio calls, would you agree that these are representative of airmanship? What else do you think could be used to evaluate airmanship?

285

Interview Questions – Civilian Advanced Participants Case no: _____

Demographics:

For how long have you been a pilot?

For how long have you been in check and training?

Have you any academic qualifications in aviation, BAv, MAvMgt?

What & from which Uni?

1. What do you think airmanship is?

2. What do you think are the main components of airmanship?

3. What skills are included?

4. Does it encompass knowledge as well?

5. What about attitudes?

6. Is there anything else that you can add to those components?

7. Can you step me through a flight, telling me what you would expect to see in terms of good airmanship.

8. Please give an example of a situation where a pilot displayed poor airmanship.

9. Do you talk about ‘airmanship’ specifically during the pre-flight briefing, if so what sorts of thing do you mention?

10. What aspects of airmanship do you think develop first in trainees?

11. What aspects of airmanship do you think develop next?

12 Do you assess airmanship specifically during your check rides?

13. What is it about (each airmanship component) that you are concerned with when assessing pilots?

14. What sorts of behaviours do you look for in pilots when evaluating airmanship? Eg what evidence or behaviour do you look for that the pilot is demonstrating good situational awareness?

286

15. The findings of a survey of instructors who are required to assess airmanship indicated that decision making, situational awareness and safety were most often mentioned in relation to airmanship assessment – do you agree or would you add something else?

16. An analysis of feedback given to such trainees showed that mention was made mostly of checks, situational awareness and radio calls, would you agree that these are representative of airmanship? What else do you think could be used to evaluate airmanship?

287

Appendix E: Conference papers based on data reported in Chapter 2

Human factors as a component of airmanship CARRICK, K. University of Newcastle, Australia. Kirstie.carrick@newcastle.edu.au

Airmanship is a quality widely assumed to be held by pilots of all license levels. Airmanship is not a specific part of the theory syllabus for flight licensing but is supposed to be included in pre-flight briefings during flight training. Assessment of airmanship in licence flight tests has been a subjective assessment by the testing officer. Human factors (HF) and threat and error management (TEM) are covered in the human performance and limitations part of the theory syllabus and will be tested from mid-2009 in licence flight tests. Criteria to assess competency in HF and TEM during flight tests are being introduced by CASA as being part of ‘old-fashioned good airmanship’. What, then, do Australian pilots think ‘airmanship’ consists of, and does this include the five elements that will be assessed? An on-line survey that asked about pilots’ concepts of airmanship was promoted at CASA safety seminars and in the CASA Flight Safety magazine and resulted in 40 responses. The survey found that the elements of HF identified by CASA for assessment were reflected in pilots’ definitions of airmanship or in discussions about airmanship training, but each of the elements was mentioned by fewer than half the respondents.

Airmanship is a quality widely assumed to be held by pilots of all license levels. The International Civil Aviation Organization (ICAO) defines airmanship as ‘the consistent use of good judgement and well developed skills to accomplish flight objectives’ (ICAO, 2006). A military definition is ‘the safe and efficient operation of an aircraft, in the air and on the ground’ (BAE Systems, 2006). To become a pilot, one has to pass a sequence of theory exams, undergo flight training and pass a series of flight tests. During this process, it has generally been hoped that good airmanship would be learned by experience and from the example of instructors and other pilots. Airmanship itself, however, remains a rather elusive quality when it comes to trying to pin down specifics (Kern, 1996).

Safety is certainly a common theme amid discussion of airmanship and definitely the desired outcome. In aviation, it is much to be preferred that the flight conclude with the aircraft safely on the ground at the end of a flight, than not. If anything does go wrong, it is the Captain and crew who are expected to display good airmanship and bring the stricken aircraft safely through the flight. Accident analysis shows that 80% or more of aircraft accidents involve some human factor among the causes. Conversely, where a flight has experienced an external threat (like a bird strike) or an internal error on the flight deck, 100% of flights that did not crash (or have been survivable) have involved humans saving the day and landing safely, sometimes in circumstances that constitute what James Reason calls ‘heroic recoveries’ (Reason, 2008). So while the human factor can be a source of problems, it is definitely a part of the solution.

In recognition of the role of human factors in safety and in accidents, theory training for pilots for the last 20 years has included the study of human factors. In Australia human factors forms a part of the theory syllabus called ‘Human performance

288

and limitations’ (HPL). The HPL syllabus covers aspects such as human physiological response to high altitude, perceptual problems like visual and vestibular illusions due to ambiguous visual cues and unusual aircraft attitudes, and issues relating to information processing such as memory, attention and decision making and the effects of fatigue, alcohol and stress on human performance. Consideration of communication, situational awareness and threat and error management are also part of the syllabus.

In the Australian civilian aviation industry ‘airmanship’, as such, is not part of the theory syllabus for flight licenses. However it is supposed to be included in pre-flight briefings during flight training (CASA, 2006) and Licence flight tests have included a subjective assessment of airmanship carried out by the testing officer.

From mid-2009, the human factors (HF) and threat and error management (TEM) issues covered in the human performance and limitations part of the theory syllabus, will now be explicitly tested in pilot licence flight tests. The criteria to assess competency in HF and TEM in flight tests are being introduced by the Civil Aviation Safety Authority (CASA) on the basis of being part of ‘old-fashioned good airmanship’ (CASA, 2008) and the suggestion is that for “CASA to link airmanship to human factors, is in effect, tantamount to bringing science to the often nebulous concept of airmanship” (CASA, 2008, p 8).

The aspects of human factors that CASA is going to require in flight testing from July this year are the following five elements:

• maintain effective lookout;

• maintain situational awareness;

• assess situations and make decisions;

• set priorities and manage tasks; and

• maintain effective communications and interpersonal relationships.

(CASA, 2008, p7) In addition, there will also be assessment of recognition and management of

threats, errors and undesired aircraft states, as the threat and error management part of the syllabus, but these will not form part of the current discussion.

Given the emphasis that the introduction of human factors testing is part of airmanship formulated into competencies, it would be interesting to know if the general aviator agrees with CASA on this. What, then, do Australian pilots think ‘airmanship’ consists of, and does this match the criteria that CASA is introducing?

Method. An on-line survey was developed to find out about the concepts of airmanship

among general aviation pilots in Australia, as part of a wider project on airmanship and its development and training. The survey was promoted at CASA safety seminars in the winter of 2008, and in the CASA Flight Safety magazine of May-June 2008. Contact details were provided to allow respondents who preferred to respond off-line to obtain a paper copy. The survey asked 11 open ended questions, including ‘what do you think airmanship is?’ Other questions asked about airmanship in initial flight training, development of airmanship, awareness of airmanship in current operations and also asked for examples of good and poor airmanship.

289

By September 2008, 40 responses had been recorded. Respondents held a variety of flight licences at all levels from 5 ATPL (Airline Transport Pilot Licence) holders, through 15 CPL (Commercial), 13 PPL (Private), 4 at GFPT (General Flying Proficiency Test) and 1 Student, one military pilot and one military ATC (Air Traffic Controller). One of the PPL holders was also an ATC. Only three of the respondents were initially trained in the military environment. Sixteen respondents held an instructor rating. The experience in civilian flying ranged from less than one year to 55 years, and from 28 hours to 22,000 hours (mean = 3776 hours, median 1100 hours).

Results Assessment of content and themes was carried out using NVivo7 as the medium

of analysis. Tree nodes, corresponding to the components of the Kern (1996) model of airmanship were established initially, with free nodes created as themes were discovered in the survey responses. A total of 36 nodes were created, each representing a concept that the respondents related to airmanship. The most frequently mentioned was safety, brought up by 25 respondents (or 62%). At the other end of the rankings, ‘human factors’ was specifically mentioned by only one respondent and error management by one. Clearly, the respondents had different ideas about what makes up airmanship. For the purposes of this paper, the nodes that correspond to the five areas or elements of human factors, defined by CASA and that will be included in flight testing from July 2009, will be explored.

All five elements of assessment for human factors were raised in discussion by the respondents. While none of the respondents mentioned all five of the CASA elements of competency, only seven omitted them completely. The two who mentioned four elements, both missed ‘set priorities and manage flight’. The majority of respondents (62%) mentioned two or more of the CASA elements of assessment, as shown in Table 1.

Table 1

Number of Elements Mentioned by Respondents to the Airmanship Survey Elements N of Respondents

All 0 Four 2 Three 9 Two 14 One 8 None 7

Table 2 shows some of the things that respondents wrote about when asked about airmanship. The left column shows themes related to the CASA elements of assessment and the right column shows some of the other themes mentioned by respondents. The themes are also ranked by number of respondents mentioning that theme.

290

Table 2: Themes related to Airmanship

‘CASA related’ Nodes N Rank Other Nodes N Rank Safety 25 1 2. Situation awareness 20 4 Relationships 22 2 1. Lookout 19 6 Legislation 20 4 5. Communication 16 7 Skill 12 9 Courtesy 13 8 Experience 12 9 3. Judgement/Decisions 10 14 Aircraft Knowledge 11 12 Planning 10 14 Common sense 9 18 4. Set priorities and manage flight

5 24 Responsibility 5 24

Discussion

Situational awareness, lookout and communication were frequently mentioned by respondents in writing about airmanship and are high in the ranking of themes; decision making or judgement was not mentioned quite so frequently; and ‘set priorities and manage flight’ was the least frequently mentioned of the five CASA elements of assessment.

CASA has included ‘Maintain situational awareness’ as an element of competency to be assessed in flight tests. Situational awareness can be defined as “the perception of the elements in the environments within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future” (Endsley, 1988, cited in Endsley, 1999, p258). More simply phrased, it is knowing what is going on in the aircraft and in the surrounding airspace, as well as what is likely to happen next, often referred to as being ‘ahead of the aircraft’. Twenty of the respondents, or 50%, made a total of 30 references to situational awareness being part of airmanship, in their responses to the survey. Some did not use those exact words, but it was clear what was meant. Examples: ‘I believe airmanship to be an awareness and understanding of an aviator of their surroundings and what is taking place’ or a negative phrasing of the same concept: ‘The only things that can kill a pilot are his lack of regard to the task at hand, other aircraft and birds’.

Lookout is the idea of looking outside of the aircraft to identify and avoid other aircraft or unexpected landforms, it can also be expanded to include ‘listen out’ or keeping track of the position of other aircraft by listening to radio calls on the local frequency. Only two respondents mentioned ‘lookout’ as part of their initial definition of airmanship, but a total of 19 (47%) mentioned lookout at some stage in the survey, generally in response to a question on the concepts included in initial flight training. Listen or listen-out was mentioned, once or twice, but was coded with ‘communication’.

The next most frequently mentioned element of competency was to ‘maintain effective communications and interpersonal relationships’. Communication was brought up by 16 respondents in their answers to various questions, with seven including communication specifically in their definition of airmanship. For example, one definition was, in part, ‘Practicing a good lookout [and] keeping in contact with other aircraft’ and another was ‘ability to communicate with other users of aircraft to achieve safe reliable outcomes’. Several of the examples of good and poor airmanship included

291

comment on communication – or lack of it. Examples: ‘clear concise radio calls in the ctaf area’, and on the other hand, ‘often strike pilots not using radios when landing on public strips’.

The other half of this element is concerned with ‘interpersonal relationships’. The elaboration of this element in the CASA document includes a competency on assertiveness to ensure safe outcomes, but otherwise repeats the importance of clear communication to passengers and other stakeholders (CASA, 2008). Relationships with others were mentioned by the respondents very frequently, but perhaps not in quite the way that the CASA Flight Testers would be looking for. Respondents to the survey included comment on the influence of a good relationship with their instructors when learning to fly, and the hanger talk experienced in flying clubs, on the development of their airmanship. A few respondents mentioned the effect on them of the deaths of friend and acquaintances in aircraft accidents, and how this led to them being more aware of good airmanship. Such comments were coded as ‘relationships’, and the table shows that they were mentioned almost as frequently as ‘safety’.

An interesting aspect in definition and discussion of airmanship was the inclusion by 13 of the respondents, (33% or roughly a third) of concepts to do with courtesy and respect, which relates closely to the idea of interpersonal relationships and communication. Some examples of definitions that included this theme are that airmanship is ‘A combination of flying skills, common sense and courtesy’ from one respondent and ‘Being alert and abiding by all the rules and being courteous to other pilots’ from another. ‘Doing the safe & right thing by yourself and to others, showing respect to other airmen’ was another definition. Such responses show that some importance is placed on the tone of communication and of interpersonal relationships, but not the same aspect as CASA considers in suggesting that Flight Testers should look for ‘assertiveness that ensures safe completion of the flight’(CASA, 2008).

The element of assessment ‘assess situation and make decisions’ was mentioned, or indicated but not using those exact terms, by 10 respondents or 25%, in discussing airmanship. For example mention was made of ‘thinking before doing’, coded as showing an awareness of judgement and decision making. More direct examples are comments like ‘I think of good judgment as good airmanship’ and ‘Airmanship is very individual but is usually referring to judgement and decision making more so than manipulative skill’.

Finally the element of competency ‘set priorities and manage tasks’ was infrequently mentioned. Managing the flight was specifically mentioned by only two respondents, one of these was: ‘Having a system of managing my flight from the flight plan all the way through till the aircraft is put away at the end of the day’. There were a further three who wrote a response that indicated a concern with ongoing management during flight, such as ‘The necessity to plan the flight and fly the plan is very important. Spontaneous or unthought out changes from the plan are what creates unsafe situations’ and ‘Airmanship is awareness of present, imminent, and future situations, giving priority as is needed to each element. Aviate, Navigate, Communicate, is one example’. More comment was made by respondents about planning, mentioned by 11 respondents (25%), in 16 references, although this was generally in reference to prior planning before flight. The implication of the CASA assessment elements was more towards ongoing management during flight.

292

Conclusion

The on-line survey responses do show that the elements selected by CASA for inclusion in flight testing are generally seen as part of airmanship by general aviation pilots. While none of them mentioned all of the elements in their discussions, all but 7 (83%) did mention at least one of the elements. The diversity of concepts brought up by respondents, and the lack of agreement in terms of mentioning the same components, shows the nebulous nature of ‘airmanship’. Even though only one respondent mentioned ‘human factors’ specifically, it is clear that many human factors concepts, like situational awareness, judgement and communication, are seen as important parts of airmanship.

This survey only asked open ended questions about airmanship, there could be different pattern of responses if a list to tick off had been included. CASA’s selection of human factors elements for assessment is discussed here only in terms of whether a sample of General Aviation aviators think that these are part of airmanship, and raised them spontaneously. In addition, it is a small sample of aviators and may not be particularly representative. It did not include pilots currently flying in major airlines, for example.

References BAE Systems (2006). Airmanship Introductory Lecture. [Powerpoint presentation]

Tamworth, Australia: Author

CASA (2006). Flight instructor manual. Canberra, Australia: Author

CASA (2008). CAAP 5.59-1(0) Teaching and assessing single-pilot human factors and threat and error management Retrieved 15 October 2008 from http://www.casa.gov.au/wcmswr/_assets/main/download/caaps/ops/5_59_1.pdf

Endsley, M.R. (1999). Situation awareness in aviation systems. In D.J. Garland, J.A. Wise, & V.D. Hopkin, (Eds) Handbook of Aviation Human Factors. Mahwah, N.J.: Erlbaum

ICAO (2006). Explanatory Factors. Retrieved 10 November 2006 from www.icao.int/anb/aig/Taxonomy/R4CDExplanatoryFactors.pdf

Kern, T. (1996). Redefining airmanship. New York; McGraw-Hill

Reason, J.T. (2008). The human contribution. Aldershot, UK: Ashgate.

293

Airmanship - Voices from an on-line survey

KIRSTIE CARRICK* AND KERRY CHALMERS

University of Newcastle

Background Airmanship is considered important for aviators (Ebbage & Spencer, 2003). Kern (1996) describes a model that provides a theoretical structure of airmanship, based on research conducted in the USA. Kern states that the fundamental principles of airmanship are: “skill, proficiency and the discipline to apply them in a safe and efficient manner” (p21). In addition, there are six areas of critical expertise, which are shown as pillars of knowledge: self, team, aircraft, environment, risk and mission (Kern, 2009). According to Kern (1996) it is the combination of the basic principles with expert knowledge that enables an aviator to maintain situational awareness and exercise good judgement in decision making, and thus display good airmanship. Ebbage and Spencer present a simplified model of airmanship with three main factors: discipline, control and judgement. Discipline includes self-improvement, vigilance, co-operation and confidence. Judgement covers situational awareness, problem solving, mental workload and foresight, while control refers to flying skills, automation skills, information management, navigation and communication skills. This model specifies broader skill sets than the Kern model. Both models have a similar emphasis on discipline. Ebbage and Spencer consider the foundations of airmanship to be the underlying knowledge, skills and attitudes and that these should be included in any flight training program.

The current research addresses whether Australian aviators consider airmanship to include similar concepts to Kern’s model and whether there are any additional themes specific to the Australian industry. Method An on-line survey was developed and promoted at CASA safety seminars in winter 2008 and in the Flight Safety magazine of May-June 2008. There were 40 responses from pilots by September 2008. The respondents varied in level of licence held, in length of time involved in aviation and hours of experience, from a student pilot on 28 hours through to an ATPL holder with 55 years in the industry and 22,000 hours experience (M = 17.8 years, 3776 hrs; SD = 13 yrs, 5518.6 hrs). Sixteen respondents held Instructor ratings. The survey asked 11 open ended questions about: pilots’ definition of airmanship; the inclusion of airmanship in initial training; development of airmanship in respondents; awareness of airmanship during current aviation activities; examples of good and poor airmanship; and finally any other comments. The survey responses were analysed using NVivo8 as the structure for determining concepts and themes raised in the responses. Initially, tree nodes were established to reflect the concepts included in Kern’s model. Further free nodes were created as other concepts were discovered in the responses. A total of 36 nodes were developed, each representing concepts that respondents related to airmanship or issues that they raised. Results The questions of most interest for this discussion are Question 1: “What do you think airmanship is?” and Question 2, which asked the respondent to elaborate on the answer given in Question 1. Other questions considered here include questions that asked about the most important aspects of airmanship (Question 6), the situations in which respondents were most aware of airmanship (Question 7) and about good and poor examples of airmanship (Questions 9 and 10). Questions 3 to 5 and 8 were more concerned with the development and training of airmanship, and are discussed elsewhere (Carrick & Chalmers, 2010).

294

Table 1 shows the variety of concepts found in the responses, the number of respondents who mentioned each concept in the Questions 1 and 2, and the total number of respondents mentioning the concept over all the questions. The table is set up to compare the structure of the Kern model, on the left, with some of the other concepts raised by respondents.

The main themes of the Kern model are reflected in some of the respondents’ comments, but there are clearly other concepts, represented in the right hand column, that were also expressed. No-one mentioned all of the Kern concepts. The mean number of concepts raised per respondent was 2.95 in Question 1. Ten respondents did not answer Question 2, the mean across the 30 who did was 2.90 concepts per respondent.

Safety was the concept with the most mentions by respondents in Question 1, Question 2 and across all questions. Kern did not include this in his model, possibly considering it an outcome of good airmanship rather than part of it. The survey respondents were quite concerned about safety with comments like: “Airmanship is a way of being safe”, “An aviator's own attitude towards his / her own safety and the safety of others” and “Sensible aviation practices that take into consideration the safety of yourself and others”.

Table 1

Comparison of Kern concepts and other concepts raised by respondents – number of respondents who mentioned each concept in Questions 1 and 2, and overall (Questions 1 to 11). N = 40

Kern Concepts Other Concepts Q1 Q2 Tot Q1 Q2 Tot Judgement or decision making 2 5 10 Safety 13 14 25 Situational awareness 10 9 22 Courtesy/ethics/respect 12 12 12 Knowledge 6 5 9 Attitude / personal quals 10 0 22 Self 2 3 5 Common sense 9 5 9 Aircraft 2 1 11 Communication 4 4 15

Team 0 0 1 Set priorities, manage flight 3 0 5

Environment 0 0 1 No shortcuts 2 3 3 Rules, regs 9 4 20 Lookout 2 1 19 Risk 3 5 10 Responsibility 2 0 5 Mission: Planning 9 10 16 Knowledge - medical 1 0 1 Skill & Proficiency 12 2 12 Discipline 4 4 5

Judgement or decision making is the major capstone outcome according to Kern, but was not considered by respondents to be particularly central to airmanship, in that only two mentioned it in response to Question 1 and five in their elaboration in Question 2. In contrast, situational awareness (SA) clearly is seen as a significant concept relating to airmanship, with 10 respondents mentioning it in Question 1, nine in Question 2 and 22 respondents including the concept over all questions. Some examples of comments are: “awareness of the operating environment”, “being aware of the situation around you i.e. the weather, other traffic” and “I believe airmanship to be an awareness and understanding of an aviator of their surroundings and what is taking place”. Lookout is an activity upon which SA is dependent, in VFR, and is also frequently mentioned although not so much in response to the first two questions, being mentioned three times, with 19 respondents including it over all questions.

Thirteen respondents referred to knowledge: six made mention of knowledge generally, others specifically related to self (2), the aircraft (2) or to risk (3), with one person commenting on knowledge about medical aid (not something Kern was concerned about). There were no references to knowledge of the team in responses to the definition questions, but this may have

295

been a function of the mostly general and recreational aviation sample, flying mostly single pilot. Nor was there specific reference to the environment. However, when elaborating upon the model, Kern divides environment into the physical, regulatory and organisational environments. When considered at this level, nine respondents mentioned rules, regulations and operating procedures in their definition, and 20 overall.

So far there has not been a published definition of “mission” by Kern, just the inclusion of it in the diagrams presented at a recent seminar series (Kern, 2009). Those respondents who included flight planning, planning generally, preparation and thinking about what one was doing in their responses are shown in “mission” in Table 1. Clearly there is consideration among respondents that planning of what will be going on in flight is part of airmanship, even if it may not have been referred to as “mission”.

Skill and proficiency are separate items in Kern’s model. Twelve respondents to the survey included skill in their definitions of airmanship, or stated “ability to control the aircraft”, “aircraft handling” or “operate an aircraft”. One of these stated that airmanship “is usually referring to judgement and decision making more so than manipulative skill”. Only one respondent specifically mentioned proficiency: “as a pilot without a lot of hours I need to fly regularly enough to remain competent”. One respondent gave as an example of good airmanship: “a flying competition that required pilots to touchdown as close to a spot as possible on the runway” which indicates a skill based component within their conception of airmanship.

Discipline is the cornerstone of airmanship, according to Kern (1996), and it was included in their definitions by four of the respondents, with added comment in response to Question 2 for example: “airmanship is the discipline that you apply to your flying” and “Discipline, implying self discipline should be exercised at all times as an essential element of airmanship”. Three other respondents wrote of “not cutting corners” or “no short-cuts” which could be interpreted to indicate the application of discipline to stick to the task in full.

In addition to the Kern model concepts, the respondents included a number of other concepts in their definition and elaboration of airmanship. There were many mentions of a complex of ideas that can be represented as courtesy, as well as mention of courtesy itself, such as consideration of other airspace users, helping others, manners, doing the right thing, or ethical behaviour (elaborated as “honest in all dealings and treat others as you would wish to be treated”). Courtesy was always mentioned as part of a complex that makes up airmanship (rather than alone) such as the definitions: “A combination of flying skills, common sense and courtesy” and “Being alert and abiding by all the rules and being courteous to other pilots”.

Common sense is a concept that was included by nine of the 40 respondents. Unfortunately common sense is not always very common. Three of the nine elaborated on common sense in Question 2: “Know your abilities and operate within that area of competence” and “the ability to think a situation through logically”. The other response was just another mention of it: “Recognition of the need to apply common sense”. The respondents who included comments on common sense in their responses tended to have been involved in aviation for longer (M = 25.8 years; SD = 15.1) and to have more hours of experience (M = 6213 hours; SD = 8697) than the respondents who did not mention common sense (M = 15.6 years, SD = 11.6; and M = 3068 hours, SD = 4138). What may be considered common sense to someone who has been in the industry for a number of years, may not yet have been learned by a novice. The use of the term may reflect that airmanship has become an ingrained part of the more expert individual to the extent that they do not remember what it was like to be a novice.

Kern discusses communication when writing about characteristics of good team members. Communication is rather more active than just a “knowledge of” item would imply, and is included as a skill set by Ebbage and Spencer (2003). Good communication is essential to maintaining situational awareness and thus safety. This is recognised by the six respondents who mentioned communication in their initial definitions or elaboration, and by 15 respondents overall. Examples of such responses to Question 1 include: “Ability to communicate with other users of aircraft to achieve safe reliable outcomes” and “keeping in contact with other aircraft”.

Some respondents mentioned attitude and various personal qualities as forming part of airmanship, for example: “A combination of knowledge, discipline and attitude toward flying”

296

and “Attitude to safety and fellow aviators.” This category also included other personal characteristics, for example: “Furthermore good airmanship is a reflection of the quick thinking never panicking pilot, who seems to have a dogged determination to keep his plane and contents in a safe configuration”. There was also mention of a desire to do as good a job as possible and having a sense of responsibility.

Finally, the responses included the idea of setting priorities and managing the flight, one of the human factors competencies for flight tests being introduced by CASA (Carrick, 2009), although only mentioned by five respondents overall. The other CASA human factors competencies are lookout, situational awareness, decisions and communication, all mentioned by respondents as part of airmanship.

Table 2 indicates the main concepts raised by respondents when asked: “In your current aviation activities, are you normally aware of airmanship as part of your operation? What would be the most important things that come to mind?” (Question 6). Table 2 also indicates the concepts mentioned in the examples of good and poor airmanship (Questions 9 and 10).

Situational awareness and lookout are frequently mentioned in responses to these questions, supporting the emphasis given to the concepts in the earlier questions. Similarly there were frequent mentions of planning as being important to airmanship, and standard operating procedures (SOPs) and checks were mentioned as frequently, somewhat related to rules and regulations, although these were mentioned only once. Again, there were few explicit mentions of skill or discipline. Communication, or lack of it, was another major issue raised in examples of good and poor airmanship.

Table 2

Concepts raised in Question 6 or in the examples of good and poor airmanship (Questions 9 and 10) N = 40

Concept Frequency

Q6 (Important)

Q9 (Good)

Q10 (Poor)

Judgement/Decision 1 2 Situation awareness 9 8 5 Lookout 9 1 Communication 2 8 6 Knowledge 2 Rules, regulations 1 Planning 8 1 1 SOPs / Checks 9 Skill 1 1 Discipline 1 Attitude/personal quals 1 1 3 Courtesy etc. 1 3 Safety 1 1 Other 8

297

Table 3

Frequency of mention of scenarios or phase of flight in Questions 7, 9 or 10, N = 40

Concept Frequency

Q 7 (When)

Q9 (Good)

Q10 (Poor)

Weather/light 2 9 6 Preflight/planning 5 3 Taxi/runway use 2 5 Circuit / CTAF 8 9 10 Traffic 5 2 1 A/c problem 3 1 Approach/landing 5 1 1 Take-off / landing 8 All the time 10 Agricultural flying 2 1 Non-specific 4 3 Other 4 2 No response 1 10 13

The scenarios shown in Table 3 were mentioned by respondents in answer to Question 7 (“In your current aviation activities, during what phases of flight or events would you be most actively aware of airmanship issues?”) and in the examples of good and poor airmanship (Questions 9 and 10). There is frequent mention of weather and fading light, especially in the examples. Concern about flying in the circuit (landing and take-off could be included here) and in high traffic situations shows the importance of airmanship in interacting with other aircraft and their pilots. A gratifying 25% of the respondents stated that they were aware of airmanship at all times.

Conclusions

While the concepts included by Kern (1996) form a useful structure for consideration of airmanship, based on the responses to this survey it appears that airmanship in Australian aviation is more broadly defined than Kern’s model might suggest. There is little emphasis on discipline, despite its central role in both of the two models of airmanship, which may reflect the primarily civilian sample. Communication appeared as an important concept, it is included in the Ebbage and Spencer (2003) model and perhaps should be more explicitly included in the Kern model. The occurrence of the concept of courtesy shows an awareness of the importance of good interpersonal interaction among the sample group. The frequency of mention of personal qualities and attitudes as part of the overall airmanship construct is also interesting. It implies that airmanship is integral to an aviator’s personality and lifestyle, rather than just something that one does or has when flying. Use of the term “common sense” also reflects this idea of integration within the individual.

298

References

Carrick, K. (2009). Human factors as a component of airmanship. Paper presented at The 8th Industrial and Organisational Psychology Conference, Sydney, 25-28 June, 2009.

Carrick, K. & Chalmers, K. (2010). Development of airmanship. Paper presented at The 9th International Symposium of the Australian Aviation Psychology Association, Sydney, 18th – 22th April, 2010

Ebbage, L. & Spencer, P.D. (2003). Airmanship training for modern aircrew. Paper presented at the Advanced Technologies for Military Training Symposium, Genoa, Italy, 13-15 October 2003.

Kern, T. (1996). Redefining airmanship. New York: McGraw-Hill

Kern, T. (2009). CASA Wrap-up. Power points from the CASA Error Management Roadshow. June 2009.

299

Development of airmanship

KIRSTIE CARRICK* AND KERRY CHALMERS

University of Newcastle

Background

Effective development of airmanship among aviators is important if airmanship is considered an essential quality for safe and efficient flight. Ebbage and Spencer (2003) discuss some ways in which the development of airmanship can be encouraged in military pilots, suggesting that it should be an integrated part of overall flight and ground training rather than a by-product of conventional training. Kern (1996) also promotes the integration of airmanship within ongoing aircraft handling skills training, and suggests three phases of airmanship training. These are: for the instructor to explain the importance of airmanship and instil motivation for improvement; model and teach airmanship; and to evaluate airmanship actively and aggressively. Kern emphasises the importance of the instructor as a mentor to introduce a sound aviation culture, model appropriate behaviours and enforce standards and expectations. Ebbage and Spencer parallel these three phases, also proposing three basic elements to airmanship training: explanation of the concept and its importance, by discussion of case studies; instruction covering the required knowledge, skills and attitudes; and use of objective assessment with provision of feedback. Kern encourages the use of his model of airmanship as a template for the structuring of pre-flight briefings and post-flight debrief and reflection, an activity that should continue throughout one’s flying career, not just take place during initial training.

As part of an ongoing study of airmanship in Australian aviation, aviators were asked to participate in an on-line survey that included questions about the development of airmanship. The aim was to identify the airmanship instruction and development experiences of current Australian pilots.

Method

An on-line survey was promoted at CASA safety seminars in May to September 2008 and in the Flight Safety Australia magazine of May-June 2008. This resulted in 40 responses by the end of September 2008. The experience of respondents in civilian flying ranged from less than one year to 55 years (M= 17.8 yrs, SD = 13.0) and from 28 hours to 22,000 hours (M = 3776 hours, SD = 5518.6). Sixteen of the respondents held an instructor rating (5xCFIs, 4xGrade 1, 2xGrade 2 and 5xGrade 3). The survey comprised of 11 open-ended questions asking respondents about their interpretation of the term “airmanship” (Carrick & Chalmers, 2010) and also about the development of airmanship, the focus of this paper. Questions of interest here are: the inclusion of airmanship in initial training (Questions 3 and 4); factors influential in the development of airmanship in respondents (Question 5); and, for the instructors who responded, how they ensured the development of airmanship in their students (Question 8).

300

Results Inclusion of airmanship in initial training

The majority of respondents stated that airmanship formed some part of their initial flight training. The questions were designed to determine what happened in the air, as part of flying instruction (Question 3), as distinct from what was included in briefs or ground instruction (Question 4).

Question 3: Was the development of airmanship a specific part of the initial flight training you did? Please explain what this involved.

Eighteen respondents stated “yes”, some with added explanation. Three listed airmanship related activities such as checks, lookout and procedures. A further six responses were along the lines of “not really, but…” and included concepts to do with airmanship. Two respondents indicated that airmanship was part of their initial instruction, but that this was only skill related or did not include human factor related concepts. This gives a total for inclusion of airmanship (however defined) as 29, or 73% of the respondents. Eight respondents indicated that airmanship was not addressed (20%). Two respondents indicated that it was so long ago that they could not remember, one stating: “I cannot recall that airmanship was specifically discussed when I was learning to fly in 1954. I sometimes wonder if common sense was more expected in those days.” One respondent did not answer the question. Examples of positive responses are: “Yes. From the start my instructors emphasised that airmanship was what defined a good pilot and they demonstrated that right through my training: lookout, ground checks, aircraft handling and so on were all part of each flight” and “in each brief, airmanship was mentioned in regard to lookout, communications and aircraft awareness”. Several responses to this question included mention of briefs, so the distinction between Questions 3 and 4 was not necessarily picked up by the respondents. Question 4 (a) In your initial flight training, was airmanship, or concepts related to airmanship, discussed in ground training or pre- or post- flight briefings?

There was a response of “Yes” from 67.5% of respondents (N=27) and “No” from 25% (N=10). Three respondents did not answer the question. One of those did not answer any questions except Question 1, the other two had answered Question 3, one with “don’t remember”, the other with “no”.

Question 4 (b) If so which concepts and what sort of discussions?

Nine respondents left this question blank as they had replied “No” to Question 4a (or did not answer), a further four said that airmanship was not a major item, or only discussed in a general sense and two simply mentioned discussion in the bar or “hanger talk” as the source of information about airmanship.

Most respondents included more than one item in replying to Question 4b. The most frequent concept mentioned was lookout listed by 14 respondents. Related responses were: traffic (4 mentions), communication or radio use (4), and listen-out (2). Checks were listed by six respondents and procedures by two. Safety issues were mentioned by three respondents. Planning was included by four respondents. There were various other considerations including: handing-over/taking-over (2); and one mention each of: developing a flow of activity; relationship with instructor; fuel logs;

301

courtesy; and common sense. Two respondents noted that the content was specific to the lesson.

There were ten aircraft or skill related comments including general handling (2 mentions) and understanding of systems (2), and one mention each of: using airworthy aircraft; aircraft awareness; taxi speeds and run up into wind; aircraft performance; temperatures and pressures; and energy and fuel state. Five of these respondents did not include any non-aircraft handling concepts.

Most respondents had some discussion of airmanship in their initial training, but for some of the respondents airmanship was not discussed, or was limited to aircraft handling or aircraft state issues alone. While these are part of airmanship, a broader view of airmanship includes the human factors aspects of situational awareness and judgement. Of the five respondents who only listed aircraft or handling issues in response to these questions, only one really had the broader view when the definitions given in the first questions defining airmanship were reviewed. Two wrote a little about issues that could be interpreted as situational awareness and the other two gave quite narrow, aircraft handling related definitions. Two of these respondents had over 20 years of experience, but the other three were instructed fairly recently, with 2.5, 3 and 6 years of experience as PPL holders. Given that human factors has been part of the syllabus for the PPL, as human performance and limitations, for approximately 20 years, it is interesting that there are pilots who have not incorporated these concepts into their understanding of airmanship. On the other hand, it may be that their instructors had not discussed human factors in terms of airmanship, or airmanship in terms of human factors.

Looking at the responses from another angle, there were at least 12 respondents who made no mention of aircraft handling skills when writing about the discussion of airmanship in their initial training. This may be a reflection of the military training system of assessing technical skills and airmanship as separate aspects of flight proficiency (Carrick, Graham, Healey & Pickard, 2008). Although only three respondents indicated a background in the military, it may be that the instructors of the non-military respondents had a military background, and so perpetuate the separation of technical skill and airmanship.

Influences on the development of airmanship

When asked what had led to the development of their own sense of airmanship, respondents mentioned experience, reading, developing a sense of responsibility and the influence of the people around them. Self-preservation and fear were also included.

Question 5: Thinking about your progress to date as a pilot, what do you feel led to the development of your sense of airmanship?

As it was an open-ended question, several respondents listed more than one influence on their development. Only one person did not provide a response. Experience was included by 13 of the respondents. Ten respondents listed a personal desire to get it right, or developing maturity or responsibility. The influence of other people was commonly included as: their training (by six respondents); talking to others (6); reading (6); the mistakes or experiences of others (5); losing a friend or colleague in an accident (3); and the Club environment (2). Self-preservation or fear was mentioned by five respondents. Helping to teach others was included by three respondents and common sense also by three respondents. One person mentioned luck, and another simply wrote

302

“leaving a fuel cap off”. Some examples are: “Fear of having an accident because of a minor mistake, and a desire to be [as] prepared and professional as possible”; “Watching and talking with other pilots to develop my own ideas of airmanship from examples around me I identified with [and] being scared and not wanting to go through a similar experience again”; and “Common sense and experience. Be ready for the unexpected.”

Ongoing experience in aviation is seen as important in developing a sense of airmanship, but also the observation of what others do, whether these others are initial instructors, fellow aviators, or observed through reading. This supports the comments of both Kern (1993) and Ebbage and Spencer (2003) that early instructors provide a significant role model, but extends further to imply that learning from others continues as the aviator develops post ab-initio training. There is also a clear internal motivation to improve or excel, to be as professional as possible, among some of the respondents. This is included by Ebbage and Spencer as one of the attitudes that form the basis of airmanship and that should be encouraged in initial training.

Practice of current instructors

Instructors reported that they ensure the development of good airmanship among their students by modelling good airmanship themselves, as well as by discussions and briefs during training.

Question 8: If you are currently an instructor, what do you do to ensure that your students develop good airmanship?

There were 16 instructors among the sample, and 17 responses and 23 not applicable or nil answers to this question. One instructor did not answer the question, and two respondents who were not instructors, did and stated that they intended becoming instructors. Of the 17 responses, eight respondents wrote that they provide good example. Two mentioned getting their students to read, one specifically mentioned the book, Redefining Airmanship (Kern, 1996). Five respondents listed discussion, briefs and in-flight emphasis and one wrote that he makes sure that students have a system for managing flights. There were single mentions of the following: right attitude and clear understanding; encourage them to think; try to instil points mentioned; part of post-flight self-evaluation; and “keep yourself alive and keep a good lookout”. One person noted that he “hit them (not very hard) when they turn without looking (they all get hit at least once).”

The instructors who responded to the survey did not mention either model of airmanship in responding, although one did mention Kern (1996) as a resource for students. While the pattern of instilling motivation, teaching the skills and providing feedback may be part of the instructor training system and be a recommended structure for instruction, there were only five respondents who wrote about briefs, and one who specifically mentioned post-flight self-evaluation. The role of an instructor as mentor or model, providing a good example, was included by eight of the respondents, or 50% of instructors, so acceptance of this aspect of developing airmanship is more widespread.

303

Conclusions

Based on the responses to this survey, it appears that airmanship is generally included in student instruction in Australian aviation, but perhaps not consistently across all training establishments, given that 20% of respondents stated airmanship was not part of their initial training. In addition, individual experiences and the quality of airmanship displayed by the people with whom the student pilot associates is clearly influential in the development of airmanship, and continues to be important after initial training.

The close examination of responses to Questions 3 and 4 indicates that where the initial training did include airmanship concepts, it did not always reflect a truly integrated view of airmanship. There appear to be three groups of respondents: those for whom airmanship was presented as an integration of flying skills and human factors concepts; those for whom airmanship referred only to flying skills and aircraft knowledge related items; and those for whom airmanship referred to only the human factors related concepts, not including technical flying skills. Responses to the initial questions on the survey asking for a definition of airmanship included only 12 mentions of skill or technical proficiency (Carrick & Chalmers, 2010); thus 28 responses did not include these aspects in their initial definition of airmanship and so probably consider airmanship to be predominantly about non-technical aspects of flying.

It should be noted that this survey concluded before the CASA Safety Seminars addressing airmanship occurred, and that those seminars did present an integrated model of airmanship. Of the respondents who had some level of integration in their responses, none outlined fully either of the two models that have been presented here. It would be interesting to investigate whether there has been any change in pilots’ conceptions of airmanship following the CASA airmanship Safety Seminars and an article on airmanship in the Flight Safety Australia magazine by Marchbank, (2008). The CASA Error Management Roadshow featuring Tony Kern in June 2009 also presented an integrated model of airmanship, and a parallel model for maintenance engineers, in the closing session of the day.

A limitation of any generalisation from these results is that it is a very small sample, recruited from among those interested enough in safety to attend CASA Safety Seminars, or to read Flight Safety Australia, and thus may not reflect the views of the wider aviation community. Also it is possible that the respondents were writing only the most important aspects of their reactions to each question, and were not willing to take much time to explain their activities or views in depth. Most respondents wrote no more than a few words or a single sentence in response to the questions asked. By comparison, interviews tend to elicit far more information and allow additional questions to be asked. However, interviews are not very practical for an extensive survey.

304

References Carrick, K. & Chalmers, K. (2010). Airmanship: Voices from an on-line survey. Paper

presented at The 9th International Symposium of the Australian Aviation Psychology Association, Sydney, 18th – 22th April, 2010

Carrick, K, Graham, S., Healey, A. & Pickard, S. (2008) An assessment of feedback and evaluation at a flight training school. The 8th International Symposium of the Australian Aviation Psychology Association, Sydney, 8th – 11th April, 2008

Ebbage, L. & Spencer, P.D. (2003). Airmanship training for modern aircrew. Paper presented at the Advanced Technologies for Military Training Symposium, Genoa, Italy, 13-15 October 2003.

Kern, T. (1996). Redefining airmanship. New York: McGraw-Hill

Kern, T. (2009). CASA Wrap-up. Power points from the CASA Error Management Roadshow. June 2009.

Marchbank, M. (2008, Nov-Dec) Is airmanship dead? Flight Safety Australia, 65, 8-13

305

Appendix F

Survey reported in Chapter 6

306

307

308

309

310

311

312

Page left deliberately Blank

313

Appendix G

Between and within domain differences from Chapter 6

An expectation of the data was that the ‘within-domain’ relatedness would be

greater than the relatedness of the relevant pairings with concepts from other domains

(between-domain). This can be explored on three levels: concept level, domain level

and overall. At the concept level, each concept has two within-domain comparison

means (shown in bold) and a row and column of the mean relatedness of the between-

domain comparisons, shown in Table 6.4. Ideally, the bold means should be greater than

the non-bold means. Secondly, each domain has three within-domain relatedness means,

and four sets of nine between-domain comparisons (represented by the intersections of

the domain names above the columns and beside the rows in Table 6.4). The within-

domain means should exceed the between-domain means, and this can be assessed by

calculating a ‘mean relatedness’ for each set and comparing these means through

analysis of variance. Thirdly, the overall the within-domain relatedness mean should

exceed the overall between-domain relatedness mean, a comparison between the means

along the diagonal and the non-diagonal un-bolded means. A combined relatedness

variable can be calculated for each of these and a comparison conducted.

Overall.

At the overall level, a repeated measures t-test indicated that overall within-

domain relatedness (M = 4.11, SD = .412) is significantly greater than overall between-

domain relatedness (M = 3.63, SD = .490), t (69) = 13.06, p < .001, and this is a large

difference (d = 1.07).

Domain level.

An example of the domain level comparison is that the preparation within-domain

pairs should show greater mean relatedness than preparation concepts paired with any of

the non-preparation concepts. This can be assessed by creating a domain relatedness

rating for each participant, by adding together the relatedness ratings for the pairs in

each domain combination that included preparation (three means for the within domain

pairs and nine for each of the between-domain sets) and finding the mean for each set,

creating four between-domain relatedness means. This results in a preparation within-

domain relatedness mean, and a between-domain relatedness mean for each of

314

preparation and knowledge, preparation and non-technical skills, preparation and

personal attributes, and preparation and technical skills.

Figure I.1 shows the means for group relatedness. There is one panel for each

concept group, showing the mean relatedness for each set of pairs, thus in the first

panel, displaying the ‘knowledge’ mean relatedness, the X-axis shows K_K for the

within group mean; K_NT for the knowledge-non-technical skills group mean; K_P for

the knowledge-preparation group mean; K_PQ for the knowledge-personal attributes

group mean and K_T for the knowledge-technical Skills group mean. The second panel

indicates that same pairings but with Non-technical Skills rather than knowledge. The

knowledge-nontechnical skills relatedness mean uses the same data as in the knowledge

panel, but may be slightly different based on the list-wise treatment of missing values

for each set of means. Similarly, all ‘out-group’ group relatedness data are used twice,

in a panel for each member of the pair. In addition each panel differs in the scaling used

on the y-axis.

The results of repeated measures ANOVA using the group relatedness scores for

each concept group is reported below. SPSS23 (IBM, 2015) was used with guidance

from Allen & Bennett (2010). As there were numerous comparisons made, the

Bonferroni adjusted alpha level was again set to p < .001. List-wise deletion for missing

values for each analysis led to different numbers of participants for each analysis, so the

number of participants included in each analysis is reported in each summary, as this is

not obvious from the degrees of freedom due to the use of the Huynh-Feldt correction.

315

Figure I.1 Group relatedness means for Knowledge (K), Non-technical skills (NT),

Preparation (P), Personal attributes (PQ) and Technical skills (T), showing 95%

confidence intervals. Be aware of the change in the scaling of the y-axis between

panels.

Appendices 316

Knowledge comparisons. The Fmax statistic indicated that the assumption of

homogeneity of variance was not violated. The Kolmogorov-Smirnov statistic for the

assumption of normality indicated that the K_K distributions was not normal, but that

all the other distributions were normal. Descriptive data indicated that the skew and

kurtosis of the K_K distribution was not excessive, but the detrended Q_Q plot showed

some deviation from normality. Mauchly’s test indicated that the sphericity assumption

was violated and consequently the Huynh-Feldt correction was employed.

The repeated measures ANOVA indicated that there is significant difference

among the Knowledge group relatedness means, F (3.376, 236.343) = 19.50, n = 71, p <

.001, ƞ²ʟ = .112 (a large effect size).

A series of pairwise comparisons revealed that K_PA relatedness (M = 4.03, SD =

0.58) was significantly greater than group relatedness for K_K (M = 3.68, SD = 0.56),

K_NT (M = 3.57, SD = 0.53), and K_T (M = 3.52, SD = 0.44) and these did not differ

among themselves. The K_PA group relatedness did not differ significantly from the

K_P group relatedness (M = 3.81, SD = 0.52) and the K_P group relatedness was

significantly greater than the K_T group relatedness.

This result does not support the expectation that mean relatedness of the

Knowledge within group pairings would be significantly greater than the other pairings

of concepts with Knowledge concepts.

Non-Technical skill comparisons. The Fmax statistic indicated that the

assumption of homogeneity of variance was not violated. The Kolmogorov-Smirnov

statistic for the assumption of normality indicated that the distribution of scores for all

the groups, except the NT-NT distribution, were normal. The NT_NT distribution was

not significantly skewed, nor was the kurtosis significant, and the Q-Q plots were as

expected. Mauchly’s test indicated that the sphericity assumption was violated and

consequently the Huynh-Feldt correction was employed.

The repeated measures ANOVA indicated that there is significant difference

among the Non-technical skill group relatedness means, F (3.548, 266.093) = 23.827, n

= 76, p < .001, ƞ²ʟ = .086 (a medium effect size).

A series of pairwise comparisons indicated that group relatedness for NT_NT (M

= 4.01, SD = 0.56) was significantly greater than group relatedness for NT_K (M =

Appendices 317

3.60, SD = 0.62), PA_NT (M = 3. 70, SD = 0.69) and T_NT (M = 3.52, SD = 0.57), but

did not significantly differ from group relatedness for P_NT (M = 3.81, SD = 0.56).

Group relatedness for P_NT was significantly greater than group relatedness for T_NT.

The NT_K, NT_PA and the T_NT means did not differ significantly among themselves.

This result does show support for the expected closer relatedness of the within

group non-technical skills concepts compared with the pairs between these and other

group concepts. Non-technical skills and preparation concepts are more closely related

that non-technical and other domain concepts.

Preparation comparisons. The Fmax statistic indicated that the assumption of

homogeneity of variance was not violated. The Kolmogorov-Smirnov statistic for the

assumption of normality indicated that the distribution of scores for the P_P group

relatedness scores was non-normal, however the other four distributions were normal.

The box-plots of the data indicate that the P_P distribution has a ceiling effect, the

values are near the top of the range. Mauchly’s test indicated that the sphericity

assumption was violated and consequently the Huynh-Feldt correction was employed.

The repeated measures ANOVA indicated that there is significant difference

among the Preparation group relatedness means, F (3.576, 271.8) = 76.879, n = 77, p <

.001, ƞ²ʟ = .30 (a very large effect size).

A series of pairwise comparisons indicated that group relatedness for P_P (M =

4.27, SD = 0.542) was significantly greater than for K_P (M = 3.85, SD = 0.55), P_NT

(M = 3.82, SD = 0.59), PA_P (M = 3.81, SD = 0.71) (which did not differ among

themselves) and P_T (M = 3.06, SD = 0.60). In addition, group relatedness for P_T was

significantly lower than for any of the other concept group pairs.

This result supports the expectation that the within group relatedness mean for

preparation concepts would be significantly higher than the group relatedness of the

out-group pairings between preparation concepts and other concepts.

Personal attributes. The Fmax statistic indicated that the assumption of

homogeneity of variance was not violated. The Kolmogorov-Smirnov statistic for the

assumption of normality indicated that the distribution of scores for the K_PA, PA_P

and PA_T group relatedness scores were normal, with the distributions of PA_PA and

PA_NT being non-normal. The box-plots of the data indicate that the PA_PA

Appendices 318

distribution was restricted in range as it has a ceiling effect, although the skew was not

significant. Similarly, skew and kurtosis of the PA_NT distribution were not significant.

Mauchly’s test indicated that the sphericity assumption was violated and consequently

the Huynh-Feldt correction was employed.

The repeated measures ANOVA indicated that there is significant difference

among the Personal attributes group relatedness means, F (2.961, 228.017) = 25.716, n

= 78, p < .001, ƞ²ʟ = .089 (a medium effect size).

The pairwise comparisons indicated that group relatedness did not differ

significantly for PA_PA (M = 4.15, SD = 0.69) and PA_K (M = 4.06, SD = 0.59)

however each was significantly higher than group relatedness for PA_NT (M = 3.68, SD

= 0.68), PA_P (M = 3.79, SD = 0.66) and PA_T (M = 3.62, SD = 0.68) which did not

differ significantly among themselves.

This result partially supports the expectation that the personal attributes within

group relatedness (PA_PA) would be higher than relatedness with out-group concepts,

but lack of difference with group relatedness for pairings with knowledge concepts

(PA_K) is not as expected.

Technical skills. The Fmax statistic indicated that the assumption of homogeneity

of variance was not violated. The Kolmogorov-Smirnov statistic for the assumption of

normality indicated that the distribution of scores for the T_T group relatedness scores

was not normal, with the other four distributions being normal. The box-plots of the

data indicate that the T_T distribution has a ceiling effect (the scores are against the

upper limit of the scale) and hence has a negative skew. Mauchly’s test indicated that

the sphericity assumption was violated and consequently the Huynh-Feldt correction

was employed.

The repeated measures ANOVA indicated that there is significant difference

among the Technical skills group relatedness means, F (3.233, 226.301) = 145.3, n =

71, p < .001, ƞ²ʟ = .465 (a very large effect size).

Post-hoc pairwise comparisons showed that group relatedness for T_T (M = 4.54,

SD = 0.46) was significantly higher than group relatedness for K_T (M = 3.51, SD =

0.45), T_NT (M = 3.47, SD = 0.55), PA_T (M = 3.57, SD = 0.71) and P_T (M = 2.99,

SD = 0.51) In addition, group relatedness for P_T was significantly lower than group

Appendices 319

relatedness for the other combinations of concepts. Group relatedness for K_T, T_NT

and PA_T did not differ significantly.

This result supports the expectation that the within group relatedness for technical

skills would be greater than for pairs with concepts from other groups. It also indicates

that preparation and technical skills are seen as particularly unrelated.

Summary. The group relatedness means indicate that preparation and technical

skills are distinct groups; in both cases the within-group group relatedness mean was

significantly higher than the out-group group relatedness means. The non-technical

skills group relatedness mean was significantly greater than all but the out-group

combination with preparation, and thus non-technical skills may be related to

preparation, rather than any of the other groups. The personal attributes within-group

group relatedness mean was significantly greater than all the out-group combinations

except that with knowledge, indicating a relationship between personal attributes and

knowledge. The knowledge within-group group relatedness mean was significantly

lower than the out-group combination with personal attributes, and not significantly

different from any of the other out-group group relatedness means, lending further

support to the idea of such a relationship.

Concept level

Visual inspection and comparison of each bold (within domain) mean with the

other means in the relevant row and column (between-domain pairs) in Table 6.4, shows

that in general the within-domain mean relatedness tends to be higher than the mean

relatedness for between-domain pairs. The exception is seen among pairs made with

‘knowledge’ concepts. While ‘knowing SOPs’ and ‘knowledge of rules & regulations’

were seen as quite closely related (high relatedness), these concepts paired with ‘aircraft

type knowledge’ were less closely related, and most of the means for the between-

domain pairs in the aircraft type knowledge column are greater than the within-domain

means. This contrasts with the ‘knowing SOPs’ and ‘knowledge of rules & regulations’

comparisons, for which none of the between-domain means exceeded the mean

relatedness between these two concepts.

Carrying out 15 ANOVAs on 14 means each, is probably over-analysing the data,

and a more appropriate way to look at the detail of this information is through the multi-

dimensional scaling (MDS) analysis reported in Chapter 6.

Appendices 320

Appendix H

Copyright Permissions

1. Tony Kern

Kirstie

I am the copyright owner and you have my permission. Good luck!

Tony

Sent from my Verizon Wireless 4G LTE smartphone

-------- Original message -------- From: Kirstie Carrick <kirstie.carrick@newcastle.edu.au> Date: 5/4/2016 23:30 (GMT-07:00) To: Tony Kern <Tony@convergentperformance.com> Subject: Use of Airmanship Model diagrams

Hello Tony,

I am contacting you to seek written permission to copy and communicate the following material within the electronic version of my PhD thesis titled ‘Airmanship in Australian Aviation’.

1. The diagram of the airmanship model as it appears on page 22 of Redefining Airmanship, (1996, McGraw Hill)

2. The Airmanship model revised as it appeared in Slide 11 of the Roadshow Wrap-up presentation at the CASA Threat and Error Management Roadshow (2009).

If you are not the rights holder for this material, I would be grateful if you would advise me who to contact.

The thesis will be made available on the internet via the University of Newcastle’s online digital repository http://nova.newcastle.edu.au/

Regards,

Kirstie Carrick

Appendices 321

2. Phil Spencer

3. Australian Transportation Safety Bureau

Creative Commons licence

With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright ,this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.

Creative Commons Attribution 3.0 Australia Licence is a standard form license agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.

The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau

Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you want to use their material you will need to contact them directly.