View
7
Download
0
Category
Preview:
Citation preview
Airmanship in Australian Aviation
Christine Ellen Carrick, BSc (Hons) MSc
Thesis submitted for the degree of Doctor of Philosophy
(Aviation)
December 2015
i
Declaration
I hereby certify that the work embodied in this thesis is contains no
material which has been accepted for the award of any other degree or
diploma in any university or other tertiary institution and, to the best of my
knowledge and belief, contains no material previously published or written
by another person, except where due reference has been made in the text. I
give consent to the final version of my thesis being made available
worldwide when deposited in the University’s Digital Repository, subject to
the provisions of the Copyright Act 1968.
Christine Carrick
ii
Acknowledgements
I owe many thanks to my supervisor Dr Kerry Chalmers for her
guidance, patience and tenacity. My thanks also go to the organisations
that allowed me access to their personnel and to the many friends and
acquaintances who participated in or promoted my research.
I am also very grateful to my children, Luke and Ruth, and my
mother, Sheila (1925-2015), for their support and encouragement.
iii
Publication Notes
Excerpts and summaries from two chapters of this thesis have been presented
as conference papers, as shown below.
Carrick, K. (2009, June). Human factors as a component of airmanship. Paper presented at the 8th Industrial and Organisational Psychology Conference, Sydney, Australia.
Carrick, K. and Chalmers, K. (2010, April). Airmanship: Voices from an on-line survey. Paper presented at the 9th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.
Carrick, K. and Chalmers, K. (2010, April). Development of airmanship. Paper presented at the 9th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.
Carrick, K., Wahlberg, J., Wright, K. and Chalmers, K. (2014, August). Australian aviators’ conception of airmanship. Paper presented at the Massey Academic Forum of the 2014 PACDEFF, Auckland, New Zealand.
iv
v
Table of Contents
Declaration
Acknowledgements
Publication Notes
Table of Contents
List of Tables
List of Figures
Abstract
i
ii
iii
v
vii
ix
x
Chapter 1. Airmanship in Australian Aviation 1
Models of Airmanship 2
Training and Development of Airmanship in Australia 25
Current Research 29
Chapter 2. Airmanship: Definition and Development 33
Methodological considerations 34
Method 40
Results and Discussion 41
Conclusion 55
Chapter 3. Airmanship Across Industry Sectors 59
Method 62
Results - Definition of Airmanship 67
Conclusion – Definition of Airmanship 91
Results - Training and Development of Airmanship 93
Conclusion – Training and Development of Airmanship 128
Chapter 4. Current Airmanship Training and Assessment 131
Method 132
Results – Analysis of Demographics 135
Results – Definition of Airmanship 137
Discussion - Definition of Airmanship 156
Results - Training and Assessment of Airmanship 162
Discussion: Training of Airmanship 185
vi
Chapter 5. The Structure of Airmanship 193
Comparison with Previous Research 193
Model of Airmanship 195
Conclusion 200
Chapter 6. Airmanship Concept Domains 201
Methodological Considerations 201
Pilot Study 203
Method 203
Results 205
Discussion 207
Main Study 207
Method 208
Results 210
Discussion 218
Chapter 7. Airmanship Conclusions 221
Survey 1 222
Survey 2 224
Interviews with Current Instructors 229
Interviews: Training and Development of Airmanship 233
The Structure of Airmanship 237
Airmanship Concept Domain Testing 237
Definition of Airmanship 238
Training of Airmanship 240
Limitations of the Current Research 242
Future Research 242
Conclusion 243
References 247
Appendices 255
vii
List of Tables
Table Title Page Table 2.1 Number of Respondents (of a Total of 40) who Mentioned Each Concept in
Questions 1, 2 and Overall (Qs 1 to 11). 42
Table 2.2 Number of Respondents who Mentioned Each Concept in Questions 6, 9 and 10 (of N = 40).
47
Table 2.3 Number of Respondents who Mentioned Listed Scenarios or Phases of Flight in Questions 7, 9 or 10
49
Table 2.4 Number of Respondents Answering Question 3 49 Table 2.5 Number of Respondents (of total N=40) who Mentioned the Listed Concepts
in Response to Question 4b 51
Table 2.6 Number of Respondents (of a total N=40) who Mentioned the Listed Concepts in Response to Question 5
53
Table 2.7 Number of Respondents (of a total N=40) who Mentioned the Listed Concepts in Response to Question 8
54
Table 3.1 Comparison of Word Count across Industry Sectors 65 Table 3.2 Comparison of Hours of Experience and Years of Experience across
Industry Sector 66
Table 3.3 Percentage of Respondents who Mentioned Listed Concepts from Airmanship Models in Response to Combined Definition of Airmanship Questions
69
Table 3.4 Percentage of Respondents who Mentioned Listed Additional Concepts in Response to Combined Definition of Airmanship Questions
70
Table 3.5 Percentage of Respondents who Mentioned Listed Events or Phases of Flight in Response to Combined Definition of Airmanship Questions, by sector
71
Table 3.6 Percentage of Respondents who Mentioned Listed Concepts from Airmanship Models in Response to Combined Definition of Airmanship Questions
73
Table 3.7 Percentage of Respondents who Mentioned Listed Additional Concepts in Response to Combined Definition of Airmanship Questions
79
Table 3.8 Percentage of Respondents who Mentioned Listed Events or Phases of Flight in Response to Combined Definition of Airmanship Questions
83
Table 3.9 Number of Respondents in each Sector of Background and of Current Operation who Responded to Question 14 and Question 15
84
Table 3.10 Airmanship Concepts Ordered by Frequency of Mention in the Definition Questions, Rating in Question 14 and Ranking in Question 15
90
Table 3.11 Percentage of Civilian or Military Sectors Respondents by era of training (before 1991 or 1991 and after) Who Responded to Question 3(a)
94
Table 3.12 Percentage of Civilian or Military Sector Respondents by era of training (before 1991 or 1991 and after) who Mentioned the Listed Concepts From Airmanship Models in Response to Question 3(b)
95
Table 3.13 Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned the Listed Additional Concepts in Response to Question 3(b)
99
Table 3.14 Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Responded to Question 4
101
Table 3.15 Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned Concepts From the Airmanship Models in Response to Question 4
102
Table 3.16 Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned the Listed Additional Concepts in Response to Question 4
104
viii
Table Title Page Table 3.17 Percentage of Respondents in Each Sector who Mentioned the Listed
Factors in Response to Question 5 108
Table 3.18 Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 6
114
Table 3.19 Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 9
118
Table 3.20 Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 10
124
Table 4.1 Numbers of Instructors in Each Group, Experience in Aviation and Tertiary Degree Status
135
Table 4.2 Percentage of Participants in Each Group, and of Total Participants, who Commented on the Process of Definition
137
Table 4.3 Percentage of Participants in Each Group, and of Total Participants, who Commented on the Process of Definition
138
Table 4.4 Percentage of Participants in Each Group who Spoke of Listed Concepts in Their Initial Definition of Airmanship
140
Table 4.5 Percentage of Participants in Each Group who Mentioned Specific Definitions or Listed Concepts from the Airmanship Models During Interview
142
Table 4.6 Percentage of Participants in Each Group who Mentioned Listed Additional Concepts Relevant to the Definition of Airmanship
147
Table 4.7 Comparison of Means of Years of Experience by Mention of Common Sense 150 Table 4.8 Percentage of Participants in Each Group who Mentioned Listed Actions
and Considerations Relevant to the Definition of Airmanship 154
Table 4.9 Concepts Constituting Airmanship Ordered by Percentage of Participants who Mentioned each, from the Interview Data and from the Online Survey
160
Table 4.10 Percentage of Participants who Commented on Aspects of the Training Program
163
Table 4.11 Percentage of Participants in Each Group and Overall who Mentioned Inclusion of Listed Airmanship Items in Initial Flight Training
171
Table 4.12 Percentage of Participants who Commented on Listed Aspects of the Training Process
177
Table 6.1 Domains and representative concepts 204 Table 6.2 Mean relatedness between pairs of concepts in the pilot data grouped by
domain 206
Table 6.3 Mean relatedness for forward and reverse worded pairs and outcome of Wilcoxen Signed Ranks (WSR) tests and Pearson correlation for each pair
212
Table 6.4 Mean relatedness between pairs of concepts grouped by domain 204
ix
List of Figures
Figure Title Page
Figure 1.1 The Kern model of airmanship. An adaptation from Redefining Airmanship T. Kern, 1996, McGraw-Hill p. 22
3
Figure 1.2 The Kern model of airmanship. An adaptation from a diagram in Kern (2009a) CASA threat and error management roadshow (wrap-up power-point presentation, slide 11).
4
Figure 1.3 Schematic representation of the elements of airmanship in the Ebbage and Spencer model. From "Airmanship training for modern aircrew", by L Ebbage and P. D. Spencer, 2003
9
Figure 1.4 Threat and Error Management Model from "Error management training: Defining best practice” by M.J.W. Thomas (2005). ATSB
15
Figure 1.5 Mismatch emergency model from "Error management training: Defining best practice” by M.J.W. Thomas (2005). ATSB
16
Figure 3.1 The rank of mean ratings for the 26 concepts rated in Question 14
85
Figure 3.2 The mean rankings for the 26 concepts ranked in Question 15
87
Figure 5.1 A revised airmanship model 196
Figure 6.1 Scree plot of stress values for one to seven dimension solutions
216
Figure 6.2 MDS 2-dimensional plot 216
Figure 6.3 MDS 3-dimensional plot at 20 degrees of rotation 217
Figure 6.4 MDS 3-dimensional plot at 60 degrees of rotation 217
Figure 6.5 Revised model of airmanship 220
x
Abstract
The current project involved investigation into the understanding of airmanship among
Australian aviators and the way in which airmanship is trained in Australia. The starting
point was Kern’s model of airmanship (1996, 2009a), developed from research with aviators
in the USA. The current research revealed two insufficiencies in the Kern model. First that it
includes few of the non-technical skills that become important as an aviation career
progresses, and secondly it emphasises knowledge over application. The Ebbage and
Spencer (2003) model of airmanship, developed in the UK, includes non-technical skills
more overtly and some additional concepts not included in the Kern model. Both Kern and
Ebbage and Spencer recommend that training of airmanship should include three phases: 1)
instil the importance of airmanship; 2) overtly teach and model airmanship; and 3) assess
and provide feedback.
In the current project, an initial on-line survey reached mainly general aviation pilots.
The participants mentioned many of the concepts included in the existing models but also
some additional concepts. A second survey reached more participants and enabled
comparison of differences in views between military and civilian background aviators. It also
established the relative importance placed by the participants on the component concepts of
airmanship. A third study involved the interviewing of a cross-section of flight instructors
and trainers from civilian and military sectors, at ab-initio and advanced levels of training. It
appears that the training of airmanship generally meets the final two phases of the suggested
training process, but there seemed to be a lack of formal introduction to airmanship and no
use of a structure to facilitate development of airmanship. A model of airmanship was
developed and its structure tested empirically. A revised model is presented, which provides
a more balanced approach to the importance of the component concepts than earlier models
and also recognises the influence of context on the development and expression of
airmanship. The use of the revised model to expressly embed airmanship in training
programs may alleviate the paradox of the use of the term ‘airmanship’ both as a global
expression of safe and efficient flight and also to describe only the non-technical skills
component, in some training environments.
1
Chapter 1. Airmanship in Australian Aviation
In the last hundred years aviation has moved from being an experimental
adventure to being an accepted, even expected, form of transportation, and an integral
part of military capability. In the process, aviation has also become a closely regulated
activity. The International Civil Aviation Organisation (ICAO), a branch of the United
Nations, coordinates the aviation regulations of signatory nations. In each nation, the
Air Navigation Act (or similar legislation) determines what aviators can and cannot do.
The process of gaining a pilot’s licence requires passing several exams on
aerodynamics, navigation, meteorology, aircraft systems and human factors, while the
practical side of learning to fly is also governed by a regulator-set syllabus and
assessment procedure. Once qualified, the pilot’s flying activity is subject to standard
operating procedures (SOPs) set by the manufacturer of the aircraft flown or by the
operating organisation, and regular training and assessment activities are required to
maintain a licence.
It is the human pilot who is ultimately responsible for flying the aircraft safely,
and despite the regimentation and regulation, there is still room for variation in the
operation of the aircraft. The literature concerned with aviation accident analysis
frequently refers to the idea that anywhere upwards from 70% of aircraft accidents
involve some human factor among the causes (Helmreich & Foushee, 1993). On the
other hand, there has been more recent recognition that of all the flights on which
something unexpected has happened, of those that land safely or are survivable, 100%
are due to the intervention of a human on the flight deck (Reason, 2008). Generally the
media will tell us that good airmanship has saved the day. What is good airmanship?
There are many definitions. James Reason considers airmanship to be professionalism,
and defines it further.
Let me begin by teasing out what I understand by “professionalism’. It is a fuzzy-edged term embracing a number of different qualities. Aviators call it airmanship; mariners call it seamanship. These labels describe abilities that go well beyond the competent deployment of technical skills. They imply the capacity to see the broader picture, to think ahead and to draw upon a wide range of knowledge and experience so as to perform demanding work safely, elegantly and effectively. It means having a deep understanding of all the various factors that can impact upon task performance for good or ill. It also
2
entails a willingness to engage in all aspects of the job – tedious or otherwise – to the best of one’s ability. (Reason, 2008, p. 161)
Airmanship is defined by ICAO as “the consistent use of good judgement and
well developed skills to accomplish flight objectives” (ICAO, 2006, cited in CASA,
2008). Australian military pilot trainees are taught that it is “the safe and efficient
handling of the aircraft on the ground and in the air” (BAE Systems, 2006, slide 8).
Edwards (2013) condenses a variety of ideas to “the (technical, operational and non-
technical) knowledge, skills and attitudes aircrew employ to operate an aircraft
effectively, efficiently and safely” (p. 13). Certainly good airmanship is considered an
essential quality for aviators (Ebbage & Spencer, 2003), however it can be a rather
elusive concept to pin down (Kern, 1996). The most extensive attempt to bring order to
the definition has been the model of airmanship developed by Kern (1996).
Models of Airmanship
While there are many definitions, opinions and discussions of airmanship on
public forums such as aviation magazines and various web-pages, there are just three
published models presenting a solid formulation of what constitutes airmanship. These
are the books by Kern (1996, 1997 and 1999); a paper presented at a military flight
training conference by Ebbage and Spencer (2003) and the book by Edwards (2013). A
brief description of each of these models follows. There are also several models of
human performance in aviation environments that reflect some aspects of airmanship,
which will be discussed in a further section, below.
Kern (1996, 2009a) Model of Airmanship
Overview. Kern (1996) presented a model of airmanship represented as a building
façade, based on the foundation or bedrock principles of “skill, proficiency and the
discipline to apply them in a safe and efficient manner” (p. 21). Kern (1996)
summarised his model as:
Airmanship is the consistent use of good judgement and well-developed skills to accomplish flight objectives. This consistency is founded on a cornerstone of uncompromising flight discipline and developed through systematic skill acquisition and proficiency. A high state of situational awareness completes the airmanship picture and is obtained through knowledge of one’s self, aircraft, team, environment and risk (p. 22).
3
These foundation or bedrock principles of discipline, skill and proficiency, with
the pillars of knowledge, support the capstone outcomes of situational awareness and
good judgment in decision making, leading to superior airmanship. A diagrammatic
representation of the model is presented Figure 1.1 (from Kern, 1996). The goals of
airmanship are stated as “safety, mission effectiveness and efficiency” (Kern, 1996, p.
254). Rather than thinking of these components of the model as separate, Kern (1999)
presents the idea that they should be a seamless whole that informs and directs the
conduct of the aviator at all times.
Figure 1.1 The Kern model of airmanship. An adaptation from Redefining Airmanship T. Kern, 1996, McGraw-Hill p. 22.
Bedrock Principles. Flight discipline is seen as the support or backbone for
airmanship (Kern, 1997). Kern (1996) describes discipline as the ability and willpower
to fly safely “within operational, regulatory, organisational and common-sense
guidelines” (p. 29). Kern (1997) devotes an entire book to the importance of flight
discipline, and upgraded discipline to a separate sub-heading, “cornerstone” more
recently (2009a). In writing for a more general audience (rather than just aviation) Kern
(2009b) suggests that there is an individual responsibility for preventing personal error,
predominantly through self-discipline, or empowered accountability.
Skill and proficiency are shown as separate foundations in Kern’s (1996) model,
but are discussed together. Skill is considered to be the ability to handle the aircraft
generally. According to Kern, skill should build up with experience from the minimum
safe qualification through the levels of effectiveness and efficiency, culminating in an
4
attitude of precision and improvement wherein the aviator is aiming to improve with
each flight. Kern (1996) also points out that many aviators are happy to sit at the
effective or efficient levels and never progress, thus demonstrating a lower level of
airmanship. Proficiency is presented as personal assessment of the currency and
recency of flying skills, with good airmanship including a personal responsibility to
ensure that these are not degraded by lack of practice. Kern mentions in passing that as
well as the obvious flying skill, skills in communication, decision making, teamwork
and self-assessment are also needed for good airmanship, however these are not
elaborated upon in the discussion of skill and proficiency.
Pillars of Knowledge. Kern’s (1996) original model included five pillars of
knowledge, knowledge of one’s self, the team, the aircraft, the environment and risk
associated with the flight. An additional pillar was added more recently (Kern 2009a)
representing knowledge of the mission (see Figure 1.2).
Figure 1.2 The Kern model of airmanship. An adaptation from a diagram in Kern (2009a) CASA threat and error management roadshow (wrap-up power-point presentation, slide 11).
Discussion of knowledge of one’s self includes physiological airworthiness, such
as the effects of fatigue, illness, stress and nutrition, and also considers hazardous
attitudes. The five main hazardous attitudes are anti-authority, impulsivity,
invulnerability, machismo (or showing-off) and resignation. A strong desire to get to the
planned destination despite developing problems, known as “get-there-itis”, is also
5
considered a hazardous state (Trollip & Jensen, 1991). Kern adds some further
hazardous attitudes: complacency, emotional jet-lag (or over-perfectionism and
dwelling on errors), excessive professional deference and passenger/co-pilot syndrome
(the idea that the other pilot will pick up any errors).
Kern (1996) returns to the theme of individual responsibility in discussing
knowledge of the team: he considers that one should be sufficiently competent to
contribute effectively to the team. Crew resource management (CRM) is discussed, with
the comment that “airmanship is the individual structure on which CRM builds” (Kern,
1996, p. 139). Kern notes that many of the topics included in CRM training programs
are elements in the airmanship model, with a list of eight basic inclusions in CRM
training being situational awareness, group dynamics, communication, risk
management, decision making, stress awareness and management, mission planning,
review and critique, and physiology and human performance. Leadership and
followership are also considered as part of this “team” pillar.
Under knowledge of the aircraft, Kern (1996) states that there should be a
thorough understanding of the aircraft type being flown, including its subsystems,
limitations and emergency procedures. It is emphasised that this knowledge should be
continuously rehearsed and updated. Kern notes that discussions with other pilots of the
same aircraft type will bring to light idiosyncrasies that may not appear in the flight
manuals.
Knowledge of the environment includes the physical, regulatory and
organisational environments. The physical environment includes terrain, meteorology,
light, and airspace. The regulatory environment includes everything in the regulations,
including air traffic control regulation. The organisation’s environment includes an
awareness of corporate culture and challenges to airmanship that may arise with
conflicting demands (e.g. depart on time vs. checking a minor malfunction). Kern
(1996) does not draw attention to Standard Operating Procedures and other
organisational policies, procedures or practices that are part of the organisational
environment.
Kern (1996) places importance upon the knowledge of risk and the effect of risk
upon the various components of the model. Kern defines risk as “the probability and
severity of a loss linked to a hazard” (p. 202) and a hazard is any event, or item, that
6
affects a flight with the potential for damage, injury or delay. There is no flying without
some risk, but the risks should be recognised and managed in a display of good
airmanship.
A more recent addition to the model (Kern, 2009a) is the idea that the knowledge
of the mission forms a further support pillar for good airmanship. This seems logical,
knowing what each flight will involve, including preparation, flight planning and
obtaining weather forecasts, is clearly going to assist in developing and maintaining
situational awareness and safety, and thus is good airmanship. Figure 1.2 shows the
Kern model of airmanship presented in 2009 (Kern 2009a) including the increased
emphasis on discipline and the additional pillar for knowledge of the mission.
Capstone Outcomes. Kern (1996) places situational awareness feeding into good
judgement at the apex of his model, as “capstone outcomes”. A simple definition of
situational awareness (SA) is “What happened previously, what is happening now, and
what will happen in the future” (Sarter & Woods, 1991). A more comprehensive
definition is “the perception of the elements in the environments within a volume of
time and space, the comprehension of their meaning and the projection of their status in
the near future” (Endsley, 1988, cited in Endsley, 1999, p. 258). Kern interprets and
discusses these three levels of SA, being perception, comprehension, and projection. He
also reports a survey of F15 fighter pilots (Wagg, 1993, cited in Kern, 1996) who
reported seven elements crucial to good SA: building the big picture; integration of
information from multiple sources; keeping track of spatial position and relationship
with other aircraft; keeping up with changes in the situation; prioritizing tasks and data;
making good, timely decisions; and prediction into the future.
Kern (1996) emphasises the importance of recognising when SA has been lost and
of taking steps to remain safe while trying to recover SA. In aviation, good SA is
sometimes referred to as “staying ahead of the aircraft” while lack of SA is considered
“being behind the aircraft”. Once it is recognised that one is being “left behind”,
catching up again becomes a priority.
Keeping track of multiple variables in a dynamic environment, the essential task
of SA, is challenging but essential for making decisions, hence its central importance in
airmanship. Judgement – or the ability to make sound decisions – is ultimately how
flights remain safe, effective and efficient.
7
Kern (1996) takes the view that judgement is an integral part of every action an
aviator takes, in the air and on the ground, and relies upon the interaction and
interdependence of all the aspects of airmanship discussed thus far. Judgement and
decision making are treated as synonyms and both of the terms are used in his
discussion. A concern is expressed by Kern (1996) that increased use of checklists and
procedures while leading to more predictable decisions and safer flight, may actually
mean that pilots are overly reliant upon automation and manuals, reducing their ability
to take independent and timely action when it is called for. Contrary to what is
expressed as common opinion (that one intrinsically has good judgement or one
doesn’t) good judgement is presented as an acquired skill that can be systematically
developed. Kern’s discussion of the decision making process appears to be based on the
models of naturalistic decision making and concentrates on assessing the situation and
determining alternatives (Kampf & Klein, 1994).
Integration of model components. Kern (1996) considers airmanship as an
integrated holistic notion that encompasses the elements described above. To be
effective pilots must be ready to utilise their airmanship whenever the opportunity to
demonstrate it occurs, whether that is handling a regular routine flight or coping with
disaster and pulling off what Reason (2008) terms a “heroic rescue”. Kern presents
Captain Al Haynes’ leadership of the crew of United Flight 232 and the crash landing at
Sioux City in 1989 as one such heroic rescue. Flight deck crew coordination and careful
aircraft handling overcame loss of all control hydraulics and, despite the aircraft
breaking up on landing, most of the passengers and crew survived. A more recent
example of such an integration of skill, knowledge, situational awareness and
judgement is Captain “Sully” Sullenberger’s ditching of US Airways Flight 1549 in the
Hudson River on 15 January, 2009, with no casualties, after multiple bird-strikes led to
the loss of both engines just after take-off. In April, 2009 Sullenberger was awarded the
Jabara Award for Airmanship by his alma mater the USAF Academy. He later wrote
On January 14, 2009, my life had been a series of thoughtful opportunities to be the best pilot, leader, and teammate I could be. I was an anonymous regular guy... On January 15, circumstances changed everything, a reminder that none of us ever knows what tomorrow will bring. I flew thousands of flights in the last forty-two years, but my entire career is now being judged by how I performed on one of them. This has been a reminder to me: We need to try to do the right thing every time, to perform at our best, because we never know which
8
moment in our lives we’ll be judged on (Sullenberger & Zaslow, 2009, p. 313-314).
Captain Richard de Crespigny also commented on teamwork and excellence in
crew performance. He led the crew aboard the “Nancy-Bird Walton”, Qantas Flight 32
out of Singapore on 4 November, 2010. An uncontained engine explosion on climb
destroyed one engine, put holes through the wing and fuel tanks leading to fuel leaks,
severed many control and information conduits leaving the remaining engines degraded,
and peppered the fuselage with shrapnel. After the event, he wrote:
So from my perspective and having made it through the toughest challenge of my professional life with no injuries or deaths, I can honestly say that everyone on board was lucky that they had a crew full of knowledgeable, case-hardened, experienced pilots who worked effectively as a team as we wrestled to keep Nancy-Bird Walton in the sky and then bring her back to earth... You get a first officer like [name] because the airline recognises his role as a professional, as a “safety and profit” centre not a “cost centre”, and therefore actively develops his excellence... The QF32 was a great flight, but we were only one of thousands of flights that take off every day, each of which are manned with excellent crews (de Crespigny, 2012, p. 332-3).
It appears that both Captains agree with Kern, that the integration of skill,
knowledge and judgement – good airmanship – brought off the almost impossible
“heroic rescue” that an unusual circumstance had presented them with. They also imply
that the combination of factors, airmanship, did not come about by chance. Both
extracts indicate a process of development and experience that resulted in a crew ready
to handle the emergency. Kern (1996) after defining and describing airmanship, goes on
to present a method for continuous development towards excellence in airmanship,
discussed below under the heading training and development of airmanship.
Ebbage and Spencer Model of Airmanship
Ebbage and Spencer (2003) present a model of airmanship based on their work
with military ab-initio training in Britain, and acknowledge the influence of the Kern
(1996) model in the structure they present. They define airmanship as:
A personal state that enables aircrew to exercise sound judgement, display uncompromising flight discipline and demonstrate skilful control of an aircraft and a situation. It is maintained by continuous self-improvement and a desire to perform optimally at all times (p. 2).
A schematic representation of the model is presented in Figure 1.3. In this model,
the three central facets of good airmanship are judgement, control and discipline.
9
Judgement is presented as a product of situational awareness, problem solving, mental
workload and foresight. Control of the aircraft is required for safe flight and is shown as
developing from flying skills, automation and information management, navigation and
communication skills. Discipline is indicated as arising from self-improvement,
vigilance, cooperation and confidence. Ebbage and Spencer (2003) state that discipline
is needed to detect errors early and ensure that judgements are considered and actions
controlled.
Ebbage and Spencer (2003) suggest that airmanship is built on a foundation of
knowledge, skills and attitudes. They present these foundation requirements as a table
of syllabus items for the training of airmanship.
Knowledge. Ebbage and Spencer (2003) refer to three types of knowledge, that of
the aircraft, the environment and risk. Aircraft knowledge is specified as including
thorough familiarity with the aircraft systems, approved procedures, flight
characteristics and limitations of the aircraft, and with automation. Knowledge of the
environment includes the same three sub-categories as Kern (1996), the physical,
regulatory and organisational environments. Physical environmental features, such as
weather and terrain, are considered in terms of their effects on aircraft control. The
Figure 1.3 Schematic representation of the elements of airmanship in the Ebbage and Spencer model. From "Airmanship training for modern aircrew", by L. Ebbage and P. D. Spencer, 2003.
10
regulatory and organisational environments are presented as possibly posing challenges
to airmanship. An understanding of the risks that may occur is the third part of the
knowledge foundation.
Skills. Ebbage and Spencer (2003) list a variety of skills that they consider central
to good airmanship and that should be developed in military flight training courses.
They group these skills into five categories: physical skills; cockpit management;
communication; cognitive skills and team skills. Flying skills, including instrument
flying and emergency handling and recovery, navigation skills and combat survival are
listed as required physical skills. Cockpit management includes information
management and specific mention of “avoiding the pitfalls of automation (over-reliance,
complacency, bias)” (Ebbage & Spencer, 2003, p. 6). Communication skills are spelt
out to include vigilant monitoring, active listening, appropriate, clear and concise
phraseology and inquiry. Unlike Kern (1996), Ebbage and Spencer include situational
awareness and decision making as cognitive skills, rather than as outcomes. They also
include problem solving, workload handling and self-assessment as cognitive skills.
Finally, team skills cover performance monitoring, leadership and initiative,
interpersonal skills, coordination and decision making, and team communication and
SA. There is some overlap between cognitive and team skills, perhaps in order to
emphasise that these are as important in single-pilot operations as multi-crew, and that
in multi-crew (team) environments additional attention is needed to ensure that all team-
members are “in the loop” and aware of what is happening.
Attitudes. Within the consideration of the development of appropriate attitudes,
Ebbage and Spencer (2003) give prominence to an understanding of hazardous attitudes.
These were mentioned briefly in Kern (1996) as part of “knowledge of self” and are
based on the hazardous attitudes discussed by Trollip & Jensen(1991). The idea is that if
trainee pilots are exposed to the sorts of attitudes that can lead to dangerous behaviour,
and their antidotes (sayings used to defuse a hazardous thought), then they are less
likely to indulge in or be led astray by them. Thus, awareness of the impact of such
hazardous attitudes on airmanship and safety is a key part of the training regime that
Ebbage and Spencer (2003) propose. On a more positive note, they include the
development of a sense of professionalism, as being “understanding the values and
principles embodied in airmanship” (p. 6), and an attitude of self-improvement,
11
including the motivation for life-long learning, understanding of the need for self-
assessment and the will to develop excellence.
Finally, Ebbage and Spencer (2003) consider discipline to be an attitude that
should be fostered and exercised in terms of ongoing maintenance of knowledge and
skills, and of self-discipline (seen as managing one’s own attitudes and stress). They
also consider discipline in terms of various aspects of flying activity such as
preparation, vigilance and lookout, maintenance of situational awareness, and keeping
within operational and regulatory policy. Post-flight evaluation is included as an area of
application of discipline, perhaps in recognition that once flight training is completed
(with its formal post-sortie debriefings) such evaluation might be skimped and yet can
be a very useful reflection.
Edwards’ Model of Airmanship
Edwards (2013) defines airmanship as the technical, operational and nontechnical
knowledge, skills and attitudes required to fly safely, effectively and efficiently. He
presents a discussion of airmanship (as opposed to a formal model) in four main
sections: technical elements, operational elements, non-technical elements and other
factors. Each of the elements is described in terms of the knowledge, skills and attitudes
required to exhibit good airmanship.
Technical elements. Edwards (2013) considers these to be knowledge about the
aircraft flown, the skills to fly accurately and smoothly and an attitude of study, practice
and self-discipline.
Operational elements. Knowledge of the people, systems and processes related
to the job are the first part of the operational elements. Operational skills include
forming and executing a plan of action, time management, and awareness of what part
of the plan requires implementation at which time. Operational attitudes embrace a
balance of safety and getting the job done, self-discipline, preparation, thinking ahead,
and not making assumptions.
Non-technical elements. Edwards notes that this section includes human factors,
of which crew resource management is a subset. Non-technical knowledge covers the
psychological and physiological operation of humans in the aviation environment. Non-
technical skills are discussed as a set of standards of performance covering
12
communication, leadership and teamwork, workload management, situational
awareness, and problem solving and decision making. Non-technical attitudes are noted
as overlapping those of the other sections, but including in addition: self-respect and
confidence; respect for others and tolerance; willingness to learn from experience;
integrity and honesty; and being positive and ensuring success rather than looking for
excuses for failure.
Other factors. These are aspects that are not included in the three main elements
but awareness of which Edwards (2013) notes as being important for good airmanship.
The factors discussed include stress; fatigue; hazardous attitudes; culture; automation;
monitoring; and threat and error management. Most of these could be considered
covered by the ‘knowledge’ sections in each element of airmanship.
Comparison of the Airmanship Models
Ebbage and Spencer (2003) acknowledge the influence of Kern (1996) on the
formulation of their ideas about airmanship and therefore there are many similarities
between the two models, but there are also some differences. Both the Kern and the
Ebbage and Spencer models of airmanship described above have their roots in the
military environment, with Kern using many military case studies as well as civilian,
and Ebbage and Spencer specifically addressing military flight training. This may have
influenced the emphasis on discipline in both these models. Ebbage and Spencer list
areas in which discipline should be applied, while Kern develops the concept of flight
discipline much further, and expands upon it with the discussion of rogue pilots who
eschew discipline and the consequences of such behaviour (Kern, 1999). Edwards
(2013) acknowledges his own background in military aviation but uses exclusively civil
aviation examples in his discussion of airmanship. Self-discipline is mentioned as a
desirable attitude but is not over-emphasised.
Compared with the Kern (1996, 2009a) model, the Ebbage and Spencer (2003)
and the Edwards (2013) models have a more comprehensive list of the skills required
for good airmanship, drawing more attention to communication and team skills than
does Kern. Edwards (2013) and Ebbage and Spencer (2003) include situational
awareness and decision making as skills, while Kern (1996) discusses them as outcomes
of the exercise of skill and use of knowledge.
13
Kern (1996), Ebbage and Spencer (2003) and Edwards (2013) agree that
airmanship includes a thorough understanding of the aircraft type flown, its subsystems,
limitations, emergency procedures and so on. Both the Kern and Ebbage and Spencer
models include knowledge of the physical, regulatory and organisational environments,
however neither of them draw attention specifically to Standard Operating Procedures
(SOPs) and other organisational policies, procedures or practices. In contrast, Edwards
discusses operational elements of aviation in a general sense, without mention SOPs
specifically, except in terms of exercising self-discipline to conform to procedures and
checklists “even when there doesn’t seem to be the need” (p 33).
Kern (1996) includes in the knowledge pillars, knowledge of one’s self and of the
team, while Ebbage and Spencer cover these areas with their comments on team skills
and with self-assessment included as part of cognitive skills, and Edwards (2013)
discusses human factors and teamwork quite specifically under non-technical elements.
All three models consider awareness of the same hazardous attitudes, Ebbage and
Spencer include these more prominently as part of an attitudes grouping, while Kern
includes these and some others within the pillar for knowledge of the self, and Edwards
discusses these among his ‘other factors’. Both Kern and Ebbage and Spencer consider
the knowledge of risk and the effect it has upon the various components of their models,
while Edwards does not, except insofar as threat and error management is covered. The
Kern model (2009a) includes knowledge of the mission, but with no elaboration, while
the Ebbage and Spencer model simply includes “preparation” as an area in which
discipline should be applied. In Edwards this is covered among the operational skills, in
developing, organising and implementing a plan for the flight.
The Kern and the Ebbage and Spencer models place importance on situational
awareness as a precursor to good judgement. Kern places these at the apex of his model,
as capstone outcomes, while Ebbage and Spencer place judgement as one of the three
main elements of airmanship, depending upon situational awareness, problem solving,
mental workload and foresight. Edwards includes situational awareness and decision
making as two of his six non-technical standards.
Compared with Kern (1996), both Ebbage and Spencer (2003) and Edwards
(2013) put more emphasis on attitudes, including hazardous attitudes. They also list
more prominently skills of communication, crew co-ordination, interpersonal skills,
management within the flight-deck or cockpit, self-assessment and team leadership. In
14
addition, Ebbage and Spencer emphasise professionalism and self-improvement as part
of airmanship, which the other two approaches do not, but do seem to assume.
None of these models present any empirical evidence for the structure of the
model or the concepts included within them. Nor has there been any formal evaluation
of the models against the views of current pilots.
Other Models Related to Airmanship
In addition to the models of airmanship discussed above, there are a number of
other models of aviation related human performance that are relevant to the general
discussion of airmanship.
The end goal of airmanship is to maintain safety at all times. Operating safely can
be seen either as part of airmanship or as the outcome of good airmanship. In the
announcement of the release of the Guild of Air Pilots and Air Navigators (GAPAN)
threat and error management (TEM) program for General Aviation and regional airlines
it was suggested that the TEM model is a framework for airmanship, and that TEM
training “brings airmanship training into the 21st century” (Elms, 2007, p. 1). TEM
training addresses the safe conduct of flight and thus follows the intent of the broadest
definitions of airmanship. TEM was first proposed by Helmreich, Klinect and Wilhelm
(1999) as part of the revision of crew resource management (CRM) training for airline
pilots, and is a return to the original intent of CRM, this being error handling and crew
interaction in order to avoid accidents (Helmreich, Merritt & Wilhelm, 1999). However,
TEM is also relevant to single pilot operations and this was the basis for the GAPAN
program. The TEM model provides a framework that demonstrates that management
actions by the flight crew in response to external threats and crew errors can render the
hazards arising from those threats and errors inconsequential, however, mis-
management can lead to an incident or accident. The relationship between these
elements is shown in Fig 1.4. The model has been found to be useful for aircrew in
presenting a way of thinking about flight-deck management, such that ICAO has
mandated its inclusion in ongoing flight crew training. (CASA, 2008).
15
Figure 1.4 Threat and Error Management Model from "Error management training: Defining best practice” by M. J. W. Thomas (2005). Australian Transportation Safety Bureau. Copyright 2005 ATSB.
The model can also be used for post hoc analysis of the kinds of threats or errors
that occurred in a flight and to draw attention to the skills and actions that led (or would
have led) to a safe conclusion to the flight. Helmreich (2002) lists the core CRM
behaviours for addressing TEM as being: “active Captain leadership; briefing known
threats; asking questions, speaking up; decisions made and reviewed; operational plans
clearly communicated; preparing/planning for threats; distributing workload and tasks;
and vigilance through monitoring and challenging” (p. 8).
The model itself does not address specifically the recognition of threats and
errors, nor the actions that need to be taken to manage them. This lack is addressed in
Thomas’s (2005) “mismatch emergence model” (see Fig 1.5). This model indicates the
balance between situational awareness of the actual state of the system and the mental
model of the desired state of the system. A mismatch between the two starts from a
feeling that “something is wrong” or recognition of an abnormal state, threat or error. It
is suggested that an aviator who is ready to deal with threats and errors has good
situational awareness as well as an accurate mental model of the desired state of the
system (based on a good knowledge of the aircraft and the operating environment). In
addition, the pilot must be able to develop a dynamic plan to achieve a safe outcome.
The “mismatch emergence model” is presented as an approach to the training of threat
16
and error management, thus providing for threat or error recognition, missing from the
TEM model.
Figure 1.5 Mismatch emergency model from "Error management training: Defining best practice” by M. J. W. Thomas (2005). Australian Transportation Safety Bureau. Copyright 2005 ATSB.
Thomas (2005) summarises the non-technical skills and generic error-
management strategies that are required to detect and effectively manage threats and
errors as being cognitive skills such as situation awareness, vigilance and monitoring,
self-regulation and problem identification, and interpersonal skills including
communication, negotiation, assertiveness and task management. The parallel with
airmanship is seen in the central requirement for situational awareness, and an
assumption that skill and proficiency is present in order to effectively correct and
recover, and self-regulation could be considered a form of discipline. The inter-personal
skills of communication and task-management were also emphasised in Ebbage and
Spencer (2003) and Edwards (2013), although assertiveness and negotiation are not
mentioned in the two main models of airmanship.
Mavin (2010) proposes a model for assessment of pilot performance (MAPP) that
has very similar components to the Kern (1996, 2009a) Edwards (2013) and Ebbage and
Spencer (2003) models of airmanship. Mavin reports asking simulator instructors what
qualities they were most concerned about in the captaincy upgrade training of airline
pilots. In other words, what is it that makes a pilot competent to move from First Officer
to the rank of Captain, with its associated responsibility and authority. The outcome of
this research was the MAPP model which presents three essential skills that pilots must
have: (1) situational awareness; (2) decision making and consideration of risk; and (3)
the ability to maintain the aircraft within acceptable tolerances. These essential skills are
supported by the enabling skills of aviation knowledge, management (of workload,
threat and error), and communication. Mavin and Roth (2014) represent these elements
17
as ‘six discursive repertoires’ (p. 215) for the discussion of pilot performance without
dividing them into essential versus enabling skills. Mavin’s model, like Ebbage and
Spencer (2003), places more emphasis on cockpit management and communication,
than does the Kern model of airmanship. Mavin (2010) does not specify in detail the
areas of knowledge required. Discipline is not mentioned, nor other personal qualities or
attitudes that the other models include, perhaps because these are not as relevant to
simulator instructors making decisions about upgrade training. The central importance
of situational awareness, decision making (or judgement) and aircraft handling skills
parallel the airmanship models very closely.
Airmanship Research
There has been little direct research on airmanship as a holistic concept, although
there is a great deal of research about many of the individual components, such as
situational awareness, decision making or CRM issues. However, there is some research
on what might be considered a “good pilot”, which directly relates to good airmanship.
Swauger (2011) reported on a survey conducted among South West Airlines First
Officers concerning what they expected of good Captains. When the results of this
survey were distilled down to a single descriptive sentence, it can be said that First
Officers value a Captain who is technically competent, psychologically confident, and
promotes an effective CRM environment. In a more detailed report of the study,
(Swauger, June, 2006) it was reported that 180 First Officers returned the surveys
(about a 7% response rate). Sixteen main categories of comment were reported in three
tiers based on how frequently the comments were made. The top three areas of
comment were: establishment of a good CRM environment (empowering and allowing
contribution); personality (friendly, relaxed, balanced) and technical expertise (good
book and aircraft knowledge, briefings and standardization). Second most frequently
mentioned characteristics were: instructional abilities (good teachers); flying skills
(precise, smooth and accurate); easy-going nature (unflappable, able to defuse stressful
events); and professional (without further definition). The final tier of characteristics
included elements that were mentioned less often, but were still considered important:
team building; open communication; company asset (considered to have a selfless
approach towards promoting company success, good ambassadors); flight management
(organising the flow of the flight); sense of humour; mentoring (passing on accumulated
18
wisdom, rather than just technical training); situational awareness; decision making; and
experience.
There are obvious commonalities between the characteristics reported in Swauger
(2006) and the airmanship models of Kern (1996, 2009a), Ebbage and Spencer (2003)
and Edwards (2013), although clearly the characteristics are not ranked in the same
order of importance as implied in these models. In Swauger, situational awareness and
decision making were mentioned least often of those characteristics listed while
knowledge (or at least aircraft knowledge) was considered of primary importance, along
with active promotion of CRM, team building and communication. Flying skills were
also reported as part of a good Captain, at the second tier of reported frequency.
Discipline was not directly mentioned by Swauger’s survey participants, although this
could have been implied in the mention of professionalism. Additional concepts,
mentioned in this survey but not directly included in the airmanship models, are the
mention of personality and nature, sense of humour, teaching and mentoring.
A slightly different selection of concepts were reported by Nergård, Hatlevik,
Martinussen and Lervåg (2011), following questioning of pilots’ views of desirable
personal attitudes in a series of interviews (15 group and 30 individual interviews for a
total of 250 pilots) held during or just after CRM courses at two Scandinavian airlines.
The pilots were asked to discuss and define a good pilot. All groups used the term
‘airmanship’ and commented on ‘the attitudes of a good airman’. They were then asked
to define airmanship and attitude. Nergård et al report that there was no single definition
for airmanship, for a good pilot or for the abilities or qualities of a good pilot. Skills and
abilities were seen by the participants to depend upon the kind of operation in which the
pilots were engaged. Good communication skills were also seen as important. A good
attitude was reported as being reflected in acceptance within the pilot community,
acting and socialising appropriately within an unwritten code of conduct. Being a good
crew-member was mentioned, and included knowing how to work with others and
exhibiting cooperation and interaction. Humility was discussed in terms of recognising
one’s own errors and helping others with problems. Self-awareness, self-efficacy and
self-esteem were also included as features of a good pilot. Finally, learning to fly the
aircraft using CRM was regarded as essential. Nergård (2014) reported on an additional
20 interviews, as well as those reported earlier, and included additional comment on the
ability to fly the aircraft being central to airmanship. The ‘rationale’ for airmanship was
19
reported as a strong awareness that the pilot’s actions create safety, and that one can die
from making mistakes.
Nergård et al (2011) report a second study that involved 174 participants
completing a questionnaire developed from the statements evoked in the first study. A
principle components analysis extracted four factors (explaining 60% of the variance):
principles in airmanship knowledge, self-awareness, flying skills and CRM. The
airmanship factor included thinking ahead of one’s actions, awareness of aircraft
limitations, initiative to improve, monitoring for fatigue and stress, being a good role
model, applying good airmanship, knowing rules and regulations, awareness of own
errors and limitation and helping others. The latter two also loaded on the CRM factor,
which included statements about cooperation and interaction, communication and
coordination. The self-awareness factor included awareness of own errors and
limitations, communication, asking for help, and interaction. Flying skills covered
knowledge of rules and regulations, mastery of procedures, manual flying skills and
operating ahead of the aircraft.
The pilots involved in the Nergård et al (2011) and Nergård (2014) studies
mentioned the skills and abilities to fly the aircraft, agreeing with the basic airmanship
requirements of the Kern (1996), Ebbage and Spencer (2002) and Edwards (2013)
models of airmanship. However, like Swauger’s (2006) sample, these pilots also
omitted mention of discipline. CRM competence was considered essential and while
good crewmember skills such as communication, cooperation and interaction were
included, there appears to have been no mention of situational awareness (as such,
although thinking ahead and operating ahead could be indicative of it) nor of judgement
or decision making. Among this group, judgement and situational awareness do not
appear to have the same central focus as that accorded them by Kern (1996). In
Nergard’s study, the only knowledge mentioned was that of rules and regulations,
although awareness of aircraft limitations implies knowledge, again not reflecting the
central, supportive role that Kern considers for all aspects of knowledge.
Other research specifically claiming to address airmanship includes Ali,
Gluckenberger, Rossi and Williams (2000) who carried out an evaluation of training in
a time-speeded environment and self-instructional techniques for what they referred to
as “airmanship tasks on a flight simulator”. It turned out that these were simply basic
flight manoeuvres (straight and level, climbing and turning) assessed relative to
20
tolerances in altitude, heading and airspeed. The term “airmanship”, while in the title,
was not mentioned within the paper itself. Clearly they see airmanship as being hands-
on stick and rudder skills.
A similar approach is seen in the 8th Edition of “Van Sickle’s Modern
Airmanship” (Welch, Bjork & Bjork, 1999; first edition 1957) which, interestingly,
does not use the term “airmanship” anywhere in the 950 pages of text. One is left to
assume that a sound grasp of the material included in the text will somehow translate
into “airmanship’. The chapter headings include all the technical aspects of operating an
aircraft from basic aerodynamics, through propulsion to avionics and meteorology, with
a brief discussion of aeromedicine. There is no discussion of group dynamics,
communication, or personal limitations and nothing remotely resembling crew resource
management or threat and error management. The section on accidents mentions pilot
error, inexperience, overconfidence, and poor judgement, merely concluding that they
will improve with experience (p. 941). In this context, airmanship appears to be equated
with a good working knowledge of aircraft systems and environment, and the
aeromedical aspects of self-knowledge, and glosses over what would be considered
human factors or non-technical skills in modern aviation training, and not mentioning
situational awareness or decision making directly, nor discipline at all.
Airmanship and National Culture
The model of airmanship presented in Kern (1996) is based on historical aviators
and interviews and a survey carried out in the USA. The Ebbage and Spencer model and
the Edwards model were developed in the UK, although utilising information from
around the globe. Swauger (2008) surveyed American pilots, Nergård (2011, 2014)
interviewed and surveyed Scandinavian pilots and Mavin (2010, 2014) interviewed
Australian pilots. It is possible that the differences between the models and between
findings reported in the research papers, are due to the national culture in which
participants were sourced. However, the pilot cultures in these countries are more like
each other than they are like other nationalities for example, Asian or African countries.
(Helmreich & Merritt, 1998).
Helmreich and Merritt (1998) presented a survey of pilot groups around the
world. Their 15,454 pilot participants came from 23 different countries and 33 airline
groups. They reported that there was clear recognition across all nationalities that
21
communication and coordination on the flight deck are important, and secondly that all
pilots appeared to want to talk freely with their peers and superiors. There were some
items on which there was strong difference of opinion across the different nationalities;
however generally the Anglo and Scandinavian countries were at the same end of each
distribution of nationality scores, and often British Hong Kong, Australia and the USA
were adjacent or quite close in the distribution of countries. The only exception to this
was a set of questions that related to realistic appraisal of personal performance under
stress. The British Hong Kong pilots were at the top of the distribution (most realistic)
while the USA and Australia were about half way down. The Scandinavian countries
fell between these, in the upper half of the distribution.
A cluster analysis of responses to questions about work values grouped the
‘Anglo’ countries together (Australia, USA, NZ, Ireland and the USA) and the
Scandinavian countries together (Norway, Sweden and Denmark); other clusters were a
Germanic cluster, South America plus some Asian countries, and a cluster of just Korea
and Taiwan. Italy was unique. The Anglo, Scandinavian and Germanic clusters were
more like each other than they were like the other clusters in their higher ranking of
financial reward and advancement and of questions relating to independence and lower
ranking of questions relating to order within the job.
Views on airmanship, or what makes the best pilot, are quite likely to vary across
different national cultures, given the different values placed on inter-personal
interactions in different cultures. However, the USA, Australia and UK appear to be the
least likely to differ given their common cultural background and the findings of Merritt
and Helmreich (1998).
Airmanship, Crew Resource Management and Selection
An historical view of what constituted airmanship, at various times, can be gained
by looking at what was selected for (presumably to identify good pilots or potentially
good pilots) and trained for (presumably to improve pilot performance), on the
assumption that a good pilot exhibits good airmanship.
Historically (up until 50 years ago) selection of pilots was dominated by
assessment of flying skills or the ability to develop such skills based on performance on
ability and cognitive tests (Hunter and Burke, 1994). Based on this selection activity
and on Van Sickles’ Modern Airmanship editions from 1957 to 1999 focussing on
22
technical knowledge alone (Welch, Bjorg & Bjorg, 1999), airmanship appears to have
been about flying the aircraft skilfully and precisely. There were some early comments
about the kind of person who made a good pilot, unfortunately somewhat contradictory:
‘high spirited and happy-go-lucky sportsman’ (Rippon & Manuel, 1918, cited by
Nergård et al, 2011) and ‘quiet, methodical men’ (Dockeray & Isaacs, 1921, cited by
Nergård et al, 2011).
Concern for the human-machine interface and pilot performance were frequent
research topics reported in the years after the Second World War. In addition there was
progress in relating psychological research in attention, perception, learning, decision
making and psycho-motor skills to aviation (Bond, Bryan, Rigney & Warren, 1968;
Jensen, 1997). This includes a history of aviation psychology conferences dating back
to the 1960’s (Maurino, 1994). Awareness of human factors as they effect pilot
performance was becoming more widespread, and the first human factors course for
airline pilots was initiated in 1977 by KLM (Hawkins, 1987).
This perhaps reinforced the perception of airmanship as an individual quality, but
gradually widened the scope of airmanship from just technical skills to include other
aspects of personal behaviour, and on occasion left technical skills behind:
Airmanship is used to describe a level of performance, but it is more often used to describe a personality trait. As performance, it describes excellence in the decision making and judgement of the pilot, as a trait, it is the ability and tendency to define the relevant aspects of an aerial problem or opportunity. (Mané, 1981, p. 161)
In the 1970s a series of aircraft accidents occurred in the USA that involved ‘pilot
error’ or poor performance of the flight crew in handling unusual circumstances. The
main aspects of such poor performance were identified as failures of communication,
decision making and leadership, rather than anything lacking in ‘stick and rudder’
aircraft handling skills. A workshop on the issue in 1979, sponsored by NASA, led
many of the attending airlines to develop new training programs to address this lack of
inter-personal skills. The label ‘Cockpit Resource Management’ was applied to such
programs, although this later evolved to Crew Resource Management (Helmreich
Merritt & Wilhelm, 1999). The initial program in Australia was run by Trans Australian
Airlines (TAA) in the mid-1980s, as Aircrew Team Management (Margerison, McCann
23
& Davies, 1986), although as other Australian airlines developed their own courses and
international interaction progressed, the term CRM was also applied in Australia.
The focus and content of CRM courses has evolved over the decades since
(Helmreich, Merritt & Wilhelm, 1999), but the essence of the training remains as (or
has returned to) the development of inter-personal skills to manage errors and threats as
they occur during flight (Helmreich, Klinect & Wilhelm, 1999). In recent years, CRM
has expanded its application to other team environments in aviation, including air traffic
control and maintenance crews, and has also been introduced to other teamwork
environments, for example marine bridge crew, railway crews and surgical teams (Flin,
O’Connor & Crichton, 2008; Hayward & Lowe, 2010).
Topics covered under the CRM umbrella relating to inter-personal behaviour on
the flight deck include situational awareness, decision making, communication,
teamwork and leadership (Helmreich & Foushee, 2010, Jensen, 1997). Such skills are
also termed non-technical skills, which were defined by the European Joint Aviation
Authorities as “the cognitive and social skills of flight crew members in the cockpit, not
directly related to aircraft control, system management and standard operating
procedures” (Flin, et al, 2003, p. 96). As CRM training became accepted, so the skills
that it represents have become absorbed into the overall definition of airmanship.
Inclusion of both human factors and CRM topics in the syllabus for pilot licence
training followed the ICAO 1986 Resolution on Flight Safety and Human Factors
(ICAO, 1989). In Australia, Human Performance and Limitations (HPL) was introduced
into the theory syllabus in 1991. HPL became part of the examination system in 1992
for the Airline Transport Pilot Licence and in 1995 for the Private Pilot and Commercial
Pilot Licences. Mandating the study of HPL as part of basic pilot licencing reinforces its
importance as part of good airmanship.
The accident record and training response has drawn attention to the inter-
personal behaviours required for effective multi-crew performance, however concern
that CRM training alone may not be ‘enough’ to counter ingrained non-CRM
behaviours has led to the increasing inclusion of such skills in selection procedures
(Bartram & Baxter, 1994; Damitz, Manzey, Kleinmann & Severin, 2003; Salas, 2014).
The challenge for selection has been how to assess and predict such behaviours,
especially in the selection of ab-initio student or cadet pilots. One response was the
24
development of assessment centres to examine ‘live’ social interaction (Damitz, et al,
2003), however the expense of such centres and problems with inter-rater reliability and
construct validity has led to recent consideration of whether social competence
questionnaires may replace such centres (Hoerman & Goerke, 2014). The importance of
such behaviours for aircraft operation has certainly percolated to the workforce. An
assessment of the knowledge, skills, abilities and other factors required for effective
operation on advanced flight-decks found that while cognitive and psycho-motor
abilities were seen as remaining relevant, it was inter-personal skills that gained the
highest relevance rating from pilots themselves (Goerters, Mashchke and Eiβfeldt,
2004, cited by Damos,, 2014).
Personality has figured in selection activity from the early days of flight, however
the use of measures of personality has showed only modest relationships to training
outcomes (Campbell, Castenada & Pulos, 2010; Martinussen, 1996). Development of
the Big Five factors of personality organisation (Costa & McCrae, 1989) led to further
attempts to relate personality to flight performance, with various degrees of success
(King, 2014; Rose, Barron, Carretta, Arnold & Howse, 2014). Openness to experience,
conscientiousness and extroversion show some potential, however application depends
on specific job analysis and assessment of predictive validity (Campbell et al, 2010;
King, 2014; Rose et al, 2014; Tvarynas, 2014). In addition, neuroticism/emotional
stability has been suggested as a screening variable (King, 2014). Personality is seen as
a potential source of predictive power for selection purposes, and has been part of some
discussions of airmanship. Kern (1996, 2009a) does not include any personality
variables, unless discipline is counted as such, in his model of airmanship. Discipline is
also included by Ebbage and Spencer (2003) and Edwards (2013). Ebbage and Spencer
mention confidence and self-improvement, while Edwards adds self-respect, integrity
and honesty, all of which could be considered at least personal qualities, although not
formal personality traits.
The history and development of aviation selection and training shows the
increasing emphasis on human factors and psychological aspects of the flight task.
These include decision making and situational awareness, and inter-personal skills like
communication and teamwork. It would therefore be expected that these concepts have
made their way into pilot’s generally held ideas of airmanship. While such concepts
feature in the models of airmanship, these models have not been tested empirically.
25
Conclusion
Kern’s (1996) model of airmanship is based on the study of historical heroes of
aviation, accident analysis and interviews with military aviators in the USAF. The
responses of 300 USA aviators, including civilians, to a survey were also drawn upon
(T. Kern, personal communication, June 10, 2009). This research activity was not
presented in a scientific, peer reviewed format, but rather as an instructional textbook
aimed at ab-initio student pilots. Similarly, Edwards (2013) book is presented as a
guide, apparently based on personal experience and analysis of aircraft accidents. The
model of airmanship proposed by Ebbage and Spencer (2003) was presented as a
theoretical paper at a military training conference with little discussion of any research
basis on which the model was founded, although both of the authors were employed by
the training provider to the RAF. Thus, none of the major models of airmanship have
been published in a peer reviewed format, nor does it appear that there are peer
reviewed reports of validation by application in a real world setting. It is worth
considering whether any of these models are relevant to and descriptive of how aviators
actually see airmanship. The research of both Swauger (2006), Nergård et al (2011) and
Nergård (2014) suggests that there may be some differences between Kern and Ebbage
and Spencer’s ideas and the opinions of airline pilots. Some of the differences may be a
result of the military opposed to civilian flying environments, with discipline being a
more military concept and personality and CRM issues being more relevant in civilian
environments.
The existing models of airmanship are not based on systematic analysis of pilots’
views, nor have they been empirically tested. The current research aims to investigate
the views of Australian aviators about airmanship.
Training and Development of Airmanship in Australia
Both Kern (1996) and Ebbage and Spencer (2003) suggest that airmanship should
be an active and integrated part of overall flight and ground training rather than being
merely a by-product of conventional training. Kern (1996) promotes the integration of
airmanship within ongoing aircraft handling skills training, and suggests three phases of
airmanship training: (1) for the instructor to explain the importance of airmanship and
instil motivation for improvement; (2) to model and teach airmanship; and (3) to
evaluate airmanship actively and aggressively. Kern emphasises the importance of the
26
instructor as a mentor to introduce a sound aviation culture, model appropriate
behaviours and enforce standards and expectations. Kern encourages the use of his
model of airmanship as a template for the structuring of pre-flight briefings and post-
flight debrief and reflection, an activity that should, he writes, continue throughout
one’s flying career, and not just take place during initial training. He was writing for
both civilian and military audiences, with sections of the text for instructors as well as a
section that could be used for self-guided development by student pilots.
Ebbage and Spencer (2003) outline a similar three phase approach to training
airmanship, proposing:(1) explanation of the concept and its importance, by discussion
of case studies; (2) instruction covering the required knowledge, skills and attitudes; and
(3) use of objective assessment with provision of feedback. Given the presentation at a
military training conference, presumably the paper was directed mainly at military flight
training.
Edwards (2013) includes a section on ‘How is this learnt?’ in each chapter on his
three main aspects of airmanship: technical, operational and non-technical elements. He
does not set out an overall training strategy for airmanship, but consistently returns to
study, practice and focussed training for each part of each element. Edwards emphasises
the importance of the debrief activity, whether after a training session or crew or self-
debrief after a flight.
The type of initial training differs between military and civilian sectors of the
aviation industry, certainly in Australia. Military flight training caters for highly
selected cadets who have to meet strict performance expectations in a very high tempo
course. There are intensive civilian training programs leading to a commercial pilot
licence with the expectation for graduates of proceeding on to an airline career.
However, these programs are not quite as high-tempo or inflexible as the military
model. In contrast, most civilian training in flying clubs may be in response to interest
or enthusiasm rather than a career move, and tends to be self-funded and self-paced.
Airmanship is included in current military ab initio training, in both ground
school and flight training in Australia (Carrick, Graham, Healey & Pickard, 2008, see
Appendix A). Airmanship instruction takes the form of classes covering safety and non-
technical skills, which are examinable. Airmanship is demonstrated during flight
training and is assessed, along with preparation and technical aircraft handling, at
27
intervals throughout the flight training syllabus. Thus “airmanship” in this environment
presumably does not include preparation or aircraft handling skills, unlike the two
models described above.
In the Australian civilian aviation industry airmanship, as such, is not part of the
theory syllabus for flight licenses, although the study of human factors is included,
forming the Human Performance and Limitations part of the VFR Syllabus and ATPL
Syllabus. Airmanship is supposed to be included in pre-flight briefings during flight
training (CASA, 2006) and Licence flight tests have included a subjective assessment of
airmanship carried out by the testing officer. From 2009, the human factors (HF) and
threat and error management (TEM) issues covered in the human performance and
limitations part of the theory syllabus have been explicitly tested in pilot licence flight
tests. The criteria to assess competency in HF and TEM in flight tests were introduced
by the Civil Aviation Safety Authority (CASA) on the basis of being part of “old-
fashioned good airmanship” (CASA, 2008) and the suggestion is that for “CASA to link
airmanship to human factors, is in effect, tantamount to bringing science to the often
nebulous concept of airmanship” (CASA, 2008, p. 8). CASA requires the following
five elements to be assessed during the flight test: maintain effective lookout; maintain
situational awareness; assess situations and make decisions; set priorities and manage
tasks; and maintain effective communications and interpersonal relationships (CASA,
2008, p. 7). In addition, there is also assessment of recognition and management of
threats, errors and undesired aircraft states, as assessment of the threat and error
management part of the syllabus (Carrick, 2009).
The CASA publication “Flight Instructor Manual – Aeroplane” does include brief
comment on airmanship in most of the air exercises covered, generally relating to
lookout and smooth handling. The introduction to the manual provides the following
requirements for airmanship: “The instructor must possess and outwardly demonstrate a
high standard of discipline, common sense and initiative, which are the cornerstones of
airmanship. The instructor must also display leadership qualities, without which the
above qualities will be lacking” (CASA, 2006, p. iii). This brief description does
mention discipline and leadership, but not any other elements of the various models of
airmanship.
Ongoing training differs somewhat between industry sectors in aviation. Ongoing
training in the military includes conversion training onto specific aircraft types, post-
28
conversion training and development in operations and tactics, refresher and currency
training. For some, career development may include advanced training, instructor or test
pilot courses, or exchange duty with air forces of allied nations. In the airline sector,
there are several types of pilot training including: induction and procedural training for
newly hired pilots; upgrade training to change aircrew status (such as first officer to
captain); conversion training onto a specific aircraft type within an airline; and
mandatory recurrent training (usually six-monthly) which ensures that aircrew maintain
the required skills, proficiency and knowledge to operate a particular aircraft (Orlady,
2010). Under the current structure of the industry in Australia, “air work” covers
operations that are commercial but not airline, including flying doctor services,
helicopter operations, agricultural operations and charter flying, as part of the broad
general aviation sector. Ongoing training for “air work” pilots varies between operators,
and may parallel the processes described above for airline pilots, or may be simply
regular proficiency checks. In the private side of general aviation, business, club and
recreational flying, post licence training can be infrequent, and while all pilots are
expected to keep current on their aircraft type, this may be no more than an occasional
check ride with a flying instructor.
Crew resource management (CRM) is a formal part of pilot training in airlines,
mandated by ICAO and in Australia, CASA, with the intent to improve team
coordination and monitoring by utilising all available resources - people, equipment,
and information (Salas, Burke, Bowers, & Wilson, 2001). Airlines around the world
have introduced and maintained CRM from the mid 1980’s as adjunct courses dealing
with interpersonal communication, decision making, leadership, situational awareness,
conflict resolution, and group processes, though course content and duration has varied
(Flin & Martin, 2001; MacLeod, 2005). More recently threat and error management
(TEM) has been incorporated into CRM training, such that the two terms are often used
together or synonymously (CRM/TEM). The term “non-technical training” has also
been used to describe human factors and CRM/TEM training, to distinguish it from
technical training (hands-on, stick and rudder and procedures training). Military training
in Australia has included CRM from the late 1990’s (Shepherd, 2005). The themes
covered in the CRM courses are closely aligned with the concepts that are included in
definitions of airmanship, such as situational awareness, decision making,
29
communication and team building, although not taught using airmanship as part of the
terminology.
The question remains, whether airmanship itself is considered part of the training
of Australian aviators, or is it something that “hopefully” develops without formal
instruction, almost as a side-product of technical and non-technical training.
Current Research
Airmanship is a term that continues to be used in the aviation industry and media,
despite suffering from a lack of specific definition, and plethora of ideas about what that
definition should be (Marchbank, 2008). Kern (1996, 2009a) is the most widely quoted
source and his model of airmanship includes discipline as the cornerstone of
airmanship, skill and proficiency as the bedrock principles on which the six pillars of
knowledge rest. Knowledge of one’s self, the team, the aircraft, the environment, risk
and the mission are essential to support good situational awareness resulting in good
judgement (or decision making) to achieve the goal of safety, mission effectiveness and
efficiency. The model developed by Ebbage and Spencer (2003), specifically addressing
military training, is based on Kern’s model but draws more attention to attitudes and
self-assessment, and to skills required in multi-crew flight such as communication, crew
co-ordination, interpersonal skills, leadership and flight-deck management. Edwards
(2013) considers the knowledge, skills and attitudes across technical, non-technical and
operational elements of aviation that lead to safe, efficient and effective flight.
None of the existing models of airmanship have been objectively, empirically
assessed in a peer reviewed format. Both the Kern and Ebbage and Spencer models
have a military influence. The different emphasis seen in Swauger’s (2006) research and
that of Nergård et al. (2011) and Nergård (2014) may be a reflection of a military vs
airline contrast.
Both Kern (1996) and Ebbage and Spencer (2003) recommend the integration of
airmanship training into initial flight training activities, and specify the phases that
should be included in airmanship training. This does occur in ab initio military training
in Australia (Carrick et al., 2008) albeit with an apparently narrower definition. Civilian
ab-initio training includes some of the elements of airmanship as defined by the models,
while not labelling them as such. It appears at first glance that “airmanship” may be a
by-product of civilian training rather than a deliberate part of the process, in Australia.
30
However, this is a conclusion drawn from the training publications, rather than from
direct observation of training activity. There has been no specific research into the
training of airmanship, per se, in civilian ab-initio training environments in Australia.
The current project aims to redress the lack of empirical evidence regarding
aviators’ views of airmanship and to develop an overview of what it is that Australian
aviators consider “airmanship” to be, whether airmanship is specifically included in
pilot training in Australia, and if so, how. Differences in views between aviators in
military and civilian environments and differences in approach to training in these
environments and at the different levels of training are also considerations.
There are four main parts to this research. Firstly, two surveys of Australian
aviators were conducted to find out what they consider “airmanship” to be. The initial
survey covered the views of mainly civilian general aviation pilots while the more
extensive second survey was able to include a comparison of responses between
aviators from military and civilian background and different current sectors of aviation
activity. In both surveys a comparison was made between the participants’ views and
the main models of airmanship. Secondly, the same aviators were asked what their
experience was in ab initio training with regard to the development of airmanship, and
its inclusion in the current training that they undertake. Thirdly, a selection of military
and civilian flight instructors at ab-initio and advanced levels of training were
interviewed about their definition of airmanship and how they include these concepts in
their training of students. A comparison of these responses between sectors and levels
was conducted, and considered in terms of the training recommendations of Kern
(1996) and Ebbage and Spencer (2003). Fourthly, the structure of a proposed new
model of airmanship was tested through a survey of Australian pilots.
An analysis of a first survey of Australian pilots forms Chapter Two of this thesis.
This chapter explores the results of an initial on-line survey that reached mainly pilots
from the general aviation sector and addresses their definition of airmanship, training
experiences and the development of airmanship during their flying experience.
Comparison with the Kern (1996, 2009a), Ebbage and Spencer (2003) and Edwards
(2013) models is also reported.
Results from a more extensive and enlarged online survey are presented in
Chapter Three. A greater number of respondents allowed a comparison between civilian
31
and military background pilots. The match between the concepts raised by the
respondents and the Kern (1996, 2009a), Ebbage and Spencer (2003) and Edwards
(2013) models is explored. The training experiences reported and current training
activity is analysed with reference to the recommendations made by Kern (1996) and
Ebbage and Spencer (2003).
The next chapter, Chapter Four, examines the responses of flight instructors in
military and civilian environments, at ab initio and more advanced levels, to questions
presented in individual interviews about the definition of airmanship, its inclusion in
flight training and the techniques used to assess airmanship. The responses of ab-initio
and advanced instructors in each sector are compared, and matched with the models of
airmanship. The responses between the military and civilian instructors are compared
and discussed in terms of the match with the recommendations for instruction made by
Kern (1996) and Ebbage and Spencer (2003).
The first three studies triangulate onto the representation of airmanship among
Australian pilots, and the fifth chapter summarises the conclusions regarding the
definition of airmanship and presents a new model of airmanship. This model proposes
five domains of airmanship concepts and includes consideration of the context in which
airmanship occurs.
The sixth chapter reports a further survey, in which Australian pilots were asked
to rate the relationship between the main concepts making up airmanship, to confirm the
domain structure of concepts in the airmanship model presented in Chapter 5. This
resulted in some revision of the model.
The final chapter summarises the conclusions reached in the foregoing chapters.
Limitations of the current research are discussed and directions for future research are
explored.
32
Chapter 2 Page | 33
Chapter 2. Airmanship: Definition and Development
among Australian Pilots
The term “airmanship” is used throughout the aviation industry. It appears in
the civilian “Flight Instructor Manual – Aeroplane” (CASA, 2006) as a general aim
for aviators, is part of the flight tests for licensing and is a part of the assessment of
trainee military aviators (Carrick, Graham, Healey & Pickard, 2008). There is some
debate as to the meaning of the term in general usage, Marchbank (2008) suggesting
that we need something better than the common statement about airmanship: “I know
it when I see it” (p. 11).
This chapter addresses whether Australian aviators consider airmanship to
include similar concepts to those described in the Kern (1996; 2009), Ebbage and
Spencer (2003) and Edwards (2013) models of airmanship, and whether there are any
additional themes brought up by members of the Australian aviation community. The
concepts included by Kern rest upon discipline as a key cornerstone for airmanship
as well as the bedrock principles of skill and proficiency. Knowledge of self, the
team, the aircraft, the environment (including physical, organisational and legislative
environments), risk and mission also support the “capstone outcomes” of situational
awareness and judgement (or decision making). In addition to these concepts,
Ebbage and Spencer (2003) suggest that the pilot should display suitable attitudes,
including professionalism and self-improvement, and the avoidance of hazardous
attitudes such as invulnerability and impulsivity. Further skills are included in this
model, specifically communication, crew coordination, interpersonal skills,
management of the flight deck, self-assessment and team leadership. Edwards (2013)
while organising his model differently, does not include anything that the above two
models do not. Together, these models provide a fairly wide range of concepts
covered by “airmanship”, but it is possible that Australian aviators would come up
with some alternatives in addition to, or instead of, these concepts. Rather than
provide a tick-list of possibilities, allowing for recognition and selection of concepts
for inclusion in a definition of airmanship, open ended questions requiring
spontaneous production of responses were preferred as a more useful method of
investigation.
34
Both Kern (1996) and Ebbage and Spencer (2003) recommend the integration
of airmanship training into initial flight training, thus an additional aim for the
current investigation is to identify the experiences of Australian pilots with regard to
instruction in airmanship during their ab-initio training. Kern (1996) suggests that
airmanship should be a career-spanning process of reflection and improvement and
Edwards (2013) also advocates life-long learning through study, training and
debriefing. Any further influences on the development of airmanship reported by
Australian pilots would be of interest to determine whether their experiences
conform to this ideal.
The aims for this study are to explore what Australian pilots think airmanship
is, and to discover what part airmanship played in their training. These aims were
addressed in the development of a short on-line survey of open-ended questions.
Specifically the intent of the survey was to identify the concepts that Australian
pilots associate with the term “airmanship”, to assess whether they think that
airmanship was part of their initial training and to determine what these pilots
consider led to them developing a sense of airmanship.
Methodological considerations
Collecting definitions of airmanship and views about airmanship training from
a selection of aviators has the potential to provide a rich source of information and
opinion. There is some challenge in determining a suitable method of analysis that
preserves the individuality of participants’ contributions while also ensuring that
comparison with existing models of airmanship can be conducted. This section
locates the research perspective within the range of qualitative epistemological and
methodological frameworks and outlines the approach taken to the analysis of the
qualitative data in the survey responses.
According to Gray (2009), there are two main philosophical perspectives
through which qualitative analysis is undertaken: naturalistic and progressive. A
naturalistic paradigm assumes that there is an objective reality that can be
approached by extensive collection of data in real settings. Researchers with this
outlook tend to be somewhat detached from the setting and objective in outlook,
collect data from a number of representative sources, and aim for minimal
interference and bias in the collection and coding of verbatim accounts. Within this
35
perspective common methodological frameworks include grounded theory, thematic
analysis and content analysis.
The naturalistic approach contrasts with progressive approaches such as post-
modernism and constructivism, which are based on the idea that reality and science
are socially constructed. In these approaches, researchers form part of the research
setting and it is considered that researchers should engage in reflexive and self-
critical dialog which forms part of the analysis. The aim is to reveal hidden realities,
lived experiences and/or structures of consciousness among the participants.
Methodological frameworks include phenomenology, ethnography and hermeneutics
(Liamputtong, 2013).
The current research is aligned with the naturalistic perspective, stemming
from an objectivist epistemology. The plan is to ask direct questions about aviators’
thoughts and experiences relating to airmanship and conduct an objective assessment
of their contributions. The most appropriate methodological framework is that of
content analysis. Grounded theory and thematic analysis assume separation from
existing ideas or models related to the topic area. The expectation in these
approaches is that themes and theories will be developed independently and are
based in the data. In the current research, existing models are well known, and a
deductive or directed approach to analysis of content is preferable and more
appropriate.
Content Analysis
Content analysis adheres to the naturalist paradigm in analysing text materials
(Hsieh & Shannon, 2005), but includes a range of approaches from intuitive and
interpretive to systematic, strict and formulaic (Krippendorf, 2013). While some
approaches to content analysis look only for ‘manifest’ or obvious surface content of
communication, there is also room for more impressionistic interpretation. Hsieh and
Shannon describe three main applications of content analysis: conventional, directed
and summative. Summative content analysis is concerned with contextual usage, and
as such is language focused rather than content focused. Conventional content
analysis is useful when theory and previous research is scarce, as researchers look for
categories arising from the data and avoid preconceived ideas. This approach shares
a similar initial analytical approach to grounded theory (Corbin & Strauss, 2008) in
36
which there is no guidance from existing theory to develop coding, but content itself
leads to generalisations and thence to theory development (Potter & Levine-
Donnerstein, 1999). It does need to be recognised, as Braun and Clarke (2006)
comment, that researchers cannot free themselves completely from their theoretical
background and data are not coded in an epistemological vacuum. Directive or
deductive content analysis recognises that pre-existing theory and research is known
to the researcher. Thus existing theory and research are used overtly in the
determination of coding frameworks (Mayring, 2000). Researchers begin by
identifying key concepts or variables that have been presented in the relevant
literature. Established theoretical positions are used to create initial coding
categories. A first run through the data helps produce operational definitions
(Krippendorff, 2013). As further ideas or concepts that are not covered by the coding
framework are encountered in the data, new categories or sub-categories can be
created and defined (Hsieh & Shannon, 2005). Findings can offer both supporting
and non-supporting evidence for existing theory, because both are searched for in the
coding process. As Hsieh and Shannon note “a directed approach makes explicit the
reality that researchers are unlikely to be working from the naïve perspective that is
often viewed as the hallmark of naturalistic designs” (p. 1283).
The intent in all forms of content analysis is to reduce text into categories. A
category may consist of one or more words, or may represent an idea or concept
expressed in various ways (Weber, 1990). Content analysis should not be seen as
simple word count: interpretation is required to identify synonyms, explanations or
euphemisms within the material under analysis (Stemler, 2001). Nonetheless, content
analysis can result in the production of frequency data (Wilkinson, 2000) and thus
allow some quantitative analysis of what started off as qualitative data (Ryan &
Bernard, 2000). In fact Weber (1990) notes that perhaps the best content analysis
uses both qualitative and quantitative operations on textual material, thus combining
what are usually thought of as antithetical modes of analysis.
Reliability and Validity in Content Analysis
Consideration of reliability and validity is of considerable importance in any
qualitative research methodology. Potter and Levine-Donnerstein (1999) address the
reliability and validity of the coding process in content analysis. They consider that
37
reliability and validity should be conceptualised differently across various forms of
content analysis. They consider that two foundational issues must be determined, the
nature of the data content and the role of theory, before considering the extent of
coding checks required. Content can be manifest, latent or projective. That is, either
the coder is coding the obvious surface expression of the participants (manifest
content); or looking beyond the surface to latent meaning. Latent meaning could
include looking for patterns in the data or may require coder interpretation of deeper
meanings. Projective content is revealed as coders look at content as symbols and use
their own mental schema to judge the meaning within the content. The role of theory
has been discussed above, deductive content analysis uses existing research or theory
to guide the development of coding schemes. The alternatives presented by Potter
and Levine-Donnerstein are not using theory at all, or inductive development of
theory from data after analysis.
Validity is a two-step process: firstly, the development of a coding scheme that
details the codes and code definitions; and secondly, comparison of coder decisions
against an expert standard. Validity can be maximised by ensuring a strong coding
scheme that reduces the need for coders to develop their own interpretations. Using a
coding scheme ensures that coding is uniform and can be considered systematic and
thus scientific. Reproducibility of coding output is the strongest realistic method of
determining reliability in content analysis (Potter & Levine-Donnerstein, 1999). This
quantitative approach to reliability in qualitative analysis involves the use of multiple
coders over common data, counting the areas of agreement and disagreement and
then calculating inter-rater reliabilities (Krippendorff, 2013).
Mayring (2000) reports increasing criticism of ‘classical’ quality criteria for
reliability applied in content analytic research. Alternatives that Mayring presents
include documentation of method, interpretation safeguards, and triangulation, that
is, using multiple sources, methods, investigators and theories (Lincoln & Guba,
1985). Braun and Clarke (2006, 2014) agree with this critique, charging that iterative
processes of checking and discussing coding to reach consensus is sufficient check
on reliability without calculating a number artificially.
38
Challenges for Reliability and Validity in Manifest Deductive Content Analysis
Potter and Levine-Donnerstein (1999) consider that the primary threat to
reliability in the analysis of manifest data is coder fatigue, due to the high levels of
concentration required. Ensuring that coding sessions are relatively short and spaced
out, coupled with several runs over the data, address this problem. Hsieh and
Shannon (2005) suggest that in a deductive content analysis, the researcher may
approach the data well informed but biased, and be more likely to find supportive
than non-supportive evidence of the theoretical framework. Discussion between
coders about uncategorised content, update of the coding guidelines and repeated
runs over the data address this challenge. In addition, Hsieh and Shannon indicate
that a desire to please the researcher may lead to participants agreeing with
suggestions or hints made by the researcher related to the theoretical framework.
Audit of the survey and interview questions and training of interviewers counter this
possibility. The third challenge that Hsieh and Shannon raise, is that over-emphasis
on the theory can lead to researchers ignoring context within which the phenomenon
occurs. One way to address this issue may be to specifically ask about context, or
include context references in the coding framework.
Procedures for the Current Research
In the current research, content is primarily manifest; that is, attention is paid to
the overt content of the data, with some interpretation in dealing with synonyms or
descriptions. The approach is that of deductive content analysis with existing models
of airmanship providing the initial coding categories (or concepts).
Hsieh and Shannon (2005) state that all content analysis should use the ‘seven
classic steps’, with variations depending on the approach to content analysis being
used. These steps are: formulate the research question; select the sample to be
analysed; define the categories to be applied; outline the coding process and conduct
coder training; implement the coding process; determine the reliability of the
analysis; and analyse the results of the coding process (Kaid, 1989, cited by Hsieh &
Shannon, 2005, p 1285). Mayring (2000) is more prescriptive regarding the approach
used, specifying that the sequence for deductive content analysis should include
specification of the initial, theory based categories, and revision of the categories as
coding progresses.
39
In the current research, the process used followed the basic classic steps, with
attention to the need to carefully construct the initial coding categories and revise
them as new categories (or concepts) appear in the data. The process included a final
return to the coded concepts in order to sort through them to establish possible
combinations of concepts and identification of any themes within the data. The
process also attempted to address reliability and validity requirements and counter
the difficulties and challenges for deductive content analysis raised by Potter and
Levine-Donnerstein (1999) and Hsieh and Shannon (2005). Coding sessions were not
prolonged, the coders frequently discussed the coding process, guidelines and any
difficult cases until consensus was reached, and there were several runs over the data.
The process can be outlined as follows:
1. Determination of the research question and identification of relevant
models
2. Development of data-capture strategies and decisions regarding participant
groups
3. Definition of the category system (concepts) from the models
4. Definition of coding guidelines (definitions, examples and coding rules)
5. Training of coders, discussion of coding guidelines
6. Initial run-through, preliminary coding and identification of additional
concepts with definitions and anchor examples
7. Revision of category system with additional concepts, discussion between
coders to ensure reliability of coding.
8. Further work through of material (iterate with step 6)
9. Final establishment of concepts, concept combinations and themes
10. Analysis and interpretation
This general process was applied in each qualitative study in the current
research, any deviations from it will be addressed in the report on each study.
40
Method
Participants
There were 40 participants in the survey. The respondents varied in level of
licence held from a student pilot through to an airline transport pilot licence (ATPL)
holder; in hours of experience (28 hours to over 20,000 hours, M = 3776 hours SD =
5,518 hours); and in length of time involved in aviation (from less than one year to
55 years in the industry, M = 17.8 years, SD = 13 years). Sixteen respondents held
Instructor ratings.
Measure
The full questionnaire is presented in Appendix B. The first part of the
questionnaire asked demographic questions about kind of licence held and flying
experience. The second part consisted of 11 open ended questions about: the
respondents’ definition of airmanship; the inclusion of airmanship in initial training;
development of airmanship in respondents; development of airmanship in students;
awareness of airmanship during current aviation activities; and examples of good and
poor airmanship. The final question asked for any other comments.
Procedure
Ethics approval was granted through ethics application, H-647-1207.
The on-line survey was promoted at CASA safety seminars from May to
August 2008 and in the May/June 2008 issue of Flight Safety magazine. This
magazine is produced by CASA and, at the time, distributed to all Australian pilot
licence holders. There was an option for participants to contact the researcher to
obtain a paper copy of the survey, instead of completing it on-line. Only one
participant took this option. There were 40 responses by September 2008.
Analysis
The survey responses were analysed according to the approach detailed in
‘Methodological Considerations’, above. Two coders worked on the data, both had
training in the analysis of qualitative data and the use of NVivo. NVivo9 (QSR,
2011) was used as the coding platform for determining concepts raised in the
responses. Response length ranged from 42 words to 781 words, with a mean of 308
41
words. Existing models of airmanship were used to create initial coding categories
(tree nodes) covering the basic concepts of those models. Initially, 12 tree nodes
were established to reflect the concepts included in Kern’s model and nine to cover
the additional Ebbage and Spencer (and Edwards) themes. Further categories (free
nodes) were created as other concepts were discovered. A total of 44 nodes were
developed, each representing concepts that respondents related to airmanship or
issues that they raised. The coders worked through the survey responses until they
were satisfied that a reliably coded data set had been achieved.
Initially, it was intended to conduct comparisons by experience, but a median
split would have been at 1000 hours or 20 years of experience, which would give a
“less experienced” group that was actually quite experienced. In addition, there was
some concern that hours had been mis-recorded for several respondents (over 30
years but only 700 to 1400 hours) although this may reflect a group dominated by
recreational aviation.
Results and Discussion
Definition of Airmanship
What is airmanship? The questions of most interest for the definition of
airmanship are Question 1: “What do you think airmanship is?” and Question 2,
which asked the respondent to elaborate on the answer given in Question 1. Other
questions relevant to the definition of airmanship include questions that asked about
the most important aspects of airmanship (Question 6), the situations in which
respondents were most aware of airmanship (Question 7) and about examples of
good and poor airmanship (Questions 9 and 10). Questions 3 to 5 and 8 are discussed
in a later section about the development of airmanship.
Table 2.1 shows the number of respondents who mentioned each of the
concepts found in the responses to Questions 1 and 2, and the total number of
respondents who mentioned each concept in any of the questions (i.e. questions 1 to
11). In the discussion of the concepts mentioned by the participants, any quotations
are identified by a three-digit participant number.
42
Table 2.1
Number of Respondents (of a Total of 40) who Mentioned Each Concept in Questions 1, 2 and Overall (Qs 1 to 11).
Concepts Q1 Q2 Overall Nil responses 0 10 0 N of respondents 40 30 40 Kern concepts Judgement or decision making 2 5 10 Situational awareness 10 9 22 Knowledge 6 5 9 Self 2 3 5 Aircraft 2 1 11 Team 0 0 1 Environment 0 0 1 Rules, regulations 9 4 20 Checks 1 5 14 Risk 3 5 10 Mission: Planning 9 10 16 Skill 9 2 12 Proficiency 1 1 1 Discipline 4 4 5 Concepts specific to Ebbage & Spencer/Edwards Attitudes: Hazardous 0 0 0 Professionalism 0 2 4 Self-improvement 1 1 6 Skills: Communication 4 4 15 Crew co-ordination 0 0 0 Interpersonal 1 1 2 Management of flight-deck 3 0 5 Self-assessment 3 1 4 Team leadership 0 0 0 Additional concepts (not in existing models) Safety 13 14 25 Courtesy/ethics/respect 12 12 12 Attitude / personal qualities 10 0 22 Common sense 9 5 9 No shortcuts 2 3 3 Lookout 2 1 19 Responsibility 2 0 5 Knowledge - medical 1 0 1
As shown in Table 2.1, the main themes of the Kern model were reflected in
some of the respondents’ comments. No respondents listed all the Kern concepts, nor
all of the concepts included in the Ebbage and Spencer or the Edwards models. There
were additional concepts that were expressed by respondents but are not included in
43
any of the existing models. The mean number of concepts raised per respondent was
2.95 in Question 1 (N=40). The mean for respondents who answered Question 2 was
2.90 concepts per respondent (N=30). Additional concepts were mentioned by more
of the respondents over the entire set of questions, compared with just Question 1 or
Question 2, as further probes led to more concepts being generated.
A chi-square goodness of fit (with α = .05) was used to assess whether the
Kern model concepts were equally likely to be mentioned across all the survey
questions (the Overall column in Table 2.1). The chi-square test was statistically
significant, χ² (13, N = 137) = 60.737, p < .001, indicating that some concepts were
mentioned more frequently than others. As an index of effect size, Cohen’s w was
0.67, which can be considered large.
Safety was the concept with the most mentions by respondents in Question 1,
Question 2 and across all questions. Kern did not include safety in his model,
possibly considering it an outcome of good airmanship rather than part of it. The
survey respondents mentioned safety as central to airmanship, with comments like:
“Airmanship is a way of being safe” (010), “An aviator's own attitude towards his /
her own safety and the safety of others” (016) and “Sensible aviation practices that
take into consideration the safety of yourself and others” (019). It should also be
noted that ‘safety’ was used as an example in the text of Question 2 (“For example, if
you wrote ‘being safe’, write in more detail what ‘being safe’ would involve”). This
may have influenced participants to include comment on safety, however fewer than
half the participants wrote about safety in either Question 1 or Question 2.
Judgement or decision making is the major capstone outcome according to
Kern, but was not considered by respondents to be particularly central to airmanship,
in that only two mentioned it in response to Question 1 and five in their elaboration
in Question 2. In contrast, situational awareness (SA) clearly is seen as a significant
concept relating to airmanship, with 10 respondents mentioning it in Question 1, 9 in
Question 2, and 22 respondents including the concept over all questions. Some
examples of comments coded in this theme are: “awareness of the operating
environment” (020), “being aware of the situation around you i.e. the weather, other
traffic” (012) and “I believe airmanship to be an awareness and understanding of an
aviator of their surroundings and what is taking place” (015). Lookout is an activity
upon which SA is dependent, in visual flight, and is also frequently mentioned
44
although not so much in response to the first two questions (being mentioned three
times) with 19 respondents including it over all questions.
Thirteen respondents referred to knowledge: six made mention of knowledge
generally; others specifically related to self (2), the aircraft (2) or to risk (3); with one
person commenting on knowledge about medical aid (something none of the existing
models included). There were no references to knowledge of the team in responses to
the definition questions, but this may have been a function of the mostly general and
recreational aviation sample, flying mostly in single pilot operations. Teamwork is
more important in the multi-crew cockpits of commercial aviation or military multi-
crew operations.
There were few specific references to the environment. However, when
elaborating upon the model, Kern divides environment into the physical, regulatory
and organisational environments. When considered at this level, nine respondents
mentioned rules, regulations and operating procedures in their definition (Q1 and 2),
and 20 respondents did so overall questions.
Those respondents who included flight planning, planning generally,
preparation and thinking about what one was doing, in their responses are shown as
“mission” in Table 2.1. Clearly there is consideration among respondents that
planning of what will be going on in flight is part of airmanship.
Skill and proficiency are separate items in Kern’s model. Twelve respondents
included skill in their definitions of airmanship, or stated “ability to control the
aircraft” (004); “aircraft handling” (037); or “operate an aircraft” (029). One of these
stated that airmanship “is usually referring to judgement and decision making more
so than manipulative skill” (022). Only one respondent specifically mentioned
proficiency: “as a pilot without a lot of hours I need to fly regularly enough to remain
competent” (009). One respondent gave as an example of good airmanship: “a flying
competition that required pilots to touchdown as close to a spot as possible on the
runway” (017) which indicates a skill based component within their conception of
airmanship.
Discipline is the cornerstone of airmanship, according to Kern (1996), and it
was included in their definitions by four of the respondents, with added comment in
response to Question 2 for example: “airmanship is the discipline that you apply to
45
your flying” (011) and “Discipline, implying self-discipline should be exercised at all
times as an essential element of airmanship” (034). Three other respondents wrote of
“not cutting corners” or “no short-cuts” (009, 033, 039) which could be interpreted to
indicate the application of discipline to complete each task in full.
The concepts included in the Ebbage and Spencer (2003) and Edwards (2013)
models that were not also included by Kern are shown in the middle section of Table
2.1. While Kern mentions good communication when writing about the
characteristics of teamwork, as part of his “Knowledge” structure, communication is
rather more active than just a “knowledge of” item would imply, and is included as a
skill set by Ebbage and Spencer and Edwards. Good communication is seen as
important to maintaining situational awareness and thus safety. This is recognised by
the four respondents who mentioned communication in their initial definitions or
elaboration, and by 15 respondents overall. Examples of such responses to Question
1 include: “Ability to communicate with other users of aircraft to achieve safe
reliable outcomes” (001) and “keeping in contact with other aircraft” (021).
Management of the flight-deck and self-assessment were each mentioned by
three respondents as part of their initial definition of airmanship. An example of this
is “The desired behaviours are such that I am effectively managing flight in a way
that both achieves the intended mission of the flight and manages the risks to both
my aircraft and to others” (040).
In addition to the concepts included in the existing models, the respondents
included a number of other concepts in their definition and elaboration of
airmanship. There were many mentions of a complex of ideas that can be represented
as courtesy, as well as mention of courtesy itself. These included consideration of
other airspace users, helping others, manners, doing the right thing, and ethical
behaviour, elaborated as “honest in all dealings and treat others as you would wish to
be treated” (001). Courtesy was always mentioned as part of a grouping of
characteristics that makes up airmanship, rather than alone, such as in the definitions:
“A combination of flying skills, common sense and courtesy” (002) and “Being alert
and abiding by all the rules and being courteous to other pilots” (008).
Common sense is a concept that was included by nine of the 40 respondents.
Three of the nine elaborated on common sense in Question 2: “Know your abilities
46
and operate within that area of competence” (001) and “the ability to think a situation
through logically” (037). The third response was just another mention of it:
“Recognition of the need to apply common sense” (036).
Some respondents mentioned attitude and various personal qualities as forming
part of airmanship, for example: “a combination of knowledge, discipline and
attitude toward flying” (025) and “attitude to safety and fellow aviators” (004). This
category also included other personal characteristics, for example: “Furthermore
good airmanship is a reflection of the quick thinking never panicking pilot, who
seems to have a dogged determination to keep his plane and contents in a safe
configuration” (003). There was also mention of a desire to do as good a job as
possible and having a sense of responsibility. None of the respondents specifically
mentioned the hazardous attitudes that are mentioned in the existing models and
listed as something to avoid, in their formulation of airmanship.
Several people mentioned the difficulty of defining airmanship, and also that it
is an integral or integrated part of flying.
Finally, the responses included the idea of setting priorities and managing the
flight, one of the human factors competencies for flight tests being introduced by
CASA (CASA, 2008), although only mentioned by five respondents overall. The
other CASA human factors competencies are lookout, situational awareness,
decisions and communication, and were all mentioned by respondents as being part
of airmanship as noted above, although none of the respondents listed all five.
Most important aspects of airmanship. Table 2.2 indicates the main concepts
raised by respondents when asked: “In your current aviation activities, are you
normally aware of airmanship as part of your operation? What would be the most
important things that come to mind?” (Question 6). Table 2.2 also indicates the
concepts mentioned in the examples of good and poor airmanship (Questions 9 and
10), as these are indicative of the airmanship related concepts that are relevant in
“critical incidents”, although this term was not specifically used.
47
Table 2.2
Number of Respondents who Mentioned Each Concept in Questions 6, 9 and 10 (of N =40). Concepts Q6 Q9 Q10 Nil Response to Question 3 9 13 Number of Respondents 37 31 27 Kern concepts Judgement or decision making 1 2 0 Situational awareness 5 8 5 Knowledge 0 2 0 Self 1 0 0 Aircraft 2 0 0 Team 0 0 0 Environment 0 0 0 Rules, regulations 0 1 0 Checks / SOPS 9 0 0 Risk 1 0 0 Mission: Planning 8 1 3 Skill & Proficiency 0 1 1 Discipline 0 1 0 Concepts specific to Ebbage and Spencer/Edwards Attitudes: Hazardous 0 0 0 Professionalism 0 0 0 Self-improvement 0 1 0 Skills: Communication 3 8 6 Crew co-ordination 0 0 0 Interpersonal 0 0 0 Management flight deck 1 0 0 Self-assessment 0 0 0 Team leadership 0 0 0 Additional concepts (not in models) Lookout 9 1 0 Safety 1 0 0 Courtesy/ethics/respect 2 3 0 Attitude / personal qualities 2 1 3 Common sense 1 1 0 No shortcuts 1 0 0 Everything 4 3 0
Situational awareness and lookout are related concepts that were frequently
mentioned in responses to these questions, supporting the emphasis given to the
concepts in the earlier questions. Similarly there were frequent mentions of planning
as being important to airmanship. Standard operating procedures (SOPs) and checks
were also mentioned frequently. SOPs are somewhat related to rules and regulations
48
as a knowledge item but were listed as the organisational side of environmental
knowledge. Rules and regulations were mentioned only once. Again, there were few
explicit mentions of skill or discipline in response to these questions.
Communication, or lack of it, was a major issue raised in examples of good and poor
airmanship, raised at least as often as situational awareness. Four respondents refused
to specify any most important aspect, stating that all aspects are important, or that
everything should be included.
There were no mentions of teams or the skills of crew co-ordination, inter-
personal skills or team leadership, perhaps a reflection of the predominately single-
pilot GA background of most of the respondents. There were also no mentions of the
physical aspects of the environment, hazardous attitudes or professionalism.
When is airmanship important? The scenarios shown in Table 2.3 were
mentioned by respondents in answer to Question 7 (“In your current aviation
activities, during what phases of flight or events would you be most actively aware of
airmanship issues?”) and in the examples of good and poor airmanship (Questions 9
and 10). There was frequent mention of weather and fading light, especially in the
examples. The most frequently mentioned scenario was of flying in the circuit.
Landing and take-off is a similar phase of flight to circuits, also mentioned
frequently in response to Question 7. Mention of high traffic situations, most likely
to occur in the circuit and round aerodromes, adds to the impression that constant
awareness of where other aircraft are in relation to one’s own, and one’s own in
relation to the ground, is a major part of airmanship. Twenty-five percent of the
respondents stated that they were aware of airmanship at all times. Some of the
respondents, while giving an example of good or poor airmanship, did not specify
any event or phase of flight, these are shown in Table 2.3 as “non-specific”
responses.
49
Table 2.3
Number of Respondents who Mentioned Listed Scenarios or Phases of Flight in Questions 7, 9 or 10 (of total N=40). Scenario or Phase Q7(When) Q9(Good) Q10(Poor) Nil Response 1 10 13 Responses to Qs 39 30 27 Weather/light 2 9 6 Pre-flight/planning 5 0 3 Taxi/runway use 2 0 5 Circuit / CTAF 8 9 10 Traffic 5 2 1 Aircraft problem 0 3 1 Approach/landing 5 1 1 Take-off / landing 8 0 0 All the time 10 0 0 Agricultural flying 2 1 0 Non-specific 0 4 3 Other 4 2 0
Development of Airmanship
Inclusion of airmanship in initial training. The majority of respondents
stated that airmanship formed some part of their initial flight training. The questions
were designed to determine what happened in the air, as part of flying instruction
(Question 3), as distinct from what was included in briefs or ground instruction
(Question 4). Responses to Question 3 (“Was the development of airmanship a
specific part of the initial flight training you did? Please explain what this involved”)
are shown in Table 2.4. Responses to Question 4 are summarised in Table 2.5.
Table 2.4
Number of Respondents Answering Question 3 (Inclusion of airmanship in initial training; N=40) Answer N Yes 18 Not really, but... 6 Related activities 3 Skill related 2 No 8 Can’t remember 2 No answer 1
Eighteen respondents stated “yes”, some with added explanation, to Question 3
as shown in Table 2.4. Three respondents listed airmanship related activities such as
50
checks, lookout and procedures. A further six responses were along the lines of “not
really, but…” and included concepts to do with airmanship. Two respondents
indicated that airmanship was part of their initial instruction but that this was only
skill related or did not include human factor related concepts. This gives a total for
inclusion of airmanship (however defined) as 29, or 73% of the respondents. Eight
respondents indicated that airmanship was not addressed (20%). Of the remaining
three respondents, two indicated that it was so long ago that they could not
remember, one stating: “I cannot recall that airmanship was specifically discussed
when I was learning to fly in 1954. I sometimes wonder if common sense was more
expected in those days” (034). One respondent did not answer the question.
Some examples of positive responses are: “Yes. From the start my instructors
emphasised that airmanship was what defined a good pilot and they demonstrated
that right through my training: lookout, ground checks, aircraft handling and so on
were all part of each flight” (037) and from another respondent “in each brief,
airmanship was mentioned in regard to lookout, communications and aircraft
awareness” (013). Several responses to this question included mention of briefs, so
the distinction between Questions 3 and 4 was not necessarily picked up by the
respondents.
Question 4 had two parts. Question 4a asked “In your initial flight training,
was airmanship, or concepts related to airmanship, discussed in ground training or
pre- or post- flight briefings?” and allowed only a Yes or No response. Question 4b
asked for elaboration, “If so which concepts and what sort of discussions?”
There was a response of “Yes” from 67.5% of respondents (N=27) and “No”
from 25% (N=10) in answer to Question 4a. Three respondents did not answer the
question.
Nine respondents left Question 4b blank as they had replied “No” to Question
4a or did not answer. A further four wrote that airmanship was not a major item, or
only discussed in a general sense and another two respondents simply mentioned
discussion in the bar or “hanger talk” as the source of information about airmanship.
Table 2.5 indicates the concepts that were raised in response to Question 4b.
Most respondents included more than one idea in replying to Question 4b
(mean of 2.2 concepts per respondent). The most frequent concept mentioned was
51
lookout, listed by 14 respondents. Related responses were: traffic, communication or
radio use, and listen-out. Checks, procedures, planning and safety issues were also
mentioned. There were various other considerations raised as shown in Table 2.5.
Table 2.5
Number of Respondents (of total N=40) who Mentioned the Listed Concepts in Response to Question 4b Concept N Nil response 9 Respondents to Q4b 31 Lookout 14 Aircraft knowledge 7 Communication/radio/listen 6 Checks 6 Traffic 4 Skills: general aircraft flying 3 Safety Issues 3 Planning 3 Specific to lesson/changed 3 Procedures 2 Hand over/ take over 2 Courtesy 1 Common sense 1 Developing a flow of activity 1 Relationship with instructor 1 Appropriate decisions 1 Bar Talk 2 General sense, not a major item 4
There were ten aircraft or skill related comments including general handling (3
mentions) and aircraft knowledge included mention of: understanding of systems (2
mentions), using airworthy aircraft; aircraft awareness; taxi speeds and run up into
wind; aircraft performance; temperatures and pressures; and energy and fuel state.
Five of these respondents included only aircraft handling concepts in their response
to this question.
It appears that most respondents had some discussion of airmanship in their
initial training, but for some of the respondents airmanship was not discussed, or was
limited to aircraft handling or aircraft state issues alone. While these are part of
airmanship, a broader view of airmanship includes the human factors aspects of
52
situational awareness and judgement. Of the five respondents who only listed aircraft
or handling issues in response to these questions, only one really had the broader
view when the definitions given in the first questions defining airmanship were
reviewed. Two wrote a little about issues that could be interpreted as situational
awareness and the other two gave quite narrow, aircraft handling related definitions.
Two of these respondents had over 20 years of experience, but the other three were
instructed fairly recently, with 2.5, 3 and 6 years of experience as PPL holders.
Given that human factors has been part of the theory syllabus for the PPL, as human
performance and limitations, for approximately 20 years, it is concerning that there
are pilots who have not incorporated these concepts into their understanding of
airmanship. On the other hand, it may be that their instructors had not discussed
human factors in terms of airmanship, or airmanship in terms of human factors.
Looking at these responses from another angle, there were at least 12
respondents who made no mention of aircraft handling skills when writing about the
discussion of airmanship in their initial training. This may be a reflection of the
military training system of assessing technical skills and airmanship as separate
aspects of flight proficiency (Carrick, Graham, Healey & Pickard, 2008). Although
only three respondents indicated a background in the military, it may be that the
instructors of the non-military respondents had a military background, and so
perpetuate the separation of technical skill and airmanship.
Influences on the development of airmanship. When asked what had led to
the development of their own sense of airmanship, (Question 5: “Thinking about
your progress to date as a pilot, what do you feel led to the development of your
sense of airmanship?”) respondents mentioned experience, reading, developing a
sense of responsibility and the influence of the people around them. Table 2.6 shows
the numbers of respondents who mentioned various influences on their development
of airmanship.
53
Table 2.6
Number of Respondents (of a total N=40) who Mentioned the Listed Concepts in Response to Question 5 Concept N Nil Response 1 Respondents to Q 5 39 Experience 13 Desire to “Get it right’ 10 Self-preservation / fear 5 Teaching others 3 Common sense 3 Luck 1 Specific episode 1 Influence of other people (28) Training 6 Talking to others 6 Reading 6 Mistakes/experiences 5 Loss due to an accident 3 Club environment 2
As it was an open-ended question, several respondents listed more than one
influence on their development. Only one person did not provide a response. Some
examples are: “Fear of having an accident because of a minor mistake, and a desire
to be [as] prepared and professional as possible” (025); “Watching and talking with
other pilots to develop my own ideas of airmanship from examples around me I
identified with [and] being scared and not wanting to go through a similar experience
again” (022); and “Common sense and experience. Be ready for the unexpected”
(001). Another simply wrote “leaving a fuel cap off” (030).
Ongoing experience in aviation is seen as important in developing a sense of
airmanship, but also the observation of what others do, whether these others are
initial instructors, fellow aviators, or observation through reading. This supports the
comments of both Kern (1993) and Ebbage and Spencer (2003) that early instructors
provide a significant role model, but extends further to imply that learning from
others continues as the aviator develops post ab-initio training. There is also a clear
internal motivation to improve or excel, to be as professional as possible, among
some of the respondents. This is included by Ebbage and Spencer as one of the
54
attitudes that form the basis of airmanship and that should be encouraged in initial
training.
Practice of current instructors. Question 8 asked: “If you are currently an
instructor, what do you do to ensure that your students develop good airmanship?”
There were 16 instructors among the sample but 17 responses to this question, with
23 not applicable or nil answers. One instructor did not answer the question, and two
respondents who were not instructors did answer, and stated that they intended
becoming instructors and wrote about what they would do when instructing; it was
decided to include their responses. Responses are shown in Table 2.7.
Table 2.7
Number of Respondents (of a total N=40) who Mentioned the Listed Concepts in Response to Question 8 Concept N Nil response 23 Respondents to Q8 17 Provide a good example 8 Briefs, discussion, in-flight emphasis 5 Lookout 2 Get students to read 2 System for managing flight 1 Attitude and understanding 1 Encourage them to think 1 Instil points mentioned 1 Post-flight evaluation 1
Half of the responding instructors reported that they ensure the development of
good airmanship among their students by modelling good airmanship or providing a
good example themselves.
While the formula of instilling motivation, teaching the skills and providing
feedback may be part of the instructor training system and be recommended by both
Kern (1996) and Ebbage and Spencer (2003) as a structure for instruction, there were
only five respondents who wrote about briefs and/or discussion, and one who
specifically mentioned post-flight self-evaluation. Of the two instructors who
mentioned getting their students to read, one specifically mentioned the book,
Redefining Airmanship (Kern, 1996). The two mentions of lookout were to “keep
yourself alive and keep a good lookout” (032) and “I hit them (not very hard) when
55
they turn without looking (they all get hit at least once)” (025). Only five of the
respondents mentioned more than one concept.
Conclusion
While the concepts included by Kern (1996) form a useful structure for
consideration of airmanship, based on the responses to this survey it appears that
airmanship in Australian aviation is slightly differently defined than the Kern,
Ebbage and Spencer or Edwards model. There is little emphasis on discipline among
the current respondents, despite its central role in the Kern and Ebbage and Spencer
models of airmanship. Similarly, discipline is not mentioned by pilots in research
into what makes a good pilot (Nergård, 2011; Swauger, 2006). This may reflect the
primarily civilian samples compared with the military influences on both Kern’s and
Ebbage and Spencer’s models. By contrast, communication appeared as an important
concept, mentioned quite frequently. Communication is included in the Ebbage and
Spencer and Edwards models as part of the required skill set, but is only listed as a
knowledge item (under “Team”) by Kern. Communication is a central skill for
TEM/CRM (Helmreich, 2002; Kanki, 2010).
Just over half of the respondents wrote about situational awareness, and far
fewer respondents mentioned decision making (no-one used the term ‘judgement’).
Both are central concepts in all three of the existing models of airmanship, and
generally seen as important non-technical skills (Flin & Martin, 2001; Helmreich &
Foushee, 2010; Mavin, 2014; Orasanu, 2010; Thomas, 2004).
The occurrence of the concept of courtesy among responses to the definition of
airmanship questions shows an awareness of the importance of good interpersonal
interaction in this participant group. The frequency of mention of personal qualities
and attitudes as part of the overall airmanship construct is also interesting. It implies
that airmanship is seen as integral to an aviator’s personality and lifestyle, rather than
just something that one does, or has, when flying. Use of the term “common sense”
also reflects this idea of integration within the individual, and also the use of the term
within the CASA Flight Instructors Manual (2006). These are not concepts that have
attracted much attention in the research literature, except insofar as personality is
seen as a potentially useful predictor in selection of pilots (Campbell, Castenada &
Pulos, 2010; King, 2014).
56
All three existing models place importance on relevant knowledge in various
domains. Respondents to this survey also listed various areas of knowledge or simply
mentioned “knowledge” as a general reference. The emphasis among participants
was more on rules, regulations and operating procedures, and on planning, than on
the other areas of the models, with very little mention of knowledge of “Self” or
“Team”. Knowledge of one’s aircraft was mentioned more in later questions in the
survey than in the initial definition section, so while this aspect may not have come
immediately to mind when asked about airmanship, respondents did bring it into
responses as a part of airmanship later in the survey.
Based on the responses to this survey, it appears that airmanship is generally
included in student instruction in Australian aviation, but perhaps not consistently
across all training establishments, given that 20% of respondents stated airmanship
was not part of their initial training. In addition, individual experiences and the
quality of airmanship displayed by the people with whom the student pilot associates
is clearly influential in the development of airmanship, and continues to be important
after initial training.
The close examination of responses to Questions 3 and 4 indicates that where
the initial training did include airmanship concepts, it did not always reflect a truly
integrated view of airmanship. There appear to be three groups of respondents: those
for whom airmanship was presented as an integration of flying skills and human
factors concepts; those for whom airmanship referred only to flying skills and
aircraft knowledge related items; and those for whom airmanship referred to only the
human factors related concepts, not including technical flying skills. This group
appears to be the largest, based on consideration of the initial definition questions.
Responses to the initial question on the survey asking for a definition of airmanship
included only 10 mentions of skill or technical proficiency (see Table 2.1), thus 30
respondents (75% of the respondents) did not include these aspects in their initial
definition of airmanship and so probably consider airmanship to be predominantly
about non-technical or human factors aspects of flying.
A factor that may limit any generalisation from these results is the very small
sample. In addition, respondents were recruited from among those interested enough
in safety to attend CASA Safety Seminars, or to read Flight Safety Australia, and
thus may not reflect the views of the wider aviation community. Also it is possible
57
that the respondents were writing only the most important aspects of their reactions
to each question, and were not taking the time to explain their activities or views in
depth. Most respondents wrote no more than a few words or a single sentence in
response to each of the questions asked.
A further limitation of the survey is the lack of both military and airline pilots
among the respondents, and the possibility that the resulting view of airmanship is
not representative of all sectors of the aviation community. It was also evident that
the respondents did their initial training in different eras, and this might have some
bearing on their training in airmanship. With the median number of years of
experience at 20, half of the respondents did their ab-initio training before human
factors was part of the theory syllabus. Although extensive exposure through
magazine and CASA seminars, among other methods, have introduced such issues to
pilots generally over the years since then, airmanship may well not have included
consideration of human factors at the time of their training, before 1991.
Examinations in human factors were introduced from 1992 in the airline licence
(ATPL) exams and 1995 in the private and commercial level exams (PPL and CPL).
Human factors questions were included in the Helicopter ATPL examinations from
1998. Respondents who were trained to fly after 1991 should include more human
factors, or non-technical, related issues in their definition of airmanship, and mention
such aspects as included in their initial training more frequently, than those trained in
the earlier era, who may have experienced a more technically oriented, aircraft
handling approach to airmanship. This aspect of group comparisons is addressed in
Chapter 3, where a larger respondent group and more detailed demographic
questions in an extended questionnaire made such divisions more practical for
statistical analysis. The comparison of responses from military and civilian
background respondents as well as era of training are reported in Chapter 3.
58
59
Chapter 3. Airmanship Across Industry Sectors
The survey responses described in Chapter 2 matched some of the major
aspects of the Kern (1996), Ebbage and Spencer (2003) and Edwards (2013) models
of airmanship, while differing on others, and mentioned some concepts that the three
models did not include. Overall, situational awareness was frequently mentioned,
with judgement/decision-making appearing in responses about half as often.
Australian aviators included various knowledge items, with an emphasis on
knowledge of rules and regulations and planning rather than on other areas of
knowledge listed by Kern or Ebbage and Spencer.
Discipline is a central tenet of both the Kern (1996) and Ebbage and Spencer
(2003) models of airmanship, with Kern (1997, 1999) continuing to emphasise the
idea of discipline as the cornerstone of airmanship. The survey described in Chapter
2 found that Australian aviators did not place nearly as much emphasis on discipline,
in fact it was rarely mentioned by respondents. In contrast, communication was
mentioned fairly frequently by respondents to the survey, confirming Ebbage and
Spencer’s and Edwards’ inclusion of it as a skill, while Kern made only passing
reference to communication as part of the discussion of the components of
knowledge of team. Common sense, courtesy, attitude and various personal qualities
appeared among the responses to the survey, but were not considered in the models
of airmanship, except insofar as Ebbage and Spencer included interpersonal relations
among their skill set and Edwards included some personal attributes as “attitudes”.
The training of airmanship is a central concern in both Kern (1996) and Ebbage
and Spencer (2003). Of the respondents to the survey reported in Chapter 2, 20%
indicated that airmanship was not included in their initial training, with a few others
indicating that it was picked up more from “hanger talk” and meeting at the bar. The
flight instructors who responded, indicated that they are careful to set a good
example for their students, and many respondents mentioned that the example of
others had influenced their development of airmanship.
The small number of respondents and the lack of military and airline pilots
among them were identified as limitations of the survey described in Chapter 2.The
respondents were mostly from general aviation (GA) and while 16 were instructors,
many of the remainder appeared to be recreational fliers. There is the possibility that
60
the resulting view of airmanship is not representative, with such a narrow group of
respondents. On the other hand, given the difference in training between military and
civilian pilots, there may not be a convergence of ideas, and thus no wholly
representative view.
It was decided to undertake a second survey of Australian aviators, in an
attempt to gain a larger and wider sample of pilots, specifically aiming to include
military and airline pilots as well as GA pilots. Changes to the survey itself consisted
of more specific demographic questions to allow separation of pilots into a number
of career-related sub-groups (described below) and the inclusion of further questions
building from the findings of the first survey.
The broad aims of this survey were (1) to determine the definition of
airmanship by Australian pilots and (2) to examine whether, and how, airmanship is
included in the training of Australian pilots. In more detail, the first aim includes the
definition of airmanship by a wider cross section of Australian aviators and whether
there are any consistent differences between military and civilian pilots as well as
whether their views on airmanship have changed over the course of their careers in
flying. How closely the Australian definitions align with the models of Kern (1996),
Ebbage and Spencer (2003) and Edwards (2013) is also of interest. The second aim
addresses the inclusion of airmanship in ab-initio and on-going training and also
considers how pilots think that they developed airmanship. This information can be
assessed against the recommendations for airmanship training presented by Kern
(1996) and Ebbage and Spencer (2003).
Depending upon the number of respondents to the survey, a comparison could
be made between the pre- and post- introduction of human factors to the training
syllabus for pilots. It is possible that undertaking initial training in different eras
might have some bearing on training in and understanding of airmanship and the
inclusion of human factors aspects. In Australia, the Human Performance and
Limitations (HPL) part of the syllabus was only introduced into the theory syllabus
in 1991. HPL became part of the examination system in 1992 for the Airline
Transport Pilot Licence (ATPL) and in 1995 for the Private Pilot and Commercial
Pilot Licences (PPL and CPL). Human factors competencies were specifically
included in flight testing only from July, 2010, but a general “airmanship”
assessment has always been in the flight test. Thus, there is a basis for dividing
61
survey participants into training eras of before and after the introduction of human
factors (in the HPL syllabus) in 1991. An additional demographic question identified
when the respondents began their training.
There may also be differences in ideas about airmanship dependent upon
pilots’ initial training in military or civilian sectors. Views on airmanship may also
be influenced by the sector in which the pilot has been flying, as a military pilot, or
in civilian industry broadly divided into general aviation (GA) or airline flying.
There are also pilots who began their careers in the military then moved into the
airlines or into GA thereafter. This results in five sectors reflecting current and past
involvement in aviation: civilian airline, civilian GA, military followed by airline
flying, military followed by GA, and pilots who have only flown in the military.
Several additional questions were intended to identify whether initial training was
civilian or military, current sector of activity, and experience (years and hours) in
civilian and/or military flying.
The open ended questions remained almost the same as in the first survey, with
one change in order and two new questions. The questions about the definition of
airmanship remained, the questions about airmanship included in ground training and
in “in air” flight training were reordered so that the ground training question came
first. An additional question about continuing training was added, to gain insight into
the ongoing training of pilots in airmanship across their careers. To compliment this
question, another was added asking if the respondents’ ideas about airmanship had
changed over time, and if so in what way. This question was intended to capture any
change in views with increased exposure to human factors issues since 1991, and
also add to the information gained from the question about development of
airmanship.
In addition to the open-ended questions, two sets of rating and ranking
activities were added to the questionnaire. Based on the findings of the previous
survey, a list of 26 concepts was constructed. The respondents to the second survey
were asked to rate each concept for level of importance in airmanship and to rank the
five they felt to be the most important.
In this second survey, comments on the definition of airmanship were expected
to differ by the sector of background and current activity. Comments on initial
62
airmanship training were expected to differ according to the sector in which initial
training was undertaken and the era of training, and comments about ongoing
training by sector of current activity.
Method
Participants
Eighty-three pilots participated in this study. Of these participants, 51 had
completed their initial flying training through civilian operators, 11 of these
participants had then moved on into the military and therefore participated in military
ab-initio training as well, because civilian training is not considered equivalent to
military training. Thirty-two participants had completed their initial flying training
through the military sector, 17 of these moved on to civilian flying, but they did not
repeat ab initio training in the civilian environment as military training is recognised
and equivalent licencing is issued. Sixty-six of the participants reported an
instructional or training background, of these 35 had civilian, 9 had military, and 22
reported both military and civilian instructional or training experience. Seventy-
seven participants received their initial training in Australia, and six had taken their
initial training overseas (three in the UK, two in Canada, and one in New Zealand)
but were currently flying in Australia. There was a wide variation in both years of
experience (range: 2 to 50 years) and total flying hours accrued (range: 120 to 25 000
hours) among participants.
Aviation sector classification was determined for each participant by
examining their licence type, flying history (proportion of flying hours and years of
experience in civilian and military operations), and how the participant self-identified
when asked about their most recent aviation activity. Participants were divided into
two overarching groups: those with civilian experience only (40 participants), and
those who had a military background, but may currently be working either in military
or civilian environments (43 participants). Civilian pilots were classed as those who
fly in regular public transport airlines (airline sector) and those in general and
recreational aviation (GA sector) including charter and helicopter operations. Among
the participants who reported military flying experience, three sub-categories were
created: military+airline to cater for those respondents with a military background
who identified as working in the airlines sector; military+GA for those with a
63
military background who identified as operating in the general aviation sector; and
military-only. The latter group included participants who identified as military pilots
and whose civilian hours totalled less than 20% of their military hours. The reverse,
predominately civilian flying with military experience consisting of less than 20% of
their total, were included in the military+airline or military+GA sectors, depending
on the nature of their civilian experience. The distribution of participants across
aviation sectors indicated 21 respondents in airlines, 19 in general aviation, 21 in
military+airlines, 7 in military+GA, and 15 respondents in the military-only sector.
Measure
The on-line survey reported in Chapter 2 was modified as follows. Questions
were added to the demographic section of the survey to determine sector of aviation
operation. In the second section, consisting of open ended airmanship related
questions, additional questions about ongoing training resulted in a total of 13
questions including a last “any other comment” question. A new third part of the
survey included two questions. In the first of these, participants were asked to rate
each of 26 concepts (selected from the Survey 1 findings) with regard to their
interpretation of airmanship, on a five-point Likert type scale from “not important”,
through “marginal”, “moderate”, “quite” to “extremely important”. The second
question asked respondents to pick the five most important concepts from among the
same 26, and number them “1” for the most important down to “5” for the fifth most
important. A final question asked the participants if they had any further comments.
The modified questionnaire can be found in Appendix C.
Procedure
Three methods were used to recruit participants. The first strategy employed a
snowballing recruitment method, in which all pilots known to the researcher and
colleagues were emailed and asked to complete the on-line survey. Their assistance
was sought to forward the email to other pilots whom they knew. Secondly, the
researcher attended several aviation conferences and sought participation by
conference attendees and further distribution of the invitation through their
organisations. Lastly, assistance was sought from CASA to include the survey details
in CASA safety seminars (as was done for Survey 1). However, this did not occur, or
not widely, possibly due to changes in management and personnel since the first
64
study. Participation in the survey was voluntary. There were 198 hits on the survey
website of which 37 were disregarded as there were no responses recorded for the
survey. Data from 80 participants were excluded as they completed the demographic
details only. Two respondents completed the survey via a word document, off-line.
Analysis
The analysis process followed the sequence described in Chapter 2. Three
coders were involved, all had undertaken training in qualitative analysis and the use
of NVivo and all communicated frequently concerning the coding process.
Ultimately a coded data set was developed that all coders agreed was reliably and
validly coded. The data set was small enough that one of the coders was able to
check code all the data. PASW20 (IBM, 2012; Allen & Bennett, 2010) was used to
conduct statistical analyses on the demographic details from Section 1 (demographic)
and Section 3 (Questions 14 and 15) of the survey. Responses to the open ended
questions in Section 2 were entered into NVivo9 (QSR, 2011) used as the platform to
record concepts and themes raised in these responses. As in the previous study, 12
tree nodes were established to reflect the main concepts included in the Kern (1996,
2009) model. A further 9 tree nodes represented minor concepts from Kern and
aspects of Ebbage and Spencer’s (2003) and Edwards’ (2013) model not covered by
Kern. Additional free nodes were created as other concepts emerged in the responses.
A total of 91 free nodes were developed. PASW20 was then used to make
comparisons between node frequencies across the demographic groupings. A Chi
Square test of contingencies was used to compare the responses of pilots in civilian
flying with those of the pilots with any military background. Fisher’s Exact test was
used where there were one or more cells with an expected frequency less than 5,
unless otherwise noted.
Amount written by respondents in each industry sector. A word count was
conducted on each response. It is possible that any interpretation of concepts
mentioned by participants could be confounded by the amount written by participants
in each sector. The mean word counts for each sector were compared to address this
possibility. The civilian sector word counts were skewed, so a Mann-Whitney U test
was appropriate. This indicated that there was no significant difference between the
mean ranked word counts (U = 667.5, p = .079) for the civilian and the military
background participants (see Table 3.1). Given the possible division of the
65
participants into five sectors, word counts were checked at this level also. The
distribution of word counts in the airline sector violated assumptions of normality, so
a Kruskal-Wallis one-way ANOVA was conducted, which indicated that there were
no differences in mean word counts between sectors (H (corrected for ties) = 3.626,
df = 4, N = 81, p = .459, see Table 3.1). On the whole, the military-only background
respondents tended to write more in their answers, and the civilian GA participants
wrote the least, but the differences were not statistically significant.
Table 3.1
Comparison of Word Count Across Industry Sectors Civilian Military
Mean
409.1 507.4 Std Dev
293.0 286.6
Airline GA
Military +Airline
Military +GA Military only
Mean
449.1 364.9 504.4 495.0 517.5 Std Dev
337.2 236.0 303 142.8 326.1
Level of participants’ experience by industry sector. Another possible
confounding factor is the level of experience among respondents. It is generally
accepted in aviation that military pilots fly fewer hours than airline pilots. It costs a
lot of money to fly a fast military jet and the Defence budget does not allow for
anything near the maximum allowable hours for civilian pilots (800 hours per year,
with some exceptions) and military pilots move into desk jobs as their rank increases,
reducing the amount of flying undertaken. A comparison of hours of experience for
military and civilian groups by means of a Mann-Whitney U test (used because the
distributions are skewed) indicated that there is no significant difference in hours of
experience between the two groups (U = 663.5, p =.139; see Table 3.2 for means and
standard deviations). Comparison of years of experience was conducted by a Mann-
Whitney U test (the military group distribution was skewed,) and indicated a
significant difference between the two groups (U = 435.5, p < .001, see Table 3.2).
Although there is no difference in hours of flying experience, the military group was
significantly more experienced in terms of years of involvement with aviation.
66
The two groups are made up of participants in different sectors of aviation and
who are likely to fly different patterns of hours across a career. A military pilot who
moves on into airline flying will probably fly more hours than a military-only pilot
who moves into a desk job, and airline pilots will generally fly far more hours than a
GA hobby pilot. A five-sector examination of experience was carried out to
determine if these differences existed between participants in the different industry
sectors. The distribution of hours of flying experience was skewed for airline and GA
respondents, therefore a Kruskal-Wallis one way ANOVA was conducted. This
indicated significant differences (H(corrected for ties) = 36.937, df = 4, N = 81, p <
.001, large effect size (Cohen’s f = 0.926)) among the mean ranks of hours of
experience for the participants in the 5 sectors (see Table 3.2 for means and standard
deviations).The hours of experience of participants in the airline, military+airline and
military+GA sectors are not very different, but mean hours of participants in the GA
and military-only sectors are much lower.
Table 3.2
Comparison of Hours of Experience and Years of Experience Across Industry Sector
Civilian Military p value Hours Experience
Mean
8878.8
10371.2
.058 Std dev
7924.5
6233.5
Years Experience Mean
21.5
31.8
.001
Std dev
12.4
10.9
Airline GA Military +Airline
Military +GA
Military only p value
Hours Experience Mean
13307.5 3983.9 14512.6 10294.3 5161.3 .001
Std dev
7232.7 5455.5 4555.5 7616.1 2667.8 Years Experience
Mean
26.6 15.9 34.6 32.3 26.7 .001 Std dev
12.1 10.5 7.4 14.6 12.5
An examination of years of flying experience across the five sectors used a
Kruskal-Wallis one way ANOVA because, although distributions were not skewed,
the Levene Statistic (3.302, df = 4,76, p = .015) indicated that the assumption of
homogeneity of variances had been violated, and since the group sizes differ widely,
a non-parametric test should be performed. The Kruskal-Wallis test indicated
67
significant differences between the mean ranks of years of flying experience (H
(corrected for ties) = 21.474, df = 4, p < .001, Cohen’s f = .605, see lower rows of
Table 3.2 for means and standard deviations). Participants in the GA group had the
fewest years of experience; military only and airline only participants had about the
same, with military + GA and military + airline participants having far more years of
aviation experience. The latter two groups represent pilots who have engaged in two
careers, in military and in civilian flying. Therefore it is not altogether surprising that
they also have more years of experience, but not many more hours of experience as
military flying does not involve as many hours, as noted above. The GA group of
respondents has fewer years of experience and fewer hours – so is simply a less
experienced group than the others. GA flying generally does not result in as many
hours per year as airline flying.
Results - Definition of Airmanship
Six of the survey questions were designed to elicit the individual’s ideas about
the definition and components of airmanship. The first two questions in the survey
addressed this directly (Q1. What do you think airmanship is? Q2. Thinking of your
response to the first question, can you elaborate on each of the key words in your
response? For example, if you wrote ‘being safe’, write in more detail what ‘being
safe’ would involve). Questions 7 asked about awareness of airmanship and what
might be most important about it, while Question 8 directed attention to the phases of
flight and events that might influence airmanship (Q7. In your current aviation
activities, are you normally actively aware of airmanship as part of your operation?
What would be the most important things that come to mind? If you are no longer
flying, were you normally actively aware of airmanship as part of your operation?
Q8. During what phases of flight or events would you be most actively aware of
airmanship issues?). Questions 11 and 12 elicited examples of good and poor
airmanship, indirectly addressing what respondents considered constitutes
airmanship. (Q11. Please describe a flight, that you observed or know about, in
which good airmanship was displayed, including why you feel it was particularly
good airmanship. Q12. Please describe a flight, that you observed or know about, in
which poor airmanship was displayed, including why you feel it was particularly
poor airmanship).
68
The content analysis resulted in frequency tables of the number of participants
who mentioned particular themes: from Kern (16 nodes); from Ebbage and Spencer
or Edwards (but not in Kern, 9 nodes); or free nodes (not included in either model).
Re-analysis of the free nodes for these questions led to the combination of similar
concepts and a reduction of nodes to 28 from all 6 questions, and identification of 18
phases of flight or events mentioned in relation to airmanship. In the following
discussion, participants are cited by participant number when comments are quoted.
Differences between Sectors.
Initially, differences between participants grouped into the two overarching
sectors of civilian background (40 participants) and military background (43
participants) were considered.
Due to the low cell frequencies for many concepts, the frequency counts for
each node were combined across all six questions relating to the definition of
airmanship. Care was taken to ensure that each respondent was still only counted
once for each concept, so that repeated mention of the same concept by individual
participants in response to different questions did not weight the analysis. Table 3.3
shows the percentage of respondents who mention each of the concepts included in
the Kern model or additional concepts included in the Ebbage and Spencer or
Edwards models of airmanship, and the results of the comparison tests between the
two groups (Civilian or Military background) for each concept. Due to the large
number of comparisons, a Bonferroni adjusted critical value of p < .001 was used.
While an examination of the data reveals some apparent differences between
numbers of pilots in each sector who mentioned certain concepts, on examination
none of these were significant.
Reference to Table 3.3 shows that about twice as many military background
respondents (54%) as civilian (27.5%) mentioned aircraft handling skills as being
part of airmanship and three times as many (23.3% vs 7.5%) mentioned proficiency,
however these differences are not significant. None-the-less this is interesting given
that airmanship and technical skills are assessed separately in early military training,
implying that airmanship is something other than technical handling skill (Carrick,
Graham, Healey & Pickard, 2008). It appears that in the intervening years since their
69
training more than half of the military respondents have developed a more inclusive
view of airmanship.
Table 3.3
Percentage of Respondents who Mentioned Listed Concepts from Airmanship Models in Response to Combined Definition of Airmanship Questions.
Civilian Military
Significance values
Respondents 40 43
% %
p
Kern model 7.5 4.7 Judgement or decision making 52.5 58.1 Situational awareness 57.5 60.5 Knowledge: General 22.5 27.9 Self 22.5 25.6 Aircraft 35.0 58.1
Team 10.0 11.6
Environment: General 20.0 11.6
Legislative 32.5 55.8
Organisational 27.5 53.5
Physical 15.0 20.9
Risk 27.5 39.5
Mission: Planning 10.0 18.6
Skill 27.5 51.2
Proficiency 7.5 23.3
Discipline 15.0 23.3
Additional Ebbage & Spencer or Edwards concepts Attitudes: Hazardous 10.0 18.6
Professionalism 20.0 11.6 Self-improvement 5.0 0.0 Skills: Communication 37.5 30.2 Crew co-ordination 27.5 25.6 Interpersonal 7.5 2.3 Management flight-deck 12.5 16.3 Self-assessment 7.5 11.6 Team leadership 15.0 7.0
Table 3.4 shows the mentions of additional concepts by respondents. This table
includes a count of the use of either of two common phrases that occurred in
responses. There is a standard definition of airmanship used in the military: “safe and
efficient operation of a flight, both in the air and on the ground”. Another comment
that was made several times was to refer to “the saying that airmanship is the
70
exercise of superior judgement to avoid the use of superior skills”. These statements
were only made by military sector pilots, and appear at the top of Table 3.4.
Table 3.4
Percentage of Respondents who Mentioned Listed Additional Concepts in Response to Combined Definition of Airmanship Questions.
Civilian Military Significance values
Number of Respondents 40 43
% %
p
Military definition 0.0 23.3
.001 Superior judgement/skill 0.0 9.3
Other concepts: Safety 72.5 76.7 Personal qualities 52.5 41.9
Efficiency/effective 32.5 39.5 Experience 22.5 37.2 Common sense 12.5 37.2
Courtesy/consideration 40.0 11.6
CRM 7.5 20.9 Care for aircraft 12.5 14.0 Responsibility 15.0 11.6 Foresight 5.0 18.6 Thinking 17.5 7.0 Briefing/discussion 5.0 16.3 Act proactively 7.5 11.6 Time management 10.0 9.3 TEM 12.5 7.0 Cognitive capacity 5.0 9.3 Normal ops not aware 5.0 7.0 Gut feeling/ 6th sense 2.5 9.3 Human factors 7.5 4.7 Organisational culture 10.0 2.3 Prioritising 5.0 7.0 Checks 5.0 4.7 Management of
automation 7.5 0.0 Mistakes 5.0 2.3 Documentation 2.5 2.3 Examination 2.5 2.3
71
The only significant difference between the two groups of respondents was the
use of the military definition (χ² (1, N = 83) = 10.577, p < .001, effect size φ = .37 a
medium effect), which was only reported by military background respondents.
Table 3.5 indicates the phases of flight or events mentioned by respondents.
Three significant differences were detected between responses of civilian and
military background participants.
Table 3.5
Abnormal or emergency events were mentioned by very few civilian
background respondents (7.5%) while military background respondents brought
these up much more frequently (44.2%), the effect size being medium to large (χ² (1,
N = 83) = 14.32, p < .001, w = .42). Abnormal or emergency events were only
mentioned in response to questions 7 and 8, which asked about what phases of flight
Percentage of Respondents who Mentioned Listed Events or Phases of Flight in Response to Combined Definition of Airmanship Questions, by Sector.
Civilian Military Significance
values Number of Respondents 40 43
% % p Phase of flight or event
All the time 47.5 51.2 Weather/visibility 27.5 53.5 Departure/arrival 32.5 30.2 Traffic 32.5 27.9 Abnormal or Emergency 7.5 44.2 .001
Pre-flight/preparation 17.5 30.2 Altered plans 12.5 20.9 Fuel 15.0 16.3 Ground operations 20.0 9.3 Pressure 2.5 25.6 Circuit 12.5 7.0 Fatigue 10.0 9.3 Landing (and approach) 7.5 7.0 Training 0.0 9.3 Workload 7.5 2.3 Bird strike 5.0 2.3 ATC 0.0 4.7 Automation 5.0 0.0 Stress 5.0 0.0 Cruise 2.5 0.0
72
pilots’ were most aware of airmanship and in response to the questions asking about
good and bad examples of airmanship, and responses were evenly split between the
two questions..
Overall, there were very few significant differences between the civilian
background and military background respondents. This may be because there are not
very many pilots in each group or in fact civilian and military pilots do not raise
different concepts. There is also some confounding between the groups, as some of
the military background respondents have also worked as airline pilots (after their
military career). There were not enough pilots in each sub-grouping to consider
comparisons at that level
Comparisons with Models
As there were so few significant differences in the mention of themes between
the two groups (civilian vs military background) in response to the definition
questions, the content of the definition responses can be discussed for all the
respondents together.
Kern model. The 15 concepts representing the Kern (1996, 2009a) model of
airmanship were mentioned by many respondents, although no-one listed all the
themes. Five respondents indicated knowledge of the Kern model itself, referring to
it in their responses. Table 3.6 shows the total percentages of participants who
mentioned each of the Kern, Ebbage and Spencer (2003) or Edwards (2013) model
concepts in the definition questions.
A Chi square goodness of fit on the total frequencies across Kern themes tested
whether there were significant differences in the frequencies of mention. This was
statistically significant (χ² (14, N = 81) = 95.039, p <.001). The index of effect size,
Cohen’s w was 1.08, which can be considered large. The test confirms that the
themes relating to the Kern model of airmanship were mentioned by significantly
different numbers of respondents, rather than being mentioned equally.
Situation awareness and judgement/decision-making were mentioned most
frequently, by more than 50% of participants. Knowledge of aircraft, legislation and
organisation, and risk were mentioned quite frequently, by between 30 and 50% of
73
participants. Of the Ebbage and Spencer/Edwards additional concepts,
communication was most frequently mentioned, by 33.7% of the participants. With
the exception of knowledge of self (mentioned by 24.1% of participants) the least
frequently mentioned concepts seem to be those that relate to the self or the team,
including knowledge of the team and team leadership (each 10.8%), self-assessment
(9.6%), interpersonal skills (4.8%) and self-improvement (2.4%).
Table 3.6
Percentage of all Respondents who Mentioned Listed Concepts from Airmanship Models in Response to Combined Definition of Airmanship Questions.
Total
Number of Respondents
83
%
Kern model
6.0 Judgement/decision
making
55.4 Situational awareness
59.0
Knowledge: General
25.3 Self
24.1
Aircraft
47.0 Team
10.8
Environment: General
15.7
Legislative
44.6 Organisational
41.0
Physical
18.1 Risk
33.7
Mission: Planning
14.5 Skill
39.8
Proficiency
15.7 Discipline
19.3
Ebbage and Spencer or Edwards Additional Concepts
Attitudes: Hazardous
14.5 Professionalism
15.7
Self-improvement
2.4 Skills: Communication
33.7
Crew co-ordination
26.5 Interpersonal
4.8
Management flight-deck
14.5 Self-assessment
9.6
Team leadership
10.8
74
Situational awareness was the most commonly mentioned theme, closely
followed by judgement or decision making, these being Kern’s capstone outcomes.
Frequently, the respondents simply listed “situational awareness”, while others gave
a definition such as: “Situational awareness: Having enough brain space to fly the
aircraft and consider all external aspects of the aviation environment enabling
yourself to predict the aircrafts position and state with enough time to avoid 99% of
hazards” and “Situational awareness is the ability of the pilot to be able to know:
where his aircraft is in relation to airports, other air traffic, mountains, adverse
weather and range from the remaining fuel available” (military respondents, 038 &
090); “Situational awareness - appreciating what is going on around you, what risks
and threats exist, what effect your actions can have” and “Situational Awareness = In
simplest terms where are we now and where are we going to be at some time in the
future, this process starts before flight planning continues throughout the flight and
concludes when the last post flight issue has been dealt with” (from airline
respondents, 032 & 040). One GA respondents remarked “the ability to see changes
in the operational environment, know how the changes will impact your flight
progress and then take actions to adjust aspects of the flight to maintain expected
operational margins” (024). Those respondents who gave a definition of situational
awareness seemed to have a good grasp of the meaning according to one of the
standard definitions “the perception of the elements in the environments within a
volume of time and space, the comprehension of their meaning and the projection of
their status in the near future” (Endsley, 1988, cited in Endsley, 1999, p258).
Kern’s model refers to judgement, and while some respondents used that term,
in general respondents reported that decision making was part of airmanship. Some
made direct reference to judgement e.g. “I regard the outcomes of airmanship as
increased situational awareness, the development of sound judgement, and the
correct employment of operational risk management” while others were more
indirect “The captain's job is to choose the best solution when faced with lots of
options about weather, maintenance, options before and during flight” (both military
respondents, 080 & 102). Other comments on this theme include: “The ability to
consider all factors involved with operating the aeroplane and make positive
decisions to maintain safe operations at all times” (156, an airline respondent) and
“making effective decisions regarding risks and hazards” (178, a GA respondent).
75
The “Pillars of Knowledge” did not get equal mention by respondents.
Knowledge of aircraft was most frequently mentioned of the “Pillars”, with
significantly more military respondents placing more emphasis on this aspect of
knowledge. Some examples of comments made include: “essential knowledge
includes aircraft”, “the serviceability state of the multitude of systems in the
aircraft”, “knowledge of aircraft capabilities and systems” (all from military pilots,
025, 090, 126) and “having a complete and thorough knowledge of the aircraft
technical systems, performance capabilities and its handling characteristics in the
operational environment” (009, an airline respondent), “knowledge of the aircraft
performance envelope, aircraft systems...” (024, a GA respondent).
Knowledge of the legislative and organisational environments was also
frequently mentioned, and again more often by military than civilian sector
respondents. Comments along these lines were generally listed clearly, such as:
“knowledge of rules (SOPs)”, “compliance with regulations, rules, procedures”,
“Knowledge of orders and instructions” (from military respondents, 025, 090 &
126); “Operating an aircraft in a professional manner having regard for rules of the
air and Company Standard Operating Procedures”, “Considering rules, procedures”
(from airline respondents, 058 & 039) and “good knowledge of the rules”,
“Respecting and adhering to rules and regulations” and “knowledge of air law and
rules to conduct a safe flight from start to stop” (from GA respondents, 054 & 113).
Respondent’s comments about knowing their own capabilities or being aware
of their own limitations were coded as “knowledge of self”. These were fairly evenly
distributed between civilian and military background respondents.
Risk was mentioned somewhat frequently by 33.7% of respondents overall.
Mention was made of knowledge of risks, risk assessment or application of risk
management. Sometimes this was elaborated, for example: “An aviator has to
understand risk. An aviator has to understand severity, frequency, likelihood and
consequence. An aviator has to understand the finality of death” (an airline
respondent, 009) and from a military respondent: “Risk Management. This is the
what-if side of airmanship. We continually, both formally and informally, assess the
outcomes of any hazard we encounter. We compare the outcome with the likelihood
of the hazard will manifest itself and we make a decision based upon the risk of the
occurrence and the possible, or expected, outcome” (165).
76
The physical environment also mentioned relatively infrequently, by 18.1% of
respondents. These were usually references to terrain or meteorology in terms of
situational awareness or in examples of good or poor airmanship. “Mission”
(planning) was also mentioned infrequently (14.5%), with comments on preparation
and flight planning included in the concept node when coding.
Least often mentioned of the Knowledge items, was “Team” (10.8%). One
respondent (151), who referred to Kern, included “knowledge of the team (other
crew, ATC, engineering, cabin crew) in so far as what their roles and responsibilities
are” in his comments on airmanship. Otherwise, mention of “team” or “crew”
tended to be part of list of considerations.
Of the “bedrock foundations”, skill (aircraft handling) was mentioned most
frequently overall (39.8% of respondents), and more often by military (51.2%) than
civilian (27.5%) respondents. This is interesting considering that in their training,
military cadets are marked on preparation, technical skills and airmanship as separate
items (Carrick, Graham, Healey and Pickard, 2008). However the respondents to the
survey were generally very experienced, and had perhaps developed a more
integrated view of airmanship. That civilian respondents did not mention technical
flying skills very frequently may reflect the separation of technical skills and human
factors in the theory and testing for pilots licences and emphasis on human factors in
aviation publications, and their knowledge that the survey was being run from a
School of Psychology (stated in the introduction to the survey). Proficiency was also
differentially mentioned, more by military respondents, but overall by only 15.7% of
respondents. Discipline was mentioned by 19.3% of respondents, quite low given the
emphasis placed on discipline in Kern’s writing (Kern, 1997). Examples of
respondents’ comments coded as discipline related include: “everyone has different
abilities and comfort levels, and we must have the ‘self-discipline’ to adhere to our
‘personal limits’”, “Discipline - adherence to rules and procedures; self-discipline”,
and “Obeying the rules” (046, 086, 138).
Ebbage and Spencer and Edwards models. The additional concepts found in
the Ebbage and Spencer and Edwards models were not mentioned with equal
frequency by respondents, but there were no significant differences between the
civilian and military background respondents. Communication was mentioned most
frequently of the Ebbage and Spencer concepts, by 33.7% of respondents. Comments
77
included: “the ability to effectively communicate such that messages are clear,
concise and related”; “being prepared to share information” (airline pilots, 176, 032);
“Good communication – clear communication; English language proficiency;
readback; for multi-crew ops – seeking feedback, creating an environment where
superiors encourage input from their subordinates” (a GA pilot, 035); “listen to the
rest of your crew” (military+airline, 191); “airmanship involves taking action or
speaking up when something is observed to be out of place...” (military+GA, 200);
“good communication among the crew was essential” (military-only, 110).
Crew co-ordination was raised as part of airmanship by 26.5% of the
respondents, including mention of crew resource management, teamwork and
working well with colleagues (airline pilots); “working together with other crew
members, non-dictatorial management style” (a GA pilot, 035); “timely and
appropriate guidance to the crew” (military+airline, 091) and “management of the
crew resource” and “the Captain’s ability to employ all crew members and stake
holders” (military-only, 090, 155)
Professionalism was mentioned as part of a list of desirable qualities by 15.7%
of respondents, also including: “airmanship is the professional application of
everything that you have learned...” (military-only, 046) and “operate in a
professional manner” (airline, 058).
Management of the flight deck was mentioned directly or indirectly, by 14.5%
of respondents including comments like “organisation in terms of the flight” (GA,
063) and “flight management – approach planning to minimise crew workload during
the approach” (military+airline, 013).
Hazardous attitudes were included in descriptions of airmanship by 14.5% of
respondents, sometimes specifically (‘avoiding hazardous attitudes” by a GA pilot,
178) or by the use of examples, such as: “it seems helicopter pilots have very
cowboy attitudes’(GA pilot, 037);‘cowboy and cavalier practices are unacceptable”
(military+airline, 190); “owner trying to impress potential buyer” (military+GA,
076); and “beware of “continue-on-itis”’(military-only, 046).
Team leadership was listed among airmanship attributes by 10.8% of
respondents, such as “very aware of team skills and leadership” (military+airline,
034) with few elaborations however there was one example of good airmanship that
78
noted good leadership in that having been asked to conduct a flight in marginal
weather, the flight leader decided not to fly and under pressure to change his mind,
“stood his ground and displayed the kind of leadership that is second to none”
(military-only, 046)
Self-assessment was not frequently mentioned, only 9.6% mentioned this, and
only 2.5% specifically wrote about self-improvement. Examples are “Experience
allows a pilot to realise his own as well as other people’s limitations” (military+
airline, 089); “airmanship is what causes us to continually look for improvement”
(military+GA, 055); “post flight personal review of performance” (military-only,
025).
Other concepts. Table 3.7 shows other concepts raised that were not included
in any of the models and the percentage responses among all respondents. Safety was
raised as part of airmanship by the majority of respondents (74.7%) and to a rather
greater extent than in the earlier survey (62.5%, Table 2.1). While safety is not a
formal part of the airmanship models, it is stated as the aim of airmanship, and it is in
this context that it was mentioned by the respondents.
Personal qualities were frequently mentioned (by 47% of respondents). This
concept included such comments as “being neither a fool nor a coward” (009),
patience, selfless attitude, conscious of others limitations, integrity, honesty,
“underlying values and attitudes of the aircrew” (111) and “to develop good
airmanship you have to be passionate about the industry” (140).
Running an efficient and/or effective flight was mentioned as being part of
airmanship by 36.1% of respondents. One respondent defined this as: “Efficiently
conduct a flight: pilot the aircraft from A to B in the minimum amount of time, using
the least amount of fuel, and ensuring the minimum amount of stress on the aircraft”
(100). Another explained this as “Operating the aircraft so as to achieve the required
outcomes using the least amount of resources that could reasonably be expected”
(111).
79
Table 3.7
Experience was mentioned in the definition of airmanship by 30.1% of
respondents. Examples of mentions of this concept include: being aware of the
experience or lack of it among other airspace users; application of knowledge and
experience to a given situation; and “store the experience away so it becomes part of
the memory bank of airmanship” (166).
Common sense was mentioned by 25.3% of respondents overall. Some
examples from military background respondents include: “common sense application
Percentage of Respondents who Mentioned Listed Additional Concepts in Response to Combined Definition of Airmanship Questions. Total Number of Respondents 83
%
Other concepts: Safety 74.7
Personal qualities 47.0 Efficiency/effective 36.1 Experience 30.1 Common sense 25.3 Courtesy/considerate 25.3 CRM 14.5 Care for aircraft 13.3 Responsibility 13.3 Foresight 12.0 Thinking 12.0 Briefing/discussion 10.8 Act proactively 9.6 Time management 9.6 TEM 9.6 Cognitive capacity 7.2 Normal ops not aware 6.0 Gut feeling/ 6th sense 6.0 Human factors 6.0 Organisational culture 6.0 Prioritising 6.0 Checks 4.8 Management of automation 3.6 Mistakes 3.6 Documentation 2.4 Examination 2.4
80
of hazard analysis / risk management”; “Common sense compliance with regulations,
rules, procedures” (086); “In many respects, I think Airmanship is little more than
formalized common sense, but in an aviation context. During my own training I
recall the phrase ‘Common Sense and Airmanship’ being used repeatedly. In
hindsight, I think it was a redundant phrase!!” (099); and “An active form of
‘common sense’ which is a mix of experience, training, attitude and perspective
which informs decisions and affects the interpretation of incoming information”
(124).
A longer explanation came from a military pilot:
Aviation is a very regulated and rule based environment, however there are situations that arise that will require those rules and regulations to be broken. This by its nature is an exceptional violation and would require post flight de-brief with all key players, ie the regulator, the AOC holder etc. Example: Arrive at isolated destination with adequate fuel reserves, weather is below ILS minima and lasts for longer than your fuel reserves allow. Rule says you can't descend below DA however once you burn fuel to minimum it is now time to fly the ILS to the ground in control vice running out of fuel and crashing out of control (038).
An airline respondent wrote “A combination of situational awareness, common
sense and common courtesy. Common sense - appreciating that the safest way to
complete a task will ultimately be the most efficient” (032). A GA respondent
included: “Respecting and adhering to rules and regulations whilst realising common
sense can sometimes have a significant influence over an operation” (113). In general
then, common sense is used to modify the idea of conformity and being overly rule-
bound, and to inform the decision-making process.
Courtesy and consideration for others was also mentioned by 25.3% of
respondents overall, and was slightly (not significantly) more frequently mentioned
by civilian than military background pilots and rather more frequently by GA
(47.4%) than airline pilots (33.3%) (again, not significantly) if the detail is examined.
This is appropriate given the environment in which GA pilots operate: in and out of
small airfields and having to interact with other operators directly, more-so than
airline pilots, who interact mainly through ATC using formalised phrases. The
differences are understandable given the environment in which GA pilots operate: in
and out of small airfields and having to interact with other operators directly, more-
81
so than airline pilots, who interact mainly through ATC using formalised phrases.
Some examples of the comments made are: “common courtesy – appreciating that
other airspace and airport users are also wanting similar outcomes and being
prepared to share information and facilities” (airline, 032); “politeness – considering
others and how you like to be treated” (airline, 039), “considerate of other airspace
users [but] you would not be able to be considerate if your flight safety was in
question” (airline, 184); “considerate – ensuring all parties can achieve their own
objectives without outside influence” (GA, 116); “treat others the way you like to be
treated” (military+ airline, 191). Courtesy is thus an expression of how the
relationship with other pilots should be conducted. A more detailed comment is:
Courtesy - Acting in a manner which creates a good outcome for all users of the air, not only yourself, no matter how to the rule it is. Definitely not an excuse to break a rule, rather think of another way in which to legally and efficiently make a situation better for all concerned. E. g. Extend downwind for another 30 seconds or so, to enable the guy on the ground who's been waiting to take off for some time, to get away safely (GA, 054);
Crew resource management (CRM) was included in responses by 14.5% of
respondents, overall. Rather more military than civilian pilots mentioned it, but this
was not a significant difference. Some examples are: “airmanship (for me) was
brought clearly into focused summary with the introduction of CRM training”
(military+airline, 034); “I consider CRM principles to be an element of airmanship”
(military, 080); “modern training refers to airmanship as cockpit resource
management” (military+ airline, 096). A military participant (169) commented
“Well I am of the opinion that I have never observed a significant lapse in
airmanship. CRM on the other hand...”. Thus he is implying that CRM is not part of
airmanship.
Comment about caring for the aircraft was included by 13.6% of respondents
as being part of airmanship. This included comment such as “operating the aircraft so
that no undue stress is placed on the airframe or engines” (military+GA, 111) and
“don’t bend or break the aircraft” (military+airline, 099). The same proportion of
respondents indicated that responsibility is important, some stating individual
responsibility such as “accountability for your actions” (military-only,165) or
“individual responsibility” (airline, 051), and another suggesting “ultimate
responsibility falls on the supervisors at Squadron level” (military-only, 046), while a
82
final person saw it as a shared obligation “owning up to your actions and if need be
taking responsibility for others actions to ensure the outcome is safe” (airline, 176).
Foresight was brought up as a separate concept by 12.3% of respondents. This
included comment such as “rigorous forward thinking” (military-only, 165), and
“ability to foresee problems” (military+airline, 088). Comment about thinking was
mentioned as frequently, some examples are: “a constantly evolving or changing
environment through which you intend to fly requires constant adaptation,
consideration, adjustments and decisions” (military+airline, 088) “their decision...
demonstrated great thinking “outside the box” (military+airline, 099) and “poor
airmanship is really about not thinking efficiently” (airline, 026).
Briefing or discussion among crew was mentioned as part of airmanship. A
military+GA respondent wrote “briefing other members of the crew and passengers,
to clarify their duties and encourage them to have a positive input” (101), while
others simply mentioned the concept.
The remaining concepts were brought up by fewer than 10% of respondents.
The range of ideas that were included in the descriptions of airmanship indicates the
lack of an overarching consistent definition or interpretation of the term. However,
the underlying theme of ensuring safe flight was certainly present, even if not clearly
verbalised.
Phases of flight or events that occur in flight were brought up in relation to
direct questions or in relation to the examples of good and bad airmanship, and these
are listed in Table 3.8. The interesting outcome from asking respondents when they
were most aware of airmanship was that nearly half (49.4%) challenged the question
by responding that they were aware of airmanship at all times during flights. Some
respondents expanded the scope of the question to indicate that pre-flight preparation
(24.1%) and ground operations (14.5%) are also times during which they are aware
of airmanship. Weather and visibility were frequently mentioned in response to the
question about examples of good and poor airmanship, 41% of respondents included
some comment about weather or used examples that included a weather related
event. Respondents listed several phases or circumstances of flight that are known to
involve more risk or higher workload: when there is increased traffic (30.1%), in the
circuit (9.6%), during approach and departure (31.3%) and landing and approach
83
(7.2%). Concern regarding the management of fuel featured as well, another critical
feature of flight. Individual factors of feeling under pressure, high workload or stress
as well as fatigue were mentioned by some respondents.
Table 3.8
Percentage of Respondents who Mentioned Listed Events or Phases of Flight in Response to Combined Definition of Airmanship Questions. Total Number of Respondents 83
%
Phase of flight or event All the time 49.4
Weather/visibility 41.0 Departure/arrival 31.3 Traffic 30.1 Abnormal or Emergency 26.5 Pre-flight/preparation 24.1 Altered plans 16.9 Fuel 15.7 Ground operations 14.5 Pressure 14.5 Circuit 9.6 Fatigue 9.6 Landing (and approach) 7.2 Training 4.8 Workload 4.8 Bird strike 3.6 ATC 2.4 Automation 2.4 Stress 2.4 Cruise 1.2
Concept Rating and Ranking
The final two questions of the survey asked participants to rate (Question 14)
and to rank (Question 15) the 26 concepts related to airmanship selected from the
Kern model and from frequent responses found in the earlier survey (reported in
Chapter 2). The concepts can be seen in the questionnaire at Appendix B.
Three participants did not complete either question. A further three omitted
Q15 and another 5 completed it incorrectly, generally by using only a few numbers
repeatedly instead of just 1 to 5. Thus there were 80 responses to Q14 and 72 to Q15,
84
distributed over the sector of background and of current operation as shown in Table
3.9.
Table 3.9
Number of Respondents in Each Sector of Background and of Current Operation who Responded to Question 14 and Question 15
Sector Q14 Q15 Background: Civilian 38 33 Military 42 39 Current operation: Airline 21 17 General Aviation 17 16 Airline and Military 20 17 GA and Military 6 6 Military 16 16 Total 80 72
Rating Question. Question 14 asked the participants to rate the importance of
the 26 concepts, with regard to their interpretation of airmanship, from “not
important”, “marginal”, “moderate” and “quite” to “extremely important’. One
participant used only “extremely important” for all except one of the 26 concepts.
Seventeen participants used only the top two ratings and only five participants rated
anything using the lowest rating. There appears to be an upward bias, rather than a
central one.
A comparison of ratings between the five sectors of background and current
activity, using a Kruskal-Wallis ANOVA, indicated no significant differences with a
Bonferroni adjusted critical value of p < .002.
A comparison of ratings (a Mann-Whitney U analysis) between the civilian
background and military background participants (combining the five sectors into
two) indicated that there was one significant difference, with a Bonferroni adjusted
critical value of p < .002. The significant difference in rating occurred for
communication with other aircraft (U = 502.0; Z = -3.104; p = .002, r = .347, a
medium effect). The civilian background participants rated the concept higher than
did the military background participants.
85
As there was only one significant difference, the sectors were combined to
allow an overall consideration of the ratings for the 26 concepts. A Friedman two-
way ANOVA indicated that there is significant variation in ratings over the concepts
(χ² = 461.731, df = 25, N = 80, p < .001). Figure 3.1 shows the distribution of the
ranking of the mean ratings, as calculated by this analysis.
Figure 3.1 The rank of mean ratings for the 26 concepts rated in Question 14
Situational awareness was rated highest, overall. There followed a group of
five concepts with approximately equal ratings, responsibility, discipline, judgement-
and-decision-making, and preparation. It is interesting to see responsibility and
discipline rated so highly, given that they were not mentioned very frequently in the
earlier “free recall” open ended questions. After fairly steep steps down from
common sense to medical fitness, the ranks decrease more gradually from crew
interaction to knowledge-of-the-regulatory/organisational-environment, and then
there is a tail of lower rated concepts from ability-to-operate-automation to courtesy.
86
Follow-up pair-wise comparisons with the Wilcoxen Signed Rank test and a
Bonferroni adjusted alpha of .0125 were made between situation-awareness and
responsibility, responsibility and risk assessment (in the upper slope), risk
assessment and communication-with-other-aircraft (in the lower slope) and between
communication-with-other-aircraft and ability-to-operate-automation (in the “tail’).
All four comparisons were significant.
Situation awareness (Mean Rank =18.93) was rated significantly more
important than responsibility (Mean Rank =17.5), T = 112, z = -2.524, (corrected for
ties), N-Ties = 14, p = .012. This effect can be described as large, r =.675.
Responsibility (Mean Rank =17.5) was rated significantly more important than risk
assessment (Mean Rank=15.4), T = 45, z =-2.803, (corrected for ties), N-Ties = 22, p
= .005, r =.598, a large effect. Risk assessment (Mean Rank =15.4) was rated
significantly more important than communication-with-other-aircraft (Mean rank =
12.18), T = 759.5, z = -2.956, (corrected for ties), N-ties = 45, p = .003; r = .441, a
medium to large effect. Finally, Communication-with-other-aircraft (Mean rank =
12.18) was rated significantly more important than ability-to-operate-automation
(Mean rank = 9.58), T = 300.00, z = -2.611, (corrected for ties), N-Ties = 45, p =
.009; r =.389, a medium effect. The overall significant variation in the ratings of
concepts is most likely because of differences between mean ratings across the range,
with some concepts rated alike, but others stepped apart, where several steps leads to
a significant difference in mean rating.
Ranking Question. Question 15 asked “If you had to pick the five most
important concepts, which would you pick?” Participants were asked to number their
top five from 1 (most important) to 5 (fifth most important), and to not number any
other concepts. As mentioned above, five participants felt that ranking only five was
too few and ranked more than five concepts; these participants were removed from
this analysis.
The resulting data was reversed scored, so that the most important concept now
had a rank of 5, down to the least important rank of 1 and the concepts not ranked
were given a 0. This resulted in a more meaningful ordinal scale.
87
A comparison of ratings between the five sectors of background and current
activity, using a Kruskal-Wallis ANOVA, indicated no significant differences, with a
Bonferroni adjusted critical value of p < .002.
A comparison of ratings (a Mann-Whitney U analysis) between the civilian
background and military background participants also indicated that there were no
significant differences with a Bonferroni adjusted critical value of p < .002.
With no significant differences, the sectors were combined to allow an overall
consideration of the ratings for the 26 concepts. A Friedman two-way ANOVA
indicated that there is significant variation in ratings over the concepts (N = 71, χ² =
402.549, df = 25, p < .001). Figure 3.2 shows the distribution of the mean rankings,
as calculated by the analysis.
Figure 3.2 The mean rankings for the 26 concepts ranked in Question 15
88
Follow-up pair-wise comparisons with the Wilcoxen Signed Rank test and a
Bonferroni adjusted α of .008 checked the differences between the three top-ranked
concepts and between concepts stepped along the distribution shown in Figure 3.2.
The latter comparisons were between risk analysis and discipline, risk analysis and
responsibility and discipline and crew interrelations. Of the six comparisons, four
were significant.
Situation awareness was ranked significantly higher than judgement/decision-
making with Z = -3.009 (corrected for Ties), N-Ties = 64, p = .003, 2-tailed, r = .376
(a medium effect). Forty-six respondents ranked situation awareness higher than
judgement/decision-making; 18 ranked judgement/decision-making higher than
situation awareness, and there were 8 ties (did not rank either concept). Situation
awareness was also ranked significantly higher than risk assessment: Z = -4.459
(corrected for ties), N-Ties = 62; p = .001, 2-tailed, r = .566 (a large effect). Forty-
five respondents ranked situation awareness higher than risk assessment; 17 ranked
risk assessment higher than situation awareness and there were 10 ties (did not rank
either concept).
The rankings of judgement/decision-making and risk assessment were not
significantly different, nor were the rankings of risk assessment and discipline. The
rankings of risk assessment and responsibility were significantly different, Z = -
2.673 (corrected for ties), N-Ties = 34; p = .008, 2-tailed, r = .458 (a large effect).
Twenty-six respondents ranked risk assessment higher than responsibility, 8 rated
responsibility higher than risk assessment, and there were 38 ties (did not rank either
concept). The rankings of discipline and crew interaction were also significantly
different, Z = -3.618 (corrected for ties), N-Ties = 22; p < .001, 2-tailed, r = .771, a
large effect. Nineteen respondents ranked discipline higher than crew interrelations,
3 ranked it lower and there were 50 ties (did not rank either concept).
The significant variation of rankings across the 26 concepts was partly due to
the high ranking of situational awareness and judgement/decision-making by many
of the respondents, and partly due to the differences between concepts at various
steps through the distribution shown in Figure 3.2.
Following Question 15, the final question asked for any other comment and
eight participants made some comment about the final two questions. They all
89
pointed out that it was difficult to separate the top five, several at some length. For
example: “How can you expect anyone to insert anything other than VERY
IMPORTANT in any of the boxes?” (042), “It is difficult as I would count a few of
these as equally important” (043) and “The foregoing concepts are inter-related, and
differing situational contexts may alter the assignment of importance. I would
include consistency, balance, established specialisation, a broad knowledge base, and
an enthusiasm for excellence.” (080). All but one of those that commented completed
the question.
Comparison among question types. There are some differences in the order
of importance ascribed to the concepts based on the three different styles of question:
open ended, rating and ranking. These can be seen most clearly in Table 3.10, where
the concepts raised in response to the combined definition questions and that were
either included in the Question 14 and Question 15 list or are similar to those
concepts, are ordered by frequency of mention in the left of the table, and the
Question 14 and Question 15 concepts are ordered by the Friedman analysis means,
in the centre and right of the table.
It is clear that situation awareness is seen as the most important aspect of
airmanship, with judgement/decision-making a close second, overall. Risk
assessment (or knowledge of risk) also maintained a fairly high profile across all
three question types. While discipline was not mentioned overly frequently in the
definition questions (half-way down the list), it is close to the top of the list when the
concepts were rated, and a bit lower for the ranking exercise but still near the top.
Similarly, common sense and preparation (included as mission planning in the
analysis of the open-ended definition questions) both moved further up the listing in
the last two questions. Knowledge of the environment was rated and ranked fairly
highly (in the top 10 of both Q14 and Q15) but was a bit lower down in the ranks in
the open-ended definition questions. Specific mention of the physical environment
and mention of the environment without specification were separately counted, and
combining the count would bring the open-ended definition questions ranking nearer
to the top.
90
Table 3.10
Airmanship Concepts Ordered by Frequency of Mention in the Definition Questions, Rating in Question 14 and Ranking in Question 15 Definition Questions N Question 14 Rate Question 15 Rank Situation Awareness 49 Sit Awareness 18.93 Sit Awareness 21.21
Judgement/DM 46 Responsibility 17.5 Judgement /DM 18.9
Know Aircraft 39 Discipline 17.42 Risk assessment 16.34
Personal qualities 39 Judgement/DM 17.38 Common sense 15.94
Know Legis (Regs) 37 Preparation 17.08 Preparation 15.63
Skill 33 Common sense 16.79 Discipline 14.69
Know Risk 28 Know Env 15.98 Know Env 14.48
Communication 28 Risk assessment 15.4 Know Aircraft 14.3
Crew co-ordination 22 Lookout 15.23 Proficiency 14.18
Common sense 21 Psych fitness 14.71 Flying skill 14
Courtesy 21 Know Aircraft 14.08 Team management 13.31
Know Self 20 Medical fitness 13.76 Responsibility 13.03
Discipline 16 Crew interrelations 12.97 KnowReg/Org 13
Know Env (Phys) 15 Team management 12.83 Personal attitude 12.68
Know Env(General) 13 Self-improvement 12.68 Comm Aircraft 12.46
Proficiency 13 Personal attitude 12.64 Psych fitness 12.07
Mission: Planning 12 Cooperation 12.27 Courtesy 12.06
Manage flight deck 12 Comm Aircraft 12.18 Lookout 11.75
Responsibility 11 Comm ATC 12.12 Crew interrelations 11.69
Know Team 9 Proficiency 11.86 Assertiveness 11.65
Team leadership 9 Know Reg/Org 11.66 Comm ATC 11.46
Self-assessment 8 OperateAutomation 9.58 Cooperation 11.46
Interpersonal 6 Comm Maint 9.48 Medical fitness 11.32
Manage automation 3 Flying skill 9.07 OperateAutomation 11.2
Self-improvement 2 Assertiveness 9.03 Self-improvement 11.18
Lookout 1 Courtesy 8.41 Comm Maint 11 Note: Know = Knowledge of; Legis = legislation; Regs = regulations, Org = organisation; Env = environment; Comm = communication with; Maint = maintenance
Flying skill, “Skill” in the definition questions, was mentioned frequently, in
the top 10, as it was in the Question 15 rankings. However in Question 14, flying
skill was not so highly placed, being down near the end of the list. Courtesy followed
a similar pattern, in the upper half of ordered concepts in the definition questions,
courtesy was at the end of the list in Question 14, but climbed to be in the middle
third of the Question 15 rankings. Communication was separated into three in Q14
and Q15, perhaps reducing its impact, as it dropped somewhat in the latter two
91
columns. Crew co-ordination (as coded in the definition questions) was nearest in
connotation to crew interrelations, which rated and ranked a bit lower than crew co-
ordination. Team leadership (in the definition questions) and team management (in
Q14 and 15) showed the opposite pattern, improving in rating and rankings.
Responsibility was not mentioned often in response to the definition questions, but
was near the top of the ratings, and about half way up the rankings.
The free recall in the definition questions led to a variety of different concepts
being brought up. Questions 14 and 15 listed only 26 concepts. The rating question
allowed every one of these 26 concept to be rated, and some respondents used more
of the available scale than others which may have meant that their negative (or less
positive) opinion of some concepts weighted the ratings disproportionately.
Meanwhile, in Question 15 only five concepts could be ranked, and perhaps a bit
more thought went into exactly which five concepts could be considered most
important. The graph shows flattening because all unselected concepts were assigned
“0”, thus being equally weighted in the analysis.
Conclusion – Definition of Airmanship
One of the aims of this study was to determine if there were differences in how
civilian and military pilots define airmanship. There were only two significant
differences between the civilian and military background participants, in terms of the
number of respondents mentioning each concept. Military background pilots
mentioned a specific military definition of airmanship while civilian pilots did not.
Significantly more military background pilots mentioned abnormal or emergency
events than did civilian background pilots. It is possible that more military
background pilots have experienced abnormal or emergency events, because military
flying pushes the envelope rather more than normal civilian flying. Overall, it cannot
be said, on the basis of this study, that there are significant differences in the
definition of airmanship between civilian background and military background
pilots.
The concepts included in the Kern (1996, 2009a) model of airmanship were
generally well represented in the free recall definition questions, although the
number of mentions differed significantly across the concepts. Situational awareness
and judgement/decision-making were most frequently written about by respondents.
92
Knowledge of the aircraft was the most frequently mentioned knowledge element,
although knowledge of the legislative and of organisational environments were also
frequently included in responses. Knowledge of the team was least often mentioned.
Knowledge was written about more as an application of knowledge than simply
possession of knowledge. The additional Ebbage and Spencer (2003) and Edwards
(2013) concepts were not as well represented, apart from communication and crew
co-ordination, with self-improvement and interpersonal skills least often mentioned.
Most frequently mentioned further concepts included safety, personal qualities,
experience, common sense and courtesy. Common sense tended to be used to
moderate the application of knowledge and decision-making, while courtesy was
associated with interaction with other pilots.
The rating and ranking questions supported situational awareness and
judgement/decision-making as the most important airmanship concepts. While
discipline was not mentioned very frequently in the free recall definition responses,
when it was available for rating or ranking it appeared high in the order of
importance. Preparation was similarly infrequently mentioned in the free recall
definition responses (coded as the Knowledge element: mission planning), but was
highly rated and ranked in the final two questions. Overall, the Kern model elements
were ranked as more important than any of the other concepts included in the
available options, with only common sense and responsibility intruding in the top
half of the rankings. The Kern elements were rated highly as well, again
responsibility and common sense were inclusions in the upper ratings, while
proficiency, knowledge of regulations/organisation and flying skill were low in the
order of ratings.
The concepts mentioned by Australian pilots and the importance placed on a
variety of airmanship related concepts, clearly supports those included in Kern’s
model of airmanship (2009a). However, the Kern model lacks reference to many of
the crew resource management concepts that were included by Ebbage and Spencer
(2003) and Edwards (2013) and that were mentioned by the respondents in this
study. Clearer reference to communication, the framing of knowledge items as
actively applied rather than passively held and inclusion of crew co-ordination and
team management should be part of any revised model of airmanship.
93
These results also support the importance placed on situational awareness,
decision making, risk assessment and flying skill (‘maintaining aircraft within
tolerances’) that are central to Mavin’s (2010) model for the assessment of pilots.
That these comments were so frequently mentioned supports the importance placed
on them by many researchers in the area of pilot performance (Flin & Martin, 2001;
Helmreich & Foushee, 2010).
Results - Training and Development of Airmanship
The survey included five questions that addressed the training and development
of airmanship as experienced by the respondents, and one question that asked any
respondents who were flight instructors or had a check/training role what they did to
ensure airmanship developed in their students. These questions were included to
address the second aim of this study, to examine how airmanship is included in flight
training and what contributes to the development of airmanship.
Initial Training
Questions 3 and 4 asked about the initial training of the participants. In order to
assess the responses to Questions 3 and 4, the participants were categorised by their
answer to the demographic question about the sector of their initial training and any
who were initially trained somewhere other than Australia (N = 6) were omitted from
this analysis. There were 37 participants whose initial training was in the civilian
sector, 28 whose initial training was with the military. A further 7 respondents who
were initially trained in the civilian sector but who then moved into the military (and
therefore started again in military ab initio training) clearly identified in Questions 3
and 4 that they were writing about their military experience and so were added to the
military-trained group, bringing that group up to 35 respondents. An additional 5
respondents who were initially trained in the civilian sector but then moved into the
military were removed from the analysis because it was not possible to tell which
sector they were actually referring to when asked about their initial training.
The participants were divided into “era of training” groups based upon when
they started their initial training (a question in the demographics section of the
questionnaire). They were placed in “before 1991” or “1991 and after” groups,
94
referring to the year in which human factors training was introduced to the civilian
Day VFR syllabus. It is possible that the introduction of human factors as a syllabus
area in the theory examinations system may have raised awareness of topics linked
with airmanship, such as situation awareness, decision making, communication and
teamwork, and that this awareness flowed on into flight training.
There is no equivalent change that occurred in military training, except perhaps
the move of ab initio training from Point Cook, Victoria, to Tamworth, NSW, and
the introduction of a civilian contractor to run the program (BAE Systems), however
part of the contract was to conform to the existing military training system. From
1986, all military cadets were required to take a degree at the Australian Defence
Force Academy before proceeding to flying training and this might be an appropriate
division point, but again, there was no official change in training activity at this date.
Ground Instruction – Question 3. Question 3 (a) of the questionnaire asked
“In your initial flight training, was airmanship, or concepts related to airmanship,
discussed in ground training or pre- or post- flight briefings?” and the survey was
set up so that they had to answer either “yes” or “no’. Table 3.11 indicates the
responses made by each group. Nearly all the participants indicated that airmanship
or airmanship related concepts were included in their early training, with a slightly
stronger response from the military trained groups. There was no significant
difference between eras within the civilian sector, nor within military sector, nor
when the sectors were combined. Combining eras, there was no difference between
sectors.
Table 3.11
Percentage of Civilian or Military Sectors Respondents by era of training (before 1991 or 1991 and after) who Responded to Question 3(a)
Civilian Military
Before After Before After
1991 1991 1991 1991
Number of Respondents 22 15 29 6
% % % %
Yes 86.4 86.7 93.1 100.0 No 13.6 13.3 6.9 0.0
95
Question 3 (b) asked “If so, which concepts, and what sorts of discussion?”
Table 3.12 summarises the concepts mentioned in the responses that match the
existing airmanship models.
Table 3.12
Percentage of Civilian or Military Sector Respondents by era of training (before 1991 or 1991 and after) who Mentioned the Listed Concepts From Airmanship Models in Response to Question 3(b)
Civilian Military
Before 1991
After 1991
Before 1991
After 1991
Number of Participants 22 15 29 6 Missing answers 4 2 4 0 Number of Respondents 18 13 26 6
% % % %
Kern concepts: Judgement or decision making 5.6 0 26.9 33.3 Situational awareness 33.3 7.7 19.2 50 Knowledge: General 5.6 0 7.7 0 Self 0 7.7 7.7 0 Aircraft 0 7.7 11.5 16.7 Team 0 15.4 0 0 Environment: General 0 7.7 7.7 0 Legislative 5.6 15.4 23.1 0 Organisational 0 0 34.6 0 Physical 0 0 3.8 0 Risk 5.6 7.7 3.8 0 Mission: Planning 5.6 7.7 15.4 0 Skill 5.6 0 15.4 0 Proficiency 5.6 0 3.8 0 Discipline 16.7 0 3.8 0
Additional Ebbage and Spencer or Edwards concepts: Attitudes: Hazardous 5.6 0 0 0
Professionalism 0 0 0 0 Self-improvement 0 0 0 0 Skills: Communication 22.2 15.4 3.8 0 Crew co-ordination 5.6 0 3.8 0 Interpersonal 0 0 0 0 Management flight-deck 5.6 0 3.8 0 Self-assessment 0 0 0 0 Team leadership 5.6 0 0 0
96
When considering ground training (the theory part of learning to fly) and the
pre and post flight briefings, most participants recalled that there was discussion of
the things that they consider to be airmanship. The details of what was mentioned
varied considerably, and cell numbers were generally too low for formal comparison
except in the case of situation awareness, judgement, safety, courtesy and lookout,
when frequencies were combined across eras of training to allow comparison across
sector, using a Pearson’s chi-square test of contingencies (with a Bonferroni adjusted
α such that p < .01).
Situation awareness was the second-most often mentioned concept, and there
was no significant difference between the sectors with regard to frequency of
mention. Mention of safety, courtesy and lookout did not differ significantly between
sectors. However, mention of judgement or decision making was nearing
significance, with more military-trained respondents mentioning than civilian-trained
respondents (χ² (1, N = 63) = 7.310, p = .013, w = .341, a medium effect).
The only civilian-trained respondent’s comment about judgement was the
comment “Mainly revolved around the concept of having the necessary skills to
make good decisions” (151). Some of the military-trained respondents simply listed
“making decisions” (197) or “applying sound judgement” (126) or similar, or else
expanded on the theme, like the comment “Post flight usually focussed on
performance on those aspects as well as any wrong decisions being made, or just the
lack of decisions which I believe is a more important skill to learn” (110).
Situational awareness was generally clearly stated by respondents, although
there were some who stated something else but the comment was coded as situation
awareness, such as “traffic monitoring” (035, civilian-trained), “interpretation of
incoming information” (124, military-trained) and “pre-flight tends to be listen and
lookout, traffic separation and a good understanding of where other aircraft are when
approaching an uncontrolled airfield” (200, military-trained).
Comments counted as general references to knowledge included the mention of
“knowledge” in a list of topics covered during briefings by one civilian trained
respondent and one military-trained respondent. Another military-trained person’s
response seemed to be separating knowledge from airmanship, writing “In training,
97
we were assessed on preparation, knowledge, the proficiency of our execution, and
airmanship considerations that applied to our prep and subsequent execution” (080).
The concept “knowledge-of-self” included comments about knowing one’s
own limitations and capabilities. References to knowledge-of-aircraft included
“aircraft limitations” (155) and also “knowledge of orders and instructions; aircraft
capability and systems” (126) and “judicious fuel management and monitoring on-
board indications to ensure the safe operation of all aircraft systems including power-
plant and life support” (187).
Knowledge-of-environment comments tended to be just references amid a list
of requirements, as were references to knowledge-of-team, for example “Knowledge
about one's aircraft, environment, crew, and operation” (162, military-trained).
Comments from respondents coded as knowledge of the legislative
environment and knowledge of the organisational environment were often linked, for
example “adherence to rules and procedures” (119, military-trained). Comments that
mentioned rules or regulations were coded in “knowledge-legislative”. Comments
that mentioned procedures, SOP’s or instructions were coded in “knowledge-
organisational”. The knowledge-of-the-physical-environment comment was a
reference to terrain. Knowledge about risk included comments about risk
management or risk assessment, or simply awareness of risks and threats. Comments
about preparation or planning or knowing about the operation were included in
knowledge of the mission.
Flying skill was included in the answers to Question 3b, for example “All
concepts of airmanship listed above were constantly discussed and reviewed in most
Mass Briefs and in all individual instructor flight briefs and debriefs. It's a
combination of several factors all working together [including] the ability to safely
and competently fly and navigate an aircraft.” (089, military trained). There were two
references to proficiency or “highly refined flying standards” (051, civilian trained).
Discipline was coded from mention of discipline, obedience and “not taking
shortcuts” (142, civilian trained).
From the additional Ebbage and Spencer/Edwards concepts, only
communication was frequently mentioned, especially among the civilian trained
respondents. Comments coded here included mention of making radio calls or radio
98
procedures as well as the use of the word communication. There was only one
reference that could be interpreted as relating to hazardous attitudes, which was “it
was a kind of ‘do unto others’ approach, as in not being a ‘cowboy’” (142, civilian
trained). Crew co-ordination included the listing of “crew support” (176, civilian
trained) and “timely and appropriate guidance to the crew” (091, military trained)
which was also coded as management in cockpit, as was “management skills” (040,
civilian-trained). Team leadership was represented by mention of “leadership skills”
(040, civilian-trained). There were no mentions of professionalism, self-
improvement, interpersonal skills or self-assessment.
Table 3.13 summarises the additional concepts and other comments made by
the participants in response to Question 3 (b).
Safety was the most frequently mentioned concept raised as discussed during
ground training, or in pre or post flight briefings. It was either included in a list of
things discussed, or specific attention was drawn, such as with the comment
“Airmanship was usually discussed pre-flight with respect to safety considerations”
(119, military-trained) and “the safety related aspects of airmanship were particularly
emphasised during training” (100, civilian-trained). Lookout was also frequently
mentioned, for example “Keep a good look out. See and be seen was always stressed
from day one” (026, civilian trained).
Courtesy was mentioned by respondents from both civilian and military
training backgrounds. Comments coded included mention of courtesy, “do unto
others”, being polite, showing consideration or having a pleasant manner, all
mentioned as something that was considered part of airmanship and discussed during
training.
Prioritising was mentioned more often by military-trained respondents as
something specifically taught. The saying “aviate, navigate, communicate” was
quoted as the order in which to prioritise. The remaining concepts listed in Table
3.13 were each mentioned by fewer than five respondents, but the variety shows the
range of different concepts remembered as being part of airmanship training.
99
Table 3.13
Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned the Listed Additional Concepts in Response to Question 3(b)
Civilian Military
Before 1991
After 1990
Before 1991
After 1990
Number of Participants 22 15 29 6 Missing answers 4 2 4 0 Number of Respondents 18 13 26 6
% % % %
Other concepts: Safety 27.8 15.4 38.5 16.7
Lookout 22.2 7.7 19.2 33.3 Courtesy 22.2 15.4 11.5 16.7 Prioritising 0 7.7 11.5 16.7 Caring for the a/c 11.1 0 7.7 0 Taxi 16.7 0 3.8 0 Common sense 5.6 0 7.7 0 Efficiency 5.6 7.7 3.8 0 Professionalism 5.6 7.7 0 16.7 CRM/TEM 0 15.4 3.8 0 Checks 0 0 7.7 0 Fuel 5.6 0 3.8 0 Weather 5.6 0 3.8 0 Abnormal/emergency 0 0 0 16.7 Documentation 5.6 0 0 0
Other Comments: Non-specific use 16.7 0 15.4 33.3
Can't remember 0 15.4 15.4 0 Pre-HF/TEM 16.7 0 3.8 0 Experience 0 7.7 11.5 0 Examination 0 0 11.5 0
Other comments that were made by respondents included complaint about the
non-specific use of the term “airmanship”. This included such comments as
“Airmanship was always used as a ‘catch all’ phrase with no specific definition”
(042, civilian trained), “Airmanship was a vague concept but in general related to
100
behaviours” (173, civilian trained), “Yes, but the concept was vague” (201, military
trained) and “Most lectures struggled to find an accurate definition. We were given a
few different definitions of airmanship from various writers, however there was not
one singled out” (185, military trained). A longer comment was:
In my military training, the concept of airmanship was a formal feature in both the sortie briefing and debriefing process (not so much ‘airborne’). On reflection, it was mainly used as a tool to define and/or reinforce those behaviours/outcomes during the flight which were acceptable or needed improvement or were unacceptable (189, military trained).
There were six respondents who knew that airmanship was discussed (replied
“yes” to question 3a) but could not remember the details. Four respondents made the
point that their training pre-dated the introduction of human factors or Threat and
Error Management training, for example, “Self-assessment and the management of
potential threats were not mentioned in those days (1960's)” (051, civilian trained)
and “The term threat and error management was not used back then, but Airmanship
was taught in the same sort of terms” (201, military trained). A few respondents
implied that later experience was required to firm up the concept of airmanship and
several military-trained respondents made mention of the fact that they were
examined on or assessed on airmanship as well as everything else.
101
In-air training – Question 4. When asked whether airmanship was part of
instruction in the air (Question 4 “Was the development of airmanship a specific part
of the initial ‘in the air’ flight training you did? If so, please explain what this
involved.”), some participants were equivocal, and did not give a definite yes or no
answer, and this question did not force the choice. Table 3.14 shows the pattern of
yes/no responses to Question 4.
Table 3.143
Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Responded to Question 4
Civilian Military
Before After Before After
1991 1991 1991 1991 Number of participants 22 15 29 6 Missing Answer 2 2 2 1 Number of respondents 20 13 27 5
% % % %
Yes 30.0 53.8 77.8 80.0 No 50.0 23.1 7.4 0.0 Other 20.0 23.1 14.8 20.0
There were no significant differences in the yes/no responses between eras
within each sector, although the Civilian sector looks as though it may have neared
significance (however p < .107). There was no significant difference between eras if
the sectors are combined. However, a comparison between sectors (ignoring era)
indicated that the military-trained respondents were more likely than the civilian-
trained respondents, to indicate that airmanship was discussed during their in-air
training (χ² (1, N = 53) = 11.841, p < .001, w = .473, a large effect).
102
Table 3.15 indicates the concepts that were mentioned by respondents in
answer to Question 4. Numbers of mentions were too low to carry out any statistical
tests, except for situational awareness, which was non-significant.
Table 3.15
Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned Concepts From the Airmanship Models in Response to Question 4.
Civilian Military
Before 1991
After 1991
Before 1991
After 1991
Number of Participants 22 15 29 6 Missing answers 6 3 2 1 Number of Respondents 16 12 27 5
% % % %
Kern Concepts: Judgement or decision making 0 0 7.4 20.0
Situational awareness 18.8 25.0 14.8 40.0 Knowledge: General 0 0 0 0 Self 18.8 0 0 0 Aircraft 6.3 0 7.4 20.0 Team 0 0 0 0 Environment: General 6.3 0 3.7 0 Legislative 0 0 7.4 20.0 Organisational 0 0 11.1 0 Physical 0 0 3.7 0 Risk 0 0 0 0 Mission: Planning 0 0 3.7 0 Skill 0 0 0 60.0 Proficiency 0 0 7.4 0 Discipline 0 0 11.1 0
Additional Ebbage and Spencer or Edwards concepts: Attitudes: Hazardous 0 0 0 0
Professionalism 0 0 0 0 Self-improvement 0 0 0 0 Skills: Communication 18.8 0 14.8 0 Crew co-ordination 0 0 0 0 Interpersonal 0 0 0 0 Management flight-deck 0 0 0 0 Self-assessment 0 0 0 0 Team leadership 0 0 0 0
103
Respondents from all groups mentioned that situational awareness was part of
their in-air training. Mostly this was clear inclusion of the term in their response, but
sometimes other words were used, for example “Having due respect and awareness
for other aircraft and their safety” (181, civilian-trained) or “Knowing and
developing an ‘air picture’ of what was going on around” (155, military-trained)
were included as situational awareness. Judgement, actually decision-making, was
mentioned only by military-trained respondents. One said that as students, decision-
making was forced upon them and that “how we addressed these problems was a
factor in determining our level of airmanship awareness” (165). The other two
mentions of decision-making were similar comments, that “airmanship was ...
positively reinforced when a good decision was made and negatively reinforced if a
poor decision was made” (187) and “when good decisions were made, this was
complemented as good airmanship and that when poor decisions were made, that this
was labelled as poor airmanship” (200).
The concept of knowledge-of-self included comments about personal abilities
and limitations. Knowledge-of-the-aircraft comments referred to limitations of the
machine and the flight envelope. There was a reiteration of comment on rules and
procedures, coded in knowledge of the environment, legislative and organisational.
The reference to the physical environment was a comment regarding awareness of
geographical position.
Skill was mentioned by one respondent who wrote that “things like lookout and
correct control of the aircraft was integral” (155, military trained) while two others
noted that preparation, technique and airmanship were assessed separately, thus
distinguishing between flying skill (technique) and airmanship. Proficiency was
mentioned by two additional military trained respondents as an expectation of
reaching a certain standard of flying skill. Discipline was a simple inclusion of
“Mental control of one self. (Discipline)” (162, civilian trained). Comments classed
as communication included reference to listening, listen-out, radio use, read-back,
and “maintenance of audio communication in relation to the position of other
aircraft” (090, military-trained).
Table 3.16 lists the additional concepts mentioned by respondents in answer to
Question 4, these covered what was included in the in-air training and also comments
about how the training was conducted
104
Table 3.16
Percentage of Civilian or Military Sector Respondents by Era of Training (before 1991 or 1991 and after) who Mentioned the Listed Additional Concepts in Response to Question 4.
Civilian Military
Before 1991
After 1991
Before 1991
After 1991
Number of Participants 22 15 29 6 Missing answers 6 3 2 1 Number of Respondents 16 12 27 5
% % % %
Additional Concepts: What Lookout 18.8 0 33.3 20.0
Courtesy 12.5 16.7 0 0 Abnormal/Emergency 0 0 7.4 20.0 Safety 6.3 16.7 0 0 TEM 0 8.3 7.4 0 Prioritise 0 8.3 3.7 0 Care for aircraft 0 0 3.7 0 Common sense 6.3 0 0 0 Survival 6.3 0 0 0 Taxi 6.3 0 0 0 Untrusting 0 8.3 0 0 Weather 6.3 0 3.7 0 Additional Concepts: How
Briefs 6.3 16.7 18.5 40.0 Demonstration 0 0 18.5 0 Examination 0 0 7.4 60.0 Correction 6.3 8.3 7.4 0 Training process 0 0 3.7 20.0 Good example 0 8.3 0 0 Other Comments:
Can't Remember 0 8.3 0 0 Informal 0 0 3.7 0 Pre-CRM/HF 6.3 0 3.7 0 Experience 12.5 8.3 7.4 0 Non-specific use 31.3 16.7 3.7 20.0
Again numbers were too low to carry out statistical tests, except for the
mention of lookout. Combined across eras, analysis of lookout comments indicated
that the military-trained respondents mentioning lookout as a topic for instruction
rather more than civilian-trained respondents but the difference was not significant.
Courtesy was only mentioned by civilian-trained respondents and performance
105
during abnormal or emergency events only by military-trained respondents. Safety
was commented upon only by a few civilian trained respondents.
There was some mention of the recognition and management of threats,
prioritisation and single mentions of several other relevant issues, including never
trusting anyone and keeping a close eye on the weather.
Some respondents included an explanation of the methods used in conducting
training, and these are listed as the “how” of training in Table 3.16. The inclusion of
airmanship in briefings was reiterated, despite the question being about in-air
training. Some of the military-trained respondents noted that airmanship was
demonstrated, although a few of these comments were actually about the
demonstration of flying sequences (and so technique related). Examination and
assessment was also revisited. There was mention of being corrected or reminded of
airmanship points during flights. A couple of respondents noted that they were
provided a consistently good example through their flight instructor.
One respondent noted that the training process included allowing him to
“progress down a ‘non-optimal’ course of action as long as safe in order to provide a
teaching point and as a chance to recognise a dangerous situation next time it
occurred” (124, military-trained). A further respondent gave an extensive explanation
of the training process:
A "building block" approach was used. A new student was not expected to handle complex emergencies from day one. But, once appropriate lectures, exams, experience and training had occurred (e.g. radio failure procedures) the student was generally exposed to a simulated exercise to “immerse” or “cement” familiarity with that aspect or discipline. Subsequent reinforcement might occur with no notice and an acceptable standard of proficiency was expected. By the end of the course syllabus students were expected to be able to use airmanship to diagnose and manage complex, and often compound (i.e. multiple concurrent) emergency scenarios to a successful outcome. (197, military-trained)
There were a few comments that were neither about what was included nor
how training was carried out – one such comment being that the respondent did not
remember the details. One person stated about airmanship training: “Not formally.
Flight training inherently critiques and develops airmanship” (119, military-trained)
106
and two noted that their training in airmanship pre-dated introduction of crew
resource management and human factors training.
There were also some comments coded as experience, but that reflected the
lack of regard for airmanship in their training, “No. It was like seamanship you
picked it up as you went along.” (009, civilian-trained), “Airmanship is something
that is gained with experience” (026, civilian-trained). “Not really. Well not for me
anyway. I think it's something that develops more with experience and the more
comfortable you are with the operation” (064, civilian-trained), “It was spoken of as
an evolving ethos, as it were, that came with experience and practice. Good
airmanship would allow you to apply some limited knowledge to the new/unexpected
circumstances” (088, military-trained) and “When you start training you as a student
are not aware of the word airman-ship. It is only after you survive the experience
that you realize that you have developed airman-ship” (096, military-trained).
Finally, there were some respondents who, as part of their response, indicated
that airmanship was integrated throughout the training rather than being a specific
area of instruction, but that it was not necessarily well defined. Some examples are:
“If you did something wrong or poorly, it was criticised as poor airmanship. When
you pushed the instructor for a definition of airmanship, they could only give
examples based on their recent experience” (042, civilian-trained), “It varied from
trainer to trainer. As far as I recall airmanship was a commonly used term but no one
could really explain what it meant.” (151,civilian-trained), “Not so much a ‘specific’
part, but it was just something which was constantly being taught throughout all my
training.” (054, civilian-trained) and “I don’t recall that it was specifically included
in training” (200, military-trained).
Discussion - initial training. Most of the respondents indicated that
airmanship was included in their ground training and/or pre and post flight briefs
(Q3), while more military than civilian trained respondents reported that airmanship
formed part of their in-air flight training (Q4). There were no significant differences
between eras of training. Airmanship (however defined) was reported to have been
included by respondents from both eras. Tracing any differences between the eras in
concepts raised by respondents was made difficult by the low numbers once the
respondents were divided into four groups, especially the military post-1991 group.
107
Safety was the most frequently mentioned concept raised as having been
discussed in ground-training or in briefs, followed by situational awareness, lookout,
judgement/decision-making, courtesy and then knowledge of regulations and
procedures. Most frequently mentioned concepts in the responses about in-air
training were lookout, situational awareness and communication, followed by
knowledge of the aircraft and courtesy. There is consistent inclusion of Kern’s
capstone outcomes of airmanship: situational awareness and judgement, as well as
the overall aim of airmanship – safety. Lookout is one activity that leads to
development of situational awareness, communication being another, and these were
both quite frequently mentioned. There was a range of other concepts included in
responses, covering many of the other Kern model elements as well as some Ebbage
and Spencer/Edwards elements.
Consistently omitted from responses were reference to self-improvement, self-
assessment, interpersonal skills (unless courtesy is interpreted as interpersonal skills)
and team related comments. The lack of referral to teamwork in ab-initio training is
understandable, as initial flying is not a team activity. However, it is interesting to
note that the respondents were not encouraged to consider self-assessment, self-
improvement or much in terms of knowledge of themselves, during their initial
training. A few comments on being aware of one’s own limitations were all that was
noted.
It is also note-worthy that complaint about airmanship being assessed without
provision of a clear definition was frequently brought up in responses to both
questions.
Development of Airmanship
Participants were asked, in Question 5, “Thinking about your progress as a
pilot, what do you feel led to the development of your sense of airmanship?” For
analysis of this question, participants were divided by sector as they were for the
earlier definition questions, civilian background (both airline and GA) and military
background including people who moved on to fly in airlines or in general aviation
after a period in the military. Table 3.17 shows the development factors included by
the respondents.
108
Table 3.17
Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 5
Civilian Military
Participants 40 43 Missing answers 4 2 Respondents 36 41
% %
Internal development factors: Experience 27.8 56.1
Personal qualities 30.6 19.5 Common sense 0 12.2 Responsibility 8.3 2.4 Thinking/reflection 8.3 2.4 Continual learning 2.8 4.9 Proficiency 8.3 0 Knowledge of Aircraft 0 2.4
External development factors: Good/poor examples 27.8 29.3
Training 16.7 22.0 Reading 5.6 24.4 Instructors 11.1 12.2 Talking with others 13.9 7.3 Mistakes 8.3 9.8 Safety 8.3 8.3 Organisational culture 5.6 12.2 Discussions 2.8 9.8 HF training/CRM 5.6 4.9 Supervision 0 9.8 Survival/threat 2.8 7.3 Death 0 7.3
The same question was presented in the previous survey and responses
separated into personal reasons and influences of others (see Table 2.6). The current
data can be arranged to reflect a similar pattern. Personal or internal forces include
one’s own experiences, motivation, common sense, responsibility, thinking /
reflection and continual learning. This selection could also include safety,
proficiency and aircraft handling. The influence of others over the development of
airmanship can be seen in the examples provided by others and their mistakes,
training including HF/CRM, instructors, supervision, organisational climate, reading,
109
talking to others and discussions. The death of a workmate could also be seen as an
external influence.
Formal comparison between the numbers of respondents from each sector was
conducted for the seven concepts mentioned most frequently. The Bonferrroni
adjusted critical value was p < .007. None of the comparisons were significant.
The most commonly included factor that influenced the development of
airmanship was experience. Mentioned more often by military background than by
civilian background participants (but not significantly so) generally this factor was
mentioned in combination with other factors. An example is “experience and
common sense as well as studying and reading books and magazines” (089, a
military + airline participant) or “I believe experience was probably the biggest
factor in developing a sense of airmanship” (100, airline participant).
Various personal qualities were mentioned as factors in developing airmanship,
slightly (but not significantly) more often by civilian than military background
participants. Mostly these were comments reflecting an internal motivation. Some
examples from civilian respondents include: “a personal desire to be better” (040),
“self-development” (151), “continual self-improvement” (179) and “the personal
rewards that come from keeping high standards” (051). Some military background
respondents wrote: “My own desire to apply and gain the appropriate skills to
produce acceptable results” (119), “I also wanted to be ‘the best’ at my job, bar
none” (046) and “the drive to fly well” (126). There were some other kinds of
comments. Two military participants mentioned fear of failure as a motivator to
develop airmanship, and one mentioned “attitudes of peers” (124), without indicating
whether these were positive or negative.
Only military participants mentioned common sense in response to this
question, but without much elaboration for example “Experience and common sense
as well as studying” (084), “common sense and good instructors” (025) and “using
common sense” (126). A lengthier response was “Repetition!! I'm not sure that
[people] have a particularly strong innate level of common sense. I suspect that
many aviation accidents may well be attributable to either a lack of airmanship (read
common sense in the air), or at best a poorly developed level of it” (099). Here it is
not so much that common sense was a driver for developing airmanship, as that
110
repetition develops common sense, seen as equivalent to airmanship by this
participant.
Some respondents mentioned that the increase in responsibility as they took
over running a business or reached command level was significant. A few
participants commented that reflection or thinking about their experiences led to
learning and development of airmanship, while some pointed out that the
development of airmanship is continual “I learnt about airmanship on every flight
that I flew - airmanship and its development never ends” (110, military). There were
a few civilian participants who indicated that their airmanship was improved because
of having to maintain their flying skills at a high level of proficiency. One military
background participant indicated that changing aircraft type is a key factor, saying
that “the most significant influence on the development of airmanship [is] the
changes in aircraft handling and performance [between types]” (013).
Observation and learning from the good (and poor) examples of others was the
most frequently mentioned external factor significant to the development of
airmanship, included by both groups of participants. Some examples of comments
from civilian background participants include: “Watch those who you considered to
be good pilots” (008), and “observing other pilots’ behaviour, good or bad” (178).
Some comments from military background participants are “the example set by
senior pilots” (091), “the initial training set the scene but seeing other pilots display
poor airmanship or good airmanship reinforced these principles in me” (111) and
“experience, and observing the correct and incorrect execution by more (and less)
experienced aircrew” (080).
Training or instruction was mentioned by both groups as a positive influence
on development of airmanship. Some examples from civilian background
respondents are “my initial training” (178) and “a consistently high standard of
training” (042). Military background examples are “excellent instruction” (088) and
“quality on-going check and training” (119). One dissenting voice stated “There was
nothing in my formal training – this was all self-development” (151, civilian).
Reading was significantly more often mentioned by military than by civilian
background participants (χ² (1, N = 77) = 5.169, p = .023, w = .259). Most of these
participants specifically indicated the reading of accident reports and/or flight safety
111
magazines. Several mentioned study or research, including one civilian participant
who indicated that his University degree was a factor in developing his sense of
airmanship.
Instructors or mentors were listed as forces behind the development of
airmanship by both civilian and military participants. As well as simply listing
instructors as influences, there were other comments, for example from two military
background participants: “the availability of (informal) mentors from time to time”
(089) and “mentors (ie pilots whose professionalism and skills I wanted to emulate)”
(197).
Talking to other pilots was more often listed by civilian than military
background participants (but not at statistically significant levels) and included
comments like “I ‘hung on every word’ of what I considered good advice /
knowledge imparted by those more experienced than myself” (166) and “Experience
of chatting with others, I have learnt much about ‘what not to do’ after chatting with
more experienced aviators. This has really developed airmanship” (037). Two
military participants made much the same comments “studied aviation from ‘top to
bottom’ and ‘picked the brains’ of anyone who I thought could help” (046) and
“while flying with more experienced people [took] the opportunity to seek their
knowledge or ideas on how to best do things” (165). One of each group mentioned
“war stories” as an influence, this being in reference to stories about flying
experiences, not necessarily wartime.
Learning from one’s own mistakes, or those of others, was another influence
listed by a few participants from each group, as was a sense of developing and
maintaining safety. Organisational culture was mentioned as an influence by the
military background participants, rather more than by civilian participants, perhaps a
reflection of pride in their military background. One comment from a civilian
participant was “In my first job... the company had what I consider to be a
professional approach to business” (058). Military background participants
commented: “operational culture on the squadron (very strong peer pressure and
inevitable disciplinary-based motivators!!!)” (189), “great training in a disciplined
and supportive organisation” (192) and “flying in a large flying organisation that
lived and breathed flying and allowed us to learn from the experiences (sometimes
mistakes) of others” (102).
112
Some respondents mentioned discussion, this was not quite the same as an
active attempt to gain knowledge from talking to others, but appeared to be more
training related. Mention of briefings was included in this category. Several
participants specifically mentioned training in human factors, crew resource
management or aviation psychology as having led to improvement in airmanship. A
few military background participants drew attention to the importance of good
supervision. The idea that one has to survive in a threat-filled environment
encouraged some participants in their development of airmanship, closely related to
this concept are the few who listed the death of a workmate or friend as something
that influenced them, for example “I attended more than ‘Forty’ funerals of fallen
comrades.....and I did not want to be one of the next” (046, military background
participant).
Discussion – development of airmanship. Overall, it is experience that is
mentioned most often as an influence in the development of airmanship, whether it is
personal experience or using the experiences and mistakes of others as salutary or
inspiring examples. Internal motivation is recognised as an important personal
quality that is engaged in the development of airmanship, which supports the position
of Ebbage and Spence (2003) that attitudes of professionalism and self-improvement
are important foundations for airmanship that should be developed in early training.
Kern (1996) also considers that the motivation for improvement should be instilled
early in training, and should be continued life-long as “empowered responsibility”
(Kern, 2009b). This is in contrast to the responses to Questions 3 and 4, in which the
ideas of self-improvement and self-assessment were barely mentioned.
The further development of airmanship with increased responsibility and the
continued reflection and thinking about experiences confirm that this aspect of
internal motivation continues beyond the initial training stages. The frequent mention
of early training, instructors and mentors, organisational culture and supervision,
recognises the importance of these roles in development of airmanship, supporting
the emphasis that both Kern (1996) and Ebbage and Spencer (2003) place on the
inclusion of airmanship in ab initio training, and the importance of positive role
models. However, as with the earlier findings presented in Chapter 2, it is not just the
ab initio instructor who provides this important role model, but later career
influences in other pilots as mentors that is also seen as important in developing
113
airmanship. There is also some recognition that airmanship is a case of life-long
learning, among a few of the respondents. It is also interesting to note some more
negative influences, in seeing aviation as something to be survived, and for a few, the
influence of a death forcing confrontation with the need to improve airmanship.
Current Training
Question 6 asked “Do you participate in ongoing training? Does this include
airmanship and if so, what does this entail?” and Table 3.18 provides a summary of
responses to this question. Because this question asked about the participants’ current
activity, their sector of current, or most recent, activity was used to examine the
responses, regardless of background. The Airline group consists of people from
either civilian or military background, currently or most recently flying for an airline.
The general aviation (GA) group is those participants from either civilian or military
background, currently or most recently flying in general aviation, including
recreation, flying training clubs or in charter or helicopter operations. The Military
group covers any participants who are currently, or were most recently, flying with
the military.
There was no statistically significant difference between the three sectors with
regard to participation in training. Where there were sufficient responses, chi-square
analysis was carried out, but there were no significant differences in frequency of
mention of any concept between the sectors.
Several respondents appeared to have misunderstood the questions and
answered with comments about what they did as trainers or instructors, rather than
the type of training that they engaged in themselves, these are shown as ‘instructor’
in the table.
Many respondents indicated that they participated in training, but that this did
not specifically include airmanship. Some comments from respondents in the GA
sector include “but it doesn't really include much airmanship” (064), “Once again it
is an implied notion rather than a specific detail” (073), and “It's more something we
talk about over a debrief/beer with the rest of the crew or other crews. Very informal
but lessons are learnt” (064). Some comment from airline participants includes
“Airmanship training, not specifically” (006), “airmanship is only acquired with
experience, not something that can be taught” (026), “but airmanship seems to be an
114
unspoken concept in modern training programs” (032) and “as it [airmanship] is
assumed knowledge at my level of operation” (156).
Table 3.18
Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 6
Airline GA Military
Number of Participants 42 26 15 Missing answers 3 4 1 Number of Respondents 39 22 14
% % %
Yes 84.6 72.7 64.3 No 12.8 18.2 14.3 Other 2.6 9.1 14.3
Concepts: Instructor 5.1 13.6 28.6
Yes, but not airmanship 28.2 36.4 21.4 Centrality 20.5 22.7 14.3 Inclusion with CRM/TEM/HF 23.1 9.1 28.6 Safety 2.6 0.0 21.4 Common sense 7.7 0.0 0.0 Lack of formal definition 10.3 0.0 0.0 Formality/requirement 0.0 4.5 0.0 Kind of Training 28.2 31.8 28.6 What:
emergencies 5.1 0.0 0.0 night ops 2.6 0.0 0.0 decisions 0.0 4.5 7.1 safe, legal, efficient 0.0 4.5 0.0 reading, talking, courtesy 0.0 4.5 0.0 Aircraft handling 2.6 0.0 0.0 SOPs 7.7 0.0 0.0 Note. ‘Other’ includes statements that did not indicate whether or not the respondent engaged in training.
Some respondents indicated that airmanship is central to all flight operations,
including training, and thus is not a separate item. For example, from airline
respondents “Organisationally, airmanship is at the core of all training and checking
activity. Rarely would it be considered a stand-alone subject” (051), “Each and every
day your management and airmanship skills are being assessed. It may be by ATC,
115
your crew or customers. It is a skill that needs to practiced and constantly
developed.” (176) “All flights and training entail the application and subsequent
assessment of airmanship principles” (119) and “[Airline] is pro-active in the
development of airmanship and it is reinforced by many monitoring and training
functions” (192).
Several respondents drew the parallel between airmanship and the content of
Crew Resource Management, Threat and Error Management and/or Human Factors
training, indicating that it is in these non-technical training areas that airmanship is
addressed. Examples of comments from the airline sector are “but now [airmanship]
is called different things CRM, Security, OH&S training, New Technology Training,
Technical training modules + Highly proscriptive operating procedures” (040), “It
does entail airmanship in a different guise; not airmanship per se but the study and
application of Human Factors and Crew Resource Management where we are taught
how to work together to achieve safe and effective operation and study the cause and
effect of how poor airmanship can contribute to accidents.” (166) and “CRM training
will often involve discussion on recent events and how we might handle them” (190).
From a GA respondent, “Crew resource management is a part of training nowadays,
which is an important aspect of airmanship” (101) and from military participants:
“Included, principally under CRM, during formal triennial refresher training” (086)
and “Crew Resource Management and Aviation Risk Management training is
refreshed annually” (097).
One airline participant referred to a cabin safety briefing “where safety aspects
of our operation are discussed” (058). The military respondents drew attention to
safety culture, safety ‘stand-downs’ and the responsibility for safe operations, as
being part of the airmanship training in which they participate.
There were some references to common sense, all from airline respondents, for
example “One could possibly summarise it as the effective application of good old
common sense!!” (166) and “I teach safety management systems, flight operations
management and aviation quality assurance. I stress common sense which in the final
analysis is airmanship” (009).
There was some complaint about the lack of definition of airmanship, from
airline respondents, and the consequent subjectivity of assessments. Some examples
116
are “Airmanship is an item required in our line check system, while not defined
anywhere, it concerns the general conduct of the flight” (043), “Airmanship is
always an assessable component, and is generally assigned a somewhat subjective
score by the check captain” (100), and “Airmanship seems to be a very subjective
thing to assess and it is entirely up to the assessor as to what their opinion might be”
(184).
About 30% of respondents overall gave a description of the type of ongoing
training that they are involved in, without necessarily indicating how airmanship was
included in these activities. Some airline examples are: “CRM and Recurrent training
simulators” (053), “always undertaking simulator check rides as part of licence
renewal requirement” (096), “ongoing training in the form of check flights and
simulator training at regular intervals” (100), “recurrent simulator checks” (179) and
“Training programs associated with my work as an ATPL pilot” (173). Among GA
respondents comment included: “I undergo 3 checks a year as part of my
employment” (054), “We conduct check and training every six months as part of our
companies ops manual” (113), and “If you can count instructor renewals as training”
(116). Military respondents included “Yes, annual check flights” (165) and “formal
triennial refresher training” (086) as well as reference to safety stand-downs.
There were few actual descriptions of what was included in training, whether
airmanship or not. One description came from an airline participant and included
“Day proficiency consists of a session of in-flight emergencies and refresher training
then the remainder of the two day shift observing normal operations with particular
emphasis on SOP's. Night proficiency is night sortie usually to a difficult bush strip
where night operations including some emergencies are checked.” (058). Standard
operating procedures (SOPs) were mentioned by other airline respondents, for
example one wrote “My organisation has a very strong SOP-based culture and
assessment is conducted at both technical and non-technical levels” (189).
Two respondents wrote about decision making. From one airline respondent
this was a negative comment: “Airmanship is rarely mentioned in ongoing training
and if mentioned is generally as a negative reinforcement to a poor decision” (187),
while a military instructor wrote that he takes over some tasks to allow the students
“a little more free brain space to work with, in problem solving and decision making”
(165). One GA respondent listed “reading as much as I can in the industry, asking
117
my peers about changes, respecting other fliers” (138) in response to the question
and another indicated keeping safe, legal and efficient.
Discussion – current training. In summary, most participants engage in some
form of on-going training, if they are still active pilots, and this does not generally
have any formal inclusion of airmanship as such, but there is recognition that
airmanship underpins all aviation activity. Quite a few respondents recognised the
application of CRM and HF training in developing airmanship. In addition, the lack
of formal definition and subjective assessment of airmanship was noted among some
airline pilots. There are some minority views expressed, such as airmanship being all
about aircraft handling, or that it is only common sense anyway, and that it cannot be
taught but depends upon experience.
Instructors’ Practices
In Question 9, participants were asked about their practices as instructors or
trainers with regard to airmanship: “If you are currently an instructor or hold a
check/training role, what do you do to ensure that your students develop good
airmanship? (or if you were an instructor, what did you do?). Responses to this
question are summarised in Table 3.19. The participants were divided into three
groups as they were for the analysis of Question 6, based on current sector of
activity, being airline, general aviation (GA) or military.
There was much less detail in these responses compared to previous questions,
possibly some respondents felt that they had already written enough. The general
aviation (GA) group, especially, gave very concise answers. Three respondents
referred to previous questions, content of which was not included in the analysis for
Question 9, but they also wrote additional comment, which was included. The kinds
of responses made fell into two main types: 1) indications about how training was
carried out and 2) discussion of what sorts of things students were taught. The
frequency of mention of each topic was low, and only briefings and examples were
tested for differences between sectors, in both cases, the differences were not
significant.
118
Table 3.19
Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 9
Airline GA Military
Participants 42 26 15 Missing answers 18 14 4 Respondents 24 12 11
% % %
Critical Comment 12.5 0 0 How:
Briefing/discussion 41.7 33.3 18.2 Examples 29.2 33.3 45.5 Scenarios 16.7 0.0 27.3 Observe 16.7 0.0 9.1 Correction 4.2 0.0 9.1 Information load 4.2 0.0 9.1 Experience 4.2 0.0 9.1 What:
Safety 4.2 33.3 27.3 Knowledge 8.3 25.0 27.3 Think/Introspection 20.8 0.0 18.2 Decisions 16.7 0.0 18.2 Situation awareness 16.7 0.0 9.1 Attitude 0.0 33.3 9.1 Preparation 4.2 8.3 9.1 Work cycle/routine 8.3 0.0 0.0 Communication 4.2 0.0 0.0 CRM 4.2 0.0 0.0 Crew coordination 4.2 0.0 0.0 Comparing types 4.2 0.0 0.0 Consideration 0.0 8.3 0.0 Skill 0.0 0.0 9.1
Three airline participants made critical comment about the content of the
question. One of these was that “All aspects of training input include airmanship as
intrinsic” (051) and thus no specific practices were required, one assumes, as no
further comment was made. An opposing view was presented as: “I NEVER use the
un-defined word Airmanship. I NEVER try to develop an undefined skill” (042) and
again no further comments were made. A third participant noted that “Yes, however
the due to the experience level of trainee already has a high level of airmanship
119
instilled.” (013), but did go on to give further comment. Airline participants are
likely to be training pilots who are undergoing type or upgrade training, rather than
initial training, handled by the GA instructors. The Military instructors could have
been doing either kind of training.
Briefings or discussion were the most frequent type of comment. One airline
respondent mentioned using “Kern's airmanship model frequently as a template for
discussion” (151). Others emphasised the importance of briefings and discussions for
highlighting issues to do with airmanship including “Briefings before and after flight
are extremely important in getting the right mind-set for safe and efficient flight”
(034), and “debrief airmanship issues ... analyse errors in order to help students avoid
repeating the same mistake in future” (197). Respondents from the GA sector wrote
much the same sort of comments, for example “discussion prior and post flight are
the best times to highlight airmanship.” (073) and “emphasize the importance of
these points during all stages of briefing and flight and post flight, even in general
conversation around a beer” (118). The military respondents wrote that briefs should
be thorough and consistent.
Being a good example as an instructor was brought up by many respondents.
An airline respondent stated: “First of all as an instructor, airmanship is taught by
example” (100) and another: “LEAD BY EXAMPLE!!!... The role of the mentor
cannot be overlooked in developing Pilot's Airmanship” (192). A GA respondent
wrote: “try to model appropriate attitudes and provide an example” (124). Some
military respondents had identical messages, including: “Demonstrate how to
develop good airmanship by example on each and every flight.” (090) and “Lead by
example. Ensure I do practice what I preach” (183).
Respondents from the airlines and military wrote about the use of different
scenarios in the training environment, so that what is learnt covers a “broad range of
operations” (025) and might be “as simple as an in-flight malfunction, through to a
major divert or critical malfunction... it also might include an unusual flight regime
or requirement.” (165), both are comments from military respondents. However, one
airline respondent warns, “adhere to the building-block approach (ie don't give an
impossible scenario for the student's level of experience)” (197) and another that “it
may be necessary to create a situation or allow a situation to develop to a stage where
the student ‘stares death in the face’ in order to get the message across” (166).
120
The latter approach is echoed by a few comments about simply observing how
the student manages events. Airline examples are “allow circumstances to develop
without comment or action and observe the crew(s) performance” (119), and “I
would let them manage the situation as much as was appropriate and their
authority/limits allow” (190), however a military respondent points out “another
noteworthy dilemma is how far you let a student go ‘safely’ to learn a valuable
lesson for life at the risk of letting him or her go too far and cause an accident” (102).
The final few comments concerning the ‘how’ of instruction included comment
on correcting students, “if they do something that would be considered poor
airmanship” (043), on not overloading them with too much information, and on the
role of experience. Two people commented that while they can provide the basics,
“finesse comes with experience and is an extension of the basics” (110) and “for the
most part pilots “evolve”, from student to Co-pilot to Captain the ability to see
problems and avoid or react increases with experience” (192).
One airline respondent made an additional statement in the next question that is
relevant here: “I actually provide my students with more than a comment along the
lines of ‘airmanship is really important’. I give them a framework of why it is
important, what is required, and how to get there!” (151), this being a reference to
the Kern model.
The ‘what’ of flight training looks quite familiar, the main components of the
two airmanship models are represented in the list in Table 3.18. Safety (the expected
outcome of the models) was the most frequently mentioned concept, as something
that the instructors were conscious of developing in their students. One airline
respondent wrote “I try to develop good, safe habits.” (042), echoed by GA
respondents in comments like “My aim is to instil a good attitude and good practices
that promote good airmanship and safe operation” (111) and “emphasizing the
importance of regulations and flight manual limitations to the safety of flight” (124).
A military instructor wrote “Teach them, and insist on TOTAL ADHERANCE to the
safe principles of flight....’Professional airmanship’......if they don't get the
picture....FAIL THEM” (046).
Knowledge was also a frequently mentioned concept among the ‘content’
concepts. This appeared in terms of imparting knowledge or checking it, and in some
121
cases knowledge was left unspecified. Specific mention was made of rules,
regulations, SOPs, and aircraft systems and limitations. One longer example is
Make sure they know the rules, SOPs etc, but that they don't get bogged down by them. Rules are for the guidance of the wise, and the blind obedience of fools - the trick is knowing when it's appropriate to bend/break the rules!! (099).
Both airline and military trainers explained that they try to get their students to
think about what they are doing and the consequences of their actions (or inactions),
and one added “encourage introspection” (173) as part of developing airmanship. It
was clear that the instructors among the participants were concerned that their
students develop good decision making skills. An airline participant noted that “it is
also essential that students be developed by allowing them to make decisions...”
(192), and military respondents added “decision making with the basics always being
used were the main areas for the students to have under control.” (110) and “force
issues upon students where a decision needs to be made. The making of this decision
will generally involve most of the airmanship considerations already discussed.”
(165). Situation awareness is also key aspect of airmanship, and was noted by both
airline and military instructors as something to be developed, for example, “I try to
develop constant, accurate situational awareness.” (042) and “Direction of attention
to ensure the student is picking up the correct clues (traffic etc.) whilst airborne to
update their situational awareness” (185).
There was some mention of developing the right attitude to aviation, which is
included in the Ebbage and Spencer (2003) model of airmanship and mentioned by
Kern (1996) as the first step in developing airmanship. This was most clearly stated
by a GA respondent as:
The most important thing you can do as an instructor is have the student obtain a state of mind that makes him or her an aviator... To develop good airmanship you have to be passionate about the industry you are entering and be wanting and willing to absorb and apply the knowledge and experience you gain (140).
A military respondent stated something similar, almost as a warning, “Be
aware of the likelihood of the development of habit patterns and attitudes in students
who are at a formative stage” (097).
122
Respondents from the GA and military trainers mentioned preparation as part
of the expectation they had of students. A GA respondent noted that he “Promoted
and tested preparation (knowledge and thinking) for flight” (024) and a military
respondent expected “pre-flight preparation (including positive mental preparation
by knowing how the sortie was going to be flown before getting into the briefing
room)” (110).
Creating a routine or ‘work cycle’ in the cockpit was mentioned by two airline
respondents, and communication, Crew Resource Management, and crew co-
ordination were each mentioned once by airline respondents. One airline respondent
stated that “as an instructor, the highlighting of traps and pitfalls and comparing to
previous aircraft types highlights airmanship issues” (013). One GA respondent
noted an expectation of courtesy in stating that “I demand that the pilot consider
others at all times” (111). Finally, a military respondent mentioned the requirement
for flying skill, stating “demonstrate of the discipline, requirement for skill and
knowledge across broad range of operations” (025). This was also the only mention
of discipline.
Some comment about instruction appeared in response to the following
question, but is relevant to Question 9. One of these comments was a serious
criticism of the competence of current instructors:
If there is one criticism of the aviation industry I have, it is the lack of intuitive airmanship skills in many young pilots. I put this down to the lack of “career instructors” now in the industry. Too often pilots are being trained by inexperienced instructors who are not particularly interested in their jobs, as all they want to do is progress to an airline position. So you have the situation where an inexperienced pilot who has not developed good airmanship knowledge / skills, being unable to pass those skills on to the student. (166)
123
Discussion – instructor’s practices. The three stages of development of
airmanship as noted by Kern (1996) and by Ebbage and Spencer (2003) are:
1) explanation of the importance of airmanship (Kern adds to instil motivation
to improve, while Ebbage and Spencer add discussion of case studies as a
method)
2) model and teach good airmanship (Ebbage and Spencer specifying skills,
knowledge and attitudes)
3) evaluate and provide feedback
None of the respondents, individually, clearly articulated the three stages as
presented in the two major models of airmanship. However, the components are
there in many of the responses. There were frequent mentions of discussion and
briefing of airmanship, and of the necessity to be a model of good airmanship. The
“what” of airmanship instruction covered many of the skills, knowledge and attitudes
specified in each model. There were not quite so many mentions of evaluation or
feedback, although some comment was made by a few respondents.
Change in Airmanship over Time
Question 10 asked the participants “Have your ideas about airmanship
changed over time? If so, in what way?” The response to this question may be
influenced by both the sector of training and background (civilian vs military) and by
the time spent flying in the different sectors (current activity). Table 3.20 shows the
responses to the question across the five sectors.
124
Table 3.20
Percentage of Respondents in Each Sector who Mentioned the Listed Factors in Response to Question 10 Background Civilian Military Current Activity Airline GA Airline GA Military Participants 21 19 21 7 15 Missing answers 2 5 3 0 1 Respondents 19 14 18 7 14
% % % % %
Yes 57.9 57.1 50.0 28.6 64.3 No 26.3 35.7 38.9 57.1 28.6 Other 15.8 7.1 11.1 14.3 7.1 Factors:
Experience 21.1 35.7 11.1 42.9 42.9 General development 10.5 14.3 11.1 14.3 28.6 CRM/Human Factors 10.5 7.1 11.1 14.3 0.0 Criticism of modernity 5.3 14.3 22.2 28.6 0.0 Awareness/content 10.5 0.0 27.8 0.0 0.0 Examples 15.8 7.1 0.0 28.6 0.0 Importance 15.8 0.0 5.6 0.0 7.1 For modernity 21.1 0.0 0.0 0.0 0.0 Training 10.5 14.3 5.6 0.0 0.0 Basics remain the same 5.3 0.0 5.6 0.0 0.0 Eastern Philosophy 0.0 7.1 0.0 0.0 0.0
In general, more than half of the respondents indicated that they felt that their
ideas about airmanship had changed over time, however, the respondents with a
military background but currently active in general aviation differed in that more
than half stated that their ideas had not changed. A chi-square analysis indicated that
there was no significant differences across sectors in the Yes/No responses to the
question (χ² (4, N = 64) = 2.856, p = .582). There were a number of respondents who
made some sort of general comment about airmanship but did not state a clear yes/no
response to the question. These responses are shown as ‘other’. In this question,
unlike Question 3a, the yes or no response was not set up with a tick box to force
response.
Several of those who said that their ideas had not changed, then qualified that
with a general observation such as “Not really changed, just developed more to
125
include other things” (054) or “No, just reinforced” (055) or “Matured but not
essentially changed” (086) .
The lower part of Table 3.20 shows the factors that were mentioned by
respondents in their answers. Responses included reference to experience, most
frequently across all sectors, except for military background airline pilots. A chi
square analysis indicated that any differences between the sectors were not
significant (χ² (4, N = 64) = 5.740, p = .219). Some examples of the use of
experience include: “Initially when I was inexperienced airmanship seemed like a
series of dot points (taxi speed, lookout, etc.). Over time, more experience and
knowledge of aviation in general has helped to define the concept.” (100, airline),
“No, I don't believe my ideas have changed over time, however airmanship is an
evolutionary process as one tends to build on it with experience.” (166, airline), “My
ideas about airmanship have not so much changed, but have evolved and developed
with increasing experience” (021, GA), “Yes. The balance of my importance on the
many aspects involved in airmanship has changed according with my experience.”
(038, military only), and “I have probably become more conscious of the application
of the principles of good airmanship as I accumulated experience and grew older”
(126, military-only).
There were eleven comments overall, that reflected an acknowledgement of
some general development over time. Numbers were too low to run any formal tests,
but the comments were fairly evenly spread across the sectors, with a few extra from
the military-only respondents. Some examples are: “I think I have always been in
pursuit of good airmanship in myself and others. Having the benefit of working with
some fine organizations and some excellent people I think my understanding of
airmanship has matured and become more inclusive” (058, airline), “Yes, as I have
matured more into a flight and instructional role I have found myself being more
hard on myself to set a higher standard” (118, GA). A military+GA participant wrote
“Yes I think I may have observed something in the past and not taken action or
thought ‘someone else will take care of that’. These days I tend to speak up straight
away” (200). Two military-only respondents wrote “Matured but not essentially
changed” (086) and “I guess it is a continual refining of what you knew before and
applying it to your latest situation” (185).
126
Six respondents made mention of CRM, TEM, non-technical behaviours or
human factors. Some comments were positive for example “Not only I have I grown
in that time but the industry has really changed regarding airmanship, safety and
CRM over the past 10 years and in a positive way. Training is improving and the
more we become educated about things like human error and accident prevention the
safer the skies will be. Airmanship is perhaps what links all this together” (043,
airline). Another was “Yes, initially my understanding of airmanship was simply
about being a safe pilot, now I view airmanship as being an array of pilot skills
involving technical and non-technical skills.” (178, military+GA). One comment was
not quite so positive: “As described, it has become de-identified by human factors,
CRM and TEM.” (013, military+airline).
There was some criticism of ‘modern trends’ or new pilots, mostly among
military + airline respondents, but with one comment from an airline respondent,
which was quoted above, regarding the lack of career-instructors and a prevalence of
young inexperienced instructors. This conclusion was echoed by a GA respondent:
“My ideas about airmanship have not changed only that I am concerned about a
growing lack of airmanship and awareness among many recreational pilots” (113). A
Military+Airline participant wrote: “Very critical of the current trend in risk-analysis
as it attempts to quantify and categorise a natural process that is inherent in those
who possess good airmanship. The ongoing process of experienced operators
coaching junior aircrew and/or ostracizing those who are aberrant is now seen to be
old fashioned and not PC” (119). Another noted: “The modern flight deck
environment and extensive use of automation and computerisation has clouded the
airmanship concepts eg. crew programming complex arrival procedures when a more
simple technique would suffice especially when time constrained” (013).
In contrast, there were four airline participants who were more positive about
changes in modern aviation. Two reflected on positive influence of CRM and human
factors research: “Yes. As human factors research has provided great insights into
behaviour during the last 20 years” (173). Another more general approving comment
was “As I get older and I see more new technology at play I realise that Airmanship
today is just as important as always, due to the fact that aircraft are managed around
the sky and airmanship is a required part of the management process.” (040), and
finally, with reference to another more recent initiative, “Operational safety audits
127
have done more than any other initiative to provide pilots with both good and bad
examples of airmanship among their peers. Ultimately, the organisation that provides
the individual pilot with tools to SELF MANAGE, particularly in terms of individual
strengths and susceptibilities, will achieve the highest order of airmanship.” (051).
Several respondents commented that their understanding of airmanship had
improved in terms of awareness of wider content. For example an airline respondent
wrote: “Yes, it was once considered the polite thing to do. Since then, airmanship for
me is much more diverse with many aspects” (043). Among military+airline
respondents three comments are: “Yes. I became more aware of how vague, or
general, the topic can be; but also more aware of the various elements of the subject.”
(034), “As one gets older and is no longer immortal then safety is something one
strives more actively to achieve rather than seen as the inevitable result of sound
aircraft operation” (071) and “I think the whole understanding of airmanship has
changed over the years. We have changed with it. It's not so magical any longer but
we are working at making it much more tangible” (081).
There were five comments that reflected increased awareness of the importance
of airmanship for example “Yes, become much more important in my daily
operations” (053, airline), “Yes. I have become more aware of the principles of
airmanship and the importance of its application.” (097, military) and “It's like
growing up! The older you get, the more important and involved you realize it
is.’(190, military+airline).
Examples drawn from other pilots were raised as a significant factor in
developing their airmanship by six respondents, some examples are: “They have
probably strengthened as a result of seeing how a lack of airmanship can lead to
poor, or in worst case, tragic, outcomes.” (032, airline), “Having the benefit of
working with some fine organizations and some excellent people” (058, airline) and
“My perceptions on airmanship have changed through flying/speaking with more
experienced aviators. Their mistakes have been there to help me to remember not
what to do!” (037, GA).
There were five mentions of training or instruction, one positive: “Training is
improving and the more we become educated about things like human error and
accident prevention the safer the skies will be.” (043, airline) and the others more
128
negative, for example: “yes it is not as good as it was I feel the training is not there”
(138, GA), “We are now expected to ‘teach’ airmanship as a subject, some of this is
appropriate but much about airmanship is how you live.” (140, GA), “Yes, there is
too much checking and not enough training” (191, military+airline).
Finally two respondents noted that the basics have remained the same, and one
GA respondent noted “Yes - I see eastern philosophy such as mediation as having a
role in performance enhancement” (143). It is possible that ‘meditation’ was the
word intended.
Discussion – development of airmanship. There were no significant
differences across the groups of respondents in the initial yes/no response to the
question. There were quite a variety of views expressed, with some people taking
advantage of the survey to voice some cherished complaints about the industry, while
others were more positive. On the whole, most people recognised that their views of
airmanship had changed, developed or matured during their time in aviation. Mostly
this was put down to experience, to the introduction of CRM and HF training or to
the examples of others. Many also just commented on general development,
increased awareness or developing a sense of the importance of airmanship, without
referring to any mechanism through which this might have occurred. The numbers of
comments were too low to carry out any extensive statistical comparisons.
Conclusion – Training and Development of Airmanship
One aim of this project was to investigate whether any differences in the
training of airmanship had occurred with the introduction of human factors and crew
resource management to the theory syllabus. It became apparent that the very
experienced participants were mostly trained before that 1991 introduction,
especially the military background participants. Nonetheless, there did not appear to
be any differences between sectors or eras when asked about inclusion of the
concepts of airmanship in ground training and briefing. The military participants
were more likely to report inclusion of airmanship in in-air training, but differences
between eras was not significant. Decision-making in briefings was mentioned by
more military than civilian participants, but not significantly so. Situational
awareness and lookout were among the most frequently mentioned concepts. The
lack of mention of any self-awareness or self-improvement concepts in response to
129
the questions about initial training is interesting, especially compared with the
number of referrals to these sorts of personal qualities that were mentioned in
response to the question about the development of airmanship
The second aim was to determine whether airmanship training conforms to the
three part formula that Kern (1996) and Ebbage and Spencer (2003) recommended.
As discussed above, none of the respondents, individually, clearly articulated these
three stages. The main components are there in many of the responses, with mention
of briefing, debriefing and having taken note of examples of airmanship or providing
a good example.
Assessment of airmanship without reference to a strict definition appeared as a
fairly common complaint in response to the initial training questions and (less
commonly) in response to the question about current training.
The Instructors among the participants were asked about their practices to
ensure airmanship is developed in their students, and generally referred to the
importance of briefings and of providing a good example. There was an emphasis on
safety and development of knowledge among GA and military respondents, while
airline respondents were more concerned with thinking, decisions and situational
awareness among their students. This relates well to the findings of Mavin (2010)
and Mavin and Roth (2014) regarding assessment in airline training.
The final question in this section, about any change in ideas over time, resulted
in a number of comments about the role of experience. There was also at least some
mention of the influence of CRM and/or human factors training on their development
of airmanship. Criticism of modern techniques and young, inexperienced instructors
by some of the more experienced respondents was balanced by others who
recognized the positive influence of the introduction of human factors training and
other safety related initiatives.
The majority of the participants were trained quite some time ago (62% before
1991) so asking about their ab-initio training provides information that has to be
considered time related. Of those that are current instructors, not all answered the
question about current training practices, nor can it be ascertained whether they were
referring to ab-initio or more advanced training in that question. The inclusion of
airmanship in current training and the way in which this is carried out, may be better
130
considered by a more direct questioning approach, addressed in the next part of the
project, the interviewing of current instructors in military and civilian sectors, at ab-
initio and advanced levels.
131
Chapter 4. Current Airmanship Training and Assessment
The conduct of online surveys, as described in the preceding chapters, allowed
for the questioning of a cross-section of participants in the aviation industry about
their views on airmanship. The questions used were limited in the extent to which
one can be sure that the responses referred to current practice or reminiscence from
past experience. Respondents to these surveys included some current aviation
instructors but the length of the survey and the position of the questions about
training towards the end of the survey resulted in lack of response by some
instructors. It was not clear whether instructors who did respond were referring to ab-
initio or more advanced training, or were reporting ideas that were based on a
combination of all types of instruction they had been involved with. Further directed
enquiry would allow a more detailed examination of the role of airmanship in flight
training.
The initial stage of learning to fly is referred to as ab-initio training. This is
generally carried out in flying clubs on a casual basis, in flying schools catering for
those who wish to make commercial aviation their career, or for flying cadets in the
military, at the Basic Flight Training School (BFTS) in Tamworth. After passing this
course, Air Force, Army and Navy cadets progress separately to further training on
their service’s specific aircraft types.
As a flying career progresses, training becomes more specific. In the civilian
airlines, it is a requirement that pilots undergo training and checking every six
months. In the larger airlines, this check and training process generally occurs in a
simulator environment, although line checks also occur during commercial flights.
Airline pilots can be selected to transfer to different types of aircraft within the same
company, requiring a conversion course. Command training occurs when a pilot at
First Officer level is ready to move up to a Captain position. This training is
generally carried out by more senior Captains who have received training as trainers
within the company. In the RAAF, pilots also have additional training to convert
onto specific types of aircraft. It is also possible to be selected to undergo training to
become a Qualified Flight Instructor at Central Flying School (CFS, East Sale), and
then instruct at BFTS or at other training squadrons within the RAAF.
132
The purpose of this section of the research project was to discover what current
flight instructors in the military and civilian environments both at ab-initio and at
more advanced levels of training, think airmanship is. A further interest is in how
(and if) they include airmanship considerations in the training and assessment that
they carry out. To this end, several civilian flight training schools were approached
for instructors to participate and one of the major Australian airlines were asked for
access to instructors in their training sections. Two additional former airline trainers
(from a different airline) known to the researcher also agreed to participate.
Permission was gained from the RAAF to approach BFTS, the Central Flying School
and a training squadron at Williamtown for participants. The training program at
BFTS is administered by a civilian contractor, and some of the instructors are
civilians, while others are military. Both sets of instructors were sampled, with
permission from the civilian organisation. Interviews were carried out allowing more
depth of discussion than was achieved in the on-line surveys.
Method
Participants
A total of 39 participants from the organisations outlined above were
interviewed. The combinations of civilian and military activities and level of training
undertaken resulted in five groups of instructors: two groups representing the ab-
initio level of military training at BFTS with the instructors divided by whether they
were military or civilian; instructors at the more advanced level within the military,
some of whom train trainers at CFS and others carry out jet fighter conversions at a
Williamtown based Squadron; ab-initio instructors from two civilian flying schools
in regional NSW; and Check and Training Captains from two major airlines, the
advanced level of civilian training. All the participants were male and all had been
active in Australian aviation, whether military or civilian, for at least six years. The
range of aviation experience was from 6 years to 38 years. Experience as an
instructor ranged from 4 months to 17 years.
Measure
The interviews were semi-structured, with some set questions that varied a
little between groups. The interviewer probed and asked for elaboration as necessary,
without leading the respondent. The participants were asked about their years of
133
experience and any tertiary qualifications. Tertiary qualifications were questioned as
the Griffith University undergraduate aviation program has a specific “Airmanship”
course, whereas the other Australian university aviation courses do not, but do have
human factors courses. The first question was simply along the lines of “what do you
think airmanship is?” The interviewer asked for clarification if the participant simply
listed concepts or made general statements.
Once the participant seemed to have covered all their initial points, they were
asked what other skills, knowledge or attitudes might also be a part of airmanship.
Then they were asked to provide an example of a situation in which good airmanship
was displayed, and then an example in which a pilot displayed poor airmanship. The
participants were asked if they specifically mention airmanship in the pre-flight
briefing, and what sorts of things are mentioned. If they were an ab-initio instructor
they were also asked what aspects of airmanship were included in the initial lessons.
They were then asked about the assessment of airmanship and what sorts of
behaviours they look for in making any such assessment. Finally participants were
told about the findings of a survey of instructors (Carrick, Graham, Healey &
Pickard, 2008) which indicated that decision making, safety and situational
awareness were most often mentioned in regard to assessment of airmanship and that
written reports of feedback generally referred to checks, situational awareness and
radio calls in regard to airmanship. The participants were asked if they agreed with
these findings, and would they add anything or not include anything. The interview
schedules are presented in Appendix D. In general each interview ran for about 40 to
60 minutes.
Procedure
Organisations were contacted with a request for volunteers. A convenient date
for interviews was then established. The researcher conducted interviews at the place
of employment in vacant offices, conference rooms, or the participants own office, as
available. The interviews were electronically recorded using an iRiver MP3 player
with a small tape recorder used as a backup. The resulting voice files were
transcribed to word files by a transcription agency, and then checked by the
researcher to ensure that the transcript matched the audio file. The researcher
conducting the interviews had attended training in interview techniques, had
conducted many selection interviews (for CASA and ATSB) and lectures in
134
assessment techniques, including interviews, in undergraduate and post-graduate
psychology programs.
Analysis
The transcribed interview files were loaded into NVivo10 (QSR, 2013) as five
separate projects, one for each group. The analysis process followed the sequence
described in Chapter 2. Two coders were involved, both had undertaken training in
qualitative analysis and the use of NVivo and communicated frequently concerning
the coding process. In the same fashion as the survey analyses reported in earlier
chapters, tree nodes were established representing the main concepts included in
Kern (1996, 2009a) and the aspects of the Ebbage and Spencer (2003) and Edwards
(2013) model not covered by Kern. Additional free nodes were established to
represent further concepts discovered during coding. Cross-checking between the
project files ensured that the free nodes covered the same concepts in each group of
interviews. Once coding was completed, nodes were compared and combined where
there was overlap in concepts. Concepts that were unique to each group or were not
widely mentioned were still represented in the final node set. A total of 67 free nodes
were coded, relating to the main themes of the definition of airmanship (38), general
comments and overviews about airmanship (17) and relating to the training and
assessment process (12). The final coded data set was agreed by both coders to be
reliably and accurately coded. Each participant file was identified by a three letter
code and these codes are used to identify participants when quoting their comments
below.
135
Results – Analysis of Demographics
Experience, education and transcript length were examined to ensure that no
systematic bias was present in the data sets.
Experience and Education
Table 4.1 shows the numbers of instructors in each group and their years of
experience, in aviation in total and as an instructor. The number of participants in
each group with tertiary degrees is also presented.
Table 4.1
Numbers of Instructors in Each Group, Experience in Aviation and Tertiary Degree Status. Civilian
Ab-initio trainers
Civilian Advanced
trainers
Civilian at BFTS
Military at BFTS
Military Advanced
trainers N 7 7 9 6 10
Total experience (years) Range 7 - 17 25 - 38 10 - 32 6 - 18 8 - 23 Mean 11.29 30.28 22.22 11.0 15.5 Std.dev 3.545 4.231 7.496 5.019 6.381
Instruction experience (years) Range .33 - 14 4 - 17 1.5 - 14 .3 - 15 .15 - 15 Mean 7.76 10.0 7.94 4.03 6.37 Std.dev 5.453 4.041 5.0 5.769 5.219
Tertiary Education Bachelor degree 3 0 0 4 4 Masters degree 1 2 0 1 1
The distribution of years of experience as an instructor was non-normal for the
Military BFTS group and the test of homogeneity of variance was significant for
total experience, thus the conservative approach is to use a non-parametric test to
check the differences between the means for both variables. Kruskall-Wallis
ANOVAs indicated that there were no significant differences among the groups
means for number of years of experience instructing (H (corrected for ties) = 4.942,
df = 4, N = 39, p = .293 and Cohen’s f = 0.387). The effect size f could be considered
large, so there is a difference among the means, possible not significant because of
the low number of participants. There was a significant difference among the groups
in terms of mean total experience, H (corrected for ties) = 22.397, df = 4, N = 39, p <
.001 and Cohen’s f = 1.198. The effect size is very large.
136
The civilian advanced trainers were the most experienced group and had the
most time in the job, and the military ab-initio participants were the least
experienced. This reflects the differing expectations in the two sectors: in airlines,
experienced Captains are deliberately chosen for Check and Training roles; in the
military, the training role is only undertaken after one or more operational postings,
but this can occur fairly early on in a military career and as it happened there were
several relatively new instructors among the participants. In contrast, civilian ab-
initio training roles tend to be taken up by low-time pilots building experience
towards their ATPL, although this is not always the case, and the participants
included some long term ab-initio instructors.
The participants were asked about any higher education undertaken. Two of the
civilian ab-initio instructors had completed the BSc (Aviation) and one had the
Master of Aviation Management from the University of Newcastle, one other
instructor had a B Computer Science. Two of the Civilian Advanced group of
instructors held the Master of Aviation Management from Griffith University. None
of the Civilian background instructors at BFTS had completed degrees. Among the
Military BFTS instructors, four had bachelor degrees from the Australian Defence
Force Academy (ADFA) of which only one was related to aviation, being in
aeronautical engineering. One additional instructor in this group held a Master in
Science (Aviation) from University of NSW (UNSW). Four of the Military
Advanced instructors held Bachelor level degrees, none related to aviation. One
instructor held a Master of Aviation Management from University of Western
Sydney. It is not a requirement to have taken a degree program to enter the aviation
industry, however it is becoming more common. Since 1986, military pilots have
been required to take a degree program at ADFA before moving on to the flight
training school at Tamworth, unless they are direct entry recruits or have taken a
degree elsewhere before recruitment. None of the degree programs taken by
participants had a specific airmanship course.
Transcript length.
An analysis of the number of words in each transcript was undertaken, to
determine if there were any differences between the groups. Table 4.2 shows the
relevant data for each group. The tests for normality and skew were acceptable and
Levene’s statistic was non-significant, F(4,34) = 1.750, p = .162, so the assumption
137
of homogeneity of variance was not violated. An ANOVA was statistically
significant indicating difference among the group means for transcript length, F (4,
34) = 2.701, p = .047. The effect size was f = .564 which can be considered a large
effect. Post hoc tests, using Gabriel’s procedure due to uneven group sizes, showed
that the only significant difference was between the civilian ab-initio group and the
civilian advanced group (p = .026). The civilian ab-initio instructors did not talk as
much as the civilian advanced trainers,
Table 4.2
Mean Transcript Word Length for each Participant Group, Standard Deviation and Range. CAB CAD CBF MBF MAD Total N 7 7 9 6 10 39 Mean words 5730.29 9361.14 7180.33 7393.83 7724.00 7483.74 Std deviation 2535.63 2517.04 1233.13 1436.11 2378.08 2280.43 Minimum 2753 7270 5383 4934 3788 2753 Maximum 10021 13675 9120 9265 11594 13675 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced
Results – Definition of Airmanship
Process comments.
Comments were made about the process of defining airmanship, by some of
the participants. Some mentioned the difficulty of defining airmanship, others made
the point that airmanship is all-encompassing and/or made statements that it is very
informal and non-specific. Table 4.3 shows the distribution of these comments across
the groups of instructors. In the discussion of the concepts mentioned by the
participants, any quotations are identified by a three letter participant code.
138
Table 4.3
Percentage of Participants in Each Group, and of Total Participants, who Commented on the Process of Definition CAB CAD CBF MBF MAD Total
N 7 7 9 6 10 39
Comments % % % % % %
Difficult to define 14.3 42.9 0.0 16.7 60.0 28.2 All-encompassing 14.3 71.4 100.0 100.0 60.0 69.2 Subjective 42.9 71.4 11.0 0.0 10.0 25.6 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced
More of the advanced military instructors said that airmanship is difficult to
define than instructors in the other groups. One military instructor stated “it is
broadly talked about all the time, but is almost impossible to define” (MEG) while
another instructor was quite adamant, “airmanship is subjective, I will not define
good airmanship” (MEA). Instructors from the other groups simply stated that it is
difficult to define.
Many of the participants noted that airmanship is an “all-encompassing”
concept, including all of the instructors interviewed at BFTS. Comments from the
civilian BFTS instructors included “we score preparation, technique and airmanship,
to me they all go hand-in-hand” (BCA) “they are all interwoven” (BCB), “it is an
overarching umbrella to your performance” (BCC) and “I don’t think that you can
separate it… it is a combination of everything … how you handle everything”
(BCE). The military BFTS instructors often stated that the boundaries are blurred,
and that preparation, technique and airmanship are all closely linked and explained as
“it is a general feel of how he is operating the whole aircraft” (BME). Comments
from the advanced civilian instructors included that it is a global view and that it is
an integration of many skills. This longer statement from a civilian advanced
instructor illustrates the idea that airmanship is everything put together:
Because sometimes we know we’ll use that, you know, “that was fabulous airmanship” and that just covers a lot of things because the whole operation whether it be technical or soft side of things was well done and put together and the result was what we wanted to see as far as the aeroplane arriving where it’s supposed to arrive safely. (CCE)
139
About a quarter of the participants also noted that airmanship is fairly
subjective and broad brush (CAC, CAD, BCC) and used adjectives like “intangible”
(CCB) “holistic” (CCC) and “unquantifiable” (CCE) or as one advanced military
instructor said: “it is all esoteric… you can’t write it down” (MEA). This instructor
summarised the interview with the comment:
I really can’t go beyond it’s the way that you go about doing your job, and even if the job is exactly the same day after day after day, the environment that you’re working in, the people that you’re working with change on a daily basis and that affects airmanship. So even though a certain proportion of the way that you do your job doesn’t change on a daily basis, the airmanship side of it does and it’s very, very subjective (MEA).
Despite some saying that airmanship is difficult to define, all-encompassing
and unspecifiable, all the participants were willing to talk extensively about what
they thought was involved in airmanship.
Initial definitions.
The participants’ initial reaction to the question “what do you think airmanship
is?”, after explaining the difficulties in defining it, generally resulted in some attempt
at a definition or mention of important aspects. Among the 39 participants, there
were 25 different combinations of concepts mentioned in the first reaction to the
question. Overall, 12 of the participants mentioned situational awareness, in
combination with various other concepts, eight included following of rules and
regulations, and seven talked about decision making. Nine of the responses included
the basic military definition of safe and efficient flight, some with additional
concepts and two included courtesy. Nine of the respondents mentioned common
sense, two of them only included common sense and the other seven talked about it
in combination with other concepts as well. Table 4.4 shows the various
combinations of concepts, and how they were represented across the groups of
instructors. Apart from the numbers of instructors who used the military definition as
a base for their own definition, there were few other patterns, but some common
themes such as frequent mention of situation awareness, decision making and
common sense. The military definition of airmanship, as mentioned in Chapter 3, is
the phrase: “the safe and efficient operation of a flight, both in the air and on the
ground”.
140
Table 4.4
Percentage of Participants in Each Group who Spoke of Listed Concepts in Their Initial Definition of Airmanship CAB CAD CBF MBF MAD Total Number of Respondents 7 7 9 6 10 39
% % % % % %
Concept Combinations Safe and efficient 0 0 9.1 66.7 20 17.9 Safe &efficient, courtesy 28.6 0 0 0 0 5.1 SA (but vague) 0 14.3 0 0 10 5.1 SA and safety 14.3 0 0 0 0 2.6 Care for aircraft, SA, ongoing study 14.3 0 0 0 0 2.6 SA, knowledge, perception, handle self, manage risk 0 0 9.1 0 0 2.6 SA, care for aircraft, safety 0 0 9.1 0 0 2.6 Professional pride, CS and SA 0 0 9.1 0 0 2.6 CS, SA and Safety 0 0 9.1 0 0 2.6 CS 0 0 9.1 0 10 5.1 CS and thinking 0 14.3 0 0 0 2.6 CS and experience 0 14.3 0 0 10 5.1 Responsible conduct CS, courtesy 14.3 0 0 0 0 2.6 Thinking and DM 0 14.3 0 0 0 2.6 SA, DM, rules 0 0 0 16.7 20 7.7 DM and prioritizing 0 0 0 0 10 2.6 Knowledge, attitudes, TEM 0 14.3 0 0 0 2.6 Non-technical skills 0 14.3 0 0 0 2.6 Thinking behaviour and interaction for safe flight 0 14.3 0 0 0 2.6 Preparation and mental agility, ability 0 0 9.1 0 0 2.6 Discipline, rules and regulations, courtesy 0 0 0 16.7 0 2.6 Discipline, knowledge, safety 0 0 9.1 0 0 2.6 Checks, lookout, safety 0 0 9.1 0 0 2.6 Follow the rules 28.6 0 0 0 0 5.1 Reducing risk 0 0 0 0 10 2.6 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced; SA = situation awareness; CS = common sense; DM = decision making; TEM = threat and error management
Once a basic definition was stated, the participants were asked what made up
the components of that definition, or were asked to elaborate on some of the points.
141
The next section deconstructs the full interview in terms of the concepts mentioned
and their distribution over the various instructor groups.
Model related concepts.
The Kern model was mentioned specifically by two of the civilian advanced
group of participants, almost as soon as they had been asked to define airmanship
and by one of the military advanced group, towards the end of his interview. In all
three cases, the participant did not step through the Kern model, but rather, having
noted that they had read Kern (1996) went on to talk about their own ideas about
airmanship. One of the civilian advanced instructors went on to talk about common
sense, knowledge and experience. The other civilian advanced instructor talked about
the use of threat and error management as a structure around which to build
airmanship. The military advanced participant who had read Kern (1996) stated that
he did not find the book particularly helpful and did not discuss any details of the
model as such.
Most of the concepts included in the models of airmanship, Kern (1996, 2009b)
Ebbage and Spencer (2003 ) and Edwards (2013), were spoken about by participants,
but not all of them by any one participant. Table 4.5 shows the percentage of
participants who raised each concept during the interview.
The majority of participants in each group mentioned the capstone outcomes of
the Kern model, situational awareness and decision making (the term “judgement”
was not generally used) at some point in the interviews. Eight participants were
prompted by the interviewer about situational awareness and/or decision making
once the question based on previous research was reached, all but one then agreed
that these concepts were an important part of airmanship and spoke about their
relevance to airmanship. The one exception was a participant in the civilian BFTS
group, who deflected discussion into the idea of thinking through problems and
coming up with solutions, but did not actually mention decision making, as such.
142
Table 4.5
Percentage of Participants in Each Group who Mentioned Specific Definitions or Listed Concepts from the Airmanship Models During Interview
CAB CAD CBF MBF MAD Total
Number of Participants 7 7 9 6 10 39 Concepts – Kern model % % % % % % Kern mentioned by name 0.0 28.6 0.0 0.0 10.0 7.7 Judgement/DM 100.0 100.0 88.9 100.0 100.0 97.4 Situational Awareness 100.0 100.0 100.0 100.0 100.0 100 Knowledge: General 28.6 100.0 44.4 50.0 90.0 64.1 Self 28.6 28.6 66.7 16.7 30.0 35.9 Aircraft 71.4 42.9 66.7 83.3 70.0 66.7 Team 0.0 0.0 0.0 0.0 0.0 0.0 Environment: General 0.0 14.3 33.3 0.0 10.0 12.8 Legislative 100.0 71.4 88.9 83.3 80.0 84.6 Organisational 57.1 71.4 77.8 66.7 90.0 74.4 Physical 14.3 28.6 66.7 66.7 40.0 43.6 Risk 14.3 42.9 22.2 33.3 20.0 25.6 Mission: Planning 100.0 14.3 56.0 33.3 20.0 43.6 Knowledge: Application 14.3 42.9 88.9 100.0 30.0 53.8 Skill 71.4 100.0 88.9 66.7 100.0 87.2 Included in Airmanship 57.1 57.1 22.2 33.3 60.0 46.2 Proficiency 28.6 28.6 11.1 16.7 10.0 17.9 Discipline 28.6 42.9 55.6 16.7 30.0 35.9 Discipline + implied 57.1 71.4 77.8 83.3 50.0 66.7 Additional Ebbage and Spencer or Edwards Concepts Attitudes: Hazardous 14.3 14.3 33.3 0.0 0.0 12.8 Professionalism 28.6 14.3 22.2 33.3 40.0 28.2 Self-improvement 28.6 14.3 0.0 16.7 0.0 10.3 Skills: Communication 71.4 100.0 56.0 50.0 40.0 61.5 Crew co-ordination 0.0 42.9 22.2 16.7 20.0 20.5 Interpersonal 0.0 71.4 0.0 0.0 10.0 15.4 Mgt of flight-deck 28.6 85.7 22.2 16.7 30.0 35.9 Self-assessment 0.0 28.6 22.2 33.3 10.0 17.9 Team leadership 0.0 71.4 11.1 0.0 0.0 15.4 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced
The pillars of knowledge were not all mentioned. No one mentioned
‘knowledge of the team’ specifically – although if one considers this as including
crew resource management and crew related issues, then there was some mention
especially among the civilian advanced trainers, but such comments were coded as
Ebbage and Spencer/Edwards additional concepts or among the free nodes. Just over
143
half of the participants overall (53.8%) made comments that showed a concern with
the application of knowledge during flight, at the right time or appropriately. Nearly
all of the civilian and military instructors at BFTS made such comments. Over half
the participants (64.1%) made a general comment about knowledge. In some cases
this was prompted by the question “What other skills, knowledge or attitudes do you
think might be included in airmanship?” which was used to generate further
discussion.
Knowledge of one’s self was represented by comments about knowing and
applying one’s personal limits (CAG, BCB, BMD, MWB), recognising one’s health
and fitness (BCH, CCD) or when one is sick, seeing the medic (BCA). There was
also one reference to the “I’m safe” mnemonic (CAD) which suggests consideration
of illness, medication, stress, alcohol, fatigue and eating habits before going flying.
Knowledge about the aircraft was frequently mentioned, by 66.7% of the
participants overall. Generally this was expressed as abiding by the aircraft speeds
and limitations, the implication being that the student should know what these are
(CAA, BCA, BCE, BMB). There was also comment about knowing the aircraft
systems (BMA, BME, CCB), knowing the manuals (CCC) and general comments
about knowledge of the aircraft (MEA, MEB).
There were not many comments about the environment in general terms, an
example of this sort of comment is “they’re gonna need to be pretty well educated
about the technical aspects of the equipment they’re using and the environment in
which they use it” (MEC). The most mentioned aspect of the environment was the
legislative environment usually specific mention of following rules and regulations.
Overall, 86.4% of the participants noted this aspect, including all of the civilian ab-
initio instructors, who frequently paired comment about knowledge with comment
about abiding by them, or simply talked of abiding by the rules, implying that
therefore these need to be known. For example, “a good broad knowledge of the
rules and regulations is very important obviously to abide by them” (CAB) and
“you’ve got rules, you must abide by them, don’t go breaking them” (CAC). Finally,
one participant took it a little further, saying “not only a knowledge of the rules, but a
desire, you know, that ethical, moral desire to go out and learn the rules, knowing
that there’s likely to be a rule governing this” (MEG).
144
Knowledge of standard operating procedures (SOPs), and in the military, also
of standing orders and rules of engagement, were coded as knowledge of the
organisation, as well as knowing what an organisation expects, for example, “taking
the time to learn how a particular company operates their aircraft in respect of power
settings, fuel flows…” (CAG). Other comments included “knowing your SOP’s”
(CCF), “knowledge of all the procedures, which would definitely include all
emergency procedures” (BCG), “obviously you need to know the rules of
engagement” (BMA), “he conducts the aircraft in accordance with the student air
training guides, limitations and the standing instructions” (MED).
Knowledge concerning the physical environment was not mentioned as much
as the legislative and organisational environments. It was more frequently mentioned
by the instructors at BFTS, both military and civilian, where it was usually expressed
as awareness of airspace, and also geographic awareness, in terms of where would
one land if suddenly faced with an emergency forced landing, and one instructor
added “knowledge of weather” (BCE).
Risk was not mentioned by many participants, and when it was mentioned, the
comments tended to be about the assessment or management of risk, rather than
knowledge of risk. Most such comments were made by the civilian advanced group
of instructors. One example, which spells out the application aspect, is “the
application of practical risk management on the run, rather than reviewing it as a
desk-top exercise” (CCB).
Knowledge of mission included references to planning of flights with rather
more detail than just a comment about preparation. All of the civilian ab-initio
instructors commented on this aspect of knowledge, just over half of the civilian
BFTS instructors but not many of the military instructors or the advanced civilian
instructors. A typical comment is the following:
their general preparation of even their navigation equipment, maps, charts, the obtaining of appropriate briefing for weather and NOTAMS and making sure that those things have been interpreted and assessed conservatively rather than a “well come on lets go” type attitude (CAA).
Skill was specifically addressed, as part of the question: “What other skills,
knowledge of attitudes does airmanship include”. Although not all interviewees took
145
up the discussion of technical flying skill in this context, overall 87.2% of the
participants made some mention of technical flying skill. The line after ‘skill’ in
Table 4.5 indicates whether the participant included flying skill within their concept
of airmanship. There is a clear difference in frequency of inclusion of flying skill
within airmanship: most of the BFTS instructors, both military and civilian,
specifically excluded flying skill from airmanship – only 2 from each group (33%
and 22% respectively) suggested that flying skill was part of airmanship – despite
saying that the components are all interrelated. This relates to the practice at BFTS of
separately assessing technical flying skill, airmanship and preparation (Carrick,
Graham, Healey and Pickard, 2008). The other three groups included technical flying
skill as part of airmanship about half the time. Proficiency was mentioned
infrequently, only by one or two interviewees in each group.
Discipline, the keystone foundation concept for the Kern model, was
mentioned by 35.9% of respondents across all the five groups. Interestingly,
discipline was least frequently mentioned by the military instructors at BFTS and
most frequently mentioned by the civilian BFTS instructors. Examples are “It’s the
discipline to be well educated in your craft and the discipline to apply your
knowledge to keep the operation safe” (BCC) and “there’s a certain level of ethics
required and self-discipline” (MEG). There is an implication that discipline is
required to adhere to rules and procedures, and not cut-corners, and if comments
which mentioned these aspect are included in the count for discipline, then about
66.7% of participants either mentioned discipline or implied its presence, and the
military BFTS instructors are then shown as implying a need for discipline most
frequently.
Hazardous attitudes were raised by some of the civilian BFTS instructors and
one each of the civilian ab-initio and advanced instructors. They did not use the
specific terminology of hazardous attitudes, necessarily, but used similar wording,
for example, rather than “invulnerability” BCA used “it won’t happen to me”.
Professionalism was mentioned most by the military participants, generally as an
adjective: “a professional desire to fly ... in a safe and efficient manner” (BMB) and
“it comes down to a professional attitude for airmanship” (MEB). Self-improvement
was not frequently mentioned, three out of the four mentions were by civilian
instructors, and included reference to keeping up to date with technology (CAE) and
146
continually striving for improvement (CAG, CCD). Self-assessment is a similar
concept, listed in the Ebbage and Spencer/Edwards skill sets, and was mentioned by
one or two instructors in all but the civilian ab-initio group. This was phrased as
“having a good honest level of self-critique” (BMD) or “a good ability to self-
criticise or self-analyse what they do” (CCA).
The civilian advanced level trainers most frequently mentioned the remaining
skill sets: communication, crew coordination, interpersonal skills, management of the
flight-deck and team leadership. These are all skills closely related to the ideas of
crew resource management and highly relevant for multi-crew flight decks, for
which environment this group of participants are training crew. In contrast, the ab-
initio trainers and the military advanced trainers are more concerned with single pilot
operations, and mention of the above concepts was less frequent.
Of these skill sets, communication was most frequently mentioned across the
groups, overall 61.5% of participants raised communication in relation to airmanship,
and 100% of the civilian advanced group did so. The civilian ab-initio instructors
also raised communication frequently, 71.4% included it in discussion. Comments
from the civilian advanced instructors were generally about sharing information
within the flight deck crew, and comment was also made about some reluctance to
communicate openly in the check and training environment:
They feel if they were to talk openly that they might reveal, incriminate themselves, so to speak. Which we normally try to point out to them that we’re sitting in the back with nothing much to do other than watch what they’re doing so there’s not much that they can tell us that we haven’t already seen but they still have a reluctance to do it. I think … it’s a society thing I suppose. Our society, especially men, are not particularly open to talking about their mistakes. (CCF)
Most of the comments from the ab-initio instructors were concerned with
communication with the Tower and other aircraft, and the use of standard
phraseology on the radio. There were also some comments about encouraging
student pilots to speak up if they feel uncomfortable about what is happening in the
air. One of the military advanced instructors related the importance of
communication in terms of rank and hierarchy, noting that this is not generally an
issue but that junior officers should have the communication skills to be able to
express their views (MEB).
147
Additional concepts.
There were many other concepts that were brought up in the interviews that did
not fit within the framework presented by the models of airmanship. Table 4.6
presents the concepts that were relevant to the definition of airmanship and related to
behaviours or qualities.
Table 4.6
Percentage of Participants in Each Group who Mentioned Listed Additional Concepts Relevant to the Definition of Airmanship CAB CAD CBF MBF MAD Total Number of Respondents 7 7 9 6 10 39
% % % % % %
Military definition 28.6 14.3 22.2 83.3 50.0 38.5 Personal qualities 85.7 85.7 100.0 100.0 70.0 87.2 Safety 85.7 57.1 100.0 100.0 50.0 76.9 Preparation 57.1 42.9 88.9 83.3 90.0 74.4 Experience 57.1 57.1 22.2 83.3 90.0 61.5 Attitude 57.1 85.7 33.3 50.0 50.0 53.8 Prioritising 14.3 71.4 44.4 66.7 60.0 51.3 Common sense 42.9 85.7 44.4 33.3 50.0 51.3 Thinking Cognition 14.3 85.7 55.6 16.7 60.0 48.7 Captaincy/Command 42.9 28.6 55.6 66.7 10.0 38.5 Foresight 42.9 28.6 66.7 50.0 0.0 38.3 Courtesy 100.0 28.6 11.1 33.3 10.0 33.3 TEM/error management 28.6 71.4 44.4 16.7 0.0 30.8 Efficiency 28.6 0.0 22.2 83.3 0.0 23.1 C R M 0.0 28.6 0.0 33.3 30.0 17.9 Responsibility 42.9 28.6 0.0 16.7 0.0 15.4 Information gathering 0.0 28.6 0.0 0.0 40.0 15.4 Time management 0.0 0.0 44.4 0.0 10.0 12.8 Human factors 28.6 0.0 22.2 0.0 0.0 10.3 Authority gradient 0.0 42.9 0.0 0.0 10.0 10.3 Automation 0.0 57.1 0.0 0.0 0.0 10.3 Non-Tech Skills 0.0 42.9 0.0 0.0 0.0 7.7 Behaviour 0.0 28.6 0.0 0.0 0.0 5.1 Multi-tasking 0.0 14.3 0.0 0.0 10.0 5.1 Cooperation 0.0 14.3 0.0 0.0 0.0 2.6 Situation management 0.0 0.0 0.0 16.7 0.0 2.6 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced
148
The formal military definition of airmanship is “safe and efficient flight both in
the air and on the ground”, which is more a statement of the outcome of good
airmanship. This definition was quoted across all the groups, but most frequently by
the military BFTS group. Two civilian ab-initio instructors added courtesy, phrasing
it as “safe, efficient and courteous flight” as their initial response (CAA, CAG). One
participant from the civilian advanced group and a few from each of the military
groups did not use this definition in their initial response but brought it up later in the
interview, so the figures in Table 4.6 differ from those in Table 4.4. The definition of
airmanship as being safe and efficient flight was used more frequently by the military
than civilian instructors, as would be expected.
Many of the participants spoke about the personal qualities which they felt
were relevant to airmanship, both in the positive and negative sense. Concepts coded
as personal qualities included honesty, integrity, commitment, diligence, intelligence
and motivation. On the negative side, distractibility, complacency, over-confidence
and fear of failure were mentioned as not conducive to good airmanship. Overall,
87.2% of the participants raised one or more such personal qualities.
Safety was the concept next most frequently mentioned, overall 76.9% of
participants mentioned safety as part of or as an outcome of airmanship. It is
interesting that safety was more frequently mentioned by the ab-initio instructor
groups (86 to 100%), rather than the instructors at the more advanced level (57% and
50%). Possibly safety is more of a concern for instructors and the ab-initio level,
where inexperienced student pilots may be more likely to become unsafe than the
more advanced pilots.
Preparation seemed to be more of concern to the military (including civilian
instructors at BFTS) than to the civilian instructors. Preparation is something that is
assessed at the ab-initio level in military training, explaining its frequent mention.
Generally preparation was raised in reply to the questions about good or poor
examples of airmanship. These responses generally began with a comment about the
level of preparation evident in the student. Civilian instructors talked more about
flight planning which was coded as “Mission”, about which the BFTS instructors did
not talk as much.
149
Experience was included in the initial definition of airmanship by two
advanced level instructors (one civilian, one military) and was mentioned later in the
interview by a further twenty-two participants across all the groups. Nearly all of the
military instructors spoke about experience, and about half of the civilian instructors.
In general, experience was considered to be important in the development of
airmanship and that thus training provides a variety of experiences. Instructors stated
that situation awareness and decision making improve when the pilot has more
experience to help make sense of situations and inform decisions.
Attitude was included in the early probe question “what other skills, knowledge
or attitudes do you think are included in airmanship?” Not all participants talked
about attitude, and some of the comments coded overlapped with personal qualities,
like arrogance (or lack of it) or willingness to take correction. Having a good attitude
or an attitude of trying to be the best that you can possibly be, were mentioned by
civilian instructors (CAE, CCD). Other comments included that good airmanship
involves a professional attitude, a positive attitude and/or general attitude towards
flying.
The idea that prioritising is part of airmanship came up frequently with all
except the civilian ab-initio group of instructors. A commonly used phrase was to list
the terms “aviate, navigate, communicate” as the ordering of priority, with some
adding “administrate” as the final, lowest priority activity.
Common sense was included by instructors across all groups, most frequently
among the civilian advanced level participants. In general common sense was seen as
being responsible, logical and aiming for safe outcomes. It boiled down to acting for
self-preservation with concern for safety. One civilian advanced instructor
summarised common sense as the process of asking “is it safe, is it smart, is it
legal?” (CCG). Based on the speculation suggested in earlier analysis of survey data
that more experienced aviators may be more likely to equate airmanship with
common sense, an additional analysis was conducted to compare the mean
experience (both total and as an instructor) between those who mentioned common
sense and those who did not, combined over groups. Neither comparison was
significant, but there was a small to medium effect size, slightly greater for total
experience. The relevant values are presented in Table 4.7
150
Table 4.7
Comparison of Means of Years of Experience by Mention of Common Sense Mention of
common sense
N Mean (years)
Std Dev
t df p Effect size d
Total experience
No 19 16.158 8.092 1.456 37 154 .4664
Yes 20 20.250 9.375
Instruction experience
No 19 6.332 4.927 1.110 37 274 .3556
Yes 20 8.167 5.369
Comments about thinking, cognition or processing of information were made
by nearly half of the participants. The civilian advanced trainers made the most
mention of thinking. Their emphasis was that airmanship is thinking about
consequences of actions and having a plan ready for whatever might happen. The
military advanced trainers also considered airmanship to be more about thinking and
the thought processes behind decisions than about skill on the controls. It is not
necessarily about making quick decisions every time, but rather recognising that
there may be time to think things through more thoroughly in some circumstances.
To several of the ab-initio instructors, it was important that students think about what
they are doing, and they included this in airmanship. One instructor went as far as to
say that he saw his role mostly as teaching the students to think in certain ways, and
that if he can achieve that then everything else gets sorted out easily (Civilian BFTS,
BCG).
The idea of captaincy or command was raised by some of the ab-initio
instructors, although two of the civilian advanced trainers also spoke about command
abilities. Captaincy is part of the assessment of airmanship at BFTS (Carrick,
Graham, Healey and Pickard, 2008) and tends to cover decision making or the
decision making process, but also includes the integration of the demands of the
exercise with knowledge of rules and regulations and thinking ahead to end up well
positioned for the next stage of the sortie. One military BFTS instructor stated “It’s
not just decision making, again you know, you’re in charge of the aircraft, and whilst
you’re flying you’re making decisions all the time… which may encompass a
situation and what you are going to do” (BMA). The civilian ab-initio instructors had
a similar idea of command, one saying “I need to have people developing command
151
ability and command attitude, which involves significantly their ability to make safe
and prompt decisions” (CAA). There is an obvious overlap here with the outcome
level of Kern’s model of airmanship, the idea of good judgment or decision making,
but it is interesting that is given a different title or label by some participants: rather
than just talking of making decisions, this is “Captaincy” with an obvious initial
capital. Also significant is the idea that this is a skill that is developed early on. Not
all instructors who talked about decision making called it Captaincy, but all
Captaincy discussions involved mention of decision making (and the comments were
also coded as such).
Courtesy was predominantly mentioned by civilian ab-initio instructors. All of
this group mentioned courtesy in one form or another, including, as mentioned
earlier, as part of a formal definition of airmanship, or simply mention of
consideration for other pilots operating in the airspace. A couple of the civilian
advanced instructors mentioned courtesy, one saying that it was a reflection of the
history of the gentlemanly art of flying but really has nothing to do with real
airmanship (CCB) and the other commented that being courteous and polite is way to
ensure good relations and exchange of information with others (CCC). Two of the
military BFTS instructors also mentioned courtesy, as good manners and politeness
in operating among other pilots, and that it is part of airmanship (BMA, BMC).
Similarly, the one participant mentioning this concept from among the civilian BFTS
instructors suggested that manners are part of good airmanship (BCA). The one
military advanced instructor who mentioned courtesy, also commented that it should
not get in the way of getting the job done and said “I guess there’s no nice way to say
yes put those bombs down now, how do you be nice to someone you’re about to
shoot?” (MEB).
Foresight was coded among the free nodes as something rather more than the
part of situation awareness that suggests pilots should be predicting current events
into the future. This node included comments that covered the concept of thinking
ahead to the next part of the sortie or flight, planning what might happen next or
predicting the movements of other aircraft, mentioned independently of definitions of
situational awareness. More of the civilian BFTS instructors, than other instructor
groups, mentioned some aspect of foresight. They emphasised the idea that the
trainee should always be thinking ahead (BCA, BCB), should always be ready and
152
have an ‘escape route’ if things went wrong (BCE). The military BFTS instructors
expressed this as thinking ahead of the aircraft (BMC, BMF). One civilian advanced
instructor spoke about pulling trainees up for not thinking far enough ahead nor
thinking about the ramifications of their actions further along in the flight (CCG).
Foresight was not coded among the responses of the military advanced instructors,
while they did speak about situational awareness and a few mentioned the aspect of
prediction into the future, none made a point of thinking ahead or exhibiting
foresight as part of airmanship.
Threat and error management (TEM) was mentioned as such by civilian ab
initio and civilian advanced instructors. The military trainers also talked about
management of error, but without using the TEM label, and this has been recorded
alongside TEM in Table 4.6 above. The use of the TEM model was introduced into
the ab-initio “Day VFR” syllabus in 2010, and has been part of the regular crew
resource management training in airlines for some time. The concept has clearly been
incorporated into ideas of airmanship among the civilian instructors.
Crew resource management (CRM) was not mentioned by many of the
instructors. It was mentioned by some advanced civilian instructors as may be
expected since it is a required part of airline training. However, only two civilian
advanced instructors mentioned CRM, as such, despite many of them including the
skill sets that make up CRM in the discussion. Five of the military instructors
mentioned CRM. One military BFTS instructor used the term in passing while
discussing situational awareness, saying “it’s a standard crew resource management
type scenario where you are using all your sources to update your model and change
anything if needs be” (BME). Another military BFTS instructor qualified some
comments as “in the training environment”, prompting the researcher to ask about
the real world which led to a discussion of multi-crew flying and the comment
I’m sure you’re aware with the whole crew resource management thing. I think that’s linked to a degree with airmanship as well because again it comes down to the good manners and the ethics of aviating. …I’ve heard of some real classic breakdowns of CRM that lead to, you know, possible breaches of safety for the aircraft (BMC).
One of the military advanced instructors gave a more positive example, saying
that the outcome of decision making is more acceptable if:
153
you’ve taken the time to make a logical, thoughtful decision that you’ve run past the crew, you know, the CRM and all that sort of thing, you don’t always have to have a consensus but you’ve got a general feeling that yes there’s a reason to do this (MEG).
A further two military advanced instructors just commented that CRM is a part
of good airmanship.
Responsible conduct was included in the definition of airmanship by one of the
civilian ab-initio instructors, and mentioned by another as important a bit later in the
interview. A third said that employees have a responsibility “to ensure that a
particular person is trained and mentored properly” (CAG). Two of the civilian
advanced instructors stated that responsibility was part of airmanship, with one of
them relating this to discipline and not cutting corners. One of the military BFTS
instructors mentioned responsible conduct among all aircrew, not just pilots.
Information gathering was mentioned specifically only by instructors from the
military and civilian advanced training groups. Two of the civilian trainers
mentioned collecting information preparatory to making a decision, and four of the
advanced military instructors included the taking in and assimilating of information
in a reasonable time-frame as part of their definition of airmanship, and separately
from the general idea of perception and recognition of input that is part of situational
awareness.
There are a number of concepts that were mentioned only by a few civilian
advanced instructors towards the end of Table 4.6. These are authority gradient,
automation, non-technical skills, behaviour (that is, specifically identifying
airmanship as a form of behaviour), multi-tasking and co-operation. These are all
skills or concepts that are more relevant in multi-crew environments and in
commercial aircraft, or are talked about more in that context, so it is unsurprising that
these were only mentioned by instructors in that environment.
Mention of human factors was made by the civilian ab-initio and civilian BFTS
instructors only, as being relevant to or equivalent to airmanship. Human
performance and limitations has been part of the Day VFR syllabus since 1993, so
the civilian instructors should be well aware of the importance of this aspect of
knowledge.
154
External considerations.
In discussion of the nature of airmanship, a number of activities, actions and
considerations were mentioned by participants that would need to be executed or
taken into account in demonstrating good airmanship. These are shown in Table 4.8.
Table 4.8
Percentage of Participants in Each Group who Mentioned Listed Actions and Considerations Relevant to the Definition of Airmanship
CAB CAD CBF MBF MAD Total
Number of Respondents 7 7 9 6 10 39
% % % % % %
Checks 85.7 57.1 100.0 100.0 30.0 71.8 Lookout 100.0 14.3 100.0 100.0 30.0 66.7 Radio 85.7 57.1 100.0 66.7 30.0 66.7 Weather 71.0 71.0 67.0 17.0 40.0 53.8 Pre-flight Inspection 100.0 0.0 55.6 50.0 30.0 46.2 Go-round criteria 28.6 14.3 66.7 66.7 30.0 41.0 Care for aircraft 85.7 14.3 44.4 50.0 10.0 38.5 Admin paperwork 42.9 0.0 44.4 16.7 40.0 30.8 Traffic 28.6 0.0 44.0 83.0 10.0 30.8 Fuel 28.6 57.0 22.2 16.7 20.0 28.2 Pressure 14.3 57.1 33.0 17.0 20.0 28.2 Taxi 42.9 0.0 44.4 50.0 0.0 25.6 Work cycles 0.0 14.3 55.6 66.7 0.0 25.6 Passengers 28.6 0.0 22.2 0.0 0.0 10.3 Hand over 14.3 0.0 11.1 0.0 0.0 5.1 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced
Carrying out checks and checklists was more relevant at the ab-initio level, as
were lookout and use of and listening to the radio. These are concerns that need to be
actively taught at the ab-initio level and perhaps are assumed at the more advanced
levels of training, and this is reflected in the frequency of mention in regard to
airmanship. In fact, one civilian BFTS instructor initially defined airmanship simply
as doing checks and keeping a good lookout.
Weather, traffic and fuel were considerations raised by many of the
participants, though not evenly across the groups of instructors, in their descriptions
155
of good or poor examples of airmanship. The monitoring of fuel was raised by a few
participants in each group. Comments included “not running out of fuel” (BMA,
CAB) as part of a list of good airmanship practices, managing fuel (BCB, BCG),
being aware of remaining fuel (CAA, MWB, MWC) and balancing fuel load against
expected weather (CCB and CCG).
The next most frequently mentioned aspect of aviation mentioned in discussion
of airmanship was the carrying out of pre-flight inspections, that is, the checking of
oil and fuel and general serviceability of the aircraft before flight. Civilian ab-initio
instructors all mentioned this aspect of airmanship. The civilian advanced trainers
did not mention this at all, and this may be because they were simulator trainers, so
pre-flight checks would not be relevant in that situation. About half of the instructors
at BFTS mentioned this, and fewer than a third of the military advanced instructors.
‘Go-round criteria’ was listed among the free nodes as part of airmanship.
These are the criteria by which the pilot decides whether it is save to land, given how
well (or how badly) the approach has gone. There is a formal set of criteria used in
military instruction, and thus the compliance with these criteria was far more
frequently mentioned by the BFTS instructors.
Caring for the aircraft was an interesting inclusion in airmanship,
predominantly among the civilian ab-initio instructors, but cropping up fairly
frequently among the BFTS instructors as well, more-so than among the trainers at
the more advanced level. This included things like not starting the aircraft on gravel
(which damages the propeller), not taxi-ing over boggy ground, looking after the
aircraft generally, and being aware of temperatures and pressures and other
limitations of the aircraft and engine. The consequences of not caring for the aircraft
may be expensive repairs or more frequent maintenance, which can eat into the profit
of a flying school or small company so the instructors may be instilling a commercial
consideration at an early stage of training. Severe overstressing of the aircraft in
flight and ignoring extreme engine indicators can lead to airframe or engine failure,
with possibly fatal consequences, so there is certainly a safety aspect to this concern.
Administration and paperwork were regarded and mentioned as part of
airmanship by a few pilots in each group, except the civilian advanced group. These
156
comments referred to the necessary paperwork that needs to be done at the end of a
flight or sortie: the filling in of logs, maintenance reports and student records.
Taxi speed and considerations whilst taxiing were mentioned by some
participants in all the ab-initio groups, but not the military and civilian advanced
instructors. This may be because it is something that needs to be actively taught at
the ab-initio level, and assumed at the more advanced levels.
‘Work cycles’ is a term used by the military to refer to the structure of
practices in the cockpit, thus most mentions were from civilian and military BFTS
instructors. There was one additional mention from a civilian advanced instructor,
who happened to be ex-military. He stated that “flying tends to be made up of
routines and perhaps work flows, patterns, cycles of activity” (CCB) and the
suggestion that this assists with cognitive capacity.
Time pressure and commercial pressure were mentioned by four of the civilian
advanced instructors as something that has to be coped with while on the job, and
thus part of one’s airmanship. One ab-initio instructor spoke about some students
putting pressure on themselves to get things done, whereas he felt that simply putting
off the flight rather than rushing makes more sense (CAG). There was also mention
of stress during flight tests for ab-initio students.
The passenger brief was mentioned by two of the civilian ab-initio instructors,
as being something that should be carried out, and thus part of good airmanship
(CAA, CAB). The two mentions by the civilian BFTS instructors differed, one
including the safety of passengers as part of the definition of airmanship (BCG), the
other suggesting that thinking ahead and flying smoothly gives a more enjoyable
flight for the passengers (BCE). The “handing-over, taking over” exchange was
mentioned as one of the first airmanship lessons in the context of always being aware
of who has control of the aircraft with the formal exchange taking place in the
instructional environment (CAC, BCF).
Discussion - Definition of Airmanship
Similarities and Differences among Groups
There was agreement between the different groups of instructors about the
capstone outcomes of airmanship: situational awareness and decision making. All the
157
instructors included these in their discussion of airmanship. Of these 33 included
these concepts in their discussion of the definition of airmanship. The remaining six
(15%) included them after being prompted by a later question. There was also
general agreement that airmanship involves the knowledge and application of the
rules and regulations that form the legislative environment and the operating
procedures that are part of the organisational environment. Personal qualities were
brought up by many of the participants, across all groups. While the specific qualities
mentioned varied, it is interesting that so many felt that airmanship also included
aspects of the individual’s personality. There was also consistency across the groups
in the lack of specific mention of ‘knowledge of the team’ as such.
Differences between the groups could not be tested statistically, due to the low
numbers of participants in each group. However, the frequency of comments can be
contrasted informally by looking at which groups mentioned particular concepts
most often.
Instructors from the civilian ab-initio group made more mentions of mission
planning, self-improvement, and courtesy than did instructors from other groups.
They made next to highest mention of communication (after the civilian advanced
group) and safety (after the BFTS instructors). They made fewer mentions of the
physical environment, organisational procedures, knowledge of risk and the
application of knowledge than the other groups of instructors. Working in the volatile
GA environment preparing student pilots for independent flying may have influenced
the emphasis on adequate flight planning and the concern with communication and
courtesy in dealing with others.
The civilian advanced instructors made more mentions of multi-crew related
concepts, as these relate closely to their activities training within multi-crew
operations. This group made most mentions of communication, crew coordination,
interpersonal skills, management of the flight deck, team leadership and threat and
error management, and the only mentions of authority gradient, automation
management, non-technical skills, behaviour, multi-tasking and cooperation. They
also made most comments about thinking or cognition, attitudes and common sense.
They made least mention of aircraft knowledge and mission planning, and were next
to lowest on mention of knowledge of the physical environment. These later points
may be because of the simulator environment in which this group works.
158
The civilian instructors, both ab-initio and advanced, included mention of
proficiency, communication and responsibility more frequently than the other
instructors: and they were in the middle grouping in mentions of experience.
Civilian instructors at BFTS made the least mention of the role of experience
within airmanship, but made the most comments about knowledge of self, foresight,
time management and hazardous attitudes. They mentioned thinking about as often
as the military advanced instructors, less often than the civilian advanced group but
more than the civilian ab-initio and military BFTS groups. The military BFTS
instructors made the most mention of efficiency and the only mention of situation
management. Together the BFTS instructors made most mention of safety,
knowledge of the physical environment, captaincy and the application of knowledge,
and with the civilian advanced group, made most mention of discipline or implied a
disciplined approach. They did not tend to include flying skill in airmanship. The two
BFTS instructor groups, with the military advanced group, made most comment
about preparation. The role within BFTS of these instructors would have influenced
their comments about Captaincy, emphasis on preparation and separation of flying
skill from airmanship as these are marking criteria for this organisation. Other
concepts which they mentioned most are aspects that perhaps are emphasised more
in the military training environment than elsewhere. Given the contrast between the
initial statements of BFTS instructors that airmanship is an overarching concept and
that the three areas that are assessed are intertwined, it might be appropriate to re-
label what is currently assessed as airmanship, as “non-technical skills” and abstract
“airmanship” to represent the integration of the three assessable areas in terms of the
overall safe and efficient conduct of the flight.
The military advanced instructors mentioned information gathering more than
other instructors, and with the military BFTS instructors made most mention of
experience playing a part in airmanship. The two military groups, along with the
civilian advanced group, commented most on prioritising. The backgrounds of these
instructors either in combat or military service roles may have ensured prompt
processing of data and appreciation of the role of experience. Operational activity
would have led to the understanding of the importance of prioritizing tasks on a high
demand flight-deck, something also raised by the advanced civilian instructors,
veterans of busy airline operations.
159
Comparison with Survey Results
The relative importance (in terms of frequency of mention) of the concepts that
are part of airmanship raised in the interviews is very similar to the outcome of the
survey reported in the previous chapter. Table 4.8 compares the results combined
across groups, with the outcome from the on-line survey.
The interviews led to a higher percentage mention on most of the concepts than
did the survey, due to the different method with the interview allowing more probe
questions and questions about specific aspects of airmanship than did the survey, and
the participants in the interviews talking much more than the survey respondents
were willing to write down, so that there were more areas covered in the interviews.
The consistency across the two data collection formats and sets of questions
used indicates the robust nature of the idea of airmanship among Australian aviators.
Nineteen of the top 25 concepts are the same, although the ordering of concepts is
not exactly the same, it is clear that situational awareness, decision making and
safety are the central concepts in the formulation of airmanship. Personal qualities
are high on the lists, which may reflect a concern for the kind of person an aviator is.
It should also be noted that there was not much consistency among the qualities that
were coded, just that there tended to be a feeling that some qualities promoted good
airmanship and less positive qualities detracted from it.
Knowledge concepts, especially of rules and regulations (legislative),
procedures (organisational) and of the aircraft, were all high on the lists.
Communication came in eleventh on each list, just ahead of experience, with
common sense not far behind. Of the six concepts that were in the top 25 from the
interview data but not from the survey, three did appear further down the list in the
survey data – prioritising, thinking and foresight. Captaincy, preparation, and attitude
did not appear in the survey data, although hazardous attitudes did. The six survey
concepts that were not in the interview top 25 concepts, did all appear lower in the
list from the interview results: knowledge of risk, crew coordination, proficiency,
professionalism, hazardous attitudes and general comments about the environment.
160
Table 4.9
Concepts Constituting Airmanship Ordered by Percentage of Participants who Mentioned Each, from the Interview Data and from the Online Survey (top 25) Interview data N=39
Survey data N=83
% % Situational Awareness 100.0
Safety 74.7
Judgement/Decisions 97.4
Situational Awareness 59.0 Personal qualities 87.2
Judgement/Decisions 55.4
Knowledge Legislative 84.6
Knowledge Aircraft 47.0 Safety 76.9
Personal qualities 47.0
Knowledge Organisational 74.4
Knowledge Legislative 44.6 Preparation 74.4
Knowledge Organisational 41.0
Discipline 66.7
Skill (technical) 39.8 Knowledge Aircraft 66.7
Efficiency/effective 36.1
Knowledge – general 64.1
Knowledge: Risk 33.7 Communication 61.5
Communication 33.7
Experience 61.5
Experience 30.1 Attitude 53.8
Crew co-ordination 26.5
Common sense 51.3
Knowledge - general 25.3 Prioritising 51.3
Common sense 25.3
Thinking Cognition 48.7
Courtesy 25.3 Skill (technical, included) 46.2
Knowledge: Self 24.1
Knowledge Physical Env 43.6
Discipline 19.3 Knowledge Mission 43.6
Knowledge Phys Env 18.1
Captaincy/Command 38.5
Knowledge Env General 15.7 Safe and efficient 38.5
Proficiency 15.7
Foresight 35.9
Professionalism 15.7 Knowledge Self 35.9
Knowledge Mission: 14.5
Management of flight-deck 35.9
Hazardous Attitudes 14.5 Courtesy 33.3
Management of flight-deck 14.5
Note. Env = environment; Phys = physical.
The ubiquitous appearance of common sense is interesting. It is not something
that is included in either model of airmanship, nor perhaps should it be. “Common
sense” seems to be a short cut way to express using one’s acquired knowledge
sensibly and appropriately and not taking unnecessary risks. There is also an
implication that this knowledge has been so well integrated that it is what the
interviewee sees as something any normal aviator would abide by. That there is an
near medium effect size in relation with experience supports the idea that the more
experienced aviators are more likely to speak about common sense, as they have had
more time to develop that integration of knowledge and practical application.
161
Comparison with the Models of Airmanship
The analysis of the interviews is consistent with the basic structure that the
Kern model indicates. However, the results also draw attention to the areas that were
not at the forefront of that model – the aspect of operations in multi-crew
environments, and support the Ebbage and Spencer and Edwards models which add
emphasis in this area. Also, it was evident that simply listing areas in which
knowledge is required is not sufficient in the active training environment, where
there was emphasis on the application of knowledge, the ability to know when and
how to use acquired knowledge. Discipline was viewed rather as compliance with
rules, regulations and procedures. There is a contrast here with the survey data
reported in the previous chapter, where there were several comments that rules had to
be applied with a certain amount of common sense – the implication being that there
are occasions in which breaking a rule may be the best course of action.
Communication was again brought up in all areas, as part of good airmanship, and
probably requires more conspicuous placement in models of airmanship.
162
Results - Training and Assessment of Airmanship
The interview schedule included some specific questions about the place of
airmanship in the training programs at the various organisations from which
instructors were interviewed. These included questions about inclusion of airmanship
in the pre-flight briefing as well as the aspects of airmanship included in the initial
flight lessons (or sorties). The participants were also asked about the assessment of
airmanship during training in their organisations. There was also a lot of free range
discussion of training generally, including topics such as post-flight debrief, dealing
with students, handling of student errors and the training of instructors.
Airmanship in the Training Program Structure
There were a number of consistent types of comment made about the inclusion
of airmanship within a general structure of training programs at the relevant training
levels. The comments included that at advanced training levels, airmanship is
assumed to be adequate. The inclusion (or not) of airmanship in the pre-flight brief
and post-flight debrief was discussed and some participants commented on the
learning rate expected in ab-initio training. The combination of the idea that student
pilots will learn airmanship from the behaviour of others and the requirement for
instructors to provide a good example or airmanship, as well as the influence of the
organisational culture in which the training occurs were also raised. There were some
related comments on the progressive introduction of new content within the program
and the instructors reported observation of the development of airmanship over time.
Finally, there was some discussion of curriculum issues associated with training
programs, student record keeping and the relevance of ground school instruction.
Table 4.10 summarises the distribution of such comments across instructor groups. In
the discussion that follows, participants are identified by a three letter code (e.g.
CAA).
163
Table 4.10
Percentage of Participants who Commented on Aspects of the Training Program
CAB CAD CBF MBF MAD Total
Number of Participants 7 7 9 6 10 39
Comments % % % % % %
Assumed at this level 14.3 42.9 0.0 0.0 60.0 25.6 Pre-flight brief/talk 100.0 14.3 100.0 100.0 70.0 76.9 Post debrief 42.9 42.9 66.7 50.0 80.0 59.0 Learning rate 14.3 0.0 22.2 33.3 0.0 12.8 Learn from others 14.3 57.1 11.1 16.7 10.0 20.5 Provide an example 14.3 57.1 33.3 50.0 40.0 38.5 Organisational culture 14.3 71.0 0.0 33.3 40.0 41.0 Learn+example+culture 42.8 85.7 33.3 66.7 50.0 53.8 Progression 0.0 14.3 55.5 83.3 40.0 38.5 Development with time 0.0 42.9 22.2 83.3 30.0 33.3 Curriculum structures 0.0 0.0 66.7 16.7 0.0 23.1 Student records 14.3 0.0 44.4 66.7 0.0 23.1 Ground school 0.0 0.0 44.4 66.7 10.0 20.5 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced
Airmanship assumed. When airmanship training began to be addressed with
the participants, there was an interesting pattern of response among the participants.
Many of the advanced trainers (52.9%), especially in the military, made comments
about the experience that had been acquired by their students by this time in their
careers and that adequate airmanship is assumed at their level of training. Three of
the advanced civilian instructors noted that by the time pilots got to fly the sort of
aircraft they instructed on (large public transport aircraft) they have reasonable
airmanship (CCA), are assumed to have it (CCB) or are all good pilots (CCF). The
military advanced instructors made the point that the pilots they are dealing with
have all had squadron experience (MEG), “have above average airmanship” (MEC,
MED) and “there is nothing new to teach them about airmanship” (MEA). The one
ab-initio instructor who made this type of comment was referring to a higher level of
training, indicating that airmanship is not briefed or taught as much at that higher
level (CAB).
164
Pre-flight brief. The ab-initio trainers and Williamtown advanced military
trainers were asked if airmanship was included in the pre-flight briefs, the military
advanced trainers at East Sale were asked if they talked to the flight instructor
candidates (FICs) about airmanship, however the advanced civilian instructors were
not all asked an equivalent question.
All the civilian ab-initio instructors noted that airmanship issues were covered
in the pre-flight briefing from the first flight. Briefings were standardised within the
organisations and there would be four to seven airmanship points at the end of each
brief. The airmanship content of the briefs would depend on what was being covered
in the flight, for example safety considerations, situational awareness, weather
effects, the rules and regulations relating to the specific flight (such as right of way
entering the circuit), clearing turns and checklists.
The military and civilian BFTS instructors were consistent in all saying that
airmanship is always covered in their briefs. The pre-flight briefs are standardised
starting with the aim of the sortie, then the airmanship points relevant to the sortie
and finally any additional considerations (BMB, BMC, BMD, BME). These
participants noted that the airmanship points vary according to the sortie and how far
through the training program the student has progressed (BCA, BCD, BCI, BMA).
They stated that lookout and situational awareness are included throughout the
program (BCB, BCC, BCG, BCH, BMF) and other topics initially might include
safety, pre-flight checks and ten-minute checks (BCA, BCE). The “considerations”
are things that are not so vitally important to the exercise but would be good to see
demonstrated. Items in the “considerations” section in the early part of the flying
program move up into the “airmanship” section in later sorties, although not
everything is listed every time (BCF, BMD). One participant noted that the students
also had mass briefs in the classroom about the flying program before fronting for
their individual briefs with their instructors (BME).
Among the civilian advanced instructors, one participant responded to the
question about pre-flight briefs including mention of airmanship with a firm “no”,
and the comment that “if they are still in the system their airmanship is not too bad”
(CCA). Another participant in this group indicated that their organisation provides a
list of discussion points to be covered with each trainee and that this list includes
many points that cover airmanship, including “check the big picture”, as well as
165
technical knowledge (CCG). There may be a problem with terminology, as one
civilian advanced instructor noted, “for example on a check assessment form you’re
unlikely to see the heading airmanship. These days you’re more likely to see the term
non-technical skills” (CCB) or non-technical markers (CCE) that cover the sorts of
behaviours that were included in the definition of airmanship. The pre and post flight
briefs are more likely to address non-technical behaviours, rather than use the term
airmanship in the civilian advanced environment. The implication then, is that
technical skills are not considered part of airmanship.
The responses of the advanced military instructors at Williamtown, who teach
conversion to jet fighters, were consistent with the BFTS instructors. Their briefs
follow the same ‘aim, airmanship and considerations’ framework and it was also
noted that there are mass briefs as well as individual briefs (MWB, MWC).
The advanced military instructors at East Sale, who train future instructors,
were asked if they discussed airmanship with the flight instructor candidates (FICs).
The responses varied considerably, from “No, they should already have good
airmanship” (MED) via “it gets discussed with an instructional flavour” (MEC) to
“Yes, that is pretty much all I talk about” (MEF). One participant said that he gives
two briefs for each sortie, one directed at the hypothetical student (which he later role
plays) and one for the FIC, emphasising how to maintain their own airmanship and
situational awareness while delivering the training package (MEC). Given how
standardised the military is, this may be the general rule but the interview question
did not reliably elicit this information. The remaining four participants said that they
talk about airmanship in a general way but do not spend much time on it. One
participant noted that there is a series of lectures for the FICs about how to teach and
assess, but while it touched on airmanship it is mostly about the technical aspects of
flying. The situation was summarised by the statement:
I think we rather rely on the airmanship that they’ve demonstrated to the point that has got them here as reasonably good pilots, and we hope that implicitly they realise that the behaviour that has got them here is the behaviour that they’re now going to try and encourage of their students when they get where they’re going. So I don’t think we formalise too much instruction of airmanship. (MEC)
This situation is confirmed by some of the comments made by the military
BFTS instructors, when talking about their training at East Sale. They reported that
166
most of instruction was on how to get across the technical skills, the “hands and feet”
skills, and that airmanship was considered to be about providing the right example
and enforcing that (BMB, BMD, BMF).
Post-flight debrief. The post-flight debrief was not consistently mentioned by
the ab-initio groups of instructors, but there was no specific question in the interview
schedule that addressed the post-flight de-brief. In several cases, such as among the
advanced military East Sale group and advanced civilian instructors, in saying that
airmanship was not covered in pre-flight briefs, it was added that any deficiencies
were addressed post-flight. In the military environment, there is a required post-flight
debrief in which the student’s performance is appraised and indication given to
which areas need further work (MEA, MEE, MEF). A post-sortie debrief is also
expected in normal flying activities in the military, a self-debrief if necessary, or a
debrief with other pilots who had taken part in the flight (MEB, MEC). In the
training environment, rather more of the military advanced and military and civilian
BFTS instructors spoke about the debrief than did the civilian ab-initio and civilian
advanced instructors. Among these civilian instructors who mentioned the debrief,
comments indicated that the whole flight was debriefed and that would include
airmanship behaviours (CAB). There was comment from the civilian advanced
instructors that if the crew can come up with their own explanation of what might
have gone wrong, or not exactly as planned, then they are happier with the outcome
(CCA, CCF). In contrast, the ab-initio instructors phrased their comments to indicate
that they provide the feedback for the student, rather than facilitate the student’s own
self-analysis (CAA, CAC).
Training rate. The military ab-initio course at BFTS is a set length and
students are expected to keep up with the training program, with some allowance for
remedial training if required. There were a few comments that this can be a challenge
for the instructor as well as the student (BCA, BMB). One of the civilian ab-initio
instructors mentioned the challenge of the opposite situation, with private students
coming in for lessons infrequently making it difficult to reinforce desirable
behaviours, in contrast to the culture change that is achievable in a formal intense
course (CAA).
Learning from others and providing an example. Overall 20% of the
participants talked about their own or their students’ airmanship developing by
167
learning from others (CAD, BCF), watching what others do in the same organisation
(CCD, CCG) or from reading crash comics (reports of aircraft accidents) (CCA,
CCB), with most comment coming from the civilian advanced group. A parallel
concern is with providing a good example for students, and others, to learn from,
brought up by 38.5% of participants overall. This appears to be a central assumption
in the military instructor training process, as discussed above. The civilian advanced
instructors made the most mentions of providing a good example or having been
provided with a good example early in a career (CCB, CCD).
Organisational culture. Comments about good examples were also reflected
in the mentions of the importance of the promotion of a positive organisational
culture. Participants discussed the idea of a no-blame approach and concern with
why a mistake or error happened, paired with non-acceptance of deliberate violation
of rules (CCD, MED), generally known as a just culture, although that terminology
was not used. The lack of authority gradient, allowing the crew to speak up, was
given as another example of a positive culture (CCE, CCF). There was criticism of
cultures that normalise deviance or threat, and that hide rather than admit to
problems (CAG, CCA, CCB, BMB). This was related back to the training
environment being required to present this sort of culture in the first place (CAG,
BMC, MEE), and to reinforce it consistently across the different areas of the
organisation, different squadrons in the military (MEF) and different fleets in airlines
(CCD).
Learning from others, providing a good example and organisational culture are
related in that all are comments about the way in which airmanship is inculcated into
student pilots. Combining over the three concepts, without counting any participant
twice, results in an indication that 53.8% of the participants mentioned one or more
of these three concepts, and the civilian advanced group had the highest mention rate
(85.7%).
Progression. The BFTS instructors made the point that the flying training
program is a staged process (BCF, BMA, BMC, BMF), and that there is a graduation
to allow experiences to build up and ease transition to new tasks (BCB). There was
also comment made that after a while, the student is expected to take responsibility
for the pre-flight aircraft checks and start-up, but now and again the instructor will
check that it is being done correctly (BCA, BME). One of the civilian advanced
168
instructors made a similar comment, that the training is staged so that everything is
not happening at once and to allow the student to get the feel of the aircraft (CCA).
Only one of the advanced military instructors made a comment along these lines,
noting the gradual introduction and exposure to situations with a demonstration of
the appropriate response (MEC), and he may have been referring to the BFTS
process. Another advanced military instructor noted that as the FIC course progresses
there is a change from attention on the FIC’s airmanship to the impact of instructor’s
actions on the airmanship of the (future) student (MEE). Other military advanced
instructors commented that they try to extend the FIC by loading him up with tasks
(MEB, MED) as the course develops. The civilian ab-initio instructors did not
comment on the way in which the early lessons progress, but the advice from the
regulator sets out a progression of lessons (CASA, 2006).
Development. Some of the civilian advanced instructors talked about how they
may see pilots who work in the organisation infrequently, and notice how they have
developed as their career has progressed, in responsibility, decision making, or
simply being better at the job (CCD, CCE). The BFTS instructors see the same
development on a shorter time scale as the trainees move through basic flight
training, and commented on the instructors” expectations for the trainee to exhibit
increasing levels of airmanship (BCC, BMA, BME). The advanced military
instructors made much the same comments about basic flight training (MEA) and
across a full career (MEC).
Curriculum structures. The instructors in the military system made some
comments about the curriculum structures in place to ensure consistency. The
curriculum itself is determined by the Central Flying School at East Sale and this
provides a structure, syllabus and guidance material (BCB, BCH). There is
recognition that different instructors do things a little differently with regard to
airmanship (BCC, BCG) and that can cause difficulties for students who move
between instructors (BME). However, there are meetings to standardise the approach
to assessment and to exchange ideas (BCG, BCH, BCI). One participant noted that
with increases in workload, it is not as easy as it once was to get together and
compare notes (BCF).
Student records. The majority of comments about student records came from
the BFTS instructors, because the BFTS has a formal system of reporting on each
169
sortie. The one other comment came from a civilian ab-initio instructor who
expressed the desire to see much more formal record of student progress, “not only
talking about the, the tolerances and standards of the flight but also the elements of
airmanship that we observed” (CAA).
At BFTS, the student record is written into a structured document including
tick boxes for the aspects of flight covered in that lesson and a space for written
narrative (BCC). The written commentary is based on the post-flight debrief of the
student and tends to be a summary of the flaws observed in flight and the required
corrective action (BCA, BCD). It was also noted that the emphasis tends to be on
technical skills, although airmanship errors are also reported (BCC). Several of the
military BFTS instructors indicated their belief that the report is not primarily for the
student, but rather to inform another instructor, should one have to take over the
student (BMB, BME, BMF). One of the civilian BFTS instructors went as far as to
indicate that the student record is not there to give positive feedback to the student, it
is to provide documentation to enable removal of poorly performing students from
the training program (BCF). One of the military BFTS instructors reported that the
training undertaken to become a QFI did not include formal instruction on how to
write the reports, but rather feedback on practice write-ups (BMD). A typical
comment is
[The student record] is not there to give the student good feelings about himself, it’s there to pick up on every mistake that he makes so that the next instructor can try and correct those. The unfortunate reality is now we write them with the view that the student will read them and as such, you know, they become a little less, well a little bit less harsh I guess (BME).
Ground school. Ground school aspects of flying training programs were only
discussed by some of the BFTS participants and one military advanced participant.
Essentially the comments indicated that the students at BFTS have a fairly intensive
ground theory component, in which they are lectured in airmanship, medical aspects
of flying, limitations of the aircraft, rules and regulations and begin to learn their
basic techniques, work cycles and checks (BCB, BCH, BMB, BMC). They also
attend mass briefings which introduce each sortie and form part of their preparation
for flying (BCA, BCC, BMA, BME). Similarly, in more advanced training such as at
Williamtown, the students have tutorials, an air training guide and mass briefs, as
well as the individual briefs, before sorties are flown (MWB).
170
Initial Training of Airmanship
The participants who work with ab-initio student pilots were asked what they
started with in terms of including airmanship in initial flight training. Table 4.11 lists
the kinds of things that were mentioned across the three ab-initio instructor groups,
loosely ordered to reflect Kern’s model of airmanship.
After noting that airmanship is introduced from the first pre-flight briefs, the
different ab-initio instructor groups came up with a variety of different aspects of
airmanship that are introduced in the first few lessons, but most noticeably they all
talked about lookout being the most important initial airmanship related skill to be
taught. Two civilian ab-initio instructors specifically mentioned teaching the clock
code for identifying where other aircraft are located in the sky (CAB, CAE). Now
that the use of digital watches and clocks is very widespread, this may not be so
intuitive a method as it once was.
Weather awareness was noted by some of the instructors as something that they
drew attention to or required students to have found out about before a lesson (CAB,
CAG, BCC, BCG, BCH). Interestingly, only one instructor (CAB) indicated that the
flying school he was associated with had set up a procedure for getting weather and
doing pre-flight planning and fuel calculations so that the habit of doing so is
established early on. Five BFTS instructors mentioned that they expected students to
arrive prepared for the sortie, which may include such pre-planning, as well as being
aware of the activities they would be conducting during the session (BCC, BCE,
BCF, BCH, BMF).
In addition to unanimous mention of lookout, listen-out to the radio and
making appropriate radio calls was included by more than half the instructors (54.5%
overall) in initial lessons. Listening is associated with lookout in that the combination
allows the development of good situational awareness. Also in relation to developing
situational awareness, six of the BFTS instructors spoke about getting their students
to think ahead of the aircraft and about what would be happening next, from the early
stages of training (BCC, BCE, BCF, BMA, BMC, BMF) and two noted a concern
with ensuring fuel awareness on the part of the student (BMA, BMF).
171
Table 4.11
Percentage of Participants in Each Group and Overall who Mentioned Inclusion of Listed Airmanship Items in Initial Flight Training
Civilian Ab-initio
Civilian BFTS
Military BFTS Total
N 7 9 6 22 Item % % % % Pre-sortie Weather 28.6 33.3 0.0 22.7 Flight planning 14.3 0.0 0.0 4.5 Preparation 0.0 44.4 16.7 22.7 Situational Awareness Lookout 100.0 100.0 100.0 100.0 Listen-out and radio use 42.9 55.6 66.7 54.5 Thinking ahead 0.0 33.3 50.0 27.3 Fuel awareness 0.0 0.0 33.3 9.1 Judgement/Decisions Prioritising 0.0 11.1 33.3 13.6 Decisions/ Captaincy 0.0 0.0 33.3 9.1 Knowledge How to apply knowledge 0.0 11.1 50.0 18.2 Safety 14.3 0.0 0.0 4.5 Knowledge of rules 14.3 0.0 33.3 13.6 Aircraft limits/parameters 28.6 22.2 33.3 27.3 10 minute checks 0.0 33.3 83.3 36.4 Checks/checklists 42.9 55.6 83.3 59.1 Go round criteria 0.0 11.1 50.0 18.2 Skills Start-up 14.3 22.2 0.0 13.6 Taxi 42.9 33.3 33.3 36.4 Hand-over take-over 14.3 0.0 0.0 4.5 Smooth control 14.3 0.0 0.0 4.5 Procedures 14.3 0.0 0.0 4.5 Work cycles 0.0 33.3 50.0 27.3 Correct altitude 0.0 0.0 33.3 9.1 Emergency procedures 0.0 0.0 33.3 9.1 Discipline Pre-flight Inspection 14.3 0.0 16.7 9.1 Paperwork 0.0 0.0 16.7 4.5
A couple of the military BFTS instructors mentioned the expectation that
students would begin to exhibit decision making and exercising captaincy, a bit later
in the sequence of early flights (BMA, BMF). There was also an expectancy that
172
students would begin to start prioritising tasks, especially in busy phases of flight
(BCE, BMC, BMF).
Half of the military BFTS instructors, and one of the civilian BFTS instructors,
made a point of their role being to teach the students how to apply their knowledge in
the air. The ground school component of the military training program presents what
the students need to know, but recognising how to use that information requires some
coaching (BCC, BMC, BME, BMF)
One of the civilian ab-initio instructors noted that one of the first things that he
does with new students is ensure that they understand safety aspects of being near
aircraft on the ground (CAF). Several instructors expected students to know about
the rules and regulations, and standing instructions in the military case, appropriate
to the sortie (CAC, BMC, BMD). There was also mention of knowledge of the
aircraft limitations and parameters, for example the appropriate speeds, temperatures
and pressures against which to compare operational readings (CAC, CAE, BCF,
BCH, BMB, BMD).
Knowing and carrying out 10 minute checks were included by many of the
BFTS instructors. These checks are to ensure that the aircraft systems are operating
correctly and are inculcated early on. Knowing their checks and appropriate use of
checklists was also frequently mentioned across all three groups, overall by 59.1% of
the instructors. Four of the BFTS instructors spoke about the students having to
know and apply the go-round criteria specified for student flights (BCF, BMA,
BMD, BME).
In terms of flying skills that were included in the listing of airmanship points,
three instructors mentioned aspects of the aircraft start-up (CAE, BCF, BCG) and
eight included taxiing, generally in terms of appropriate speed and separation from
other aircraft. In the air, the ritual of the formal “handing-over – taking-over” routine
and knowing who is in control of the aircraft was mentioned by just one of the
civilian instructors (CAC). Another civilian instructor made the point of wanting to
see the student develop a relaxed and smooth handling of the aircraft (CAA). One
civilian instructor talked about the student learning their procedures (CAC), while six
of the BFTS instructors spoke about the student following their work cycles (BCA,
BCC, BCI, BMC, BME, BMF), these referring to correctly carrying out specific in-
173
air aircraft handling sequences. Flying at the correct altitude and being ready for
emergency procedures as they were introduced were both mentioned by two military
BFTS instructors (BMD, BMF).
There was mention of the requirement for carrying out a pre-flight inspection
(CAB). One military instructor said that after the first few sorties, the student was
expected to carry this out unsupervised (BME). Finally there was mention that the
flight was not over until any necessary paperwork was completed, such as noting any
problems with the aircraft in the maintenance log (BMF).
Clearly this is a snapshot of what the instructors came up with in an interview
setting, and in answer to one question, not an exhaustive list of everything each
organisation requires to be included in early flight training. There are commonalities
across the groups, and differences seem to be mainly in terminology: flight planning
vs preparation and procedures vs work cycles. The items that are most frequently
mentioned can be seen as those that are considered most important: lookout, listen-
out, and checks.
Assessment of Airmanship
The participants were asked what they look for in assessing airmanship, which
led to discussion of whether airmanship was assessed at all, at least with the civilian
instructors. The BFTS instructors were clearer about what they were assessing, and
how to do so, although there was not complete consistency. The advanced military
instructors were divided, in that the instructors at Williamtown responded in a similar
fashion to the BFTS instructors, while the East Sale instructors of instructors were
more inclined to say that there is a more subjective approach.
In the civilian ab-initio context, airmanship is assessed in an informal sense, in
that the instructor is expecting the student to have picked up and used the airmanship
points raised during the pre-flight briefing (CAC, CAD, CAG). There is not really a
formal assessment as such: generally the instructors de-brief the students on where
they see problems or a need for improvement, which would include airmanship
points (CAA, CAB, CAC, CAD, CAF). At one of the schools, there is a report form
on which the instructor has a tick box “airmanship has been maintained”, which the
participant said was not detailed enough, and a further break-down of what to look
for was required (CAE).The flight test pro-forma (which the regulator, CASA,
174
supplies) requires an assessment of the elements of airmanship, including
maintenance of situational awareness, and the comment was made that therefore the
instructors don’t put people up for the flight test unless they show these elements of
good airmanship (CAB, CAE).
Among the civilian advanced instructors there was general agreement that the
term “airmanship” is not used in assessment. Rather, assessment is divided into
technical and non-technical skills, where non-technical skills include situational
awareness, decision making, communication, leadership and automation
management, and technical skills include aircraft handling skills, such as flying an
ILS approach or managing an engine failure on take-off. As one instructor said,
“there isn’t an airmanship box … if you looked at the big umbrella it’s in there
encompassing the whole thing. What we try and do is sort of break it down a little bit
and look at specific areas” (CCE). However, while they are assessed, pilots are not
failed (on a simulator check) unless a problem in the non-technical (or airmanship)
factors is associated with an error or problem in technical skills (CCG). In contrast,
when assessing pilots for Captaincy upgrades, comment was made that non-technical
skills “are about all they look at” (CCF).
When asked about assessment, several of the civilian BFTS instructors
mentioned the “word pictures” that they use, or in some cases, no longer feel they
need to use. These are a set of rubrics for assigning marks for preparation, technique
and airmanship, on the basis that the behaviours exhibited by the student can be
matched to the levels described in the rubric. Thus, assessment consists of
determining whether the student is meeting the required standard for that stage of
training, or is making errors or simply not doing very well. The basic inclusions in
the rubric are the same as were expressed in the definition of airmanship: lookout,
listen-out, awareness of other aircraft, where they are, where the weather is, doing
checks and following work cycles, planning ahead and being aware of consequences,
getting appropriate clearances, applying go-round criteria and so on (Carrick,
Graham, Healey & Pickard, 2008), with more expected of students as they progress
through the program (BCA, BCI).
Comment was made that when the instructor was not sure if the student was
actually on top of things, they would “set them up”. For example demonstrate a
manoeuvre and hand over with the aircraft pointing outside the training area and see
175
if the student notices; another being to spring a practice engine failure and observe
whether the student was ready for it and knew which way to turn for an appropriate
forced landing area (BCC, BCG). There was also comment that there is more
emphasis on the technical handling of the aircraft (BCC) and on what the student has
done wrong (BCE, BCI) in the write up of the sortie.
Military ab-initio BFTS instructors made similar comments to those made by
the civilian BFTS instructors. The word pictures were mentioned (BMC) with the
comment that these are a guide and individuals develop their own interpretation of
them (BMD, BME). There was also comment that expectation of performance
changes as the student progresses (BMA, BMB, BME). More than half of the
instructors in this group noted that the three components that they mark (preparation,
technique and airmanship) are interlinked and fairly blurred as to where any problem
originated or where to allocate marks (BMA, BMB, BMC, BMF). Two of this group
of instructors indicated that they start off assuming everything will go well, then
mark down according to what went wrong or was incorrectly handled (BMA, BMD).
One mentioned setting the student up to make an error, so that they remember it
(BMB). Only two BFTS instructors (one civilian and one military) commented that
what they are looking for in airmanship is fairly well defined (BMB, BCD).
In the context of assessment, the Williamtown based military advanced
instructor participants made comments more similar to the BFTS instructors, than to
the East Sale CFS instructors. Their work in tactical instruction for pilots converting
to jet fighters is more like to the basic instruction at BFTS than to the training of
instructors that occurs at East Sale. Two of the three instructors at the Williamtown
squadron spoke about the use of word-pictures to guide the assessment of all the
different categories to be marked for a mission, one of which is airmanship (MWA,
MWB). One noted that assessment of airmanship tends to be on what went wrong or
non-optimally (MWA) while another noted commenting on and reinforcing good
airmanship as well (MWC).
The CFS participants train flight instructor candidates (FICs), who, if they
qualify, then progress as qualified flight instructors (QFIs) to the various training
squadrons or to basic flight training schools at Tamworth (New England tablelands,
NSW) or Pearce, near Perth in WA. These instructors, when asked about assessing
airmanship, did not mention word pictures at all. In the answer to the question about
176
assessment of airmanship, three of the seven East Sale instructors said that the FICs
were expected to have high to above average airmanship when they were sent for
QFI training, most would have flown in combat and have a lot of experience (MEC,
MED, MEE). However, two of these also said that during the initial phase of
training, conversion to the training aircraft, airmanship is assessed because of the
change-over to flying a different aircraft type and because “people still make
mistakes” (MED, MEE). There was also a comment about the extent to which
airmanship is defined by context, what is appropriate airmanship in one context (like
an operational squadron) may not be appropriate in another (such as a training
school) (MEF). So at this level of instruction, it appears that initially airmanship is
assessed and attention drawn to any areas that may need improvement (MEB), later
in the training it is more a case of drawing attention to issues of airmanship that
should be raised with their future trainee pilots (MEE, MEG). One participant was
adamant that airmanship is subjective and that it should not be reduced to a set
checklist, saying “airmanship isn’t about what you do, and as soon as it’s not about
what you do you can’t objectively assess against a checklist, and if people do
airmanship from a checklist well that isn’t airmanship” (MEA) and added that
assessment of airmanship is rather about evaluation of thought processes and the
resultant decisions that are made.
In summary, assessment of airmanship is apparently quite informal among the
civilian ab-initio instructors, only one mentioned formal recording of airmanship
behaviours. The civilian advanced instructors recognise that what they are assessing
is airmanship, but the term is not used, rather non-technical skills are assessed along
with technical handling, the combination of which forms the overarching idea of
integrated airmanship. In contrast, the military and civilian instructors at BFTS, and
the advanced trainers at Williamtown, while talking about airmanship as an
integration of technical and non-technical skills, are quite clear that they assess
preparation, technical skills and airmanship separately, where airmanship is
described as consisting of what the civilian advanced trainers would call non-
technical skills. Finally, the military advanced trainers at CFS were reluctant to say
that they assess the airmanship of the FICs, noting their experience and existing high
levels of airmanship. It was clear that they separated the technical handling skills
from airmanship in that the FICs have to relearn the specific technical handling
177
requirements that CFS include in the basic flying syllabus, but that airmanship
(excluding flying skills) is assumed to be fine and discussion relates more to how the
FIC will imbue their future students with this aspect of airmanship.
Training Process
In the course of the interviews the instructors made a number of comments
about the way in which they go about the training process. Some voiced a philosophy
of training or comment on their perception of their role as an instructor and what they
see as necessary instructor qualities. Other comments included dealing with the
students themselves and the advice they give to students, how they use errors made
by students to elaborate on the training program, and the use of queries and scenarios
in their instruction. There were also comments on the cognitive capacity of the
students. There was some explanation of instructor training, mostly by the BFTS and
advanced military instructors. There were also a number of comments about training
generally that were not further categorised. Table 4.12 shows the distribution of these
comments across the five instructor groups.
Table 4.12
Percentage of Participants who Commented on Listed Aspects of the Training Process
CAB CAD CBF MBF MAD Total Number of Participants 7 7 9 6 10 39
% % % % % %
Philosophy 42.9 42.9 44.4 16.7 50.0 41.0 Instructor qualities 42.9 42.9 33.3 33.3 20.0 33.3 Dealing with Students 28.6 14.3 55.6 50.0 40.0 38.5 Student advice 14.3 57.1 55.6 33.3 20.0 35.9 Dealing with errors 42.9 28.6 22.2 50.0 40.0 35.9 Queries and scenarios 42.9 71.4 11.1 16.7 10.0 28.2 Cognitive Capacity 0.0 42.9 88.9 66.7 50.0 51.3 Training general 28.6 0.0 33.3 50.0 50.0 33.3 Instructor training 28.6 0.0 55.6 100.0 60.0 48.7 Note. CAB = Civilian ab-initio; CAD = Civilian advanced; CBF = Civilian BFTS; MBF = Military BFTS; MAD = Military advanced
Philosophy. One of the civilian ab-initio instructors made clear that he tried to
develop the idea in the student that he (or she) was expected to learn and memorise,
certain things, which then, he felt, led to a feeling of competence on the part of the
student (CAA). Another noted that if the student appeared to be handling things well,
178
he would just push them a little to see if they can handle a bit more (CAC). A third
participant in this group talked about the importance that the training organisation
teaches within the syllabus and that instructors are responsible for making a real
effort to develop students’ “technical and airmanship skills” (CAG).
The civilian advanced participants presented a consistent position that trainers
need to recognise what is wrong with a student’s performance and know how to fix
it, and ensure that the teaching is about practical application (CCA, CCC, CCD). One
participant in this group noted that it is harder to teach behaviours, so many trainers
just stick to teaching skills, but more often it is thinking through problems and
consequent behaviours that bring the student at this level unstuck (CCF).
The civilian BFTS instructors made some similar comment, that the instructor
should pick up what the student is doing wrong and advise how to fix it (BCC, BCG)
and show the student how to apply what they have learnt in theory (BCB, BCC). One
instructor in this group indicated his opinion that the job is to get the student through
the course at as high a level as possible (BCA), and one of the military BFTS
instructors made much the same comment (BME). In discussing a well-known
accident case study, a further civilian BFTS instructor indicated that taking the
training seriously at all levels can pay off when it counts (BCF).
One of the advanced military instructors commented that the point is to mould
the students into a consistent level of acceptable behaviour reflecting good
airmanship (MEF). Further, they do not train co-pilots (all candidates are expected to
eventually captain aircraft), so the captaincy part of airmanship is important (MEG).
The idea of fixing problems was mentioned here, with the comment that the tricky bit
is to “make it sound like the application of common sense, and make it sound like it
is something the student could realistically seek to achieve themselves” (MEC) and
thus the FIC has to learn to do this. There was also the comment that the FIC’s have
already got the skills and the airmanship, the job of the instructor (at this level) is to
get them to work out how to pass these on to their (future) students (MED). Finally,
there was the comment that airmanship is an art form not a science, and that example
and discussion lead to improvement (MEA).
179
The common theme across the instructor groups is that problems need to be
identified and fixed in a practical fashion, and the student coached in both technical
skills and airmanship behaviours in order for them to pass the course.
Instructor qualities. Providing a good example of airmanship for the students
has been discussed already, however, several instructors indicated that the level of
airmanship displayed needs to be “over the top” (CAA, BCC, MEA). Another
common comment about the qualities that must be displayed was that instructors
have to be alert, as things can change fast and get out of hand quickly, so they have
to know when to step in (CAC, CCG, BCF, BMC). There was also comment that
instructors need to be sure that their own skills are adequate (BMA) and that they
change the way they explain if the student does not understand (BCC). At the higher
levels, instructors should be willing to learn from the ways that other people operate
(CCD). There were a couple of comments on the perceived lack of experience and
knowledge among new young instructors in the civilian ab-initio training
environment (CAG, CCA).
Dealing with students. Some of the instructors, 38.5% of participants overall,
talked about how they interact with their students, not in response to specific
questions but as part of the general discussion. Among the civilian ab-initio
instructors, one spoke of ensuring that the student was at ease and comfortable in the
aircraft and instruction environment (CAA), while another spoke of the difficulties
he had found in instructing overseas students (CAC). Only one of the civilian
advanced instructors talked about interactions with students, saying that the student
needed to be comfortable, and also physically well, and noted the need to avoid
demoralising the student if errors were made (CCA).
The instructors at BFTS made rather more comments about their interaction
with students than the civilian instructors and military advanced instructors. Three of
them commented that they try to keep up their students’ confidence and not shatter it
(BCC, BMA, BMB), although one noted that fear tends to be a good motivator for
avoiding errors. Two of the instructors stated that they sometimes have to deal with
the students’ personal problems affecting their ability or concentration (BCD, BMA).
There is an expectation that students will speak up if they do not understand (BCA)
and will act in an adult manner, for example attend the medic if unwell (BCB). The
aim is to get the student thinking about what they are doing in the air (BCE) and to
180
recognise that correction is for their improvement (BME) although it was noted that
sometimes a fail is required to get the student to concentrate on improving their
progress (BCD).
The military advanced instructors commented more on what they tried to teach
the FIC’s with regard to handling their future students. One noted the need to set the
tone of the sortie so that the student will speak up if they think something is wrong or
if they are not comfortable (MEB). They noted that they give hints as to where
students may have problems in certain sorties (MEB) and strategies for dealing with
things the students may do (MEE) including surprise errors from otherwise good
students (MED). There was also a comment about trying to relate the things that the
FIC has done in operational squadrons to what they are trying to teach the student
and trickle in ideas about airmanship along the way (MEG).
Advice to students. What the instructors give as advice to their students was
mentioned by 35.9% of the participants overall, and most comment came from the
civilian advanced group and the civilian BFTS instructors. One of the civilian ab-
initio instructors indicated that he advised students to abide by the “I’m safe”
mnemonic (CAD). This is a strategy for self-assessment of the question “Am I fit to
fly?” It covers illness, medication, stress, alcohol, fatigue and eating habits, the idea
being to ensure that the pilot is free of adverse indicators. Among the civilian
advanced instructors, advice included that everyone makes mistakes (CCA), that
“you’ve got to really keep an eye on the small details and the big picture at the same
time” (CCD) and the need to be good at time-sharing (CCB). It was also pointed out
that standard operating procedures (SOPs) should always be followed, because if
something goes wrong and they were not adhered to, “you’re done for” (CCG).
Two of the civilian BFTS instructors mentioned an emphasis on preparation
(BCC, BCE). Other advice included an expectation that the student will speak up if
uncomfortable (BCB), keep fit and healthy (BCC), always expect an engine failure
on take-off (BCF), keep ahead of the aircraft (BCE), have a thorough knowledge of
emergency procedures (BCG) and know that the job is not done until the paperwork
is completed (BCG). Further advice was to follow the rules because they are written
in blood (BCF). Advice from the military BFTS instructors was to make a decision,
and remake it later if necessary (BMB) and to pay more attention to the problems
that are raised about a sortie in the debrief, than to the marks given (BMF).
181
Some advice from the military advanced instructors echoed other instructors in
advising that respect for and adherence to the rules be instilled and to always expect
the unexpected from students (MEC). Further advice to the FICs was to always
provide a good example of airmanship, because the students are always watching
(MEG) and to keep even tempered with the students (MED).
Dealing with errors. The participants made comment about how they deal
with the errors that students make. A consistent discussion point across all groups
was that they allow an error to develop so that the student has a chance to recognise
and correct it, so that the student really learns from it, but to not allow it to go outside
their own comfort zone, nearing which point they would intervene (CAA, CAB,
CAC, CCG, BCC, BCF, BMC, MED, MWA, MWC). Related comment included
setting up the aircraft improperly and seeing if the student notices or put them in a
situation where they are sure to miss something (BCC, BMB). The training process
for FICs includes discussion of how to intervene when an error is about to be made
(MEE). There was also comment from a civilian advanced participant that their
organisation is hard on errors, and that more attention should be paid to error
recovery (CCF).
Queries and scenarios. It appeared to be common to use queries and scenarios
to encourage situational awareness and effective decision making (CAG, CCA, CCC,
CCG) as well as to gauge recognition of problems (CAB, CAD, CCB) and aircraft
state (BCB). Questions are also considered useful to assess knowledge (MEE) and to
determine why certain decisions were made (BMC).
Cognitive capacity. There were a number of comments from participants
(mostly from the BFTS and military advanced participants) regarding the cognitive
capacity of their students - essentially whether they were able to keep up with all the
different tasks they are required to do in flying the aircraft and then have the ability
to cope with any emergencies. Several instructors noted that students” capacity was
limited at first, but that as they progressed they were able to take more in, handle the
tasks and broaden their situational awareness (CAB, BCD, BCH, BMB). It was
reported that it is noticeable that when students become overloaded they lose
situational awareness, stop listening, looking, monitoring and thinking ahead (BCA,
BCB, BCE, BCG, BMA, MEF).
182
It is possible to teach how to build up capacity, as one of the civilian advanced
instructors said, “you can teach people timesharing and work allocation, workload
priority, workload management in order to generate spare capacity” (CCB). A
civilian BFTS instructor agreed, “If the student has studied their content and
practiced their drills then they will have the brain space to be able to do the job and
still have that ability to reach out and be situationally aware” (BCF) and similar
comments were made by other instructors (BMA, BMC). The implication among the
comments was that the students at BFTS should know checks and procedures from
ground school and mass briefs, so that the instructor just demonstrates how to apply
them and to develop effective techniques which then helps free up brain space to
expand their situational awareness. Instructors from BFTS noted that they are
constantly assessing capacity of the students, their role being to recognise when the
student is overloaded and take over or reduce task loading for the student, and then
advise on remedial action (BCH, BCI, BMA).
The military advanced group of instructors made more general comments about
capacity, noting that better airmanship means more “ brain capacity” to think and
make decisions (MEE, MWA) and that poor situational awareness occurs when one’s
mental capacity gets overloaded (MEF) something to be alert for when flying alone
(MWC). There was also comment that while experience frees up “brain space” for
faster decisions, overload occurs at different rates for different people and in
different situations (MWB). As one of the civilian advanced instructors noted: “you
can always load people up to the point where they lose situational awareness”
(CCC). However, it was also noted that this is inappropriate in a training
environment. A large modern airliner cannot be flown single pilot, an individual does
not “have the mental capacity or the physical capacity to do it (alone), so you rely
heavily on the people around you” (CCF) sharing information with other crew
members to ensure good situational awareness and airmanship.
Training comments. There were some general comments made about training
that were not consistent with the other categories. There were several comments
about having to step in and take over, when operating in controlled airspace or if the
student is about to head into cloud (CAB), if the student is not using their feet
properly and the instructor has to take over to maintain a straight path on take-off
(BCC) or complete a radio call for the student if something unusual is said by ATC
183
(BMB). One of the civilian ab initio instructors stated that he punishes students for
poor airmanship, such as taxiing too fast or starting over stones, generally by making
them wash an aircraft, or run beside a taxiing aircraft to experience the correct taxi
speed (CAE).
There was comment about the sequence of events in an instructional sortie at
BFTS: the student should have the thought processes in place, then the technique is
demonstrated, they can think about it before the next sortie, when it is demonstrated
again and they have a go themselves (BCB). If an airmanship item is not performed
as it should be, the student is corrected and observed to see how he progresses during
that sortie and so marked. If it continues to be a problem then he will be marked
down on airmanship (BME).One of the BFTS trainers summarised good airmanship
as the student utilising the techniques that he had been taught (BMA). At BFTS, the
instructors generally have students from the same course, so at roughly the same
stage, but may have a mix and so possibly instruct students at very different stages of
the training program. The instructors can revise sortie content via the computer
system before briefings (BCI).
At East Sale there are two programs, A Flight takes on the new FICs and B
Flight runs abbreviated courses for former QFI’s who have been doing something
else and are now returning to an instructional position. B Flight is also the Roulettes
display team (MEC). During sorties, the instructor will talk through their thought
processes round how they are going to go about something, so that the FIC learns the
thinking behind decisions (MEA) or they will talk about why an airmanship breach
occurred, how that may have developed (for example from lack of checks or
inadequate lookout) and how that will impact on the airmanship score (MEB). The
comment was made that “we are very specific and standardised with how we want
the aircraft to be flown” and that sorties are carefully designed to achieve a specific
outcome (MED). A Williamtown instructor noted that because they fly in a tandem
aircraft, while there is a certain amount of observation of the student, it is difficult to
see what is going on, so the instructor will get the student to talk through what they
are doing (MWB).
Instructor training. During discussion about airmanship and training in
general, there were some comments made about training of instructors, however not
by the civilian advanced participants. One of the civilian ab-initio instructors
184
commented that when training instructors (which he also does), he makes the point
that what characterises bad drivers is much the same as poor airmanship, so good
airmanship is similarly being a considerate and responsible pilot (CAA). As was
pointed out by another civilian ab-initio instructor:
there’s no actual component of airmanship in an instructor rating … However part of the testing criteria at the end is that the pilot has shown sound airmanship during the course of the flight so that includes the decision-making, the situational awareness, the radio calls and all that stuff. But there’s no actual learning component or theory component behind it in the instructor rating anyway (CAD)
In contrast, another civilian ab-initio trainer who also trains instructors said that
there is a component of airmanship in the instructor rating, but that it is more focused
on such things as the principle methods of instruction and effective communication
(CAE). He also noted that instructor candidates should already know about
airmanship issues, but that frequently they did not.
Three of the civilian BFTS instructors commented that although they were
previously instructors in civil aviation, when they started at BFTS they went through
an on-the-job process of training to ensure that they flew as required by the Central
Flying School (CFS) (BCF, BCG), but that there was not much specifically on
airmanship, just what the briefings covered (BCH). One of the instructors noted the
use of the word pictures but felt that training in its use could be improved and the
rubric itself expanded (BCC). It was also pointed out that there is internal
continuation training for the instructors, although the more experienced did not
require much, and an annual CFS check (BCI). As far as airmanship goes, the
instructors talk about standardisation and testing and what is being looked for in a
student and there is an Instructor Guide that covers all the sorties including the
airmanship points that should be made (BCI).
All of the military BFTS instructors had been through the CFS course at East
Sale, and all but one commented that there was not a great deal included on
airmanship in that course. The course was said to be more about teaching how to
teach and assess students (BMB, BMD, BME). The process at CFS was remembered
as being like on-the-job training, with the instructor playing the role of the student
and later critiquing the FIC’s demonstration, debrief and write-up (BMD, BMF).
Again there was comment that the incoming FICs already had a good standard of
185
airmanship (BMB, BMD, BMF) although one participant noted that they would be
“pinged” if they did something in breach of good airmanship (BMC). Once at BFTS,
staff continuation training ensures that instructors are up to standard for
demonstration of techniques and aircraft handling (BMA).
The military advanced trainers from East Sale went into a great deal of detail
about the training of FICs. Their statements aligned with those of the military BFTS
participants, in saying that airmanship is not presented as a separate module, but
rather is an integral part of what they need to get across in each sortie and debrief
after it (MEA, MEG), and that airmanship is discussed in tutorials (MED).
Airmanship is briefed in relation to the sortie that is being practised and the FICs
attention is drawn to where their future students might have problems and also to
situations where the instructor might have difficulty maintaining good airmanship
while simultaneously observing what the student is doing (MEB, MEG). One of the
participants explained that the primary objective in the instructor course is to train
FICs to get across a physical skill, the secondary objective is to evaluate how well
the student understood and was able to apply that skill, and therefore how their
airmanship is developing. There is only a finite time to achieve this and so there is a
reliance on the FICs existing airmanship and not much formalisation of instruction in
airmanship (MEC). Some of the time in the air is spent for the FIC to refine their
skills and become expert in handling the training aircraft, so that they can
demonstrate exactly what the student is supposed to learn (MED). In addition, the
FICs are taught the most appropriate way to deal with students, and to use various
techniques to develop airmanship in their students (MEE). The course is “not really
focussing the light on their [the FIC’s] airmanship but focussing their attention on
their requirement to enhance or to educate or to bring on the airmanship of their
student” (MEC).
Discussion: Training of Airmanship
Main Findings
The major difference between the ab-initio instructors and the advanced
instructors is that just over half of the advanced trainers (civilian and military) stated
that airmanship is assumed at their level of training. The advanced military trainers at
CFS were fairly consistent in maintaining that they did not teach airmanship at their
186
level, just showed how to teach it to future students. The civilian advanced trainers
went on to talk about including airmanship (as non-technical skills) in the training
process. The ab-initio instructors all talked about the inclusion of airmanship in the
training that they do.
Learning from others, providing a good example and comments on
organisational culture indicate a concern with embedding airmanship –mention of
each concept was made by 20% to 40% of the participants in any one group, with the
majority of comments coming from the civilian advanced group of instructors. Six of
the seven civilian advanced instructors (85.7%) mentioned one or more of these
concepts and combined, 53.8% of the participants overall made a comment about one
or more of these concepts.
The BFTS instructors more often mentioned that the flight training program is
staged in that there is a gradual build-up of content during the program. Some
comment was made by the military advanced instructors, but this was not mentioned
by the civilian ab-initio instructors and by only one of the civilian advanced
instructors.
All of the ab-initio trainers said that airmanship was included in their pre-flight
brief. The military advanced trainers at Williamtown stated that it is included and the
comments from the military advanced trainers at CFS imply that it is part of the pre-
flight brief there as well. Whether the civilian advanced group include airmanship in
the brief is uncertain, but possibly under the label “non-technical skills’. Those
participants who mentioned the debrief either included airmanship or stated that the
debrief covered the whole flight. There were more mentions of debriefing from the
civilian and military BFTS instructors and the military advanced instructors than
from the civilian ab-initio or advanced groups.
The BFTS instructors talked about curriculum structures in place to ensure
consistency, student records and the relationship between the ground school and
flight training. While these aspects of a training program may exist in the other
training levels and types, they were not mentioned during the interviews, apart from
one civilian ab-initio instructor desiring more formal records and one military
advanced instructor mentioning mass briefs.
187
The responses to questions about the airmanship inclusions in the initial
lessons (or sorties) were consistent in unanimous mention of lookout and frequent
(more than half the participants) mention of listening and radio use. Other consistent
mentions across groups were checks and checklists, aircraft limits or parameters and
taxi speed. There could be some terminology differences as the civilian ab-initio
instructors mentioned flight planning while the BFTS instructors talked of
preparation; and the civilians mentioned procedures while BFTS instructors
commented on work cycles. The civilian ab-initio instructors included safety on the
ground, handover/takeover, and smooth control. The BFTS instructors added much
more detail about inclusions and there were some variations between the civilian and
military instructors about what was included.
Assessment of airmanship was most clearly articulated by the BFTS instructors
(both civilian and military) as being distinct from the assessment of preparation and
technical handling skill. However, several of the instructors noted that there was
some blurring across the lines, saying it is not easy to separate the three. Their
comments on assessment also contrasted with the earlier statements by these same
instructors that airmanship is global and all inclusive, although most of them then
specifically excluded technical skills from the definition of airmanship. Assessment
of airmanship was not so clear among the civilian ab-initio instructors, but there is
awareness of the flight test requirements for situational awareness and threat and
error management competencies to be evident. There does not appear to be a formal
process of assessment during the training program, at least in the organisations
sampled. The civilian advanced instructors noted that the terminology more
commonly used is non-technical skill (and thus excluding technical skills) which is
assessed but is not a fail item unless paired with a problem in technical skills. The
military advanced instructors at Williamtown had similar views to the BFTS
instructors while the CFS instructors appeared reluctant to say that airmanship was
assessed, only two or three made reference to any assessment of the FICs.
The discussion of assessment raised the question of whether technical skill is
part of airmanship – in general discussion about the definition of airmanship
technical flying skill appears to be included. However, in specific questions about
assessment airmanship appears to reduce to non-technical or behavioural aspects of
flight tasks – everything except technical skills. It seems, especially among the
188
military instructors, that there are two levels of thinking about airmanship: the
general, the umbrella concept that airmanship is everything that goes to make a safe
and efficient flight; and the specific, airmanship as assessed at BFTS/CFS that does
not include the technical skills or preparation. Among the civilian instructors that
two-level way of thinking does appear as well, although not as starkly. There were
comments that airmanship is the non-technical, threat and error management sorts of
activities and not the aircraft handling actions, among the ab-initio instructors, when
considering the inclusion of airmanship items in licence flight testing among the
civilian advanced instructors when considering simulator checks.
There was discussion of the training process including comment on training
philosophy, with a view that the instructor’s role is to correct students and suggest
fixes for any problems they have in the air, at least among the civilian advanced and
civilian BFTS instructors. Getting the students through the course and moulding the
students into a consistent level of good airmanship were other approaches among
military instructors.
Instructor qualities mentioned by the participants included the over-the-top
display of airmanship and being alert to take over if things got too far out of hand.
Participants also spoke about dealing with students – generally ensuring students are
at ease and maintaining their confidence. There were many and various spontaneous
comments about the advice given to students. There was discussion of dealing with
student errors and this mainly raised the “comfort zone” issue, across all groups,
concerning when to intervene if the student is progressing down an erroneous course
of action. Questions and probes were regularly mentioned as ways to keep track of
students’ level of situational awareness and knowledge. Cognitive capacity was
discussed in depth by mostly BFTS instructors (possibly because it requires a tick-off
on the student record form) but also by military advanced and some civilian
advanced instructors. General training comments varied with little consistent
comment.
Questions about instructor training led to comment that airmanship is not
explicitly included in the civilian instructor rating. The military BFTS instructors had
little memory of any extensive inclusion of airmanship in the CFS instructor training
program and the military advanced instructors at CFS agreed. Their position is that
FICs already have good airmanship and to instruct in it is unnecessary especially as
189
there is limited time available in the program, so they concentrate on how to teach
the technical skills, with airmanship being covered by hints on how to encourage
airmanship in the students and advice to provide a good example to them. Again this
discussion seemed to exclude technical skills and how to teach them from the
discussion of airmanship.
Comparison with recommendations.
The three step process that Kern (1996) proposed for the integrated instruction
of airmanship consists of initial indoctrination concerning the importance of
airmanship; the specific teaching and modelling of airmanship; and the enforcement
of airmanship standards. Ebbage and Spencer (2003) echo the first two steps and as
the third step, recommend assessment and provision of feedback. This process was to
some extent supported. The military BFTS program includes lectures on airmanship
in the ground school and at the Williamtown conversion training squadron. There
was mention of tutorials on airmanship at CFS. However, these comments need to be
considered in the light of their definition of airmanship as excluding technical skills.
There did not appear to be an equivalent introduction to airmanship, per se, in the
civilian ab-initio training organisations visited. The CASA set theory syllabus for the
licence exams includes human performance and limitations, and this and the
technical subjects would be covered in ground schools or technical college courses as
appropriate. Similarly, the civilian advanced level instructors tended to talk about
non-technical skills being included within the study materials but did not address
airmanship as such.
Overt inclusion of airmanship in ab-initio flight training in terms of the pre-
flight briefs and initial lessons was common to both the civilian and military
environments, covering both basic aircraft handling skills and lookout, checks and
other non-technical skills. Civilian advanced training includes both aircraft handling
and non-technical skills (although it was the non-technical skills that were equated to
airmanship). In the more advanced military training at CFS, airmanship was
considered assumed, and while there was some instruction it seemed focussed on
how to instruct future students rather than focussing on the FICs themselves
(regarded as largely unnecessary), and again was considered separately to aircraft
handling skills.
190
Assessment of airmanship forms the third part of the training process: to be
evaluated actively and aggressively as Kern (1996) put it. This may be more the case
in the BFTS environment, where there is overt assessment and feedback on the three
components of preparation, airmanship and technical skill. The lowest score is taken
as the overall mark and low marks can lead to a fail on the sortie. Persistent problems
and low scores can lead to ejection from the program, so there is equal attention to
the technical and non-technical skills being developed by the students. In the civilian
ab-initio environment, there does not appear to be overt assessment of airmanship (at
least in the organisations sampled) however comment was made that the instructors
know that their students ultimately have to meet the requirements of the flight test,
which CASA specifies as including aircraft handling skills and competency in
various aspects of non-technical skills and threat and error management, and thus
train to that standard. In the civilian advanced training environment, technical skill
appears to take precedence over non-technical skills in that poor performance in non-
technical skills can only cause a fail on the check if it is paired with a technical skill
problem. Thus, while non-technical skills may be assessed and debriefed, poor
performance on non-technical skills alone is not a failure. In the more advanced
military settings, assessment of performance in aircraft type conversion at
Williamtown appeared to parallel BFTS to some extent, with preparation, technical
handling and airmanship (as non-technical skill) being separately considered. At
CFS, assessment was not elaborated on as much, except that airmanship, it appears,
is not assessed. The FIC’s performance as an instructor was under scrutiny and led to
feedback at debrief, but there did not appear to be a formal assessment process.
Conclusion: Airmanship Training and Assessment
Airmanship as a term is not used specifically in the civilian training
environment, it seems, with the emphasis rather being on technical handling skills
and non-technical behaviours. However, these can be seen to combine to form the
integrated airmanship concepts that Kern (1996, 2009a), Ebbage and Spencer (2003)
and Edwards (2013) promote. The military system refers to airmanship as being
everything except technical handling skills when it comes to assessment, but the term
is used to embrace both in more general conversation, reflecting the integrated idea.
The military training programs each appear to consist of a set syllabus with defined
briefs and progression of lessons that integrate technical and non-technical skills
191
(called airmanship), with assessment and enforcement at each level. The civilian ab-
initio process, while guided by CASA publications, is more open to interpretation by
individual training establishments and may not have the same level of formal
assessment and enforcement of airmanship behaviours as is evident in military
training. Organisations at the more advanced levels of civilian training seem to be
somewhat restricted by the agreements with pilot organisations regarding the
evaluation of the non-technical side of the airmanship equation. This results in the
impression that non-technical skills are considered as somehow secondary to the
physical handling of the aircraft, which given the preponderance of human factors
issues in aircraft crashes, and spectacular saves, may be a mistaken viewpoint.
The interviews in this study resulted in a rich and detailed tapestry of
information about airmanship in a variety of instructional environments. The most
salient feature of the discussion is that the instructors seem able to use the term
‘airmanship’ both as a global, all-encompassing term and also a specific reference to
only non-technical skills when considering assessment. This must surely be
confusing for their students. The following chapter discusses the structure of
airmanship and the relationship of the findings in the present research to the existing
models and pilot performance literature.
192
193
Chapter 5. The Structure of Airmanship
The outcome of the research presented in Chapters 2, 3 and 4 is that in the
general sense ‘airmanship’ is a fairly loosely defined term that refers to the ability to
combine the qualities, skills and knowledge areas required to fly an aircraft safely,
efficiently and effectively. These three investigations into the definition and training
of airmanship have shown quite consistent results and broad agreement on the
concepts that make up airmanship, such as situational awareness, decision making
and communication. When writing or talking about airmanship in a general sense,
most participants included technical flying skills within airmanship, noting the all-
inclusive nature of airmanship. It was only when becoming more specific that the
trainers in military environments distinguished between airmanship, aircraft handling
skills and preparation; the three criteria which they assess. When discussing
assessment, the advanced civilian trainers also tended separate the technical handling
of the aircraft from other skills, these being labelled “non-technical” skills.
Furthermore, these trainers defined specific skills within the range of non-technical
skills that they look for during assessment. These non-technical skills were consistent
with the concepts discussed as part of the definition of airmanship.
There appears to be a tension between the general case of airmanship as “safe
and efficient flight” that requires technical skills as well as non-technical skills, and
the specific case of assessment of airmanship. The assessment of “safe and efficient
flight” requires a separation into component skills in order to allow for reasonably
objective appraisal. One approach is to separate technical handling and non-technical
skills, with further categorisation possible within each set of skills. The military has
labelled the non-technical skills as “airmanship”, resulting in some inconsistency
when discussing component concepts of airmanship in the general case and then in
terms of assessment. In the following discussion, airmanship will be considered in
the general case.
Comparison with Previous Research
Many of the findings of the research reported in the preceding chapters are
consistent with findings of previous research conducted in Australia and elsewhere
that has taken an overview of pilot performance. The most frequently mentioned and
194
highest rated and ranked concepts in the current research, including situational
awareness, decision making, knowledge, flying skills, communication, crew
coordination and preparation – have all been noted in such research. Situational
awareness and decision making have been presented as central concepts for good
TEM and for good captains (Mavin & Roth, 2014; Swauger, 2006; Thomas, 2005),
although were not specifically mentioned by Nergård (2011). Mavin (2010) notes
aviation knowledge as an enabling skill for good Captains. Book and aircraft
knowledge were also among the aspects of a good Captain reported by Swauger
(2006) and Nergård’s participants discussed knowledge of rules and regulations and
aircraft limitations. Mavin (2010) includes accurate flying as an essential skill,
Swauger’s participants also included flying skills and Nergård reports extensive
discussion about flying skills. Helmreich (2002) and Thomas (2004) list
communication as essential for TEM, Mavin includes it as an enabling skill and it is
included as a skill of a good pilot as reported in both Swauger (2006) and Nergård
(2011). Crew co-ordination is reported as another skill needed for good pilots in
Nergård (2011); Swauger (2006) reported team building and management of the
flight deck, and this wording is also used by Helmreich (2002) in listing essentials
for TEM. Preparation represented by briefing and planning are also considered of
great importance (Cahill, McDonald & Losa, 2013).
Some of the aspects of airmanship raised by participants (and reported in
Chapters 3 and 4) were less frequently noted in the existing literature. For example,
discipline was mentioned by some survey participants and was rated and ranked
quite high, and overall 36% of interviewees talked about some form of self-
discipline. The term was not reported in any of the studies mentioned above,
although Thomas (2005) included ‘self-regulation’ as a requisite for TEM, which
may relate to self-discipline. Personal qualities mentioned by participants in the
current research included responsibility, patience and honesty, and a desire to excel
or to improve. These are not reproduced in the existing research, although different
personal qualities were mentioned. Thomas (2005) included negotiation skills and
assertiveness as TEM requirements. Swauger (2006) noted mention of some personal
qualities such as having a friendly and relaxed personality and an easy-going nature.
Nergård et al (2011) reported ‘good attitude’, this being acting and socialising
appropriately, and also humility, explained as recognising one’s own errors and
195
asking for help as well as helping other. Self-awareness, self-efficacy and self-esteem
were also reported in that study.
Reference to crew resource management (CRM), as such, was not frequent in
the current research, participants tended more towards reporting components of CRM
(e.g. decision making and communication). In contrast, Swauger (2006) found that
‘establishing a good CRM environment’ was the most frequently mentioned aspect
of a good captain. Similarly, Nergård et al (2011) noted that participants indicated
that the ability to fly the aircraft using CRM was essential.
Model of Airmanship
The concepts raised by participants in writing or talking about airmanship can
be categorised into five main domains, as shown in Figure 5.1. The participants also
mentioned the context in which airmanship is developed or later occurs, and this is
also included in Figure 5.1.
Knowledge
Knowledge is essential to good airmanship. Knowledge is acquired during the
learning involved in obtaining a pilot licence and is represented by the theory
syllabus specified by the regulator, or the ground school learning in the military
sector. Knowledge includes an understanding of the operational parameters of
particular types of aircraft. It also involves updating one’s knowledge of rules and
regulations and procedures as changes occur. It is important to note that the
participants in the current research did not just mention the necessity for acquiring
relevant knowledge, they also indicated that the correct application of knowledge is
required for good airmanship.
Knowledge is specified in both the Kern (1996, 2009a) and Ebbage and
Spencer (2003) models, and is considered a major part of each element in Edwards’
(2013) discussion of airmanship. In addition the Swauger (2006) study indicated the
importance of knowledge, with book and aircraft knowledge being part of one of the
three most frequently mentioned categories of a ‘best captain’. Nergård (2014)
reported that the pilots in his study considered the gaining and application of
practical knowledge to be embedded in the concept of airmanship.
196
Figure 5.1 A new airmanship model
Preparation
Preparation and flight planning were mentioned in both the surveys and
interviews. The concept of preparation was important in both the rating and ranking
questions reported in Chapter 3. The details mentioned by participants when talking
about preparation included the noting of current and likely weather, what the flight
would involve that day, correct maps and charts and any notices to airmen
(NOTAMS) that might affect the flight. These things are covered by mission
knowledge in Kern (2009a). Ebbage and Spencer (2003) included flight preparation
as an example or sub-set of discipline and Edwards (2014) refers to preparation as
part of the operational element of airmanship. Other preparation items that were
mentioned in the current research are an assessment of one’s own limitations and
issues that may affect performance of other crew members. There were also noted by
Nergård (2011). The preparation and delivery of briefings, or attending them, and
pre-flight inspection of the aircraft were mentioned by participants. Risk assessment
197
is a preparation item, although awareness of risks and threats could also be
considered an ongoing activity during the flight.
Non-technical skills
The most common concepts referred to by the participants when defining
airmanship were the skills and abilities that are generally referred to as non-technical
skills: the human factors of flight and the more cognitive activities of the pilot. These
include situation awareness, the next most frequently mentioned concept (after
safety) throughout the current research, and rated and ranked most highly in the
rating and ranking questions reported in Chapter 3. Good situation awareness
requires lookout and listen-out, the earliest items that instructors introduce in
teaching airmanship, as noted by the instructors interviewed (reported in Chapter 4).
Update of the aircraft state by regular scan of the instruments (coded within ‘checks’)
is part of situational awareness, as mentioned by almost all of the instructors.
Decision making was mentioned almost as frequently as situation awareness
and was also ranked and rated highly as important to airmanship. Decision making
(as judgement) is the capstone outcome in the Kern model (1996), and is a major part
of both the Ebbage and Spencer (2003) model and the non-technical standards
discussed by Edwards (2013).
Communication was frequently mentioned as an important part of airmanship
in the surveys and the interviews in the current research. However, the appearance of
communication in the rating and ranking questions was split up into four phrases:
crew interrelations, communication with other aircraft, communication with ATC
and communication with maintenance. Possibly as a result of this dilution of
communication, each phrase ended up in the lower half of the ratings and rankings.
As noted above, communication is seen as an important component of CRM and
good piloting in the literature.
Crew-related and teamwork skills, including crew co-ordination, management
of the flight deck and leadership, were mentioned in the surveys in the current
research. These concepts were most frequently included as part of airmanship by the
advanced civilian instructors during the interviews (reported in Chapter 4). Such
skills are seen as essential to TEM (Helmreich, 200) and for good pilots (Swauger,
2006; Nergård et al, 2011).
198
Non-technical skills, apart from situational awareness and judgement, were
subsumed under ‘knowledge of the team’ in the Kern (1996) model, and formed
‘team skills’ in the Ebbage and Spencer (2003) model. Edwards (2014) considers
non-technical skills to be a major element of airmanship.
Technical Skills
The overall concept of airmanship includes aircraft handling or flying skills as
part of safe and efficient flight. A few participants in the first on-line survey,
(reported in Chapter 2) implied that airmanship is only about control of the aircraft.
In the second survey (reported in Chapter 3), significantly more military background
respondents mentioned aircraft handling skill as part of the definition of airmanship
than did civilian background respondents. By overall number of mentions, flying
skill was the sixth most frequently mentioned concept in the second survey. In
addition flying skill made it into the top ten concepts in the ranking question, but was
at the low end of the distribution of mean ratings. In the interviews (reported in
Chapter 4) flying skill was one of the most frequently mentioned concepts with
regard to airmanship. However, not all participants were prepared to include flying
skill within their definition of airmanship; most of the military ab-initio instructors
excluded technical skills. Nonetheless, when considering airmanship as a global
concept of safe and efficient flight, the skill to effectively control the aircraft has to
be included.
Kern (1996) included ‘skill’ as one of the bedrock principles of airmanship,
along with proficiency. Ebbage and Spencer (2003) included physical flying skills as
part of their model of airmanship and Edwards (2013) presented technical skills as
the first of his elements of airmanship. Swauger (2006) found that flying skill was
one of the middle tier elements (in terms of frequency of mention) of a “best pilot”,
and Nergård (2011) reported extensive discussion of technical abilities as part of
‘what makes a good pilot’.
Personal Attributes
There were many participants who listed various personal qualities or attributes
they viewed as part of good airmanship. These included discipline, self-
improvement, responsibility, common sense and diligence.
199
Discipline was mentioned by participants in all three studies. In the second on-
line survey, discipline was rated third-highest concept and ranked sixth, so it was
considered important by those participants. Self-discipline was also implied in the
comments about complying with rules and regulations and not cutting corners. Kern
(1996, 1997) sees discipline as the cornerstone of good airmanship, essential for
maintaining standards and safety. Rather than showing it as a separate element of the
model, as Kern does, it is included as a personal attribute in Figure 5.1.
Another personal attribute mentioned by participants in the current research
was the idea that the aviator should be internally motivated to improve, to be the best
they can and/or work towards improvement. This is consistent with the Ebbage and
Spencer inclusion of self-assessment and self-development in their model. Kern
(1996) also discusses these concepts in ‘knowledge of the self’. Edwards (2013)
included a number of personal attributes in his discussion of the non-technical
elements of airmanship (e.g., enthusiasm, motivation, confidence, honesty and
perseverance).
Common sense was frequently mentioned and rated and ranked highly in the
relevant questions in the second survey. Common sense is not particularly well
defined, but seems to be used to express the sensible, logical use of skills and
knowledge to achieve the safest outcome, especially where rules and procedures do
not apply. It is included in personal attributes rather than non-technical skills because
it is poorly defined, somewhat idiosyncratic, and not a skill, as such.
The current research also supports the idea that there is a place for individual
responsibility in airmanship, with many participants mentioning responsibility.
Responsibility was rated the second highest concept and was ranked twelfth in the
second survey (reported in Chapter 3), and was mentioned by many of the civilian
ab-initio instructors in their interviews (reported in Chapter 4).
Context
The influence of the context on airmanship is indicated by the sixth block in
Figure 5.1, presented in a different colour to contrast with the internal aspects of
airmanship (the blue blocks).
200
Kern (2009a, 2009b) suggests that constant striving for perfection and firm
discipline should be central to the idea of airmanship and can defeat any tendency to
make errors. However, there are occasions when the situation stacks up against the
individual, reflected in the position that errors, incidents and accidents can be
ascribed to systemic faults and training problems within organisations, and not
necessarily primarily related to fault on the part of an individual (Reason, 1990). The
individual pilot cannot be considered in isolation, but must be seen as part of a
system. Thus airmanship is also a product of the environment in which pilots were
trained, the examples they have had, and is modified by their current organisational
culture. These factors were all brought up by various participants in the current
research. The ideal situation is one in which the individual pilot accepts
responsibility for maintaining their own competence at the highest possible level (i.e.
good airmanship), supported by a positive organisational culture, effective training,
well maintained aircraft and operating within a well-regulated system. In a less ideal
world, a pilot may feel they have to rise above the inadequacies of a poor
organisational culture, inadequate training, defective aircraft and/or a dysfunctional
system. In this situation, maintaining individual standards of good airmanship
regardless of the context may make a significant difference to the safety of their
flight.
Conclusion
The model of airmanship proposed here is intended to provide a more fluid
impression than the rigid temple façade of the Kern (1996, 2009a) model and avoids
the hierarchy of the components of airmanship in that model. The domains in the
structure of the model presented in Figure 5.1 reflect grouping of compatible
concepts, raised by participants in the research reported here. Furthermore, this
model acknowledges the influence of the context in which airmanship is displayed.
201
Chapter 6. Airmanship Concept Domains
The airmanship model developed from the findings of the surveys and
interviews, reported in earlier chapters, and presented in Chapter 5, is a logical and
semantic organisation of the airmanship-related concepts mentioned by participants.
The model suggests five domains of concepts: knowledge, preparation, technical
skills, non-technical skills, and personal attributes. It also recognises that the exercise
of airmanship takes place within a context that includes the training environment,
mentorship, experience, organisational culture and the regulatory system.
A further study was required to confirm that the grouping of concepts into
domains in the proposed model of airmanship reflects the conceptualisation of
airmanship among Australian aviators. The study consisted of two parts: a pilot
study, in which a questionnaire was developed and tested for suitability and the main
study, which was an on-line survey using the refined questionnaire. The pilot study is
reported first, followed by the main on-line survey.
The aim of this study was to investigate whether Australian aviators consider
the concepts that were raised in discussion of airmanship in the earlier studies and
formed the basis of the model presented in Chapter 5, are appropriately grouped in
the five domains indicated in the model. Support for the domains and the model of
airmanship would be provided if the concepts paired with other concepts from the
same domain (within-domain pairs) are regarded as more closely related than pairs of
concepts from different domains (between-domain pairs).
A set of 15 concepts were drawn from the proposed model of airmanship.
Participants were asked to rate how closely related they thought pairs of these
concepts were, in the context of their interpretation of airmanship.
Methodological Considerations
A form of paired comparisons was considered more likely to produce useable
results than creating statements and asking participants how strongly they agreed
with them, or asking how important participants thought each of a selection of
concepts was to airmanship. A questionnaire asking about the relevance, or
202
importance, of the concepts in the model to airmanship is likely to suffer from an
upward bias, as was found in the rating section of the survey reported in Chapter 3,
wherein most concepts were rated as important or very important. This kind of
upward bias was also found by Nergård (2011). In a study of ‘what makes a good
pilot’, Nergård asked Scandinavian pilots to rate their agreement on a 10 point
Likert-type scale to a set of 18 statements; 15 of the mean scores for the statements
were between 8.20 and 9.49, very close to the top of the scale. Such upward bias
would reduce the differentiation among concepts and possibly rule out factor analysis
due to violation of assumptions of normality. The intent for the current study was
therefore to ask about the association between various concepts already known to be
relevant, from responses made by participants in the earlier studies, and use multi-
dimensional scaling to examine the clustering of concepts.
Multi-dimensional scaling (MDS) is a multi-variate statistical technique in
which quantitative comparisons between pairs of concepts can be used to determine
the structure of a data set (Bartholomew, Steele, Moustaki & Galbraith, 2002). MDS
has advantages over factor analysis in that MDS does not require that the underlying
data distribution be normal, nor that the relationships be linear, and generally
produces more parsimonious descriptions of underlying data structures (Brazill &
Grofman, 2002). One aim of MDS analysis, then, is to discover whether the data set
can be represented in a small number of dimensions. Co-location of data points in the
appropriate dimensional representation should reflect underlying groupings of data,
or the data may show variability along dimensions inferred from the analysis
(Steyvers, 2002).
In MDS, the goodness of fit of the data set to the dimensional representation is
referred to as the ‘stress’; values of stress that are close to zero indicate that the MDS
solution is a good fit. The standard given is that a stress of 0.20 is poor, 0.05 is good
and 0.00 is perfect (Kruskal, 1964, cited by Bartholomew, Steele, Moustaki &
Galbraith, 2002). Borg and Groenen (2005) note that the index of stress is closely
related to the amount of error (or variability) in the data: as this rises, so does the
stress.
There is a trade-off between the number of dimensions required to improve fit,
and the interpretability of the resulting multi-dimensional plot (Borg & Groenen,
203
2005). Too few dimensions may distort the MDS structure due to over-compression,
while too many dimensions may blur the structure due to over-fitting of noise
components. Thus, a data set with some variability in the estimates of distance may
lead to increased stress values, resulting in a decision to use a plot of higher
dimensionality than is supportive of the underlying data structure (Borg & Groenen,
2005).
Pilot Study
A pilot study was conducted to ensure that the survey instructions and the
format and content of the questions were clear. It also allowed estimation of the time
required to answer the questions.
Method
Participants
Thirty-five pilots known to the researcher were contacted and asked to try out
the survey. Of these nine returned the survey within the required time-frame (a 25%
response rate). These were all Australian males. Six of the participants trained
initially as civilians, two had initial civilian training but moved on to the military and
one did not answer this question. Two participants held a CPL, six held an ATPL and
one had military qualifications only. Five participants flew for airlines, two in the
military and one was a civilian instructor and one was in charter operations.
Experience ranged from 1500 hours to 18,000 hours (M = 4372.22, SD = 3707.18)
and from 3 years to 30 years (M = 14.14, SD = 8.69).
Measure
A survey was developed that asked pilots to indicate how closely related pairs
of concepts appeared to them. Three representative concepts were selected from each
of the five domains of airmanship shown in the model in Figure 5.1: knowledge,
preparation, technical skills, non-technical skills and personal attributes. In the cases
of knowledge, preparation and non-technical skills, these were the most frequently
mentioned concepts in each domain, consistently most frequently mentioned across
the data sets reported in Chapters 3 and 4. For the technical skills concepts that were
rated or ranked highly in the second survey were used and for the personal attributes
204
domain, concepts were extracted from those included in the personal qualities
composite node in the second survey.
Three concepts were chosen from each domain. Adding more concepts would
have made the number of comparisons too large for participants to complete in a
reasonable time frame. Each of the 15 concepts was paired with every other concept
(105 pairs). There were also 10 additional pairings of concepts from within the same
group and eight pairs of concepts from different groups, presented in reverse order,
for a total of 123 pairs. Reverse pairs were included as a check on the reliability of
the ratings. The domains and concepts are shown in Table 6.1.
Participants were first asked to think about their “ideas about ‘airmanship’ and
the components that make it up, the things that you need in order to display good
airmanship”. They were then presented with each concept in turn and asked to rate
the ‘relatedness’ of the concept with each of eight other concepts (two concepts had
9 others), some from the same domain and some from other domains. The concepts
were presented in random order, and the comparison concepts were randomised
within the set. The five point Likert-type rating scale was from “very low
relatedness”, through “low”, “medium relatedness”, “high” to “very high
relatedness”. The pilot survey was presented in an excel file in which the participants
indicated responses by adding a cross in the relevant column for the Likert scale.
Table 6.1 Domains and representative concepts
Domain Concepts
Knowledge aircraft type knowledge; knowing SOPs (standard
operating procedures); knowledge of rules and regulations
Preparation flight planning; pre-flight briefing; preparation
Technical skills aircraft handling; control of the aircraft; proficiency
Non-technical skills situational awareness; decision making; communication
Personal attributes discipline; motivation to excel; responsibility
205
Procedure
Ethics approval was granted through a variation to the ‘parent’ ethics
application, H-647-1207.
The researcher emailed 35 pilots known to her asking them if they would be
willing to complete the questionnaire (attached as an excel file) which was being
developed for her research. The participants were asked to return the completed
questionnaire, by email, together with any comments they wished to make. They
were also asked to provide an estimate of the time taken to complete the
questionnaire.
Analysis
The data from the returned excel sheets were converted into SPSS (IBM, 2015)
for analysis of mean relatedness.
Results
Mean relatedness between pairs of concepts is shown in Table 6.2. Within-
domain relatedness is shown in bold. For clarity, any data located in the upper
diagonal half of the table were transposed to the lower half, thus leaving only the
‘reverse pairs’ in the upper section. Two issues emerged from observation of the
pilot data. First, due to an oversight, one pair of concepts (pre-flight briefing with
discipline) was omitted from the pilot questionnaire. Second, the repeat measure
pairings were concentrated on pairs with situational awareness and preparation,
rather than more evenly distributed across the table.
206
Table 6.2 Mean relatedness between pairs of concepts in the pilot data grouped by domain
Domains Concepts
Knowledge Non-technical skills Preparation Personal attributes Technical skills
Aircraft type
knowledge
Know
ing SOPs
Know
ledge of rules &
regs
Com
munication
Decision
Making
Situational aw
areness
Flight planning
Pre-flight briefing
Preparation
Discipline
Motivation to
excel
Responsibility
Aircraft
handling skills
Control of aircraft
Proficiency
Know
ledge
A/c type knowledge 2.75 Knowing SOPs 3.00 4.50 3.00 Knowl of rules & regs 3.25 4.88 3.38
Non-
technical skills
Communication 2.00 2.40 3.00 4.38 4.13 Decision Making 3.80 4.10 4.00 4.25 4.63 3.75 4.25 Situational awareness 2.88 3.13 3.25 4.25 4.38 3.38 Prepara-
tion
Flight planning 3.10 3.63 4.00 3.00 4.25 3.10 4.75 3.25 Pre-flight briefing 3.75 4.13 4.38 3.50 3.75 3.75 4.75 4.75 Preparation 3.63 4.13 4.10 3.40 4.10 3.88 4.75 4.13 Personal
attributes
Discipline 3.63 3.80 4.38 2.80 3.63 3.63 3.63 4.38 3.75 Motivation to excel 3.40 3.40 4.00 2.75 3.50 2.30 2.75 3.00 3.10 4.50 3.88 Responsibility 4.00 3.60 4.10 3.00 3.80 3.88 3.50 3.30 4.38 3.88 4.00 3.00 Technical
skills
A/c handling skills 3.30 2.80 1.75 2.38 3.00 2.30 1.40 1.75 2.75 3.25 2.80 2.75 4.38 Control of a/c 4.13 3.00 1.80 1.50 2.30 3.38 1.38 1.80 2.00 3.30 2.50 3.40 5.00 Proficiency 4.40 4.75 4.38 3.30 3.88 4.38 3.30 3.13 3.60 3.88 3.30 3.13 4.25 4.50
Note. Within-domain means are shown in bold. A/c = aircraft, regs = regulations.
207
The pilot data, while from a very small sample, show that the within-domain
pairings were generally rated as more closely related than the between-domain
pairings. An exception was the mean relatedness of ‘aircraft type knowledge’ with
other concepts, nearly all of which were higher than the relatedness with the within-
domain knowledge concepts. More importantly the repeated pairs (shown in the
upper diagonal half of Table 6.2) showed similar relatedness ratings to the same
pairs in the reverse order in the lower diagonal half of the table. This indicates that
the relatedness did not depend upon the order in which the concepts within a pair
were presented and that relatedness is measured reliably.
The participants reported taking between 15 and 20 minutes to complete the
survey. Comments were that the survey was easy to understand and quite clear. One
participant mentioned that ‘relatedness’ was a clumsy term, without offering an
alternative.
Discussion
The pilot study drew attention to some problems with the distribution of
questions across the range of concepts. This was corrected by ensuring equal
representation of comparisons between each concept domain in the survey proper. It
appeared the length of the survey was reasonable and was not considered too time
consuming by the participants. The data followed the expected patterns, although
there was some indication that knowledge of aircraft type was judged differently
from the other knowledge concepts. Further clarification of this was expected to
emerge after a larger sample was surveyed.
Main Study
The aim of the study was to examine whether airmanship concepts are seen by
Australian aviators to be related to each other in terms of the five domains indicated
in the model presented in Chapter 5.
208
If this model of airmanship accurately represents the views of Australian
pilots, then it is expected that concepts paired with other concepts from the same
domain in the model (i.e. within-domain pairs) will be regarded as more closely
related than pairs of concepts from different domains (i.e. between-domain pairs)
when pairs of concepts are presented for comparison. It was also expected that multi-
dimensional scaling of the resulting data set would show clusters of concepts that
reflect the domains shown in the airmanship model.
Method
Participants
The criteria for inclusion were: 1) a pilot who had trained in Australia; or 2) a
pilot trained elsewhere but who was currently based in or flying in Australia. Of the
232 hits on the on-line survey site, 87 met one of the two criteria. The majority of
participants were pilots trained in Australia (n= 79). A further eight were pilots
trained elsewhere, but flying in, or based in Australia. Of the remaining 145 hits on
the site, 19 pilots who were neither Australian trained nor Australian based, and 126
did not complete the survey.
Ten participants were female and 76 were male, with one participant choosing
not to answer this question. Participants were asked in which sector they undertook
their initial training. Twenty-one participants reported initial training in the military,
ten reported initial civilian training and then a move to the military. The remaining
56 participants trained in the civilian sector. Experience ranged from 235 to 25,000
hours (M = 7037.78; SD = 5810.95), and from one year to 46 years in aviation (M =
18.97; SD = 11.02).
As reported above, the majority of the participants learned to fly in Australia (n
= 79). The remaining eight participants were currently flying in or were based in
Australia, but were trained in New Zealand (N = 3), UK (N = 3) or the USA (N = 2).
In total, 73 participants were currently flying in or were based in Australia; nine in
Hong Kong; three in Saudi Arabia; one in China; and one in Singapore. Participants
also reported their level of flying qualification: only two participants held a Private
Pilot’s Licence (PPL) or lesser qualification; 19 reported having a Commercial Pilot
209
Licence (CPL); 57 reported an Airline Transportation Pilot’s Licence (ATPL) and
nine reported Military qualifications.
Participants were also asked “what sort of flying do you mostly do, currently?”
Three participants chose not to answer this question. Forty-five participants reported
airline regular public transport activity, seven reported private or recreational flying,
five reported general aviation/airwork activity, three reported charter flying, twelve
reported civilian instruction and nine military instruction, and finally, three
participants reported military flying (not instruction).
Measure
An online survey was developed from the pilot survey. The same frequently
mentioned concepts were used as in the pilot study, three from each of the five
domains of concepts that make up the model of airmanship proposed in Chapter 5
(listed in Table 6.1). Assessment of the reliability of the relatedness ratings was
examined by adding 20 reverse pairs, 10 within-domain and 10 between-domain
pairs, balanced such that there were two pairs from each within-domain set and a
sample pair from each possible between-domain pairing, ensuring that the same
concepts were not over-represented. This gave a total of 125 pairs for participants to
rate.
Participants were first asked to think about their “ideas about ‘airmanship’ and
the components that make it up, the things that you need in order to display good
airmanship”, to establish the context of airmanship. They were then presented with
each concept in turn and asked to rate the relatedness of the concept to each of eight
or nine of the other concepts, some within-domain and some between-domain. The
15 concepts were presented in random order and each set of eight or nine comparison
concepts were also randomised. The five point Likert-type rating scale was from
“very low relatedness”, through “low”, “medium relatedness”, “high” to “very high
relatedness.” Some demographic questions concluded the survey, these included
questions about gender, initial training, country of initial training and current base,
level of pilot licence, current type of flying and amount of experience. A copy of the
questionnaire can be found in Appendix F.
210
Procedure
A snowballing recruitment strategy was employed, in which all 120 pilots
known to the researcher and colleagues were emailed and invited to complete the on-
line survey. Their assistance was sought to forward the email to other pilots whom
they knew and were comfortable contacting. Several contacts reported notifying their
airline training section and there was generally further distribution of the invitation
through these organisations.
The recruitment email included the URL for the survey site. The initial page of
the survey contained information about the survey and contact details for the
researchers and ethics committee. Continuation to the survey questions was
considered to indicate consent to participate. The closing page thanked the
participant for their time and indicated that if the participant wanted a report on the
results of the survey they could email the researcher and request a copy. There were
several requests from pilots not known to the researcher, indicating that some
snowballing had occurred.
Analysis
SPSS23 (IBM, 2015) was used to conduct statistical analysis of reliability and
relative relatedness of within-domain and between-domain concepts. Matlab (2015)
was used to perform multi-dimensional scaling to determine the explanatory
clustering within the data.
Results
Reliability
Two approaches to estimating reliability were used: first whether the mean
relatedness differed for the forward-worded concept comparisons and their reverse-
worded pairs; and second whether the ratings of these forward-worded concept
comparisons and their reverse-worded pairs were correlated. If the survey method is
reliable, the mean relatedness of the forward-worded and reverse-worded pairs
should not differ significantly and the ratings for the forward-worded concepts and
their reverse worded pairs should be highly correlated. Table 6.3 presents the
forward/reverse worded pairs, their mean relatedness and the outcomes of the
211
reliability checks. Due to the number of comparisons being made, the Bonferroni
adjusted critical value of p < .001 was used to control the family-wise false positive
error rate.
Because the five-point scale used to rate relatedness is essentially an ordinal
(rather than interval or ratio) scale, comparison of the mean relatedness of the
duplicated pairs was performed using the Wilcoxon signed ranks test. The Wilcoxon
tests on each set of pairs showed significant differences (at p < .001) between three
pairs: knowledge of rules and regulations with aircraft type knowledge; motivation
to excel with aircraft type knowledge; and aircraft handling skills with flight
planning. The remaining 17 duplicated pairs did not show significant differences in
mean relatedness as a function of forward vs reverse pairing of concepts.
Table 6.3 also shows the outcome of the calculation of a Pearson product
moment correlation coefficient for each of the forward/reverse worded pairs. The
majority of the pairs were significantly correlated, with correlation coefficients
ranging from .32 to .69. The between-domain pairs tended to have higher
correlations than the within-domain pairs.
Overall, few (three) of the sets of pairs showed significant differences in mean
rated relatedness when the order of the concepts was reversed. The majority (17) of
the forward/reverse-worded pairs’ ratings were significantly correlated. Although it
is not absolutely certain that reported relatedness would be the same regardless of the
order in which the concepts appeared in the pairings in the survey, it is reasonable to
say that the reported relatedness is sufficiently reliable to permit further analysis.
212
Table 6.3 Mean relatedness for forward and reverse worded pairs, and outcome of Wilcoxon Signed Ranks (WSR) tests and Pearson correlations for each pair
Relatedness means
WSR Pearson correlations
Concept Pairs forward reverse p r p
Within domain
Knowledge of rules & regs – Aircraft type knowledge 3.33 2.90
* .51 *
Knowledge of rules & regs - Knowing SOPs 4.37 4.18
.65 *
Situational awareness - Communication 4.13 4.33
.35 *
Situational awareness - Decision making 4.59 4.53
.22
Preparation - Flight planning 4.31 4.48
.32
Preparation - Pre-flight briefing 4.39 4.46
.43 *
Responsibility - Discipline 4.28 4.24
.55 *
Responsibility - Motivation to excel 3.84 4.09
.46 *
Control of aircraft - Aircraft handling skills 4.71 4.72
.39 *
Proficiency - Aircraft handling skills 4.30 4.28
.29
Between domains
Pre-flight briefing - Aircraft type knowledge 3.26 3.16
.58 *
Motivation to excel - Aircraft type knowledge 4.08 3.24
* .52 *
Control of aircraft - Knowing SOPs 3.29 3.08
.46 *
Situational awareness - Knowledge of rules & regs 3.52 3.64
.59 *
Discipline - Communication 3.67 3.51
.60 *
Proficiency - Decision making 4.18 4.06
.40 *
Preparation - Situational awareness 4.09 3.91
.52 *
Motivation to excel - Flight planning 3.40 3.11
.69 *
Aircraft handling skills - Flight planning 2.58 2.23
* .64 *
Aircraft handling skills - Responsibility 3.25 3.46
.60 *
Note: * p < .001; regs = regulations
213
Within-Domain and Between-Domain Differences
Table 6.4 presents the mean relatedness for the comparisons between each of
the 15 concepts and every other concept. The reverse worded pairs have been
omitted from the table and from any further analysis. Any pairs positioned in the
upper diagonal half of the table due to the concept order of the paired comparison
have been transposed to the lower diagonal half, so that all relatedness means fall in
the lower section. The concept pairs from within the same domain are shown in bold
in the table, along the diagonal.
An expectation of the data was that the ‘within-domain’ relatedness would be
greater than the relatedness of the relevant pairings with concepts from other
domains (between-domain). Construction of an overall within-domains relatedness
variable (combining the ratings that made up the means along the diagonal in Table
6.4) and an overall between-domains variable (combining the ratings that made up
the non-diagonal means) allows testing of this expectation.
A repeated measures t-test indicated that overall within-domain relatedness (M
= 4.11, SD = .412) is significantly greater than overall between-domain relatedness
(M = 3.63, SD = .490), t (69) = 13.06, p < .001, and this is a large effect (d = 1.07).
Within-domain and between-domain differences can also be examined at the
domain level and at the concept level. These are presented in Appendix G.
Using calculated overall relatedness means (as has been done above and in
Appendix G) supported the expectation that there will be systematic differences
between within-domain and between-domain means. However using such summary
means results in a loss of the information contained in the variations among
individual concept-pair ratings. An MDS analysis using the full set of concept pairs
takes into account such variation among individual comparisons.
214
Table 6.4 Mean relatedness of pairs of concepts grouped by domain
Domain Concepts
Knowledge Non-technical skills Preparation Personal Attributes Technical skills
Aircraft type
knowledge
Know
ing SOPs
Know
ledge of rules &
regs
Com
munication
Decision
Making
Situational aw
areness
Flight planning
Pre-flight briefing
Preparation
Discipline
Motivation to
excel
Responsibility
Aircraft
handling skills
Control of aircraft
Proficiency
Know
ledge
A/c type knowledge Knowing SOPs 3.41 Knowledge of rules & regulations 3.33 4.37
Non-
technical skills
Communication 2.71 3.47 3.32 Decision Making 3.77 4.08 4.16 4.14 Situational awareness 3.65 3.60 3.52 4.13 4.56 Prepara-
tion
Flight planning 3.33 3.55 3.90 3.13 3.98 3.73 Pre-flight briefing 3.26 3.84 3.97 3.90 3.83 4.00 4.13 Preparation 4.29 4.16 4.05 3.34 4.15 4.09 4.31 4.39 Personal
attribute
Discipline 3.81 4.46 4.17 3.67 3.97 3.84 3.71 3.95 4.09 Motivation to excel 4.08 3.76 3.71 3.39 3.36 3.04 3.40 3.49 3.69 4.30 Responsibility 4.02 4.11 4.24 3.93 4.23 3.82 3.76 3.82 4.14 4.28 3.85 Technical
skills
A/c handling skills 3.85 3.19 2.68 2.69 3.33 3.35 2.58 2.65 3.34 3.74 3.69 3.25 Control of aircraft 4.26 3.29 2.78 2.86 3.71 3.76 2.61 2.48 3.94 3.60 3.03 3.69 4.71 Proficiency 3.83 4.20 3.94 3.55 4.18 4.22 2.87 3.45 3.38 4.03 3.87 3.72 4.30 4.18
Note. Within-group means are shown in bold. A/c = aircraft; regs = regulations
215
Multi-dimensional scaling
The data presented in Table 6.4 were used in an MDS analysis using Matlab. In
order to do this, values for the diagonal cells were required (i.e. the comparison of
each concept with itself). These were set at ‘5’, based on the assumption that each
concept would be considered identical to itself and ‘5’ represents closest relatedness.
A second assumption was that the data could be reflected about the diagonal. The
checks of reliability reported above support this assumption; the order of concepts
within the pair comparisons does not lead to different reported relatedness for the
majority of pairs. The final assumption is that distance apart is represented by the
complement of the relatedness of the concepts (i.e. distance = 5 minus relatedness).
A non-metric analysis was used due to the concepts being a psychological estimate,
made on an essentially ordinal scale, rather than a physical measurement (Borg &
Groenen, 2005; Steyvers, 2002).
The scree plot of the MDS solutions is shown in Figure 6.1. As noted in the
discussion of methodology earlier in the chapter, a stress value of .05 is considered
good, but would require a 5-dimensional solution in this analysis. The ‘elbow’ in the
stress curve occurs at two dimensions and the 2-dimensional plot (Figure 6.2) is
easier to interpret, but the stress value is approximately .15. The 3-dimensional
solution has somewhat lower stress, at .10, and expresses the relationships between
the concepts more exactly. Matlab is able to show a rotation of the 3-dimensional
plot, making the relative positions of concept points more explicit. Figures 6.3 and
6.4 are two views of the 3-dimensional solution.
The MDS 2-dimensional solution is a good reflection of the expected
relatedness of the concepts within three of the five domains. The non-technical skill
concepts, the technical skills concepts and the preparation concepts are clearly
distinct from each other and from the knowledge and personal attribute concepts.
216
Figure 6.1 Scree plot of stress values for one to seven dimension solutions.
Figure 6.2 MDS 2-dimensional plot. Note that NT1 = communication, NT2 = decision making, NT3 = situation awareness, P1 = flight planning, P2 = pre-flight briefing, P3 = preparation, T1 = aircraft handling skills, T2 = control of the aircraft, T3 = proficiency, K1 = aircraft type knowledge, K2 = knowing SOPs, K3 = knowledge of rules and regulations, PA1 = discipline, PA2 = motivation to excel, PA3 = responsibility.
217
Figure 6.3 MDS 3-dimensional plot at 20 degrees of rotation. Note that NT1 = communication, NT2 = decision making, NT3 = situation awareness, P1 = flight planning, P2 = flight planning, P3 = preparation, T1 = aircraft handling skills, T2 = control of the aircraft, T3 = proficiency, K1 = aircraft type knowledge, K2 = knowing SOPs, K3 = knowledge of rules and regulations, PA1 = discipline, PA2 = motivation to excel, PA3 = responsibility.
Figure 6.4 MDS 3-dimensional plot at 60 degrees of rotation. Note that NT1 = communication, NT2 = decision making, NT3 = situation awareness, P1 = flight planning, P2 = flight planning, P3 = preparation, T1 = aircraft handling skills, T2 = control of the aircraft, T3 = proficiency, K1 = aircraft type knowledge, K2 = knowing SOPs, K3 = knowledge of rules and regulations, PA1 = discipline, PA2 = motivation to excel, PA3 = responsibility.
218
Figure 6.3 shows that knowledge of the aircraft type (K1) is closely associated
with control of the aircraft (T2); aircraft handling skills (T1) is relatively close by;
and all three concepts are quite distinct from the other concepts. Proficiency (T3) is
far closer to the personal attributes (PA) concepts, which form their own grouping
between the other knowledge items (K2, rules and regulations, and K3, SOP’s) and
the technical skills. The preparation and non-technical skills concepts appear mixed
together in Figure 6.3, but in the rotated view in Figure 6.4 it is clearer that they are
actually separated.
Discussion
The intent of this study was to determine if the airmanship model presented in
Chapter 5 accurately reflects how Australian pilots view airmanship. To do so,
concepts already shown to be relevant to airmanship through previous surveys and
interviews (discussed in Chapters 2, 3 and 4) were selected to represent the domains
of the airmanship model and participants were asked to rate the relatedness of pairs
of these concepts in an on-line survey.
The initial expectation was that the within-domain relatedness would be greater
than that of the relevant between-domain concept pairs. The significant difference
between the overall within-domain and between-domain relatedness variables
supports this expectation.
It was also expected that the multi-dimensional scaling of the concept
relatedness data would reflect the five concept domains as presented in the
airmanship model proposed in Chapter 5. The MDS solutions support the idea that
there at least three separate groups in the data, corresponding to the technical skills,
non-technical skills and preparation domains. Personal attributes seem to be fairly
closely associated with two of the knowledge concepts (knowing SOPs and
knowledge of rules and regulations) while the knowledge of aircraft type concept is
more closely related to aircraft handling and control of the aircraft from the technical
skills group. The remaining technical skill concept, proficiency, seems more closely
linked to personal attributes than to the other technical skill concepts.
219
It appears that the participants were considering the knowledge concepts in a
functional way, in terms of how each pair of terms related to the tasks at hand. Thus,
knowledge of the aircraft type is most highly related to (and co-located with) control
of the aircraft and, to a slightly lesser extent, aircraft handling skills for which such
knowledge is needed. Both knowing SOPs and knowledge of rules and regulations
are located fairly close the personal attribute concept of discipline, perhaps reflecting
what is required to maintain such knowledge. Similarly, proficiency is close to
discipline and motivation to excel, reflecting attributes required to attain proficiency.
The interleaving of knowledge with aviation activity is reflective of the
comment made in earlier surveys and interviews (reported in Chapters 3 and 4) that it
is not just having relevant knowledge that is important, but also the ability to apply it
appropriately. Similarly, personal attributes are not simply required of themselves,
but are important for the effect that they have on other domains within airmanship.
Possibly, without responsibility, preparation is not adequately completed; without
discipline, knowledge is not maintained; and without the motivation to excel,
proficiency is not achieved.
The model of airmanship presented in Chapter 5 has been adapted to take into
account the relationships that have become apparent in the MDS analysis. This
model reflects how Australian aviators report the relationships between concepts that
are part of airmanship. The model illustrates airmanship as having three main
domains of technical skills, non-technical skills and preparation activities, each of
which is informed by the appropriate knowledge and reinforced by relevant personal
attributes. Figure 6.5 illustrates the reworking of the model.
A limitation of the current study is that there were only 15 concepts included in
the survey. The inclusion of additional concepts may have unnecessarily complicated
the resulting plots, but may also have added further explanation. It would be
interesting to observe how the inclusion of additional concepts would interact with
those presented in the current study. However additional concepts would add
considerably to the time taken by participants to compete the survey.
220
Figure 6.5. Revised model of airmanship
In general, the concepts selected for use in the survey were the most frequently
mentioned in each domain. As noted earlier, the exceptions were the technical skills
and personal attributes concepts. It is possible that slightly different choices
regarding the included concepts may have led to different findings for relatedness
and thus a different pattern in the MDS plots.
A further study could use one representative concept used in this study from
each domain (as anchor points for comparison) with a different selection of other
concepts from each domain to see if the basic structure determined in this study is
replicable and not dependent only on the specific concepts selected here.
221
Chapter 7. Airmanship Conclusions
The current project has involved four different investigations into the
understanding of airmanship among Australian aviators and the way in which
airmanship is trained in Australia. The starting point for the investigation was formed
by the models of airmanship found in the aviation literature. These are the Kern
(1996, 2009a), the Ebbage and Spencer (2003) and Edwards (2013) models of
airmanship.
The Kern model of airmanship (1996, 2009a) appears frequently in discussion
of airmanship among aviators. This model was developed from research with
aviators in the USA and biographies of noted aviators. Kern (1996) presented his
model of airmanship as an aide for learning in a textbook for ab-initio trainees, but
also outlined some ideas about the training of airmanship and made the suggestion
that ongoing self-assessment and striving for improved performance is all part of the
armoury of a good aviator. One short-fall of the Kern model is that it leaves out, or
hides amid knowledge items, some of the non-technical skills that become important
as an aviation career progresses. In addition, while noting the influence of context on
development and exhibition of airmanship, context is not included within the model
itself. The Ebbage and Spencer (2003) model of airmanship includes the non-
technical skills more overtly; some that Kern had included in the ‘knowledge of
team’ part of his model, plus some additional concepts relating to crew coordination
and self-assessment and improvement. The Ebbage and Spencer model was
developed in the UK based on the training of military pilots in Britain. The Edwards
(2013) definition of airmanship is concerned with the knowledge, skills and attitudes
associated with the three areas of technical, operational and non-technical elements
of aviation activity, and also discusses some other factors, including stress, fatigue
and managing automation. Presented as a guide book for pilots, this presentation
appears to be based on Edwards’ own experience and on a series of case studies of
aircraft accidents.
None of the above models presents any empirical evidence for the structure of
the models nor have they been formally evaluated. While there has been considerable
research into the components of airmanship, there has been little published research
222
on the overall concept of airmanship, nor any investigation into the definition and
training of airmanship in the Australian aviation industry.
In the current project, an initial on-line survey reached mainly general aviation
(GA) pilots. This provided a starting point for considering the match between the
existing models and the views of Australian aviators. Recognition that the survey had
only managed to reach a narrow sector of currently operating pilots led to the second
survey. The wider distribution of this expanded survey allowed comparison of
differences in views between military and civilian background aviators, and a better
idea of the relative importance placed by the participants on various concepts that
make up airmanship. However, the numbers were low when divided across any
further differentiation of sub-sectors within the industry. The third study involved the
interviewing of a cross-section of flight instructors and trainers from civilian and
military sectors, at ab-initio and advanced levels of training. The interview method
allowed greater detail to be garnered in responses to questions about the definition
and training of airmanship. Together the first three studies triangulated on the views
of airmanship held by Australian pilots. A model of airmanship was developed from
the results of these studies, and was tested in a fourth study. In this study,
participants rated the ‘relatedness’ of pairs of concepts from the model. These data
were then used in a multi-dimensional scaling exercise, which confirmed the basic
structure of the model and indicated the influence of the domains of knowledge and
personal attributes on the other domains of the model.
This chapter summarises the main findings of each study and outlines some
directions for future investigation.
Survey 1
The results of the first survey were discussed in Chapter 2. The questionnaire
included some demographic questions about the kind of licence held and flying
experience (in years and in hours) in both civilian and military sectors. The main part
of the questionnaire consisted of open ended questions about the participants’
definition of airmanship and about their experience of airmanship during their
training and during subsequent flying activities. The 40 participants made mention of
many of the components of the Kern model, the most frequently mentioned being
situational awareness and rules and regulations. The latter are part of Kern’s
223
‘Knowledge of the environment’ component. It was interesting that only a few
participants made specific mention of discipline, Kern’s cornerstone to good
airmanship. However some participants did note that there should be no short cuts or
cutting corners – implying a discipline to conform to procedures and tasks in full.
Communication was the most frequently mentioned of the Ebbage and
Spencer/Edwards additional concepts. Safety was included in responses by more than
half the participants and was the most frequently mentioned concept overall. The
existing airmanship models mention safety as an outcome of good airmanship but
safety is not specifically included in the models. Additional concepts raised by
participants that did not appear in any of the models included various attitudes and
personal qualities, lookout, courtesy and common sense.
With regard to the training of airmanship, most of the participants indicated
that there was some form of inclusion of airmanship in their initial training, and
many stated that airmanship was discussed in their ground briefings during initial
training. Conversely, this can also be seen as more than a quarter of participants
indicating that airmanship was not included in their initial training. Nearly half of the
participants mentioned both aircraft handling and non-technical skills as being part of
airmanship instruction. However some participants omitted any mention of aircraft
handling skills as part of airmanship (30%) while a few participants (12.5%) did the
reverse, and mentioned only aircraft handling and not human factors or non-technical
skills, when indicating what was included in initial instruction in airmanship.
Many participants indicated that the predominant influence on the development
of their airmanship was the influence of others (44% of total mentions). This
conclusion was echoed by the instructors among the respondents, half of whom
reported that they try to provide a good example of airmanship, in order to instil
airmanship in their students.
The main limitations of this study were the low participation rate (with only 40
respondents), the predominance of General Aviation pilots and lack of many airline
or military participants. However, it provided information that was used to expand
the questionnaire for a second survey.
224
Survey 2
The details of the second survey were reported in Chapter Three. The
questionnaire developed in Survey 1 was expanded to include additional
demographic questions to identify participants’ main area of activity within the
aviation environment. A rating question and a ranking question were also added.
Each of these questions listed the concepts included in the airmanship models as well
as some of the additional concepts raised in the responses to the first survey.
Participants were asked to rate each concept and then to rank the five most important
concepts. Participants included both military and civilian trained pilots, currently
flying in airlines, in general aviation and in the military. They formed two main
groups, one of participants who had only civilian experience (40 participants), and
the other those who had a background in the military but were currently flying in
either military or civilian environments (43 participants).
Definition of Airmanship.
Analysis showed that when asked about the definition of airmanship there were
significant differences in the frequency of mention of some concepts between
civilian and military background participants. Among the Kern model concepts,
military background participants mentioned knowledge of the aircraft, rules and
regulations, organisational factors (such as procedures and orders), aircraft handling
skill and proficiency significantly more frequently than did civilian background
participants. Military background participants mentioned common sense significantly
more frequently than did civilian background participants, but civilian background
participants mentioned courtesy or consideration for others significantly more
frequently than did military background participants. There were other concepts on
which the two groups differed, but the actual number of mentions of each concept
was too low to calculate significance.
Overall, in the open ended questions about the definition of airmanship, the
most frequently mentioned concept was safety (by 74.7% of participants), followed
by situational awareness and judgement/decision making, personal qualities,
knowledge of the aircraft and rules and regulations. The responses to the rating and
ranking questions reflected much the same ordering of concepts, however with some
interesting differences. Situational awareness was rated and ranked as most
225
important, and significantly ahead of the next closest rated or ranked concepts. There
were no statistically significant differences among the top-most rated, or ranked,
concepts after situational awareness. Judgement/decision making was ranked second,
and the mean rating of the concept placed it fourth. Discipline was 13th most
frequently mentioned in the open ended questions, but was third in the mean ratings
and ranked sixth. Common sense was 10th most frequently mentioned, but rose to 6th
in the ratings and 4th in the rankings. Courtesy was mentioned as frequently as
common sense overall, but was more frequently mentioned by civilian respondents.
Courtesy had the lowest rating and was ranked quite low (coming in 17th). Courtesy
was included in the “top 5” by only six respondents. Comments about preparation,
coded as mission planning in the open ended questions, were 17th in order of
frequency of mention, but preparation was rated and ranked higher up at 5th.
Communication was 8th in order of frequency of mention, but dropped to a mean
rating of 18th and mean ranking was 15th.
The changes in ordering may simply reflect the difference between free recall
and recognition, in that while some participants may not have mentioned some of the
concepts, once prompted by their presence in the supplied list they worked with the
additional concepts and ascribed importance to them regardless of whether they had
been mentioned earlier. While the difference in rating and rankings was statistically
significant across the range of 26 concepts, the degree of difference between closely
rated or ranked concepts is not significant.
The elements of the various models were well represented in the open ended
questions and performed well in the rating and ranking questions, with some aspects
being more highly rated and ranked than the frequency of mention in the open ended
questions may have appeared to predict, for example, discipline and
preparation/mission knowledge. Situational awareness is clearly seen as the most
important feature of good airmanship, being mentioned most frequently (after safety)
and rated and ranked most highly.
Training and Development of Airmanship.
In the analysis of the questions about training and development of airmanship,
the participants were separated into two groups based on their sector of initial
training (civilian or military). For those who were trained first as a civilian and then
226
in the military, if it was obvious they were writing about their military training they
were added to that group. If it was ambiguous, they were omitted from the analysis.
Participants were also separated by the era of their training (‘before 1991’ or ‘1991
and after’) to reflect the introduction of human performance and limitations to the
theory syllabus. This resulted in rather uneven groups, as there were only six military
participants who began their initial training in 1991or later.
The majority of participants indicated that airmanship or airmanship-related
concepts were included in their ground training or briefings, with no significant
difference between eras or training background. They then listed the concepts that
were discussed, with safety being most frequently mentioned, followed by situational
awareness, with no significant difference in frequency of mention between groups.
Decision making was significantly more frequently mentioned by military trained
participants. There was mention of knowledge, most often to do with rules and
regulations or procedures. Communication was more frequently mentioned by
civilian trained participants, however the number of mentions by military trained
participants was too low to allow a statistical comparison. Lookout and courtesy
were among the concepts most frequently mentioned by both civilian and military
trained participants.
When asked about inclusion of airmanship in their in-air flight training, slightly
more of the civilian participants trained before 1991 noted that airmanship was not
included, than did participants in the other groups, and the military participants
reported inclusion of airmanship significantly more frequently than did the civilian
participants. A variety of concepts were listed, most frequently situational awareness,
but not as many concepts as were mentioned in the previous question. Lookout was
more frequently mentioned by the military trained participants, the difference was
not quite significant, but had a low-medium effect size. Courtesy and safety were
only mentioned by civilian participants, and performance during emergency or
abnormal events and examinations were only mentioned by military participants.
Five participants related that airmanship was not taught, that it was gained by
experience or picked up as one progressed. A further four participants noted that
airmanship was not well defined or not a specific part of the training.
There was little mention of team related concepts or inter-personal skills
(except for the mention of courtesy). These are more related to multi-crew flying, so
227
their omission from mention when discussing ab-initio training is logical. The
participants did not mention self-improvement or self-assessment, nor was there
frequent mention of “knowledge of self” concepts apart from some comment on
being aware of one’s own limitations.
The participants were asked about the development of airmanship as they
progressed as pilots, and it is in response to this question that concepts related to self-
development arose more frequently than previously. Kern (1996) argues that ongoing
reflection on one’s own performance is an important part of maintaining airmanship,
and it appears that some of the participants in this survey agree. Experience was most
frequently mentioned in response to this question, and mentioned significantly more
frequently by military background participants than by civilian background
participants. Experience was generally mentioned in combination with other
concepts. Civilian participants slightly (not significantly) more frequently mentioned
various personal qualities reflecting an internal motivation such as self-development
or wanting to be “the best’. Third most frequently cited as a factor in the
development of airmanship was the example provided by others (whether good or
bad).
Current training. A question about current training participation showed no
significant differences in frequency of mention of specific concepts, between the
three groupings used for analysis of this question (current or most recent activity in
airlines, general aviation or military). Most participants reported engagement in on-
going training, and that airmanship as such is not generally included. However, some
noted that there is recognition that “airmanship” is an integral part of aviation
activity and is thus not separately addressed. Many of the respondents in airlines and
the military noted the relationship between crew resource management and human
factors training and airmanship. In addition, the lack of formal definition and
subjective assessment of airmanship was commented on by some airline pilots.
There were some minority views expressed, such as airmanship being all about
aircraft handling, or that it is equivalent to common sense, or that it cannot be taught
but depends upon experience.
Instructor practices. Respondents to this question were grouped by area of
current activity: airline, general aviation (GA) or military. Responses included
comment on how they impart airmanship and also about what it is that they are
228
concerned with developing. The airline and GA instructors most frequently
mentioned briefings and discussions as the mode of instruction, while the military
instructors mentioned provision of a good example most frequently, followed by
mention of the use of scenarios and then briefings; however there were no significant
differences between the groups. Airline instructors also wrote about the use of
scenarios in the training environment to cover a broad range of operations. There
were some comments about observing how the student handles events and when to
intervene should the situation deteriorate.
What is included in training of airmanship covered many of the familiar
concepts represented in the airmanship models, and some additional concepts. Safety
was most frequently mentioned, along with getting students to think, develop
decisions, and maintain situational awareness, as well as various aspects of
knowledge.
None of the respondents clearly articulated all three stages of development of
airmanship as presented in Kern (1996) or Ebbage and Spencer (2003) however the
components are there in many of the responses. The three stages are to: (a) explain
the importance of airmanship; (b) model and teach good airmanship; and (c) evaluate
performance and provide feedback. Few mentions were made about evaluation and
feedback, although there was a notable exception among the military instructors who
advised failing the students who did not grasp the importance of adherence to the
safe principles of flight.
Changes over time. The majority of respondents agreed that their ideas about
airmanship had changed over time or made some other comment that indicated a
maturing or development of airmanship. Only about a third of respondents indicated
that their ideas had not changed, and there were no significant differences across
respondent groupings. There was a variety of views expressed in response to the
questions, with some participants taking advantage of the survey to voice some
cherished complaints about the industry (regarding modern techniques and young
inexperienced instructors), while others were more positive. Change was mostly
ascribed to experience, to the introduction of CRM and HF training or to the
examples of others. Many respondents simply commented on general development,
increased awareness or developing a sense of the importance of airmanship, without
referring to any mechanism through which this might have occurred.
229
Concluding Comment
The majority of the participants undertook their ab-initio training before 1991,
so asking about their early training provides information that has to be considered
time-related. Of those respondents who were current instructors, not all answered the
question about current training practices, nor was it easy to ascertain whether they
were referring to ab-initio or more advanced training in that question. The inclusion
of airmanship in current training and the way in which this is carried out, may be
better considered by a more direct questioning approach, which was addressed in the
next part of the project: the interviewing of current instructors in military and civilian
sectors, at ab-initio and advanced levels of training.
Interviews with Current Instructors
Definition of Airmanship
The third study within the current project aimed to directly address issues
regarding the definition and training of airmanship with the people who are most
concerned with the initial training and ongoing development of Australia’s pilots: the
instructors employed in ab-initio and advanced training in civilian and military areas
of operation. While the participants in the two on-line surveys included instructors,
the questions about training were towards the end of the survey and resulted in a lack
of response or very brief response. Face-to-face interviews can elicit more detailed
responses to targeted questions about current practice. A series of interviews were
carried out with flight instructors at two civilian flying schools, at the basic flight
training school (BFTS) for the military with both military and civilian background
instructors, at a jet training conversion unit at Williamtown and at the military
Central Flying School (CFS) at East Sale, as well as among training captains
recruited from major Australian airlines. Thus the interviewees formed five groups:
civilian ab-initio trainers; civilian advanced trainers; a group with civilian
background currently employed to train in the military (civilian at BFTS); military
trainers at BFTS and military advanced trainers (at CFS and Williamtown).
Process comments. There was comment about the process of defining
airmanship, with many of the military and civilian advanced instructors saying that
airmanship is hard to define and all of the BFTS instructors and more than half of the
advanced instructors (military and civilian) indicating that airmanship is all-
230
encompassing or an integration of many skills. About a quarter of the participants
added that airmanship is unspecifiable in some way, being subjective, holistic,
esoteric or unquantifiable.
Initial definition. Among the 39 interviews, there were 25 different
combinations of concepts included in the initial definition of airmanship. There were
12 mentions of situational awareness among the various combinations, nine that were
based on the formal military definition (of “safe and efficient flight both in the air
and on the ground’), nine mentions of common sense and seven of decision making.
Model related concepts. The Kern model was mentioned by two participants
(both civilian advanced instructors) in response to the question about the definition
of airmanship, but neither based their definition on that model. A third participant
mentioned the model later in the interview, as not being very helpful. Despite this
lack of recognition, most of the concepts included in the Kern (1996, 2009a), Ebbage
and Spencer (2003) and Edwards (2013) models were included in the discussions of
airmanship by participants, although not all of them by any one participant.
Essentially, all participants agreed that situational awareness and decision making
were important to airmanship. Knowledge was also considered important, with
general comments made about knowledge requirements and knowledge of rules and
regulations, legislative environment, being the most frequently mentioned knowledge
related concept. Aspects of teamwork and crew related issues were also discussed.
Of these concepts, communication was most frequently mentioned within each
group.
Skill was a term included in one of the questions and the majority of
participants made some mention of technical flying skill in relation to airmanship.
Most of the instructors at BFTS (both civilian and military) specifically noted that
technical flying skill and airmanship were distinct, despite earlier noting that all the
concepts were interrelated. About half of the remaining participants included flying
skill as part of airmanship. Discipline was not widely mentioned as such, with overall
just over a third of participants making mention of discipline. However, if comments
that implied a need for discipline are included in the count, then two thirds of
participants mentioned or implied a need for discipline.
231
Additional concepts. The formal military definition of airmanship as “safe and
efficient flight, both in the air and on the ground” was used by the majority of the
military BFTS instructors and by half of the military advanced instructors. There
were also a few mentions of this definition by instructors in the other groups. It is a
useful short and snappy definition but one still needs to ask what makes a flight safe
and efficient. Safety was mentioned as part of airmanship by three-quarters of the
participants, most often by the ab-intio instructors. Personal qualities were also
frequently mentioned but this may be somewhat misleading as many different
qualities were coded within this node, such as honesty, integrity and commitment,
and also distractibility, complacency and over-confidence were mentioned as not
being conducive to good airmanship.
Preparation is assessed in the military training context and thus was mentioned
more frequently by military instructors (including the civilian BFTS group).
Experience was raised as important in airmanship by 61.5% of participants overall,
but much more frequently by military than civilian instructors. Attitudes were
mentioned, often in response to a specific question about “other skills, knowledge or
attitudes” that might be included in airmanship. Prioritising was also mentioned
across the instructor groups. Common sense was most frequently mentioned by
civilian advanced level instructors, with fairly evenly spread of mentions across the
other instructor groups.
Other concepts that were mentioned included: thinking, captaincy or command,
courtesy, foresight, threat and error management (TEM) and CRM, responsibility,
and information gathering. Participants also mentioned a number of actions or
activities that demonstrate good airmanship. These included: checks, lookout, radio
usage, weather awareness, and pre-flight inspections, amongst other things.
Group similarities and differences. There was consistency among the
participants concerning the importance of situational awareness and decision making,
and also knowledge of rules and regulations and operating procedures. Many
participants also referred to aspects of personality, which were coded as personal
qualities. There was a lack of mention of “knowledge of the team” across all groups
of participants, although skills relating to team-work were mentioned. The civilian
ab-initio group of instructors made more mentions of mission planning, self-
improvement and courtesy than did instructors from other groups. The civilian
232
advanced instructors made more comments about skills related to multi-crew
operations including communication, crew coordination, interpersonal skills,
management of the flight deck, team leadership and threat and error management.
They also commented most on thinking, attitudes and common sense. They made
fewest mentions of aircraft knowledge and mission planning, perhaps because of the
simulator environment in which they work.
Each of the instructor groups tended to comment most about aspects of
airmanship most relevant to their operating environment, thus the civilian advanced
group spoke about multi-crew skills, the BFTS groups about preparation and
discipline, and the civilian ab-initio group about planning and courtesy.
Comparison with survey results. The ordering of concepts by frequency of
mention for the interview data and the results of the second survey led to very similar
lists (see Table 4.8). There are some differences in ordering, but only by a few places
either way. Nineteen of the top 25 concepts appeared on both lists, those concepts
appearing in the “top 25” of only one list, appeared lower down the rankings in the
other list. Situational awareness, decision making and safety were at the top of both
lists. The “personal qualities” node was high on both lists, the node included many
kinds of qualities and the number of comments may indicate a concern about the type
of person an aviator is. Near the top of both were comments about the legislative and
organisational environment (rules and regulations and operating procedures).
The consistency across the two data collection methods and sets of questions
used indicates the robust nature of the idea of airmanship among Australian aviators.
Comparison with the models of airmanship. The main concepts included in
the Kern model of airmanship were mentioned frequently by the participants. The
interview data also suggests that skills relevant to multi-crew environments are
important aspects of airmanship for aviators in that more advanced environment, but
these skills were not at the forefront of the Kern model. Communication, especially,
was mentioned frequently by participants from all groups, but in the Kern model is
simply a part of “knowledge of the team’. There was also a sense that the application
of knowledge is important, knowing when and how to use relevant knowledge was
emphasised by some participants. Discipline was expressed as compliance with rules,
regulations and procedures, and was brought up rather more frequently than in the
233
open-ended survey questions. There was some confusion over whether technical
flying skills are part of airmanship. In terms of an overall discussion, most
participants included flying skills as part of a global integrated view of airmanship.
However, military instructors (and civilians working in the military BFTS
environment) later excluded technical flying skills from airmanship, in the sense that
they assess airmanship and flying skills separately, and from that position, define
airmanship as everything else except flying the aircraft.
Conclusion – definition of airmanship. The interviews indicated that while
there are many differences in initial definition of airmanship, after some discussion,
the same main themes emerged as important components. Not all of these main
themes are well represented by the Kern model alone.
Interviews: Training and Development of Airmanship
Having covered the definition of airmanship, the interviews then addressed the
way in which airmanship is trained and developed at the different levels of
instruction represented by the participants.
Airmanship in the training program structure.
The most interesting finding was that just over half of the advanced civilian
and advanced military instructors stated that at that level, airmanship is assumed to
be adequate and so little additional attention is required. The majority of instructors
at the ab-initio level stated that they talk about airmanship in the pre-flight briefing,
and that the briefing is a standard format. In civilian advanced training, it was noted
that the term airmanship is not used, rather “non-technical skills” is used. The
military advanced instructors were divided: the jet-conversion instructors responded
in a similar fashion to the ab-initio instructors, while among the instructors at CFS
the general opinion seemed to be that not much time was spent on development of
airmanship, rather emphasis was placed on how to present a good example to future
students.
Learning from others, providing a good example to others and ensuring there is
a positive supportive organisational culture was most often mentioned by the civilian
advanced participants, and also by the BFTS instructors and military advanced
participants, and not so often by the civilian ab-initio group.
234
Initial airmanship training. After noting that airmanship is introduced from
the first pre-flight briefs, the participants from the different ab-initio instructor
groups all talked about lookout being the most important initial airmanship related
skill to be taught. They also mentioned a variety of different aspects of airmanship
that are introduced in the first few lessons. Most frequently and fairly consistently
mentioned across groups were: listen out, or radio usage; checks and checklists; how
to taxi; and knowing aircraft parameters and limitations. Some of the differences
between the groups seem to be mainly in the terminology used by civilian and
military groups, such as: flight planning versus preparation and procedures versus
work cycles.
Assessment of airmanship. In the civilian ab-initio context, airmanship is only
informally assessed until the formal flight test for a licence, and only insofar as
elements of airmanship are included in the competencies assessed. Several of the
civilian advanced-level instructors stated that they do not use the term airmanship,
rather assessment is divided into technical and non-technical skills. Non-technical
skills include situational awareness, decision making, communication, leadership and
automation management, and technical skills cover aircraft handling and automation
management skills. However it was explained that pilots are not failed on non-
technical skills alone, but only if such problems are associated with a technical
handling skill problem.
By contrast, the military related groups (BFTS civilian and military
participants and the non-CFS military advanced participants) were clearer about
assessment of airmanship as a separate fail-able item and mentioned the word-
pictures that form a rubric for assessment. However, it should again be noted here
that the military definition of airmanship for assessment purposes is everything
except preparation and aircraft handling – so essentially consisting of non-technical
skills. The military advanced trainers at CFS were reluctant to say that they assess
the airmanship of their flight instructor candidates (FICs), noting their experience
and existing high levels of airmanship. It was clear that they separated the technical
handling skills from “airmanship” in that the FICs have to relearn the specific
technical handling requirements that CFS include in the basic flying syllabus, but
that airmanship (excluding flying skills) is assumed to be fine and discussion relates
more to how the FIC will imbue their future students with this aspect of airmanship.
235
Training process. Among the civilian advanced and civilian BFTS instructors
comment was made that that the instructor’s role is to correct students and suggest
fixes for any problems they have in the air. Many of the participants indicated that
instructors should demonstrate exemplary airmanship themselves and must be ready
to take over from their students if necessary. There was concern expressed across all
groups of instructors about how long to allow students to continue along an error
path before stepping in and taking over.
Instructor training. Participants mentioned that airmanship is not explicitly
included in the civilian instructor rating, candidates simply have to exhibit good
airmanship during the instructor rating flight test. The military BFTS instructors did
not recall that airmanship was a major part of the CFS instructor training program
and the military advanced instructors at CFS agreed with this impression. Their
position is that flight instructor candidates already have good airmanship and to have
it as part of the training is unnecessary especially as there is limited time available in
the program. Again this is taking the view that airmanship excludes technical flying
skills, which are focussed on in the CFS program.
Comparison with model recommendations. The components of instruction
in airmanship revealed by the participants do not entirely reflect the three phase
process for instruction of airmanship presented in the Kern (1996) and Ebbage and
Spencer (2003) models of airmanship. The first step in each model is to instil the
importance of airmanship. It is not clear whether this is addressed as such either at
ab-initio or advanced levels of instruction in civilian or military environments.
The second phase of the suggested process of training airmanship specifies the
teaching and modelling of airmanship. The ab-initio civilian and military programs
include this stage, with airmanship (and technical skills) being included in briefings
and debriefs, and with many of the instructors noting their intent to provide a good
example. The comments by at least half the advanced level instructors that
airmanship is “assumed” at their level of training indicates that perhaps the active
inculcation of airmanship does not occur at this level. Other civilian instructors at
this level stated that technical aircraft handling and non-technical skills are instructed
and assessed, while the military advanced CFS instructors focussed more on how to
instruct future students rather than improvement of the FIC’s airmanship.
236
The third phase of the process is assessment and feedback. The military BFTS
and advanced conversion programs (non-CFS) provide feedback in debriefs and
include formal assessment of preparation, technical flying skill and what they call
airmanship (everything else, or non-technical skills) which can be failed, thus
meeting Kern’s (1996) requirement of aggressive evaluation. However the CFS
program for development of future instructors did not appear to have quite so overt
an assessment process for airmanship, being concerned more with instructional
abilities. Civilian ab-initio programs, while they may not have an internal pass/fail
evaluation process, do ensure that their students meet CASA requirements for
technical and non-technical skill, which are assessed in licence flight tests. At the
advanced civilian level, it appears that airmanship is assessed, but that poor non-
technical skills are only critical (for failure) if paired with a technical skill problem.
Conclusion – training and development.
It was interesting, because unexpected, that when asked about training of
airmanship the initial reaction of about half of the advanced level instructors was to
state that airmanship is assumed to already have been acquired, or already exists at a
high standard. Further discussion indicated that civilian advanced training covers
technical and non-technical skills, but that neither is referred to as “airmanship”.
There were some similarities in the process of instilling airmanship discussed
by the ab-initio instructors in military and civilian environments. Commonalities
included the use of standardised briefings and the skills that are introduced in initial
lessons. Structured ground school instruction and formal assessment of flying
performance were features of the military program that were not reflected in the
civilian programs (of the flying schools sampled). Participants from all instructor
groups were concerned about providing a good example to students and about when
to intervene when errors occurred.
The three phase training structures proposed by Kern (1996) and Ebbage and
Spencer (2003) were not fully reflected in the comments made by participants. The
military BFTS program is closest, having specific training and evaluative assessment
of both technical and non-technical skills. It does not appear common to use
airmanship as a structure to relate knowledge acquisition, skill development and
237
personal attributes to ongoing training and flying efficiently and safely, as Kern
(1996) proposed.
The Structure of Airmanship
A discussion of the similarities and differences between the findings in the first
three studies and the research reported in the literature that takes an over-view of
pilot performance is presented in Chapter 5. The concepts raised by the participants
were categorised into five principle domains of airmanship: knowledge, preparation,
non-technical skills, technical skills and personal attributes. These form a model of
airmanship that also acknowledges that airmanship occurs in a context of experience,
organisation and regulation.
Airmanship Concept Domain Testing
An examination of whether Australian aviators consider airmanship concepts
relate to each other in terms of the airmanship model is reported in Chapter 6. Pairs
of representative airmanship concepts formed a 125 item questionnaire, in which
participants were asked to rate the relatedness of each pair of concepts on a 5-point
Likert scale. Participants were 87 Australian (or Australian based) pilots. Overall,
pairs of concepts from the same domain (within-domain pairs) were rated as more
closely related than pairs of concepts from different domains (between-domain
pairs). The pairs formed with the knowledge domain concepts did not fit this pattern.
A multi-dimensional scaling analysis using the relatedness data indicated, in a 3-
dimensional solution, that concepts from the domains of preparation, non-technical
skills, technical skills, and personal qualities clustered together. The knowledge
concepts were separated, with aircraft type knowledge falling close to the technical
skills concepts and the other knowledge concepts (knowing SOPs and knowledge of
rules and regulations) appearing closest to the personal attribute concept of
discipline.
The relatedness of the concepts was interpreted as showing the functional
approach taken by the participants, looking at how each pair of concepts might relate
to flying tasks. The model presented in the previous chapter was reworked slightly,
not to change the domains themselves, but to indicate that the three main ‘activity-
238
related’ domains of airmanship, preparation, non-technical skills and technical skills,
are influenced by the underlying knowledge and personal attributes of the pilot. The
rearranged model was presented in Figure 6.5.
Definition of Airmanship
The first aim of the present research was to redress the lack of empirical
evidence regarding aviators’ views of airmanship and to develop an overview of
what it is that Australian aviators consider “airmanship” to be. The surveys and
interviews carried out during this research have provided considerable evidence of
Australian pilots’ ideas about airmanship.
The outcome of the present research is that in the general sense airmanship is a
fairly loosely defined term that refers to the ability to combine together all the skills,
attributes and knowledge areas required to fly an aircraft safely, efficiently and
effectively. The present research has demonstrated that Australian aviators include a
diverse selection of skills and attributes when asked to define airmanship, but that
there are some strong common concepts that were raised repeatedly. The findings of
the current research were used to support the development of a model of airmanship.
This model was then tested and found to be a reasonable representation of how
Australian aviators consider component concepts of airmanship to be related. The
model includes three main domains of airmanship activity, namely preparation, non-
technical skills and technical skills. These domains are influenced by knowledge and
personal attributes. The model also allows for the exercise of airmanship to be
considered within a context, aspects of which include personal experience and
organisational and regulatory influences.
The model of airmanship presented in Chapter 6 (the ‘Carrick Model’ for ease
of reference) groups concepts into representative domains, rather than trying to
specify everything. The Kern model (1996, 2009a) tries to define and include every
aspect of knowledge, leading to knowledge visually dominating the model design.
The ‘Carrick model’ groups personal attributes into one domain, in contrast to both
Kern and Ebbage and Spencer (2003) who both concentrate on discipline to the
exclusion of other attributes. This research did not find such an emphasis on
discipline, although it was considered important when suggested to participants in
239
the survey rating and ranking questions. Personal attributes have been included by
Edwards (2013), although called ‘attitudes’, in each of the elements of his model of
airmanship.
The ‘Carrick Model’ tries to form a balance between the major domains of
preparation, technical and non-technical skills without separately specifying certain
skills at the expense of others. For example, the Kern model places situational
awareness and judgement at the peak of the model as capstone outcomes, and other
non-technical skills such as communication and crew coordination are relegated to a
knowledge column. The Edwards (2013) model also has three main elements
including technical and non-technical skills. Edwards’ third element of ‘operational
elements’ is similar in content to the ‘preparation’ domain of the ‘Carrick model’, in
covering planning, but suggests that knowledge of other areas of operations, such as
ATC, should be acquired.
Mavin and Roth (2014) separate out the non-technical skills of situational
awareness, decision making, communication and management as separate elements
of their model, and also include as elements technical skills (as flying within
tolerances) and aviation knowledge. They do not include personal attributes as part
of their model, nor do they appear concerned with preparation and planning. Their
model is intended for specific use in the assessment of pilots within airlines
undertaking upgrade training (Mavin, 2010).
Nergård et al (2011) does not formulate a model of airmanship, but does
present the outcome of a principal components analysis based on responses to the
questions that his participants in an earlier study helped to frame. Some of these are
phrased as having to do with a ‘good pilot’ and others relate to ‘crew’. The four
factors are labelled: ‘principles in airmanship knowledge’ (although this includes
items relating to initiative, role modelling and monitoring for stress and fatigue);
self-awareness (which includes further personal attribute items); flying skills; and
CRM (covering non-technical skills). Nergård essentially covers four of the domains
that the current research found, (although his ‘knowledge’ factor included many non-
knowledge items), however does not consider preparation, and neither is context
considered.
240
The concepts that were raised by the participants in the current research were
generally the same concepts that have been considered in research and theory
throughout the literature, especially the non-technical skills that have become well
established in the industry through CRM courses and the HPL syllabus and listed as
essential for good threat and error management (Helmreich, 2002; Thomas, 2005). In
addition to these concepts, two ‘new’ concepts not mentioned in other research were
found: common sense and courtesy. Courtesy was mostly raised by pilots who need
to interact with other airspace users in unstructured environments (i.e. general
aviation) and these pilots may have more need to moderate their communication and
maintain working relationships outside their immediate crew or organisation.
Courtesy was generally mentioned in the context of communication or helping
others. Common-sense tended to be raised by more experienced pilots, not always in
a context that led to easy interpretation. This researcher’s interpretation is that these
pilots have been operating in aviation for so long that airmanship has become
ingrained, they no longer remember having to learn it all, and thus it becomes
‘common sense’ to them. It is not, however, a helpful term to use with less
experienced pilots, (neither is saying that airmanship depends on experience) as such
terms may be interpreted to mean that airmanship being something that will develop
naturally with time rather than something that can be actively learned.
The current research has demonstrated the awareness of airmanship among
Australian aviators and the variety of concepts that they feel are related to it. A
model of airmanship that better represents the concepts of airmanship and their
interrelatedness has been developed, presented and tested.
Training of Airmanship
The second aim of the current research was to investigate whether and how
airmanship is taught. Both Kern (1996) and Ebbage and Spencer (2003) suggest a
three phase approach to ensuring that good airmanship develops in pilots under
instruction. First, explain and promote the concept of airmanship; secondly, instruct
and model airmanship; and thirdly, evaluate and assess airmanship, and provide
feedback to the student or trainee.
241
Some of the participants in the survey indicated that airmanship was not part of
their initial instruction, but the majority indicated that it was. All of the ab-initio
instructors interviewed indicated that they did include airmanship in their briefings
before flights, instructed and demonstrated airmanship concepts in the air and
expected airmanship to be displayed by their students. It was interesting to repeatedly
hear from aviators at more advanced levels of training, that airmanship itself is
“assumed at this level”. That is, if the aviator does not have good airmanship he or
she would not have made it (somehow) to advanced levels of training. Despite that
statement, both non-technical skills and technical handling aspects of flying make up
the training process at advanced levels. At the ab-initio level, the instructors stated
that they ensure that airmanship is briefed, instructed and is well practiced, and also
they are actively aware of providing a good example. Most the components of
airmanship appear to be addressed in current training programs (apart from, perhaps,
the personal qualities raised by some participants), and what is introduced is
consistent across military and civilian ab-initio training. Thus the second phase of the
recommended sequence is currently being met.
Assessment of airmanship varied in intensity between the military and civilian
instructors at the ab-initio level. The military programs allow for regular assessment
and the possibility of failing, whereas this appeared to occur only at the stage of the
licence flight test for civilian trainees. In both sectors and at both ab-initio and
advanced levels, feedback is provided in post flight debriefing. This goes some way
towards meeting the third stage of the suggested sequence, although the civilian
process may not meet the suggestion that assessment be evaluated aggressively
(Kern, 1996).
The first stage in the instructional process is the presentation of the importance
of airmanship. This may well occur, but there does not appear to be any widespread
overt approach to using a model or structure that attempts to explain airmanship
conceptually as well as practically. The use of such a structure would provide the
trainee with a framework on which to build skills, knowledge and practices as his or
her career evolves. Only one respondent mentioned doing this, using the Kern model
for the purpose. The model of airmanship presented in Chapter 6 may provide an
alternative structure for this purpose.
242
Limitations of the Current Research
The first survey was limited by the low numbers of participants and their
homogeneity, as most were general aviation pilots. The second survey redressed this
issue in that there were more participants and this included general aviation, airline
and military pilots. However most of the participants began their training before
1991, especially those with a military background. This was an advantage in that the
participants were all very experienced, but does not give an up to date picture of
airmanship training, or how more recent training might influence the definition of
airmanship. The nature of the survey meant that it was difficult to undertake
statistical comparisons between groups, because the frequency of mention of some
concepts was too low. However, qualitative evaluation of the comments made by the
participants did allow a picture of the general structure of airmanship to be
determined. As with all such surveys, there is a possibility that the people who chose
to respond are not necessarily representative of the population of pilots, in this case
being perhaps more open to participating in research run from a School of
Psychology.
The third study in the present research was to interview a number of current
flight instructors and check and training captains. Again, data did not allow for
quantitative analysis, but there was more than sufficient information for a qualitative
assessment of the research questions. The civilian participants may not have been
representative of the entire civilian industry as it was difficult to access participants
from a wide range of organisations. In hindsight, it would have been interesting to
have administered the rating and ranking questions from the second survey, to the
interviewees in the third study. This would have allowed a more quantitative
comparison to be made between the groups within the interview study and between
the interview and survey participants.
The final survey to test the structure of the ‘Carrick Model’ of airmanship also
had a somewhat limited number of participants. However, this was adequate for the
statistical analyses employed.
Future Research
The revised model of airmanship developed from the present research could be
further tested for validity, one method of doing this would be to consult a selection of
243
expert aviators about the relevance and applicability of the revised model. In addition
the acceptability of the revised model among aviators in general could be assessed.
This would take the form of asking pilots about how appropriate the revised model is
as a representation of airmanship and about their preference between the revised
model and existing models. Questions could also be asked about how useful they
think the revised model might be for self-development and for use in flight training.
Assuming that the response to the revised model of airmanship is positive, it
could then be promoted for use in flight instruction. The model provides a structure
that would encourage the conceptual development and integration of airmanship
among flight students as they develop their flying skills and study for theory
examinations.
One possible line of future research could be a cross-cultural comparison of
concepts of airmanship. The revised model presented here is based on comment from
Australian pilots. Existing models were developed in the USA and the UK. Pilots
from other cultures may have different views on airmanship and it would be useful to
compare the results of surveys run in other countries with the Australian findings.
Future research could use a similar approach using on-line surveys with a
combination of open-ended and rating and ranking questions, to determine concept
structures around professionalism in other high-risk/high-reliability environments.
Concepts similar to those that make up airmanship are being introduced into aircraft
maintenance as “maintenance professionalism”, but apparently without any
theoretical or conceptual structure. An investigation of aircraft maintainers’ thoughts
regarding maintenance professionalism could result in a useful model. Airmanship
parallels the idea of seamanship, although there are obvious differences between the
two environments. Determining the main concepts that define seamanship would
highlight similarities and differences in the expression of professionalism in a marine
environment compared with aviation.
Conclusion
The aim of this research was to examine what Australian aviators consider to
be the definition of airmanship and whether there are any differences between
military and civilian pilots, or between military and civilian instructors at ab-initio
and advanced levels of training. There were few differences in frequency of mention
244
of concepts between military and civilian pilot participants in the survey study. There
were some differences in the frequency of concepts mentioned, between the flight
instructors who were interviewed, but numbers were too low to conduct statistical
tests. The civilian advanced level instructors made more mention than did other
instructors, of concepts relating to multi-crew environments, such as communication,
crew coordination, leadership and management of the flight-deck. The ab-initio
instructors at BFTS (both military and civilian) made more mentions of safety,
captaincy and the application of knowledge, than did the other instructors.
Differences are related more to sphere of usual operation than to military or civilian
background.
The first three studies led to the proposal of a model of airmanship based on
the concepts that were raised by participants when asked about the definition of
airmanship. The model presents concepts grouped into domains of knowledge,
preparation, non-technical skills, technical skills and personal attributes, and also
makes note of the effect of the context in which airmanship is developed and
expressed. A further study confirmed the domains and also indicated the influence of
knowledge and personal attributes on the other three domains. The model was
revised slightly to reflect this influence. The use of the revised model to expressly
embed airmanship in training programs may alleviate the paradox of the use of the
term ‘airmanship’ both as a global expression of safe and efficient flight and also to
describe only the non-technical skills component, in some training environments.
A second intent of this research was to investigate the extent to which
airmanship is included in civilian and military aviation training in Australia, at ab-
initio and at more advanced levels of training. At ab-initio level, the kinds of
airmanship skills that are first introduced were very similar between civilian and
military instructors, and both groups noted the use of standard briefings that include
airmanship pointers. The military training program considers the assessment of
airmanship to cover non-technical skills, as distinct from preparation and technical,
aircraft handling skills. This was not so clear cut among the civilian instructors, and
it appeared that little formal assessment is made, except insofar as the student is
prepared for a flight test. The main difference between the two levels of training is
that about half of the advanced level instructors stated that airmanship is assumed at
that level. The advanced level instructors went on to describe training that, among
245
the civilians included both non-technical and technical skills, and among the military
instructors at the Central Flying School, concentrated on technical skills. At the jet
conversion unit, both non-technical (airmanship) and technical skills were addressed.
Only one of the instructors interviewed in the third study noted the use of a model to
explain the structure of airmanship and how their training related to the development
of such skills. It is suggested therefore that it may be worthwhile to introduce the
model of airmanship developed from the present research to flying instructors and
trainers as a useful tool for that purpose. As eluded to above, this might also solve
the problem of military instructors use of airmanship both as a global all-inclusive
term and also to refer to non-technical skills when assessing performance.
There are a variety of alternatives for the continuation of this line of research.
The revised model of airmanship requires further investigation to establish that it is
representative and useful. Looking at professionalism in other sectors of the aviation
industry or in other environments may draw useful parallels, as well as providing
potentially useful models of professionalism for specific industries.
246
247
References
Ali, S. F., Gluckenberger, D., Rossi, M. & Williams, M. (2000). Evaluation of above
real-time training and self-instructional strategies for airmanship tasks on a
flight simulator. USAF Research Laboratory Report ADA 391561.
Downloaded from:
http://oai.dtic.mil/oai/oai?verb=getRecord&metadataPrefix=html&identifier=A
DA391561
Allen, P. & Bennett, K. (2010). PASW statistics by SPSS, a practical guide.
Melbourne Aus: Cengage Learning.
BAE Systems. (2006). Airmanship ASH 01 Intro. Power point training aid,
Tamworth, Australia: BAE Systems.
Bartholomew, D.J., Steele, F., Moustaki, I. & Galbraith, J.I. (2002). The analysis and
interpretation of multivariate data for social scientists. Boca Raton, FL:
Chapman & Hall/CRC.
Bartram, D. & Baxter, P. (1996). Validation of the Cathay Pacific Airways selection
program. The International Journal of Aviation Psychology, 6(2), 149-169.
Bond, N.A., Bryan, G.L., Rigney, J.W. & Warren, N.D. (1968). Aviation psychology.
Los Angeles, CA: USC Press.
Borg, I & Groenen, P.J.F. (2005). Modern multidimensional scaling: Theory and
applications (2nd Ed). NY: Springer.
Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative
Research in Psychology, 3(2), 77 – 101.
Braun, V., Clarke, V. & Terry, G. (2015). Thematic analysis in P. Rohleder & A.C.
Lyons (Eds) (2015). Qualitative Research in Clinical and Health Psychology.
(pp. 95 – 113) NY: Palgrave McMillan.
Brazill & Grofman (2002). Factor analysis versus multi-dimensional scaling: binary
choice roll-call voting and the US Supreme Court. Social Networks, 24, 201-
229.
248
Cahill, J. McDonald, N. & Losa, G. (2013). Understanding and improving flight
crew performance of the preflight, flight planning and briefing task. The
International Journal of Aviation Psychology, 23(1), 27-48.
Campbell, J.S., Castenada, M. & Pulos, S. (2010). Meta-analysis of personality
assessments as predictors of military aviation training success. The
International Journal of Aviation Psychology, 20(1), 92-109.
Carretta, T. R. (2011). Pilot Candidate Selection Method: Still an effective predictor
of US Air Force pilot training performance. Aviation Psychology and Applied
Human Factors, 1, 3–8.
Carretta, T.R., Teachout, M.S., Ree, M.J., Barto, E.L. King & Michaels, C.F. (2014).
Consistency of the relations of cognitive ability and personality traits to pilot
training performance. The International Journal of Aviation Psychology, 24(4),
247-264.
Carrick, K. (2009, June). Human factors as a component of airmanship. Paper
presented at the 8th Industrial and Organisational Psychology Conference,
Sydney, Australia.
Carrick, K., Graham, S., Healey, A. & Pickard, S. (2008, April). An assessment of
feedback and evaluation in a flight training school. Paper presented at the 8th
International Symposium of the Australian Aviation Psychology Association,
Novotel Brighton, Sydney, Australia.
CASA (2003). Human factors and crew resource management training (AC121A-
09). Canberra, Aus: CASA.
CASA (2006). Flight instructor manual aeroplane. Canberra, Aus: CASA.
CASA (2008). Teaching and assessing single-pilot human factors and threat and
error management. (CAAP 5.59-1(0)). Downloaded
from: http://www.casa.gov.au/wcmswr/_assets/main/download/caaps/ops/5_59
_1.pdf
Corbin, J. & Strauss, A. (2008). Basics of qualitative research: Techniques and
procedures for developing grounded theory (3rd Ed). Thousand Oaks, CA:
Sage.
249
Costa, P.T. Jnr. & McCrae, R.R. (1989). The NEO Personality Inventory Manual.
Odessa, Fl: Psychological Assessment Resources.
Damitz, M., Manzey, D., Kleinmann, M. & Severin, K. (2003). Assessment center
for pilot selection: Construct and criterion validity and the impact of assessor
type. Applied Psychology: An International Review, 52(2), 193-212.
Damos, D. (2014). Editors preface to the special issue on pilot selection. The
International Journal of Aviation Psychology, 24(1), 1-5.
de Crespigny, R. (2012). QF32. Sydney, Aus: Macmillan.
Ebbage, L. & Spencer, P.D. (2003, October). Airmanship training for modern
aircrew. Paper presented at the RTO HFM Symposium on Advanced
Technologies for Military Training, Genoa, Italy, RTO-MP-HFM-101.
Downloaded from http://www.skybrary.aero/bookshelf/books/1131.pdf
Edwards, C. (2013). Airmanship (2nd Ed). Blacker Ltd.
Elms, G. (2007). GAPAN Press release. Downloaded from
http://www.atsb.gov.au/media/12581/gapan_press_release_20_08_07.pdf
Endsley, M.R. (1999). Situation awareness in aviation systems. In D.J. Garland, J.A.
Wise, & V.D. Hopkin (Eds) Handbook of Aviation Human Factors. Mahwah,
N.J.: Erlbaum.
Flin, R & Martin, L. (2001). Behavioural markers for crew resource management: A
review of current practice. International Journal of Aviation Psychology, 11,
95-118.
Flin, R. Martin, L., Goerters, K., Hoermann, J., Amalberti, R., Valot, C. & Nijhuis,
H. (2003). Development of the NOTECHS (non-technical skills) system for
assessing pilots’ CRM skills. Human Factors and Aerospace Safety, 3, 95-117.
Flin, R., O’Connor, P. & Crichton, M. (2008). Safety at the Sharp End: A guide to
non-technical skills. Aldershot, UK: Ashgate.
Flin, R., O’Connor, P. & Mearns, K. (2002). Crew resource management: Improving
team work in high reliability industries. Team Performance and Management
8(3/4), 68-78.
250
Gray, D.E. (2009). Doing research in the real world (2nd Ed). Thousand Oaks, CA:
Sage.
Hawkins, F.H. (1987). Human factors in flight. Aldershot, UK: Ashgate.
Hayward, B.J. & Lowe, A.R. (2010). The migration of crew resource management.
In B.G. Kanki, R.L. Helmreich & J. Anca (Eds.), Crew Resource Management.
(pp. 137-342). San Diego, CA: Academic Press.
Helmreich, R.L. (2002). Culture, threat and error: Assessing system safety. In Safety
in aviation: the management commitment: Proceedings of a Conference.
London, UK: Royal Aeronautical Society.
Helmreich, R. L., & Foushee, H. C. (1993). Why Crew Resource Management?
Empirical and theoretical bases of human factors training in aviation. In E.
Wiener, B. Kanki, & R. Helmreich (Eds.), Cockpit Resource Management (pp.
3-45). San Diego, CA: Academic Press.
Helmreich, R. L. & Foushee, H. C. (2010). Why CRM? Empirical and theoretical
bases of human factors training. In B.G. Kanki, R.L. Helmreich & J. Anca
(Eds.), Crew Resource Management. (pp. 3-58). San Diego, CA: Academic
Press.
Helmreich, R.L., Klinect, J.R. & Wilhelm, J.A. (1999). Models of threat, error and
CRM in flight operations. In Proceedings of the Tenth International
Symposium on Aviation Psychology. Columbus, OH: The Ohio State
University.
Helmreich, R.L. & Merritt, A.C. (1998). Culture at work in aviation and medicine:
National, organisational and professional influences. Aldershot, UK: Ashgate.
Helmreich, R.L., Merritt, A.C. & Wilhelm, J.A. (1999). The evolution of crew
resource management training in commercial aviation. International Journal of
Aviation Psychology, 9, 19-32.
Hoerman, H-J. & Goerke, P. (2014). Assessment of social competence for pilot
selection. The International Journal of Aviation Psychology, 24(1), 6-28.
Hsieh, H-F. & Shannon, S.E. (2005). Three approaches to qualitative content
analysis. Qualitative Health Research, 15(9), 1277 – 1288.
251
Hunter, D. & Burke, E.F. (1994). Predicting aircraft pilot-training success: A meta-
analysis of published research. The International Journal of Aviation
Psychology, 4(4), 297-313.
IBM (2012). PASW20 Statistics package for the social sciences.
ICAO, (1989). Human factors digest No. 1: Fundamental Human Factors Concepts.
(Circular 216). Montreal, CA: Author.
Jones, D.G. & Endsley, M.R. (2004). Use of real-time probes for measuring situation
awareness. International Journal of Aviation Psychology, 14, 343-367.
Kaempf, G.L. & Klein, G. (1994). Aeronautical decision making: The next
generation. In Johnston, N., McDonald, N. & Fuller, R.
(Eds), Aviation psychology in practice. (pp. 223-254). Aldershot, UK: Avebury
Technical
Kanki, B.G. (2010). Communication and crew resource management. In B.G. Kanki,
R.L. Helmreich & J. Anca (Eds.), Crew Resource Management. (pp. pp. 111-
146). San Diego, CA: Academic Press.
Kern, T. (1996). Redefining airmanship. New York; McGraw-Hill.
Kern, T. (1997). Flight discipline. New York; McGraw-Hill.
Kern, T. (1999). Darker shades of blue: The rogue pilot. New York; McGraw-Hil.l
Kern, T. (2009a, June). CASA threat and error management roadshow. Brisbane,
Australia.
Kern, T. (2009b). Blue threat: Why to err is inhuman. Monument, Co: Pygmy Books.
King, R.E. (2014). Personality (and psychopathology) assessment in the selection of
pilots. The International Journal of Aviation Psychology, 24(1,) 61-73.
Krippendorf, K. (2013). Content Analysis: An introduction to its methodology (3rd
Ed). Thousand Oaks, CA: Sage.
Liamputtong, P. (2013). Qualitative research methods. Melbourne, Aust: Oxford.
Lincoln Y.S. & Guba, E.G. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage.
252
MacLeod, N. (2005). Building safe systems in aviation: A CRM developer’s
handbook. Aldershot, UK: Ashgate.
Mané, A. (1981). Airmanship: an introduction. In Proceedings of the First
International Symposium on Aviation Psychology. Columbus, OH: The Ohio
State University.
Marchbank, M. (Nov-Dec 2008). Is airmanship dead? Flight Safety Australia.
CASA.
Margerison, C.J., McCann, D.J. & Davies, R. (1986). Human resource management
for TAA pilots and flight engineers. Asia-Pacific Journal of Human Resources,
24(2), 32-37.
Martinussen, M. (1996) Psychological measures as predictors of pilot performance:
A meta-analysis. The International Journal of Aviation Psychology, 1(1), 1-20.
Maurino, D. (1994). Foreword. In N. Johnston, N. McDonald & R. Fuller (Eds)
Aviation psychology in practice. (pp. xv-xx). Aldershot, UK: Ashgate.
Mavin, T. J. (2010, Nov). Challenges for training and checking. Paper presented at
the Pacific and Asia CRM Developers, Evaluators and Facilitators Forum,
PACDEFF-2010, Adelaide, Australia.
Mavin, T.J. & Roth, W-M. (2014). A holistic view of cockpit performance: An
analysis of the assessment discourse of flight examiners. The International
Journal of Aviation Psychology, 24(3), 210-227.
Mayring, P. (2000). Qualitative content analysis: Theoretical foundation, basic
procedures and software solution. Qualitative Social Research, 1(2), 1 – 142.
Retrieved from http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395173
Nergård, V. (2014). Airmanship – a qualitative approach. Aviation, 18(3), 147-156.
Nergård, V., Hatlevik, O.E., Martinussen, M. & Lervag, A. (2011). An airman’s
personal attitude: Pilots’ point of view. Aviation, 15(4), 101-111.
Orasanu, J.M. (2010). Flight crew decision making. In B.G. Kanki, R.L. Helmreich
& J. Anca (Eds.), Crew Resource Management. (pp. 147-180). San Diego, CA:
Academic Press.
253
Orlady, L.M. (2010). Airline pilot training today and tomorrow. In B. Kanki, R.
Helmreich, & J. Anca (Eds.), Crew resource management, 2nd ed.. (pp. 469-
491). San Diego, CA: Academic Press.
Potter, W. J. & Levine-Donnerstein, D. (1999). Rethinking validity and reliability in
content analysis. Journal of Communication Research, 27(3), 258 – 284
QSR (2011). NVivo data analysis software. Downloaded from
www.qsrinternational.com.
Reason J. (2008). The human contribution: Unsafe acts, accidents and heroic
recoveries. Aldershot, UK: Ashgate.
Rose, M.R., Barron, L.G., Carretta, T.R., Arnold, R.D. & Howse, W.R. (2014). Early
identification of unmanned aircraft pilots using measures of personality and
aptitude. The International Journal of Aviation Psychology, 24(1), 36-52.
Ryan, G.W. & Bernard, H.R. (2000). Data management and analysis methods. In
N.K. Denzin and Y.S. Lincoln (Eds) Handbook of qualitative research (2nd Ed)
Thousand Oaks, CA: Sage.
Salas, E. (2014). Commentary on the article by Hoermann and Goerke: In search of
collective orientated members in the flight deck and beyond – A need a
challenge. The International Journal of Aviation Psychology, 24(1), 29-31.
Salas, E., Burke, C.S., Bowers, C.A., & Wilson, K.A. (2001). Team training in the
skies: Does crew resource management work? Human Factors, 43, 641-674.
Sarter, N.B. & Woods, D.D. (1991). Situation awareness: A critical but ill-defined
phenomenon. The International Journal of Aviation Psychology, 1, 45-57.
Shepherd, G. (2005). Foreword. In DFS-ADF (Ed.), Focus on human factors in
aviation. Canberra, Australia: Directorate of Flying Safety-ADF.
Stemler, S. (2001). An overview of content analysis. Practical Assessment, Research
and Evaluation. 7 (17). Retrieved from
http://PAREonline.net/getvn.asp?v=7&n=17
Steyvers, M. (2002) Multidimensional scaling. In L. Nadel (Ed) Encyclopedia of
Cognitive Science. (pp. 1-5). NY, NY:Wiley.
254
Sullenberger, C.B. & Zaslow, J. (2009). Highest duty: My search for what really
matters. NY: Harper Collins.
Swauger, S. (June, 2006). The human factor: Best captains – what FOs want in a
captain. Reporting Point Express/Crossfeed. Southwest Airlines Pilot
Association.
Swauger, S. (2011, November). Identifying the qualities of the ideal captain. Paper
presented at the Pacific and Asia CRM Developers, Evaluators and Facilitators
Forum PACDEFF-2011, Queenstown, NZ.
Tabachnik, B.G. & Fidell, L.S. (2007). Using multivariate statistics (5th Ed.). Boston,
MA: Pearson/Allyn & Bacon.
Thomas, M.J.W. (2004). Predictors of TEM: Identification of core non-technical
skills and implications for training system design. The International Journal of
Aviation Psychology, 14, 207 – 231.
Thomas, M.J.W. (2005). Error management training: Defining best practice.
Canberra, Australia: ATSB.
Trollip, S.R. & Jensen, R.S. (1991). Human factors in general aviation. Englewood,
Co: Jeppesen Sanderson.
Tvaryanus, A.P. (2014). Commentary on the article by Rose et al.: Search for the
right stuff. The International Journal of Aviation Psychology, 24(1), 57-60.
Weber, R.P. (1990). Basic content analysis. Thousand Oaks, CA: Sage
Welch, J.F., Bjork, L. & Bjork, L. (1999). Van Sickle’s modern airmanship – 8th
edition. New York: McGraw-Hill.
255
Appendices
Appendix A: Conference Paper referred to in Chapter 1 257
Carrick, K., Graham, S., Healey, A. & Pickard, S. (2008, April). An assessment of feedback and evaluation at a flight training school. Paper presented at the 8th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.
Appendix B: Airmanship Questionnaire reported in Chapter 2. 269
Appendix C: Airmanship Questionnaire reported in Chapter 3. 273
Appendix D: Interview Schedules reported in Chapter 4. 279
Appendix E: Conference papers based on Chapter 2 287
Carrick, K. (2009, June). Human factors as a component of airmanship. Paper presented at the 8th Industrial and Organisational Psychology Conference, Sydney, Australia.
Carrick, K. and Chalmers, K. (2010a, April). Airmanship: Voices from an on-line survey. Paper presented at the 9th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.
Carrick, K. and Chalmers, K. (2010b, April). Development of airmanship. Paper presented at the 9th International Symposium of the Australian Aviation Psychology Association, Sydney, Australia.
Appendix F: Airmanship Questionnaire reported in Chapter 6. 305
Appendix G: Domain Differences from Chapter 6 313
Appendix H: Copyright Permissions 320
256
Page left deliberately blank
257
Appendix A: Carrick, Graham, Healey and Pickard (2008)
An assessment of feedback and evaluation in a flight training school.
Kirstie Carrick, Suzanne Graham, Alison Healey
University of Newcastle
Stephen Pickard
BAE Systems
Background
An Australian flight training provider requested an independent examination of feedback and evaluation of flying activity in their flight training school, with a view to identifying any areas that might require improvement.
Learning to fly an aircraft is essentially the learning of motor skills, related to the physical handling of the aircraft, and acquiring the judgement, decision making and situational awareness skills that can be referred to as ‘airmanship’. Acquiring the essential theoretical knowledge of such things as aerodynamics, systems, engines and legislation, underpins successful learning and execution of flying skills. Thus the process of learning to fly will involve ground based study and the learning and practice of relevant skills in the aircraft. Part of the process is the provision of feedback from an instructor as to the effectiveness of the learner’s performance. There may be two different aims here: to provide information about the accuracy and appropriateness of actions in order that the trainee acquire and improve his/her skills; and secondly, to assess (and let the trainee know) whether he/she is meeting the standard of proficiency and rate of progress expected or required by the training organisation. Educational literature refers to these two intentions as formative and summative feedback. Formative feedback is that conducted during a task to provide information about how accurately the trainee is carrying out the tasks and how the trainee can improve their performance. Summative feedback or evaluation is generally conducted after the task performance and sums up how well the task was performed relative to some standard (Telfer and Biggs, 1985).
The education and flight training literature suggests that evaluation of performance should be based on objective criteria or competencies (Reich, Bye & Henley, 2003). Roscoe and Childs (1980) were early advocates of effective assessment and feedback to trainee pilots using an objective, criterion referenced approach. However, one important challenge facing assessment methods in aviation is to avoid overloading the instructor with so many indices of performance that he (or she) cannot observe them all during in-flight assessment, particularly while maintaining aircraft safety and instructional benefits to the student.
258
Most recent expression of the concept of criterion referenced evaluation has been adopted in the wider field of skill training as the setting of ‘competencies’, the level of satisfactory performance of job tasks (Tovey and Lawlor, 2004). This is well known in workplace and trade training, and has been introduced to civilian flight training in the development by CASA of National Competency Standards for pilots. There has been some debate regarding the applicability of competency-based assessment when assessing higher level competencies such as critical thinking, judgment, and decision making (all of which relate to the concept of airmanship) due to the difficulty in describing these more subjective outcomes in terms that are directly observable and assessable (Reich, Bye, & Henley, 2003).
The provision of formative feedback about task performance to the trainee is a crucial step in the learning process in any training and assessment program, not just in aviation (Kunchinke, 2000; Reich et al, 2003; Weaver, 2006). Feedback is an essential aspect of the communication that takes place between an instructor and a trainee and, as with any communication, the trainee needs to receive and comprehend the feedback they are given in order for it to be successful (Wood et al, 2004). Feedback about both strengths and weaknesses in task performance will allow trainees to maximise their potential and continually improve, if they are willing to accept the feedback (Weaver, 2006). In a study of summative feedback related to assessment of text (in a written assignment) Fritz, Morris, Bjork, Gelman & Wickens, (2000) found that feedback is more effective if it is not just an overall numeric score, but rather a qualitative account of several aspects of the task. In a further study of feedback on written work, Weaver (2006) found that feedback is perceived as helpful by students if it contains information on both strengths and weakness, is specific but related to general improvements that can be made in the next task, is understandable at the receiver’s level and related to assessment criteria.
In the management and organisational psychology literature, it is advised that constructive feedback should be based on trust and be direct. It should provide specific examples and be given when the receiver is ready to receive and accept it. Constructive feedback should also be practical and contain an appropriate amount of information. Most importantly feedback should be valid and be aimed to benefit the person who is receiving it, so that development occurs (Wood et al, 2004). Feedback about the effectiveness of workplace performance is linked to motivation and job satisfaction (Furnham, 2005).
Combining these ideas suggests that good post-flight debrief feedback should include:
• Clear, understandable communication • Evaluation related to criteria, and seen to be valid • Not just a total score, include sub-scores and qualitative evaluation • Strengths and weaknesses • Specific examples of each • Improvement tips
259
Assessors should be aware that evaluation must be consistent, reliable and unbiased as possible especially where a number of assessors are evaluating a range of students. The nature of the instructor-student relationship and the values and beliefs of the instructor can impact on the assessment of individual performance (Williams, Klamen & McGaghiel, 2003). Macleod (2005) notes the effect of ‘observer bias’ on evaluation in aviation contexts, and discusses standards, recency and primacy, halo and horn effects, leniency and severity, central tendency and ‘scale abuse’, the latter referring to those who feel that perfection can never be achieved and so never award the top grade. It is important for accurate evaluation, that the specific details of the assessment requirement are described and conditions of assessment are standardised as much as is possible. In addition the assessors should be aware of the possibility of the above-mentioned external influences and actively minimise them, so that trainees are assessed as having met or not met the performance criteria as objectively as possible.
The aim of this study was to evaluate feedback provided to trainees and ask the trainees and instructors about their views of feedback and evaluation in the current system.
Method
Materials
Observation of actual post-flight debriefs was considered to be too time consuming and intrusive. The instructors at this organisation, however, do produce written feedback from evaluation flights. This feedback is in addition to the post-flight debrief, provided as a record of assessment and is sighted and signed off as accurate by the trainee. Thus it was felt to be a fairly accurate record of content of the actual debrief. Each feedback sheet included three evaluation scores, one each for preparation, technique and airmanship, the lowest of the three being considered the final score for the flight. A sample of 98 feedback sheets was collected for analysis. Sheets were selected from across the range of flights, from different students and instructors. All identifying information was removed before the material was provided to the researchers.
A survey of instructor and trainee opinions about current evaluation and feedback practices was also conducted. The survey questionnaires differed slightly for each group and included a few demographic questions. The main part of the questionnaires consisted of a mix of open-ended questions, allowing respondents to write down their ideas or opinions, and some statements to which respondents were asked to indicate their degree of agreement. This was indicated by placing a cross on a line which was anchored at each end with the statements ‘not at all’ and ‘completely’, and had intermediate indications of ‘somewhat’, ‘moderately’ and ‘mostly’. In analysis, responses were scored between zero, for ‘not at all’ and 8 for ‘completely’, with estimation to the nearest to 0.5.
Procedure
260
The researchers attended the flight training establishment and presented a brief talk on the literature relating to feedback and evaluation to the assembled instructors. The aim of the research was outlined and the expert opinion of the instructors requested. They were told that participation in the survey was voluntary and anonymous, and that questionnaires could be returned via a box left in reception or by post. Questionnaire packs containing an information sheet, the questionnaires and a reply paid return envelope were then distributed. Trainee questionnaire packs were left with management for distribution to trainees.
Participants
The response rates among the two populations were similar, with 11 of the 20 instructor questionnaires being returned (55%), and 20 of the 40 trainee questionnaires (50%). The instructors ranged in experience from 0 - 2 years as an instructor, to over 16 years experience. The mean fell in the ‘3 to 5 years’ group. The trainee participants ranged in age from 19 to 31, mean 22.62 years. The trainee participants were from two instruction groups, one (9 participants) was at a stage of 25 hours training or less, the other group (11 participants) had a spread of 40 to 70 hours training.
Results
Analysis of feedback sheets
Evaluation scores: In the sample of 98 feedback sheets, three were missing assessment scores. In the remaining 95 sheets, it was found that preparation was rarely the lowest score. The range of preparation scores was from 1 to 4, with a mean score of 3.58. The most common score was 4. For both Technical skill and Airmanship, the range was 0 to 4 and the means were similar (Technique = 2.81, Airmanship = 2.80). It appears that the scores awarded are quite low and a score of 5 was never awarded in the sample examined. Statistical comparison of the means using a Kendell’s W (non-parametric equivalent of an ANOVA) revealed that there was no difference between the rank mean scores for airmanship and technique, but that the rank mean score for preparation was significantly higher (Kendall’s W=.432, p<.001). There is also a strong significant correlation between the airmanship and technical skills scores (r = .819, p<.001) indicating that trainees generally are awarded similar scores for each. The same score was awarded for preparation, airmanship and technique on 21 (22%) of the feedback sheets, and airmanship and technique were awarded the same score (with preparation higher) on 47 sheets (49%). There were also high significant correlations between each of technique and airmanship scores and the final score, as would be expected, given that the final score is the lower of the other two and they were so often the same. Correlation between final score and technique score was r= .881 (p<.001) and final score and airmanship r=.909 (p<.001).
Feedback comments: The number of positive and negative comments written on the feedback sheets was counted. A positive comment occurred where there was an indication that something was done well. A negative comment included an indication of
261
poor performance. Neutral comments, usually simply a record of what was flown, were not counted. An assessment was made of the number of learning comments on the feedback sheets, where a learning comment is defined as advice regarding future improvement, or how to do things better, or what not to do in future. Examples of these are: ‘Do not fixate on runway but maintain an appropriate workcycle’ or ‘Avoid swapping hands to set frequencies’.
Statistical comparison of the mean number of positive, negative and learning comments per feedback sheet indicated a significant difference among them (Kendall’s W=.664, p<.000). The rank mean for learning comments was significantly lower than the rank means for positive and negative comments, and there was no difference between positive and negative rank means.
Content analysis of the positive, negative and learning comments allowed subtotals to be calculated for the three assessment categories, preparation, airmanship and technical skills. Comments that mentioned preparation were classed as preparation, Comments were classed as airmanship if they occurred under a general heading of airmanship, or if they corresponded to airmanship criteria included in the assessment word pictures. Anything else was considered to be related to technical skills. Table 1 indicates the frequency count for type of comment (positive, negative or learning) by aspect of assessment (preparation, technique or airmanship).
Table 1:
Frequency of type of comment by aspect of assessment
Type of Comment
Positive Negative Learning Totals
Preparation 57 15 3 75
Technique 403 579 32 1014
Airmanship 131 66 9 206
Totals 591 660 44 1295
There were relatively few comments about preparation on the feedback sheets, and most of these were positive. There were more negative than positive comments about technique and most feedback sheets included both negative and positive comments. The majority of learning comments related to technical skills. There were fewer comments made about airmanship than aircraft control technique. Comments on airmanship were more frequently positive than negative, and nine learning comments related to airmanship.
A statistical comparison of the mean numbers of positive and negative comments per feedback sheet in each of the three categories of assessment was carried out. Learning
262
comments were not included because of the low numbers. There were significantly more comments regarding the technical aspects of flight than preparation or airmanship, and there were more positive than negative comments on preparation and airmanship, but more negative than positive comments about technical skills (Kendall’s W = .541, p<.000). Figure 1 shows the rank means for this analysis.
0
1
2
3
4
5
6
Prep Air Tech
Aspect of Assessment
Mea
n Ra
nk (K
enda
ll)
PosNeg
Figure 1: Type of comment by aspect of assessment: rank means
Survey analysis
Verbal/written brief consistency: Instructors were asked ‘how much does the content of your verbal debrief reflect the content of the written feedback sheet’. Response was by placing a cross on a scale from ‘not at all’ (scored as zero) to ‘completely’ (scored as 8). For this question the mean was 7.1, with all but one response above ‘mostly’ (scored as 6), indicating an ‘almost completely’ response. Trainees were asked a similar question, the mean of the responses was 6.3, just above ‘mostly’. The difference between the means was tested (using non-parametric tests) and showed a significant difference between the two groups (Wilcoxen, z=2.58, p<.01). This indicates that instructors were more confident that the content of the feedback sheets reflected the verbal debrief than were the trainees.
Objectivity of feedback: Trainees were asked how much they thought the written feedback reflects their flight performance. The mean of the responses was 5.7 (just below ‘mostly’), with a range of 4 (moderately) to 7 (between ‘mostly’ and ‘completely’), as illustrated in Figure 2.
263
7.006.005.004.00
reflectperformance
12
10
8
6
4
2
0
Freq
uenc
y
Mean =5.6905Std. Dev. =0.73274
N =21
reflectperformance
Figure 2: Trainee response regarding how much written feedback reflects actual performance.
Trainees were then asked separately about whether the current assessment of airmanship and aircraft control techniques, accurately reflects their actual performance. Response to the question about airmanship resulted in a mean score of 4.9, (SD = 1.20, range 2.5 to 7) and the technical skills in a mean of 5.9, (SD = 0.89, range 4 to 7). These two means are significantly different (Wilcoxen, z=2.573, p<.01). This indicates that the trainees are more confident that assessment of technical skills reflects their actual performance than they are that the assessment of airmanship reflects their actual performance.
The instructors were asked how much they felt that the current assessment is an objective process. Response to the assessment of airmanship question resulted in a mean of 3.6 (SD = 1.38, range 2 to 5.5). The mean response to the question about assessment of technical skills was 4.9 (SD = 1.72, range 1 to 6.5). These two means are significantly different (Wilcoxen, z=2.187, p<.029). This result indicates that in general the instructors feel that their assessment of technical skill is more objective than their assessment of airmanship.
Statistical comparison of trainee and instructor scores indicates that there is a statistically significant difference between the mean responses for each group for the question about airmanship (Wilcoxen, z=2.398, p<.016). Trainees appeared more confident that the assessment reflects their true ability than the instructors were confident in their own objectivity when assessing airmanship.
There was no significant difference between the two groups of respondents for the responses to the same question about aircraft control techniques. The trainees and instructors were equally confident that technical skill assessment (between moderately and mostly) reflects ability and is objective.
Inter-rater reliability: Instructors were asked how consistent the system allowed them to be when assessing different trainees. The mean response to this question was 5.2, (SD
264
= 1.87, range 1 to 8), indicating an average position between ‘moderately’ and ‘mostly’. The distribution illustrated in Figure 3 indicates that six of the eleven instructor respondents see themselves as between mostly and completely consistent.
8.007.006.005.004.003.002.00
diffstudconsistent
4
3
2
1
0
Freq
uenc
y
Mean =5.1818Std. Dev. =1.87447
N =11
diffstudconsistent
Figure 3: Instructor opinion of their consistency across trainees
Trainees were asked two questions about instructor consistency, first, ‘how consistent do you think instructors are when assessing different students’ and then ‘if two instructors were to assess the same student how consistent do you think their assessment would be’, an attempt to estimate ‘reliability’.
The response to the consistency question resulted in a mean of 4.3 (SD = 1.78, range 1 to 7), indicating ‘moderately’, a position just over half way along the scale. Response to the question about reliability among instructors assessing the same trainee resulted in a mean of 3.8, (SD = 1.72, range 1 to 6), suggesting that trainees see instructors as being a bit less than moderately reliable, slightly below the middle of the scale. These means were compared statistically, but were found not to be significantly different (Wilcoxen z=1.483, p=.138).
The means representing instructor consistency and trainee view of consistency were compared and found not to be significantly different. It seems that instructors are considered to be (and consider themselves) generally internally consistent. Trainees report that different instructors assess the same trainees moderately consistently and there is no significant difference with their opinion regarding individual consistency across trainees. The range of scores indicates a general feeling of ‘moderately’ consistent, but with some spread of opinion.
System problems and improvements: Instructors were asked about problems with the current system of assessment for airmanship and aircraft control technique, and also to make suggestions for improvement. All the respondents wrote something about these. There were a range of views including suggestions for improving the consistency of instructor evaluation, by changes to the assessment guidelines and/or training for instructors in debriefing, assessment and standardisation. Two responses drew attention to the workload resulting from too many students and not enough instructors, resulting in adverse effect on quality of training/debriefing. One instructor (out of 11) stated that
265
no changes need to be made. There were almost twice as many responses (9) to the questions about problems with assessment of airmanship and suggested improvements, than there were to the same questions about technique (5).
Trainees were asked similar questions and also raised the issue of consistency and standardisation. Eight of the twenty responses to the question about the current system were positive, stating such things as ‘comments are a good indication of where to improve’. Five responses mentioned instructor differences and inconsistencies, and one specifically noted variation in the amount of feedback given. Another commented on the narrow range of scores awarded, noting it was thus hard to show improvement over time. Three responses commented that the course is short, challenging and high workload. Asked about improvements to the system, seven responses (out of a total of 13) were concerned with standardisation and consistency across instructors. Three further comments had to do with instructor issues, having the same instructor regularly, personality clashes and assessment of instructors. The other three responses were more general comments on the assessment system, averaging of scores, making it pass/fail rather than having scores, and providing more detail on the assessment criteria to trainees.
Discussion
Distribution of scores: There were no top scores, ‘5’s, in the sample of feedback sheets despite there being allowance in the assessment guidelines for some errors to occur in a ‘5’ evaluation. This non-use of the full range of possible scores is something that Macleod (2005) would term ‘scale abuse’. It was also noted that scores for technique and airmanship were often the same (71% of sample sheets), which may indicate that instructors are awarding a global score rather than making truly separate assessments.
Feedback content: Analysis of the feedback sheets indicated clear imbalance in the types of comments made and in the aspects of assessment commented upon. Few learning comments were recorded, preparation was not often mentioned, there were some comments on airmanship and the majority of comments related to technique.
The literature suggests that good feedback should include suggestions about how the student can improve their performance (Weaver, 2006). These were identified as learning comments in the analysis of the sample of feedback sheets. Only 44 learning comments were identified on 34 of the 98 sheets. Although negative comment by implication includes some learning intent, constructive advice is generally considered to be of more use to students. This may indeed occur in the formative feedback in air and/or in the post-flight debriefing, and not make its way into the feedback sheets. However, instructors reported that the sheets are a close reflection of the post-flight debrief. Inclusion of some indication of what should be done to attain a ‘5’ would help the trainee improve.
The majority of written feedback comments were concerned with technique. It could be that the instructors are more comfortable commenting on skill development within
266
the context of the technical criteria defined for the sortie, rather than the less easily observed aspects of airmanship (decision making, situational awareness, etc). There are also many different technical flying activities in each sortie which would require comment, while airmanship criteria may be considered across the whole flight. Comments on airmanship sometimes appeared to be in the form of a mechanical record of whether radio, checks and/or situational awareness were satisfactory, or not, rather than learning comments regarding airmanship. Little comment on preparation is made on the feedback sheets, with 75 comments of various types, made across the sample of 98 sheets. This may be because the feedback of a high score (‘4’) may be considered sufficient comment, and it would save time not to add anything else.
Instructor consistency: There was no statistical difference between the mean responses of instructors and trainees on the question addressing consistency across trainees, although the instructors’ responses were, in general, higher on the scale than the trainees’. On the whole the responses to these questions suggest that there is a perception of moderate to mostly consistent assessment. The ideal, of course, would be to have ‘completely’ consistent assessment.
In response to the final open questions, five of the trainees raised issues of instructors’ lack of consistency as concerns about the current system (out of 20 comments), and ten responses about future improvements (out of thirteen) were concerned with instructor issues. The content of the questions they had answered may have primed the respondents to some extent to report such issues, but clearly some dissatisfaction has been tapped among some of the trainees. One went as far as to suggest that there should be more “scrutiny of instructors and the use of feedback from trainees to assess instructors”. Research suggests that credibility of assessors is important when presenting negative feedback for it to be taken on board by the student (Steelman & Rutowski, 2004).
It is also clear that few of the instructors felt that they (or their colleagues) were able to be completely consistent across trainees. There were some suggestions among those made to the final questions asking about problems and improvements, which addressed issues of consistency. A couple of instructors raised a concern that standards might vary with instructors’ experience. One suggestion was to have regular meetings of instructors to ensure consistency and standardisation of assessment, another that some training in assessment and debriefing would be useful as these are not emphasised in instructor training.
Conclusions:
Feedback to flight trainees included both positive and negative comments, but lacked many learning comments, although these may be made during debriefs and not written into the feedback sheets.
267
Consistency in evaluation across different instructors appears to be of some concern, to instructors and trainees, and several suggestions were made to address the problem however it should also be noted that the indicators of consistency were towards the top of the scale rather than the bottom, so while consistency may not be seen as perfect, it is not seen as drastically lacking.
The relative lack of comment on airmanship in written feedback, the indication that airmanship is slightly less objectively assessed and concerns raised among the survey open responses may indicate that while there are clear criteria for demonstration of technical skills these may not matched by as definitive criteria for demonstration of airmanship. Definition and evaluation of airmanship is an area that is being explored further.
References
Fritz, C., Morris, P., Bjork, R. A., Gelman, R. & Wickens, T. D. (2000). When further learning fails: Stability and change following repeated presentation of text. British Journal of Psychology, 91, 493-511.
Furnham, A. (2005). The psychology of behaviour at work: The individual in the organisation. Hove, UK: Psychology Press.
Kuchinke, P. K. (2000). The role of feedback in management training settings. Human Resource Development Quarterly, 11(4), 381-401.
Macleod, N. (2005). Building safe systems in aviation: a CRM developer’s handbook. Aldershot, UK: Ashgate.
Steelman, L. A. & Rutkowski, K. A. (2004). Moderators of employee reactions to negative feedback. Journal of Managerial Psychology, 19(1), 6-18.
Reich, A., Bye, J., & Henley, I. (2003). Assessment and Evaluation. In I.M.A. Henley (Ed.), Aviation education and training. Aldershot, UK: Ashgate.
Roscoe, S. N., & Childs, J. M. (1980). Reliable, objective flight checks. In S. N. Roscoe (Ed.), Aviation Psychology. Ames, Iowa: Iowa State University.
Telfer, R. & Biggs, J. (1985). The Psychology of Flight Training. Cessnock, Australia: Aircrew Training Centre.
Tovey, M.D. & Lawlor, D.R. (2004). Training in Australia. Sydney, Australia: Pearson Prentice Hall.
Weaver, M. R. (2006). Do students value feedback? Student perceptions of tutors’ written responses. Assessment and Evaluation in Higher Education, 31(3), 379-394.
Williams, R. G., Klamen, D. A. & McGaghie, W. C. (2003). Cognitive, social and environmental sources of bias in clinical performance ratings. Teaching and Learning in Medicine 15(4), 270-292.
268
Wood, J., Chapman, J., Fromholtz, M., Morrison, V., Wallace, J., Zeffane, R. M., Schermerhorn, J. R., Hunt, J. G. & Osborn, R. N. (2004). Organisational Behaviour: A Global Perspective (3rd ed). Sydney, Australia: Wiley.
269
Appendix B: Airmanship Questionnaire reported in Chapter 2
Airmanship Survey
Thank you for deciding to complete this questionnaire. Please consider that what we want is your personal view, not that of an employer or organisation, or what you might have read, but what you, yourself, think.
Experience – these questions are factual. Please complete truthfully!
Copy and paste as required
1. Licence level: (tick one)
Unlicensed
Student
GFPT
PPL
CPL
ATPL
Military
2. Instructor rating and grade (if held) Rating
Grade 1
Grade 2
Grade 3
CFI
3. Where did you take your initial flight training (tick one)?
Military or
Civilian
4. Military experience (if any): Years of service: ____
Hours flown: ________
5. Civilian experience: Years of experience: _______
270
Hours flown: _______
6. Have you any Aviation related studies or qualifications (besides licensing requirements), if so, what? ____
7. Please indicate your general educational level:
Left before Yr 12
HSC, Matric, or equivalent toYr12
Trade or TAFE qualifications
University degree
8. Any Comments? ______
Questions about airmanship – please type in your responses as if you were speaking to someone. Just start typing on the line below the question.
1. What do you think airmanship is? (Take as much space as you want)
2. Think of your response to the first question, can you elaborate on each of the key words in your response? For example, if you wrote ‘being safe’, write in more detail what ‘being safe’ would involve. (Take as much space as you want)
3. Was the development of airmanship a specific part of the initial flight training you did? Please explain what this involved.
4. (a) In your initial flight training, was airmanship, or concepts related to airmanship, discussed in ground training or pre- or post- flight briefings?
271
4 (b) If so, which concepts, and what sorts of discussion?
5. Thinking about your progress to date as a pilot, what do you feel led to the development of your sense of airmanship?
6. In your current aviation activities, are you normally actively aware of airmanship as part of your operation? What would be the most important things that come to mind?
7. In your current aviation activities, during what phases of flight or events would you be most actively aware of airmanship issues?
8. If you are currently an instructor, what do you do to ensure that your students develop good airmanship?
9. Please describe a flight, that you observed or know about, in which good airmanship was displayed, including why you feel it was particularly good airmanship.
10. Please describe a flight, that you observed or know about, in which poor airmanship was displayed, including why you feel it was particularly poor airmanship.
11. Any other comments?
272
Page left deliberately blank
273
Appendix C: Airmanship Questionnaire reported in Chapter 3
Airmanship Survey
Thank you for deciding to complete this questionnaire. Please consider that what we want is your personal view, not that of an employer or organisation, or based on what you might have read, but what you, yourself, think.
When you have finished, save the survey and send as an attachment to kirstie.carrick@newcastle.edu.au
The survey will be separated from any identifying material (like email address) before analysis.
Demographics: we would like to know a bit about you so we can compare different groups of pilots.
Copy and paste as required
1. Most recent Licence level: (tick one)
Unlicensed
Student
GFPT
PPL
CPL
ATPL
Military
Other _________
2. Are you an instructor or trainer (or have you ever been)?
Instructor rating Yes No
Military QFI/FCI Yes No
Check or training role Yes No
If you are no longer in this role, in what year did you stop? ________
274
3. Where did you do your initial flight training?
Australia
Other
4. Was your initial flight training (tick one)?
Military or
Civilian
5. What year did you start your flying training? ________
6. Military experience (if any): Years of service: ____
Hours flown: ________
7. Civilian experience: Years of experience: _______
Hours flown: _______
8. If you no longer fly, in what year did you stop? ______
9. How would you describe your current aviation activity (or most recent if no longer flying)?
Recreational
Instruction
Aerial work
Charter
Regional airline
Major airline
Military instruction
Military
10. Any Comments? ______
275
Part 2
Questions about airmanship – please type in your responses as if you were speaking to someone. Just start typing on the line below the question.
1. What do you think airmanship is? (Go into as much detail as you want)
2. Thinking of your response to the first question, can you elaborate on each of the key words in your response? For example, if you wrote ‘being safe’, write in more detail what ‘being safe’ would involve. (Go into as much detail as you want)
3. (a) In your initial flight training, was airmanship, or concepts related to airmanship, discussed in ground training or pre- or post- flight briefings?
3 (b) If so, which concepts, and what sorts of discussion?
4. Was the development of airmanship a specific part of the initial ‘in the air’ flight training you did? If so, please explain what this involved.
5. Thinking about your progress as a pilot, what do you feel led to the development of your sense of airmanship?
6. Do you participate in ongoing training? Does this include airmanship and if so, what does this entail?
7. In your current aviation activities, are you normally actively aware of airmanship as part of your operation? What would be the most important things that come to mind? If you are no longer flying, were you normally actively aware of airmanship as part of your operation?
276
8. During what phases of flight or events would you be most actively aware of airmanship issues?
9. If you are currently an instructor or hold a check/training role, what do you do to ensure that your students develop good airmanship? (or if you were an instructor, what did you do?)
10. Have your ideas about airmanship changed over time? If so, in what way?
11. Please describe a flight, that you observed or know about, in which good airmanship was displayed, including why you feel it was particularly good airmanship.
12. Please describe a flight, that you observed or know about, in which poor airmanship was displayed, including why you feel it was particularly poor airmanship.
13. Have you any other comments?
Scroll down for Part 3
277
Part 3
14. Please rate the importance of each of the following concepts or ideas with regard to your interpretation of airmanship.
Please enter an X in one of the boxes for each concept (one per line).
Not important
Marginal Moderate Quite Extremely important
Awareness of own medical fitness
Awareness of own psychological fitness – e.g. personal stress
Flying Skill (‘stick and rudder’) Ability to operate automation Proficiency (currency and expertise)
Knowledge of your aircraft type Knowledge of environment (terrain, weather)
Courtesy Knowledge of regulatory/ organisational environment
Risk assessment Self-improvement Team management Communication with other aircraft
Situational awareness Discipline Judgement and decision making Crew interrelationships Personal attitude/qualities Communication with aircraft maintenance personnel
Communication with ATC Assertiveness Common sense Cooperation Maintain lookout Preparation/ planning Responsibility
278
15. If you had to pick the five most important concepts, which would you pick? Please number from 1 (most important) to 5 (fifth most important). Just number 5, don’t number the rest!
Rank 1-5
Awareness of own medical fitness Awareness of own psychological fitness – e.g. personal stress Flying Skill (‘stick and rudder’) Ability to operate automation Proficiency (currency and expertise) Knowledge of your aircraft type Knowledge of environment (terrain, weather) Courtesy Knowledge of regulatory/ organisational environment Risk assessment Self-improvement Team management Communication with other aircraft Situational awareness Discipline Judgement and decision making Crew interrelationships Personal attitude/qualities Communication with aircraft maintenance personnel Communication with ATC Assertiveness Common sense Cooperation Maintain lookout Preparation/ planning Responsibility
16. Any final comments?
Thank you for participating in this research! Results will be published when analysis is completed and a summary sent for publication in Flight Safety Australia.
279
Appendix D: Interview Schedules reported in Chapter 4
Interview Questions – Military Ab-initio Participants Case no: _____
Press Record!
Demographics:
For how long have you been an Instructor? Mil or Civ?
Have you any academic qualifications in aviation, BAv, MAvMg?
What & from which Uni?
1. Can you describe your concept of airmanship?
2. What do you think are the main components of airmanship? Can you say what each component is (eg what is SA?)
3. What skills are included?
4. Does it encompass knowledge as well?
5. What about attitudes?
6. Is there anything else that you can add to those components?
7. Please give an example of a situation in which good airmanship was displayed.
8. Please give an example of a situation where the pilot displayed poor airmanship.
9. Do you talk about ‘airmanship’ specifically during the pre-flight briefing, if so what sorts of thing do you mention?
10. What aspects of airmanship do you think develop first in trainees?
11. What aspects of airmanship do you think develop next?
12. What is it about (each airmanship component) that you are concerned with when assessing trainees?
280
13. What sorts of behaviours do you look for in trainees when evaluating airmanship? Eg what evidence or behaviour do you look for that the trainee is demonstrating good situational awareness?
14. The findings of the 2006 survey at FTT indicated that decision making, safety and situational awareness were most often mentioned in relation to airmanship assessment – would you change the order or add anything else?
15. The analysis of PT51 forms in 2006 showed that mention was made mostly of checks, situational awareness and radio calls when giving feedback on airmanship, would you agree that these are representative of airmanship? What else do you think could be used to evaluate airmanship?
16. Do you think that your assessment of and feedback to trainees has altered since the feedback given in late 2006?
281
Interview Questions – Sector Military Advanced Participants Case no: _____
Demographics:
For how long have you been an Instructor?
Military only, or any civilian?
Have you any academic qualifications in aviation, BAv, MAvMgt?
What & from which Uni?
1. What do you think airmanship is?
2. What do you think are the main components of airmanship?
3. What skills are included?
4. Does it encompass knowledge as well?
5. What about attitudes?
6. Is there anything else that you can add to those components?
7. Can you walk through a flight telling me what you would expect to see in terms of good airmanship?
8. Please give an example of a situation where a pilot displayed poor airmanship.
9. Do you talk about ‘airmanship’ specifically during the pre-flight briefing, if so what sorts of thing do you mention?
10. What aspects of airmanship do you think develop first in trainees?
11. What aspects of airmanship do you think develop next?
12 Do you assess airmanship when instructing trainees?
13. What is it about (each airmanship component) that you are concerned with when assessing trainees?
14. What sorts of behaviours do you look for in trainees when evaluating airmanship? Eg what evidence or behaviour do you look for that the trainee is demonstrating good situational awareness?
282
15. The findings of a survey of instructors who are required to assess airmanship indicated that decision making, safety and situational awareness were most often mentioned in relation to airmanship assessment – do you agree or would you add something else?
16. An analysis of feedback given to such trainees showed that mention was made mostly of checks, situational awareness and radio calls, would you agree that these are representative of airmanship? What else do you think could be used to evaluate airmanship?
283
Interview Questions – Civilian Ab-initio Participants Case no: _____
Demographics:
For how long have you been an Instructor? 1 to 5, 5 to 10 or 10 to 15 yrs, more?
Have you any academic qualifications in aviation, BAv, MAvMgt?
What & from which Uni?
1. What do you think airmanship is?
2. What do you think are the main components of airmanship?
3. What skills are included?
4. Does it encompass knowledge as well?
5. What about attitudes?
6. Is there anything else that you can add to those components?
7. Can you step me through a flight, telling me what you would expect to see in terms of good airmanship.
8. Please give an example of a situation where a pilot displayed poor airmanship.
9. Do you talk about ‘airmanship’ specifically during the pre-flight briefing, if so what sorts of thing do you mention?
10. What aspects of airmanship do you think develop first in trainees?
11. What aspects of airmanship do you think develop next?
12 Do you assess airmanship when instructing trainees?
13. What is it about (each airmanship component) that you are concerned with when assessing trainees?
14. What sorts of behaviours do you look for in trainees when evaluating airmanship? Eg what evidence or behaviour do you look for that the trainee is demonstrating good situational awareness?
15. The findings of a survey of instructors who are required to assess airmanship indicated that decision making, situational awareness and safety were most often
284
mentioned in relation to airmanship assessment – do you agree or would you add something else?
16. An analysis of feedback given to such trainees showed that mention was made mostly of checks, situational awareness and radio calls, would you agree that these are representative of airmanship? What else do you think could be used to evaluate airmanship?
285
Interview Questions – Civilian Advanced Participants Case no: _____
Demographics:
For how long have you been a pilot?
For how long have you been in check and training?
Have you any academic qualifications in aviation, BAv, MAvMgt?
What & from which Uni?
1. What do you think airmanship is?
2. What do you think are the main components of airmanship?
3. What skills are included?
4. Does it encompass knowledge as well?
5. What about attitudes?
6. Is there anything else that you can add to those components?
7. Can you step me through a flight, telling me what you would expect to see in terms of good airmanship.
8. Please give an example of a situation where a pilot displayed poor airmanship.
9. Do you talk about ‘airmanship’ specifically during the pre-flight briefing, if so what sorts of thing do you mention?
10. What aspects of airmanship do you think develop first in trainees?
11. What aspects of airmanship do you think develop next?
12 Do you assess airmanship specifically during your check rides?
13. What is it about (each airmanship component) that you are concerned with when assessing pilots?
14. What sorts of behaviours do you look for in pilots when evaluating airmanship? Eg what evidence or behaviour do you look for that the pilot is demonstrating good situational awareness?
286
15. The findings of a survey of instructors who are required to assess airmanship indicated that decision making, situational awareness and safety were most often mentioned in relation to airmanship assessment – do you agree or would you add something else?
16. An analysis of feedback given to such trainees showed that mention was made mostly of checks, situational awareness and radio calls, would you agree that these are representative of airmanship? What else do you think could be used to evaluate airmanship?
287
Appendix E: Conference papers based on data reported in Chapter 2
Human factors as a component of airmanship CARRICK, K. University of Newcastle, Australia. Kirstie.carrick@newcastle.edu.au
Airmanship is a quality widely assumed to be held by pilots of all license levels. Airmanship is not a specific part of the theory syllabus for flight licensing but is supposed to be included in pre-flight briefings during flight training. Assessment of airmanship in licence flight tests has been a subjective assessment by the testing officer. Human factors (HF) and threat and error management (TEM) are covered in the human performance and limitations part of the theory syllabus and will be tested from mid-2009 in licence flight tests. Criteria to assess competency in HF and TEM during flight tests are being introduced by CASA as being part of ‘old-fashioned good airmanship’. What, then, do Australian pilots think ‘airmanship’ consists of, and does this include the five elements that will be assessed? An on-line survey that asked about pilots’ concepts of airmanship was promoted at CASA safety seminars and in the CASA Flight Safety magazine and resulted in 40 responses. The survey found that the elements of HF identified by CASA for assessment were reflected in pilots’ definitions of airmanship or in discussions about airmanship training, but each of the elements was mentioned by fewer than half the respondents.
Airmanship is a quality widely assumed to be held by pilots of all license levels. The International Civil Aviation Organization (ICAO) defines airmanship as ‘the consistent use of good judgement and well developed skills to accomplish flight objectives’ (ICAO, 2006). A military definition is ‘the safe and efficient operation of an aircraft, in the air and on the ground’ (BAE Systems, 2006). To become a pilot, one has to pass a sequence of theory exams, undergo flight training and pass a series of flight tests. During this process, it has generally been hoped that good airmanship would be learned by experience and from the example of instructors and other pilots. Airmanship itself, however, remains a rather elusive quality when it comes to trying to pin down specifics (Kern, 1996).
Safety is certainly a common theme amid discussion of airmanship and definitely the desired outcome. In aviation, it is much to be preferred that the flight conclude with the aircraft safely on the ground at the end of a flight, than not. If anything does go wrong, it is the Captain and crew who are expected to display good airmanship and bring the stricken aircraft safely through the flight. Accident analysis shows that 80% or more of aircraft accidents involve some human factor among the causes. Conversely, where a flight has experienced an external threat (like a bird strike) or an internal error on the flight deck, 100% of flights that did not crash (or have been survivable) have involved humans saving the day and landing safely, sometimes in circumstances that constitute what James Reason calls ‘heroic recoveries’ (Reason, 2008). So while the human factor can be a source of problems, it is definitely a part of the solution.
In recognition of the role of human factors in safety and in accidents, theory training for pilots for the last 20 years has included the study of human factors. In Australia human factors forms a part of the theory syllabus called ‘Human performance
288
and limitations’ (HPL). The HPL syllabus covers aspects such as human physiological response to high altitude, perceptual problems like visual and vestibular illusions due to ambiguous visual cues and unusual aircraft attitudes, and issues relating to information processing such as memory, attention and decision making and the effects of fatigue, alcohol and stress on human performance. Consideration of communication, situational awareness and threat and error management are also part of the syllabus.
In the Australian civilian aviation industry ‘airmanship’, as such, is not part of the theory syllabus for flight licenses. However it is supposed to be included in pre-flight briefings during flight training (CASA, 2006) and Licence flight tests have included a subjective assessment of airmanship carried out by the testing officer.
From mid-2009, the human factors (HF) and threat and error management (TEM) issues covered in the human performance and limitations part of the theory syllabus, will now be explicitly tested in pilot licence flight tests. The criteria to assess competency in HF and TEM in flight tests are being introduced by the Civil Aviation Safety Authority (CASA) on the basis of being part of ‘old-fashioned good airmanship’ (CASA, 2008) and the suggestion is that for “CASA to link airmanship to human factors, is in effect, tantamount to bringing science to the often nebulous concept of airmanship” (CASA, 2008, p 8).
The aspects of human factors that CASA is going to require in flight testing from July this year are the following five elements:
• maintain effective lookout;
• maintain situational awareness;
• assess situations and make decisions;
• set priorities and manage tasks; and
• maintain effective communications and interpersonal relationships.
(CASA, 2008, p7) In addition, there will also be assessment of recognition and management of
threats, errors and undesired aircraft states, as the threat and error management part of the syllabus, but these will not form part of the current discussion.
Given the emphasis that the introduction of human factors testing is part of airmanship formulated into competencies, it would be interesting to know if the general aviator agrees with CASA on this. What, then, do Australian pilots think ‘airmanship’ consists of, and does this match the criteria that CASA is introducing?
Method. An on-line survey was developed to find out about the concepts of airmanship
among general aviation pilots in Australia, as part of a wider project on airmanship and its development and training. The survey was promoted at CASA safety seminars in the winter of 2008, and in the CASA Flight Safety magazine of May-June 2008. Contact details were provided to allow respondents who preferred to respond off-line to obtain a paper copy. The survey asked 11 open ended questions, including ‘what do you think airmanship is?’ Other questions asked about airmanship in initial flight training, development of airmanship, awareness of airmanship in current operations and also asked for examples of good and poor airmanship.
289
By September 2008, 40 responses had been recorded. Respondents held a variety of flight licences at all levels from 5 ATPL (Airline Transport Pilot Licence) holders, through 15 CPL (Commercial), 13 PPL (Private), 4 at GFPT (General Flying Proficiency Test) and 1 Student, one military pilot and one military ATC (Air Traffic Controller). One of the PPL holders was also an ATC. Only three of the respondents were initially trained in the military environment. Sixteen respondents held an instructor rating. The experience in civilian flying ranged from less than one year to 55 years, and from 28 hours to 22,000 hours (mean = 3776 hours, median 1100 hours).
Results Assessment of content and themes was carried out using NVivo7 as the medium
of analysis. Tree nodes, corresponding to the components of the Kern (1996) model of airmanship were established initially, with free nodes created as themes were discovered in the survey responses. A total of 36 nodes were created, each representing a concept that the respondents related to airmanship. The most frequently mentioned was safety, brought up by 25 respondents (or 62%). At the other end of the rankings, ‘human factors’ was specifically mentioned by only one respondent and error management by one. Clearly, the respondents had different ideas about what makes up airmanship. For the purposes of this paper, the nodes that correspond to the five areas or elements of human factors, defined by CASA and that will be included in flight testing from July 2009, will be explored.
All five elements of assessment for human factors were raised in discussion by the respondents. While none of the respondents mentioned all five of the CASA elements of competency, only seven omitted them completely. The two who mentioned four elements, both missed ‘set priorities and manage flight’. The majority of respondents (62%) mentioned two or more of the CASA elements of assessment, as shown in Table 1.
Table 1
Number of Elements Mentioned by Respondents to the Airmanship Survey Elements N of Respondents
All 0 Four 2 Three 9 Two 14 One 8 None 7
Table 2 shows some of the things that respondents wrote about when asked about airmanship. The left column shows themes related to the CASA elements of assessment and the right column shows some of the other themes mentioned by respondents. The themes are also ranked by number of respondents mentioning that theme.
290
Table 2: Themes related to Airmanship
‘CASA related’ Nodes N Rank Other Nodes N Rank Safety 25 1 2. Situation awareness 20 4 Relationships 22 2 1. Lookout 19 6 Legislation 20 4 5. Communication 16 7 Skill 12 9 Courtesy 13 8 Experience 12 9 3. Judgement/Decisions 10 14 Aircraft Knowledge 11 12 Planning 10 14 Common sense 9 18 4. Set priorities and manage flight
5 24 Responsibility 5 24
Discussion
Situational awareness, lookout and communication were frequently mentioned by respondents in writing about airmanship and are high in the ranking of themes; decision making or judgement was not mentioned quite so frequently; and ‘set priorities and manage flight’ was the least frequently mentioned of the five CASA elements of assessment.
CASA has included ‘Maintain situational awareness’ as an element of competency to be assessed in flight tests. Situational awareness can be defined as “the perception of the elements in the environments within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future” (Endsley, 1988, cited in Endsley, 1999, p258). More simply phrased, it is knowing what is going on in the aircraft and in the surrounding airspace, as well as what is likely to happen next, often referred to as being ‘ahead of the aircraft’. Twenty of the respondents, or 50%, made a total of 30 references to situational awareness being part of airmanship, in their responses to the survey. Some did not use those exact words, but it was clear what was meant. Examples: ‘I believe airmanship to be an awareness and understanding of an aviator of their surroundings and what is taking place’ or a negative phrasing of the same concept: ‘The only things that can kill a pilot are his lack of regard to the task at hand, other aircraft and birds’.
Lookout is the idea of looking outside of the aircraft to identify and avoid other aircraft or unexpected landforms, it can also be expanded to include ‘listen out’ or keeping track of the position of other aircraft by listening to radio calls on the local frequency. Only two respondents mentioned ‘lookout’ as part of their initial definition of airmanship, but a total of 19 (47%) mentioned lookout at some stage in the survey, generally in response to a question on the concepts included in initial flight training. Listen or listen-out was mentioned, once or twice, but was coded with ‘communication’.
The next most frequently mentioned element of competency was to ‘maintain effective communications and interpersonal relationships’. Communication was brought up by 16 respondents in their answers to various questions, with seven including communication specifically in their definition of airmanship. For example, one definition was, in part, ‘Practicing a good lookout [and] keeping in contact with other aircraft’ and another was ‘ability to communicate with other users of aircraft to achieve safe reliable outcomes’. Several of the examples of good and poor airmanship included
291
comment on communication – or lack of it. Examples: ‘clear concise radio calls in the ctaf area’, and on the other hand, ‘often strike pilots not using radios when landing on public strips’.
The other half of this element is concerned with ‘interpersonal relationships’. The elaboration of this element in the CASA document includes a competency on assertiveness to ensure safe outcomes, but otherwise repeats the importance of clear communication to passengers and other stakeholders (CASA, 2008). Relationships with others were mentioned by the respondents very frequently, but perhaps not in quite the way that the CASA Flight Testers would be looking for. Respondents to the survey included comment on the influence of a good relationship with their instructors when learning to fly, and the hanger talk experienced in flying clubs, on the development of their airmanship. A few respondents mentioned the effect on them of the deaths of friend and acquaintances in aircraft accidents, and how this led to them being more aware of good airmanship. Such comments were coded as ‘relationships’, and the table shows that they were mentioned almost as frequently as ‘safety’.
An interesting aspect in definition and discussion of airmanship was the inclusion by 13 of the respondents, (33% or roughly a third) of concepts to do with courtesy and respect, which relates closely to the idea of interpersonal relationships and communication. Some examples of definitions that included this theme are that airmanship is ‘A combination of flying skills, common sense and courtesy’ from one respondent and ‘Being alert and abiding by all the rules and being courteous to other pilots’ from another. ‘Doing the safe & right thing by yourself and to others, showing respect to other airmen’ was another definition. Such responses show that some importance is placed on the tone of communication and of interpersonal relationships, but not the same aspect as CASA considers in suggesting that Flight Testers should look for ‘assertiveness that ensures safe completion of the flight’(CASA, 2008).
The element of assessment ‘assess situation and make decisions’ was mentioned, or indicated but not using those exact terms, by 10 respondents or 25%, in discussing airmanship. For example mention was made of ‘thinking before doing’, coded as showing an awareness of judgement and decision making. More direct examples are comments like ‘I think of good judgment as good airmanship’ and ‘Airmanship is very individual but is usually referring to judgement and decision making more so than manipulative skill’.
Finally the element of competency ‘set priorities and manage tasks’ was infrequently mentioned. Managing the flight was specifically mentioned by only two respondents, one of these was: ‘Having a system of managing my flight from the flight plan all the way through till the aircraft is put away at the end of the day’. There were a further three who wrote a response that indicated a concern with ongoing management during flight, such as ‘The necessity to plan the flight and fly the plan is very important. Spontaneous or unthought out changes from the plan are what creates unsafe situations’ and ‘Airmanship is awareness of present, imminent, and future situations, giving priority as is needed to each element. Aviate, Navigate, Communicate, is one example’. More comment was made by respondents about planning, mentioned by 11 respondents (25%), in 16 references, although this was generally in reference to prior planning before flight. The implication of the CASA assessment elements was more towards ongoing management during flight.
292
Conclusion
The on-line survey responses do show that the elements selected by CASA for inclusion in flight testing are generally seen as part of airmanship by general aviation pilots. While none of them mentioned all of the elements in their discussions, all but 7 (83%) did mention at least one of the elements. The diversity of concepts brought up by respondents, and the lack of agreement in terms of mentioning the same components, shows the nebulous nature of ‘airmanship’. Even though only one respondent mentioned ‘human factors’ specifically, it is clear that many human factors concepts, like situational awareness, judgement and communication, are seen as important parts of airmanship.
This survey only asked open ended questions about airmanship, there could be different pattern of responses if a list to tick off had been included. CASA’s selection of human factors elements for assessment is discussed here only in terms of whether a sample of General Aviation aviators think that these are part of airmanship, and raised them spontaneously. In addition, it is a small sample of aviators and may not be particularly representative. It did not include pilots currently flying in major airlines, for example.
References BAE Systems (2006). Airmanship Introductory Lecture. [Powerpoint presentation]
Tamworth, Australia: Author
CASA (2006). Flight instructor manual. Canberra, Australia: Author
CASA (2008). CAAP 5.59-1(0) Teaching and assessing single-pilot human factors and threat and error management Retrieved 15 October 2008 from http://www.casa.gov.au/wcmswr/_assets/main/download/caaps/ops/5_59_1.pdf
Endsley, M.R. (1999). Situation awareness in aviation systems. In D.J. Garland, J.A. Wise, & V.D. Hopkin, (Eds) Handbook of Aviation Human Factors. Mahwah, N.J.: Erlbaum
ICAO (2006). Explanatory Factors. Retrieved 10 November 2006 from www.icao.int/anb/aig/Taxonomy/R4CDExplanatoryFactors.pdf
Kern, T. (1996). Redefining airmanship. New York; McGraw-Hill
Reason, J.T. (2008). The human contribution. Aldershot, UK: Ashgate.
293
Airmanship - Voices from an on-line survey
KIRSTIE CARRICK* AND KERRY CHALMERS
University of Newcastle
Background Airmanship is considered important for aviators (Ebbage & Spencer, 2003). Kern (1996) describes a model that provides a theoretical structure of airmanship, based on research conducted in the USA. Kern states that the fundamental principles of airmanship are: “skill, proficiency and the discipline to apply them in a safe and efficient manner” (p21). In addition, there are six areas of critical expertise, which are shown as pillars of knowledge: self, team, aircraft, environment, risk and mission (Kern, 2009). According to Kern (1996) it is the combination of the basic principles with expert knowledge that enables an aviator to maintain situational awareness and exercise good judgement in decision making, and thus display good airmanship. Ebbage and Spencer present a simplified model of airmanship with three main factors: discipline, control and judgement. Discipline includes self-improvement, vigilance, co-operation and confidence. Judgement covers situational awareness, problem solving, mental workload and foresight, while control refers to flying skills, automation skills, information management, navigation and communication skills. This model specifies broader skill sets than the Kern model. Both models have a similar emphasis on discipline. Ebbage and Spencer consider the foundations of airmanship to be the underlying knowledge, skills and attitudes and that these should be included in any flight training program.
The current research addresses whether Australian aviators consider airmanship to include similar concepts to Kern’s model and whether there are any additional themes specific to the Australian industry. Method An on-line survey was developed and promoted at CASA safety seminars in winter 2008 and in the Flight Safety magazine of May-June 2008. There were 40 responses from pilots by September 2008. The respondents varied in level of licence held, in length of time involved in aviation and hours of experience, from a student pilot on 28 hours through to an ATPL holder with 55 years in the industry and 22,000 hours experience (M = 17.8 years, 3776 hrs; SD = 13 yrs, 5518.6 hrs). Sixteen respondents held Instructor ratings. The survey asked 11 open ended questions about: pilots’ definition of airmanship; the inclusion of airmanship in initial training; development of airmanship in respondents; awareness of airmanship during current aviation activities; examples of good and poor airmanship; and finally any other comments. The survey responses were analysed using NVivo8 as the structure for determining concepts and themes raised in the responses. Initially, tree nodes were established to reflect the concepts included in Kern’s model. Further free nodes were created as other concepts were discovered in the responses. A total of 36 nodes were developed, each representing concepts that respondents related to airmanship or issues that they raised. Results The questions of most interest for this discussion are Question 1: “What do you think airmanship is?” and Question 2, which asked the respondent to elaborate on the answer given in Question 1. Other questions considered here include questions that asked about the most important aspects of airmanship (Question 6), the situations in which respondents were most aware of airmanship (Question 7) and about good and poor examples of airmanship (Questions 9 and 10). Questions 3 to 5 and 8 were more concerned with the development and training of airmanship, and are discussed elsewhere (Carrick & Chalmers, 2010).
294
Table 1 shows the variety of concepts found in the responses, the number of respondents who mentioned each concept in the Questions 1 and 2, and the total number of respondents mentioning the concept over all the questions. The table is set up to compare the structure of the Kern model, on the left, with some of the other concepts raised by respondents.
The main themes of the Kern model are reflected in some of the respondents’ comments, but there are clearly other concepts, represented in the right hand column, that were also expressed. No-one mentioned all of the Kern concepts. The mean number of concepts raised per respondent was 2.95 in Question 1. Ten respondents did not answer Question 2, the mean across the 30 who did was 2.90 concepts per respondent.
Safety was the concept with the most mentions by respondents in Question 1, Question 2 and across all questions. Kern did not include this in his model, possibly considering it an outcome of good airmanship rather than part of it. The survey respondents were quite concerned about safety with comments like: “Airmanship is a way of being safe”, “An aviator's own attitude towards his / her own safety and the safety of others” and “Sensible aviation practices that take into consideration the safety of yourself and others”.
Table 1
Comparison of Kern concepts and other concepts raised by respondents – number of respondents who mentioned each concept in Questions 1 and 2, and overall (Questions 1 to 11). N = 40
Kern Concepts Other Concepts Q1 Q2 Tot Q1 Q2 Tot Judgement or decision making 2 5 10 Safety 13 14 25 Situational awareness 10 9 22 Courtesy/ethics/respect 12 12 12 Knowledge 6 5 9 Attitude / personal quals 10 0 22 Self 2 3 5 Common sense 9 5 9 Aircraft 2 1 11 Communication 4 4 15
Team 0 0 1 Set priorities, manage flight 3 0 5
Environment 0 0 1 No shortcuts 2 3 3 Rules, regs 9 4 20 Lookout 2 1 19 Risk 3 5 10 Responsibility 2 0 5 Mission: Planning 9 10 16 Knowledge - medical 1 0 1 Skill & Proficiency 12 2 12 Discipline 4 4 5
Judgement or decision making is the major capstone outcome according to Kern, but was not considered by respondents to be particularly central to airmanship, in that only two mentioned it in response to Question 1 and five in their elaboration in Question 2. In contrast, situational awareness (SA) clearly is seen as a significant concept relating to airmanship, with 10 respondents mentioning it in Question 1, nine in Question 2 and 22 respondents including the concept over all questions. Some examples of comments are: “awareness of the operating environment”, “being aware of the situation around you i.e. the weather, other traffic” and “I believe airmanship to be an awareness and understanding of an aviator of their surroundings and what is taking place”. Lookout is an activity upon which SA is dependent, in VFR, and is also frequently mentioned although not so much in response to the first two questions, being mentioned three times, with 19 respondents including it over all questions.
Thirteen respondents referred to knowledge: six made mention of knowledge generally, others specifically related to self (2), the aircraft (2) or to risk (3), with one person commenting on knowledge about medical aid (not something Kern was concerned about). There were no references to knowledge of the team in responses to the definition questions, but this may have
295
been a function of the mostly general and recreational aviation sample, flying mostly single pilot. Nor was there specific reference to the environment. However, when elaborating upon the model, Kern divides environment into the physical, regulatory and organisational environments. When considered at this level, nine respondents mentioned rules, regulations and operating procedures in their definition, and 20 overall.
So far there has not been a published definition of “mission” by Kern, just the inclusion of it in the diagrams presented at a recent seminar series (Kern, 2009). Those respondents who included flight planning, planning generally, preparation and thinking about what one was doing in their responses are shown in “mission” in Table 1. Clearly there is consideration among respondents that planning of what will be going on in flight is part of airmanship, even if it may not have been referred to as “mission”.
Skill and proficiency are separate items in Kern’s model. Twelve respondents to the survey included skill in their definitions of airmanship, or stated “ability to control the aircraft”, “aircraft handling” or “operate an aircraft”. One of these stated that airmanship “is usually referring to judgement and decision making more so than manipulative skill”. Only one respondent specifically mentioned proficiency: “as a pilot without a lot of hours I need to fly regularly enough to remain competent”. One respondent gave as an example of good airmanship: “a flying competition that required pilots to touchdown as close to a spot as possible on the runway” which indicates a skill based component within their conception of airmanship.
Discipline is the cornerstone of airmanship, according to Kern (1996), and it was included in their definitions by four of the respondents, with added comment in response to Question 2 for example: “airmanship is the discipline that you apply to your flying” and “Discipline, implying self discipline should be exercised at all times as an essential element of airmanship”. Three other respondents wrote of “not cutting corners” or “no short-cuts” which could be interpreted to indicate the application of discipline to stick to the task in full.
In addition to the Kern model concepts, the respondents included a number of other concepts in their definition and elaboration of airmanship. There were many mentions of a complex of ideas that can be represented as courtesy, as well as mention of courtesy itself, such as consideration of other airspace users, helping others, manners, doing the right thing, or ethical behaviour (elaborated as “honest in all dealings and treat others as you would wish to be treated”). Courtesy was always mentioned as part of a complex that makes up airmanship (rather than alone) such as the definitions: “A combination of flying skills, common sense and courtesy” and “Being alert and abiding by all the rules and being courteous to other pilots”.
Common sense is a concept that was included by nine of the 40 respondents. Unfortunately common sense is not always very common. Three of the nine elaborated on common sense in Question 2: “Know your abilities and operate within that area of competence” and “the ability to think a situation through logically”. The other response was just another mention of it: “Recognition of the need to apply common sense”. The respondents who included comments on common sense in their responses tended to have been involved in aviation for longer (M = 25.8 years; SD = 15.1) and to have more hours of experience (M = 6213 hours; SD = 8697) than the respondents who did not mention common sense (M = 15.6 years, SD = 11.6; and M = 3068 hours, SD = 4138). What may be considered common sense to someone who has been in the industry for a number of years, may not yet have been learned by a novice. The use of the term may reflect that airmanship has become an ingrained part of the more expert individual to the extent that they do not remember what it was like to be a novice.
Kern discusses communication when writing about characteristics of good team members. Communication is rather more active than just a “knowledge of” item would imply, and is included as a skill set by Ebbage and Spencer (2003). Good communication is essential to maintaining situational awareness and thus safety. This is recognised by the six respondents who mentioned communication in their initial definitions or elaboration, and by 15 respondents overall. Examples of such responses to Question 1 include: “Ability to communicate with other users of aircraft to achieve safe reliable outcomes” and “keeping in contact with other aircraft”.
Some respondents mentioned attitude and various personal qualities as forming part of airmanship, for example: “A combination of knowledge, discipline and attitude toward flying”
296
and “Attitude to safety and fellow aviators.” This category also included other personal characteristics, for example: “Furthermore good airmanship is a reflection of the quick thinking never panicking pilot, who seems to have a dogged determination to keep his plane and contents in a safe configuration”. There was also mention of a desire to do as good a job as possible and having a sense of responsibility.
Finally, the responses included the idea of setting priorities and managing the flight, one of the human factors competencies for flight tests being introduced by CASA (Carrick, 2009), although only mentioned by five respondents overall. The other CASA human factors competencies are lookout, situational awareness, decisions and communication, all mentioned by respondents as part of airmanship.
Table 2 indicates the main concepts raised by respondents when asked: “In your current aviation activities, are you normally aware of airmanship as part of your operation? What would be the most important things that come to mind?” (Question 6). Table 2 also indicates the concepts mentioned in the examples of good and poor airmanship (Questions 9 and 10).
Situational awareness and lookout are frequently mentioned in responses to these questions, supporting the emphasis given to the concepts in the earlier questions. Similarly there were frequent mentions of planning as being important to airmanship, and standard operating procedures (SOPs) and checks were mentioned as frequently, somewhat related to rules and regulations, although these were mentioned only once. Again, there were few explicit mentions of skill or discipline. Communication, or lack of it, was another major issue raised in examples of good and poor airmanship.
Table 2
Concepts raised in Question 6 or in the examples of good and poor airmanship (Questions 9 and 10) N = 40
Concept Frequency
Q6 (Important)
Q9 (Good)
Q10 (Poor)
Judgement/Decision 1 2 Situation awareness 9 8 5 Lookout 9 1 Communication 2 8 6 Knowledge 2 Rules, regulations 1 Planning 8 1 1 SOPs / Checks 9 Skill 1 1 Discipline 1 Attitude/personal quals 1 1 3 Courtesy etc. 1 3 Safety 1 1 Other 8
297
Table 3
Frequency of mention of scenarios or phase of flight in Questions 7, 9 or 10, N = 40
Concept Frequency
Q 7 (When)
Q9 (Good)
Q10 (Poor)
Weather/light 2 9 6 Preflight/planning 5 3 Taxi/runway use 2 5 Circuit / CTAF 8 9 10 Traffic 5 2 1 A/c problem 3 1 Approach/landing 5 1 1 Take-off / landing 8 All the time 10 Agricultural flying 2 1 Non-specific 4 3 Other 4 2 No response 1 10 13
The scenarios shown in Table 3 were mentioned by respondents in answer to Question 7 (“In your current aviation activities, during what phases of flight or events would you be most actively aware of airmanship issues?”) and in the examples of good and poor airmanship (Questions 9 and 10). There is frequent mention of weather and fading light, especially in the examples. Concern about flying in the circuit (landing and take-off could be included here) and in high traffic situations shows the importance of airmanship in interacting with other aircraft and their pilots. A gratifying 25% of the respondents stated that they were aware of airmanship at all times.
Conclusions
While the concepts included by Kern (1996) form a useful structure for consideration of airmanship, based on the responses to this survey it appears that airmanship in Australian aviation is more broadly defined than Kern’s model might suggest. There is little emphasis on discipline, despite its central role in both of the two models of airmanship, which may reflect the primarily civilian sample. Communication appeared as an important concept, it is included in the Ebbage and Spencer (2003) model and perhaps should be more explicitly included in the Kern model. The occurrence of the concept of courtesy shows an awareness of the importance of good interpersonal interaction among the sample group. The frequency of mention of personal qualities and attitudes as part of the overall airmanship construct is also interesting. It implies that airmanship is integral to an aviator’s personality and lifestyle, rather than just something that one does or has when flying. Use of the term “common sense” also reflects this idea of integration within the individual.
298
References
Carrick, K. (2009). Human factors as a component of airmanship. Paper presented at The 8th Industrial and Organisational Psychology Conference, Sydney, 25-28 June, 2009.
Carrick, K. & Chalmers, K. (2010). Development of airmanship. Paper presented at The 9th International Symposium of the Australian Aviation Psychology Association, Sydney, 18th – 22th April, 2010
Ebbage, L. & Spencer, P.D. (2003). Airmanship training for modern aircrew. Paper presented at the Advanced Technologies for Military Training Symposium, Genoa, Italy, 13-15 October 2003.
Kern, T. (1996). Redefining airmanship. New York: McGraw-Hill
Kern, T. (2009). CASA Wrap-up. Power points from the CASA Error Management Roadshow. June 2009.
299
Development of airmanship
KIRSTIE CARRICK* AND KERRY CHALMERS
University of Newcastle
Background
Effective development of airmanship among aviators is important if airmanship is considered an essential quality for safe and efficient flight. Ebbage and Spencer (2003) discuss some ways in which the development of airmanship can be encouraged in military pilots, suggesting that it should be an integrated part of overall flight and ground training rather than a by-product of conventional training. Kern (1996) also promotes the integration of airmanship within ongoing aircraft handling skills training, and suggests three phases of airmanship training. These are: for the instructor to explain the importance of airmanship and instil motivation for improvement; model and teach airmanship; and to evaluate airmanship actively and aggressively. Kern emphasises the importance of the instructor as a mentor to introduce a sound aviation culture, model appropriate behaviours and enforce standards and expectations. Ebbage and Spencer parallel these three phases, also proposing three basic elements to airmanship training: explanation of the concept and its importance, by discussion of case studies; instruction covering the required knowledge, skills and attitudes; and use of objective assessment with provision of feedback. Kern encourages the use of his model of airmanship as a template for the structuring of pre-flight briefings and post-flight debrief and reflection, an activity that should continue throughout one’s flying career, not just take place during initial training.
As part of an ongoing study of airmanship in Australian aviation, aviators were asked to participate in an on-line survey that included questions about the development of airmanship. The aim was to identify the airmanship instruction and development experiences of current Australian pilots.
Method
An on-line survey was promoted at CASA safety seminars in May to September 2008 and in the Flight Safety Australia magazine of May-June 2008. This resulted in 40 responses by the end of September 2008. The experience of respondents in civilian flying ranged from less than one year to 55 years (M= 17.8 yrs, SD = 13.0) and from 28 hours to 22,000 hours (M = 3776 hours, SD = 5518.6). Sixteen of the respondents held an instructor rating (5xCFIs, 4xGrade 1, 2xGrade 2 and 5xGrade 3). The survey comprised of 11 open-ended questions asking respondents about their interpretation of the term “airmanship” (Carrick & Chalmers, 2010) and also about the development of airmanship, the focus of this paper. Questions of interest here are: the inclusion of airmanship in initial training (Questions 3 and 4); factors influential in the development of airmanship in respondents (Question 5); and, for the instructors who responded, how they ensured the development of airmanship in their students (Question 8).
300
Results Inclusion of airmanship in initial training
The majority of respondents stated that airmanship formed some part of their initial flight training. The questions were designed to determine what happened in the air, as part of flying instruction (Question 3), as distinct from what was included in briefs or ground instruction (Question 4).
Question 3: Was the development of airmanship a specific part of the initial flight training you did? Please explain what this involved.
Eighteen respondents stated “yes”, some with added explanation. Three listed airmanship related activities such as checks, lookout and procedures. A further six responses were along the lines of “not really, but…” and included concepts to do with airmanship. Two respondents indicated that airmanship was part of their initial instruction, but that this was only skill related or did not include human factor related concepts. This gives a total for inclusion of airmanship (however defined) as 29, or 73% of the respondents. Eight respondents indicated that airmanship was not addressed (20%). Two respondents indicated that it was so long ago that they could not remember, one stating: “I cannot recall that airmanship was specifically discussed when I was learning to fly in 1954. I sometimes wonder if common sense was more expected in those days.” One respondent did not answer the question. Examples of positive responses are: “Yes. From the start my instructors emphasised that airmanship was what defined a good pilot and they demonstrated that right through my training: lookout, ground checks, aircraft handling and so on were all part of each flight” and “in each brief, airmanship was mentioned in regard to lookout, communications and aircraft awareness”. Several responses to this question included mention of briefs, so the distinction between Questions 3 and 4 was not necessarily picked up by the respondents. Question 4 (a) In your initial flight training, was airmanship, or concepts related to airmanship, discussed in ground training or pre- or post- flight briefings?
There was a response of “Yes” from 67.5% of respondents (N=27) and “No” from 25% (N=10). Three respondents did not answer the question. One of those did not answer any questions except Question 1, the other two had answered Question 3, one with “don’t remember”, the other with “no”.
Question 4 (b) If so which concepts and what sort of discussions?
Nine respondents left this question blank as they had replied “No” to Question 4a (or did not answer), a further four said that airmanship was not a major item, or only discussed in a general sense and two simply mentioned discussion in the bar or “hanger talk” as the source of information about airmanship.
Most respondents included more than one item in replying to Question 4b. The most frequent concept mentioned was lookout listed by 14 respondents. Related responses were: traffic (4 mentions), communication or radio use (4), and listen-out (2). Checks were listed by six respondents and procedures by two. Safety issues were mentioned by three respondents. Planning was included by four respondents. There were various other considerations including: handing-over/taking-over (2); and one mention each of: developing a flow of activity; relationship with instructor; fuel logs;
301
courtesy; and common sense. Two respondents noted that the content was specific to the lesson.
There were ten aircraft or skill related comments including general handling (2 mentions) and understanding of systems (2), and one mention each of: using airworthy aircraft; aircraft awareness; taxi speeds and run up into wind; aircraft performance; temperatures and pressures; and energy and fuel state. Five of these respondents did not include any non-aircraft handling concepts.
Most respondents had some discussion of airmanship in their initial training, but for some of the respondents airmanship was not discussed, or was limited to aircraft handling or aircraft state issues alone. While these are part of airmanship, a broader view of airmanship includes the human factors aspects of situational awareness and judgement. Of the five respondents who only listed aircraft or handling issues in response to these questions, only one really had the broader view when the definitions given in the first questions defining airmanship were reviewed. Two wrote a little about issues that could be interpreted as situational awareness and the other two gave quite narrow, aircraft handling related definitions. Two of these respondents had over 20 years of experience, but the other three were instructed fairly recently, with 2.5, 3 and 6 years of experience as PPL holders. Given that human factors has been part of the syllabus for the PPL, as human performance and limitations, for approximately 20 years, it is interesting that there are pilots who have not incorporated these concepts into their understanding of airmanship. On the other hand, it may be that their instructors had not discussed human factors in terms of airmanship, or airmanship in terms of human factors.
Looking at the responses from another angle, there were at least 12 respondents who made no mention of aircraft handling skills when writing about the discussion of airmanship in their initial training. This may be a reflection of the military training system of assessing technical skills and airmanship as separate aspects of flight proficiency (Carrick, Graham, Healey & Pickard, 2008). Although only three respondents indicated a background in the military, it may be that the instructors of the non-military respondents had a military background, and so perpetuate the separation of technical skill and airmanship.
Influences on the development of airmanship
When asked what had led to the development of their own sense of airmanship, respondents mentioned experience, reading, developing a sense of responsibility and the influence of the people around them. Self-preservation and fear were also included.
Question 5: Thinking about your progress to date as a pilot, what do you feel led to the development of your sense of airmanship?
As it was an open-ended question, several respondents listed more than one influence on their development. Only one person did not provide a response. Experience was included by 13 of the respondents. Ten respondents listed a personal desire to get it right, or developing maturity or responsibility. The influence of other people was commonly included as: their training (by six respondents); talking to others (6); reading (6); the mistakes or experiences of others (5); losing a friend or colleague in an accident (3); and the Club environment (2). Self-preservation or fear was mentioned by five respondents. Helping to teach others was included by three respondents and common sense also by three respondents. One person mentioned luck, and another simply wrote
302
“leaving a fuel cap off”. Some examples are: “Fear of having an accident because of a minor mistake, and a desire to be [as] prepared and professional as possible”; “Watching and talking with other pilots to develop my own ideas of airmanship from examples around me I identified with [and] being scared and not wanting to go through a similar experience again”; and “Common sense and experience. Be ready for the unexpected.”
Ongoing experience in aviation is seen as important in developing a sense of airmanship, but also the observation of what others do, whether these others are initial instructors, fellow aviators, or observed through reading. This supports the comments of both Kern (1993) and Ebbage and Spencer (2003) that early instructors provide a significant role model, but extends further to imply that learning from others continues as the aviator develops post ab-initio training. There is also a clear internal motivation to improve or excel, to be as professional as possible, among some of the respondents. This is included by Ebbage and Spencer as one of the attitudes that form the basis of airmanship and that should be encouraged in initial training.
Practice of current instructors
Instructors reported that they ensure the development of good airmanship among their students by modelling good airmanship themselves, as well as by discussions and briefs during training.
Question 8: If you are currently an instructor, what do you do to ensure that your students develop good airmanship?
There were 16 instructors among the sample, and 17 responses and 23 not applicable or nil answers to this question. One instructor did not answer the question, and two respondents who were not instructors, did and stated that they intended becoming instructors. Of the 17 responses, eight respondents wrote that they provide good example. Two mentioned getting their students to read, one specifically mentioned the book, Redefining Airmanship (Kern, 1996). Five respondents listed discussion, briefs and in-flight emphasis and one wrote that he makes sure that students have a system for managing flights. There were single mentions of the following: right attitude and clear understanding; encourage them to think; try to instil points mentioned; part of post-flight self-evaluation; and “keep yourself alive and keep a good lookout”. One person noted that he “hit them (not very hard) when they turn without looking (they all get hit at least once).”
The instructors who responded to the survey did not mention either model of airmanship in responding, although one did mention Kern (1996) as a resource for students. While the pattern of instilling motivation, teaching the skills and providing feedback may be part of the instructor training system and be a recommended structure for instruction, there were only five respondents who wrote about briefs, and one who specifically mentioned post-flight self-evaluation. The role of an instructor as mentor or model, providing a good example, was included by eight of the respondents, or 50% of instructors, so acceptance of this aspect of developing airmanship is more widespread.
303
Conclusions
Based on the responses to this survey, it appears that airmanship is generally included in student instruction in Australian aviation, but perhaps not consistently across all training establishments, given that 20% of respondents stated airmanship was not part of their initial training. In addition, individual experiences and the quality of airmanship displayed by the people with whom the student pilot associates is clearly influential in the development of airmanship, and continues to be important after initial training.
The close examination of responses to Questions 3 and 4 indicates that where the initial training did include airmanship concepts, it did not always reflect a truly integrated view of airmanship. There appear to be three groups of respondents: those for whom airmanship was presented as an integration of flying skills and human factors concepts; those for whom airmanship referred only to flying skills and aircraft knowledge related items; and those for whom airmanship referred to only the human factors related concepts, not including technical flying skills. Responses to the initial questions on the survey asking for a definition of airmanship included only 12 mentions of skill or technical proficiency (Carrick & Chalmers, 2010); thus 28 responses did not include these aspects in their initial definition of airmanship and so probably consider airmanship to be predominantly about non-technical aspects of flying.
It should be noted that this survey concluded before the CASA Safety Seminars addressing airmanship occurred, and that those seminars did present an integrated model of airmanship. Of the respondents who had some level of integration in their responses, none outlined fully either of the two models that have been presented here. It would be interesting to investigate whether there has been any change in pilots’ conceptions of airmanship following the CASA airmanship Safety Seminars and an article on airmanship in the Flight Safety Australia magazine by Marchbank, (2008). The CASA Error Management Roadshow featuring Tony Kern in June 2009 also presented an integrated model of airmanship, and a parallel model for maintenance engineers, in the closing session of the day.
A limitation of any generalisation from these results is that it is a very small sample, recruited from among those interested enough in safety to attend CASA Safety Seminars, or to read Flight Safety Australia, and thus may not reflect the views of the wider aviation community. Also it is possible that the respondents were writing only the most important aspects of their reactions to each question, and were not willing to take much time to explain their activities or views in depth. Most respondents wrote no more than a few words or a single sentence in response to the questions asked. By comparison, interviews tend to elicit far more information and allow additional questions to be asked. However, interviews are not very practical for an extensive survey.
304
References Carrick, K. & Chalmers, K. (2010). Airmanship: Voices from an on-line survey. Paper
presented at The 9th International Symposium of the Australian Aviation Psychology Association, Sydney, 18th – 22th April, 2010
Carrick, K, Graham, S., Healey, A. & Pickard, S. (2008) An assessment of feedback and evaluation at a flight training school. The 8th International Symposium of the Australian Aviation Psychology Association, Sydney, 8th – 11th April, 2008
Ebbage, L. & Spencer, P.D. (2003). Airmanship training for modern aircrew. Paper presented at the Advanced Technologies for Military Training Symposium, Genoa, Italy, 13-15 October 2003.
Kern, T. (1996). Redefining airmanship. New York: McGraw-Hill
Kern, T. (2009). CASA Wrap-up. Power points from the CASA Error Management Roadshow. June 2009.
Marchbank, M. (2008, Nov-Dec) Is airmanship dead? Flight Safety Australia, 65, 8-13
305
Appendix F
Survey reported in Chapter 6
306
307
308
309
310
311
312
Page left deliberately Blank
313
Appendix G
Between and within domain differences from Chapter 6
An expectation of the data was that the ‘within-domain’ relatedness would be
greater than the relatedness of the relevant pairings with concepts from other domains
(between-domain). This can be explored on three levels: concept level, domain level
and overall. At the concept level, each concept has two within-domain comparison
means (shown in bold) and a row and column of the mean relatedness of the between-
domain comparisons, shown in Table 6.4. Ideally, the bold means should be greater than
the non-bold means. Secondly, each domain has three within-domain relatedness means,
and four sets of nine between-domain comparisons (represented by the intersections of
the domain names above the columns and beside the rows in Table 6.4). The within-
domain means should exceed the between-domain means, and this can be assessed by
calculating a ‘mean relatedness’ for each set and comparing these means through
analysis of variance. Thirdly, the overall the within-domain relatedness mean should
exceed the overall between-domain relatedness mean, a comparison between the means
along the diagonal and the non-diagonal un-bolded means. A combined relatedness
variable can be calculated for each of these and a comparison conducted.
Overall.
At the overall level, a repeated measures t-test indicated that overall within-
domain relatedness (M = 4.11, SD = .412) is significantly greater than overall between-
domain relatedness (M = 3.63, SD = .490), t (69) = 13.06, p < .001, and this is a large
difference (d = 1.07).
Domain level.
An example of the domain level comparison is that the preparation within-domain
pairs should show greater mean relatedness than preparation concepts paired with any of
the non-preparation concepts. This can be assessed by creating a domain relatedness
rating for each participant, by adding together the relatedness ratings for the pairs in
each domain combination that included preparation (three means for the within domain
pairs and nine for each of the between-domain sets) and finding the mean for each set,
creating four between-domain relatedness means. This results in a preparation within-
domain relatedness mean, and a between-domain relatedness mean for each of
314
preparation and knowledge, preparation and non-technical skills, preparation and
personal attributes, and preparation and technical skills.
Figure I.1 shows the means for group relatedness. There is one panel for each
concept group, showing the mean relatedness for each set of pairs, thus in the first
panel, displaying the ‘knowledge’ mean relatedness, the X-axis shows K_K for the
within group mean; K_NT for the knowledge-non-technical skills group mean; K_P for
the knowledge-preparation group mean; K_PQ for the knowledge-personal attributes
group mean and K_T for the knowledge-technical Skills group mean. The second panel
indicates that same pairings but with Non-technical Skills rather than knowledge. The
knowledge-nontechnical skills relatedness mean uses the same data as in the knowledge
panel, but may be slightly different based on the list-wise treatment of missing values
for each set of means. Similarly, all ‘out-group’ group relatedness data are used twice,
in a panel for each member of the pair. In addition each panel differs in the scaling used
on the y-axis.
The results of repeated measures ANOVA using the group relatedness scores for
each concept group is reported below. SPSS23 (IBM, 2015) was used with guidance
from Allen & Bennett (2010). As there were numerous comparisons made, the
Bonferroni adjusted alpha level was again set to p < .001. List-wise deletion for missing
values for each analysis led to different numbers of participants for each analysis, so the
number of participants included in each analysis is reported in each summary, as this is
not obvious from the degrees of freedom due to the use of the Huynh-Feldt correction.
315
Figure I.1 Group relatedness means for Knowledge (K), Non-technical skills (NT),
Preparation (P), Personal attributes (PQ) and Technical skills (T), showing 95%
confidence intervals. Be aware of the change in the scaling of the y-axis between
panels.
Appendices 316
Knowledge comparisons. The Fmax statistic indicated that the assumption of
homogeneity of variance was not violated. The Kolmogorov-Smirnov statistic for the
assumption of normality indicated that the K_K distributions was not normal, but that
all the other distributions were normal. Descriptive data indicated that the skew and
kurtosis of the K_K distribution was not excessive, but the detrended Q_Q plot showed
some deviation from normality. Mauchly’s test indicated that the sphericity assumption
was violated and consequently the Huynh-Feldt correction was employed.
The repeated measures ANOVA indicated that there is significant difference
among the Knowledge group relatedness means, F (3.376, 236.343) = 19.50, n = 71, p <
.001, ƞ²ʟ = .112 (a large effect size).
A series of pairwise comparisons revealed that K_PA relatedness (M = 4.03, SD =
0.58) was significantly greater than group relatedness for K_K (M = 3.68, SD = 0.56),
K_NT (M = 3.57, SD = 0.53), and K_T (M = 3.52, SD = 0.44) and these did not differ
among themselves. The K_PA group relatedness did not differ significantly from the
K_P group relatedness (M = 3.81, SD = 0.52) and the K_P group relatedness was
significantly greater than the K_T group relatedness.
This result does not support the expectation that mean relatedness of the
Knowledge within group pairings would be significantly greater than the other pairings
of concepts with Knowledge concepts.
Non-Technical skill comparisons. The Fmax statistic indicated that the
assumption of homogeneity of variance was not violated. The Kolmogorov-Smirnov
statistic for the assumption of normality indicated that the distribution of scores for all
the groups, except the NT-NT distribution, were normal. The NT_NT distribution was
not significantly skewed, nor was the kurtosis significant, and the Q-Q plots were as
expected. Mauchly’s test indicated that the sphericity assumption was violated and
consequently the Huynh-Feldt correction was employed.
The repeated measures ANOVA indicated that there is significant difference
among the Non-technical skill group relatedness means, F (3.548, 266.093) = 23.827, n
= 76, p < .001, ƞ²ʟ = .086 (a medium effect size).
A series of pairwise comparisons indicated that group relatedness for NT_NT (M
= 4.01, SD = 0.56) was significantly greater than group relatedness for NT_K (M =
Appendices 317
3.60, SD = 0.62), PA_NT (M = 3. 70, SD = 0.69) and T_NT (M = 3.52, SD = 0.57), but
did not significantly differ from group relatedness for P_NT (M = 3.81, SD = 0.56).
Group relatedness for P_NT was significantly greater than group relatedness for T_NT.
The NT_K, NT_PA and the T_NT means did not differ significantly among themselves.
This result does show support for the expected closer relatedness of the within
group non-technical skills concepts compared with the pairs between these and other
group concepts. Non-technical skills and preparation concepts are more closely related
that non-technical and other domain concepts.
Preparation comparisons. The Fmax statistic indicated that the assumption of
homogeneity of variance was not violated. The Kolmogorov-Smirnov statistic for the
assumption of normality indicated that the distribution of scores for the P_P group
relatedness scores was non-normal, however the other four distributions were normal.
The box-plots of the data indicate that the P_P distribution has a ceiling effect, the
values are near the top of the range. Mauchly’s test indicated that the sphericity
assumption was violated and consequently the Huynh-Feldt correction was employed.
The repeated measures ANOVA indicated that there is significant difference
among the Preparation group relatedness means, F (3.576, 271.8) = 76.879, n = 77, p <
.001, ƞ²ʟ = .30 (a very large effect size).
A series of pairwise comparisons indicated that group relatedness for P_P (M =
4.27, SD = 0.542) was significantly greater than for K_P (M = 3.85, SD = 0.55), P_NT
(M = 3.82, SD = 0.59), PA_P (M = 3.81, SD = 0.71) (which did not differ among
themselves) and P_T (M = 3.06, SD = 0.60). In addition, group relatedness for P_T was
significantly lower than for any of the other concept group pairs.
This result supports the expectation that the within group relatedness mean for
preparation concepts would be significantly higher than the group relatedness of the
out-group pairings between preparation concepts and other concepts.
Personal attributes. The Fmax statistic indicated that the assumption of
homogeneity of variance was not violated. The Kolmogorov-Smirnov statistic for the
assumption of normality indicated that the distribution of scores for the K_PA, PA_P
and PA_T group relatedness scores were normal, with the distributions of PA_PA and
PA_NT being non-normal. The box-plots of the data indicate that the PA_PA
Appendices 318
distribution was restricted in range as it has a ceiling effect, although the skew was not
significant. Similarly, skew and kurtosis of the PA_NT distribution were not significant.
Mauchly’s test indicated that the sphericity assumption was violated and consequently
the Huynh-Feldt correction was employed.
The repeated measures ANOVA indicated that there is significant difference
among the Personal attributes group relatedness means, F (2.961, 228.017) = 25.716, n
= 78, p < .001, ƞ²ʟ = .089 (a medium effect size).
The pairwise comparisons indicated that group relatedness did not differ
significantly for PA_PA (M = 4.15, SD = 0.69) and PA_K (M = 4.06, SD = 0.59)
however each was significantly higher than group relatedness for PA_NT (M = 3.68, SD
= 0.68), PA_P (M = 3.79, SD = 0.66) and PA_T (M = 3.62, SD = 0.68) which did not
differ significantly among themselves.
This result partially supports the expectation that the personal attributes within
group relatedness (PA_PA) would be higher than relatedness with out-group concepts,
but lack of difference with group relatedness for pairings with knowledge concepts
(PA_K) is not as expected.
Technical skills. The Fmax statistic indicated that the assumption of homogeneity
of variance was not violated. The Kolmogorov-Smirnov statistic for the assumption of
normality indicated that the distribution of scores for the T_T group relatedness scores
was not normal, with the other four distributions being normal. The box-plots of the
data indicate that the T_T distribution has a ceiling effect (the scores are against the
upper limit of the scale) and hence has a negative skew. Mauchly’s test indicated that
the sphericity assumption was violated and consequently the Huynh-Feldt correction
was employed.
The repeated measures ANOVA indicated that there is significant difference
among the Technical skills group relatedness means, F (3.233, 226.301) = 145.3, n =
71, p < .001, ƞ²ʟ = .465 (a very large effect size).
Post-hoc pairwise comparisons showed that group relatedness for T_T (M = 4.54,
SD = 0.46) was significantly higher than group relatedness for K_T (M = 3.51, SD =
0.45), T_NT (M = 3.47, SD = 0.55), PA_T (M = 3.57, SD = 0.71) and P_T (M = 2.99,
SD = 0.51) In addition, group relatedness for P_T was significantly lower than group
Appendices 319
relatedness for the other combinations of concepts. Group relatedness for K_T, T_NT
and PA_T did not differ significantly.
This result supports the expectation that the within group relatedness for technical
skills would be greater than for pairs with concepts from other groups. It also indicates
that preparation and technical skills are seen as particularly unrelated.
Summary. The group relatedness means indicate that preparation and technical
skills are distinct groups; in both cases the within-group group relatedness mean was
significantly higher than the out-group group relatedness means. The non-technical
skills group relatedness mean was significantly greater than all but the out-group
combination with preparation, and thus non-technical skills may be related to
preparation, rather than any of the other groups. The personal attributes within-group
group relatedness mean was significantly greater than all the out-group combinations
except that with knowledge, indicating a relationship between personal attributes and
knowledge. The knowledge within-group group relatedness mean was significantly
lower than the out-group combination with personal attributes, and not significantly
different from any of the other out-group group relatedness means, lending further
support to the idea of such a relationship.
Concept level
Visual inspection and comparison of each bold (within domain) mean with the
other means in the relevant row and column (between-domain pairs) in Table 6.4, shows
that in general the within-domain mean relatedness tends to be higher than the mean
relatedness for between-domain pairs. The exception is seen among pairs made with
‘knowledge’ concepts. While ‘knowing SOPs’ and ‘knowledge of rules & regulations’
were seen as quite closely related (high relatedness), these concepts paired with ‘aircraft
type knowledge’ were less closely related, and most of the means for the between-
domain pairs in the aircraft type knowledge column are greater than the within-domain
means. This contrasts with the ‘knowing SOPs’ and ‘knowledge of rules & regulations’
comparisons, for which none of the between-domain means exceeded the mean
relatedness between these two concepts.
Carrying out 15 ANOVAs on 14 means each, is probably over-analysing the data,
and a more appropriate way to look at the detail of this information is through the multi-
dimensional scaling (MDS) analysis reported in Chapter 6.
Appendices 320
Appendix H
Copyright Permissions
1. Tony Kern
Kirstie
I am the copyright owner and you have my permission. Good luck!
Tony
Sent from my Verizon Wireless 4G LTE smartphone
-------- Original message -------- From: Kirstie Carrick <kirstie.carrick@newcastle.edu.au> Date: 5/4/2016 23:30 (GMT-07:00) To: Tony Kern <Tony@convergentperformance.com> Subject: Use of Airmanship Model diagrams
Hello Tony,
I am contacting you to seek written permission to copy and communicate the following material within the electronic version of my PhD thesis titled ‘Airmanship in Australian Aviation’.
1. The diagram of the airmanship model as it appears on page 22 of Redefining Airmanship, (1996, McGraw Hill)
2. The Airmanship model revised as it appeared in Slide 11 of the Roadshow Wrap-up presentation at the CASA Threat and Error Management Roadshow (2009).
If you are not the rights holder for this material, I would be grateful if you would advise me who to contact.
The thesis will be made available on the internet via the University of Newcastle’s online digital repository http://nova.newcastle.edu.au/
Regards,
Kirstie Carrick
Appendices 321
2. Phil Spencer
3. Australian Transportation Safety Bureau
Creative Commons licence
With the exception of the Coat of Arms, ATSB logo, and photos and graphics in which a third party holds copyright ,this publication is licensed under a Creative Commons Attribution 3.0 Australia licence.
Creative Commons Attribution 3.0 Australia Licence is a standard form license agreement that allows you to copy, distribute, transmit and adapt this publication provided that you attribute the work.
The ATSB’s preference is that you attribute this publication (and any material sourced from it) using the following wording: Source: Australian Transport Safety Bureau
Copyright in material obtained from other agencies, private individuals or organisations, belongs to those agencies, individuals or organisations. Where you want to use their material you will need to contact them directly.
Recommended