IATA Air Carrier Self Audit Checklist – Analysis Operations...Carrier Self Audit Checklist – ... IATA Air Carrier Self Audit Checklist ... The time available for pilots to decide

IATA Air Carrier Self Audit Checklist – Analysis (RERR 2nd Edition - Issued 2011)

IATA Air Carrier Self Audit Checklist - Analysis 2 RERR 2nd Edition

NOTE DISCLAIMER. The information contained in this publication is subject to constant review in the light of changing government requirements and regulations. No subscriber or other reader should act on the basis of any such information without referring to applicable laws and regulations and without taking appropriate professional advice. Although every effort has been made to ensure accuracy, the International Air Transport Association and the contributors to this publication shall not be held responsible for any loss or damage caused by errors, omissions, misprints or misinterpretation of the contents hereof. Furthermore, the International Air Transport Association and the contributors to this publication expressly disclaim any and all liability to any person or entity, whether a purchaser of this publication or not, in respect of anything done or omitted, and the consequences of anything done or omitted, by any such person or entity in reliance on the contents of this publication. The International Civil Aviation Organization (ICAO) and other contributors’ opinions expressed in this publication do not necessarily reflect the opinion of the International Air Transport Association. The mention of specific companies, products in this publication does not imply that they are endorsed or recommended by the International Air Transport Association in preference to others of a similar nature which are not mentioned. © International Air Transport Association 2011. All Rights Reserved. No part of this publication may be reproduced, recast, reformatted or transmitted in any form by any means, electronic or mechanical, including photocopying, recording or any information storage and retrieval system, without the prior written permission from: Senior Vice President Safety, Operations and Infrastructure International Air Transport Association 800 Place Victoria, P.O. Box 113 Montréal, Québec


INTRODUCTION The purpose of this industry wide audit survey was to sample industry implementation of best practices in operating commercial transport aircraft, and was specifically aimed at practices that reduce the risk of runway excursions. The IATA Safety department was very pleased to note that we received complete responses to this on-line query from 120 of the 210 carriers requested, representing an extraordinarily high response rate of 57%, and included responses from all IATA global regions. The 98 question survey was conducted from Nov 2009 to Aug 2010. It covered aircraft performance planning, takeoff and landing flight crew procedures, contaminated runway procedures, the use of deceleration devices (engine reverse thrust, ground spoilers, etc), go-around policies, and the use of Flight Data Analysis for monitoring operational performance and preventing runway excursions. The vast majority of these responses indicated that industry carriers have incorporated the majority best practices, which is reflected in the constantly improving industry safety performance. In total, 9673 yes or 91 percent. “NO” responses in some areas provide insight into potential areas for improved training, both at the individual carrier and industry level, and suggest other opportunities to reduce the risk of runway excursions. In some cases a specific issue may not be applicable to a particular operator, and therefore these percentages are not exact indicators.


Issues where some carriers indicated that they did not fully implement these procedures include:

• Identification of “critical” runways was not being accomplished by 24% of the respondents. While this may indicate that some operators do not have any “critical” runways in their system, it is more likely that the identification and management of critical runway threats is not universally understood. This is a missed opportunity.

• Training regarding directional control following an engine failure at a very low speed on takeoff. Responses to the questions regarding directional control on takeoff identify opportunities for improved training in directional control on take-off, especially with regard to adverse environmental factors and the management of low speed rejected take-offs. Some carriers’ training packages may not adequately address the significance for directional control of an engine failure below Vmcg.

• Training regarding performance of necessary abort actions when the first officer is flying is an important consideration and is affected by who makes the go/no-go decision and who handles the rejected takeoff.

• The most significant issue identified appears to be the apparent absence of a policy requiring pilots to check or verify landing performance for each arrival at 27% of the carriers.

• All responding carriers had a policy requiring a go around in response to an unstabilized approach, yet approximately 10% of respondents gave a NO answer with regard to the existence of a non-punitive culture towards go arounds.

• A significant training opportunity was identified by the 20% of the carriers who do not train pilots regarding the ability to go-around at very low altitudes and after touchdown.

• Almost 10% of the respondents do not currently limit operations on contaminated runways with aircraft having inoperative retardation systems, and 5% not currently training pilots to use maximum retardation immediately after touchdown on a contaminated runway.

• Many potential improvements in expanding Flight Data Analysis (FDA) programs were also identified

A detailed review of each of the major self audit checklist areas is in the following sections. This industry wide survey is a valuable indicator regarding the implementation of industry best practices, and may provide individual carriers with potential areas for improvement.


ANALYSIS OF RESPONSES

Total responses received as at 20 August 2010 - 120 Overview The authors of the Self Audit Checklist designed it such that the ‘right’ answer to most, if not all, questions was YES, insofar as a YES answer indicates a higher degree of risk awareness and threat management with respect to runways excursions. It is encouraging from an industry perspective to see that virtually all of the charts of responses received show an overwhelming majority of YES answers, suggesting that in the areas explored by the Checklist and among those carriers who responded, there is a good understanding of the excursion risks encountered during the take-off, approach and landing phases, and of the strategies and best practices known to address them. Of course some consideration must be given to the probability of a greater propensity to respond to the Checklist in the knowledge that the carrier was largely compliant, than if it was not, and the statistical conclusions drawn from the results should bear that in mind. Finally, what should be construed as the implication of a NO answer? Does that mean the carrier has somehow failed? Not at all – what it does is to allow the responding carrier to identify target areas on which to focus in the universal effort towards continuous improvement in air safety in general and the management of runway excursions risks in particular. Chart 1 - General The chart shows a high standard of overall risk awareness and associated monitoring but if there is a weakness it appears in the recognition and management of cumulative runway excursion risk factors. This is perhaps one of the most difficult aspects to evaluate because factors interact with each other in unpredictable ways in each set of circumstances and it is hard to define the magnitude of the combined risk. Furthermore, factors like weather and runway contamination are dynamic and the final risk assessment must necessarily be completed by the pilots in a critical high workload environment. Nevertheless, it is vital for carriers to recognise that management strategies developed in response to individual threats may not be sufficient to deal with combined or cumulative threats, and then develop programs to systematically identify and catalogue permanent threats like short runways or terrain, and monitor and highlight critical changes in dynamic threats like weather. In the Runway Excursion Risk Reduction (RERR toolkit) there is a “Risk Assessment Tool” that addresses the cumulative effect of multiple threats.


Chart 1


Chart 2 – Take-off Performance I This chart reassuringly confirms an almost universal understanding of the importance of training and implementation of take-off performance considerations, especially the dynamic influences of weather and runway conditions. What is not addressed in this, or any other of the take-off performance sections of the Checklist is the importance of starting the performance calculation with the correct aircraft take-off weight. While seemingly obvious, history shows that fundamental and substantial errors in take-off weight entry have contributed to a continuing sequence of runway excursion accidents. In all cases, carriers need to provide pilots with reliable gross error checks and error tolerant procedures to manage the potential for mistakes due to complacency, distraction and high-workload.

Chart 2


Chart 3 – Take-off Performance II The chart shows that the Checklist queried some (though perhaps not all) of the vital components of a performance training package and the answers indicate that a small proportion of respondents omit some of these elements from their program. Those who would have answered NO to any of these questions should undoubtedly review their training to ensure that those missing elements are addressed adequately in future but even those who answered YES would be wise to conduct regular auditable reviews of the training syllabus to ensure that all pilots have an up-to-date and relevant understanding of the effects of environmental and operational factors upon take-off performance.

Chart 3


Chart 4 – Take-off Performance III This is the first chart in the series to show a significant proportion of NO answers – an average of almost 20% across the 3 Checklist questions - suggesting that the identification and management of critical runway threats is not universally understood. This is a missed opportunity because the factors characterising a critical runway may be identified, quantified and catalogued at leisure in an office environment, long before they are encountered by pilots. Changes are highlighted by NOTAM and the magnitude of criticality can be reassessed and promulgated as necessary. For those carriers who do identify critical runways, it is essential that the process includes a regular reassessment of the remaining non-critical runways to confirm that their status remains unchanged – a long runway shortened by work-in-progress may immediately become performance critical and failure to recognise these changes on the day have led to several accidents and incidents.

Chart 4 Note: critical runways may not apply to all operators, but defining a critical runway length (or operational safety margin) is a good way for an operator to manage risk. A runway that is not normally “critical” may become critical due to increases in operating weight, thrust restrictions, runway construction, or many other factors.


Chart 5 – Rejected Take-off I For the most part the chart shows a sound understanding of the need for robust procedures and training for rejected take-offs, and with good reason. The time available for pilots to decide whether to stop or go, and the criticality of getting the decision right mean that they must be exposed to the full spectrum of possibilities and outcomes at regular intervals in training, and be equipped with simple and effective rules to govern their decision making. Almost 20% answered NO to question 21 regarding training applicable to rejected take offs above V1, and this may be that carriers are reluctant even to address this scenario – again with good reason. Pilots and the industry as a whole should be unequivocal in the understanding that a rejected take-off above V1 will almost inevitably result in an overrun. History supports that position.

Chart 5


Chart 6 – Rejected Take-off II The chart shows that the majority of carriers employ a comprehensive rejected take-off training program, which addresses many if not all of the potential factors in the go/no-go decision. However, the most significant proportion of NO answers relate to training for conditions which do not necessarily offer obvious go/no-go indications to the pilots – vibration, tire failure etc – and for conditions which, while disconcerting, do not warrant a rejected take-off. As mentioned in the previous chart analysis, the decision time available to pilots is minimal and the more experience and knowledge they have, the more likely they are to arrive at the best and safest conclusion. Airlines should consider discussing potential takeoff events that are not engine failures (tire failures, door warning lights, fire warning lights, etc) that may or may not require high speed rejected takeoffs.

Chart 6


Chart 7 – Directional Control on Take-off I This chart shows a small proportion of NO answers, indicating potential deficiencies in training for directional control on take-off, especially with regard to environmental factors and the management of low speed rejected take-offs. More particularly the responses show that some carriers’ training packages may not adequately address the significance for directional control of an engine failure below Vmcg. The important point regarding an engine failure below Vmcg is that the only possible means of controlling the aircraft is reducing thrust on the remaining engine(s). At very slow speeds, loss of control from a single engine failure can quickly result in a runway excursion.

Chart 7


Chart 8 – Directional Control on Take-off II Significantly the chart shows that training almost universally addresses the issue of who actually makes the go/no-go decision – essential as there is no time for debate. However, the training programs of 7% of respondents do not cover rejected take-offs when the First Officer (FO) is flying. This is an important consideration and is affected by who makes the go/no-go decision and who handles the stop manoeuvre. Implementing a decision made by the other pilot and/or handing over control to the other pilot both have implications in terms of time and performance, and must not be underestimated in the outcome of a rejected take-off. Rejected takeoffs with the FO flying should be a part of simulator training programs to ensure that the decision is properly executed in a timely manner.

Chart 8


Chart 9 – Aircraft Loading I While the majority of respondents reported comprehensive procedures for the management of aircraft loading and the reporting of loading errors and deficiencies, nearly 10% of responses overall indicated some potential gaps, especially in the mechanism for ground staff to report such incidents. The risks associated with loading deficiencies, affecting aircraft weight and balance for take-off performance among other things, may not always be apparent to those involved in loading and disciplinary processes may not offer an incentive to report errors. It is vital that the prevailing culture allows for and encourages the recognition and disclosure of loading errors before they manifest themselves in controllability and handling issues.

Chart 9


Chart 10 – Aircraft Loading II The chart shows a general compliance with best practice in the determination and confirmation of flap setting, stabiliser position and V speeds in relation to the aircraft weight and balance, and cross-checking of relevant data in the FMS to identify potentially hazardous disparities. No matter how careful or compliant an individual may be, once he has adopted a particular situational model he is far less likely to recognise flaws or errors in that model than an independent party – hence the vital need for procedural cross-checks of all selections and entries. The NO answers to Question 40 with regard to checking configuration against load information may be due to a lack of clarity in the question itself.

Chart 10


Chart 11 – Landing Performance Against a general picture of sound processes with regard to landing performance, a number of NO answers indicate that some carriers may need to review this aspect of SOPs. The most significant deficiency appears to be the apparent absence of a policy requiring pilots to check or verify landing performance for each arrival. Bearing in mind that no carrier would allow a take-off without prior calculation of the take-off performance, no matter what the conditions, and that more accidents occur on landing than on take-off, it would seem a natural conclusion that landing performance should be calculated and recorded for every landing. The recording of the data also allows for a comprehensive audit process to confirm compliance and data accuracy.

Chart 11


Chart 12 – Unstabilized Approach I Responses in this chart show a reassuringly consistent understanding and management of policy and procedures with regard to unstabilized approaches and the setting of associated criteria. The only NO answers relate to the definition of a minimum altitude for selection of landing configuration and of responses to flight path deviations on the approach, both of which are generally accepted to be essential elements of approach stabilization policy.

Chart 12


Chart 13 – Unstabilized Approach II The NO answers to question 54, relating to the use of examples of accidents resulting from unstabilized approaches represent a missed opportunity for those and any other carriers who would answer NO. The information is freely available and the message to pilots is compelling. However, care must be taken to ensure that the learning points conveyed are appropriate, accurate and kept fresh with the introduction of a changing program of accidents in the package. Most respondents’ checking standards included the confirming the ability of pilots to execute a stabilized approach and this should be seen as essential to reinforce the stated policy and training. However, the standards should also examine the ability of pilots to recognise and recover from an unstabilized approach.

Chart 13


Chart 14 – Go Around Decision All responding carriers had a policy requiring a go around in response to an unstabilized approach but the Checklist did not explore policy with regard to recovering from an unstabilized approach within defined conditions. Approximately 10% of respondents gave a NO answer with regard to the existence of a non-punitive culture towards go arounds, and/or to teaching pilots to initiate a go around if any crewmember calls for it, and both factors are critical to the go around decision. If pilots believe they may be punished for a go around they may well pursue other potentially less desirable options until it is too late – a non-punitive response is essential to achieve genuine compliance with declared policy. Any qualified crewmember is capable of recognising that an approach is unstabilized and it may be that the captain or the pilot flying is not in the best situation to decide whether or not to go around – there isn’t time to debate the matter and a go around should be automatic.

Chart 14


Chart 15 – Abnormal Touchdowns This is the second chart to show a significant number of NO answers, over 20%, this time in response to the two questions regarding policy and pilot training to react to impending or actual abnormal landings. This may be indicative of a belief that it is too late to correct in the last seconds of the landing phase and that it is therefore impossible to legislate or train for, but such a perception would be incorrect. Most modern aircraft are capable of executing a successful go around even after touchdown, as long as reverse thrust has not been selected, and this may be the safer option by a substantial margin when compared to a long, off-centreline or bounced landing. A potentially unintuitive pilot reaction of this nature will only be assured through regular training, reinforced by robust and clear policy.

Chart 15


Chart 16 – Contaminated Runways I The chart shows that most respondents regard contaminated runways as a significant threat and have implemented appropriate training, limitations and restrictions upon operations in such conditions. The small number of NO answers to the first 3 of the 5 questions may be attributable to misunderstanding – if not then carriers would be wise to revisit their training and policy to ensure that the deficiencies are addressed. Of rather more significance is the almost 10% not currently limiting operations on contaminated runways with aircraft having inoperative retardation systems, and 5% not currently training pilots to use maximum retardation immediately after touchdown on a contaminated runway. Both are vital risk management strategies for contaminated runway conditions.

Chart 16


Chart 17 – Contaminated Runways II, Asymmetrical Thrust Question 71 with regard to the use of reverse thrust in stopping performance calculations is perhaps the misfit in the Checklist, because there really is no ‘right answer’ in terms of risk management, and as a result (unsurprisingly) answers were split almost 50/50 between YES and NO. The possibility that a reverser might fail is a consideration but the same could be said of all retardation devices, and therefore the contingency procedure addressed in Question 72 would apply equally. Since main wheel braking is the retardation device most vulnerable to the effects of contamination, the importance of functioning reversers becomes proportionally greater, as does the lift dumping effect of spoilers to enhance surface water penetration for braking. Accidents records indicate that failure to deploy retardation devices in a timely manner in critical situations has contributed to the seriousness of the outcome many times and it is therefore encouraging that procedures to monitor deployment are almost universal.

Chart 17


Charts 18 to 22 – Flight Data Analysis Question 73, with regard to the existence of a flight data analysis program is fundamental to the remainder of the checklist and to operational risk management as a whole. Some 7% of respondents answered NO to this question and it would have been very interesting to ask a supplementary question “Why not?” because the availability of simple and cost effective data download systems and outsourced analysis programs suggests that the benefits will outweigh the cost. Having established that the majority do use FDA as a risk management tool it is perhaps surprising that 30% do not appear to use improvement goals as safety performance indicators - Question 75. Carriers may want to use the list of other questions as an indication of some, though not all, of the potentially useful parameters to be monitored and clearly most respondents currently monitor the majority of these. One parameter critical to runway excursion risk is touchdown point and 18% responded that this was not monitored. There are some technical difficulties associated with recording touchdown point but it is a fundamental indicator of operational behaviour and risk awareness. The final chart, Chart 22, suggests that several other important factors relating to landings and the associated excursion risks are not as well understood and perhaps consequently less well monitored than most other parameters. What this entire section of the Checklist does not address is the potential value of ‘normal operations monitoring’ in proactive and perhaps even predictive risk management, as opposed to exceedence recording, which only activates after a pre-determined boundary has been crossed. Normal operations monitoring involves the recording of specific events or parameters from every flight – altitude at which gear is selected down or time to select reverse after touchdown for example – to generate a substantial bank of data from which to gauge and manage operational behaviour, and to determine fleet-wide levels of compliance.


Chart 18


Chart 19


Chart 20


Chart 21


Chart 22


www.iata.org/safety

Documents

IATA Air Carrier Self Audit Checklist – Analysis Operations...Carrier Self Audit Checklist – ... IATA Air Carrier Self Audit Checklist ... The time available for pilots to decide