Will The Result Of The Asiana 214 Investigation Be Just More Of The Same?

As the hearings into last summer’s crash of Asiana 214 at San Francisco International Airport get underway, I can only hope that they will result in a real and meaningful look at the relationship between today’s pilots and their modern aircraft.  I expect though, the result will once again be platitudes like “over reliance on automation” and the obligatory mandate for stricter policy and more procedure. Standard operating procedure (SOP) is essential in airline operations, however, it should be used as a tool and SOP compliance should never be viewed as THE objective in establishing a safe operation.  The system will try to legislate and regulate effective decision-making through narrowly targeted training scenarios, more SOP and increased spending on airport navigation aids.  It will succeed in giving the appearance of making the airline industry safer while actually failing to address the root cause of this type of accident, ineffective threat identification and mitigation. 

In 1996 the FAA Human Factors Team issued a report on the interface between flight crews and modern flight deck automation. It was a result of a 1994 A300 crash in Nagoya as well as 1995 accidents in Cali and at Hartford, CT.   This FAA report is an exhibit in the NTSB Docket for Asiana 214.  The HF Team’s report clearly articulates their recognition that it was essential to look beyond flight crew errors for a deeper level of understanding of the issues.  After reading this report it would appear that almost all of the concerns raised in 1996 were still contributing factors Asiana 214 and UPS 1354 hull loss accidents.  Why, 20 years later, are we still unable to manage human error?

A friend of mine said this reminded him of the scene in the movie, A Few Good Men, where well meaning policy and procedure (Lt. Daniel Kaffee) collide with operational realities (Col. Nathan R. Jessup) in their iconic courtroom scene.

 Jessep (Jack Nicholson): You want answers?    
  Kaffee (Tom Cruise): I think I’m entitled to them.    
  Jessep: You want answers?    
  Kaffee: I want the truth!  
  Jessep: You can’t handle the truth!

What is the “truth” here?  Let me see if I can explain.

There are two fundamental components to aviation, the airplane and the crew.  The advancements in aviation safety have come from improvements to these two components.  The airplane was first.  With the advent of the digital age, avionics technology was able to assist the airplane with a truckload of very reliable hardware programmed with “if-then” software meant to help with crew awareness and workload.  As technology advanced so did the amount and sophistication of these hardware/software devices.  Later in the evolution came improvements to the crew, or human, component.  Crew improvements came from the development and introduction of CRM, Crew Resource Management.  CRM soon became the “poster child” for all progressive flight operations.  Unlike the technological advancements to the airplane, CRM was not universally adopted in a standardized manner and therefore lost much of its value and influence.  Each flight operation developed and trained its own CRM objectives and methods.  CRM became synonymous with teamwork and communication rather than improving the crew effectiveness component of aviation safety.  A member of the 1996 FAA Human Factors Team and the “Godfather” of error management, Dr. Robert L. Helmrich, from the University of Texas, along with James Klinect and John Wilhelm, wrote about this deterioration of CRM in their paper MODELS OF THREAT, ERROR AND CRM INFLIGHT OPERATIONS.

AQP, the Advanced Qualification Program, implemented to improve training for airline crews mandated two, among other, essential elements necessary to align training with line operations. CRM and Advanced Simulation were required.  They were included for very important reasons.  CRM for the very explanation Helmreich et al stated, “CRM is an active process by crewmembers to identify significant threats to an operation, communicate them to the PIC, and to develop, communicate, and carry out a plan to avoid or mitigate each threat.”  Advanced Simulation is essential because it is meant to create a flight deck environment identical to the real line operations.  The intent was to provide crews an opportunity to face realistic scenarios and effectively debrief their experience, not to only perform according to preplanned scripts.  Unfortunately the industry has not embraced the intent of line oriented simulation as described in a manual for debriefing simulator events by Key Dismukes, former Chief Scientist for Aerospace Human Factors in the Human Systems Integration Division at NASA Ames Research Center, with Lori K, McDonnell and Kimberly Jobe from San Jose State University. The first sentence of the introduction states, “How much crews learn in Line Oriented Simulations (LOS) and take back to the line hinges on the effectiveness of the LOS debriefing.”   Most simulator training is still about “what” and very little time is spent learning the “why”.

The ultimate result of this 20 year safety evolution of airplane and crew is the huge bureaucracy called SMS, Safety Management Systems.  SMS has tried to do to the airline industry what digital technology has done for the airplane, program the system with “if-then” software (procedures and policy).  The concept was good.  The goal was to look at risk from an integrated perspective.  Unfortunately, there are so many layers, and competing agendas, that like all big bureaucracies there are lots of people doing lots of things but accomplishing very little.  One of the things that SMS has been able to accomplish is to communicate to crews that “they, meaning the SMS group” are the genesis of threat mitigation strategies.  The problem is that SMS threat mitigation strategies are seldom, if ever, timely or appropriate.  That is because SMS can address overall risk of an organization, but it cannot manage individual threats very well.  Threats, which are generally defined as events or errors that occur beyond the influence of the crew, increase operational complexity, and must be managed to maintain the margins of safety, can only be identified and managed tactically by the crew.  The crew can and should rely heavily on procedural guidance for their threat mitigation strategies, but all the procedural guidance, all the SOP the airline can come up with is worthless if it is not effectively applied.  The bureaucracy has lost sight of the fact the machines can be programmed, humans cannot.  You can teach humans what to think about, but you cannot program humans to think only in “if-then” binary patterns.  Policy and procedure is only as good as the judgment and decision-making used to apply it.

The identification of threats, and their mitigation is the most fundamental of all aircrew responsibilities.  Conventional wisdom, including most flight standards and training managers, has replaced threat management with procedural compliance.  The system (airlines, regulators, etc.) arrogantly and narcissistically assumes it can write a policy or procedure for every condition and then naively thinks that all crew members can and will comply.  Airline managements are reticent to spend money on training that is not FAA mandated or have a corresponding dollar value.  Pilot bulletins and computer-based learning have replaced facilitated workshops where line pilots are allowed to discuss their experiences and relevant operational safety issues.  Chalkboards and instruction from experienced pilots has been replaced with PowerPoint presentations packed with dozens of slides that are just “cut and paste” text and pictures from the flight manual.   Pilot associations have their share of responsibility as well.  They are quick to point out any deficiencies in the system, but at the same time are only interested in looking at human factors as a way to avoid responsibility when mistakes are made.  They need to be willing to look introspectively at human factors in an effort to raise the level of pilot performance, i.e. threat identification and mitigation.  There have been some airline operators with robust TEM programs, however, through budget cuts and/or mergers these programs have been trivialized or abandoned altogether. 

If the airline industry is really serious about making flight operations safer it will increase its efforts to improve crew effectiveness not just pilot performance.  Training and evaluation must be outcome based not just procedure driven.  Crews should be trained and evaluated using the LOSA paradigm.  That paradigm focuses on the values of crew awareness and self-correction where they are actively engaged with the airplane and environment so they can identify and mitigate threats and errors in a timely manner.    These instructors, evaluators and crews encourage and accept peer feedback in an effort to achieve a safe outcome.  There are only three responses to a threat or error; it is not identified, it is identified and not mitigated or it is identified and mitigated.  Two of those responses require luck or things outside the crew to provide a safe outcome.  To ensure a safe operation, isn’t it worth at least as much time and effort to train pilots to identify and mitigate threats and errors as it is to teach procedure?

That outcome-focused approach is in contrast to the still employed, “I know more than you do.” ego centered dynamic utilized by many flight standards organizations. We must get away from the ancient methodology of training and evaluating performance based on procedural compliance and focus on safe outcome.  The equation, Procedure + compliance = safe operation only works in an academic environment.  Intentional and unintentional non-compliance are unavoidable realities.  You might reduce one but you never eliminate the other.

Every time there is an accident involving the crash of an aircraft with no mechanical deficiencies the focus of correction is to improve the system, or blame the crew.  Writing more policy and procedure does not ensure a more effective crew.  Flying pre-briefed scenarios in the simulator is valuable training, but still does not address the most pernicious component of any crew, ineffective threat identification and mitigation. Before the crash of Asiana 214, there have been other infamous crashes of perfectly good airplanes like the Boeing 757 at Cali, Columbia.  Why can’t we do better?  Imagine if these flights had used even the most basic TEM approach.  Approaching SFO the Asiana crew could have asked themselves, “With the ILS and the VASI out of service what are going to be our challenges flying this visual approach and how should we address them?”  Before starting their descent into Cali the crew of American 965 could have articulated something like, “It’s dark and there are a lot of high mountains on our route, we had better be sure of our flight path on the descent.”  Both crews were supplied with plenty of procedure to address their conditions, however in both cases, it was not simply an over reliance on automation, rather ineffective or absent threat identification and mitigation.  SOP is an invaluable tool in flight operations, but when crews are taught that SOP compliance is in itself, by definition, a safe operation then human error will determine the outcome.  In the absence of active TEM training and practice the only alternative crews have is to rely on things outside themselves like automation and procedure.  We know how that works out.