Five Steps Page 1 of 6

Why Do Good People Do Bad Things, And What Can We Do About It?

Five Steps to Attaining Excellence in Operations Performance

Operations and process safety managers frequently ask why experienced, knowledgeable people still make errors. “I know we’ve got good operators, yet we continue to have incidents.” “He knew better, but just went ahead and did it.” Why do good people do bad things? Managers usually have one of two different basic answers. One I call the God theory, “To err is human (i.e., God did it). It’s just human nature to make mistakes. No matter what you do, humans will make errors. So, it really is fruitless to spend too much effort on trying to ‘design out’ human error. People just have to learn by experience.” Actually, there’s a lot to be said about the benefits of learning by experience, IF you can afford the mistakes that occur in the learning process. In many high-risk operations, this is not the case. Some mistakes can be catastrophic; others just extremely expensive!

The second answer is “The devil made them do it.” Murphy’s law exists. If there is some way for a person to screw up, he will. The unfortunate corollary of this devil theory is that somehow, error is “sin,” and punishment is the appropriate corrective action. “They just need an attitude adjustment!”

My response to the question is that in most cases the devil did make them do it; but, the devil is us – the designers, and builders, and managers of the systems. We set the operators up for failure when we design and build and operate systems without proper consideration of human performance capabilities and limitations. Technical systems for the most part are built and operated by “technical” people. Engineering and technical education typically does not prepare us to design and operate people systems.

What can we do about it? The solution is to develop a comprehensive, systematic and continuous approach to improving design and management of the human side of the system. You need an approach to engineer human performance, just as you engineer all of the hardware elements of the system. This human performance engineering approach will: define precisely what performance and what level of performance is required; assure that the “design requirements” for people on the job (the knowledge, skills, abilities, and attitudes that they bring to the job) are sufficient; assure that all of the support and maintenance systems (procedures, operator aids, administrative controls, supervision, etc.) are available and are used. You can dramatically decrease the frequency of human errors, and the consequence of human error, thereby reducing equipment damage, unplanned outages, lost product, and risk to personnel, the public and the environment. Below are five basic steps to establishing a human performance engineering program that will lead your facility out of the “valley of compliance” to the “peaks of excellence”.

Step 1: Specify your performance requirements. The idea that we should know what performance we need before we try improve is so basic that it frequently is overlooked. Every job has minimum requirements for human performance – knowledge and skill level, physical requirements, personality traits, attitudes toward safety, etc. And it is possible to specify those requirements – to clearly identify and document them. In a systems-engineering design approach, human performance requirements are specified as part of the total system performance requirements. System functions and specific performance requirements are allocated to humans, to hardware, to software, to facilities, or to some combination of these major system elements. Formal job and task analysis (JTA) is used to identify specific human performance requirements. It also identifies the implications of those requirements on knowledge, skill and ability levels, on human-machine interface design, staffing levels, etc.

Five Steps Page 2 of 6

While caution should be exercised to not overkill on the level of formality and detail, applying the basic concepts and techniques of JTA to each job and task can produce a wealth of basic data for improving human performance. JTA is a means to identify who has to do what, where, when? What information is required? What equipment items are manipulated? What tools are required? What are the unusual or particular demands on the human for accuracy, time allowed, communication, or other performance attributes? What are the major constraints (e.g., high stress, confined space, etc.)? Personnel selection, training and qualification programs, design of operations and maintenance procedures, design of control and display interfaces, performance evaluations and other areas of human performance can draw on information collected and assessed in a job/task analysis. In order to get good performance you have to define what good performance is – what is required to do the job well.

Step 2: Develop good measures and advertise them. The JTA results provide a firm basis for specifying the underlying knowledge, skills, abilities, attitudes, etc. that are necessary for human performance that will meet system design requirements. These individual attributes can be measured in controlled settings. That’s what training tests or examinations do. But direct observation and measurement of those necessary components usually is not practical outside of the classroom, and more importantly, doesn’t get to the ultimate goal of tapping on-the-job performance.

We all know that performance on the job at any given time is affected by many factors other than basic knowledge and skills. What are needed are measures that discriminate levels of performance on the job. And, to the extent possible, an understanding of relationships between job performance and the underlying factors that influence performance. Then we can measure actual job performance and, when performance is less than required/ desired, determine the underlying (systemic) cause of the performance problem and fix it.

Furthermore, by advertising, i.e., by clearly specifying and communicating those performance measures, you can guide people to attain the desired performance. People tend to produce the performance for which they will be rewarded. Knowing what performance is desired clearly gives everybody a head start. It tends to eliminate “mushroom-farm management” (keep ‘em in the dark and feed ‘em lots of manure). Performance measures become a tool for attaining the performance you desire, not just a scorecard on how well you’ve been doing.

The complexity of the task of building good measures is often underestimated. The number of factors potentially influencing human performance on the job can be overwhelming. Those factors can vary greatly from person to person, within a single person from task to task, and even within the same person performing the same task on different occasions. To some degree, each performance of a task by each person is unique. Further, it is difficult to quantify the impact of many of the variables on overall “goodness” of performance. As a colleague once noted, “There is no calculus of human behavior”.

However, there are “experts” who can rate performance on the job, and usually they are readily available in your facility or organization. They are the highly experienced job incumbents. They are senior technical specialists, trainers who have been operators in the past and have had lots of experience evaluating trainees, current operators who are recognized (usually unofficially) as the best operators. These experts, or “authorities,” know good performance when they see it. And, they can discriminate levels of performance precisely and reliably. A problem is that it is very difficult for them to explain to anyone else precisely how they do what they do – what they see and how they put information together to make judgments.1

That information has to be carefully and painstakingly extracted from the experts and turned into straightforward “observable” indicators that can be used to both measure performance and to make explicit to all involved what is being measured and why. There are systematic techniques for extracting such rating strategies, and once captured they are powerful tools, not only for assessing performance, but for setting standards of performance, and for capturing the deeper

Five Steps Page 3 of 6

“expert knowledge” that comes only from years of experience on the job. A side benefit of developing such subjectively based measures is that the job incumbents (through the selected representative “experts”) at each level become intimately involved in setting their own standards of performance, in identifying collectively what is good performance, and in focusing on how good performance demonstrates itself.

Note that performance measures are not just for workers “on the floor,” but for all levels. Ideally, performance measures are developed for each job level with input from all “customers,” including internal customers at least one level above and one level below the job position. A comprehensive, integrated set of performance measures (or “family of measures”2) is the goal.

The process for developing measures should identify a comprehensive listing of what to measure. A good process will also identify how to measure. That is, a characteristic of a good indicator to be used in a measure is that data is reasonably and practically available. This does NOT mean (as so often is the case) that you simply identify the data that’s easy to collect and use that to get “the best measure you can”. This “looking under the lamppost” approach has led to many failed attempts at performance measurement. It does mean, however, that the most elegant psychological concept for human performance measurement is essentially useless if the data are too difficult or too expensive to collect. Your measurement development process should identify what data sources are to be used for indicators, how often they will be collected, in what form and by whom.

In general, a comprehensive set of measures will include three levels of indicators: organizational, functional, and individual3 - and at least three basic types of indicators: outcome, process, and behavioral. Organizational measures have to do with how well the organization identifies, communicates, and achieves its performance goals as an entity, whether it is the entire corporate organization or a sub-unit. They tend to be more global “outcome measures,” e.g., profit, availability, lost workdays. Functional measures focus on the performance of complete processes or business functions, which often cut across organizational lines – production, training, quality control. They tend to use process indicators and focus on how well the process is designed, installed, operated, maintained and evaluated. Most traditional audits involve process measures. Individual measures deal with the performance of individuals or teams of people as they perform the job. These typically are “behavioral” measures. Indicators involve observed behaviors or directly observable results of behaviors and are concerned with how well the human is performing the required task in comparison to specified requirements.

Again, there are behavioral measures for all levels of employees, from entry level to top management. Each individual or job-position measure is related to the organizational level measures for the organization of which that person is a part, and to functional level measures associated with the processes and functions in which that individual has some responsibility. There can be outcome, process or behavioral measures at any of the three levels, but typically organizational measures tend to use outcomes as indicators, functional measures tend use process indicators, and individual measures tend to use observational indicators. The challenge in building an effective family of measures is not just to identify all the indicators, but to identify and make explicit how they all relate to each other and to overall “system” performance.

Note that this is distinctly different from benchmarking. Why should your performance goals be determined by someone else’s standards of “excellence”? You know good performance when you see it. Your performance goals and your system performance requirements should drive the human performance requirements for your facility. Certainly, benchmarking is valuable. Understanding what the competition is doing well or not doing well is important. It is part of the information you can use to set your goals. But, the focus of your operations and your measurement process should be the processes, behaviors and outcomes you need to attain the optimum level of system performance. And, the target for human performance is the human performance that optimizes system performance. Recognize that the term system here is used

Five Steps Page 4 of 6

generally and is applied top-down starting from the highest level. Each sub-level has performance requirements derived from above.

Step 3: Measure current performance. You can’t figure out how to get where you want to go until you find out where you are. Given a clear idea of required performance and effective measures of performance, it is necessary to determine the current level of performance. A key concept to establish early is the culture of continuous improvement. There is no permanent fix. Performance measurement, evaluation, and improvement must be a continuous process. The initial measurement establishes where you are with respect to currently defined performance requirements and goals. As the system changes, as competition changes, as the economic conditions change, the desired/necessary performance levels will change. The only answer to “How good is good enough?” is “As good as you need to be to reach your goals at this time.”

If you have developed an effective measurement system as described in Step 2, your initial measure will be addressing not only outcomes, e.g., frequency of human errors, but processes, such as the training system design. A common mistake is to treat all human performance deficiencies as a “training” problem. In fact, human resource development specialists now recognize4 that only a small fraction of identified human performance problems are completely resolved by improved training. At a minimum, the measurement and evaluation of human performance should address the following areas:

- Personnel Selection, Training, and Qualification- Administrative Controls (Conduct of Operations)- Procedures – operating and maintenance procedures used by workers on the job- Ergonomics – the design of the human-machine interface- Organizational Culture – how the organization establishes and nurtures values such as

safety, learning, and excellence.These are the fundamental areas to consider in engineering human performance. Since very few current systems have been designed in accordance with a total systems engineering process, initial examination of these areas is likely to indicate there are some fundamental process improvements that need to be made to the human side of the system; that it is necessary to do some human performance engineering, or “re-engineering.”

Step 4: Re-engineer the people systems. The measurement and evaluation process must be capable of identifying systemic problems in these areas. Systemic problems are those of the underlying structure of the people system for which long-term basic solutions are possible. Problem identification focused on blame will not lead to long-term improvements. “The operator should have known better; he wasn’t paying attention; he should be disciplined.” Punishment rarely is effective for long-term behavior modification. Further, such solutions usually do not address the underlying cause of the error. This particular operator may not make the same mistake next time, but if there is a basic problem with the system, the error will occur again. What is most often found is that there are fundamental deficiencies in the design and management of the human system, largely because designers and managers were never trained in “human-system technology.”

The engineering (or re-engineering) of the people system needs to be accomplished from a total systems perspective. And, it needs to address the five areas listed above – personnel subsystems, conduct of operations, procedures, ergonomics, and organizational culture. A framework for engineering these people systems already exists. It is the formal systems engineering process that evolved in the military and aerospace industries for design and development of complex systems. Two key concepts are: 1) top-down definition and allocation of functions and performance requirements; and, 2) clear specification and management of interfaces among different system elements (e.g., human-machine interfaces). These systems

Five Steps Page 5 of 6

engineering design concepts can be applied to existing operational systems. In our company we refer to this as “systems performance engineering”. We aren’t engineering the system, but we are engineering performance of the existing system.

Human performance engineering (HPE) is the application of systems performance engineering to human (sub)systems. For example, in the field of training, the application of systems performance engineering to training systems is the “Instructional Systems Design” or ISD process. It is a top-down process of analysis, design, development, implementation and evaluation to produce performance-based training (and by extension, personnel selection and qualification). A central concept of human performance engineering is “user-centered design,” i.e., that all human interfaces – equipment, tools, procedures, facilities – should be designed to meet the needs, capabilities and limitations of the user. User-centered design helps to produce computer interfaces that are easier to understand and procedures that are easy to follow. In all areas, the human performance engineer seeks to identify and eliminate problems with the underlying design and management processes that are causing less than optimum human performance.

Step 5: Continuously measure and improve system performance. The final step in establishing a process for attaining operations excellence is to assure that there are mechanisms in place for continuous measurement, evaluation and improvement of human performance, and that they are fully supported by management. Figure 1 is the typical continuous quality improvement model applied to systems performance engineering. The focus here is on total system performance; and, as we’ve discussed, human performance is simply a part of overall system performance.

A critical element for improving human performance is a system for collection, analysis and feedback of operational experience – of successes and, especially, failures. And, a critical requirement for success of such a feedback system is an enlightened management attitude intent on learning from experience. A culture that views human error as simply a deficiency in the person involved, and the cure for human error a matter of identifying the guilty party and “correcting” that person, will never succeed in establishing a learning organization that benefits from mistakes and eliminates underlying causes of human error. A culture that eliminates “blame” and looks for systemic causes for error, that rewards self-critical evaluation, and that proactively seeks out and eliminates “error-likely situations” will drastically reduce the frequency of human errors.

A central element to a successful operational-experience-feedback program is a comprehensive root cause analysis program that includes root causes for human error and relates those causes back to basic human-system elements that management has the power to fix. The performance measures established in Step 2 will include direct measures of on-the-job performance that provide data for evaluation that also will be a powerful source of feedback. Other examples of data sources are “behavior-based” safety programs that involve systematic observation of performance on the job, industry information on events/occurrences applicable to your facility, and results of internal and external audits. An important issue in many facilities is how to retain the vast store of knowledge and experience that often is lost due to retirement or downsizing efforts. Some companies are setting up systematic programs for capturing this expert knowledge and incorporating it in training, procedures, required reading and other means of feedback to operations personnel. All of these sources of operational experience feedback can be valuable sources of data for improving performance, if they are systematically collected, analyzed and communicated to the right people.

Why do good people do bad things? Because they are people, of course. The design specs for people include intelligence, creativity, flexible thinking, adaptability, a modest level of sensory capabilities and physical capabilities, and many other attributes. Humans are extraordinarily, even uniquely, qualified for certain tasks. But they also have many limitations. High intelligence

Five Steps Page 6 of 6

and creativity can mean low tolerance for mundane repetitive tasks. While good memory is a distinct survival advantage, so is a good “forgettery”. Imagine life without being able to forget the infinite input of data we absorb in ordinary living each day. People have tasks and environments that they are inherently better suited for. It is the job of system designers to design the system, and the job of managers to manage the system, in a way that accounts for the inherent strengths and weaknesses of the human element.

Performance-based selection, training and qualification of personnel will help to assure a sufficient and consistent level of knowledge, skills, abilities and attitudes. User-centered design of procedures will help to assure that knowledgeable and skilled operators don’t “slip” and omit a key step. Well-designed displays and controls, labeling, lighting, etc. can help reduce both “cognitive” and “manual” errors. Rigorous administrative controls on simple things like exchanging information at shift turnover, tagout/lockouts, making changes to procedures, and numerous other routine activities can increase consistency and reduce possibilities for mental slips that all humans make. Management actions, such as rewarding those who self-report human errors that caused, or could have caused, an injury or process incident, will make clear that management really does mean those lofty statements about safety and the value of employees. And they will help eventually to inculcate the “safety culture” that we all claim we desire.

This is the job of the human performance engineer. It is a tough job. “Hardware managers,” that think the word “system” is synonymous with “hardware” do not make it easier. Increased demands on management to “do more with less” make it difficult for even the most enlightened plant manager to obtain the funding and manpower to rebuild poorly designed and implemented human systems. The good news is that there are well- established techniques for improving human performance. More managers are becoming aware, if only by the continued unacceptable level of occurrence of incidents, that all of the improvements to equipment that have been made will not assure safe and effective performance until the human element is addressed. The really good news is that these approaches will work and will improve human performance and, therefore overall system performance. And, while they do require initial investment, the cost is usually far less than hardware managers are used to spending on hardware fixes while the performance improvement can be much greater. You can attain operations excellence in your facility!

FOOTNOTES

1. Connelly, E.M., P.M. Haas, and K.C. Myers, “Method for Building Performance Measures for Process Safety Management,” International Process Safety Management Conference, Center for Chemical Process Safety, AIChE, San Francisco, CA, 1993.

2. Thor, Carl G., The Measures of Success, Creating A High Performance Organization, Oliver Wright Publications, Inc., Essex Junction, VT, 1994.

3. Geary A. Rummler and Alan P. Brache, Improving Performance, How to Manage the White Space on the Organization Chart, 2nd Addition, Jossey-Bass Publishers, San Francisco, CA, 1995.

4. Shirley, T., Performance In Practice, American Society for Training & Development Newsletter, Fall, 1997.

Five Steps Page 6 of 6

and creativity can mean low tolerance for mundane repetitive tasks. While good memory is a distinct survival advantage, so is a good “forgettery”. Imagine life without being able to forget the infinite input of data we absorb in ordinary living each day. People have tasks and environments that they are inherently better suited for. It is the job of system designers to design the system, and the job of managers to manage the system, in a way that accounts for the inherent strengths and weaknesses of the human element.

Performance-based selection, training and qualification of personnel will help to assure a sufficient and consistent level of knowledge, skills, abilities and attitudes. User-centered design of procedures will help to assure that knowledgeable and skilled operators don’t “slip” and omit a key step. Well-designed displays and controls, labeling, lighting, etc. can help reduce both “cognitive” and “manual” errors. Rigorous administrative controls on simple things like exchanging information at shift turnover, tagout/lockouts, making changes to procedures, and numerous other routine activities can increase consistency and reduce possibilities for mental slips that all humans make. Management actions, such as rewarding those who self-report human errors that caused, or could have caused, an injury or process incident, will make clear that management really does mean those lofty statements about safety and the value of employees. And they will help eventually to inculcate the “safety culture” that we all claim we desire.

This is the job of the human performance engineer. It is a tough job. “Hardware managers,” that think the word “system” is synonymous with “hardware” do not make it easier. Increased demands on management to “do more with less” make it difficult for even the most enlightened plant manager to obtain the funding and manpower to rebuild poorly designed and implemented human systems. The good news is that there are well- established techniques for improving human performance. More managers are becoming aware, if only by the continued unacceptable level of occurrence of incidents, that all of the improvements to equipment that have been made will not assure safe and effective performance until the human element is addressed. The really good news is that these approaches will work and will improve human performance and, therefore overall system performance. And, while they do require initial investment, the cost is usually far less than hardware managers are used to spending on hardware fixes while the performance improvement can be much greater. You can attain operations excellence in your facility!

FOOTNOTES

1. Connelly, E.M., P.M. Haas, and K.C. Myers, “Method for Building Performance Measures for Process Safety Management,” International Process Safety Management Conference, Center for Chemical Process Safety, AIChE, San Francisco, CA, 1993.

2. Thor, Carl G., The Measures of Success, Creating A High Performance Organization, Oliver Wright Publications, Inc., Essex Junction, VT, 1994.

3. Geary A. Rummler and Alan P. Brache, Improving Performance, How to Manage the White Space on the Organization Chart, 2nd Addition, Jossey-Bass Publishers, San Francisco, CA, 1995.

4. Shirley, T., Performance In Practice, American Society for Training & Development Newsletter, Fall, 1997.