4 Ways to Measure Effective Performance of RCA

8/22/2019 4 Ways to Measure Effective Performance of RCA

1/5 Lie Cycle Engineering 2013 www.LCE.co

Root Cause Analysis is all about improving your bottom line in saety, environmental compliance and protability. This paperexplores measuring the eectiveness o Root Cause Analysis as a business process intended to produce business results.The author provides examples o the primary KPIs that will help you keep your nger rmly on the pulse o your RCA process

Four Ways to Measure the Effectiveness

of Your Root Cause Analysis ProcessBy Sam McNair, PE, CMRP, Life Cycle Engineering


2/5

To learn more about Lie Cycle Engineering, contact: 843.744.7110|[email protected]|www.LCE.com Lie Cycle Engineering 20

Overview

Root Cause Analysis is all about improving your bottom line in saety, environmental compliance and protability. This paperexplores measuring the eectiveness o Root Cause Analysis as a business process intended to produce business results.The author provides examples o the primary KPIs that will help you keep your nger rmly on the pulse o your RCA process.

People oten dene the sole measure o success or Root Cause Analysis (RCA) as being able to kill the problem orever. Butdoes that mean dead at any cost? How big are the problems we are solving? Are we trapping mice (orever chasing irritating littleproblems) or hunting big game (only responding to major events)? Forever is a long time. Is one re-occurrence per 100 years goodenough? And is the true objective the analysis, or the actions that ollow?

It is important to distinguish between perorming an individual RCA (using a tool) and the business process o systematicallyperorming RCAs that supports any eective loss elimination, ailure elimination, or continuous improvement program. Excellent

training is available or any o the more than 100 documented RCA tools in use. This discussion is limited to measuring theeectiveness o RCA as a business process intended to produce business results. We do RCA because o the desire or goodbusiness results which include protecting people, the environment and our prots.

Please keep in mind that an RCA can be perormed both reactively and proactively. We are all amiliar with the reactive mode operorming a Root Cause o FAILURE Analysis (RCFA). But do you know that RCA adds most value when it is used proactively?One such proactive example is when developing proactive solutions to address potential ailure modes identied by risk analysistools such as Opportunity Analysis (OA) or Failure Mode and Eects Analysis (FMEA). Even when perormed in reaction to a ailuri your RCA process is eective, the ndings o an RCA can be applied proactively to similar assets, products, and situations, thuspreventing ailures rom occurring elsewhere.

I you are near the top o your class in business, most o your RCA should be ocused on eliminating chronic, money-stealing, mino

events (the nancial equivalent o saety near misses) as well as addressing adverse trends in any perormance, instead o beingreactive and doing RCA only when large or disastrous events occur. Very oten, having to perorm an RCA on a major disaster ismerely painul validation that the system was not truly under control to start with. RCA on disastrous events should be very rare.

In the business process o RCA, program eectiveness is measured using several metrics, also known as Key PerormanceIndicators (KPIs). The most useul metrics are leading indicators o correct behaviors. Lagging indicators or metrics are part o theeedback loop or the overall process and behaviors system. They are also known as the bottom line. What ollows are exampleso the primary KPIs that, i used properly, will keep your nger rmly on the pulse o your RCA process by answering these vitalquestions:

1. Is every event that meets or exceeds the published RCFA trigger criteria receiving an RCFA,

and is the RCA being initiated within the specifed time rame?

First you must understand what an RCA trigger is and why its important. It oten takes a lot o ailure analysis and other expensiveorensics to ully understand the nature o the ailure. Add to that the cost o the RCA team human resources and the cost toimplement corrective action, and an RCA can be very expensive! Trigger criteria are used to ensure consistent application o RCFAso that we ocus on solving the most important problems rst, and to ensure that the resources expended are appropriate to thepotential return. This maximizes the return on the resources expended. Without triggers, initiating an RCFA becomes anerratic and oten emotionally driven event, jumping rom partially solved problem to partially solved problem, oten withinadequate results and poor return on investment.

Four Ways to Measure the Effectivenessof Your Root Cause Analysis ProcessBy Sam McNair, PE, CMRP, Life Cycle Engineering

Page 2 o 5


3/5


Trigger criteria are a scale o business consequences including the previously mentioned important business outcomes such

as saety, environmental impact, and prots, and takes into consideration the requency with which these consequences can beexpected to occur. In short, they are a means o classiying ongoing business risk. Well-constructed trigger criteria accommodatenot only singular big events, such as a major accident, but also chronic events such as repeated rst aid injuries. And they will takeinto account not only those events that have happened, but those that can realistically be expected to happen i no action is takensuch as the output o an FMEA, or multiple, similar saety near-misses. The triggers, usually in tabular orm, relate the overall riskreduction potential (benets) to the amount o resources that the business is willing to spend to reduce the risk (RCA cost). It is asimple tool or classiying the business case and determining i sucient Return on Investment (ROI) exists to justiy perorming anRCA, and what level o RCA resources are justiable to invest.

Some words o caution: Many organizations have developed elaborate multi-tiered trigger systems that select the type o RCAanalysis tool to use based on the loss incurred. By speciying the type o RCA tool to be used, what they are really implying is theallowable resource cost o the entire analysis. Just as when selecting the tools to x a machine, dierent RCA tools should beselected or dierent types o problems. This slightly o-target type-based trigger approach is like saying a big hammer is more

eective than a little hammer, when in act a wrench is needed or the job. To get the best results, the tool should be chosen orits eectiveness. The allowable resources to expend to perorm the RCA should be determined by the trigger. I the only eectivetool to solve the problem costs more to implement properly than the benets o the resulting solution can support, then pauseto consider whether you should be working on this problem at all at this time. Using the wrong RCA tool well or the right toolimproperly, or simply wasting major resources on small problems, all lead to sub optimal results.

Concerning the issue o RCA timing, the time required post incident to complete an RCA is oten specied as another,inappropriate metric. I do not agree with the common usage rule that every RCA must be completed within some specied timerame, such as one or two weeks, ater the incident. The RCFA process should always be started within the specied timerame(generally hours or days) ater an incident occurs, beore memories ail and evidence escapes. It should always be completed withthe agreed-upon time. Sometimes actors such as external ailure analysis services, or tests, will drive the completion timing; thusa single xed value or all completion is not always appropriate. An RCFA is worthless i it addresses the wrong ailure. The keybehavior here is starting the RCA promptly and consistently.

The critical KPI to measure here is start interval compliance:

This is the number o RCAs started within the required specied interval ater the trigger is pulled, as a percentage o the totalnumber o events that pull the RCA trigger.

2. Are 100% o all approved corrective actions specifed in an RCFA being completed on or beore

the published date?

Notice that completing the RCA analysis within the time rame is not the most critical measure. That is because the only thing reallimportant to the business is the implemented results o the RCA. An elegant analysis and perectly worded corrective actions,

fawlessly reported is not a business result, just a step in the process. The only things that will produce results are completedcorrective actions. The RCA closes only ater all approved corrective actions are closed.

There are several good reasons or measuring corrective action compliance:

a. The RCFA is worthless i the corrective actions are not promptly completed (not to mention the hideous liability trailthat an RCA with uncompleted corrective action leaves behind.)


Page 3 o 5


4/5


b. When everyone understands that the corrective actions will be completed on time or serious consequences will ollow, the

tend to ocus on the critical ew corrective actions instead o loading the report with rivolous items or solving everything buthe real problem. This is humorously known as shot-gunning or trying to solve world hunger when the problem wasrunning out o bread at lunch time.

c. I an organization is unable to complete the necessary corrective actions, the entire RCA eort is wasted. And they obvioushave not allocated the resources and priorities to complete what they have started. This rapidly becomes an untenablesituation with a large and rising backlog o uncompleted corrective actions. Morale will suer. I this happens, then there isa hard choice that must be made: you must either raise the trigger level (reduce the infow), or add more resources andpriority (add more pumps.) It is imperative to decide and act or your RCA program will collapse in a food o incomplete inishes and reoccurring problems.

This critical KPI to measure, On-time Corrective Completion, one o the most critical to have, and most difcult one

to manage, is: The number o approved RCA corrective actions completed on or beore their specied dates, divided by the

number o corrective actions having due dates within the measurement time rame. This should always be near 100%.

3. Have all the criteria that were specifed to measure success o the corrective actions been

measurably met?

Every RCFA must have specied measurable means o assuring that the corrective actions are eective, otherwise there is nomeans o verication. Closing an unveried RCFA is engaging in wishul speculation. What behaviors, actions, inactions, orconditions need to be changed to prevent recurrence? How do we measure the success o the RCA? We all know the cautionabout what happens when people make assumptions. Assuming that the RCA correction actions were successul is a common ansometimes literally atal mistake. The ground is littered with ailed RCA programs and damaged or wrecked careers rom assuming

that the problem was xed and stayed xed.

Metrics such as the mean time between ailure (MTBF), easily available rom a good Failure Reporting and Corrective Action Syste(FRACAS), work well as a source o conrmation. Measured losses should take into account spare part costs, lost productivitycosts, scrap and material losses, lost opportunity cost, and manpower costs. For singular events such as minor accidents, near-misses and other known precursors o more serious events, the rates can be measured. Or you may have to convert the results intcommonly agreed-upon meaningul statistics such as OSHA injury rate. By ar the simplest and most reliably eective approach isto measure the actual costs where they can be obtained.

Also, when measuring, always be on the lookout or unintended negative consequences o your x. Did we x one problem tocreate a dierent one or move the same problem to some other location? It happens ar more oten that people care to admit.

It is critical that you measure success or as long as it takes to ensure that the x works and keeps on working. Do not just decla

victory and go home when the corrective actions have been implemented and the rst good data comes in. You must track or asucient period to ensure sustainability o the results. This is especially true when the corrective actions are changes in humanbehavior, such as SOP changes or training. People are not easily changed, nor do they stay changed without serious eort. A bespractice is to speciy and approve the audit period simultaneously with the corrective actions.

The critical KPI to measure, Success Rate, is:For each RCA, the actual percentage o each stated improvement goal that was reached, measured over the speciedaudit period.


Page 4 o 5


5/5


4. What is the net return on investment o the RCFA / Loss Elimination Program? How much did

you save (the avoided cost o the ongoing incidents) versus the total cost to perorm the RCFAplus the cost o the corrective actions?

We measure RCFA eects in money because it is a airly universal language, and business is in business to make money. In therare case o a non-prot organization, there is still a need to use resources wisely or measurably protect the public welare. Evenin the area o saety, where there is no price you can place on human lie or suering, there are generally accepted and agreed-upon costs to be avoided or lost human lie and suering. Most Loss Management departments or insurers can give you somevery straightorward dollar numbers to use to put a cost on avoided death, injury, and other disastrous events. The U.S. Departmeno Labor also has some useul data. It is not being callous; using cost as a mean o measuring injury risk is an integral part o riskmanagement in business and government. It is also an eective means to ensure that you are maximizing the use o your resourcesto minimize the risk to your business, your employees, and the public at large.

Its unortunate that almost any business activity except or accounting and actual production o product or service has tocontinuously re-justiy itsel, as i anything but these ew exceptions were some obvious and inherent waste o company unds.Sometimes this problem may be simply due to management ignorance o what really contributes to the value stream. As an RCApractitioner, this problem becomes your challenge. Remember, management controls the money and other resources necessaryto do RCA. The RCA program you put in place must eventually show a documented positive cash fow and good ROI, or it willeventually ace extinction. Money just happens to be a very universal way to prioritize the use o your resources. Do not start on anRCA i the ultimate impact o the trigger event or trend is not known.

The fnal (lagging) KPI, but the one most critical to sustaining your RCA program, is simple Return on Investment

(ROI): It is the cost o the ongoing incidents eliminated / (Cost o the RCFA + Cost o corrective actions.)

Ultimately RCA is all about improving your bottom line in saety, environmental compliance, and protability.


Page 5 o 5

About the AuthorSam McNair is a senior consultant with Lie Cycle Engineering (LCE). A Proessional Engineer andCertied Maintenance and Reliability Proessional, Sam has more than 34 years o experience in discretemanuacturing, chemical process industries, mining, machine processes, automation, aviation, constructionand utilities. Sam specializes in reliability engineering with a ocus on the integration o maintenance andmanuacturing unctions. You can reach Sam at [email protected].
mailto:smcnair%40lce.com?subject=LCE%20Resource%20Inquirymailto:smcnair%40lce.com?subject=LCE%20Resource%20Inquiry

Documents

4 Ways to Measure Effective Performance of RCA