Readiness is a Predictive Measure

Slide 1

Readiness is a predictive measure

James RollinsTakouba

Anyone remember the Nisqually Earthquake? Do you remember where you were? What you were doing? I was in a conference room on the second floor of a factory building in Tukwila. I remember the room sort of shaking. It wasn’t that dramatic as to be shocking, but it stopped the conversation. We looked at each other wondering, “What the heck is that.” Then slowly it sunk in that we were experiencing an earthquake. The thought was, “What do we do?” “Oh yeah! Duck and cover.” So we scrambled under the table and a few of us stood in the door frame watching the building shake. It was over in about a minute. Just enough time to realize what was going on, and then it was over. From there we headed downstairs to check and see if anyone was hurt. The power went out, so the factory was shut down. Everyone was standing around looking at each other. Then people began to evacuate, not because it was planned, but because that is what we do in case of fire – it’s the same right? We had no idea when the power would come on, a few of us tried to make phone calls, but the cell phone lines were down. The bottom line was – we weren’t ready. We were lucky, but we weren’t ready. So how should we get ready? And if we think we are ready, how do we really know? How do we measure it?

Slide 2

What is readiness?

A predictive measure of . . .

The state of being ready or prepared, as for use or action . . .

For the purpose of reducing risk . . .

by mitigating consequences.

So let’s start by discussing what readiness is Readiness is the state of being ready or prepared, as for use or action – so readiness is being CAPABLE of action. For the purpose of reducing risk . . . By mitigating consequences. Since readiness is about a capability for a future event, it must mean that the risk event has occurred – so we are left with mitigating the consequences.

Slide 3

Ris

k

Resi

dual R

isk

Read

iness

How does readiness relate to risk?

= Risk+ Probability of Occurrence

+ Severity of Hazard

+ Vulnerabilities

= Readiness+ Reduce vulnerabilities

+ Manage consequences

RiskThe danger that

loss or injury will

occur or that a

process could

fail

To build a solid definition of READINESS, it is important to understand the relationship between READINESS and RISK. Risk is the danger that loss or injury will occur or that a process could fail. From this definition, we learn that not only are sources of risk found outside of our organizations – but also inside our organizations if our processes and procedures are not robust. Risk is normally expressed as a function of its probability of occurrence, plus the SEVERITY of the consequences, plus your VULNERABILITIES. This is key, because VULNERABILITIES represent the gaps in your armor. Ethically speaking, you should be at least aware of them, and objectively looking to remove them. There are two ways to deal with VULNERABILITIES. One way is to remove them through some sort of improvement. The other way is to mitigate the consequences – which brings us to . . . READINESS. READINESS is your organization’s capability to reduce the consequences of the risk event, so in a way reduces your vulnerabilities. This is KEY – sometimes it is not feasible to fix vulnerabilities before they occur as they may be too expensive when compared to the value of the asset or the probability of occurrence. A strategy, then, to deal with these vulnerabilities is to reduce injury by reducing the consequences of the risk event, thereby leaving a much smaller residual risk.

Slide 4

What factors influence readiness?

Time

People

Money

Intellect

Maint.

Org

Leaders

Readiness

Stuff

Shelflife

KM

Standards

Move

So what factors influence READINESS? Factors are entities that improve, take-away or otherwise affect READINESS. There is TIME, MONEY (usually not enough), ORGANIZATION and STUFF. I say STUFF because it is a shorter word than “equipment” or “technology.” ORGANIZATIONS are made up of PEOPLE, with priorities set by LEADERS and hopefully, the organization has some from of KNOWLEDGE MANAGEMENT so it isn’t constantly “Reinventing the wheel.” So lets start connecting these entities together and describing their relationship to each other. Let’s start with TIME. TIME is an enemy of READINESS. What I mean is, the more time that goes by, the more READINESS deteriorates. People forget, organizations change, policies change. The inevitable march of time erodes READINESS. MONEY is definitely a supporter of readiness, but it usually has to be managed by an ORGANIZATION or used to buy STUFF. PEOPLE have knowledge and experience – they have intellect, or what some companies call “intellectual capital.” PEOPLE are mobile, they have a tendency to move around. They get promoted, or move to other jobs to expand their experience, which is good for the organization.

But sometimes they get fired or just move off to find other opportunities, usually taking some part of the intellectual capital in the organization. This affects READINESS negatively. All-in-all though, PEOPLE are definitely a positive influence on ORGANIZATION. And the quality of the ORGANIZATION positively influences READINESS. Finally there is STUFF, which is influenced by NEED FOR MAINTENANCE and SHELF-LIFE. Both of these factors influence STUFF negatively, because like TIME, when things sit, they get out-of-date, they expire or they fall into disrepair. PEOPLE also have a shelf-life too. There is forgetting. Skills must be regularly reinforced, otherwise PEOPLE forget and make mistakes. Overall, fresh, well maintained STUFF positively influences READINESS. That is quite a plate of spaghetti, huh?! With meatballs! The bottom line, is that assessing READINESS is complicated because it is influenced by so many different factors. The purpose of this brief seminar is to discuss how to structure your approach to assessing READINESS.

Slide 5

What are the steps to measure readiness?

Audit organization to test for compliance2

Conduct a test4

Evaluate organization KPI3

Evaluate your risk1

What are the steps to measure READINESS? In my estimation there are 4 steps that should be followed: First you need to evaluate your RISK. This is normally done by reviewing local All-HAZARD assessments that will give you an indication of what you are facing, how probable and how severe the threat is. However, there is more . . . Understanding what can hit you is one thing, but understanding its effect on your organization is another. In my experience, I review risk worksheets of various kinds, but it rarely describes the vulnerabilities of the business. Nor is there any cross-walk between these vulnerabilities and the countermeasures the business has in place to mitigate the effect of this vulnerability. Second you need to AUDIT your organization. Auditing is a way to see if your organization is complying with its internal policies and procedures. Auditing can and should be based on some STANDARD – and we’ll talk more about that later, but it is not absolutely necessary. The key point is that an organization’s COMPLIANCE with its own policies and procedures is a basic organizational COMPETENCY. Third is how well is your organization doing against its KEY PERFORMANCE INDICATORS? Many organizations don’t even have KPI – but should. If your organization provides any service or product, it likely has some numbers it uses to measure its performance. The key point is that an INTERNAL MEASUREMENT is a basic organizational COMPETENCY.

Lastly, a great way to see if you are ready is to conduct a TEST. By simulating the scenario you are most likely to face, you can devise a method to test your organization to see how well it will respond to a risk event. So let’s start on the top of our list . . .

Slide 6

Risk

Review past AARs2

Focus effort on critical capabilities4

Prioritize critical capabilities3

Analyze hazards1

How do assess our risk? First we analyze our hazards. As I mentioned earlier, there are various documented sources of risk information. Everything from county risk assessments, to specific geological and climactic studies made by various agencies. But what ever you do, don’t forget to just walk around and look. Sometimes the risks we face are staring right in the face. Identify the your vulnerabilities based on these hazards and use them to determine what CAPABILITIES you will need to develop. Review past After Action Reviews. You are likely not the only one who has spent some time assessing your organization’s risk. AAR’s are a rich source of vulnerability information, and some even recommend solutions (that nobody instituted). I can tell you that I have personally reviewed pages and pages of AARs with great information that no one has acted upon. PRIORITIZE. Once you understand your vulnerabilities, you will be able to decide what capabilities you need to develop in order to respond effectively to the risk event. You likely will not have enough resources to address all your needed responsibilities, so you will need to PRIORITIZE. And of course, once you have a priority it will allow you to FOCUS on what is important.

Slide 7

Audit - What does right look like?

Standards1 Agreements2

Contracts EMAC

Before we audit, we first need to define “What right looks like.” World-class organizations adhere to industry standards such as ISO or ANSI. As a result of the quality revolution in the 80’s, TOTAL QUALITY MANAGEMENT practices were codified into standards. These standards make good business sense, because everyone evaluates their basic business competencies the same way. The definitions are all the same, and it is easier to compare apples-to-apples. Because business now have standards, their overall quality improves and typically results in more efficient, cost-effective operations. AGREEMENTS are another source of “what right looks like.” For example, the Emergency Management Assistance Compact (EMAC) was hammered out between states that covers thorny issues like how to request resources, how they will be paid for, how licenses and other authorities translate across state lines and such all which establish a standard by which all can do business.

Slide 8

Why standardize?

Common language1

Lower costs2

Improves quality3

Slide 9

Audit – What do we look at?

Policies1 WHY we do it

Procedures2 HOW we do it

Resources3WHAT we have

WHO we have

STANDARDS tell us what policies we should have in our operation in order to fulfill that function. So for example, if I am looking at an Emergency Operations Center, I would likely have a policy that describes the need for the organization to have and maintain an overarching Emergency Management Plan. The policy would likely describe who is responsible for making the plan and how often the EMP would be reviewed. A procedure associated with the EMP, might describe how a advisory committee is formed, how the chair is selected and what processes they will use to advise on the EMP. Finally, we would review what resources in terms of organization and stuff (you remember stuff from earlier?), the organization has its disposal and determine its adequacy with respect to need.

Slide 10

Audit – What we are looking for

Does organization have a culture of fixing problems?2

Does the organization self-audit?4

Do leaders conduct management reviews?3

Do people follow policies and procedures?1

So you are looking at your organization – what do you look for? Are people aware of and do people actually follow policies and procedures? Do they know where procedural documents are kept? Do they refer to them? Does the organization have a method for controlling the document versions? Does the organization have a method they use to deliberately define, highlight and elevate problems so they can be fixed? Do leaders conduct management reviews? Do they pull information about institutional issues or problems up to the management level so they can allocate resources to fix the problems? Do leaders ask for internal audits to evaluate the effectiveness of their programs? Is audit a bad word?

Slide 11

KPI – Key performance indicators

Measurable outcome (number of minutes between call and response)2

Statistical process control (is process consistent and repeatable?)3

Specific to a function (call response time)1

Another way to tell if an organization is READY is by looking see if they have set up and are using KEY PERFORMANCE INDICATORS. KPI are measurements of a specific function, such as a call response time. If we are running a dispatch center, for example, we might be interested in knowing how long it takes for a resource to be dispatched to a location from the point of the call. KPIs measure a specific outcome in some repeatable way, with a standard quantity. So using the call response time example, we would track that time in minutes from the time the call was received by a dispatcher to the time a unit arrives on scene. This definition will provide us with a standard by which to measure this function. Typically KPIs can be analyzed using a statistical process control technique in order to determine if the process is consistent and repeatable. This can give an organization the ability to determine the causes of variation to an otherwise consistent process.

Slide 12

Test – how can we test?

Individual skills requirements2

Exercise!4

Collective skills requirements3

Job task analysis1

Finally – once we determine that an organization has all of the basics elements of competency in place, we can run a test. A test is a great way to provide a simulated future event, that provides context to observe the performance of the organization. Testing is a form of auditing, but differs in that it provides a future context. To build out a test, we have to develop a list of observable behaviors that an individual would perform. This first step in this process is conducting a job task analysis. The job task analysis starts by reviewing a process or procedure and determining and listing out the observable behaviors that would be associated with that role in the future context. So for example, if we are going to evaluate a logistics role an Emergency Operations Center (EOC), we would likely look through the job action sheet and associated procedures to see what actions that person would take in the performance of their job. The individual actions that an individual is responsible are called individual skills. These are the skills an individual must successfully perform in order to be individually competent for the job. Rarely does an individual operate completely independent of the organization. For most tasks, we work together. So our individual skills come together in team environment to accomplish and organizational task – such as “Write a Incident Action Plan.” We call these “collective skills” and use these to form an exercise evaluation plan. Using the evaluation plan, we can specifically evaluate each function, and each function in relation to each other.

Finally, we exercise. The exercise is a learning event were we test our processes and procedures and attempt to build the confidence of the exercise participants. It is critically important to maintain a positive outlook in an exercise, because during an exercise, we are there to fail. Today’s failures are tomorrow’s strengths.

Slide 13

Training Design Features

COMPETENCY TASK SUBTASK TRAINING DESIGN FEATURES

1.0 Estimate the effect of the hazard on the hospital

1.1 Activate a Hospital Incident Command Post

Provide a scenario briefing PowerPoint Organize HICS staff representatives:

Command

PIO

Operations Section

Planning Section

Medical Care Branch

Infrastructure Branch

Logistics Branch

EMS

1.2 Conduct a facilities vulnerability assessment

1.2.1 Estimate the effect of hazard on critical infrastructure 1.2.2 Estimate effect of consequences on hospital facilities, staff and resources 1.2.3 Develop and consider measures to mitigate hazard consequences

Staff provide an estimate process based on the following documents:

Hospital vulnerability assessment.

Weather report or other consequences information (county risk register).

Facilities manager judgment. Use Hospital Facilities Hazard Impact Assessment form to estimate post-mitigation status of systems.

1.3 Determine the degree to which the hospital can operate in post hazard environment*

Staff provides a briefing for the IC.

1.4 Determine how the hospital can safely evacuate patients before disaster strikes. 1.4a Determine how the hospital can safely evacuate patients

1.4.1 Estimate the time required to evacuate the hospital

Staff provides a rough-cut estimate using a form or calculation to estimate time to evacuate. Formula based on complexity of facility, patient mix, transport type and distance to gaining hospitals.

Slide 14

How do we develop our team?

Individual training - training that an individual requires to fulfill their

primary job. Normally provided on-the-job or through employer provided training.

So how do we develop our team? Individual training key because in ensures that an individual can fulfill the tasks in their primary job. Normally provided on-the-job or through employer provided training.

Slide 15

Collective training -training that an organization performs together

in context to learn and test processes such as planning and reporting.

And we provide CONTEXT for collective training to learn and test processes – like planning and reporting.

Slide 16

Leaders –experienced leaders whoare conditioned to the nuances of the context

and confident in the people and capabilities of theorganization

The final dimension of team-building are leaders. Exercises provide leaders (provided they participate) with an exception way to build confidence in their team, to provide a venue to test out processes and procedures and to ultimately validate that the organization is ready.

Slide 17

Time

Long hard climb

Fast decline

Re

ad

ine

ss

Typically, organizations don’t arrive at a high degree of readiness unless the spend time ramping up. Indeed, it can be a long hard climb to a high state of readiness. And the unfortunate thing is that the decline from readiness is even faster! As I explained in the beginning, time is the main detractor from readiness, because people move about, they forget, technology changes and readiness just starts to fall. So what are our challenges to maintaining READINESS?

Slide 18

Ways to make readiness last

Turn over2

Knowledge management4

Scale and capacity3

Frequency1 Documentation5

Follow up6

Train frequently enough to reinforce organizational knowledge. If an organization doesn’t regularly exercise, then readiness can fall so low that it requires substantial investment in time and resources to get it to an acceptable level again. Manage turn-over. Turn over is inevitable – but do you have a strategy to transfer knowledge? Do you train regularly enough that the new person can have an opportunity to exercise in their job? Find novel ways to exercise that don’t put a drain on the organization. One reason why full-scale and functional exercises are not done regularly, is because they take a lot of time, money and coordination. Computer aided simulations could provide a training venue that can be easily repeated, not cost as much as a full-scale exercise, and actually do a better job of simulating capacities and identifying bottlenecks. Another idea is to incorporate exercises into the regular work day – that is what happens when the urgency is over and the recovery begins. Manage your institutional knowledge. Have a way to capture processes and procedures, control versions and keep these up to date. This provides you with good idea of “what right looks like.” Without knowledge management, you will likely find yourself reinventing the wheel instead of incrementally developing your organization’s ability. Document your lessons learned. The outcome of an exercise should be a corrective action plan and updated procedures.

And for the good of the order FOLLOW UP!! Assign people complete recommendations highlighted in the corrective action plan. Give them a deadline and empower them to complete the task.

Slide 19

Time

Re

ad

ine

ss

Ris

k

Standards1 Agreements2

Resi

dual R

isk

So let’s summarize. READINESS IS A PREDICTIVE MEASURE of how well your organization would perform in context to a future event, as measured against a set of standards and agreements, reducing exposure to the consequences of a given risk event, leaving a smaller residual risk.

Slide 20

So – are you ready?


Anyone remember the Nisqually Earthquake? Do you remember where you were? Whatyou were doing?

I was in a conference room on the second floor of a factory building in Tukwila. I rememberthe room sort of shaking. It wasn’t that dramatic as to be shocking, but it stopped theconversation. We looked at each other wondering, “What the heck is that.” Then slowly itsunk in that we were experiencing an earthquake.

The thought was, “What do we do?” “Oh yeah! Duck and cover.” So we scrambled underthe table and a few of us stood in the door frame watching the building shake. It was overin about a minute. Just enough time to realize what was going on, and then it was over.

From there we headed downstairs to check and see if anyone was hurt. The power wentout, so the factory was shut down. Everyone was standing around looking at each other.Then people began to evacuate, not because it was planned, but because that is what wedo in case of fire – it’s the same right?

We had no idea when the power would come on, a few of us tried to make phone calls, butthe cell phone lines were down. The bottom line was – we weren’t ready. We were lucky,but we weren’t ready.

1

So how should we get ready? And if we think we are ready, how do we really know? How dowe measure it?

1

So let’s start by discussing what readiness is:

1. Readiness is the state of being ready or prepared, as for use or action – so readiness isbeing CAPABLE of action.

2. For the purpose of reducing risk . . .

3. By mitigating consequences. Since readiness is about a capability for a future event, itmust mean that the risk event has occurred – so we are left with mitigating theconsequences.

2

To build a solid definition of READINESS, it is important to understand the relationshipbetween READINESS and RISK. Risk is the danger that loss or injury will occur or that aprocess could fail. From this definition, we learn that not only are sources of risk foundoutside of our organizations – but also inside our organizations if our processes andprocedures are not robust.

Risk is normally expressed as a function of its probability of occurrence, plus the SEVERITYof the consequences, plus your VULNERABILITIES. This is key, because VULNERABILITIESrepresent the gaps in your armor. Ethically speaking, you should be at least aware of them,and objectively looking to remove them. There are two ways to deal withVULNERABILITIES. One way is to remove them through some sort of improvement. Theother way is to mitigate the consequences – which brings us to . . .

READINESS is your organization’s capability to reduce the consequences of the risk event,so in a way reduces your vulnerabilities.

This is KEY – sometimes it is not feasible to fix vulnerabilities before they occur, as they maybe too expensive when compared to the value of the asset or the probability of occurrence.A strategy, then, to deal with these vulnerabilities is to reduce injury by reducing the

3

consequences of the risk event, thereby leaving a much smaller residual risk.

3

So what factors influence READINESS? Factors are entities that improve, take-away orotherwise affect READINESS.

First - there is TIME, MONEY (usually not enough), ORGANIZATION and STUFF. I say STUFFbecause it is a shorter word than “equipment” or “technology.”

ORGANIZATIONS are made up of PEOPLE, with priorities set by LEADERS and hopefully, theorganization has some from of KNOWLEDGE MANAGEMENT so it isn’t constantly“Reinventing the wheel.”

So lets start connecting these entities together and describing their relationship to eachother. Let’s start with TIME. TIME is an enemy of READINESS. What I mean is, the moretime that goes by, the more READINESS deteriorates. People forget, organizations change,policies change. The inevitable march of time erodes READINESS.

MONEY is definitely a supporter of readiness, but it usually has to be managed by anORGANIZATION or used to buy STUFF.

PEOPLE have knowledge and experience – they have intellect, or what some companies call“intellectual capital.” PEOPLE are mobile, they have a tendency to move around. They get

4

promoted, or move to other jobs to expand their experience, which is good for theorganization. But sometimes they get fired or just move off to find other opportunities,usually taking some part of the intellectual capital in the organization. This affectsREADINESS negatively. All-in-all though, PEOPLE are definitely a positive influence onORGANIZATION. And the quality of the ORGANIZATION positively influences READINESS.

Finally there is STUFF, which is influenced by NEED FOR MAINTENANCE and SHELF-LIFE. Bothof these factors influence STUFF negatively, because like TIME, when things sit, they get out-of-date, they expire or they fall into disrepair.

PEOPLE also have a shelf-life too. There is forgetting. Skills must be regularly reinforced,otherwise PEOPLE forget and make mistakes. Overall, fresh, well maintained STUFFpositively influences READINESS.

That is quite a plate of spaghetti, huh?! With meatballs! The bottom line, is that assessingREADINESS is complicated because it is influenced by so many different factors. The purposeof this brief seminar is to discuss how to structure your approach to assessing READINESS.

4

What are the steps to measure READINESS? In my estimation there are 4 steps that shouldbe followed:

First you need to evaluate your RISK. This is normally done by reviewing local All-HAZARDassessments that will give you an indication of what you are facing, how probable and howsevere the threat is. However, there is more . . . Understanding what can hit you is onething, but understanding its effect on your organization is another. In my experience, Ireview risk worksheets of various kinds, but it rarely describes the vulnerabilities of thebusiness. Nor is there any cross-walk between these vulnerabilities and thecountermeasures the business has in place to mitigate the effect of this vulnerability.

Second you need to AUDIT your organization. Auditing is a way to see if your organizationis complying with its internal policies and procedures. Auditing can and should be based onsome STANDARD – and we’ll talk more about that later, but it is not absolutely necessary.The key point is that an organization’s COMPLIANCE with its own policies and procedures isa basic organizational COMPETENCY.

Third is how well is your organization doing against its KEY PERFORMANCE INDICATORS?Many organizations don’t even have KPI – but should. If your organization provides anyservice or product, it likely has some numbers it uses to measure its performance. The key

5

point is that an INTERNAL MEASUREMENT is a basic organizational COMPETENCY.

Lastly, a great way to see if you are ready is to conduct a TEST. By simulating the scenarioyou are most likely to face, you can devise a method to test your organization to see howwell it will respond to a risk event. So let’s start on the top of our list . . .

5

How do assess our risk?

Start by analyzing your hazards. As I mentioned earlier, there are various documentedsources of risk information. Everything from county risk assessments, to specific geologicaland climactic studies made by various agencies. But what ever you do, don’t forget to justwalk around and look. Sometimes the risks we face are staring right in the face. Identifythe your vulnerabilities based on these hazards and use them to determine whatCAPABILITIES you will need to develop.

Review past After Action Reviews. You are likely not the only one who has spent some timeassessing your organization’s risk. AAR’s are a rich source of vulnerability information, andsome even recommend solutions (that nobody instituted). I can tell you that I havepersonally reviewed pages and pages of AARs with great information that no one has actedupon.

PRIORITIZE. Once you understand your vulnerabilities, you will be able to decide whatcapabilities you need to develop in order to respond effectively to the risk event. You likelywill not have enough resources to address all your needed responsibilities, so you will needto PRIORITIZE.

6

And of course, once you have a priority it will allow you to FOCUS on what is important.

6

Before we audit, we first need to define “What right looks like.”

World-class organizations adhere to industry standards such as ISO or ANSI. As a result ofthe quality revolution in the 80’s, TOTAL QUALITY MANAGEMENT practices were codifiedinto standards. These standards make good business sense, because everyone evaluatestheir basic business competencies the same way. The definitions are all the same, and it iseasier to compare apples-to-apples. Because business now have standards, their overallquality improves and typically results in more efficient, cost-effective operations.

AGREEMENTS are another source of “what right looks like.” For example, the EmergencyManagement Assistance Compact (EMAC) was hammered out between states that coversthorny issues like how to request resources, how they will be paid for, how licenses andother authorities translate across state lines and such all which establish a standard bywhich all can do business.

7

8

STANDARDS tell us what policies we should have in our operation in order to fulfill thatfunction. So for example, if I am looking at an Emergency Operations Center, I would likelyhave a policy that describes the need for the organization to have and maintain anoverarching Emergency Management Plan. The policy would likely describe who isresponsible for making the plan and how often the EMP would be reviewed. A procedureassociated with the EMP, might describe how a advisory committee is formed, how thechair is selected and what processes they will use to advise on the EMP. Finally, we wouldreview what resources in terms of organization and stuff (you remember stuff fromearlier?), the organization has its disposal and determine its adequacy with respect toneed.

9

So you are looking at your organization – what do you look for?

Are people aware of and do people actually follow policies and procedures? Do they knowwhere procedural documents are kept? Do they refer to them? Does the organizationhave a method for controlling the document versions?

Does the organization have a method they use to deliberately define, highlight and elevateproblems so they can be fixed?

Do leaders conduct management reviews? Do they pull information about institutionalissues or problems up to the management level so they can allocate resources to fix theproblems?

Do leaders ask for internal audits to evaluate the effectiveness of their programs? Is audit abad word?

10

Another way to tell if an organization is READY is by looking see if they have set up and areusing KEY PERFORMANCE INDICATORS.

KPI are measurements of a specific function, such as a call response time. If we are runninga dispatch center, for example, we might be interested in knowing how long it takes for aresource to be dispatched to a location from the point of the call.

KPIs measure a specific outcome in some repeatable way, with a standard quantity. Sousing the call response time example, we would track that time in minutes from the timethe call was received by a dispatcher to the time a unit arrives on scene. This definition willprovide us with a standard by which to measure this function.

Typically KPIs can be analyzed using a statistical process control technique in order todetermine if the process is consistent and repeatable. This can give an organization theability to determine the causes of variation to an otherwise consistent process.

11

Finally – once we determine that an organization has all of the basics elements ofcompetency in place, we can run a test. A test is a great way to provide a simulated futureevent, that provides context to observe the performance of the organization. Testing is aform of auditing, but differs in that it provides a future context.

To build out a test, we have to develop a list of observable behaviors that an individualwould perform. This first step in this process is conducting a job task analysis. The job taskanalysis starts by reviewing a process or procedure and determining and listing out theobservable behaviors that would be associated with that role in the future context. So forexample, if we are going to evaluate a logistics role an Emergency Operations Center (EOC),we would likely look through the job action sheet and associated procedures to see whatactions that person would take in the performance of their job.

The individual actions that an individual is responsible are called individual skills. These arethe skills an individual must successfully perform in order to be individually competent forthe job.

Rarely does an individual operate completely independent of the organization. For mosttasks, we work together. So our individual skills come together in team environment toaccomplish and organizational task – such as “Write a Incident Action Plan.” We call these

12

“collective skills” and use these to form an exercise evaluation plan. Using the evaluationplan, we can specifically evaluate each function, and each function in relation to each other.

Finally, we exercise. The exercise is a learning event were we test our processes andprocedures and attempt to build the confidence of the exercise participants. It is criticallyimportant to maintain a positive outlook in an exercise, because during an exercise, we arethere to fail. Today’s failures are tomorrow’s strengths.

12

When creating a test – we use a competency to training design feature matrix. Byidentifying the actions we want to observe in behavioral terms, we are able to create anevaluation scheme. In this example, we are observing the competency of “Estimating theeffect of the hazard on the hospital.” This is a requirement for hospitals that have beenaffected by a natural hazard. Tasks that must be performed to enable this competency areidentified in the Task column. For example, “Activate an incident Command Post” is a taskthat is necessary to enable the first competency. “Conduct a facility vulnerabilityassessment” is another supporting task. Tasks can be broken down further into supportingelements such as “Estimate effect of hazard on critical infrastructure” and “Estimate effectof consequences on hospital facilities, staff and resources.” From here the matrix identifiesnecessary training design features that would support the evaluation of those tasks.Documents, processes, scenario injects and related items are all examples of designfeatures that would make the evaluation possible.

13

So how do we develop our team? Individual training key because in ensures that anindividual can fulfill the tasks in their primary job. Normally provided on-the-job or throughemployer provided training.

14

And we provide CONTEXT for collective training to learn and test processes – like planningand reporting.

15

The final dimension of team-building are leaders. Exercises provide leaders (provided theyparticipate) with an exception way to build confidence in their team, to provide a venue totest out processes and procedures and to ultimately validate that the organization is ready.

16

Typically, organizations don’t arrive at a high degree of readiness unless the spend timeramping up.

Indeed, it can be a long hard climb to a high state of readiness. And the unfortunate thingis that the decline from readiness is even faster! As I explained in the beginning, time is themain detractor from readiness, because people move about, they forget, technologychanges and readiness just starts to fall. So what are our challenges to maintainingREADINESS?

17

1. Train frequently enough to reinforce organizational knowledge. If an organizationdoesn’t regularly exercise, then readiness can fall so low that it requires substantialinvestment in time and resources to get it to an acceptable level again.

2. Manage turn-over. Turn over is inevitable – but do you have a strategy to transferknowledge? Do you train regularly enough that the new person can have an opportunity toexercise in their job?

3. Find novel ways to exercise that don’t put a drain on the organization. One reason whyfull-scale and functional exercises are not done regularly, is because they take a lot of time,money and coordination. Computer aided simulations could provide a training venue thatcan be easily repeated, not cost as much as a full-scale exercise, and actually do a betterjob of simulating capacities and identifying bottlenecks. Another idea is to incorporateexercises into the regular work day – that is what happens when the urgency is over andthe recovery begins.

4. Manage your institutional knowledge. Have a way to capture processes and procedures,control versions and keep these up to date. This provides you with good idea of “what rightlooks like.” Without knowledge management, you will likely find yourself reinventing thewheel instead of incrementally developing your organization’s ability.

18

5. Document your lessons learned. The outcome of an exercise should be a corrective actionplan and updated procedures.

6. And for the good of the order FOLLOW UP!! Assign people complete recommendationshighlighted in the corrective action plan. Give them a deadline and empower them tocomplete the task.

18

So let’s summarize.

READINESS IS A PREDICTIVE MEASURE of how well your organization would perform incontext to a future event, as measured against a set of standards and agreements, reducingexposure to the consequences of a given risk event, leaving a smaller residual risk.

19


20

21