Embed Size (px)
Citation preview
Taken from FEMA Professional Development Series
Exercise Design
WelcomeAdministrative Items:
-Breaks
-Lunch
-Cell phones
-Fire / Severe Weather / Evac Procedures
Introductions
30 seconds or less
Name
Organization
Role
Exercise experience Types of exercises conducted, participated in, etc.
3
4
Exercise
Requires participants to function in the same capacity as they would in a real event
Promotes preparedness
Tests policies and plans
Trains personnel
A focused practice activityusing a simulated situation
5
Unit Objectives
Identify the five major accomplishments in designing and implementing an exercise.
Define the purpose of the four exercise documents.
6
Course Objectives
Describe and apply the eight exercise design steps.
Explain the purposes and characteristics of tabletop, functional, and full-scale exercises.
Compare and contrast the design considerations for tabletop, functional, and full-scale exercises.
7
Course Objectives
Identify the tasks involved in a systematic exercise evaluation process.
Describe the purpose, benefits, and potential sources of exercise enhancements.
Design and implement a small functional exercise using the eight-step design process.
8
Course Content
Unit 1: Course Introduction
Unit 2: The Comprehensive Exercise Program
Unit 3: The Exercise Process
Unit 4: Exercise Design Steps
Unit 5: The Tabletop Exercise
Unit 6: The Functional Exercise
Unit 7: The Full-Scale Exercise
9
Course Content
Unit 8: Exercise Evaluation
Unit 9: Exercise Enhancements
Unit 10: Designing and Implementing a Functional Exercise
Unit 11: Course Summary
10
Why Exercise?
UAL 232 crash: Problems revealed in full-scale exercise had been corrected
Loma Prieta earthquake: Effective response was preceded by earthquake exercise
11
Why Exercise?
Oklahoma City bombing:
“If an integrated emergency management system is to be utilized and effective in future disasters, all levels of government must be on the same page of the book.”
12
Why Exercise
2000 Denver building collapse scenario preceded 2001 World Trade Center collapse.
13
Benefits of Exercising
Provides individual training
Leads to system improvement
What are some specific reasons to conduct exercises?
14
Regulatory Requirements
FEMA and other agencies providing Federal funds
Nuclear Regulatory Commission
SARA Title III (chemical facilities)
Licensing of public facilities (e.g., airports, hospitals)
OSHA
FINRA, SEC
15
Functions
Alert Notification (Emergency Response)
Warning (All personnel)
Communications
Coordination and Control
Emergency Public Information
Damage Assessment
16
Functions
Public Works/Engineering
Transportation
Resource Management
Continuity of Government
Continuity of Operations (Organization)
17
Unit Summary
In Unit 1, we:
Previewed the course.
Considered benefits gained from exercising and reasons to exercise.
Completed a needs assessment.
Next: The Comprehensive Exercise Program
18
Unit 2: The Comprehensive Exercise Program
19
Unit Objectives
Identify the basic components of a comprehensive exercise program.
Explain the importance of designing a comprehensive exercise program to meet the needs of your organization or community.
20
Progressive Exercising
Broad commitment: Multiple groups involved in planning, preparation, and execution.
Careful planning: Each exercise carefully planned to achieve identified goals.
Increasing complexity: Increasingly complex exercises build on each other until mastery is achieved.
Success Breeds Success!
21
Who Participates?
Organization decides
Participants also determined by nature and size of exercise
Tabletop—Key decision makers
Functional—Particular functions
Full-scale—All functions, key personnel.
22
Types of Exercise Activities
Simple Narrow InexpensiveTheoretical
1. Orientation seminar
2. Drill
3. Tabletop exercise
4. Functional exercise
5. Full-scale
Complex Broad Costly Realistic
23
Orientation Seminar
An overview or introduction
Familiarizes participants with roles, plans, procedures, or equipment
Can also be used to resolve questions of coordination and assignment of responsibilities
24
Conducting an Orientation
Be creative:
Use varied methods.
Make the session interactive.
Plan ahead: Do not try to “wing it.”
Be ready to facilitate:
Help participants stay focused.
Keep things positive and moving along.
25
Drill
A coordinated, supervised exercise activity normally used to test a single specific operation or function
No coordination, no EOC
Purpose: Perfect one small part of response plan, help prepare for more extensive exercises
26
Conducting a Drill
Prepare: Review operational procedures and safety precautions beforehand.
Set the stage: Present purpose, objectives, scenario.
Monitor the action: Intervene if necessary to keep the drill on track.
27
Tabletop Exercise
Facilitated analysis of an emergency situation
Informal, stress-free environment
Designed to elicit constructive discussion
Participants resolve problems based on existing plans and identify needed changes
28
Functional Exercise
Fully simulated interactive exercise that tests the capability of an organization to respond to a simulated event
Tests multiple functions in a coordinated response
Time-pressured
Realistic simulation
29
Full-Scale Exercise
Simulates a real event as closely as possible
Evaluates operational capability of emergency management systems in a highly stressful environment
Requires mobilization of all resources
Should test and evaluatemost functions of the plan
30
Building an Exercise Program
Built by a team
Based on operating plan
Involves:
Analysis of capabilities and costs.
Scheduling.
Public relations.
Development of long-term plan.
Provides basis for individual exercise design
31
Planning Team
Organizational program:
All departments and major functions.
Vendors, external entities.
32
Planning Tasks
Team organization
Goal setting: Long-term goals, mission statement
Sequence and scheduling:
Plan series of exercises to meet goals of all participating entities.
Organize exercises into progressive sequence.
Develop time schedule.
33
Plan Elements
Timeframe
Problem statement
Long-range goal(s)
Functional objectives
Schedule
Exercise descriptions:
Type of exercise
Participants
Purpose
Rationale
34
Unit Summary
In Unit 2, we:
Discussed the characteristics of and types of activities in a comprehensive exercise program.
Reviewed the main tasks in building an exercise program.
Initiated a comprehensive exercise program plan.
Next: The Exercise Process
35
Unit 3: The Exercise Process
36
Unit Objectives
Identify the five major accomplishments in designing and implementing an exercise.
Describe the organization of an exercise design team.
Define the purpose of the four exercise documents.
37
Exercise Process: The Big Picture
Three ways to visualize:
Organized by task sequence
Organized by task categories and phase
Organized by major accomplishments
38
1. Establishing the base
5. Exercise followup
4. Exercise critique and evaluation
3. Exercise conduct
2. Exercise development
Major Task Accomplishments
39
Using the Exercise Process
Process applies to:
All types and levels of exercises.
Any size jurisdiction.
Any type of organization.
Flexibility is the key:
Each task must be designed, tailored, and applied in a way that suits your objectives and capabilities.
40
Task 1: Establishing the Base
1. Review the current plan.
2. Conduct a needs assessment.
3. Assess capability to conduct an exercise.
4. Define the scope.
5. Select the exercise type.
6. Address costs and liabilities.
7. Develop a statement of purpose.
8. Gain support and announce the exercise.
41
Task 2: Exercise Development
1. Assess needs.
2. Define scope.
3. Write a statement of purpose.
4. Define objectives.
5. Compose a narrative.
6. Write major and detailed events.
7. List expected actions.
8. Prepare messages.
42
Task 3: Exercise Conduct
To conduct a successful exercise:
Be clear.
Sustain action.
Foster realism.
Establish timelines.
Review emergency call-off procedures.
Capitalize on problemsituations.
43
Task 4: Evaluation and Critique
Evaluate how well the objectives have been achieved.
Extent and depth of evaluation is determined by participating organizations.
Critique and report: Analyze results and giverecommendations for addressing deficiencies.
44
Task 5: Exercise Followup
Getting full benefit from an exercise requires following up on evaluation recommendations.
Assign responsibility.
Monitor progress.
Complete the cycle: Build testing of improvements into the next exercise.
45
Assessing Capability
When was your last exercise?
What exercise experience is available?
How much preparation and development time can be devoted?
What skills can people provide?
What physical facilities are available?
What communication facilities and systems are available?
What attitudes can you expect from leaders?
46
Costs and Liabilities
Cost items:
Salaries
Contract services
Equipment and materials
Fuel
Miscellaneous items
Other considerations:
Reimbursement for overtime?
Who foots the bill?
Exercise responsibilities covered in job descriptions?
Routine exercise costs recognized by leadership?
Injury and damage covered by insurance?
47
Gaining Support
Gaining support of the highest manager / leader gets participant cooperation.
To gain official support:
Seek support for entire exercise program.
Protect the organization.
“Sell” the process.
Announce the exercise.
48
Selling the Process
Use the needs assessment, capability analysis, purpose statement, and objectives as sales tools.
Present the exercise concept, including:
Need for the exercise.
Organizational capability.
Type of exercise.
Scope and purpose ofthe exercise.
49
Exercise Design Team Leader
Is responsible for entire development process
Should be someone who:
Can devote significant time.
Is familiar with the emergency plan.
Knows the participating organizations.
Is NOT a key operational member.
?
50
Design Team Responsibilities
Determine exercise objectives.
Tailor the scenario.
Develop the sequence of events and associated messages.
Assist in development and distribution of pre-exercise materials.
Help conduct pre-exercise training sessions.
May act as simulators or controllers.
51
Design Team Candidates
Include representatives of:
Participating jurisdictions.
Participating agencies.
Key departments.
If that makes too large a team, select a small core team that draws on others as needed.
Select members with varied backgrounds.
Provide technical/administrative support.
? ? ? ? ?
52
Organizing the Team
Establish clear goals.
Agree on plan of action.
Reach consensus on realistic schedule.
Meet regularly.
Share expertise and resources.
No single, standard approach.
Use teamwork strategies.
Learn about each other’s area and function.
Keep lines of communication open.
Use job aids to keep everyone on the same page.
53
Exercise Documents
For . . .Exercise Plan EveryoneControl Plan Controllers
Simulators NOT players
Evaluation Plan Evaluators Controllers Simulators
Player Handbook Players
54
Unit 3 Summary
In Unit 3, we:
Examined the overall design process and five major task accomplishments in that process.
Learned key aspects of establishing the base.
Discussed key factors in assembling a design team.
Reviewed 4 major design documents.
Next: Exercise Design Steps
55
Unit 4: Exercise Design Steps
56
Unit Objective
List and explain the eight exercise design steps
57
Step 1: Assess Needs
Begin with your plan:
Hazards and priorities
Vulnerable areas
Functions in need of rehearsal
Potential participants and program areas.
Past exercises
Exercise requirements
Capabilities
58
Assessing Needs
Learn from past exercises:
Who participated?
To what extent were objectives achieved?
What lessons were learned?
What problems were revealed, and what is needed to resolve them?
What improvements have been made since, and have they been tested?
59
Needs Assessment Results
Primary and secondary hazards
Problems, weak functions
Skills requiring practice
Improvements requiring testing
Untested facilities, personnel, equipment
Weaknesses in emergency plan or SOPs
Need for role clarification
Need for certain types of exercises
60
Step 2: Define the Scope
Factors that help define scope: Expense
Availability of resources
Seriousness of the problem
Capacity of the exercise to address the problem
Designers’ skills and experience
Exercise length
Defining the scope = Setting realistic limits
61
Defining Scope
Scope includes the:
Type of emergency.
Location.
Functions.
Participants.
Exercise type.
62
Defining Scope
Location:
Choose a realistic site.
Consider traffic problems and safety issues.
Functions:
List operations to be practiced.
Be specific about the procedures addressed within each function.
63
Defining the Scope
Participants:
Which organizations need to be involved to carry out the functions being tested?
Which representatives from those organizations should be there?
Narrow the list to those who are required in order to carry out the actions.
64
Defining the Scope
Exercise type:
What exercises are most needed?
What experience have personnel had with various types of exercises?
What stress level do we want?
What types of exercises are mandated by regulatory requirements?
65
Step 3: Write a Purpose Statement
Purpose statement: A broad statement of the exercise goal
Governs objectives, which determine subsequent steps
Clarifies reasons for the exercise
Useful for communicating with media and community, and most importantly, the associates/employees participating!
66
Step 4: Define Objectives
Objectives: Descriptions of the performance you expect from participants to demonstrate competence
Objectives are essential for:
Design process.
Exercise conduct.
Evaluation.
Followup.
67
What Makes a “Good” Objective?
Clear, concise, focused on participant performance
Should contain:
Action, stated in observable terms.
Conditions under which the action will be performed.
Standards (or level) of performance.
68
A Good Objective
Should state who should do what under what conditions according to what standards.
Within 15 minutesafter the evacuation notice is given,members of the EOCwill complete notification procedures toschool administrators.
Who
Specific action
ConditionsStandards
69
SMART Objectives
S imple
M easurable
A chievable
R ealistic
T ask oriented
70
Is This a Useful Objective?
To demonstrate an understanding of the procedures necessary in
protecting responder health and safety.
71
Points of Review
Objective: Demonstrate the adequacy of displays and other materials to support emergency operations
Points of Review:
Display Yes No
1. Status boards available in facility
2. Status boards used
3. Status boards kept updated by ________
4. Maps available
5. Maps up to date
72
Word Choice
Use concrete words.
Avoid vague verbs, such as:
Know.
Understand.
Appreciate.
Show the ability to.
Be aware of.
Use action words.
73
Step 5: Compose a Narrative
Narrative: A brief description of the scenario events that have occurred up to the exercise beginning.
Sets the mood
Provides information that sets the stage for later action
74
Narrative Characteristics
A good narrative:
Is usually 1 to 5 paragraphs long.
Is very specific.
Is phrased in present tense.
Is written in short sentences.
May develop the situation chronologically (event with warning time).
May emphasize the emergency environment.
75
Step 6: Write Major & Detailed Events
Events are:
Occurrences that happen as a result of the emergency described in the narrative.
Problems requiring actions that will meet the objectives.
Careful scripting is required to:
Produce a convincing, unified scenario.
Create an exercise governed by objectives.
76
Developing Major Events
Major events:
Big problems.
Likely events calling for realistic action.
To develop major events:
1. Identify major occurrences that would follow the narrative events.
2. Select those that might generate situations to test the objectives.
77
Developing Detailed Events
Detailed events:
Specific problem situations that will prompt one or more expected actions.
To develop detailed events:
Plan detailed events and expected actions together.
Work backwards from the actions.
List specific problems likely to stem from major events, and actions that would be expected to address them.
78
Detailed Events Example
Major event: “Estimates of passenger casualties rise to 200 deaths and at least 70 severe burn victims.”
Possible detailed events:
Mortuary cannot accept so many remains.
Local hospitals lack specialized facilities and personnel to treat large numbers of severe burn victims.
The Red Cross has agreed to set up an information center to link victims and their families.
79
Step 7: List Expected Actions
Expected actions: Actions or decisions you want participants to carry out in order to demonstrate competence.
Types of actions:
Verification.
Consideration.
Deferral.
Decision.
80
Listing Expected Actions
List only those that involve the participating organizations.
List expected actions for all exercise participants.
It is not necessary for each detailed event to generate responses from all participants.
81
Step 8: Prepare Messages
Messages:
Communicate detailed events.
Evoke a response (decision or action) to meet objectives.
Are transmitted by phone, radio, note, fax, in person.
Must come from credible source, through credible channels.
82
Messages, Events, and Actions
One message may represent one event or . . .
Several messages may be needed to notify participants of an event.
Each message is designed to generate one or more expected actions.
83
Message Variables
Message source
Transmission method
Message content
Recipient
These variables influence the action taken
WHO?
HOW?
WHAT?
TO WHOM?
To what EFFECT?
84
Message Format
CONTENT: ________________________________________________
___________________________________________________________
___________________________________________________________
__________________________________________________________
ACTION TAKEN: ____________________________________________
__________________________________________________________
___________________________________________________________
___________________________________________________________
EMERGENCY EXERCISE <MESSAGE>
TO: FROM:METHOD
NO: TIME:
85
Spontaneous Messages
In tabletop and functional exercises, most messages are prescripted.
Participants do not always respond as anticipated.
Controllers and simulators must be ready to ad lib.
Designers should give them ideas for handling situations beforehand.
86
Composing a Message
Begin with an expected action.
Decide:
What message would provoke that action?
Who will send it? Who will receive it?
What should the message say?
How will the message be transmitted?
Keep it realistic!
Practice with a partner . . . does the message “work?”
87
Master Scenario of Events List
Includes:
List of events.
Time of occurrence.
Expected actions.
Organized by time of occurrence
Used to monitor progress, keep exercise on schedule
Should NOT be shown to participants
89
Unit Summary
In Unit 4, we:
Discussed the eight-step design process.
Applied each step in the initial development of an exercise.
Next: The Tabletop Exercise
90
Unit 5: The Tabletop Exercise
91
Unit Objectives
Describe the purposes and characteristics of a tabletop exercise.
Describe the steps in facilitating a tabletop exercise.
92
Tabletop Characteristics
Purpose: Solve problems as a group.
Format:
Discussion guided by facilitator.
No simulators.
No elaborate facilities or communications.
Evaluation:
Observers may be present.
Success measured by participant feedback, impact on policies, plans, procedures.
93
Problem Statements, Messages
Present verbal problem statements to:
Group.
Individuals, then open discussion to group.
Deliver prescripted messages to individuals, who coordinate with others and respond.
Take time to resolve tough problems.
94
Facilities and Materials
Operations center or conference facility
Materials:
Emergency plans
Continuity plans
Maps
Other references
95
Facilitating the Exercise
Strategies for setting the stage:
Welcome
Briefing
Narrative
Ice breaker
96
Facilitating the Exercise
Strategies for involving everyone:
Organize messages
Encourage
Elicit—do not provide—solutions
Eye contact
Positive
Reinforcement.
97
Designing a Tabletop Exercise
Use 8-step process, job aids
Steps 5-8 are simplified:
Only partially simulated
Little scripting
No simulators
Informal evaluation
98
Applying the Design Steps Narrative:
Shorter
Printed (or in person, TV, radio)
As a whole or in parts
Discussion items
Events:
Closely related to the objectives
Only a few are required
Used as basis for problem statements
99
Applying the Design Steps
Expected Actions:
List serves as a basis for developing problem statements and messages.
“Actions” may be discussion that leads to change.
Messages:
A few may suffice.
May relate to major or detailed events.
100
Next: The Functional Exercise
Unit Summary
In Unit 5, we:
Reviewed how a tabletop exercise works.
Discussed facilitation guidelines.
Examined how the eight design steps are applied to tabletop exercises.
Developed problem statements and messages.
101
Unit 6: The Functional Exercise
102
Unit Objectives
Describe the purpose and characteristics of a functional exercise.
Explain how designing a functional exercise differs from designing a tabletop exercise.
Describe the physical requirements and participant roles in a functional exercise.
103
The Functional Exercise
Purpose: Test the capability of one or more functions in an emergency
As realistic as possible without deploying actual resources
Takes place in EOC or operations center
Large conference space
104
Key Characteristics
Format:
Interactive—challenges entire business continuity program or crisis system in place.
Involves controller(s), players, simulators, evaluators
Stressful atmosphere
Lengthy, complex
Requires careful scripting, careful planning, attention to detail
105
Key Characteristics
It is geared for policy, coordination, and operations personnel.
Players respond realistically to carefully planned and sequenced messages.
Messages reflect events and problems.
Players’ decisions and actions:
Occur in real time.
Generate responses from other players.
106
Best Uses
Assess: BCP Direction and control.
Adequacy of plans, policies, procedures, roles.
Individual and system performance.
Decision-making process.
Communication and information sharing.
Allocation of resources.
Adequacy of resources.
107
Participant Roles: Players
The players are:
BCP staff, Leadership, Human Resources, Info Tech, Records, Facilities, Support staff, etc.
Duties:
Respond to messages as in a real emergency.
Decisions and actions:
Take place in real time.
Generate real responses and consequences.
108
Participant Roles: Simulators
“Actors”—People who portray organizations that interact with the players
Duties:
Deliver prescripted messages, ad lib spontaneous messages
Simulate actions taken by the organization (must implement all directives)
Inform controller of deviations
109
Organizing Simulators
How many?
At least one per organization represented in the operations center, plus some extras
Group by function:
Nonparticipating government agencies
Participating organizations
Private or volunteer organizations and individuals
110
Participant Roles: Controller
Supervises the simulation, ensures exercise stays on track and objectives are reached
Duties:
Training, orientation, and narrative
Monitor events, adjust when needed
Make decisions on unexpected actions
Maintain order and professionalism
111
Participant Roles: Evaluators
Role: Observe exercise progress, record and report observations.
Duties:
Observe and record player actions.
Remain unobtrusive.
Evaluate achievement of objectives, identify problems, inform controller.
Document positive and negative observations.
Prepare written report.
112
How It Works: Startup
Beginning: "Announced" or “No notice”
Briefing:
Overview of objectives
How exercise will proceed
Time period simulated
Ground rules
Narrative: Verbal, TV, computer, slides, dramatized
113
How It Works: Messages
Messages may be presented on paper, by phone, by radio, in person.
Simulators communicate messages to players; players respond.
Players make requests; simulators respond.
Messages are guided by MSEL.
Success depends on simulating reality.
114
How It Works: Realism
Strive for realism.
Encourage spontaneity:
Players need normal range of available responses.
Controllers, simulators need to “go with the flow” when situation calls for it.
115
How It Works: Controlling Action
Adjust scenario for spontaneous decisions.
Adjust pace:
Adjust message flow.
Even pace among participants.
Handle time skips.
116
Time Skipping
Allows multiple phases of an emergency in a short exercise
Transitions planned to coincide with natural breaking points
Controller presents transition updates
Simulators update simulation displays
117
Facilities and Materials
Exercise where you operate.
Provide:
Adequate space for players, simulators, evaluators, controller.
Needed equipment and materials: telephones, maps, displays, forms, etc.
118
Designing a Functional Exercise
Use the full 8-step process.
Attend to the details:
Convincing narrative
Carefully chosen major and minor events
Realistic sequence
Expected actions closely tied to objectives
Many well-conceived messages (100 or more in a larger exercise)
Tightly constructed MSEL
119
Next: The Full-Scale Exercise
Unit Summary
In Unit 6, we:
Reviewed how a functional exercise works.
Discussed exercise roles.
Examined how the design process is applied to functional exercises.
Identified exercise responsibilities
120
Unit 7: The Full-Scale Exercise
121
Unit Objectives
Describe the purpose and characteristics of a full-scale exercise.
Explain how designing a full-scale exercise differs from designing a functional exercise.
Identify planning considerations for site selection and scene management for a full-scale exercise.
122
The Full-Scale Exercise
Highly realistic—resources deployed
Lengthy, complex, “on location”
Interactive
123
Why Conduct a Full-Scale?
Tests ability to perform many functions at once
Tests total coordination
Reveals resource capabilities and shortfalls
Expands program scope and visibility
Attracts public attentionand raises credibility
Satisfies regulatory requirements
124
Exercise Requirements
Exercise experience
Total commitment
Support from key officials
Adequate physical facilities (EOC, Relocation Facilities)
Communication facilities
Financial resources
Planning and logistics
125
Controller(s)
If more than one controller, chief controller takes the lead
Manages the exercise
Responsible for starting on schedule
Designates exercise control point
126
Participants
Leadership
Coordination personnel
Operations personnel
Support functions (BCP, HR, IT, Facilities, Security, etc.)
127
Simulators
Volunteers who play the part of victims
May wear makeup to simulate injuries.
Act injured, unconscious,hysterical, dead, etc.
May play the part of another function.
128
Evaluators, Safety Officer
Evaluators:
Observe action.
Keep log of significant events.
May videotape action.
Safety Officer:
Analyzes exercise from safety perspective.
Ensures safe conduct of exercise.
Has authority to terminate activities.
129
How It Works: Start-Up
Announced or “no notice”—notification through normal channels
Responders go to event site—visual narrative
Decision makers go to EOC or area of congregation for leadership
Command posts set up as needed
130
How It Works: Action
Action occurs:
In the EOC.
At the relocation site.
Functions supply information to leadership once activated.
Action is sustained through messages, actions, and responses.
131
How It Works: Locations
Main event site depends on scenario
Secondary event sites (Work-at-Home, other back-up sites)
EOC (Coordination and Control):
Policy and coordination
Information
Direction and control
Communication to the organization
132
Designing Full-Scale Exercises
Design difficulties—logistics and design
Start small, build up
Use entire 8-step process, with differences:
Things replace words
Visual narrative replaces written scenario
Requires deeper analysis and greater attention to detail
133
Applying the Design Steps
Steps 1–4: Deeper analysis, greater attention to detail
Step 5: Lengthy narrative unnecessary
Step 6: Major and detailed events:
Many are simulated with victims, props
Cannot be random or haphazard
Step 7: Expected actions:
Based on objectives
Detailed list is needed
134
Applying the Design Steps
Step 8: Messages
Initial messages: Visual
Prescripted messages move action along
Anticipate the unexpected
Be flexible
135
Special Considerations
Walk the site or the space, evaluating:
Site selection (if back-up site unavailable).
Scene management.
Personnel and resources.
Response capability.
Safety and legal liability.
Emergency call-off.
The media.
136
Site Selection
Credible emergency
Realistic without interfering with normal traffic or safety
Enough space for victims, responders, observers, vehicles
137
Scene Management
Logistics
Creation of believable emergency scene
Number of victims
Props and materials
Number of controllers
138
Personnel and Resources
Participants and volunteers
Equipment and fuel
Materials and supplies
Expenses
Response capability:
Maintain enough staffingfor real emergencies
139
Safety and Liability
Designated safety officer with authority to terminate
Address safety in:
Exercise design.
Preexercise briefing.
Simulator and evaluator packets.
Examine field locations, resolve all hazards
Provide call-off procedures
Research liability
140
The Media
Exercises draw media attention.
Take advantage of attention to:
Gain support.
Increase realism.
Make allowances for:
Observers.
Public information people.
Targets of Opportunity (Interviews)
141
Unit Summary
In Unit 7, we:
Reviewed characteristics of a full-scale exercise.
Examined how the design process is applied to full-scale exercises.
Identified local design considerations.
Next: Exercise Evaluation
142
Unit 8: Exercise Evaluation
143
Exercise Evaluation
Observing and recording exercise activities
Comparing performance with the objectives
Identifying strengths and weaknesses
144
Unit Objectives
Describe the need for a systematic approach to exercise evaluation.
Identify and explain the tasks in the exercise evaluation process.
145
Why Evaluate?
Evaluation can identify:
Whether objectives are achieved.
Needed improvements in:
Plans, procedures, guidelines.
Emergency management system / Life Safety, etc.
Training and staffing deficiencies.
Equipment deficiencies.
Need for continued exercising.
146
Evaluation Team
Team:
• Helps plan methodology
• Observes and records exercise actions
• Helps prepare report
Leader:
• Develops methodology
• Selects and trains team
• Oversees evaluation
• Prepares report
147
Training the Team
Give an orientation covering:
Scenario.
Rules of play.
Objectives.
Evaluation requirements and procedures.
Evaluation forms.
Importance of being unobtrusive.
Run practice drills if needed.
Objectives
148
Evaluation Methodology
Team Structure:
Evaluators—number and background
Sub-team organization
Lines of authority
Communication and coordination
149
Evaluation Methodology
Evaluation Criteria:
Exercise ObjectivesExercise Objectives
Expected ActionsExpected Actions
Points of ReviewPoints of ReviewSimpleMeasurableAchievableRealisticTask-oriented
SimpleMeasurableAchievableRealisticTask-oriented
150
Evaluation Methodology
Evaluation Strategy:
Data collection method
Observation process
Evaluation forms
1. Recall specific objectives, events, expected actions.
2. Identify players expected to take action.
3. Locate evaluators to observe those players.
4. Brief evaluators in what
actions to look for.
1. Recall specific objectives, events, expected actions.
2. Identify players expected to take action.
3. Locate evaluators to observe those players.
4. Brief evaluators in what
actions to look for.
151
Post-Exercise Phase
Postexercise meetings:
Player debriefing
Evaluation team meetings
After action report
Implementing change
152
Player Debriefing
Occurs immediately after exercise.
Controller reviews objectives, successes, shortfalls.
Each player comments on performance.
Comments are recorded for after action report.
Questionnaire formmay be used.
153
Evaluation Team Meetings
Compare notes.
Analyze findings.
Develop accurate account of what worked and what did not.
Prepare after action report.
154
After Action Report
Documents effectiveness of the exercise
Basis for:
Planning future exercises
Upgrading emergency plan
Taking corrective action
Format varies
155
Implementing Change
Are the procedures sound?
Are resources sufficient to support procedures?
Are personnel adequately trained to follow procedures and use resources?
The goals of an exercise are not achieved until the recommendations from the
evaluation are implemented.
156
Next: Exercise Enhancements
Unit Summary
In Unit 8, we:
Discussed the need for a systematic approach to exercise evaluation.
Reviewed aspects of the evaluation process:
Team structure.
Evaluation methodology.
Postexercise phase.
Outlined an evaluation plan for an exercise.
157
Unit 9: Exercise Enhancements
158
Unit Objectives
Define the purpose of exercise enhancements.
Identify available resources for exercise enhancements.
Describe the benefits of specific exercise enhancement techniques.
159
Why Enhance?
The greater the realism, the more likely that:
Participants will take the scenario seriously.
Participants will learn from the experience.
Shortfalls and needed improvements will be revealed.
Creativity is the key to good enhancements!
160
Types of Enhancements
Communications equipment
Displays
Video and audio recordings
Computers
Miscellaneous equipment and supplies
People and props
161
Communications Equipment
Use communications equipment to:
Set up the narrative.
Advance the scenario.
Transmit messages.
Strive for realism, using:
Landlines / Fax Machines
Cell phones
Network
162
Displays
Provide context and detail for scenario.
Include:
Maps.
Charts.
Status boards.
Black or white boards.
Chart paper and easels.
163
Maps
Include, as needed:
City/county/subdivision/downtown.
Sewer, water, electric, gas
Flood plain, contour
Police, fire district
Weather
Floor Plans of relocation space.
Display on wall or provide handouts.
Use overlays for updating.
164
Charts
Problem and event log
Damage assessment chart
Facility charts and status boards
Organization charts
MSELMASTER SCENARIO EVENTS LIST
Time Message/Event Expected Actions
07:35 Plane radios tower: losing enginepower and altitude.
1. Tower notifies dispatch center.2. Dispatch alerts police, fire, medical to proceed
to airport.
07:40 –07:50 Pi lot reports majorvibrations/noise; requests runwaydesignation.
1. Tower designates runway; notifies dispatch ofrunway and potential for mass casualtyincident.
2. Dispatch relays runway to police, fire, medical.3. Dispatch notifies hospitals.4. Crash fire rescue initiates ICS; notifies
dispatch of CP and staging locations.5. Dispatch relays CP and staging locations to
police, fire, medical.
07:55 Hospital calls dispatch requestingmore information.
1. Dispatch obtains potential number ofcasualties and relays to hospital.
2. Hospital noti fies other medical facil ities.
08:00 Media calls dispatch requestinginformation.
(etc.)
165
Video, TV, and Audio
Introduce or update narrative.
Video/TV:
News broadcasts
Interviews with officials, “victims”
Audiotape situation reports, play on radio
166
Computers
Computerized inventory lists
Weather tracking
Log of events / actions taken
Communications
167
Equipment and SuppliesSimulate outages
Equipment:
Office machines
Other equipment specific to that function being tested
Supplies:
Office supplies
Display materials
References, directories, resource lists
168
People and Props
Victims
Mannequins (casualties)
Response equipment
Smoke, “hazmat” residues
Contained fires
Rubble
169
Potential Resources
Health organizations
Educational organizations
Business and industry
Government agencies
Volunteer agencies
Service organizations
Emergency response organizations
Religious organizations
Military
Amateur radio
Search & rescue
Transportation
170
Costs and Liability Issues
Time for pickup and return
Potential for damage/replacement costs
Arrangements for timely return of items and volunteers
Liability insurance
171
Logistics
Enhancement uses
Prop placement
Prop management
Volunteer transportation
Prop pickup and return
Condition of returned items
Cleaning
172
Next: Designing a Functional Exercise
Unit Summary
In Unit 9, we:
Discussed why enhancements are used and the various types of enhancements that may be used.
Identified potential enhancement resources and logistics issues.
Planned possible enhancements for an exercise.
173
Unit 10: Designing a Functional Exercise
174
Unit Objective
Design and implement a small functional exercise using the eight-step design process.
This will include:
Developing a functional exercise based on objectives.
Conducting a functional exercise as controller, simulator, and/or evaluator.
Experiencing a functional exercise as a player.
Participating in a postexercise debriefing.
175
Unit Activities
1. Organizing:
Select BCP –type exercise.
Identify jurisdiction characteristics.
Designate team roles.
2. Designing exercises:
Review a step (whole class).
Carry out the step (teams).
Debrief (whole class).
176
Unit Activities
3. Conducting exercises:
Round 1
Team A conducts exercise (controller, simulators, evaluators)
Team B experiences exercise (players)
Round 2
Team B conducts exercise
Team A experiences exercise
4. General debriefing
177
Exercise Design
1. Needs assessment
2. Scope definition
3. Purpose statement
4. Objectives
5. Narrative
6. Major and detailed events
7. Expected actions
8. Messages
178
Written Materials
Design package
For Controller:
MSEL
Design package
For Evaluators:
Objectives
Evaluation forms
For Simulators:
Messages
For Players:
Organization profile
Divisions, departments
Player roles
Resources
Maps
Statement of purpose
Narrative
179
Needs, Scope, Purpose
NEEDS ASSESSMENT:
Review emergency plan, past exercises.
Identify:
Primary/secondary hazards.
Past problems, plan/SOP weaknesses.
Skills needing practice.
Improvements needing testing.
New facilities, personnel, equipment.
Need for role clarification.
180
Needs, Scope, Purpose
SCOPE:
Set limits related to:
PURPOSE STATEMENT:
Why exercise is being conducted.
Assignment: Assess needs, define 5 scope elements, write purpose statement.
•Type of emergency.• Location.• Functions.• Participants.• Exercise type.
181
Objectives
Describe the performance you expect from participants to demonstrate competence.
State:
Who should do . . .
What under . . .
What conditions according to . . .
What standards.
182
Write SMART objectives:
S imple
M easurable
A chievable
R ealistic
T ask oriented
Objectives
Assignment: Write 3 to 5 good objectives.
183
Narrative
Sets the mood, sets up later action
Very specific
Present tense, short sentences
May be chronological (if event has warning time) or may emphasize emergency environment (if sudden event)
Assignment: Write narrative, 1-5 paragraphs long.
184
Major and Detailed Events
Major events: Problems requiring actions that will meet the objectives
Detailed events: Specific problem situations that will prompt one or more expected actions
185
Expected Actions
Actions you want participants to take in order to demonstrate competence
Closely tied to objectives
Types of actions:• Verification• Consideration• Deferral• Decision
Assignment: Write: 5 major events. 3 or 4 detailed events per major event. 1 or more expected actions per detailed event.
186
Messages
Communicate detailed events to participants.
Each message generates one or more expected actions that meet objectives.
Elements:
Source (WHO)
Transmission method (HOW)
Content (WHAT)
Recipient (TO WHOM)
To what effect
187
MessagesStart with an expected action.
Ask:
What message will motivate a participant to take that action?
Who should send it?
Who should receive it?
What should they say?
Assignment: Write one message for each expected action.
188
Written Materials
Design package.
For Controller:
MSEL
Design package
For Evaluators:
Objectives
Evaluation forms
For Simulators:
Messages
For Players:
Community profile
Agencies, departments
Player roles
Resources
Maps
Statement of purpose
Narrative
189
Exercise Goals
Develop a functional exercise based on objectives.
Conduct a functional exercise as controller, simulator, and/or evaluator.
Experience a functional exercise as a player.
Participate in a postexercise debriefing.
Exercise Time
Time to apply what you've learned.
190
191
Agenda (Two Rounds)
Each round:
Set-up (seating, maps, etc.) 5 minutes
Orientation 5 minutes
Assign player roles
Describe community,resources, personnel, etc.
Conduct exercise 20 minutes
Debriefing 10 minutes
192
Unit Summary
Reflecting on the experience:
What went well?
Comfortable level?
Difficulties during design?
What you would do differently next time?
Next: Course Closing
193
Unit 11: Course Closing
194
Accomplishments
Learned about 5 types of exercise activities and the exercise process.
Outlined a comprehensive exercise program for your jurisdiction or organization.
Identified potential design team members.
195
Accomplishments
Learned and practiced 8 design steps.
Developed a Master Scenario of Events List.
Developed problem statements for a tabletop exercise.
Recorded future ideas for full-scale exercises.
Developed an evaluation plan.
Planned exercise enhancements.
196
Accomplishments
Developed a functional exercise using the 8-step process.
Practiced conducting a functional exercise.
Practiced participating in a functional exercise.
Participated in exercise debriefings.
197
Applications
How will you will apply the design process locally?
What challenges do you foresee?
What resources can you tap?
THANK YOU!!TURN IN YOUR SURVEY ON THE WAY OUT…
