One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Babes in the Woods:Babes in the Woods:How Naïve Analysts Clashed with Trained KillersHow Naïve Analysts Clashed with Trained Killers
to the Mutual Benefit of Allto the Mutual Benefit of All
Babes in the Woods:Babes in the Woods:How Naïve Analysts Clashed with Trained KillersHow Naïve Analysts Clashed with Trained Killers
to the Mutual Benefit of Allto the Mutual Benefit of All
Fred CameronFred CameronOperational Research AdvisorOperational Research Advisor
to the Director General Land Capability Developmentto the Director General Land Capability Development
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Defence Research and Development Canada
Kingston, Ontario
Directorate ofLandSyntheticEnvironments
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Land Capability Development Operational Research Team
Land Capability Development Operational Research Team
• Mr Fred Cameron (Team Leader)• Mr Roger Roy• Ms Eugenia (Jenny) Kalantzis• Mr Ian Chapman • Mr François Cazzolato• Maj Bruce Chapman (to join July 2006)
• The Army Experimentation Centre– Part of the Directorate of Land Synthetic Environments (Army’s home for
Computer-based Wargames, Models, and Simulations for Training and Experimentation)
• Strategic Analyst – Mr Peter Gizewski• Science Advisor – Mr Regan Reshke
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Deciding Which Method to Use – A Meta-Decision
Deciding Which Method to Use – A Meta-Decision
• A meta-model of the meta-decision process• An analysis:
– Description• What happened?
– Ascription• Were there causes and effects?
– Prescription• What should we do in future?
– Proscription• What should we avoid in future?
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Cameron’s Rubbish Bin Model of Decision Making
Cameron’s Rubbish Bin Model of Decision Making
• Born of many years of observation• Employed for organizations that
are “organized anarchies”• Basis for workshop on “Ambiguity
and Command” in 1986• Used provocatively:
– Is it really that bad?
Sources: Cohen, March, and Olsen (1972), March and Weissenger-Baylon (1986), and March (1994)
March’s Garbage Can ModelMarch’s Garbage Can Model
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
March’s Garbage Can Model of Decision Making
March’s Garbage Can Model of Decision Making
Complex interactions between:
• Actors (decision makers)
• Problems
• Choice opportunities
• Potential solutions
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
March’s Garbage Can Model of Decision Making
March’s Garbage Can Model of Decision Making
• Some ways to “improve” garbage-can decision making– Increase the heat and pressure
• Force more frequent interactions
– Put more garbage in the can• Add more potential solutions• Add more problems, more actors,
more decision opportunities too?
– Make actors “stickier” • Hope that they will carry problems or
potential solutions with them long enough to get a better fit
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
DescriptionDescription
• “Deciding Which Method to Use”– OR analysts make great critics of the decision-
making of others– But, turn the lens into a mirror
• The Analyst’s Toolbox… Our Experience
• Canadian Army Modelling and Simulation
• Specific Examples and Lessons
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
The Operational Research Tool BoxSome Examples
The Operational Research Tool BoxSome Examples
• Structured brain storming• Scenarios• Analysis of historical data • Options analysis, Decision analysis• Group ranking• Seminar war games• Analytical tools – mathematical analysis• Computer-based war games (JANUS, OneSAF)• Trials, exercises, and evaluations
– Source: Future Army Development Plan, 1998
More:• time• resources• credibility
Less:• time• resources• credibility
Soft
OR/OAMethods
Hard
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Our Early Meta-decision Rules for “Deciding Which Method to Use”
Our Early Meta-decision Rules for “Deciding Which Method to Use”
• If time and resources are short, and credibility is not an issue, then use soft OR methods
• If credibility is paramount, and time and resources are available, then use hard OR methods
• Our experience supports: “hard and soft methods can be used together at appropriate stages of a study (usually soft for problem formulation, hard for resolution)”
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Simulation TypesSimulation TypesPeople
Equipment
Real
LiveReal
Simulated
Virtual ConstructiveSimulated
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Goal Synthetic EnvironmentGoal Synthetic Environment
Live
Constructive
Virtual
Joint War Game
Simulators Simulators Simulators
Army Navy Air Force Operational
Army Navy Air Force Tactical
Link
Link
Link
Link
C2
Sys
tem
s
Rea
l Wor
ld Link Link
Link Link Link
Link Link Link
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Constructive SimulationsConstructive Simulations
• Command and Staff Trainer• UK’s ABACUS
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Constructive SimulationsConstructive Simulations
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Lesson 1: AE-7B* Armed Griffon HelicopterLesson 1: AE-7B* Armed Griffon Helicopter• Aim. To provide insights into differences in effectiveness
and survivability between an armed and an unarmed helicopter
• Purpose. To validate the incorporation of aerial firepower requirements in the CH-146 Griffon Helicopter mid-life upgrade
• Simulation Structure. Constructive and virtual computer-based simulation supported by OneSAF Testbed Baseline (OTB), with virtual workstations for Griffon helicopters and Pointer-type Unmanned Air Vehicles (UAV)
* AE = Army Experiment
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Lesson 1: AE-7B Armed Griffon HelicopterLesson 1: AE-7B Armed Griffon Helicopter
Constructive
Virtual
Tactical
Griffon
DIS
OTBSAF
Pointer
DIS
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
AE-7B – The Textbook CaseAE-7B – The Textbook CaseJu
n 02
Jul 0
2
Aug
02
Sep
02
Oct
02
Nov
02
Jan
03
Feb
03
Mar
03
Apr
03
Dec
02
Define
Develop
Conduct
Analyze
Report
Weak Synthetic Environment SpecificationLate Terrain Generation
Late GriffonSimulation Delivery
Late Scenario Build
Strong Sponsor Involvement
Excellent Participant Base
Automated Small Sample Statistics
Corrective Action• Implemented Synthetic Environment Statement of Requirement
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Lesson 2: LOE* 0301 – TUAV in Controlled AirspaceLesson 2: LOE* 0301 – TUAV in Controlled Airspace
• Aim. To investigate possible reduced airspace restrictions for UAVs if Air Traffic Controllers have better situational awareness
• Objective. To identify any differences in situational awareness enabled by various improvements to ATC situational awareness (in the vicinity of Kabul)
• Simulation Structure. Constructive and virtual computer-based simulation supported by OneSAF Testbed Baseline (OTB), with virtual workstations for ATC Tower, Air Defence and ATC radars, and UAV
* LOE = Limited Objective Experiment
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Lesson 2: LOE 0301 – TUAV in Controlled AirspaceLesson 2: LOE 0301 – TUAV in Controlled Airspace
Constructive
Virtual
Tactical
Tower Ops
DIS
OTBSAF
TUAV
DIS
AD Rdrs/EOs
DIS
ATC Rdr
DIS
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
LOE 0301 – The Short CircuitLOE 0301 – The Short CircuitM
ay 0
3
Jun
03
Jul 0
3
Aug
03
Sep
03
Oct
03
Nov
03
Define
Develop
Conduct
Analyze
Report
Staff Check
Sponsor Identification
Problem Identification
Terrain Incomplete
Iterative Prototyping
Sponsor Participant Access
Automated Situation Awareness Analysis Tools
Report Pre-Preparation
Corrective Action• Create Terrain/Visualization Cell
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
AscriptionAscription
• Does the Garbage Can Model fit? Is it really that bad?
• Decision outcomes driven by– Temporal Confluence: Decision makers,
Choice opportunities, Problems, Potential solutions
• Is the method we use for “deciding which method to use” any better that that?
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Toolbox• Structured brain storming• Scenarios• Analysis of historical data • Options analysis, Decision
analysis• Group ranking• Seminar war games• Analytical tools –
mathematical analysis• Computer-based war games
(JANUS, OneSAF)• Trials, exercises, and
evaluations
The Operational Research Skill SetThe Operational Research Skill SetSkill set• Facilitator• Scribe• Co-author of scenarios
• Enumerator of votes• Analyst of preferences
• Modeller• Mathematician/Statistician
• Experimenter in simulation space
• Experimenter in live trials
Soft
OR/OAMethods
Hard
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Roles of Soft OR MethodsRoles of Soft OR Methods
• To support thinking and planning by an individual analyst or decision maker – problem articulation
• To support discussion between consultant and decision maker – problem negotiation
• To support debate and conclusion among decision makers – group decision support
• To initiate or strengthen organisational capabilities – organisational development
– Source: Steve Cropper presentation in Holt and Pickburn (2001)
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
PrescriptionPrescription
• Consider the full spectrum in the tool box
• Prepare for the full range of required skills
• Implement a “lessons” process, and strive to improve
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Application of the Spectrum of MethodsApplication of the Spectrum of Methods• Soft and hard OA should be viewed as equally useful and
part of the analyst’s toolkit. These should be viewed as a spectrum of options within the analyst’s toolkit which should be selected and used as appropriate.
• It was recognised that hard and soft methods can be used together at appropriate stages of a study (usually soft for problem formulation, hard for resolution).
• Analysts need to be aware of all OA techniques and, broadly where they should and should not be employed. Better education may help to achieve this.
• An expert practitioner (or team) is needed with expertise in the range of methods appropriate to the problem.
– Extracts from Holt and Pickburn (2001), pp. 15, 17 and 19
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
ProscriptionProscription
• Woe, woe, and thrice woe
• Beware of complexity
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Err
or
Complexity
SpecificationError
Minimize Model Error
Err
or
Complexity
MeasurementError
Models, Complexity and ErrorModels, Complexity and Error
Err
or
Complexity
Model Error
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Err
or
Complexity
SpecificationError
Err
or
Complexity
Model Error
Err
or
Complexity
MeasurementError
If we can increase the accuracy of performance
characteristics, we can accommodate greater complexity.
Models, Complexity and ErrorModels, Complexity and Error
Overall error drops.
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Err
or
Complexity
SpecificationError
Err
or
Complexity
Model Error
Err
or
Complexity
MeasurementError
Minimize Model Error by
Decreasing Complexity
Models, Complexity and ErrorModels, Complexity and Error
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Main RecommendationMain Recommendation
• Make the actors “stickier”:– Analysts need to be aware of all OA techniques
and, broadly where they should and should not be employed. Better education may help to achieve this
– An expert practitioner (or team) is needed with expertise in the range of methods appropriate to the problem
Source: Holt and Pickburn (2001)
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
ReferencesReferences
• Michael D. Cohen, James G. March, Johan P. Olsen “A Garbage Can Model of Organizational Choice” Administrative Science Quarterly, Vol. 17, No. 1. (Mar., 1972), pp. 1-25
• James G. March and Roger Weissenger-Baylon eds. (1986) Ambiguity and Command: Organizational Perspectives on Military Decision-Making. Marshfield, MA Pitman Publishing Inc.
• James G. March (1994) A Primer on Decision Making: How Decisions Happen. New York: The Free Press
• Future Army Development Plan (1998) Kingston, Ontario: Directorate of Land Strategic Concepts
• John Holt and George Pickburn (2001) OA Techniques for the Future. Farnborough, UK: Defence Evaluation and Research Agency. 30 March 2001
One Army, One Team, One Vision Operational Research
Une Armée, une équipe, une vision Recherche opérationnelle
Questions – DiscussionQuestions – Discussion
Fred Cameron
Tel: +1.613.541.5010 ext 2470
Email: [email protected]