Embed Size (px)
Citation preview
System Simulators: The Theory and Practice Ken Thompson, Sarah Fraser & Göran Henriks
Session Code: M14 The presenters will advise the session
moderator of any relevant disclosures
before the presentation
Monday Dec 9 8:30-16:00 PM
Session Objectives
Describe the theory and practice of simulation
and how it relates to both system improvement
and learning
Assess different simulation methods and explore
ideas for computer-based simulations or
learning games
Identify, test, and critique the ways in which
simulation can improve systems and enhance
learning and development
P2
Session 1
Introductions & Welcome
Sarah (15 mins)
3
4
Objectives of the Mini Course
• Introduction and Welcome • Introducing the 3 presenters • Our Experience: simulations, OD,
leadership, team working and QI • Our History: Sweden and UK • Review the Agenda and Timings
IHI Simulation Mini Course
Agenda 1. Introductions & Welcome 2. Simulation 101 3. Collaboratively Building your first simulation 4. Spread: Innovation Diffusion Simulation 5. Simulations in the Real World 6. Simulations for System Improvement 7. Simulations for Learning & Development 8. Wrap-Up/Q&A/Feedback/Action Planning
5
Table Exercise A:
Delegate experience and
expectations
Sarah (5 mins)
6
Session 2:
Simulation 101 Ken (30 mins)
7
Introduction to Simulation
The history of Simulation
The benefits and limits of Models and Simulations (including what simulation isn’t)
The 2 main uses of Simulation:
1. Simulation for System Improvement 2. Simulation for Learning & Development Useful Criteria for spotting potential simulation opportunities
8
9
A simulation is an dynamic model of “System” which responds to user inputs in a meaningful way depending on its purpose.
Two key differentiators: 1) Model Type - Realistic/Abstract* - the degree to which the simulation aims to faithfully reproduce a target system 2) Target System Type - mechanical machine, living thing or social concept (such as an Organisation or Team or Process)
This mini course is about the abstract simulation of Social Concepts not Mechanical Machines or Living Things * The Realism required is usually proportional to: • Cost of Failure/Cost of Training • Target Size of the User Community • Lifetime of Applicability
What do we mean by Simulator or Simulation?
Focus of Session: Model Type: Abstract System Type: Social Concept
Who best understands the potential of simulation?
Lessons From Aviation! Key Areas 1. Skills Training 2. Emergency Procedures 3. Type Conversion Main BENEFITS A. Safety B. Costs C. Environment
10
Thompson’s 10 Rules of Good Simulation Design
1. All models are wrong but some are useful (attributed to Albert Einstein)
2. Machines can be faithfully simulated; organisations, people, culture and social structures are not machines
3. Simulations lack credibility if they cannot reproduce past behaviour (this necessary for usefulness but never sufficient)
4. Some simulations produce forecasts - no simulation predicts the future
5. Simulations can be precise but never accurate
6. You can have realism or usefulness in a model but not both (see Rule 1)
7. In simulation complexity reduction is the biggest challenge - abstraction is the best tool to help achieve this
8. A good simulation makes visible what is important - a poor simulation makes important what is visible
9. Requisite Complexity: every unnecessary equation or variable in a simulation reduces its potential user base by 25% (Corey Peck)
10. The value of a simulation is measured only by the new actionable insights it produces and not by anything else
11
Table Exercise B:
Simulation Design Principles - Identify Good and Bad Examples
Sarah (10 mins)
12
13
First Use of Simulation: System Improvement
To Gain Actionable New Insight into
“Systems”
14
Gain Actionable New Insight into “Systems” by: 1. Challenging existing mental models and golden rules 2. Understanding non-Intuitive Behavior 3. Identifying Process Leakages and Blockages 4. Developing Clarity on Cause and Effect 5. Safe Experimentation through What-If Analysis
15
Gain Actionable New Insight into “Systems” by: #1 - Challenging existing mental models and golden rules
What is a mental model? Internal and External Models Skidding on Ice Example
16
Gain Actionable New Insight into “Systems” by : #2 - Understanding non-Intuitive Behavior Time Delays “Why do they (we) think in this short-sighted way? The reason is simple: it is a hardwired part of our Palaeolithic heritage. For hundreds of millennia, those who worked for short-term gain within a small circle of relatives and friends lived longer and left more offspring...The long view that might have saved their distant descendants required a vision and extended altruism instinctively difficult to marshal.”
Professor Edward Wilson of Harvard, one of the world’s leading biologists.
But also ….. - Feedback Loops - Exponential Effects - System Boundary Effects - System Archetypes
017
“Leakage” is where performance falls short of stated KPI performance - can be impacting results by up to 50% over 3 years!
“Blockage” is where it is possible to go beyond the stated KPI performance - often by simply redirecting effort invested elsewhere.
Gain Actionable New Insight into “Systems” by : #3 - Identifying Process Leakages and Blockages
Syst
em P
erfo
rman
ce
18
Gain Actionable New Insight into “Systems” by : #4 Developing Clarity on Cause and Effect
• Leading and Lagging Indicators
• Kaplan and Norton
• Balanced Scorecard & Strategy Maps
• “Review View Mirror” Management
“Do people get good bodies by Swimming or do people with good bodies choose to Swim?” Thinking Error #2: The Swimmers Body Illusion from The Art of Thinking Clearly (Rolf Dobell)
19
Gain Actionable New Insight into “Systems” by : #5 Safe Experimentation through What-If Analysis
“Good Decisions come from experience. Experience comes from making Bad Decisions.”
Unknown
“You start your career with a full bag of luck and an empty bag of experience. The plan is to fill your bag of experience before your bag of luck runs out.”
Well known Pilot’s Saying
Table Exercise C:
Begin identifying some potential
simulation applications of interest in System Improvement
Sarah (15 mins)
20
21
Second Use of Simulation: Learning & Development
22
Q: How do we learn most effectively? A: When all our learning modes are stimulated!
BEHAVIOURISM Stimulus Response
SOCIAL & CONTEXTUAL Learn from and
with others
COGNITIVISM Flow of Knowledge
CONSTRUCTIVISM Mental Models
EXPERIENTIALISM Learning Experiences
Ref: “Behaviourism and Games” by CARLTON REEVE
The Experiential Learning Cycle
23
The Kolb Cycle
Second Use of Simulation: Learning & Development
What is the most valuable organisational learning?
• What is a dilemma? • Dilemma resolution versus Decision-Making • How do the best practitioners resolve the central
dilemmas in their jobs?
24
Second Use of Simulation: Learning & Development
Business simulation games are ideal "observatories" of individual, team member and group behavior:
25
• Lateral Leadership • Listening Skills • Emotional Intelligence • Relationship Building • Decision-making • Meeting practices • Conflict Resolution • Time Management • Self V Team Issues
• Brainstorming • Moods • Division of Labour • Managing Pressure • Operations • Reporting/Briefing • Mentoring & Coaching • Collective Leadership • Business Acumen
Second Use of Simulation: Learning & Development
A Newspaper is better than a magazine. A seashore is a better place than the street. At first it is better to run than to walk. You may have to try several times. It takes some
skill, but it is easy to learn. Even young children can enjoy it. Once successful, complications are minimal. Birds seldom
get close. Rain, however soaks in very fast. Too many people doing the same thing can also cause problems. One needs lots of room. If there are no complications, it can be
very peaceful. A rock will serve as an anchor. If things break loose from it, however you will not get a second chance.
26
Sense-Making & “Eureka Moments”
From “On Being Certain” by Robert Burton, M.D.
Table Exercise D:
Begin identifying some potential
simulation applications of interest in Learning & Development
Sarah (10 mins)
27
Session 3 Collaboratively Building your first
simulation
Ken (30 mins)
28
Building your first simulation
• Sim Model Building: Popular Schools of Thought
• Popular tools for simulation
• Live Model Building Session using ithink/stella
29
Sim Model Building: Popular Schools of Thinking
System Dynamics Main Perspective: Whole System Main Technique: Cause and Effect Models (Causal Loop Diagrams)
Agent Modelling Main Perspective: Individual Agents Main Technique: Agent Interaction Models (State Transition Diagrams)
Most problems can be modelled using either approach but one will be more useful, easier and insightful
Hybrid Models Blend both Perspectives in the one model
30
Popular tools for simulation include:
• ithink/stella (www.iseesystems.com)
• anylogic (www.anylogic.com)
• simulate (www.forio.com)
• insightmaker (www.insightmaker.com)
• netlogo/starlogo (NorthWestern/MIT)
• excel and addins (Microsoft etc)
• xcelsius/Dashboard Builder (SAP)
Plus many others (50+)
http://en.wikipedia.org/wiki/List_of_system_dynamics_software
31
What is ithink/stella?
• A software tool for visually modeling systems and simulating them over time
0.00 15.00 30.00 45.00 60.00 Time
1:
1:
1:
0
200
400 1: Elk
Elkreproducing
reproduction
rate
dying
death rate~
32
Modeling Building Blocks
Stock:
Any accumulation or level, such as a bank balance or the level of water in a bathtub
Flow:
Changes stock by adding or subtracting
stock
flow
Stocks & Flows
A bathtub is a stock.
The faucet and the drain are flows.
Bathtub
adding water draining water
34
Other Stocks & Flows
Same thing, different units
Self Esteem
building
35
Feedback Loops
A feedback loop occurs when a stock affects its flows
Bank ac countearn ing in terest
interest ra te
R
36
Reinforcing Loops
R
Size of Snowball adding snow
speed
37
Balancing Loops
38
Table Exercise E:
Collaborative Simulation Session:
We will identify the core model stocks in a simulation application of interest and
we will try to build it online
Ken & Sarah
39
40
Theme Hospital
Session 4 SPREAD: Innovation Diffusion Simulation
Ken (15 mins)
41
SpreadSim http://bioteamsapps.com/netsim/netsims/ken.thompson/spreaddemo/index.html
Videocast: http://www.screenr.com/MYP7 42
43
44
SpreadSim: Change Readiness screen
45
SpreadSim: Main Simulation screen
Table Exercise F:
Play with Spread Simulation online
using the manual
Sarah (30 mins)
46
Session 5 Simulations in the real world
Ken (30 mins)
47
Simulations in the real world
• Model Design: Avoiding the Black Box Trap
• The Data Quality Dilemma
• Dashboards and Scorecards
• Visualisation
48
What-If Models(I)
1. The Empirical Model This is the underpinning verifiable model of the part of the business under consideration. This should be totally objective. 2. The Causal Model This covers the dilemma decisions, how they relate to each other and the impacts they have on the Empirical Model. This will be subjective but should be based on real life experience. 3. The Value Model This captures the value produced in money terms when key objects (customers, users, orders, staff ...) transverse the Empirical Model.
Many models used by organisations have never been calibrated using the organisation's own historical data which seriously undermines their usefulness. This is generally not due to the lack of historical data but because of the way the models have been designed.
49
Ball-parking (aka Fermi Decomposition)
Data does not have to be 100% accurate nor do you need a complete scorecard of measures before you can gain valuable insights. Enrico Fermi (1901-1945) Nobel Prize winning physicist taught his students how to estimate the number of piano tutors in Chicago. What insights can you draw about your system from measures you already have or can easily estimate? For example Network Participation Rule (90/10/1) - 1% post, 9% comment and 90% lurk
However if you do ball-parking you need to understand “Confidence Intervals" Example: If I am 90% confident my team will score at least 1 goal and not more than 6 goals at their next game then my 90% confidence interval for this question is 1-6. However research shows that almost everyone systematically over-estimates or under-estimates their confidence levels unless they have been "calibrated".
50
Dashboards
Dashboards are the dynamic translation of Business Scorecards to computer. Effective Dashboards are based on the principles of Information Design:
Source: CORE DESIGN PRINCIPLES FOR DISPLAYING QUANTITATIVE INFORMATION by Stephen Few, 2006
#1: Display neither more nor less than what is relevant to your message. #2: Do not include visual differences in a graph that do not correspond to actual differences in the data. #3: Use the lengths or 2-D locations of objects to encode quantitative values in graphs. #4: Differences in the visual properties that represent values should accurately correspond to the actual differences in the values they represent. #5: Do not visually connect values that are discrete, thereby suggesting a relationship that does not exist in the data. #6: Make the information that is most important to your message more visually salient in a graph than information that is less important. #7: Augment people’s short-term memory by combining multiple facts into a single visual pattern that can be stored as a chunk of memory and by presenting all the information they need to compare within eye span.
51
52
How many rules of Information Design does this dashboard flaunt?
Public-Service Balanced Scorecard
Standard Balanced Scorecard
53
Table Exercise G:
Confidence Intervals Quiz
Ken (15 mins)
54
Ballparking Exercise: Confidence Intervals
9. When did England last win the World Cup for Soccer?
Source: How to Measure Anything by Douglas Hubbard
90% Confidence Interval Lower Bound Upper Bound
55
Session 6 Simulations for Systems Improvement
Sarah (45 mins)
56
Simulations for Systems Improvement * Demo of SimA (Sarah) * Swedish Case Study of SimA (Göran) * Overview of SimB (Ken)
57
SimA http://bioteamsapps.com/netsim/netsims/ken.thompson/SimA/index.html
58
What does SimA allow you to do?
Takes your own data or your own guess at the data (which is useful too)
Current Admission/ReAdmission rates (percentages) by different categories
Time to begin and then to complete a reduction in these admissions or readmissions rates
Simulates the results Number of readmissions Number of avoidable admissions Costs
You have many chances to play with the data, to see the possible results and then discuss the implications of each scenario.
59
SimA: Example of main “what-if” screen
60
SimA: Working at a more detailed level
61
SimB http://bioteamsapps.com/netsim/netsims/ken.thompson/SimB/index.html
62
63
SimA: Example of main decisions screen
64
Table Exercise H:
Discuss SimA Case Study and/or play
with SimA online
Sarah (15mins)
65
Session 7 Simulations for Learning & Development
Ken (15 mins)
66
Simulations for Learning & Development
C3: The Collaboration Trilogy Family of Simulations C1: Community Building Simulation C2: Social Network Simulation C3: Effective Indirect Action
67
THE COLLABORATION TRILOGY: DEVELOPING THE 3 NEW LEADERSHIP SKILLS
NETWORK LEADERSHIP: THE 3 NEW SKILLS 1. BUILDING NETWORKS RAPIDLY CREATING AN EFFECTIVE NETWORK (COMMUNITY) WITH SUFFICIENT POWER AND COMMITMENT TO HELP YOU TO ACHIEVE YOUR MISSION. 2. NETWORK DYNAMICS INFLUENCING AND COMMUNICATING WITH YOUR NETWORK IN THE MOST EFFECTIVE WAY. 3. EFFECTIVE INDIRECT ACTION ACHIEVING RESULTS THROUGH OTHER GROUPS AND INDIVIDUALS INCLUDING THOSE WHO ARE NOT CURRENTLY SUPPORTERS. 68
THE COLLABORATION TRILOGY: DEVELOPING THE 3 NEW LEADERSHIP SKILLS
#1: BUILDING NETWORKS YOUR CHALLENGE YOU HAVE BEEN ASKED TO BUILD AN ONLINE COMMUNITY TO DELIVER AN IMPORTANT PROJECT IN YOUR ORGANISATION. THE PROJECT MUST BE DELIVERED IN 36 WEEKS AND EVERY 6 WEEKS YOU MAKE YOUR DECISIONS ABOUT HOW TO INVEST YOUR EFFORT. YOUR OBJECTIVE IS TO SCORE “GREENS” ON ALL THE INDICATORS AND GET THE MAXIMUM RESULT FOR THE PROJECT. 69
THE COLLABORATION TRILOGY: DEVELOPING THE 3 NEW LEADERSHIP SKILLS
70
THE COLLABORATION TRILOGY: DEVELOPING THE 3 NEW LEADERSHIP SKILLS
#1: BUILDING NETWORKS SURFACE YOUR MENTAL MODELS ON BUILDING ONLINE COMMUNITY: 1. DEPENDENCIES BETWEEN THE DECISIONS (SEQUENCE)? 2. RULES & STRATEGIES FOR MAKING DECISIONS? 3. DIFFERENT WAYS DECISIONS CAN BE IMPLEMENTED AND THE
DIFFERENCE IT WOULD MAKE? 71
RECRUITMENT (MEMBERS)
LDRSHIP DEVEL (LEADERS)
ENV. DEVEL (ENVIRONMENT)
TRUST BUILDING (TRUST)
EXECUTION (RESULTS)
DECISIONS (RESULTS)
Table Exercise I:
Play The Network Development
Game and discuss/feedback
Ken (30 mins)
72
THE COLLABORATION TRILOGY: DEVELOPING THE 3 NEW LEADERSHIP SKILLS
#2: NETWORK DYNAMICS GAME YOUR CHALLENGE YOU ARE NOW RUNNING YOUR NETWORK/COMMUNITY AND HAVE A VERY IMPORTANT MESSAGE TO DELIVER TO EVERY MEMBER OF IT. HOWEVER YOU ARE ONLY ALLOWED TO TELL ONE PERSON IN THE COMMUNITY AND RELY ON THEM TO TELL EVERYONE ELSE? WHO SHOULD YOU TELL AND WHY? YOUR OBJECTIVE IS TO GET THE MESSAGE TO EVERY MEMBER OF THE COMMUNITY IN THE SHORTEST TIME.
73
THE COLLABORATION TRILOGY: DEVELOPING THE 3 NEW LEADERSHIP SKILLS
74
THE COLLABORATION TRILOGY: DEVELOPING THE 3 NEW LEADERSHIP SKILLS
#2: NETWORK DYNAMICS GAME SURFACE YOUR EXISTING MENTAL MODEL IN TERMS OF THE BEST PERSON/PEOPLE IN THE NETWORK TO SEND A MESSAGE TO OR TO INFLUENCE FOR THE BEST RESULTS: 1. IMPORTANCE OF THE DIFFERENT ASPECTS? THE PERSON, THEIR
RELATIONSHIPS AND THEIR NETWORK POSITION? 2. RELATIONSHIP BETWEEN THESE ASPECTS?
75
PERSON 1 PERSON 2
OUTGOING RELATIONSHIPS (STRENGTH)
INCOMING RELATIONSHIPS (STRENGTH)
ACTIVATION THRESHOLD (YOUR INFLUENCE ON THEM?)
ACTIVITY FREQUENCY (THEIR INFLUENCE ON OTHERS?)
NETWORK POSITION*
Table Exercise J:
Play The Network Dynamics Game
and discuss/feedback
Ken (15 mins)
76
THE COLLABORATION TRILOGY: DEVELOPING THE 3 NEW LEADERSHIP SKILLS
SOCIAL NETWORK ANALYSIS: SOME KEY DISTINCTIONS NETWORK POSITION* CAN BE MEASURED IN DIFFERENT WAYS BUT 3 OF THE MOST USEFUL ARE: 1. CENTRALITY
- THE MOST DIRECT RELATIONSHIPS IN NETWORK (E.G. DIANE)
2. ”BETWEEN-NESS” - IN THE BEST POSITION TO CONNECT DIFFERENT PARTS OF THE NETWORK BUT COULD BE BOTTLENECK (E.G. HEATHER)
3. CLOSENESS - HAS THE BEST ACCESS TO THE DIFFERENT PARTS OF THE
NETWORK (E.G. FERNANDO AND GARTH) 77
Session 8
Wrap-Up/Feedback/Q&A/Action Planning
Sarah (15 mins)
78
