Upload
jonathan-madden
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Operated by Los Alamos National Security, LLC for DOE/NNSAContractor Assurance Office (CAO)
U N C L A S S I F I E D
Learning About LearningProgress in Lessons Learned at Los Alamos National Laboratory
presented to the
DOE Operating Experience Coordinators WorkshopMay 5, 2010
Learning About LearningProgress in Lessons Learned at Los Alamos National Laboratory
presented to the
DOE Operating Experience Coordinators WorkshopMay 5, 2010
Bob Stuewe
Operating Experience Coordinator
Contractor Assurance Office – Performance Feedback
Los Alamos National Laboratory
Bob Stuewe
Operating Experience Coordinator
Contractor Assurance Office – Performance Feedback
Los Alamos National Laboratory
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Bottom LineResults Take About Three Years to Manifest
End of CY 2006 163 processed per year
• Pass preliminary screening• Nearly all DOE/HSS• Safety• 10 internal per year
200 web users/month
250 in searchable archive
?? document accesses per month
?? searches/month
Trend Occurrences
Track Communications
2 FTEs• 1 FTE on the Trending
FY 2010 (Now) 800 processed per year
• Pass preliminary screening• Multiple pre-planned sources• All program/function areas• 150-200 internal per year
1500-3000 web users/month
>2100 in searchable archive
>30,000 document accesses per month
100-225 searches/month
Analyze/Trend Occurrences, Causes
Track Actions (risk-based)
2.5 FTEs• 1.2 FTE on Analysis
Slide 2
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Learning About Learning Lessons About Effective Lessons Learned Programs
Champion Learning, Not Lessons
Reliable Processes Build Trust and Use
Clarify Roles to Eliminate Redundancy
Support Work Processes
Enable Risk Management
More Focused Equals More Effective
Build and Use Indicators of Effectiveness
Encourage Innovation
Slide 3
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Champion Learning Not LessonsOperating Experience Programs are Key to Knowledge Management
Slide 4
Operating Experience Program
World of Possible Knowledge
Learning happens in the work unit in many, many ways(Required reading, training, requirements updates, work team meetings, plan of the day, procedure updates,
learning teams, process improvement teams, corrective action teams etc.)
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Champion Learning Not LessonsFrom the Criteria for Performance Excellence (MBNQA)
Learning is one of the three pillars of organizational performance.
• Learning is referenced 103 times in 88 pages
• “The term “performance excellence” refers to an integrated approach to organizational performance management that results in (1) delivery of ever-improving value to customers and stakeholders, contributing to organizational sustainability; (2) improvement of overall organizational effectiveness and capabilities; and (3) organizational and personal learning.”
• “To be effective, learning should be embedded in the way an organization operates.”
Slide 5
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Champion Learning Not LessonsKnow How Your Organization Learns and Support the Methods
Nuclear and High Hazard Facilities• Conduct of Operations: required reading, procedures, FLM weekly meeting,
critiques, facility lessons learned coordinators, etc.
Subject Areas• Electrical Safety Committee, Chemical Safety Committee, Glovebox Safety
Committee, Security Incident Analysis, etc.
Lower Hazard Organizations• Worker Safety and Security Teams, Learning Teams, Management Meetings,
mentoring, on-boarding, etc.
Institutional• Management review boards, requirements updates, training updates, institutional
web pages, etc.
Slide 6
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D Slide 7
Reliable Processes Build Trust and UseTrust Requires Relationships – With Products/Services Used
Operating Experience and Lessons Learned Process– Collects, screens, risk-prioritizes, and distributes operating experiences and lessons
learned from key external and internal sources using timely communications.– Three risk-priority levels with different disposition/communication pathways
Lessons Learned and Operating Experience Archive– Captures relevant lessons and best practices reported by the Laboratory managers
and workers, contractors, subcontractors, and from other DOE sites and external sources.
– Knowledge base
MirrorMirror: Translating Events into Actionable Information
– Communicates high-leverage actions, lessons learned and best practices associated with operating experience patterns and trends.
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Reliable Processes Build Trust and UseProcess Management is Proven To Create Reliable Products/Services
1. Establish Process Management• Lean Six Sigma - Champion and Yellow Belt• Process Team (Weekly - Operations)• Integration Team (at least Annually - Integration and Strategy)
2. Benchmark (inside and outside DOE)
3. Define Product/Service Quality Specifications• Purpose -> Products/Services-> Customers-> Requirements -> Controlling characteristics• Defined processes (steps and roles) for each product (mapping)
4. Set up Process Controls and Improvement• Program Description, Procedures, Checklists, Tools• Use Contractor Assurance System: Goals, Measures, Assessments, Issues Mgt, Process
Improvement• Establish Task Roles (daily, weekly, monthly, quarterly, annual)• Annual Improvement Plan (effectiveness and efficiency)
5. Execute and Innovate
Slide 8
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Reliable Processes Build Trust and UseReliable Processes Requires Reliable Operations
Slide 9
Monday
Process Team
Updates
Upcoming
Communications Plan
Operations
Improvement Plan
Wednesday
MIRROR Team
Organized by sections of the publication
Quality review and improve key messages and actions
Design year-end and next issue
Thursday
Screening Team
Process Team and occasionally the site office
Finalize screening
Determine dispositions
Write headlines
Publish Weekly Summary Email
Daily intake and preliminary screening Work on products and services (MIRROR, lessons, support customers) Work on Communications Plan and Improvements Plan
Tuesday
Integration
CAS Management Team
Institutional Management Review Board
As Needed
Friday
Week Close Out
In case extra time is needed to complete the weeks deliverables
Backlogs are deadly
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Clarify Roles to Eliminate RedundancyFunctional Approaches Increased Costs, Decreased Effectiveness
LL requirements embedded in many directives, guides
Alignment and Integration can save taxpayer money
LANL approach• OPEX program supports multiple functions – use an integration team to plan annual
improvement (Safety, Security, Env Mgt, Project Mgt, Facilities, HPI, Training)
• Issue & Corrective Action Management - handles extent of condition for institutional/local, OPEX program helps with what’s in between
• QA/SCI – handles work-related suspect, counterfeit and defective items, OPEX serves as knowledge base, get directly involved at Urgent level
• Planned integration with internal security LL and Security Lessons Learned Center• Organizational and Subject Area LL Knowledge Bases and Process align with and link to
Institutional• Contractor Assurance System: Measures, Assessment Planning, Issue Effectiveness
Evaluation• Site office uses LANL screening and disposition process, oversight of higher risk
Slide 10
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D Slide 11
Support the Work ProcessesEasy to Think About the “Push”
Lessons Learned Integration with Work Performance
LL P
roce
ss T
eam
*M
ana
ger
sW
orke
rs
Monitor Internal & External Sources
of Lessons Learned and
Operating Experiences
1. Screen: Relevance Applicability Priority
2. Capture lessons learned in LL Archive
Communicate key Lessons Learned to
LANL managers , workers , and DOE
sites
Communicate relevant Lessons Learned to workers
Incorporate into Work Processes
Perform Work
Evaluate work to identify lessons
learned
Identify improvement opportunities
Share Lesson Learned with the
Laboratory
Lessons Learned Archive
Slide 12
Support the Work ProcessesWork is Always Underway – Don’t Try to Change Cars While Moving
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
Support the Work ProcessesBeware of the Adverse Unintended Consequences of the “Push”
Common Lesson from OPEX: Not Thinking Through a Change
Nuclear Facilities• Procedures control the quality• Modifying procedures include many steps, for good reason
Conservation of Attention Law• Only so much time in a day• Rushing to implement a lessons learned can create rework
Resource Allocation is Not an OPEX Role• Acting on a lesson competes for resources• OPEX Coordinators do not know the daily priorities on the work floor
13
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D Slide 14
Support the Work ProcessesHard to Think about the “Pull” – But Equally Important
Lessons Learned Integration with Work PlanningL
L P
roce
ss T
eam
*M
ana
gers
Wor
kers
Access & search LL Archive for lessons relevant to new or modified work scope, training requirements ,
and/or procedures
Ensure that changes associated with lessons are effectively communicated to
workers
Use lessons to plan work:Project PlansDesign InputsIntegrated Work Documents (IWD)Laboratory proceduresLocal instructions and procedures Training plans and materialsOther work planning
Keep Lessons Learned Archive up to date
Define new or modified work scope, training requirements , and /or
procedures
Lessons Learned Archive
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
Support the Work ProcessesSupporting Work Processes Requires a Knowledge Base
Slide 15
MS Sharepoint
Reduced Cost• IT Costs
• Process Costs
Aligns with Work Processes• Topical Libraries
(key systems)
• Alerts (individual and topical groups)
• How-To Tools
• Full PDF search capability
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Hazard inventory is pretty stable at the sites• Sharing known or low hazards provides little value to management• “New” and “Significant” Hazards important and valued
Risk Management• About reducing consequence or probability• Risk Management
— What is the risk? (to the mission)— How large are the potential adverse consequences?— What are you doing to handle it?— How likely are you to succeed?— Should you do more?
• Requires knowledge of organizational processes— Russian Hydro as an Example
Enable Risk Management To Engage Senior Management – Help Reduce Risk
Slide 16
Where Lessons Learned can add the most value to
management.
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D Slide 17
Work Planning Processes (Line)
TrainingDevelopment (Support)
Requirements Development (Support)
Enable Risk ManagementRisk-based Prioritization Matrix derived from O210.2 with Security from IMI levelsUrgent and Caution matrix essentially identical to Order – will add Best Practice un future
Communicate
URGENT
Caution
Informational
ClarifyScreenMonitorSources
Urgent = Immediate action is needed to ensure that the Laboratory is in a safe, secure, and environmentally sound mode relative to the information identified in the alert. Active engagement of relevant manager(s). Formal issues and corrective action tracking.
Caution = Timely action may be needed. Routine notification through Lessons Learned and Operating Experience Weekly Summary emails and LANL Home Page. LANL managers determine relevance and take needed action.
Informational = Future action may be needed. Posted to LANL Home Page if broadly applicable and actions clear. Important to knowledge base.
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
Enable Risk Management Risk Levels Differ from Site to Site
Slide 18
Pre-Screening Post Screening #HSS - Green Informational 21
HSS - BlueCaution 4Informational 44
HSS - RedCaution 4Informational 6
HSS - YellowCaution 38Informational 65
Urgent Lessons most commonly associated with DOE Safety Alerts and Advisories, defective
safety items, and other sources
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D Slide 19
Enable Risk ManagementSelect Sources to Support Risk Management – Might Change over Time
Primary Routine Sources MonitoringFrequency
DOE HSS Server/Web (LL, Alerts, OE Summaries, Security Lessons Learned Center (SLLC))
Daily
Partner Alerts (Bechtel, BWXT, UC, etc.) As Issued
ORPS (from Other Sites) Daily
Suspect Counterfeit/Defective Items Daily
U.S. Chemical Safety Board Monthly
Enforcement Actions and GAO Reports Quarterly
Consumer Products (CSPC, FDA) As Issued
Operated by Los Alamos National Security, LLC for NNSA
U N C L A S S I F I E D
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Office
U N C L A S S I F I E D
More Focused Equals More Effective Tailor Communication Methods to Audience and Media Capability
Slide 20
1%
10%
12%
28%
49%
Dispositions of Last 400 Lessons Processed
Custom
LANL Homepage Only
Weekly Summary Email Only
Weekly Summary Email and LANL Homepage
Archive for Future Use
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
More Focused Equals More Effective Weekly Summary Email – Management and LL Roles with Prioritization
Recipients• Division Leaders and above• NNSA Site Office• LL Roles and Upon Request• Augments daily ORPS
Slide 21
Specific = Relevant1. Define Most Specific Customer
2. Select Words in Headline
3. Integrity Check
4. Order (First and Last Most Read)
Scan-able (6 most important/actionable; > 6 has lower read rate)
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
More Focused Equals More Effective LANL Homepage to Reach Workers Directly
Some Communication Overlap Helps Effectiveness
Main Users• Workers
Broadest lessons• Includes worker home
safety
Specific = Relevant• Even shorter headlines• Pre-job brief is major
customers
Slide 22
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
More Focused Equals More Effective Internal Lessons Learned Submission – Content over Format
Adobe Fill-able Form• Just the Facts
— Word Docs too slow, costly, hard to share with DOE
• Field Validation – reduce errors• Fields support processing
— Screening and prioritizing— Interface to DOE/HSS
• Includes picture objects• Submit Button linked to LL
Archive and Process• Becoming standard at facility
level
Slide 23
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
More Focused Equals More Effective LANL Mirror (Quarterly Publication)
Translating Events into Actionable Information• Events binned by systems
and processes• Patterns and trends
(incidents, occurrences, causes)
• Year-End and Two-Year Running Reflection
• Use statistics to identify changes/trends
• High-leverage lessons/actions aligned with greatest performance gaps
• Recognition for developing and sharing lessons learned
Slide 24
0 5 10 15 20 25 30
Packaging and Transportation
Suspect Counterfeit Items
Authorization Basis (DSA)
RCRA Compliance
Conduct of Operations
Largest Changes in Number of Occurrences by ProcessFY09 FY08
0
5
10
15
20
25
FY08Q1 FY08Q2 FY08Q3 FY08Q4 FY09Q1 FY09Q2 FY09Q3 FY09Q4
Num
ber
Fiscal Year Quarter
Baseline of Radiation Protection Occurrences
Mean = 4.1/qtr 95% Mean Confidence Level = 1.9/qtr
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
More Focused Equals More Effective LANL Mirror: Events Causes Checklist for Improvements
Slide 25
Do managers and workers understand what work practices are to be used?
Do managers and workers understand why prescribed work practices are the right ones and how to perform them well? Are managers and workers sufficiently engaged and motivated to do the work right, rather than making optimistic assumptions and avoiding work paths that require extra effort? Are enough of the right resources (skills, equipment, tools, support services) allocated to tasks to enable timely completion of work? Are the potential adverse impacts of changes to activities, processes, and systems effectively evaluated and mitigated? When activities cross organizational boundaries, are the communication and coordination methods as timely and effective as the work flow requires?
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
Build and Use Indicators of LL Effectiveness Fact-based Improvement is the Objective
Use CAS Dashboard• LL Indicators
— Processing time performance• Urgent within 24 hours
• Caution within 7 days
— Document Access Rate— Search Rate— Knowledge Inventory— Internal LL Generation Rate
• LL Program Execution— Routine Process Milestones— Progress on Improvement Plan— Corrective Actions
Slide 26
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
Build and Use Indicators of LL EffectivenessNeed measures than can help understand Cause and Effect
Slide 27
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
Build and Use Indicators of LL Effectiveness Need measures than can help measure use for Work Planning
Slide 28
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
Encourage InnovationLANL Performance Communication Center
Modernizing Communications
• Chat with an Expert— Builds Knowledge Base
• Blog
• Role-based information— Relevant Tools
— Relevant Requirements
• Best Practices (coming soon)
— Custom version of LL Form
— Recognition
Slide 29
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
Encourage InnovationSub-Unit Approaches Evolve as their Learning Processes Mature
R&D key lessons included in “On-Boarding” materials for new hires (might become a best practice)
Local LL Archives Linked to Institutional Archive
• Project Management
• Engineering
• D&D
• Maintenance
• Facilities
Learning Teams (linked to HPI implementation)
Slide 30
Operated by Los Alamos National Security, LLC for NNSAContractor Assurance Offoce (CAO)
U N C L A S S I F I E D
Learning About LearningProgress in Lessons Learned System at LANL
Questions?
Slide 31