Learning About Learning Progress in Lessons Learned at Los Alamos National Laboratory presented to the DOE Operating Experience Coordinators Workshop May 5, 2010 Bob Stuewe Operating Experience Coordinator Contractor Assurance Office – Performance Feedback Los Alamos National Laboratory

Bottom Line Results Take About Three Years to Manifest End of CY 2006 FY 2010 (Now) 163 processed per year Pass preliminary screening Nearly all DOE/HSS Safety 10 internal per year 200 web users/month 250 in searchable archive ?? document accesses per month ?? searches/month Trend Occurrences Track Communications 2 FTEs 1 FTE on the Trending 800 processed per year Pass preliminary screening Multiple pre-planned sources All program/function areas 150-200 internal per year 1500-3000 web users/month >2100 in searchable archive >30,000 document accesses per month 100-225 searches/month Analyze/Trend Occurrences, Causes Track Actions (risk-based) 2.5 FTEs 1.2 FTE on Analysis

Learning About Learning Lessons About Effective Lessons Learned Programs Champion Learning, Not Lessons Reliable Processes Build Trust and Use Clarify Roles to Eliminate Redundancy Support Work Processes Enable Risk Management More Focused Equals More Effective Build and Use Indicators of Effectiveness Encourage Innovation

Champion Learning Not Lessons Operating Experience Programs are Key to Knowledge Management World of Possible Knowledge Operating Experience Program Learning happens in the work unit in many, many ways (Required reading, training, requirements updates, work team meetings, plan of the day, procedure updates, learning teams, process improvement teams, corrective action teams etc.)

Champion Learning Not Lessons From the Criteria for Performance Excellence (MBNQA) Learning is one of the three pillars of organizational performance. Learning is referenced 103 times in 88 pages “The term “performance excellence” refers to an integrated approach to organizational performance management that results in (1) delivery of ever-improving value to customers and stakeholders, contributing to organizational sustainability; (2) improvement of overall organizational effectiveness and capabilities; and (3) organizational and personal learning.” “To be effective, learning should be embedded in the way an organization operates.”

Champion Learning Not Lessons Know How Your Organization Learns and Support the Methods Nuclear and High Hazard Facilities Conduct of Operations: required reading, procedures, FLM weekly meeting, critiques, facility lessons learned coordinators, etc. Subject Areas Electrical Safety Committee, Chemical Safety Committee, Glovebox Safety Committee, Security Incident Analysis, etc. Lower Hazard Organizations Worker Safety and Security Teams, Learning Teams, Management Meetings, mentoring, on-boarding, etc. Institutional Management review boards, requirements updates, training updates, institutional web pages, etc.

Reliable Processes Build Trust and Use Trust Requires Relationships – With Products/Services Used Operating Experience and Lessons Learned Process Collects, screens, risk-prioritizes, and distributes operating experiences and lessons learned from key external and internal sources using timely communications. Three risk-priority levels with different disposition/communication pathways Lessons Learned and Operating Experience Archive Captures relevant lessons and best practices reported by the Laboratory managers and workers, contractors, subcontractors, and from other DOE sites and external sources. Knowledge base Mirror: Translating Events into Actionable Information Communicates high-leverage actions, lessons learned and best practices associated with operating experience patterns and trends.

Reliable Processes Build Trust and Use Process Management is Proven To Create Reliable Products/Services Establish Process Management Lean Six Sigma - Champion and Yellow Belt Process Team (Weekly - Operations) Integration Team (at least Annually - Integration and Strategy) Benchmark (inside and outside DOE) Define Product/Service Quality Specifications Purpose -> Products/Services-> Customers-> Requirements -> Controlling characteristics Defined processes (steps and roles) for each product (mapping) Set up Process Controls and Improvement Program Description, Procedures, Checklists, Tools Use Contractor Assurance System: Goals, Measures, Assessments, Issues Mgt, Process Improvement Establish Task Roles (daily, weekly, monthly, quarterly, annual) Annual Improvement Plan (effectiveness and efficiency) Execute and Innovate

Reliable Processes Build Trust and Use Reliable Processes Requires Reliable Operations Monday Process Team Updates Upcoming Communications Plan Operations Improvement Plan Tuesday Integration CAS Management Team Institutional Management Review Board As Needed Wednesday MIRROR Team Organized by sections of the publication Quality review and improve key messages and actions Design year-end and next issue Thursday Screening Team Process Team and occasionally the site office Finalize screening Determine dispositions Write headlines Publish Weekly Summary Email Friday Week Close Out In case extra time is needed to complete the weeks deliverables Backlogs are deadly Daily intake and preliminary screening Work on products and services (MIRROR, lessons, support customers) Work on Communications Plan and Improvements Plan

Clarify Roles to Eliminate Redundancy Functional Approaches Increased Costs, Decreased Effectiveness LL requirements embedded in many directives, guides Alignment and Integration can save taxpayer money LANL approach OPEX program supports multiple functions – use an integration team to plan annual improvement (Safety, Security, Env Mgt, Project Mgt, Facilities, HPI, Training) Issue & Corrective Action Management - handles extent of condition for institutional/local, OPEX program helps with what’s in between QA/SCI – handles work-related suspect, counterfeit and defective items, OPEX serves as knowledge base, get directly involved at Urgent level Planned integration with internal security LL and Security Lessons Learned Center Organizational and Subject Area LL Knowledge Bases and Process align with and link to Institutional Contractor Assurance System: Measures, Assessment Planning, Issue Effectiveness Evaluation Site office uses LANL screening and disposition process, oversight of higher risk

Support the Work Processes Easy to Think About the “Push”

Support the Work Processes Work is Always Underway – Don’t Try to Change Cars While Moving

Support the Work Processes Beware of the Adverse Unintended Consequences of the “Push” Common Lesson from OPEX: Not Thinking Through a Change Nuclear Facilities Procedures control the quality Modifying procedures include many steps, for good reason Conservation of Attention Law Only so much time in a day Rushing to implement a lessons learned can create rework Resource Allocation is Not an OPEX Role Acting on a lesson competes for resources OPEX Coordinators do not know the daily priorities on the work floor

Support the Work Processes Hard to Think about the “Pull” – But Equally Important

Support the Work Processes Supporting Work Processes Requires a Knowledge Base MS Sharepoint Reduced Cost IT Costs Process Costs Aligns with Work Processes Topical Libraries (key systems) Alerts (individual and topical groups) How-To Tools Full PDF search capability

Enable Risk Management To Engage Senior Management – Help Reduce Risk Hazard inventory is pretty stable at the sites Sharing known or low hazards provides little value to management “New” and “Significant” Hazards important and valued Risk Management About reducing consequence or probability What is the risk? (to the mission) How large are the potential adverse consequences? What are you doing to handle it? How likely are you to succeed? Should you do more? Requires knowledge of organizational processes Russian Hydro as an Example Where Lessons Learned can add the most value to management.

Enable Risk Management Risk-based Prioritization Matrix derived from O210.2 with Security from IMI levels Urgent and Caution matrix essentially identical to Order – will add Best Practice un future Urgent = Immediate action is needed to ensure that the Laboratory is in a safe, secure, and environmentally sound mode relative to the information identified in the alert. Active engagement of relevant manager(s). Formal issues and corrective action tracking. Work Planning Processes (Line) Training Development (Support) Requirements URGENT Monitor Sources Screen Clarify Communicate Caution Informational Caution = Timely action may be needed. Routine notification through Lessons Learned and Operating Experience Weekly Summary emails and LANL Home Page. LANL managers determine relevance and take needed action. Informational = Future action may be needed. Posted to LANL Home Page if broadly applicable and actions clear. Important to knowledge base.

Enable Risk Management Risk Levels Differ from Site to Site Pre-Screening Post Screening # HSS - Green Informational 21 HSS - Blue Caution 4 44 HSS - Red 6 HSS - Yellow 38 65 Urgent Lessons most commonly associated with DOE Safety Alerts and Advisories, defective safety items, and other sources

Enable Risk Management Select Sources to Support Risk Management – Might Change over Time Primary Routine Sources Monitoring Frequency DOE HSS Server/Web (LL, Alerts, OE Summaries, Security Lessons Learned Center (SLLC)) Daily Partner Alerts (Bechtel, BWXT, UC, etc.) As Issued ORPS (from Other Sites) Suspect Counterfeit/Defective Items U.S. Chemical Safety Board Monthly Enforcement Actions and GAO Reports Quarterly Consumer Products (CSPC, FDA)

More Focused Equals More Effective Tailor Communication Methods to Audience and Media Capability

More Focused Equals More Effective Weekly Summary Email – Management and LL Roles with Prioritization Recipients Division Leaders and above NNSA Site Office LL Roles and Upon Request Augments daily ORPS Specific = Relevant Define Most Specific Customer Select Words in Headline Integrity Check Order (First and Last Most Read) Scan-able (6 most important/actionable; > 6 has lower read rate)

More Focused Equals More Effective LANL Homepage to Reach Workers Directly Some Communication Overlap Helps Effectiveness Main Users Workers Broadest lessons Includes worker home safety Specific = Relevant Even shorter headlines Pre-job brief is major customers

More Focused Equals More Effective Internal Lessons Learned Submission – Content over Format Adobe Fill-able Form Just the Facts Word Docs too slow, costly, hard to share with DOE Field Validation – reduce errors Fields support processing Screening and prioritizing Interface to DOE/HSS Includes picture objects Submit Button linked to LL Archive and Process Becoming standard at facility level

More Focused Equals More Effective LANL Mirror (Quarterly Publication) Translating Events into Actionable Information Events binned by systems and processes Patterns and trends (incidents, occurrences, causes) Year-End and Two-Year Running Reflection Use statistics to identify changes/trends High-leverage lessons/actions aligned with greatest performance gaps Recognition for developing and sharing lessons learned

More Focused Equals More Effective LANL Mirror: Events Causes Checklist for Improvements

Build and Use Indicators of LL Effectiveness Fact-based Improvement is the Objective Use CAS Dashboard LL Indicators Processing time performance Urgent within 24 hours Caution within 7 days Document Access Rate Search Rate Knowledge Inventory Internal LL Generation Rate LL Program Execution Routine Process Milestones Progress on Improvement Plan Corrective Actions

Build and Use Indicators of LL Effectiveness Need measures than can help understand Cause and Effect

Build and Use Indicators of LL Effectiveness Need measures than can help measure use for Work Planning

Encourage Innovation LANL Performance Communication Center Modernizing Communications Chat with an Expert Builds Knowledge Base Blog Role-based information Relevant Tools Relevant Requirements Best Practices (coming soon) Custom version of LL Form Recognition

Encourage Innovation Sub-Unit Approaches Evolve as their Learning Processes Mature R&D key lessons included in “On-Boarding” materials for new hires (might become a best practice) Local LL Archives Linked to Institutional Archive Project Management Engineering D&D Maintenance Facilities Learning Teams (linked to HPI implementation)

Learning About Learning Progress in Lessons Learned System at LANL Questions?