1. Safety Management Systems Session Three Safety Assurance
APTA Webinar
May 31, 2016

2. 4 Components (Pillars) of SMS
Safety
Management
Policy
Safety
Risk
Management
Safety
Assurance
Safety
Promotion

3. Safety Assurance
Processes used to ensure safety risk controls developed under the safety risk management process achieve their intended objectives throughout the life cycle of a system. These processes may reveal hazards not previously identified and identify or assess the need for new risk controls, as well as the need to eliminate or modify existing controls
Evaluates whether the transit agency implements appropriate and effective mitigations
Continuous improvement process that monitors and self-corrects the safety performance of the agency

4. Safety Assurance Sub-Components
Safety Performance Monitoring and Measurement
Management of Change
Continuous Improvement

5. Decision-Making What gets measured gets done
If you don’t measure results, you can’t tell success from failure
If you can’t see success, you can’t reward it
If you can’t reward success, you are probably rewarding failure
If you can’t see success, you can’t learn from it
If you can’t recognize failure, you can’t correct it
If you can demonstrate results, you can win public support
Osborne & Gabler, Reinventing Government

6. Safety Performance Monitoring and Measurement
Focuses on monitoring the safety performance of the transit agency through routine measuring of operations and maintenance activities
Senior management needs this data to evaluate whether:
Implemented safety risk mitigations are appropriate and effective
How well the agency’s safety performance matches established safety objectives and safety performance targets

7. Performance Targets
Acceptable/desired level of a metric to be accomplished by a future date or over a period of elapsed time
Reduce the rate of fatalities to 0.25 per 100 million passenger-miles by 2017
Measurable, acceptable to stakeholders, consistent with targets set by state and federal organizations

8. Transit Safety Stakeholders
Traveling public
Transit properties and service providers
Federal transportation organizations (i.e. FTA, FRA)
State and local transportation agencies and oversight organizations (i.e. SSOA)
Industry associations (i.e. APTA)
Investigative agencies (i.e. NTSB)
Transit systems planners and designers
Transportation research organizations (i.e. TRB)
Equipment manufacturers

9. Characteristics of an Effective Safety Performance Monitoring System
Stakeholder involvement and acceptance
Focus on agency goals and activities
Clarity and precision
Credibility and robustness
Variety of measures
Number of measures
Hierarchy of measures
Forward looking measures
Integration into decision-making
Timely reporting
In context
Realism of goals and targets

10. Getting Started- Safety Performance Monitoring System
Conduct readiness assessment
Agree on outcomes and activities
Select key metrics
Identify data needs
Pilot test and collect baseline data on metrics
Set targets
Monitor performance and evaluate results
Report findings
Integrate findings into agency decision-making
Sustain the performance measurement system

11. Safety Performance Monitoring Activities
Monitor employee safety reporting programs
Monitor service delivery activities (including field observations)
Safety data acquisition and analysis
Accident/incident reporting and investigation
Infrastructure maintenance and inspection
Vehicle maintenance, inspection and repair
Rail corridor safety
Passenger operational environment

12. Safety Performance Monitoring Activities
Monitor operational and maintenance data
Conduct safety surveys
Conduct safety audits, studies, reviews and inspections
Conduct safety investigations
Evaluate data and information from external agencies or peers

13. Unsafe Acts Operator Errors Decision Errors – honest mistakes
Procedural
Choices
Problem solving
Skill-based Errors – little or no conscious thought
Inadvertent activation/deactivation of switches
Forgotten intentions
Omitted items in checklist
Due to stress, fatigue, in a hurry
Perpetual Errors
When sensory input is degraded
Misjudge distance, speed, objects
James Reason

14. Violations Routine Exceptional
Habitual – i.e. drive 5-10 mph over speed limit
Tolerated by management
Exceptional
Departing from authority – i.e. 100 mph in 55 mph zone
Unacceptable to management

15. Potential Contributing Factors Leading to Unsafe Acts
Environment
Physical
Heat, lighting, vibration, noise, toxins, weather
Technological
Design of equipment and controls
Display/interface
Checklist layout
Automation

16. Potential Contributing Factors Leading to Unsafe Acts
Operator Conditions
Adverse mental states
Fatigue
Stress
Overconfidence
Adverse physiological states
Illness
Alcohol
Drugs
Physical and/or mental limitations
Reaction time
Vision

17. Potential Contributing Factors Leading to Unsafe Acts
Management of Operations Personnel
Failures of communication among operators
Failures of communication between operators and central control/maintenance
Personal Readiness
Disregard of rules
Sleep requirements
Drug restrictions

18. Potential Contributing Factors Leading to Unsafe Acts – Organizational Influences
Organizational climate: prevailing vision with the organization, including policies, command structure and culture
Operational process: formal process by which the organization’s vision is implemented including operations, procedures, oversight
Resource management: how human, monetary and equipment resources are managed

19. Potential Contributing Factors Leading to Unsafe Acts – Unsafe Supervision
Inadequate supervision: oversight and management of personnel and resources- training, professional guidance, operational leadership
Planned inappropriate operations: management and assignment of work – risk management, crew pairing, operational tempo
Failed to correct known problems: supervisor knows deficiencies among individuals, equipment, training or other safety areas but does not correct them
Supervisory violations: willful disregard of existing rules, regulations, instructions or SOPs by managers

20. Leading Indicators
Input based measures that have an indirect relationship to an end product or goal and can influence lagging indicators
Measure and track performance before a problem arises
Examples
Running red signals
Risk or hazard assessments and job hazard analysis
Railway worker protection violation
Use of cell phones while operating vehicles or machinery
Complaints per 100,000 passengers
Workplace inspections
Audits
Percentage of safety training completed vs. expected
Near misses
Reported anomalies

21. Lagging Indicators
Outcome-based measures that are directly related to an end product or goal
Measure performance against prior goals
Examples
Fatal accidents per specified time period
Number of preventable accidents per specified time period
Employee work days lost to injuries per specified time period
Workers compensation payments per specified time period
OSHA citations (number and type)
Workers’ compensation claims (trends and amounts)

22. Safety Performance Measures
Fatalities per time period
Injuries per time period
Fatal accidents per time period
Injury accidents per time period
Property damage accidents per time period

23. Safety Performance Measures
Work day lost to injuries by type
Work related fatalities
Workers compensation payments

24. Safety Performance Measures
Percent positive drug/alcohol tests
Number rail station overruns
Number of red signal violations
Number of traffic tickets issued to bus operators
Number of anomalies reported

25. Safety Performance Measures
Number of safety improvement ideas submitted by employees per quarter
Number of employees attending weekly safety meetings
Number of close calls reported per month

26. Safety Performance Measures
Number vehicle defects reported by operators
Number of infrastructure defects reported by operators
Number of fires by location
Customer satisfaction surveys
Percent preventive maintenance inspections competed within 10% of scheduled mileage
Percent employees scoring greater than 85% on proficiency tests

27. Data Sources and Data Management
Identify data elements
Identify data source
Develop data collection procedures and instruments
Identify the data owner/manager of the data
Develop a schedule for updating data

28. Management of Change
Configuration Management- process that ensures that all property, equipment, system design elements are documented as to configuration, both accurately and completely. Any changes to individual subsystems or fleet/inventory wide changes should be recorded on as-built drawings, addressed in training courses, maintenance guides and procedures
Standards/Design Control
System Modification- New Processes/Equipment
Document Control

29. Management of Change- Safety and Security Certification
Series of processes that collectively verify the safety and readiness of a project for public use
Identification and assessment of hazards/vulnerabilities along with resolutions
Codes, guidelines and standards related to safety and security are included in design criteria
Drawings and specifications are in conformance with design criteria
Facilities, systems, equipment are designed, constructed, built, inspected and tested in accordance with specifications, codes, standards and criteria
Tests, plan and rulebooks are developed for operations
Personnel are trained to handle normal and emergency situations
Emergency response personnel are familiar with the transit system and emergency procedures

30. Continuous Improvement
Agree on outcomes critical to safety performance
Select metrics for monitoring outcomes
Gather baseline data on current conditions
Set specific targets and dates for reaching these targets
Collect data on regular basis to assess whether the targets are being met
Analyze performance data and report results
Integrate performance results in decision-making

31. Continuous Improvement- Safety Assessment
Internal Safety Management Reviews
Follow formal written process
Determine compliance with safety policies, rules, regulations, codes, procedures, assigned safety activities and requirements
External Safety Management Reviews
Compliance to policies, plans, procedures, milestones or other requirements
Performance reviews focus on the product, process and system to determine how well they meet the customer’s needs and identify areas for improvement
Peer Reviews to assess all aspects of transit operations and functions

32. Continuous Improvement- Safety Culture Assessment
Employee surveys
Written questionnaires
Face to face interviews
Focus group interviews
Ability of agency to focus on long term performance
How the agency handles conflicts
How the agency views errors and mistakes
How the organization focuses on improving safety defenses instead of assigning blame
Agency’s proactive stance toward safety