1. `+
Confidential Close Call Reporting System (C3RS) Final Lessons Learned Results
C3RS User Group April 11, 2017
Dr. Joyce Ranney Volpe National Transportation Systems Center
Cassandra Cantu, M.S.
Volpe National Transportation Systems Center
Dr. Maryam Allahyar
Federal Railroad Administration
Melinda Davey, M.S.E. Jacobs
Dr. Jonny Morell
Syntek Technologies
Volpe The National Transportation Systems Center
Advancing transportation innovation for the public good

2. Introduction and Methods General C3RS Information Results and Findings
Agenda
Introduction and Methods
General C3RS Information
Results and Findings
Implementation
Impact
Sustainability
Recommendations
Next Steps

3. C3RS Evaluation Questions
Implementation
What are the characteristics of a successful C3RS implementation?
Impact
What is the impact of C3RS on safety and safety culture?
Sustainability
What are the conditions needed to make C3RS sustainable?

4. What is the Data Collection Method for the C3RS Evaluation?

5. Evaluation Status UP North Platte CP Central NJT Amtrak (West)
Baseline
Midterm
Final UP
North Platte X CP
Central
NJT
Amtrak (West)
Cross-Site Reports
In Reviews
In Process
TODAY

6. Introduction and Methods General C3RS Information Results and Findings
Agenda
Introduction and Methods
General C3RS Information
Results and Findings
Implementation
Impact
Sustainability
Recommendations
Next Steps

7. Volume of C3RS Reports
Monthly Average Number of Reports per 100 Eligible Employees
All four sites received enough reports to generate corrective actions

8. Overall Categories Across Demonstrations:
Excess Speed and Switching Most Reported Categories
Percent of Cases

9. Distribution of Reported Categories Varies Across Sites
Three of four sites reported excess speed
All sites reported switching

10. More C3RS Cases are Unknown than Known Events
A Close Call Event is a specific situation or occurrence, observed by a railroad employee, that has a potential for more serious consequences
Unknown 56%
Known
44%
Reporters Unknown
Event reported to C3RS and identity remains unknown
56%
Reporters Known
Minor incidents reported to C3RS and communicated directly to the railroad
44%
*Sites 1, 2, 4

11. Corrective Action Examples
Category
Action Site
Action Site
Action Site
Excess Speed
Reformatting track bulletins 1 2 3
Reminder tags
Modified signs
1 2 4
Consolidating slow orders 4
Dispatcher process improvements
Switching
Improved communications
Improved switch visibility 12
Training
Poster about problem switches
Derailment
Improved communications in yard (e.g. squawk box)
Visual checks (e.g. camera, checking with yardmaster) 13
New policy about derail removal
Collision
Safety bulletin
Barriers to prevent pedestrian crossing
Communication improvements
Contractor accountability
Blue-Flag
Maintenance procedure for flag removal
Improve maintenance of blue lights
General
Improved job safety briefings
Departure checklist hang tag

12. Introduction and Methods General C3RS Information Results and Findings
Agenda
Introduction and Methods
General C3RS Information
Results and Findings
Implementation
Impact
Sustainability
Recommendations
Next Steps

13. Overall Summary: How Far Did Each Site Progress?
(Site Names Disguised)
Implementation
Successful C3RS implementation
Impact
Impact on safety and safety culture
Sustainability
Conditions to make C3RS sustainable
Implementation
Impact
Sustainability
Site 1 
Site 2
Site 3
Site 4

14. Implementation Findings
Question
Answer
What are the characteristics of a successful C3RS implementation?
Implementing C3RS as planned is possible within transportation departments in the railroad industry. While the demonstrations had varying degrees of implementation strength, each of them performed sufficient planned activities to justify an evaluation of their impacts
High reporting rates were not associated with implementation strength, impacts, or sustainability
Sites with strong implementation demonstrated significant impacts and had common characteristics
Sites identified similar improvements for implementation

15. Implementation Finding
While the demonstrations had varying degrees of implementation strength, each of them performed sufficient planned activities to justify an evaluation of their impacts.
All demonstrations’ reporters submitted and PRTs received reports from the third party
Demonstration Sites Implemented C3RS as Planned
Third parties collected reports and provided them to carriers
Labor, management, FRA participated at the carrier:
Prioritized, grouped reports
Applied problem analysis methodology
Considered possible corrective actions
Chose corrective actions to recommend
Recommended corrective actions to management
Implemented corrective actions
Monthly Average Number of Reports per 100 Eligible Employees

16.  Implementation Finding
High reporting rates were not associated with implementation strength, impacts, or sustainability.
Monthly Average Number of Reports per 100 Eligible Employees
Implementation
Impact
Sustainability
Site 1 
Site 2
Site 3
Site 4

17. Implementation Finding
Sites with strong implementation had common characteristics.
Carrier: local and senior management provide resources to the PRT and implement corrective actions
Labor: promote detailed reporting, participate in PRT, implement corrective actions
Stakeholders cooperate:
Negotiate IMOU, maintain confidentiality, and support training
Implement and monitor corrective actions
Resolve disputes in a way that encourages continued future reporting
All stakeholders agree that C3RS is valuable

18. Cause and Effect Diagram
Main Factor
A cause and effect diagram was created based on the evaluation results
Each site was rated on each detailed factor
Ratings were compared among sites
Detailed Factor
Detailed Factor
Effect
Detailed Factor
Detailed Factor
Detailed Factor
Main Factor
Diagrams and detailed ratings for each site will be available in final report

19. Successfully Implemented C3RS
Implementation Factors Model
Successfully Implemented C3RS

20. Implementation Finding
Sites with strong implementation demonstrated significant impacts.
Site 1
Site 2
Site 3
Site 4
The average rating for each of the main implementation factors for each site is shown above.
You can see that Site 4 had the poorest implementation which led to low impact.
Sites 1, 2, and 3 have more good and very good ratings which led to good impact. .

21. Implementation Finding
Sites identified similar improvements for implementation.
Communication:
Support Team to PRT: status of corrective actions
PRT and Support Team to workforce and managers wanted: C3RS accomplishments
PRT Data Analysis and Efficiency:
Analysis was done one case at a time, so the PRTs got behind
Support Team wants more data on corrective action analysis to help with prioritization
Process improvement experts in company not connected with PRT
PRT members had to sit and wait for administrative tasks
Tracking of Corrective Actions:
Some corrective actions not recorded
Status unknown
Impact and effectiveness not measured

22. Impact Findings
Question
Answer
What is the impact of C3RS on safety and safety culture?
C3RS has demonstrated bottom-line impacts in 3 of the 4 sites in derailments, injuries, discipline hearings, equipment costs, and safety culture
Derailments decreased at three sites in the presence of C3RS (1) corrective actions and (2) communication enabled by discipline protection
Injuries decreased at one site in the presence of corrective actions such as improved job safety briefings
Discipline hearings and associated costs decreased in the presence of C3RS protection
Labor perceived that safety culture between supervisors and labor improved in the presence of C3RS-enabled communication

23. Impact Finding: Derailments
Derailments decreased at three sites in the presence of C3RS
(1) corrective actions and (2) communication enabled by discipline protection.
Derailments
 41%
 30%
 20%
 --
Site 1
Site 2
Site 3
Site 4
Legend
 = Reductions (improvement)
-- = mixed results/ little to no change
% of Derailment/ Switching cases
25% of cases on average across all sites
Interview Quotes
“Now if they have a run-though switch, they call it in and it is repaired, so a derailment is not caused. I see less equipment damage caused.” (Site 2)
Corrective Actions
Improved yard communication, policy about derail removal, reverse move training etc.

24. Impact Finding: Injuries
Injuries decreased at one site in the presence of corrective actions such as improved job safety briefings.
Transportation
Personal Injuries ?
 18% --
Site 1
Site 2
Site 3
Site 4
Legend
 = Reductions (improvement)
-- = mixed results/ little to no change
? = no data
Interview Quotes
“When you experience a mistake, you learn and become more cautious.” (Site 2)
Corrective Actions
Redesigned Job Safety Briefing procedure, trained workforce, used extensively (Site 2)

25. Impact Finding: Discipline Hearing Cost
Discipline hearings and associated costs decreased in the presence C3RS protection.
Disciplinary Hearings
 90%  ?
 39%
Site 1
Site 2
Site 3
Site 4
Legend
 = Reductions (improvement)
? = no data
Interview Quotes
“C3RS protection helps us avoid discipline hearings.” (Site 1)
“A discipline hearing cost approximately $10,000 each in travel and work time.” (Site 3)
“Reducing discipline in the dispatching office helped us find an error in the software and justified a budget to fix it.” (Site 3)

26. Safety Culture Survey*
Impact Finding: Safety Culture
Labor perceived that safety culture between supervisors and labor improved in the presence of C3RS-enabled communication.
Interviews:
Labor: “Definite impact on communication between labor and management. In the past, if you messed up, you would bury it.”
Manager: “My workers feel more comfortable approaching me.”
Safety Culture Survey*
Site 1
Site 2
Site 3
Site 4
Safety Culture
Organization
Supervisor
Coworker  -
Safe Behaviors ?
Legend
 = Improvement
- = mixed results/ little to no change
? = no data
* labor participants

27. Sustainability Findings
Question
Answer
What are the conditions needed to make C3RS sustainable?
C3RS is sustainable in railroad transportation when enlisting the support of labor, management, and FRA.
Strong implementation, impact, and agreement between stakeholders are needed for a site to be sustainable
FRA has taken steps to support sustainability in the railroad industry and has expanded the program

28. Sustainability Finding: Demonstration Sites
Strong implementation, impact, and agreement on path forward between stakeholders are needed for a site to be sustainable.
Implementation:
Sustained sites had on average “good” implementation
Impacts:
Sustained Sites had safety and other positive impacts
Agreement on Future:
Sustained sites had agreement among stakeholders on a path forward with FRA and a continued focus on improving the C3RS process 1 2 3 4

29. Office of Railroad Policy and Development
Sustainability Finding: Industry
FRA has taken steps to support sustainability in the railroad industry
and has expanded the program
FRA actions:
FRA funded a national program and established an office to support C3RS
NASA was selected as the third party
Evaluation Results:
FRA used impact findings to:
Obtain FRA/Congress funding
Motivate industry to participate
New Railroads:
Five more railroads joined C3RS, including new crafts
Challenges for future:
Class 1’s are not participating at this time
$$$
Human
Performance
Division
$$$
Office of Railroad Policy and Development
Derailments 41%
Injuries 18%
Discipline Hearings 90%
Safety Culture
Class 1

30. Summary of Findings
Implementation
Implementing C3RS is possible within the railroad industry. For implementation to be strong, need commitment and involvement by labor and management.
Impact
C3RS can demonstrate bottom-line impacts on safety and safety culture, if there is strong implementation.
Sustainability
C3RS is sustainable in the railroad industry with strong implementation by carriers and continued support by FRA.

31. Introduction and Methods General C3RS Information Results and Findings
Agenda
Introduction and Methods
General C3RS Information
Results and Findings
Implementation
Impact
Sustainability
Recommendations
Next Steps

32. Recommendations Labor Management FRA NASA
Encourage detailed reports, not just frequent reports
Participate in corrective action implementation
Management
Resolve disputes in a way that encourages continued future reporting
Monitor impact of corrective actions to determine if effective
Determine impact measures based on details of corrective actions
FRA
Determine ways to measure impact in the future to ensure continued justification and funding for the program (Lessons Learned effort ending)
NASA
Consider ways to help sites share corrective actions with the industry

33. Introduction and Methods General C3RS Information Results and Findings
Agenda
Introduction and Methods
General C3RS Information
Results and Findings
Implementation
Impact
Sustainability
Recommendations
Next Steps

34. Prepare and submit Final Report for publication TRB
Next Steps for Lessons Learned Team
Prepare and submit Final Report for publication
TRB
Article in Transportation Research Record
Presentation at 2018 TRB Annual Meeting
Articles in publication process at FRA
C3RS Lessons Learned Midterm Report
Research Result: "How an Active C3RS Program Can Impact Safety Results“
Research Result: "C3RS Reporting Trends and Implementation“
Research Result: “C3RS Demonstration Pilot Uncovers Dangerous Software Glitch and Saves $890K in Costs for Disciplinary Hearings”

35. For More Information – FRA eLibrary
publications so far
The Lessons Learned team has been creating publications for FRA’s website.
There are 10 so far, and more are coming over the next couple of years.
To find them:
Go to the FRA eLibrary (the link on far right of the top menu)
Type c3rs into the search field.
Select Research Results and Technical Reports to filter the results 35

36. Volpe The National Transportation Systems Center`+
Contact Information
Dr. Joyce Ranney
(617)
Volpe National Transportation Systems Center
Cassandra Cantu, M.S.
Melinda Davey, M.S.E.
(734)
Jacobs
Dr. Jonny Morell
Syntek Technologies
Volpe The National Transportation Systems Center
Advancing transportation innovation for the public good