Results from Formal Review Process of the Guide on CSM-DT Workshop on CSM-DT, 29-30 November 2016 Dragan JOVICIC, EU Agency for Railways

Application Guide on Regulation 2015/1136 Formal Review Process 07/10/2016 – Formal Review Request of final draft of guide on CSM-DT to NSA network NRB network (CER, EIM, UNIFE, UIP, UIRR, UITP, ERFA, NB Rail, etc.) all CSM assessment bodies registered in ERADIS OTIF and CEN/CENELEC Interoperability and Safety Units + Lawyer within the Agency Formal Review Process with comment sheets Deadline 07/11/2016 8 comment sheets received  170 comments of different nature Major comments for discussion at workshop

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review Some parts of the guide deal with the overall risk assessment process and not specifically with the allocation of CSM-DT. [e.g. §2-5 not necessary for CSM DT] 30% guidance - 70% of document examples addressing application of CSM DT Suggestions: Demonstration of achieving CSM-DT for a specific case well described in EN standards  guide should focus on CSM-DT [e.g. delete §3.2.6 to §3.2.13] Less content improves understanding  shorten also example in Annex 3 The guide should address the actual understanding of CSM DT The examples should rather be collected in a general and separate document with all examples 1 Agency opinion CSM DT not separate from overall CSM RA process Flexible to change MATTER FOR DISCUSSION AT WORKSHOP

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review Figure 6 – Flowchart on choice among categories (a) and (b) Disagreement with argumentation given in the guide on Step 8 path For a “large number of people affected” at least one fatality is also possible Suggestions: if there are no multiple fatalities, the accident consequence shall be critical and not catastrophic  there must be a link from class (a) to class (b) the brackets around (multiple) in Step 8 of Figure 6 should be deleted 2 Agency opinion No connection between two branches In case of accident either: “a large number of people affected”, or “a very small number of people affected” otherwise wrong branch was entered Statistics of accidents on number of fatalities cannot be used MATTER FOR DISCUSSION AT WORKSHOP

Application Guide on Regulation 2015/1136 Choice of the appropriate severity class of CSM-DT

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review Table 6 – Only possible cases of CSM-DT vs. number of (affected persons; victims) Suggestions: Case 2 is class (b) instead of class (a), otherwise the differentiation of “large number of people affected” and “multiple fatalities” makes no sense Case 4: any class cannot be allocated  delete content 3 Agency opinion In case of accident either: “a large number of people affected”, or “a very small number of people affected” The whole population in the group can be credibly either injured or killed Statistics of accidents on number of fatalities cannot be used to choose among categories (a) or (b) MATTER FOR DISCUSSION AT WORKSHOP

Application Guide on Regulation 2015/1136 Only possible cases of CSM-DT vs. number of (affected persons; victims). 3 ? ?

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review Disagreement on non-use of statistics in terms of fatalities concerning accidents that occurred in the past – Expert judgement is fallible  better to use statistics Suggestions: Accident statistics provide a very reliable means of estimating severity, because the accident severity is independent of the causes of the accident An expert judgement of credible worst case may be based on statistical data Safeguard: experts must be aware of the limitations, otherwise Worst-Case judgements will always have to be used, which is not acceptable 4 Agency opinion (linked to next slide) Agency has feeling that some experts have same understanding as the Agency, but they do not agree with current wording in the guide other experts convinced that statistics from accidents in the past are usable to predict most credible unsafe outcome of a failure of Technical System under assessment  even not accepted by revised CENELEC 50126 MATTER FOR DISCUSSION AT WORKSHOP

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review “Most credible unsafe consequence of failure” in §2.5.5 of Reg. 2015/1136 Disagreement with explanations in sections §4.2.2(j) and§ 4.5.3 of guide It could lead to use systematically theoretical WORST CASE scenarios/consequences Suggestions: Proposed wording does not have same meaning in German as in English The term “most unsafe” is not part of the legal text and cannot thus be read as “credible most unsafe consequence of failure”  delete the term 5 Agency opinion Understands linguistic problems A common understanding is nevertheless necessary to ensure that based on expert judgement and not on statistics of number of fatalities of accidents from past: Category (b) shall not be used when (a) is expected Category (a) shall not be imposed when (b) sufficient MATTER FOR DISCUSSION AT WORKSHOP

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review Formula used in example of Annex 3 are different from what is generally used in the sector, e.g. EN 50129 – Should {Ref. 6} & {Ref. 7} (source of formulas) be used in an EU guide? Probably no available in English or German Suggestions: Correct formula or, explain and give reference document for formulas Impossible to reproduce the example based on information given 6 Agency opinion Use of scientific literature (i.e. CoP) not forbidden by CSM Calculations in draft guide correct because they use full “Detection plus Negation Time” instead of the mean time Same results both with CENELEC and guide formulas Formulas in revised CENELEC 50126 (same as in 50129) come from IEC 61165 standard which refers to {Ref.6} Formulas of draft guide do not match those in {Ref. 6} & {Ref. 7}  formulas must be corrected in final guide MATTER FOR DISCUSSION AT WORKSHOP

When Ti represents full “Detection plus Negation Time” Alain VILLEMEUR RAMS book, Eyrolles editions, on the “Reliability, Availability, Maintainability and Safety of complex industrial systems” Application Guide on Regulation 2015/1136 Major comments from Formal Review Formulas found in footnote of §A.4.2.2.1 in Appendix A of CENELEC 50129 standard 6 Formula in draft guide Where : (a) FR’s stand for potential hazardous Failure Rates of the basic events; (b) SDT stands for the safe down time; (c) SDR stands for the safe down rate, i.e. SDR=1/SDT; (A.1) may be used with Mean Test Times if periodic testing times are used as detection times for the failures. Then SDT = 1/SDR = T/2 + negation time When Ti represents full “Detection plus Negation Time” (i.e. not meant time)

Application Guide on Regulation 2015/1136 Verification of equivalence of Formulas in Villemeur and CENELEC 6

Application Guide on Regulation 2015/1136 Verification of equivalence of Formulas in Villemeur and CENELEC 6

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review Disagreement on free use of (CoP ; Ref. Syst. ; Explicit Risk Estimation) Isn’t there an order of priority among those 3 Risk Acceptance Principles? In some cases, it is mandatory to use explicit risk estimation. Suggestions: Correct (not identified) parts of the guide 7 MATTER FOR DISCUSSION AT WORKSHOP Agency opinion Unless requested by an EU (e.g. by a TSI) or a Notified National Rule, Reg. 402/2013 and Reg. 2015/1136 do not impose any order of priority  correct in guide “Without prejudice to mandatory compliance with applicable TSIs or NSR, the proposer can decide to use …” (idem flowcharts) If no EU/NSR rule, proposer free to chose Use of explicit risk estimation, of CSM DT or quantitative risk assessment is not obligatory

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review Use of CSM DT for pneumatic technical systems – Their mutual recognition Suggestions: Should be outside the scope of use of CSM DT Mutual recognition of pneumatic systems should be limited 8 MATTER FOR DISCUSSION AT WORKSHOP Agency opinion Regulation 2015/1136 does not exclude pneumatic technical systems from the scope of use of CSM DT Article 15(5) of Reg. 402/2013 sets conditions for Mutual Recognition Likely that Codes of Practice will be used for purely pneumatic systems For mixed systems (E/E/PE + Pneumatic), Hazard Identification to be extended to a higher level of detail so that CSM DT could be used for E/E/PE part whereas CoP would likely be used for purely pneumatic part Such a “recommendation” could be made in the guide

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review Agency example on trainborne Hot Box Detector deals with failures of a purely mechanical system (wheelset/axle box)  outside scope of CSM DT Suggestions: Delete entirely the example as it is not relevant for use of CSM DT Example in total contradiction with the guide itself Use of statistical data with unknown and unreliable origin 9 MATTER FOR DISCUSSION AT WORKSHOP Agency opinion Wrong understanding of Regulation 2015/1136 CSM DT not used for design of a mechanical system (wheelset/axle box) but for design of TS which will detect emerging risks & trigger safe actions Statistics not related to “good luck” circumstances but reflecting effectiveness of RU SMS [predeparture checks, periodic planned maintenance inspections and preventive maintenance operations, etc.]

MATTER FOR DISCUSSION AT WORKSHOP Application Guide on Regulation 2015/1136 Major comments from Formal Review All examples go beyond implementation of CSM DT Most of text on implementation of CSM for risk assessment in particular Annexes 3 & 4 but also Ex 1, Ex 2 and Ex 5 in Annex 5 Suggestions: Shorten strictly content to what is necessary for implementation of CSM DT Or delete the examples 10 MATTER FOR DISCUSSION AT WORKSHOP Agency opinion Regulation 2015/1136 is not a standalone text but amends Regulation 402/2013 For correct allocation of CSM DT, understanding of overall risk assessment and use of Technical System are crucial Allocation of CSM DT cannot be done separately from overall risk assessment