1. Technical Program Delivery Group
international Engineering Safety Management Independent Assessment Accreditation
Paul Cheeseman
Technical Program Delivery Group
© TPD 2017

2. iESM - Aim www.intesm.org
To assist the international railway industry in delivering products/systems with acceptable levels of safety by developing & sharing good practice in railway Engineering Safety Management worldwide.
Developed as part of the TPD internal research activities, for the good of the rail industry, supported by MTR Corporation.
TPD, having unique access to the Yellow Book originators, former YB Steering Group members and the editor, decided to address the international need with a new handbook. With the kind support of MTR Corporation we have been able to establish a new Working Group to steward and take forward ESM for a wider audience.
The primary purpose of international Engineering Safety Management (iESM) is to help people who lead and undertake railway engineering make sure that their work contributes efficiently to improved safety and helps new railways and changes to be accepted more efficiently.
The key strengths are “all in one place” and “good practice, not mandated standard”.
© TPD 2017

3. AN 4 Independent Assessment v
“Your organization must ensure that engineering safety management activities are reviewed by competent people who are not involved with the activities concerned.”
Provides more detailed guidance on iESM Guidance, Volume 2, Chapter 17 – how to do it
Aimed at Independent Assessment organizations / Independent Assessors
Useful for railway administrations, Project Managers and Project Safety Managers
iESM
Principle
These reviews may be divided between process-focused reviews (often referred to as safety audits) to check how things are being done and product- focused reviews (often referred to as independent assessments) to check what is being produced. Audits provide evidence that you are following your plans for safety. Assessments provide evidence that you are meeting your safety requirements. How often and how thoroughly each type of review is carried out, and the degree of independence of the reviewer, will depend on the extent of the risk and novelty and on how complicated the work is.
If an ESM activity is done many times, it may be better to specify it precisely and review the specification rather than the activities themselves. You will still need to confirm that you comply with the specification each time you carry out the activity but this compliance will not be subject to independent assessment.
© TPD 2017

4. “Quis custodiet ipsos custodes? ”
“Who will guard the guards themselves?“ – Juvenal (Satires)
“Who watches the watchmen“ – Star Trek
“There are checks and balances in science. There’s somebody checking the people doing the science and then there is somebody who checks the checkers and somebody who checks the checker’s checkers.” – Michael Shermer
EXCEPT THERE ISN’ T - Who assesses the independent assessors? – as far as we can tell no one!
Surely a gap in the safety assurance arrangements for clients and suppliers alike?
© TPD 2017

5. Existing safety-related schemes
Mandatory accreditation schemes for:
Assessment Bodies (under the Common Safety Methods for Risk Assessment)
Notified Bodies (under Interoperability Regulations)
Design Bodes (under National Notified Technical Rules)
Client specified schemes such as IRSE Signaling licensing
But nothing for Independent Assessment (apart from generic audit training)
Whilst similar, none of these covers the risk-based independent safety assessment role and all of them bring a significant overhead of accreditation assurance.
© TPD 2017

6. Assessment or Certification?
Risk-based:
“In this situation, with these assumptions, caveats and dependencies this is adequate”
A judgement based on professional opinion supported by objective evidence of process, inspection of output and compliance with standards
Compliance-based:
“This complies with the specified standard”
A statement of fact where no judgement, risk assessment or test of reasonable practicability is necessary
© TPD 2017

7. Life saving second opinions
Primary purpose of independent assessment is provide confidence that the product or application under consideration that the risk is controlled to an acceptable level.
A secondary purpose is to improve the ESM evidence and / or its presentation.
Not about seeking a single, elusive “right answer”
© TPD 2017

8. Emerging CENELEC requirement
The emerging draft of new CENELEC EN includes a set of key competences for an Assessor including having an “acceptance/license” from a recognized safety authority.
The standard offers no further guidance on what is expected or how it may be achieved……
© TPD 2017

9. CENELEC Requirements for an AssessorNo
Requirement 1
Be competent in the domain/technologies where independent assessment is carried out 2
Have acceptance / license from a recognized safety authority 3
Have / strive to continually gain sufficient levels of experience in the safety principles and the application of the principles within the application domain 4
Be competent to check that a suitable method or combination of methods in a given context have been applied 5
Be competent in understanding the relevant safety, human resource, technical and quality management processes in fulfilling the requirements of the EN 50126 6
Be competent in independent assessment approaches/methodologies 7
Have analytical thinking ability and good observation skills 8
Be capable of combining different sources and types of evidence and synthesize an overall view about fitness for purpose or constraints and limitations on application 9
Have an understanding of the overall system including its application environment 10
Understand the requirements of EN 50126
© TPD 2017

10. Additional iESM requirements11
Professional status in an engineering or scientific discipline relevant to the system or equipment 12
Prior experience as an independent assessor or safety engineer for a minimum of 5 years in areas relevant to the system or equipment 13
A commitment to safety 14
The flexibility to adapt to changing circumstances and the perform assessment tasks efficiently and to minimize wastage of physical and virtual resources
© TPD 2017

11. iESM Approach
Professional
Registration
Behavioural competence – including for example maintaining independence
Technical competence - e.g. safety engineering and risk acceptance techniques, assessment and audit, general consultancy skills
Knowledge – legal requirements, domain, system, technology, specialist areas
iESM training, self
assessment with
“Responsible
Manager”
The intent here is not to repeat other schemes and existing competence assessments
CV, company
processes
© TPD 2017

12. Independent Assessment
Scope & Content Appreciation
Assessment Strategy Selection and Planning
Reviewing Safety Documentation
Assessing Safety Analysis
Producing assessment reports including formation of an overall judgment on the safety of a product or system, or process used.
Managing Outcomes
(excluding audit)
© TPD 2017

13. Evidence of competence
“Have you done it?”
Or if not:
A demonstration that the Assessor would be able to apply the competency correctly in a hypothetical situation (can be applied);
The ability to answer questions pertaining to the competency based on past experience (can be tested);
Evidence of having been trained in that particular competency (has been shown).
© TPD 2017

14. Levels of competence
Level 1 is the base level which indicates that the Assessor has sufficient knowledge and understanding of good practice to be able to work on assessment tasks under supervision. Their competencies are likely to have been developed through targeted training and work on non-assessment projects.
At Level 2, the Assessor has sufficient knowledge and understanding of good practice and has sufficiently demonstrable experience, to be able to work on the tasks associated with the overall activity without the need for detailed supervision.
At Level 3, the Assessor has sufficient understanding of why things are done in certain ways, and sufficiently demonstrable skills, to be able to undertake overall responsibility for leading a task or project. A Level Three Assessor will be familiar with the ways in which systems or products have failed in the past. They are able to deal with a multiplicity of problems under pressure without jeopardising safety or quality issues.
© TPD 2017

15. Summary
New iESM Accreditation scheme is available for individuals and independent assessment service providers.
Aims to be “light-touch” but give substantial credibility to the Assessors who achieve the mark through a combination of training, experience and demonstrable competence.
Piloted in Australia and should fill a gap in the assurance arrangements for clients and suppliers.
Supported by iESM and IA training courses from TPD and their partners
Refinements will be made as experience grows and in response to emerging requirements from the international standards.
© TPD 2017