1. Safety Regulation Group FISA-2003 Slide 1 ATSSD SRG CAA (UK) Experience with Goal Based Regulations Andrew Eaton National Requirements & Strategy Specialist

2. Safety Regulation Group FISA-2003 Slide 2 Overview  What is the ATSSD SRG CAA (UK)  Our starting point  Why we decided to change to Goal based Regulations  Our understanding of Goal Based Regulations  Changes for the Regulator  Changes for the Regulatee  Lessons learnt  Outstanding Issues

3. Safety Regulation Group FISA-2003 Slide 3 What is the ATSSD SRG CAA (UK)  Civil Aviation Authority (United Kingdom)  Safety Regulation Group  Air Traffic Services Standards Department

4. Safety Regulation Group FISA-2003 Slide 4 Our starting point  Prescriptive regulations  ICAO standard based  Modified to include EU legislation  Mostly focused on interoperability  Enhanced by UK experiences  Inspection / Tick sheet driven

5. Safety Regulation Group FISA-2003 Slide 5 Why we decided to change to Goal based Regulations Prescriptive regulations proved to be: Driven by Robens & Pipa Alpha reports Incomplete / Inconsistent Slow to respond to new technology Obstructive to innovation Placing the responsibility for safety with the regulator

6. Safety Regulation Group FISA-2003 Slide 6 Our understanding of Goal Based Regulations  Are regulations that: set a state to be achieved without mandating a solution are traceable via a valid argument to a safety axiom Or are traceable via a valid argument to a legal safety requirement  Can be very detailed/prescriptive if traceability exists.

7. Safety Regulation Group FISA-2003 Slide 7 SW01 assurance goal “For arguments and assurance evidence to be available which show that the risks associated with deploying any software used in a safety related system are tolerably safe.” (CAP 670, SW01 Part 2 Section 3.)

8. Safety Regulation Group FISA-2003 Slide 8 Assurance goal decomposition This requires assurances to be made in the following areas:  Requirements Validity  Requirements Satisfaction  Non-Interference  Requirements Traceability  Configuration Consistency

9. Safety Regulation Group FISA-2003 Slide 9 1. Requirements Validity  To ensure that arguments and evidence are available which show that the Software Safety requirements correctly state what is necessary and sufficient to achieve tolerable safety, in the system context. (CAP 670, SW01 Pt 2 requirements.)

10. Safety Regulation Group FISA-2003 Slide 10 2. Requirements Satisfaction  To ensure that arguments and evidence are available, which shows that the software satisfies its safety requirements. (CAP 670, SW01 Pt 2 requirements.)

11. Safety Regulation Group FISA-2003 Slide 11 3. Requirements Traceability  To ensure that arguments and evidence are available which shows that all Safety Requirements can be traced to the same level of design at which their satisfaction is demonstrated. (CAP 670, SW01 Pt 2 requirements.)

12. Safety Regulation Group FISA-2003 Slide 12  To ensure that functions implemented as a result of Software Safety Requirements are not interfered with by other functions implemented in the software. (CAP 670, SW01 Pt 2 requirements.) 4. Freedom from interference by non safety functions

13. Safety Regulation Group FISA-2003 Slide 13 5. Configuration Consistency  To ensure that the arguments and evidence, for the safety of the software in the system context, are from: a known executable version of the software and a known set of software products, data and descriptions that have been used in the production of that version. (CAP 670, SW01 Pt 2 requirements.)

14. Safety Regulation Group FISA-2003 Slide 14 Underlying model of SW01

15. Safety Regulation Group FISA-2003 Slide 15 What SW01 is  Guidance on setting credible success criteria for judging the achievement of the regulatory objectives.  Process independent.  Lifecycle independent.  Risk focused.

16. Safety Regulation Group FISA-2003 Slide 16 What SW01 isn’t  It is not a software assurance standard.  It does not define a software development process.  It does not define a software assurance process to follow.  It does not reason how your assurance evidence satisfies a claim.

17. Safety Regulation Group FISA-2003 Slide 17 Challenges in drafting Goal based regulations  Level of detail (tend to be verbose)  Level of abstraction of the Goal Prolific low level goals are difficult to manage Low level goals imply regulatory risk if wrong High level goals risk miscomprehension  Defining success criteria not solutions  Providing guidance which can not be taken to mandatory

18. Safety Regulation Group FISA-2003 Slide 18 Goal Based Safety Cases  Claim trees for each Goal with Arguments & Evidence  Argument design is key Top down (argument driven), Middle out (standard driven), Bottom up (evidence driven) Bespoke (argument drives design), COTS (argument justifies design) Different arguments for different system components  GSN strongly preferred  Evidence filtering – pertinence to argument

19. Safety Regulation Group FISA-2003 Slide 19 Changes for the Regulator  Converting Inspectors to Auditors Evaluation of arguments Checking validity of evidence  Managing the diversity of solutions Demands on technical knowledge Diversity of safety arguments

20. Safety Regulation Group FISA-2003 Slide 20 Changes for the Regulatee  Inability to let contracts against a standard  Argument construction  Pertinence of evidence  Availability of resources / skills / knowledge  Additional workload in evaluation design options – the downside of freedom of solution

21. Safety Regulation Group FISA-2003 Slide 21 Outstanding Issues  Conformance/Type Assessment  Resources / Skills / Knowledge required during transition  Level of argumentation  The drive to make guidance mandatory  The process by which Goals become Prescriptive  The optimum level of Goal

22. Safety Regulation Group FISA-2003 Slide 22 Lessons learnt  Ensure that your understanding of a “Goal Based Regulation” is well defined  Transition needs to be planned & managed  Implement the transition top down  Be prepared to give guidance / assistance  Regulatory independence can be compromised during the transition  Expect resistance to change

23. Safety Regulation Group FISA-2003 Slide 23 Questions ?

24. Safety Regulation Group FISA-2003 Slide 24 Andrew Eaton  ATSSD, 2W, Aviation House, Gatwick Airport South, West Sussex, RH6 0YR.  01293 573504  andrew.eaton@srg.caa.co.uk andrew.eaton@srg.caa.co.uk  www.caa.co.uk