1. An academic view of Human Factors in Maintenance Simon Place Danny Jayakody Hamad Rashid RAeS conference 8 October 2008

2. A limited selection of recent research 1.Maintenance error – Proposal for Error Management in Numerical Airworthiness Terms [Simmons, 2002] 2.Maintenance error prediction modelling [Leach, 2005] 3.A critical analysis of the links between the taxonomies used by MEDA & HFACS-ME [Eshati, 2006] 4.Aviation Maintenance Monitoring Process (AMMP) [Rashid] 5.Risk Assessment in Continuing Airworthiness of Air Transport [Jayakody]

3. Error Management in Numerical Airworthiness Terms [Simmons, 2002] Error Criticality Indices (ECI) 0.00 1.00 ECI 10E-310E-610E-9 Probability MinorMajor Catastrophic Effect on aircraft Estimate the criticality of a maintenance task, based on: –Effect if left uncorrected at Release To Service (RTS) –The severity of that effect –Where in the maintenance sequence it occurs –Whether an adverse outcome has been anticipated in design

4. Error Management in Numerical Airworthiness Terms [Simmons, 2002] Compare the criticality against the probability of error derived from existing Human reliability tools. Identify any required changes needed to meet an acceptable risk Assessment of maintenance tasks Data from Human reliability analysis Difference calculation No action Introduce new controls / re-design Consider cost savings Evaluate ECI Modify with Performance shaping factors

5. Risk factor = likelihood x severity x detection Risk Factor Analysis [Leach, 2005] Expert System of Engineers (ES) analyse maintenance tasks: Likelihood of Occurrence – range from 1 (almost impossible) to 16 (almost certain) Event severity – range from 2 (not noticeable) to 24 (probable aircraft loss) Detection – range from 1 (easy to detect) to 16 (difficult) Combination of Failure modes and effects analysis (FMEA) and MSG-3

6. Risk Factor Analysis [Leach, 2005] WARNING: Flight safety possibly endangered, additional safety nets are required WARNING: Flight safety possibly endangered, additional safety nets are required MODERATE: Flight safety is not endangered, but financial penalties often exist STANDARD: existing control measures are probably sufficient 1740 550 Insufficient control measures currently applied Model proved to be effective at identifying key events - risk factor between 2 and 6144

7. To apply HFACS-ME and MEDA human factor taxonomies to the same incident and accident reports to identify the causal contributing factors of the errors. To identify the applicability, effectiveness and limitations of both tools (MEDA and HFACS-ME). Links between MEDA and HFACS-ME [Eshati, 2006]

8. Maintainer Acts Management Conditions Organizational Supervisory Working Conditions Environment Equipment Workspace Maintainer Conditions Maintainer Conditions Readiness Crew Coordination Medical Violation Error ACCIDENT HFACS - Maintenance Extension Source: Naval Safety Center - School Of Aviation Safety

9. Application of modelling options (Source: Rashid / Jayakody) This research manages maintenance errors at manufacturer (aircraft / task) and immediate workplace levels. Rotary-wing aircraft are taken as case study.

10. Aviation Maintenance Monitoring Process (AMMP) To identify Root Causes of human errors in helicopter maintenance for both individual and organizational levels. To conduct retrospective and prospective analysis, under the HERMES [Cacciabue, 2004] methodology Retrospective HFACS-ME applied to study 58 reports of incidents and accidents related to maintenance To verify integrity of the developed process within helicopter maintenance industry.

11. ANP Model Fuzzy Analytical Network Processes Model  Calculating potentiality of maintenance error for a given task (x).

12. Fuzzy ANP [Chang 1992, 1996, adapted by Rashid] Prospective: ANP weighs task errors potentialities from both designing and immediate workplace variables concerning that task. Direct product: Weighting error potentiality of maintenance tasks and subtasks. Error risk = f (Potentiality, Criticality).

13. Fuzzy Comparison Matrices PrepRemovalChecksAnalysisInstallationFunction Test Preparation 1,1,11/x, 1/y, 1/z Removal x, y, z1,1,1 Checks 1.1.1 Analysis 1.1.1 Installation 1.1.1 Function Test 1,1,1 Pair-wise comparison of importance for tasks within a given maintenance job Overall output: Full detailed risk ranking of maintenance errors for a given task (changing the main rotor of a given helicopter).

14. Risk Assessment in Continuing Airworthiness Aims of research 2007-2010 1.To design and develop a generic model for risk assessment in continuing airworthiness process 2.To determine a method to optimise regulatory oversight programme on the basis of risk 3.To determine if parts of regulatory oversight inspections could be devolved 4.To determine how expert opinion could be represented as a measurable parameter in a decision analysis tool [Jayakody]

15. Risk Assessment in Continuing Airworthiness Risk-based oversight (RBO) from Hampton Report recommendation (leading to “Better Regulation”). ICAO mandate on Safety Management Systems Follow-on from Regulatory Oversight Weighting Index (C/SI) Analytical Hierarchical Process (AHP) – adopted by Dutch CAA for assessing risk within MROs: –Assess risks from Quality (Part reqts and Quality system) and Organisation (Culture, features and process) –Make an implicit risk analysis more explicit –Informs the decision on intensity of supervision by regulator 2007-2010

16. Bayesian Belief Networks Means to quantify probability of accident scenario using “Conditional probability theory” Estimate the conditional probability of one causal factor given the presence of other factors. Conducted using fusion of data and “beliefs” of Subject Matter Experts (SME) Prior P(A) Information B Posterior P(A l B) New Posterior P(A l C, B) Information C

17. Influence Diagram Source: Luxhoj, 2002 Regulator Maint HR Mn training Repair Inspection Failure Mn org struct Corporate deficiency Organisation Task/Environment Work system design Individual Consequence Operator on oversight Knowledge/ Experience Mn facility Repair Task load Maint Mgt

18. Current status AMMP – data gathering to start this year, with helicopter designers/maintainers Risk in CAW – interest gained from military and civil operators and maintainers. Now undertaking design of models

19. x Bayes’ concept in action Total sectors flown for period Sectors failed Sectors not failed Sectors failed due to document errors Document errors present but sector not failed

20. Example of Bayes Flight documentation is inspected regularly. 20% of late flights had experienced document errors; 30% of on-time flights had document errors. 10% of flights are late. What is the probability of document errors, given that flight is on-time? Solution Let A = event that sector is late; p(A) = 0.1 and B = event that documents have errors. 10% that were late (A) had document errors (B), so p(B|A) = 0.2 90% that were successful (Ā) had document errors (B), so p(B|Ā) = 0.3 The probability of a sector being late (A), given that it has document errors (B) is written p(A|B) Using Bayes