1. NOTECHS

2. Objectives By the end of this workshop you will:  Understand facilitation techniques, and be able to apply them to manage a constructive debrief  Be able to apply non-technical skills criteria to a CRM assessment  Be aware of CRM training and recording requirements.

3. CRM Skills Assessment

4. CRM Training  Why do we train CRM?  Does CRM Work?  Statistical proof difficult  Improved Safety Attitudes?  UT Flight Management Attitudes Questionnaire  Is changing attitudes enough?

5. Assessment - What? Non-Technical Skills (NTS)  “Non-Technical Skills are skills that refer to pilots’ attitudes and behaviours in the cockpit, not directly related to aircraft control, system management and Standard Operating Procedures (SOPs), which influence flight safety.”

6. Why Assess?  If we believe that CRM skill is an important element of safe flight operations, then it should be assessed in both licensing and line checks, as are technical skills.

7. Requirements  EU-OPS require the training and assessment of pilots’ CRM skills: “The flight crew must be assessed on their CRM skills in accordance with a methodology acceptable to the Authority and published in the Operations Manual.” “The purpose of such an assessment is to provide feedback to the individual and serve to identify retraining; and be used to improve the CRM training system”. Appendix 1 EU-OPS 1.965 3 (ii)

8. NOTECHS Group  NOTECHS – Non Technical Skills  Established in 1996  German DLR, Dutch NLR, France IMASSA and University of Aberdeen  Tasked to identify or develop a feasible and efficient methodology for assessing pilots` non-technical skills  Design Requirements  to be used to assess the skills of an individual pilot, rather than a crew  suitable for use across Europe, by both large and small operators.

9. Behavioural Markers  Observable, non-technical behaviours that contribute to superior or substandard performance as contributing factors enhancing flight safety  Can be structured into a set of categories  Describe a specific, observable behaviour, not an attitude or personality trait.

10. NOTECHS Structure  A descriptive framework to cover a full range of Non-Technical Skills to be assessed  Consists of three levels  Categories (1 st level)  Q – What categories would you expect to see?

11. Categories  Co-operation  Management & Leadership  Situational Awareness  Decision Making  What’s Missing?

12. NOTECHS Structure  A descriptive framework to cover a full range of Non-Technical Skills to be assessed  Consists of three levels  Categories (1 st level)  Elements (2 nd level)

13. Elements  Co-operation  Team Building & maintaining  Considering Others  Supporting Others  Conflict Solving  Situational Awareness  Awareness of aircraft systems  Awareness of external environment  Awareness of time  Management & Leadership  Providing and maintaining standards  Planning and Co-ordination  Workload Management  Decision Making  Problem definition & diagnosis  Option generation  Risk assessment and option selection  Outcome review

14. NOTECHS Structure  A descriptive framework to cover a full range of Non-Technical Skills to be assessed  Consists of three levels  Categories (1 st level)  Elements (2 nd level)  Behaviours (3 rd level)  Examples of Good & Poor practice are provided for each element

15. NOTECHS - Co-Operation Category ElementsExample Behaviours Team Building and maintaining Establishes atmosphere for open communication and participation Blocks open communication Considering Others Takes Condition of other crew members into account Ignores suggestions from other crew members Supporting Others Helps other crew members in demanding situations Does not help other crew members in demanding situations Conflict Solving Concentrates on what is right rather than who is right Accuses other crew members of making errors

16. Ratings  Very Poor  Behaviour directly endangers flight safety  Poor  Behaviour in other conditions could endanger flight safety  Acceptable  Behaviour does not endanger flight safety but needs improvement  Good  Behaviour enhances flight safety  Very Good  Behaviour optimally enhances flight safety and could serve as an example for other pilots

17. Five Design Principles  Use of a two point rating scale  Pass/Fail  Technical consequences required  Pass if within technical limits, even if deficient in NTS  Explanation required  Mandatory for fail categories, using simple language  Repetition required  Not an isolated behaviour, but long lasting strengths and weaknesses  Only observable behaviour  Behaviour can be debriefed and changed  Character traits are subjective and cannot be changed.

18. JAR-TEL Validation  A project funded by EEC 1998 – 2001 to validate NOTECHS  Sofreavia (leader), Aberdeen University, Airbus Industrie, Alitalia, British Airways, DERA, DLR, IMASSA and NLR  85 Instructor Pilots were recruited from a representative sample of European airlines  Applied method to rate crew members in 8 scenarios filmed in a B757 simulator.

19. JAR-TEL Results  Reliable  Internal Consistency  Accuracy  Inter Rater Agreement  Usable  Culturally robust.

20. Debriefing

21. Feedback - BOOST  Feedback should be ‘UNDERSTOOD’, ‘ACCEPTED’ & ‘CAPABLE OF ACTION’ by the person receiving it  BBalanced  OObserved  OObjective  SSpecific  TTimely

22. Debrief An effective debrief should:  Be structured  Be delivered in a facilitative style  Elicit the relevant assessment points  Gain acceptance by the candidate.

23. Summary  If we believe that CRM skill is an important element of safe flight operations, then it should be assessed in both licensing and line checks, as are technical skills.  EU-OPS Requirement for CRM skills assessment by a methodology acceptable to the Authority and published in the Operations Manual.  NOTECHS behavioural marker system has been found to be  Practical  Reliable  When used by trained personnel in a carefully controlled conditions  NOTECHS may be used to meet the Operator and EU- OPS requirements.

24. Questions