"A system that trains the evaluator"

Slides:



Advertisements
Similar presentations
Note: Lists provided by the Conference Board of Canada
Advertisements

9.401 Auditing Chapter 1 Introduction. Definition of Auditing The accumulation and evaluation The accumulation and evaluation Of evidence about information.
MGTO 630C Staffing and Managing Human Resources Dr. Christina Sue-Chan Performance Management: Chapter 7 Saturday, March 15, 2003 Please note: This is.
Using Situational awareness and decision making
Assessing EM registrars’ leadership and non-technical skills.
3rd Phase: 3rd Phase: PROJECT ORGANISATION Plan  Implementation  Recruit staff and organize project team members  Assign responsibility for Work packages.
Unit 250 Developing Yourself as a Team Leader
Thinking Actively in a Social Context T A S C.
Chapter 6 Training and Development in Sport Organizations.
Health and Safety Dynamic Risk Assessment Aim To inform the students of the need for and the importance of a risk assessment.
01-1-S230-EP Unit S230-EP S230-EP Unit 1 Objectives Describe the values and principles of operational leadership. Identify the qualities.
1–1 Chapter 1 INTRODUCTION TO MANAGEMENT AND ORGANIZATIONS © Prentice Hall, 2002.
CREW RESOURCE MANAGEMENT
Inspire Personal Skills Interpersonal & Organisational Awareness Developing People Deliver Creative Thinking & Problem Solving Decision Making, Prioritising,
BR&TE H.-J. Hoermann, NOTECHS & LH-IPCs, RAeS, Apr Research & Technology Europe Usability Study of Lufthansa’s Performance Evaluation System Hans-Juergen.
Introduction to Management
McGraw-Hill/Irwin © 2003 The McGraw-Hill Companies, Inc., All Rights Reserved. 6-1 Chapter 6 CHAPTER 6 INTERNAL CONTROL IN A FINANCIAL STATEMENT AUDIT.
IT Risks and Controls Revised on Content Internal Control  What is internal control?  Objectives of internal controls  Types of internal controls.
Student Name Student Number ePortfolio Demonstrating my achievement of the NSW Institute of Teachers Graduate Teacher Stage of the Professional Teacher.
Queen’s Management & Leadership Framework
Chapter 6: THE EIGHT STEP PROCESS FOCUS: This chapter provides a description of the application of customer-driven project management.
THOMAS COOK CRM STANDARDS ASSESSMENT Captain Karen Varney CRM Manager Thomas Cook Airlines UK.
Targeted Selection This workforce solution was funded by a grant awarded under the President’s Community- Based Job Training Grants as implemented by the.
A Guide for Management. Overview Benefits of entity-level controls Nature of entity-level controls Types of entity-level controls, control objectives,
IAEA International Atomic Energy Agency Methodology and Responsibilities for Periodic Safety Review for Research Reactors William Kennedy Research Reactor.
Bridge Resource Management
Human Factors Ontario Search and Rescue Volunteer Association Team Leader Training.
Bloom, Assessment & Aims and Objectives Module: ES204 Lecturers – Dr Justin Rami.
PROBLEM SOLVING. Definition The act of defining a problem; determining the cause of the problem; identifying, prioritizing and selecting alternatives.
BEHAVIOR BASED SELECTION Reducing the risk. Goals  Improve hiring accuracy  Save time and money  Reduce risk.
NOTECHS. Objectives By the end of this workshop you will:  Understand facilitation techniques, and be able to apply them to manage a constructive debrief.
Collaborative & Interpersonal Leadership
Welcome LEARN: teamwork and communication in Quality Improvement
HRM 560 Training and Development Environment
MANPOWER PLANNING and INVENTORY by: Allan B Palomo.
3 Chapter Needs Assessment.
What does Success mean to you?
Developing teaching, learning and assessment in education and training
Part 1 Being professional
Strategic Training.
Introduction to the Validation Phase
THE CONCEPTUAL FRAMEWORK
Skills 4 Success 3.00 – Understand the world of work and skills needed for employment success – Understand various skills needed for employment success.
CRMSG meeting 18th March 2010.
Quality Workshop The Local Council Award Scheme is a great guide for good practice in our sector and a way for councils to build confidence in their.
HUMAN RESOURCE GOVERNANCE, RISK MANAGEMENT AND COMPLIANCE
Overview – Guide to Developing Safety Improvement Plan
"A system that trains the evaluator"
HSE Case: Risk Based Approach.
Human Resources Competency Framework
Overview – Guide to Developing Safety Improvement Plan
Case Study Title CRM CASE STUDY.
Skills for Success 3.01 – Understand the world of work and skills needed for employment success – Understand various skills needed for employment.
Skills 4 Success 3.00 – Understand the world of work and skills needed for employment success – Understand various skills needed for employment success.
Thursday’s Lecture Chemistry Building Musspratt Lecture Theatre,
SCANS Skills Competencies &Personal Qualities
6 6 Training Employees C H A P T E R Training Employees
Skills 4 Success 3.00 – Understand the world of work and skills needed for employment success – Understand various skills needed for employment success.
Chapter 16 Planning and Management of Health Promotion
Engaging People : Performance Monitoring
Team Leader Training Human Factors
Case Study Title CRM CASE STUDY.
How to conduct Effective Stage-1 Audit
Leadership for Safety Through the Case Method
Skills 4 Success 3.00 – Understand the world of work and skills needed for employment success – Understand various skills needed for employment success.
Skills 4 Success 3.00 – Understand the world of work and skills needed for employment success – Understand various skills needed for employment success.
A SYSTEMATIC APPROACH S Borua.
Form FRA PC/L Human Factors
Skills 4 Success 3.00 – Understand the world of work and skills needed for employment success – Understand various skills needed for employment success.
THOMAS COOK CRM STANDARDS ASSESSMENT
Presentation transcript:

"A system that trains the evaluator" From NOTECHS to a New System of Observing Behavioural Markers as a Basis for Assessing Pilots' Performance "A system that trains the evaluator" Cpt. Harry Neb, Lufthansa German Airlines FRA PC/L Human Factors Operational Input

Influences and Sources LH Interpersonal Competence LH Safety Study (1997) External Information (scientific, other airlines, RAF, etc.) NOTECHS JAA LH CRM-Markers (1992) FRA PC/L Human Factors 24.11.2018

Influences and Sources JAA FRA PC/L Human Factors 24.11.2018

Frequency Distribution by Event-Configurations 9.1% 46.2% 30.8% Now we are going to take a look on the results - where the factors are left in their configurations and compared by the frequency of reported occurrence. As we can see, the number of incidents is dependent on the complexity of aspects, with increasing complexity the frequency increases as well. Although this doesn´t come as a complete surprise, it is the first time - that we are able to prove this connection. 14.5% FRA PC/L Human Factors 24.11.2018

Aggravating Social Factors contributed to 71% of events. Deficiencies in Involvement Information management 67.2% Quality of communication 52.3% Social climate 27.2% FRA PC/L Human Factors 24.11.2018

FRA PC/L Human Factors 24.11.2018

Interpersonal Competence Elements Communication Generally, communication includes information transfer and social aspects. Crew members share information, and assure reception and understanding. Suggestions of other crew members are considered, even if one does not agree. Ambiguities and uncertainties are announced. Workload Management Leadership and Teamwork Crew members clearly prioritise operational tasks and distribute them appropriately. Available external and internal resources are used to accom-plish task completion in good time. Stress and error are inherent factors of flight, and crew members aim to minimise their negative effects. Led by the commander, the crew achieves a safe and efficient performance in a climate that is rational and free of intimidation. Social interaction conflicts have to be addressed and managed. Every crew member takes initiative to be an active and constructive part of the team. Interpersonal Competence Situation Awareness and Decision Making Crew members recognise and anticipate factors affecting the flight. After these factors are evaluated, they choose the appropriate course of action. To achieve a favourable outcome, crew members actively monitor execution and development of the situation. FRA PC/L Human Factors 24.11.2018

Descriptors Communication Atmosphere Information Transfer encourage open and honest communication achieve a positive first impression listen actively consider suggestions Information Transfer share information assure reception assure understanding Information Management clearly state plans and intentions announce ambiguities announce uncertainties speak frankly about problems within the crew FRA PC/L Human Factors 24.11.2018

Descriptors Leadership & Teamwork Command- ability Team Ability take the lead of the crew as commander establish goals, control outcome, and correct consider condition of others Team Ability act as a constructive member of the team take initiative encourage others to co-operate support others seek ideas and views from others present own point of view provide appropriate feedback propose alternative ideas if appropriate Conflict Management address and manage conflicts achieve rational climate avoid intimidation adopt assertive behaviour if appropriate and persist until attention of others is gained or corrective action taken accept appropriate criticism avoid competition between crew members FRA PC/L Human Factors 24.11.2018

Descriptors Workload Management Task Time Stress and Error prioritise operational tasks distribute tasks appropriately complete tasks in good time use external and internal resources Time plan ahead allocate time to tasks appropriately Stress and Error aim to minimise negative effects of stress aim to minimise effects of error FRA PC/L Human Factors 24.11.2018

Descriptors Situation Awareness and Decision Making Preparation act with respect to time available avoid distractions anticipate factors affecting the flight recognise factors affecting the flight Processing evaluate factors affecting the flight choose appropriate course of action monitor execution monitor development of the situation apply FOR-DEC for complex decisions F-Facts, O-Options, R-Risks & Benefits, D-Decision, E-Execution, C-Check Interaction involve others in the process discuss discrepancies FRA PC/L Human Factors 24.11.2018

Evaluation A New Concept. Why?

Evaluation Prerequisites Appropriateness and Relevance Standardization of contents and execution Validity: Accuracy of correspondence with the relevant learning goals Reliability: Constant accuracy and correspondence with comparable results Objectivity: Same results with application by different evaluators Transparency and Reasonableness: No traps and tricks Fairness: Equal treatment of candidates from: K. Steininger, Handbuch der Flugpädagogik FRA PC/L Human Factors 24.11.2018

Present Status Description of performance in the words of the individual evaluator Subjective view due to different priorities Interpretation of assessment by the training management CRM criteria (behavioural markers) not integrated And, to top all of it, the JAR demand for assessment of CRM (primarily during line checking) had to be fulfilled. But nobody within LH so far had developed distinct and detailed criteria for this. FRA PC/L Human Factors 24.11.2018

Present Status contd. Quality description in two steps: proficient repeat Trend not recognizable Flaws in the system not identifiable Another point in the present system is the problem of recognizing trends. Only yes or no is simply not sufficient to define a trend curve. And what, if deficiencies in the performance of trainees was a result of deficiencies in the training system? FRA PC/L Human Factors 24.11.2018

For all assessments Goal (Highlights) Uniformity Objectivity As I mentioned before, we started with initial line training for FO or upgrading to Commander. But developing a sound concept meant also to consider all training and check events. Take calculated measures at the earliest possible stage for any performance below standard FRA PC/L Human Factors 24.11.2018

Uniformity Demand the same performance under the same circumstances Ask the same questions Observe the same defined behaviour Treat each trainee equally FRA PC/L Human Factors 24.11.2018

For all assessments Goal (Highlights) Uniformity Objectivity As I mentioned before, we started with initial line training for FO or upgrading to Commander. But developing a sound concept meant also to consider all training and check events. Take calculated measures at the earliest possible stage for any performance below standard FRA PC/L Human Factors 24.11.2018

Objectivity Application by different evaluators leads to the same result FRA PC/L Human Factors 24.11.2018

For all assessments Goal (Highlights) Uniformity Objectivity As I mentioned before, we started with initial line training for FO or upgrading to Commander. But developing a sound concept meant also to consider all training and check events. Take calculated measures at the earliest possible stage for any performance below standard FRA PC/L Human Factors 24.11.2018

Expected Outcome Assess quality to manage quality Identify trends to assure (maintain or improve) quality with Trainee Instructor System FRA PC/L Human Factors 24.11.2018

Identified Necessary Actions Define detailed observable rating criteria => Description of company standard By observing frequencies of behaviour the progress and the absolute performance to be described Lead instructors to observe largely the same Make transparent for the trainee FRA PC/L Human Factors 24.11.2018

Identified Necessary Actions Differentiate quality of performance Define behavioural markers to describe the interpersonal competence FRA PC/L Human Factors 24.11.2018

Quality Assurance for: TRAINEE by providing well aimed support for cases of performance below standard {below satisfactory} (because identifiable) FRA PC/L Human Factors 24.11.2018

Quality Assurance for: TRAINER (Evaluator) by enabling comparable assessments (for objectivity) FRA PC/L Human Factors 24.11.2018

Quality Assurance for: SYSTEM to improve the training process (by discovering weak spots) FRA PC/L Human Factors 24.11.2018

! Goal (Trainer): No substantial change of methodical and didactic means for training (The system does not intervene in the personal style of training) FRA PC/L Human Factors 24.11.2018

Definition Trainer: "A trainer is an employee trained in training who is instructed by the traineeship provider to train trainees." Hamburg Senate "Definitions" FRA PC/L Human Factors 24.11.2018

Goal (Trainer): Keep the sequence: Observe Note Evaluate FRA PC/L Human Factors 24.11.2018

Observation Prerequisite for evaluation is observation of Quantity and Quality of behaviour how often what FRA PC/L Human Factors 24.11.2018

What Defines Quality Specified OBSERVATION CRITERIA QUALITY (i.e. behavioural markers resp. observable behaviour) QUALITY FRA PC/L Human Factors 24.11.2018

Therefore observation criteria have to be: relevant extensive observable FRA PC/L Human Factors 24.11.2018

general descriptors of the areas Structure 9 defined phases of flight (from "preparation" to "leaving aeroplane") + general descriptors of the areas technical procedural interpersonal FRA PC/L Human Factors 24.11.2018

Break FRA PC/L Human Factors 24.11.2018

What Next? Principles of Behaviour Assessment in NOTECHS/JARTEL Development of Lufthansa‘s New Behaviour Marker System Crosslinking NOTECHS to LH Interpersonal Competences The Implementation Case: Lufthansa‘s Evaluation Method Usability Study Summary