1. Advanced Qualification Program: Utilizing Proficiency Data to Drive Curriculums
In-Flight Service
Michelle L. Farkas, ASQ CQA, CMQ/OE – General Manager, Advanced Qualification Program
Jim Karsten – Manager, Program Audit

2. Advanced Qualification Program (AQP) Defined
Voluntary alternative to traditional regulatory training requirements under 14 CFR, Parts 121 & 135
AFS-230: Voluntary Safety Programs Branch
Proficiency-based qualification
Relies on data to support a systematic method for design and development
Better performance through scenario-based crew training
Advisory Circular (AC) provides the Federal Aviation Administration (FAA) guidance for approval of an Advanced Qualification Program (AQP) under AQP regulatory guidance provided in Title 14 of the Code of Federal Regulations (14 CFR) part 121, subpart Y, or in a subsequent regulation that supersedes it.
The goal of AQP is to achieve the highest possible standard of individual and crew performance.
A leading objective of AQP is to provide effective training that will enhance professional qualifications to a level above the present standards that are provided in parts 121 and 135.
AFS-230: Voluntary Safety Programs Branch
Manages voluntary safety programs such as AQP, ASAP, FOQA, LOSA, etc. and all supporting activities, including rulemaking.
AFS-230 also monitors the effectiveness of these programs.
Their offices are located in Dulles, Virginia.
Proficiency-Based Qualification
AQP replaces programmed hours with proficiency-based training and evaluation derived from a detailed job task analysis (JTA) that includes crew resource management (CRM).
A JTA is the method or procedure used to reduce a unit of work to its base components. It provides a detailed, sequential listing of tasks, subtasks, and elements with the knowledge, skill, and attitude characteristics (KSA’s) that clearly define and completely describe the job.
Relies on Data to Support a Systematic Method for Design & Development
AQP incorporates data-driven quality control processes for validating and maintaining the effectiveness of curriculum content.
AQP allows for the ability to modify training curricula, media, and intervals for “real time” learning with more control in content development and change.
Design opportunities are unlimited.
Because data collection on learner’s performance will occur at several time throughout the training day, we will now have the capability of tracking and trending performance across the workforce. If a negative trend is detected, we will be able to reevaluate our training and, if necessary, adapt the training to the learner’s need.
Better Performance Through Scenario-Based Crew Training
The AQP encourages innovation in the methods and technology that are used during instruction and evaluation, and efficient management of training systems.

3. AQP: Past, Present & Future
Application approved in June 2006
Continuing Qualification (CQ)
commenced in January 2007
Transition of Initial curriculum
(Indoctrination & Qualification)
commenced with a Small Group
Tryout in April 2009
AQP being utilized for Delta/Northwest
flight attendant integration training
- Integration Qualification (IQ)
- Aircraft Qualification (AQ)
The path to AQP approval began in October A proposal was made to the In-Flight Service leadership team to move towards an alternate form of training, qualifying, and certifying flight attendants. In-Flight Service’s official application for AQP entry was submitted in June 2006.
Training under AQP commenced in January 2007 with the introduction of the Continuing Qualification (CQ) curriculum (formerly known as recurrent) represents the 3rd year operating recurrent training under AQP and is the first year that we have been able to use data to drive the curriculum.
Combined workforce of ~21,000 flight attendants.

4. Performance Proficiency Data
Collected on flight attendants,
instructors and evaluators.
Housed in the Performance
Proficiency Database (PPDB)
Volume of Data:
Currently: ~14,000 flight attendants
Future: ~21,000 flight attendants
Future of AQP Database
Performance proficiency data is de-identified information that represents the results of an individual’s ability to successfully demonstrate the performance objectives of each curriculum. This information is captured during validation and evaluation gates as a flight attendant progresses through an AQP curriculum, such as Continuing Qualification (CQ).
This data is collected from each flight attendant’s performance and is stored in aggregate in a Performance/Proficiency Database (PPDB). This data is used to analyze training programs and/or groups of participants, not for tracking individual accomplishment. Successful completion and analysis of this data will identify and correct problems, validate AQP curriculum, and identify developing trends.

5. Other Data That Drives Curriculum
In addition to the proficiency data collected during training, the In-Flight Service AQP utilizes the following data when determining curriculum for the next year or changes needed to current curriculum.
Flight Attendant Comment
Tracking System (FACTS)
Operations Control Center
(OCC) Reports
Level 1 Evaluations
Letters of Investigation (LOIs)
Enforcement Investigative
Reports (EIRs)
Safety Management System
(SMS) Safety Roundtable
As part of the AQP, the airline must conduct airline internal analysis for the purpose of curriculum refinement and validation. In addition to the proficiency data collected during training, the In-Flight Service AQP utilizes the following data when determining curriculum for the next year or changes needed to current curriculum.
Flight Attendant Comment Tracking System (FACTS)
FACTS is the system that Delta flight attendants utilize to report on anything that effects their work environment related to twenty-one (21) different categories from Aircraft Maintenance to Uniforms. The AQP team has the ability to query trends on a monthly, quarterly, or yearly basis in order to research what is going on in the flight attendant work environment and how if effects the current training program.
Operations Control Center (OCC) Reports
OCC reports are generated daily and are a compilation of the previous days events. Events range from maintenance issues to passenger illness and how a crew responds to it.
Level 1 Evaluation
Just as the word implies, evaluation at this level measures how participants in a training program react to it. It attempts to answer questions regarding the participants' perceptions - Did they like it? Was the material relevant to their work? This type of evaluation is often called a “smilesheet.” According to Kirkpatrick, every program should at least be evaluated at this level to provide for the improvement of a training program. In addition, the participants' reactions have important consequences for learning (level two). Although a positive reaction does not guarantee learning, a negative reaction almost certainly reduces its possibility.
Letters of Investigation (LOIs)
Enforcement Investigative Reports (EIRs)
Safety Management System (SMS) Integrated Safety Roundtable

6. AQP Structure General Manager Advanced Qualification Program Manager
Program Audit
Performance Proficiency
Training Devices &
Technology
Facilitator Operations
PEOPLEWARE
COURSEWARE
The In-Flight Service (IFS) Advanced Qualification Program (AQP) reports directly to the Director of IFS Learning which falls under the “Business Operations” area of the IFS division.
Manager – Program Audit (COURSEWARE)
Responsible for overseeing and/or contributing to the design and management of complex, large-scale c curriculum for both flight attendants and instructor/evaluators in order to support the IFS AQP.
This position is also responsible for the IFS AQP Program Audit Database (PADB), including manuals, job task lists, qualification standards, and ensures that Crew Resource Management (CRM) and Threat and Error Management (TEM) principles are built into IFS AQP curricula.
Responsible for the IFS AQP Instructor/Evaluator Quality Assurance Program which ensures calibration and quality control among the Instructor/Evaluator group.
Manager – Performance Proficiency
Responsible for management of the IFS AQP Performance Proficiency Database (PPDB), specifically program compliance and statistical analysis of large complex datasets using advanced programming techniques.
Other responsibilities include the oversight of the development of detailed FAA reports analyzing performance of flight attendant training programs.
Manager – Training Devices & Technology (HARDWARE)
Responsible for the oversight of the IFS Mock-Up Shop staff in all training centers to ensure safety, regulatory compliance, and flight attendant brand alignment. This includes ensuring mock-ups, training devices and technology meet or exceed AQP standards as put forth by AFS-230.
Manager – Facilitator Operations (PEOPLEWARE)
Responsible for the oversight of facilitator operations and delivery of AQP-related curricula in order to ensure regulatory compliance and flight attendant brand alignment at all IFS training centers.
Report directly to the Director – IFS Learning and reports via dotted line to the General Manager – AQP.
HARDWARE

7. In-Flight Service Advanced Qualification Program
Continuing Qualification 2009: Driving Curriculum With Data
Jim Karsten, Manager – AQP Program Audit

8. Continuing Qualification (CQ) Overview
To better understand the data that
the In-Flight Service Advanced
Qualification Program (AQP) collects
it is best to first outline the training
programs and the evaluation gates
that are utilized.
Systems Validation (SV)
Proficiency Evaluation (PE)
Event Management Validation (EMV)

9. Data Analysis by Training Center

10. Proficiency Evaluation
In each aircraft we measure the following procedures for door operation:
Disarm (prepare for arrival)
Door Checks (disarmed)
All-call (arrival)
Open Normal Mode
Close Normal Mode
Arm (prepare for departure)
Door Checks (armed)
All-call (departure)
Open Emergency

11. Data Analysis by Aircraft

12. Data Analysis by Aircraft

13. Data Analysis by Aircraft

14. Event Management Validation
Events Management Validation events
consist of:
Emergency Procedures
Security Procedures
Medical Procedures
Passenger Handling
Dangerous Goods

15. Data Analysis by Event

16. Year-Over-Year Comparison
Proficiency Evaluation 2007 to Present:
Disarm (prepare for arrival)
Door checks (disarmed)
All-call (arrival)
Open normal mode
Close normal mode
Arm (prepare for departure)
Door checks (armed)
All-call (departure)
Open emergency

17. Year-Over-Year Comparison
Event Management Validation:
Consistent with 2008
Continue current
Emergency/Security Procedures
Increased Events

18. Data and the Curriculum
Proficiency Evaluation:
Door briefing
Re-connect with door operation
Event Management Validation:
Focus on briefing/debriefing

19. Flight Attendant Feedback….
“This is the FIRST recurrent I have EVER attended that I felt like my proficiency was actually what was "tested".  In previous years, it was like we were "tested" on the verbiage of that particular year.  This year it really was a continuing learning experience and not a subjective microscope.  From a flight attendant that has had her share of recurrent training, this was by far the BEST.  Each facilitator was more interested in seeing that we were "getting it" than being a drill instructor.  We were able to really react to the situations and drills in a real way in order to learn.  A big thanks and a well done!”
- Atlanta-Based Flight Attendant

20. Contacts Michelle L. Farkas, ASQ CQA, CMQ/OE Jim Karsten
General Manager, Advanced Qualification Program (AQP)
Address:
Phone: (404)
Jim Karsten
Manager, AQP Program Audit
Address:
Phone: (404)

21. QUESTIONS