1. Civil Air Patrol
Cross Functional Team Updates Kevin Conyers Chief, Stan/Eval 22 Aug 2018 Anaheim, CA
This is the Cross Functional Team Updates briefing for the DO pre-conference occurring on 22 Aug 2018 in Anaheim, CA. This presentation will be given by Kevin Conyers, CAP NHQ, Chief of Standardization and Evaluation.
One Civil Air Patrol, excelling in service to our nation and our members!

2. Introductions dov@capnhq.gov Kevin Conyers, Chief of Stan/Eval
Lt Col Gene Caisse, Asst Chief of Stan/Eval
Col Jack Buschmann, Glider Program Manager
CAP NHQ Stan/Eval Advisors
Col Mike Moyer
Col Carton Sumner
Lt Col Josh Shields

3. Overview Charters Methodology Performance Analysis Cause Analysis
Intervention Selection
Implementation Timelines
This slide depicts that topics that we will be discussing…

4. CFT Charters
The Aircrew Professionalism and Glider Program Cross Functional Teams (CFTs) were chartered by the National Commander in Oct The objective of ACP CFT was to increase the professionalism of CAP aircrews, while the Glider Program CFT sought to conduct a top to bottom review of the glider program to address systemic problems highlighted by recent mishaps and other identified problems.
The CFTs were comprised of CAP-USAF and CAP personnel, both employees and volunteers from across the organization. On behalf of the DO and myself, I would like to thank the CFT membership for the significant contribution of time and effort they made in support of these efforts.
Both CFTs concluded their work by early April 2018 and subsequently briefed their recommendations and a proposed implementation plan to Gen Smith. Gen Smith subsequently briefed the CSAG and BoG, as well as approving the proposed project plans. Both of these efforts fall under the CAP DO with the CAP DOV acting as the project manager.

5. Methodology 2018 CAP Pilot Survey 2018 CAP Aircrew Survey
Performance Analysis
CAP Mishap Analysis
Benchmarking
Gap Analysis
Cause Analysis
Intervention Selection
Design and Development
Evaluation
Each of the CFTs used a variety of improvement methodologies to assist in the identification of problems; conduct of measurement; analysis; and the selection of improvements or interventions. Some of the key components of the CFT methods are listed here.
For the most part, the CFTs produced recommendations and drafts. As a result, there will be a significant period of design and development before most of those recommendations are brought to fruition. This is particularly true with respect to regulatory changes.
There is one final element of the methodology that should be mentioned and that is Evaluation. The National Commander has established measures of effectiveness that will be used to evaluate implemented changes. He will report those MOEs to the BoG and, if necessary, programs will be adjusted.

6. Performance Analysis Aircraft Accident Rates
There isn’t enough time to address all the CFT findings in detail; however, I would like to take some time to discuss some of those that had a major influence on the CFT and the implementation plan. Let start by discussing CAP’s desired and actual performance in terms of flying accidents, incidents, and mishaps.
Due to its mission, organizational relationship with the USAF, cost, and other factors, CAP expects professional levels of aircrew performance. Statistics for the period 1 Oct 2013 to 30 Jul 2018 reveal an average of 2.2 flight-related accidents (NTSB reportable) per 100,000 hours (for powered aircraft). This compares very favorably to the NTSB reported accident rate for General Aviation personal flying which is 11.8 per 100K flight hours. Even though 70% of CAP’s pilots are not engaged in the business of flying, CAP’s safety performance is substantially better than that of recreational fliers. In fact, CAP’s rate falls just above the NTSB reported average for General Aviation business flying which is 1.4/100K flight hours.
Although there is room for improvement, in the aggregate, CAP’s aircrew are performing very close to professional levels on this particular outcome measure.

7. Performance Analysis Aircraft Incident Rates
In contrast CAP’s ground-related damage rate is 46.4/100K flight hours. Although there is no comparative data for this statistic, the high rate of ground-related damage has been a cause for concern for some period of time. It is reasonable to state that CAP performance does not rise to the level of professionalism with respect to aircraft ground movement. In spite of the deployment of both training and motivation interventions, the data shows no improvement in performance over the examined time period - the number of events averaging 47 per year. The most significant contributor to the GR damage rate is Improper Ground Handling.

8. Cause Analysis Human Factors (Airplane)
Analysis of the human factors causing CAP mishaps didn’t reveal any big surprises…
The largest portion of mishaps were caused by skill-based errors (over/undercontrol and procedural error), as shown in the Pareto chart on the left. The next largest contributor was judgment-based errors (poor risk assessment and decision making). With respect to contributing factors, psychological/behavioral factors (self-regulation), which include overconfidence, complacency, and pressing, were predominate. This was followed closely by the category “coordination, communication, and planning” (teamwork) which in our case was almost entirely comprised of “failures to cross-monitor performance.”

9. Cause Analysis FR-I/A Events Over Time (Airplane)
If we restrict our analysis to only the most severe events, which are Flight Related Incidents & Accidents (FR-I/A) and examine trends over time, we see that most of the hazards appear as noise. The notable exceptions are: Bird/Wildlife Strike, Landing Technique, and CP/IP Involvement.

10. Cause Analysis FR-I/A Events Over Time (Glider)
With respect to gliders, there was a recent spike in events related to CP/IP involvement, as well as a trend of poor risk management/compromised safety margins while attempting to expedite the recovery/launch process. As you might expect, IPs were responsible for all of the decision-making aimed at expediting operations that subsequently resulted in damage to a CAP aircraft.

11. Cause Analysis FR-I/A Events by Flight Purpose
The blue line is read off the right-hand axis.
The data indicates that on check rides CAP experienced damage at a rate that was 3.7 times higher than the organizations average rate per flight hour flown.
Numerous instances of check pilot or instructor pilot involvement in a flight-related damage mishap led the team to ask if this was just the result of a high percentage of check rides and instructional flights in CAPs flight experience. This chart answers that question by depicting the number of flight-related damage mishaps by mission purpose. The number of mishaps for each mission purpose by FY are shown in the bar chart. The dotted line shows the sum of the mishaps for each mission purpose. Here the greatest number of damage events occurs during proficiency flying. However, when the number of mishaps is normalized against the number of flight hours flown under each mission purpose during the timeframe, the story is much different. The dark blue line represents the exposure normalized values and is read off the right hand axis. It can be interpreted as showing that check rides resulted in 3.7 times the rate that would be expected based on the number of hours flown. In contrast, training events with instructors on-board have slightly fewer mishaps than would be expected by hours flown.
The conclusion is that check rides are CAP’s most hazardous flight events while actual missions are CAP’s lowest risk events.

12. Intervention Selection Program Risk Management
ID Hazards
SIRS data
Mishap Analysis Tool
Assess Risk
Risk tolerance
Risk acceptance
Risk categories
DoD practices
Develop Controls
Organizational guidance
Regulatory
Procedural
RAW
Organizational
70-1
Low
Med
SOP
High
Output from the mishap analysis informed a program-level risk management process aimed at identifying, assessing, prioritizing, and controlling key risks. Execution of this approach revealed a number of shortfalls in organizational policy and guidance regarding risk management. Specifically, there is no policy or guidance establishing organizational risk tolerance, risk acceptance levels, risk categorization, or requirements for implementation and sustainment of risk controls. As a result, hazards were identified, assessed, and prioritized by making assumption regarding these issues and applying common DoD practices.
Risks were assessed using a Risk Assessment Matrix (RAM) which attempts to quantify the severity and probability terms. Using this approach each risk is assigned an Hazard Risk Index (HRI) from An assumption was made that Extremely High to High risks (HRI 1-7) should be subjected to the development of formal risk controls and those mitigation plans accepted at the National leadership level. For risks assessed at lower levels controls are not developed. These risks would be communicated to program leadership and monitored over time.
This process resulted in the depicted prioritization of risks for airplane and glider operations. Recommended controls for mitigating these risks using appropriate interventions selected from each quadrant of the human performance map: learning, environment, structure, motivation, as well as culture were developed.
Note: Due to the volume of data and limited time available, only aircrew involved mishaps from FY16-FY18 (N = 308) were coded and used for hazard identification.

13. ACP and Glider Program Initiatives Recommendations
Our discussion today has focused on measures of accidents, incidents and mishaps, cause analysis of those events, and risk mitigation planning. However, it is important to know that this was only piece of what the CFTs looked at and of the recommendations they made, as shown in these charts. These recommendations have been developed into implementation plans that fall under the CAP DO and are managed as projects by the CAP DOV. We will discuss those plans and many of the key tasks in later presentations.

14. Kevin Conyers Chief, Stan/Eval kconyers2@capnhq.gov dov@capnhq.gov
Questions?
Kevin Conyers Chief, Stan/Eval
One Civil Air Patrol, excelling in service to our nation and our members!