Life Cycle Logistics Competency Development and Management Process Task Closeout LOG FIPT Presentation May 9, 2008 LOG FIPT AT&L HCSP Competency Initiative … Enabling a High-Performing, Agile Workforce

2 Framework Development Framework Development Phase I - Convene an expert panel (EP) Actions: Develop a competency framework & input model EP identifies Subject Matter Experts (SMEs) EP communicates competency effort to the SMEs Develop communications package Goal: Establish baseline of existing competency model. Communicate effort Products: FA provides list of targeted high-performing SMEs Obtains expert panel concurrence on baseline competency framework Obtain approval from Dir, HCI and FA on competency model input Phase I - Convene an expert panel (EP) Actions: Develop a competency framework & input model EP identifies Subject Matter Experts (SMEs) EP communicates competency effort to the SMEs Develop communications package Goal: Establish baseline of existing competency model. Communicate effort Products: FA provides list of targeted high-performing SMEs Obtains expert panel concurrence on baseline competency framework Obtain approval from Dir, HCI and FA on competency model input Phase IV – Validate and Assess Actions: Launch competency assessment tool Analyze results to evaluate model validity and generalizability to the workforce Goal: Identify competencies required for superior performance Evaluate proficiency gaps for validated competencies Plan for continual updates and use of competency model Products: Deliver proven (validated) competency model in HR XML format Provide competency validation and assessment and obtain Dir, HCI and FA approval Phase IV – Validate and Assess Actions: Launch competency assessment tool Analyze results to evaluate model validity and generalizability to the workforce Goal: Identify competencies required for superior performance Evaluate proficiency gaps for validated competencies Plan for continual updates and use of competency model Products: Deliver proven (validated) competency model in HR XML format Provide competency validation and assessment and obtain Dir, HCI and FA approval AT&L Competency Management Process Successful Completion! Model Testing & Refinement Model Testing & Refinement Model Development Phase III – Perform a beta test & refine model Actions: Collect and synthesize feedback from proposed model report Pre-assessment communications to workforce Identify stratified workforce sample Goal: Further refine model to include input from functional leads Obtain FA and Dir, HCI approval for validation assessment Products: Obtain concurrence from FIPT on competency model Obtain approval from Dir, HCI and FA on competency model Phase III – Perform a beta test & refine model Actions: Collect and synthesize feedback from proposed model report Pre-assessment communications to workforce Identify stratified workforce sample Goal: Further refine model to include input from functional leads Obtain FA and Dir, HCI approval for validation assessment Products: Obtain concurrence from FIPT on competency model Obtain approval from Dir, HCI and FA on competency model Phase II – Develop the model Actions: SMEs review the competency framework and provide essential job data through structured interviews and online data collection tools. SMEs engaged to identify key “work” situations and competencies contributing to successful performance Analyze results and develop competency model content Goal: Model development and identification of key behaviors Products: Deliver Proposed Model Report to Dir, HCI and FA for review Phase II – Develop the model Actions: SMEs review the competency framework and provide essential job data through structured interviews and online data collection tools. SMEs engaged to identify key “work” situations and competencies contributing to successful performance Analyze results and develop competency model content Goal: Model development and identification of key behaviors Products: Deliver Proposed Model Report to Dir, HCI and FA for review Collect Existing Competency Data Competency Validation, Assessment, and Sustainment Competency Validation, Assessment, and Sustainment May 06 – July 06Sep Jun 07 Nov 07 - Present Jan Apr 08

3 Review of Our Activities Phase I: Expert Panel Process –Held 5-6 meetings to hammer out the framework –Communicated to and identified SMEs Phase II: Model Development - SME Workshops –Held satellite sessions and in-person facilitated sessions –This number is more than sufficient for the current analysis Phase III: Pilot Test – Facilitated Review –Held a facilitated review of the final model using functional experts from DAU and CNA

4 Phase IV Participation Exceeded our aim of 1092 employees Adequate Participation Achieved: Of the 5635 LCL professionals randomly identified, 1295 completed the assessment for a total response rate of 22.9%. –This number is more than sufficient for the current analysis Focus is on Validation of the model –Ensuring competencies are related to the “job” of a Life Cycle Logistics professional –Not enough supervisor participation did not allow for workforce proficiency gap assessment Appropriate to focus on validation for this level of development of our competency model There are many options for building off the current work to look into workforce assessment and workforce planning aspects

5 Phase IV: Participation In Detail ComponentFrequencyPercent Army Navy (including USMC) Air Force th Estate (DCMA, DLA, Other)423.2 Total Career LevelFrequencyPercent Entry Level181.4 Journey Level Senior Level Total Military/Civilian StatusFrequencyPercent Civilian Military544.2 Total Primary RoleFrequencyPercent System Sustainment Acquisition Logistics Total Years LCLFrequencyPercent 1 to 5 Years LCL Experience to 10 Years LCL Experience to 15 Years LCL Experience to 20 Years LCL Experience Years LCL Experience Missing332.5 Total Average14.1 years

6 Phase IV: Participation In Detail Comparison to FY2004 Workforce Numbers Military/Civilian Status FY2004NumberPercent Civilian Military ComponentFrequencyPercent Army Navy (including USMC) Air Force th Estate (DCMA, DLA, Other)423.2 Total Military/Civilian StatusFrequencyPercent Civilian Military544.2 Total Component FY2004NumberPercent Army Navy Air Force th Estate

7 Technical Competency Assessment What Did We Ask The Participants? Frequency, Criticality, and Proficiency

8 Competency Results – Validation Frequency and Criticality CompetencyElement In Assessment Utilizing Simulation and Modeling Techniques Use modeling and simulation to estimate cost and/or performance parameters. (E2) Earned Value Management Provide supportability data to ensure inclusion in EVM tracking where appropriate. (E6) What competencies were rated highest across all ratings? Integrated Product and Process Development (IPPD) Maintaining and Managing Customer Relationships What competencies were rated lowest across all ratings? Utilizing Simulation and Modeling Earned Value Management CompetencyElement In Assessment Integrated Product and Processes Development (IPPD) Contribute to an IPT to ensure logistics considerations are represented in the program. (E12) Maintaining and Managing Customer Relationships Communicate with the customer to verify that logistics requirements are stated accurately in the system design. (E14) Use customer feedback tools or methods to refine, maintain, and/or share logistics information. (E15)

9 Exploratory Factor Analysis Unit 2: Performance-Based Logistics Planning and Management Unit 3: Collaborate with Customers and IPT Members to Enhance Supportablity Unit 1: Utilizing Test Data for Best Design, Configuration, and Processes Unit 4: Cost Evaluation for Sustainment of Systems Unit 1: Utilizing Test Data for Best Design, Configuration, and Processes Unit 2: Performance-Based Logistics Planning and Management Unit 3: Collaborate with Customers and IPT members to Enhance Supportability Unit 4: Cost Evaluation for Sustainment of Systems

10 Competency Results – By Unit Level What Unit Was Rated Highest Across All Ratings? Unit 3 Collaborate with Customers and IPT members to Enhance Supportability is the highest rated across frequency, criticality, and proficiency The next highest was Unit 2: Performance-Based Logistics Planning and Management across all ratings Which Units Were Rated Lowest? The Lowest Rated Units then vary between Unit 1 and Unit 4 in Frequency, Criticality, and Proficiency Unit 4: Cost Evaluation for Sustainment of Systems Unit 1: Utilizing Test Data for Best Design, Configuration, and Processes

11 Competency Results – Professional Competencies Professional CompetencyProficiencyFrequencyCriticality Interpersonal Skills Customer Service Written Communication Flexibility Accountability Team Building Oral Communication Influencing/Negotiating Problem Solving Creativity and Innovation Note Top 3 Note: Problem Solving and Creativity and Innovation were identified in the Phase II study as top professional competencies in Phase II. In fact, Problem solving was the top rated in the development Phase II results, but was lowly rated in the current ratings.

12 Feedback Received – Additional Results Reported 1.Were there differences between Primary Roles? (System Sustainment Vs. Acquisition Logistics Staff) 2. What about competencies with differences in Ratings?

13 Feedback Received – Additional Results Reported We Found Some Significant Differences in Primary Roles: Systems Sustainment Staff Vs. Acquisition Logistics Staff  Acquisition Logistics staff had higher ratings across frequency, criticality and proficiency on all units, and a majority of the competencies.  This has implications on training resources and development needs depending upon a learner’s role on the job Were there differences between Primary Roles?

14 Feedback Received – Additional Results Reported We found 2 competencies with relatively high frequency and criticality and low ratings in proficiency: Joint Operational Interface and Maintenance Planning CompetencyElements Joint/Operational Logistics Interface Consider Joint support opportunities/requirements in the supportability strategy. (E13) Maintenance Planning Ensure maintenance planning is incorporated in the acquisition strategy for the system (i.e., when, where, and how). (E32) What about competencies with differences in ratings?

15 What’s Next Part of the Answer: Competency to Course Matrix LCL OSD Logistics Competency Joint Operational Interface Maintenance Planning Problem Solving (Additional Competency from Other Efforts) SiteCourse DAU LOG201 XXX DAU ACQ201A X DAU CLL 006 Depot Maintenance Planning X AFIT Additional Course XXX

16 What’s Next? Part of the Answer: Design Other Interventions Beyond Education Measure Proficiency at Time 1 Measure Proficiency at Time 3 Measure Proficiency at Time 2 Design a workforce intervention INTERVENTION Design a workforce intervention INTERVENTION  Using the proficiency standards give you information to begin your analysis.

17 What’s Next? Part of the Answer: Design Other Interventions Beyond Education  Adjust human capital strategies and organizational level decision making in such areas as: Education, training and development modification based on learner characteristics Targeted recruitment and retention to shore up strengths or heal weaknesses in workforce or at the command level Conduct strategic human capital planning Workforce/manpower allocations and other resource allocations  Conduct additional workforce analysis to: Identify strengths and weakness at both a community and location level Determine mission area or other workforce groupings that may have specific challenge areas Understand occupation job series allocations Enhance learning assets through knowledge management Continue to Adjust Competency Models  Each career field will continue to refine the competency model as necessary to: Incorporate lessons-learned from the assessment process Integrate assessment results and input from subject matter experts to specialty areas  Component integration of competency management to: Establish a continuous process to define and maintain the required competencies Evaluate organizational skill requirements

18 Workforce Planning Applications: What’s the Supply and Demand? We have helped to create competencies which can be used to help look at the supply in the workforce : Gap analysis at the local or community level Identify strengths and weakness at command or location level Identify who is doing the work and their various locations/projects/missions And these studies can be combined with Additional study could look at the demand signal and look at who is doing what, and, what drives additional work. SUPPLY DEMAND

19 Logistics Competency Management Workforce Validation Questions?

20 Competency Results – Differences by Component There are no differences in Proficiency between the major Service Components Army is different in Frequency & Criticality Army (mean 2.4) does significantly less Unit 1,2, & 3 activities than Air Force or Navy Army participants see Unit 1, 2, & 3 as significantly less critical than Air Force or Navy The job of an Army Life Cycle Logistician may be significantly different than that of a practitioner in Air Force or Navy. We know there are more Army LCLoggies than others They may have a more wide variety of duties, beyond our competency model

21 Framework Development Framework Development Phase I - Convene an expert panel (EP) Actions: Develop a competency framework & input model EP identifies Subject Matter Experts (SMEs) EP communicates competency effort to the SMEs Develop communications package Goal: Establish baseline of existing competency model. Communicate effort Products: FA provides list of targeted high-performing SMEs Obtains expert panel concurrence on baseline competency framework Obtain approval from Dir, HCI and FA on competency model input Phase I - Convene an expert panel (EP) Actions: Develop a competency framework & input model EP identifies Subject Matter Experts (SMEs) EP communicates competency effort to the SMEs Develop communications package Goal: Establish baseline of existing competency model. Communicate effort Products: FA provides list of targeted high-performing SMEs Obtains expert panel concurrence on baseline competency framework Obtain approval from Dir, HCI and FA on competency model input Phase IV – Validate and Assess Actions: Launch competency assessment tool Analyze results to evaluate model validity and generalizability to the workforce Goal: Identify competencies required for superior performance Evaluate proficiency gaps for validated competencies Plan for continual updates and use of competency model Products: Deliver proven (validated) competency model in HR XML format Provide competency validation and assessment and obtain Dir, HCI and FA approval Phase IV – Validate and Assess Actions: Launch competency assessment tool Analyze results to evaluate model validity and generalizability to the workforce Goal: Identify competencies required for superior performance Evaluate proficiency gaps for validated competencies Plan for continual updates and use of competency model Products: Deliver proven (validated) competency model in HR XML format Provide competency validation and assessment and obtain Dir, HCI and FA approval AT&L Competency Management Process Model Testing & Refinement Model Testing & Refinement Model Development V 1.0 CompetencyModel Proposed Competency Model Report Approved Initial Competency Model V 0.5 Phase III – Perform a beta test & refine model Actions: Collect and synthesize feedback from proposed model report Pre-assessment communications to workforce Identify stratified workforce sample Goal: Further refine model to include input from functional leads Obtain FA and Dir, HCI approval for validation assessment Products: Obtain concurrence from FIPT on competency model Obtain approval from Dir, HCI and FA on competency model Phase III – Perform a beta test & refine model Actions: Collect and synthesize feedback from proposed model report Pre-assessment communications to workforce Identify stratified workforce sample Goal: Further refine model to include input from functional leads Obtain FA and Dir, HCI approval for validation assessment Products: Obtain concurrence from FIPT on competency model Obtain approval from Dir, HCI and FA on competency model Phase II – Develop the model Actions: SMEs review the competency framework and provide essential job data through structured interviews and online data collection tools. SMEs engaged to identify key “work” situations and competencies contributing to successful performance Analyze results and develop competency model content Goal: Model development and identification of key behaviors Products: Deliver Proposed Model Report to Dir, HCI and FA for review Phase II – Develop the model Actions: SMEs review the competency framework and provide essential job data through structured interviews and online data collection tools. SMEs engaged to identify key “work” situations and competencies contributing to successful performance Analyze results and develop competency model content Goal: Model development and identification of key behaviors Products: Deliver Proposed Model Report to Dir, HCI and FA for review Collect Existing Competency Data CompetencyValidation & Assessment Report Approved Input Competency Model Competency Validation, Assessment, and Sustainment Competency Validation, Assessment, and Sustainment May 06 – July 06Sep Jun 07 Nov 07 - Present Jan Apr 08

22 Logistics Competency Management Workforce Validation What has been done? LCL experts met with CNA to refine competency elements Website has been tested and feedback has been collected A random sample from each component has been selected and will be ed by CNA with login information Launch details have been finalized Changes are being implemented and a final round of testing will close Jan-14 to be sent co-signed from Mr. Hall and Mr. Anderson to be sent Jan-14

23 Analysis Business Rules for Inclusion/Exclusion in Sample –Extreme proficiency rating differences between supervisor and employee will be discarded –If employee-supervisor pair incomplete, only importance and frequency ratings will be factored into final results Assumptions/Approach for Reporting Results –Gap scores are combined to make a composite score –Composite score of employee and supervisor data will be compared to a standard –Gaps will be reported based on the magnitude of difference to the proficiency standard to identify most to least critical