CBT Management and Process 4/19/2019

Courseware Management and Processes Mission Statement Provide recommendations and guidelines to the Computer-based Training community that identifies the attributes of “Good CBT” processes and product. 4/19/2019

Courseware Management and Processes Objectives Re-establishing the Basics: ISD Model Establish a High Level CBT Development Process Template Establish Metric Recommendations Establish Quality Standard Recommendations Set Recommendations for ADL/SCORM as it Drives the Process 4/19/2019

Why Processes and Metrics? Streamline Approach to Courseware Development Collect Analyze Improve 4/19/2019

METRIC MANAGEMENT SYSTEM This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation In Slide Show, click on the right mouse button Select “Meeting Minder” Select the “Action Items” tab Type in action items as they come up Click OK to dismiss this box This will automatically create an Action Item slide at the end of your presentation with your points entered. METRIC MANAGEMENT SYSTEM The Boeing Company Integrated Defense Systems Mario A. Trejo 4/19/2019 V6.1a

Metric Management System Objective Identify the requirements of a metrics system for courseware development and Process Based Management as it relates to: Quality Timeliness Efficiency Cycle Time Process Compliance 4/19/2019

Metric Management System Called MMS Consists of several integrated tracking systems Tracks Quality Performance Timeliness Cycle Time Process Compliance 4/19/2019

MMS Project Prototype developed in MS Access Multi-Phase development project Each phase is based on the tracking system being integrated MMS administration tools are upgraded with each tracking system 4/19/2019

Integrated Tracking Systems Internal Discrepancy Tracking External Discrepancy Tracking Product Development Tracking Process Surveillance Tracking Quality, Timeliness, Efficiency, and Cycle Time Monitor 4/19/2019

Internal Discrepancy Tracking System Internal Discrepancy Tracking System is completed Used on all projects to track internal deficiencies Tracks Deficiency Reports Lesson Element Checkpoint Recorder Recorded Date Attribute Type deficiency quantities Notes 4/19/2019

Internal Discrepancy Tracking System Reports Hours of instruction reviewed Deficiencies per Hour Of Instruction reviewed Deficiencies by Checkpoint Deficiencies by Attribute Type per Checkpoint Deficiencies by Course 4/19/2019

External Discrepancy Tracking System External Discrepancy Tracking System is completed Used on some projects to track External Discrepancy Reports Tracks Deficiency Reports Checkpoint Lesson Element Recorder Recorded Date Discrepancy, description Recommendation, for corrective action Disposition, corrective action Recorder’s Name, corrective action reviewer Notes Attribute Type deficiency quantities 4/19/2019

External Discrepancy Tracking System Reports Hours of instruction reviewed Deficiencies per Hour Of Instruction reviewed Deficiencies by Checkpoint Deficiencies by Attribute Type per Checkpoint Deficiencies by Course 4/19/2019

Product Development Tracking System (1 of 3) Product Development Tracking System is in development Used on three different types of projects Tracks Media Types Tracks Element Types Tracks Content Types Tracks Checkpoints 4/19/2019

Product Development Tracking System (2 of 3) Tracks Project Scheduled Start and Scheduled Completion Dates Monthly Budgeted and Actual Hours Monthly Budgeted and Actual Dollars Tracks Course and Lesson Title and Number Actual Start and Actual Completion Dates Tracks Lesson Checkpoint Scheduled Person/Group and Actual Person/Group Lesson task weights 4/19/2019

Product Development Tracking System (3 of 3) Tracks similar information for Major Topics and Subtopics Tracks Cost Performance Index Tracks Scheduled Performance Index 4/19/2019

Product Development Tracking System Reports (1 of 3) Project Timeline Scheduled versus Elapsed Project Checkpoint Performance Scheduled versus Completed Project Hours Scheduled versus Used Course And Lesson Percent Completed Course Hours Scheduled and Completed 4/19/2019

Product Development Tracking System Reports (2 of 3) Monthly Hours Scheduled and Used Monthly Accumulative Hours Monthly Index Cost Performance Index Scheduled Performance Index Deficiency Report Project/Course Percent Completed Course/Lesson Percent Completed 4/19/2019

Product Development Tracking System Reports (3 of 3) Topic Performance Scheduled versus Completed Topic Hours Scheduled and Completed 4/19/2019

Process Surveillance Tracking System Process Surveillance Tracking System is completed Used on several projects Tracks Management Process Verify process existence, approval, and usage Process Requirements Verify Development Process requirements have been met Process Tasks Verify Development Process are being followed 4/19/2019

Process Surveillance Tracking System Reports IS 6.06.05 Process Compliance IS 6.06.05 Task Compliance Total quantities of Process Surveillance Nonconformance results Monthly quantities of Process Surveillance Nonconformance results Monthly Risk Level assessments 4/19/2019

QTEC Monitor Quality, Timeliness, Efficiency, and Cycle Time Status Project Level Program Level Instructional Systems Level 4/19/2019

QTEC Monitor Reports (1 of 2) Quality, Timeliness, Efficiency, and Cycle Time Reports Minimum/Maximum Acceptable Levels Goals Actual Status 4/19/2019

QTEC Monitor Reports (2 of 2) Instructional Type Development hours Total Monthly Scheduled Actuals Development hours per hour of instruction ratio Task Type 4/19/2019

Two Year Goals Finish PDT and QTEC Reports Integrate with Employee Timekeeping System Integrate with Earned Value Management Tracking System Stabilize user interface Stabilize database structure Migrate user interface to ColdFusion Convert database to Oracle Require all projects to use MMS 4/19/2019