1. Joint ILS Manager Tools Course Development T.R. “Randy” Pilling Program Director, Acquisition Logistics (703) 805-2277 thomas.pilling@dau.milthomas.pilling@dau.mil September 12, 2005

2. Joint ILS Manager Tools Course Development DAU contacted by LOGSA (Bill Edwards) seeking endorsement, partnership, co-teaching in course development and delivery DAU also aware of LMI support to Navy regarding development and delivery of ILS courses First coordination meeting 4 Aug 05: -Consensus on need, requirements and content -Assignment specific, level 200 course -Agreed to share coordination & control in course development -Level of commitment to get it done -Course objectives and development timelines in work— potential deployment 1Q FY 07 -Manpower and funding provided by LOGSA

3. ILS Manager Tools Course Preliminary Schedule

5. Metrics-That-Matter (MtM) Logistics & Sustainment Course Metrics T.R. “Randy” Pilling Program Director, Acquisition Logistics (703) 805-2277 thomas.pilling@dau.milthomas.pilling@dau.mil September 12, 2005

6. DAU Metrics-That-Matter (MtM) Metrics that Matter (MTM) Survey System serves as the standard application DAU-Wide for recording of customer support for both students and special customer requested projects. Data is collected at the end of or immediately following each course Survey questions cover areas relating to: -Instructor -Courseware -Learning Effectiveness/On-Line Learning -Job Impact -Environment

7. Report Card Grading Other includes: Courseware Learning Effectiveness Environment Job impact Online Delivery (where applicable) MtM 7 Point Scale Highly Satisfied Highly Satisfied Dissatisfied Highly Dissatisfied Highly Dissatisfied Neutral 1 2 34567 6.0: DAU Instructor rating 5.2: DAU Instructor rating 5.6: DAU other category ratings 5.2: DAU other category ratings Satisfied

8. Instructor Courseware Environment Online Delivery Learning Effectiveness Job Impact All Courses: 2 nd Quarter Summary 48 00 64 7 0 41 43 16 7 0 65 6 0 31 291 71 courses: 48 residence, 23 DL

9. Instructor Courseware Environment or Online Delivery Learning Effectiveness Job Impact 101102201A201B203204 LOG: 2 nd Quarter Summary

10. Instructor Courseware Environment or Online Delivery Learning Effectiveness Job Impact 235A235B304 LOG: 2 nd Quarter Summary

11. Program Mgmt Curriculum Ratings by Course

12. Instructor Courseware Environment or Online Delivery Learning Effectiveness Job Impact ACQACQACQPMTPMTPMTFE 101201A201B250352A352B201 PMT/ACQ/FE: 2 nd Quarter Summary

13. Engineering & Technology Curriculum Ratings by Course

14. Instructor Courseware Environment or Online Delivery Learning Effectiveness Job Impact IRMIRMIRMSAMSAMSAMPQM 101201301101201301101 E&T: 2 nd Quarter Summary

15. Instructor Courseware Environment or Online Delivery Learning Effectiveness Job Impact PQMPQMPQMPQMPQMPQMSTM 104201A201B202212301201 E&T: 2 nd Quarter Summary

16. Instructor Courseware Environment or Online Delivery Learning Effectiveness Job Impact STMSYSSYSSYSTSTTSTTST 302201A201B301101202301 E&T: 2 nd Quarter Summary

17. DAU Strategy Identify issues/inconsistencies with MtM results Review survey questions Determine applicable data filters Review processes for data collection/allocation Develop standardized procedures for collecting, filtering and reporting accurate data Continue to review MtM results/progress at executive level and incorporate data into faculty/staff assessment reports Implement “Data Mart” tool for fusion of MtM, travel, timekeeping and financial data systems

19. BACKUP

20. LOGISTICS CURRICULUM Ratings by Course

21. MTM LOG 101 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Courseware n=795n=1,705 Online Delivery n=794n=1,696 Learning Effectiveness n=768n=1,654 Job Impact n=801n=1,681 5.855.89 5.69 5.70 5.89 5.90 5.38 5.36

22. MTM LOG 102 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Courseware n=1,043n=1,594 Online Delivery n=1,039n=1,588 Learning Effectiveness n=1,009n=1,541 Job Impact n=1,019n=1,567 5.855.84 5.75 5.73 5.90 5.81 5.33 5.31

23. MTM LOG 201A Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Courseware n=261n=688 Online Delivery n=261n=683 Learning Effectiveness n=249n=654 Job Impact n=255n=678 5.925.86 5.93 5.86 5.90 5.31 5.17

24. MTM LOG 201B Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=343n=630 Environment n=341n=622 Courseware n=344n=630 Learning Effectiveness n=341n=627 Job Impact n=340n=624 6.616.64 6.056.12 5.97 6.02 5.50 5.48

25. MTM LOG 203 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Courseware n=42n=80 Online Delivery n=14n=78 Learning Effectiveness n=41n=78 Job Impact n=42n=80 6.146.15 6.07 6.25 6.04 6.02 5.50 5.55

26. MTM LOG 235A Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Courseware n=266n=791 Online Delivery n=265n=588 Learning Effectiveness n=257n=766 Job Impact n=263n=778 5.975.85 5.88 5.72 5.95 5.82 5.36 5.26

27. MTM LOG 235B Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=298n=508 Environment n=291n=500 Courseware n=298n=508 Learning Effectiveness n=296n=504 Job Impact n=291n=499 6.606.68 6.046.05 6.046.20 6.09 6.13 5.47 5.55

28. MTM LOG 204 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=46n=72 Environment n=46n=72 Courseware n=46n=72 Learning Effectiveness n=45n=71 Job Impact n=45n=71 6.666.74 5.835.41 6.156.24 6.13 6.19 5.93 6.11

29. MTM LOG 304 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=101n=212 Environment n=99n=205 Courseware n=101n=212 Learning Effectiveness n=99n=209 Job Impact n=100n=211 6.736.75 6.166.08 6.276.20 6.26 6.18 5.88

30. Program Mgmt Curriculum Ratings by Course

31. MTM ACQ 101 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Courseware n=3342n=6793 Online Delivery n=3330n=6765 Learning Effectiveness n=3279n=6672 Job Impact n=3305n=6746 5.94 5.86 6.07 5.12 6.09 5.95 5.88 5.45

32. MTM ACQ 201A Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Courseware n=1883n=3618 Online Delivery n=1876n=3600 Learning Effectiveness n=1814n=3512 Job Impact n=1848n=3572 5.75 5.54 5.85 5.25 5.84 5.74 5.53 5.24

33. MTM ACQ 201B Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=1513n=3309 Environment n=1493n=3265 Courseware n=1511n=3307 Learning Effectiveness n=1498n=3290 Job Impact n=1499n=3275 6.68 5.82 6.12 6.21 5.53 6.67 5.77 6.10 6.17 5.58

34. MTM PMT 250 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=32n=114 Environment n=0n=44 Courseware n=32n=114 Online Delivery n=32n=68 Learning Effectiveness n=30n=111 Job Impact n=32n=113 6.53 6.69 5.14 5.935.79 5.59 5.62 5.73 5.57 5.73 5.29

35. Engineering & Technology Curriculum Ratings by Course

36. MTM STM 201 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=63n=86 Environment n=63n=86 Courseware n=63n=85 Learning Effectiveness n=63n=86 Job Impact n=63n=86 6.31 4.91 5.46 5.52 5.35 6.33 4.81 5.42 5.41 5.33

37. MTM STM 302 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=35n=51 Environment n=35n=51 Courseware n=35n=51 Learning Effectiveness n=35n=51 Job Impact n=35n=51 6.40 5.35 5.62 5.81 5.44 6.34 5.63 5.50 5.83 5.32

38. MTM SYS 201A Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Courseware n=59n=101 Online Delivery n=59n=99 Learning Effectiveness n=58n=97 Job Impact n=59n=100 5.59 5.33 5.49 5.10 5.52 5.32 5.52 5.09

39. MTM SYS 201B Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=446n=879 Environment n=444n=878 Courseware n=446n=881 Learning Effectiveness n=439n=870 Job Impact n=443n=873 6.46 5.79 5.69 5.62 5.23 6.39 5.74 5.57 5.50 5.11

40. MTM SYS 301 Report Card 2 nd Qtr CategoryCurrent Qtr Rating Cumulative FY Rating Instructor n=324n=576 Environment n=322n=570 Courseware n=324n=575 Learning Effectiveness n=321n=573 Job Impact n=321n=571 6.43 5.60 5.65 5.62 6.40 5.66 5.62 5.60 5.295.34

41. Report Card Grading (survey questions)

42. Report Card Grading (survey questions)

43. Report Card Grading (survey questions)

44. 7 6 5 4 3 2 1 n/a The content was divided into small enough units to         allow for sufficient learning. 7 6 5 4 3 2 1 n/a The electronic user interface was friendly and         easy to navigate. 7 6 5 4 3 2 1 n/a The online delivery had the right level of user         interaction/involvement. 7 6 5 4 3 2 1 n/a The graphics and illustrations used were meaningful         and within context. 7 6 5 4 3 2 1 n/a The instructor was helpful. (web-based only)         7 6 5 4 3 2 1 n/a This delivery method was an effective way for me         to learn the material. 7 6 5 4 3 2 1 n/a The online help desk was beneficial.         7 6 5 4 3 2 1 n/a The exercises or knowledge reviews added value         to my learning. (facilitated online courses only On-line Survey Questions