Employment Ontario Literacy and Basic Skills Performance Management Reports Training For Service Providers

2 Evaluations Complete the first two pages and put them in the back of your participant guide. You will work with them at the end of each unit and at the end of the day.

3 Introductions Name Position Organization

4 Key Message Today’s training supports the LBS third- party network to build capacity by way of report analysis and by improving the quality of the reports (data integrity).

5 Target Audience Service Provider staff Especially those who collect data, or input data into CaMS, or perform report analysis and continuous improvement.

6 MTCU Expectations for and Beyond Achieve contracted commitments Integrate continuous improvement Collect quality site level data

7 Purpose of the Training Importance of data integrity Business Intelligence How data is used to evaluate site performance and make decisions Importance of performing service delivery analysis

8 Unit 1 Performance Management System

9 Unit 1 Objectives Who is at the centre of LBS System ‘Above and below the waterline’ Performance Management System What Business Intelligence Entails

10 Above and Below the Waterline

11 Components of a PMS

12 Continuous Improvement

13 Unit 1 Objectives Review Customer is at the centre of LBS System Solid business foundation supports service delivery PMS components and components of components Business Intelligence: technology; business processes; people

14 Unit 2 LBS Program Services And Client Path

15 Unit 2 Objectives LBS Client Path Service Quality Standard Importance of data integrity Where report data is input into CaMS

Client Path Diagram (top portion) 16

17 Service Quality Standard (SQS)

18 Core Measures Table No data. 1. Customer Satisfaction 2. Service Coordination 3. Suitability 4. Learner Progress 5. Learners Served Why is this core measure important? What data integrity issues must be anticipated related to this core measure? What strategies could the service provider implement to increase performance in this area? 2

19 Service Quality Standard (SQS) Calculation Perform Unit 2 - Exercise 2 ‘YTD All Participants - Actual’ X ‘Service Quality Weight’ X 10 (to the maximum SQS value allowed)

20 Exercise 2 Answers SERVICE QUALITY ‘Actual’ column value for: Customer Satisfaction:100% X 15% X 10 = 1.50 Service Coordination: 83% X 25% X 10 = 2.08 Suitability: 26% X 20% X 10 = 0.52 Learner Progress:100% X 30% X 10 = 3.00 Learners Served:175% X 10% X 10 = 1.75MAX 1.00 Overall SQS value = 8.10

21 Unit 2 Objectives Review Clients, linear flow through program SQS supported by core measures SQS calculation Data integrity affects report data Connect input to reports:  Table matched inputs to reports  Report user guides  Data Dictionary

22 Unit 3 Data Integrity

23 Unit 3 Objectives Definition of ‘data integrity’ Business owners of the data Importance of timely data collection Management support of staff Value of orderliness

24 What is Data Integrity? Includes: - collection of data - entry into EOIS-CaMS - paper documentation Regarding each of the above: - complete, consistent, timely, and accurate

25 Business Owners of the Data Service providers as owners: o verification and integrity o value of data is only as good as the input o link cause and effect

26 Two Groups use the Data of the PMS MTCU Service providers

27 Data Collection Service providers: o collect and report data (EOIS-CaMS) o maintain quality assurance systems o support data with documentation MTCU and service providers: o analyse data

28 Orderliness Consistent and clear processes = Resources used efficiently

29 Unit 3, Exercise 1 ‘Four Unusual Numbers’ LBS CM # SERVICE QUALITY CORE MEASURES LBS Prov. Target Perf Com Ann. Target YTacipants Num YTD al Participants Den YTD all Participants Actual % YTD of Target Learner % 2Service Coordination50% 0 450% 3Suitability30% %42% 4Progress60% %167%

30 Unit 3 Objectives Review Data Integrity: complete, consistent, timely, accurate data collection and entry into CaMS Service providers are business owners of site data (collect quality data) Timely data collection supports evidence-based decision-making Staff: collect and enter data (CaMS) Management: stress data integrity Orderliness increases efficiency & quality

31 Unit 4 Report Analysis

32 Unit 4 Objectives Reports available Analyse LBS data What drives continuous improvement Support of staff: report analysis and continuous improvement

33 Learners Served – Pro-Rated Denominator ‘YTD All Participants – Actual’ for Learners Served Performance Commitment Annual Target = 105 YTD All Participants – Numerator= 65 Months in the report dated December 31= 9 months Annual Target - Pro-Rated: 105 X (9/12) = / 79 = 82%

34 Learners Served – Pro-Rated Denominator (1)

35 Learners Served – Pro-Rated Denominator (2)

36 Continuous Improvement (2)

Client Path Diagram (re-visited) 37

38 Report 61 SORTED by: 1. Program Status; 2. Program Duration

39 Report 61 SORTED by: Program Status

40 Report 61 SORTED by: 1. Program Status; 2. Referred In

41 Report 61 SORTED by: 1. Program Status; 2. Goal Path

42 YTD All Participants Actuals Report 64 DSQR Calc 1 - Excluding ‘3. Suitability’ and ‘5. Learners Served’: YTD All Participants Num / YTD All Participants Den Calc 2 - ‘3. Suitability’: YTD All Participants Num / YTD All Participants Den / 10 Calc 3 - ‘5. Learners Served’: YTD All Participants Num / Perf Com Ann. Target (pro-rated YTD)

43 ‘% YTD of Target’ Calculation Report 64 DSQR YTD All Participants Actual / Performance Commitment Annual Target

44 Continuous Improvement (3)

45 Unit 4 Objectives Review ‘Reports Available’ section Report analysis:  Calculations; Report 61; exercises Continuous improvement:  Accurate reports  Understand results Support of staff:  Participant Guide; user guides, Data Dictionary

46 Summary Training Summary Evaluation Exercise

47 End