1. TOPSpro Special Topics Data Detective II: Data Integrity and Payment Points

2. Description: Data Integrity Reports  DIR  DIR Audit  DIR Exceptions Description: Payment Points Reports  PP Summary  PP Exceptions  PP Monitor  PP Audit “Data Integrity vs. Payment Points”

3. Displays information for students enrolled in WIA Title II programs. Includes 27 issues related to accountability, accuracy and completeness Specific Data Integrity reports:  Data Integrity by Agency  Data Integrity Exceptions  Data Integrity Audit

4. The DIR includes Summary Information at the top that highlights the most important data elements.

5. The Item Percent column displays the percentage of times an issue occurred compared to the total number of WIA II Entry records. The DIR displays Item Count column displays the number of times an issue occurred in WIA II Entry Records.

6. Note: The DIR computes Item Count and Item Percent from the total learners with an Entry Record, not the total WIA II learners overall.

7. Items 1 – 10 are important for state and federal reporting Items 11 – 14 are important for NRS core performance reporting Items 15 – 27 are important for data validation and verification

8. I work hard fixing all of these problems that come up on the DIR, but no matter how hard I try, I still can’t get rid of all the problems … Does this mean I’m in trouble?

9. Not all DIR items are errors or omissions: Some issues reflect situations for which you cannot take actions, such as Item 3, Concurrently enrolled learners. Other items are purely informational, and should be used for comparison and evaluation, for example, Item 14.

10. Not all DIR items are errors or omissions: Other issues arise depending on the time of the year, and may not require immediate action, such as Item 7b, Learners with no Update Record.

11. All Data Integrity reports have an Item List tab where you can modify your report to include or exclude specific DIR items.

12. The Data Integrity Audit Report lists all related data issues by student. The DIR Audit includes Student Level Information that highlights the missing information for each student and identifies which record includes the missing data. (E=Entry, U=Update, T=Test)

13. Data Integrity Exceptions lists DIR data issues by Item Number. This example indicates that Autumn River is missing a birth date in her student demographic record.

14. What should I do when I expect a specific student to appear on this report, but for whatever reason the DIR Exceptions does not display that person?

15. The DIR Exceptions displays only those learners with an Entry Record. The Exceptions report will show students who lack an Entry only once, under item 7a. These students may have other DIR items that are not displayed until the Entry Record is entered into TOPS.

16. TOPSpro Payment Points reports display California accountability information for CA WIA-funded agencies. Each PP report uses the same rules to determine payments, but each differs in the amount of detail it displays. Specific Payment Points reports:  Payment Points Summary  Payment Points Totals  Payment Points Monitor  Payment Points Exceptions  Payment Points Audit

18. Students who generate a significant gain may also appear in other benchmark total cells. In this example, the Two Level benchmarks are associated with two students who made a significant gain. Students will be allocated to only one Program for benchmark evaluation. Example: the students in ASE will not appear in any other program on this report.

19. Areas filled with hash marks represent Payment Points not associated with an instructional program or focus area. Example: students enrolled in EL Civics may earn significant gain benchmarks, but the achievement will be apportioned to the ESL program.

20. Total # of Enrollees Overall is the number of unduplicated learners. In this example, the total of all students in 231-funded programs = 273. Three of these students are also enrolled in Citizenship Prep.

21. In the 231 section, students are placed into only one program. In this example, a student was moved to Family Literacy, with no effect on the unduplicated enrollment total.

22. Number Selected = 313 Number included = 273 Number Dropped = 40 Number Selected = Learners with activity (Entry, Update, or Test) in a WIA Title II program Number Included = Those without a drop reason

23. Number Selected – Number Dropped = Number included in report 313 - 40 = 273

24. Payment Points Summary – Run Statistics Aren’t you overlooking the 105 dropped students from the EL Civics section?

25. Dropped from EL Civics Section: Does not affect payment point calculations in 225/231 (non-EL Civics) areas Dropped from 225/231: Does not affect payment point calculations in EL Civics areas

26. The Payment Points Monitor report details payment points information by student. This example shows a student who has met the instructional hours and learning gains criteria, but is dropped due to no update record (drop code 7).

27. The Payment Points Exceptions report identifies students who are dropped from the Payment Points reports.

28. Click “Show only learners with attainable PP’s” under Special Options to narrow the scope of this report.

29. The Drop Reasons tab on Report Setup provides the user a way to create a preliminary report that is limited to selected drop reasons. You may want to use this feature to help focus attention on specific issues.

30. The Payment Points Audit displays detailed student by student payment points information, including the exact logic TOPS used to derive number of payment points earned.

31. Payment Points may still be awarded in 231 programs, even if dropped from EL Civics qualification.

34. What I like to do is place the DIR side by side with the Payment Points report and compare. What’s frustrating is sometimes the results are different, even though some things appear to be showing exactly the same thing, such as “Less than 12 hours of Instruction.”

35. The DIR should not be used as part of Payment Point estimation. Item Count and Percent are derived from students whose data set include at least one Entry Record within the program year and assigned to a WIA Title II program. There may be students who have met all Payment Point criteria but are not included in the report’s detail section due to lack of an Entry.

36. The DIR Exceptions and Audit should not be used for Payment Point estimation Students listed are only those whose data set includes at least one WIA Title II Entry Record within the program year 1.There may be students who have met all Payment Point criteria that are not in the report due to lack of Entry record. 2.The DIR reports are a “pure count” of issues, meaning a student may be listed more than once, with the exception of students listed under Item 7a.

37. Payment Points hierarchy is used to place the student in the lowest appropriate instructional program or focus area. The first associated drop reason will be used to eliminate the student from further payment point consideration or inclusion on summary reports.

38. The DIR Audit includes students who do not have an Entry, but they only appear under item 7a. The same students may appear on the Payment Points reports, with additional Drop Reasons. The Payment Point Monitor, Summary, and Totals reports will not include all issues or drop reasons for a given student, due to hierarchy logic.

39. I was following you perfectly until you finished off with that “hierarchy logic” business…

40. Payment Points hierarchy is used to place the student in the lowest appropriate instructional program or focus area. When determining focus area placement, TOPSpro uses the following hierarchy: 1.Citizenship Preparation 2.Civic Participation 3.225/231

41. When determining placement for instructional program, TOPSpro uses the following hierarchy: 1.ESL 2.ABE 3.ASE

42. The Payment Point and Data Integrity reports may not display the same information.

43. Data Integrity ExceptionsPayment Point Exceptions

44. The Payment Point Monitor, Summary, and Totals reports will not include all issues or drop reasons for a given student, due to hierarchy logic.

45. I think I understand these differences between Payment Points and the DIR… but if the Payment Points reports are so much more reliable in identifying the data we need to correct in order to earn Payment Points, why are we required to submit the DIR?

46. The Data Integrity Report includes a lot of information that may not directly relate to Payment Points—testing information, core performance, data requirements that may not directly cost agencies Payment Points but are nonetheless requirements, plus other issues that may not be critical to Payment Points attainment, but that are important in meeting Federal reporting requirements… which we will talk about a whole lot more in Data Detective III.

47. Data Integrity reports track multiple issues for a single student, but only for WIA II learners with an Entry Record. A Payment Points report eliminates a student from consideration as soon it discovers a drop reason, thereby not providing a duplicate count – but the report will track any WIA II learners with an Entry, Update, or Test Record.

48. 1.What is the difference between duplicated and unduplicated count?______________________________________________________________ 2.On the D.I. reports, which item numbers relate to payment points? _______ 3.Does the DIR Exceptions display duplicated information ? _______ 4.On the DIR by Agency, can a student be associated with more than one item?_____ 5.On the DIR Audit, what do the designations refer to, e.g. (E). ________________ 6.A preliminary report is one that may not include all items or drop reasons; 1.Should you use a preliminary report for Quarterly/EOY reporting? ________ 2.What feature can you use to create a preliminary report? _________ 3.What item might you exclude when running the report on Sep. 30 th ______ 7.On Payment Point reports, can a student’s benchmark achievements appear in both ABE & ESL? _____ 8.Can a student’s payment point achievements appear in both Adult Secondary Education and EL Civics? ____ 9.Can a student appear on a payment point report but not on a DIR ? ______ 10. If a student is dropped from EL Civics, does that eliminate his other achievements? 11. Under what circumstance will you see a payment point labeled “Attainable?” 12.What report feature helps you focus on dropped students who have payment-point potential?

49. Thanks for participating in Web-based training!!