TOPSpro Special Topics TOPSpro Special Topics Data Detective I: Tracking Your Pre- and Post-test Results

Data Detective I: Tracking Pre- and Post-tests  Pre/post Guidelines  Capturing Tests  Instructional Reports  DIR/Other Data Reports Data Detective II: Data Integrity and Payment Points  DIR and Corresponding Reports  Payment Points Reports  Tale of the Tape: “DIR vs. Payment Points” Data Detective III: Federal Reporting  Federal Tables  Level Completion  Persistence  NRS Performance

Pre-/Post-test Guidelines  Appropriate pre/post pairs  Use of test modalities Basic Features  Test Lister  Duplicate Students  Proxy Wizard Instructional Reports Using Data Reports to Capture Tests  Data Integrity  Class Roster by Last Record  PP Audit/Fed Tables Audit

Appropriate 81R-82R-81R 81R-82RX-81R 82R-82RX-84R Inappropriate 81R-81R-81R 81R-87R 85R-83R 4

5 ABE/ASE: Use Reading or Math ESL: Use Reading or Listening Pre- and post-test pairs must always be from the same test modality.

6 What if a student has two sets of pretest and post-test scores, such as one in reading and one in listening?

7 The pretest and post-test pair with the lowest accurate pretest score determines benchmark attainment. This satisfies the Federal requirement to address the learner’s “highest area of need.”

But what if the scores are the same? If a student’s lowest listening score = 190, and the lowest reading score also = 190, what then???

Ties are resolved as follows:  If the pretest scores for two different modalities are equal, choose the modality with a higher post-test score (in order to maximize gain)  If the post-test scores are also equal, choose the modality with the earliest pretest  For single test scores the earliest one is preferred, in case two or more pretest scores among these 3 modalities are equal.

The Test Lister displays all current tests in your database. Sometimes the obvious is the best place to start.

Use the Advanced Filter icon for more complex sorts.

The Duplicate Student Report lists students who may appear in two or more sites with multiple Student ID’s in use  Students with more than one ID creates several possible problems No matching pre/post pair Inappropriately assigned pre/post-tests Entry Records with no Updates Updates with no Entry Records

The report displays all students who share the same data elements you indicated. Remember: this report does NOT necessarily identify the “duplicate” students – it only shows students with the same data. The “sleuthing" is up to you!!

Use the Merge Wizard in conjunction with the Duplicate Student Report. Once you determine which records are the duplicates, “merge” the records into one using the Merge Wizard. You must be logged in at a SITE level or higher to use this wizard.

The Proxy Wizard allows you to generate Entry Records and Test Records for the current Program Year. It reaches back as far as May 12 th to retrieve applicable records and then gives the created record a date of July 1.

Sure, I use the Proxy, I already did that last July – but how is that going to help me find missing tests?

1.The Proxy will not necessarily help you find “missing” tests, but it may be the cause of some existing pretests without post-tests. Frequently students may attend summer session, intend to continue in the fall, but not return. 2.If this is a problem, you can use the Proxy Wizard to “Remove records.” 3.Be careful when selecting tests to proxy forward. If a student completes a test in May-June, then that test score may be significantly better than what they will score in the Fall after taking off for the summer.

 Class Performance by Competency  Class Profile  Student Test Summary – by Class  Suggested Next Test Level Detail – by Class  Student Gains by Class

Class Performance by Competency displays the collective class results for each CASAS test form.

Class Profile by Competency displays individual test performance in a particular class by test form.

In TOPS 5.0, select class level reports (Class Profile, Class Performance, Class Cumulative Performance) now have the Special Option to display tests for all classes in which the student is enrolled.

Student Test Summary lists students by Agency/Site/Class and shows all tests the learner has completed.

Suggested Next Test Level Detail provides the learner’s entire test history for the reporting period.

Student Gains by Class displays both individual test performance in a particular class and California accountability information together on the same report.

Displays information for students enrolled in WIA Title II programs. Includes 27 issues related to accountability, accuracy and completeness Specific Data Integrity reports:  Data Integrity by Agency  Data Integrity Exceptions  Data Integrity Audit

Items 8 – 10 identify test related information

New Item 27 identifies conservative estimate pretests

What’s the difference between DIR Item #9 and Item #10? They seem the same to me, especially considering the Item Counts are almost identical.

Item Count in #10 reflects the pre/post “pair” as it pertains to Federal Tables logic. In some instances, GED or HSD attainment counts as a valid pair on the Federal Tables. We’ll cover that more in Data Detective III.

Sometimes I get results on these items that don’t make sense. I use the DIR Exceptions, like you always tell me to, but I see names on the list that do have tests, and can’t find others that I know don’t have any.

The most likely reason for a student to unexpectedly appear on Items #8-10 is when a student is enrolled in multiple programs. TOPSpro uses the following hierarchy to assign instructional program for WIA II learners: 1.ESL 2.ABE 3.ASE (GED, HSD) If you have a student in both ESL and ABE, for example, with math tests in the ABE class but nothing in ESL, the DIR will search ESL first, assign that person to ESL, and then notice there are no ESL tests (in reading or listening).

The most likely reason for a student to unexpectedly NOT appear on Items #8-10 is because the DIR only considers WIA II learners WITH an Entry Record. (We’ll cover this in exhaustive detail in Data Detective II.) If you have a student with a pretest but no post-test, for example, and also without an Entry Record, then that student will not be included in the Item Count or Percent. The DIR displays learners with no Entry Record in Item #7a, but those learners will not appear anywhere in any of the DIR reports except in #7a.

I’ve never heard about that Item #27… what is that about???

DIR Item #27 is new. It indicates the number and percentage of learners with a conservative estimate  pretest. Tests with conservative estimate scores are valid as both pre- and post- tests, but if your agency has a high number of these, it may indicate there is a need to adjust your pre/post-testing procedures.

Class Roster by Last Record lists students by class and displays each person’s most recent Entry, Update, and Test Record activity.

In this report you can display students with paired tests, students without any tests, or the default selection, students with tests but no pair. Another valuable feature of this report is you can arrange the results by Last Activity.

The Payment Points Audit displays detailed student by student payment points information, including the exact logic TOPS used to designate the appropriate paired pre/post-test.

The Federal Tables Detail Audit provides a page by page account of the Federal Table logic applied to each individual student in the database, including the precise logic used to determine an appropriate pre/post pair for the Federal Tables.

1.When seeking information on a single student, what tool might be faster than creating a report? 2.Name the term that describes two students sharing the same ID: 3.What term describes one student using multiple IDs? 4.Which report and feature can you use to eliminate known duplicates? 5.Where can I find the average (mean) scale score for students who have taken the same test? 6.DoesTOPS always use the same tests to calculate a student’s preliminary payment point and the final payment point? 7.Which DIR item helps ensure accurate testing? 8.Which report helps you find a student’s last activity? 9.Do the Payment Points, Federal Tables, and Data Integrity reports use the same logic? 10.Why doesn’t the Suggested Next Test Level report tell me which test to administer? 11.If my student’s most recent test was a 83R on June 5 th, can I give him an 83R when we test for the first time in the next program year?

Thanks for participating in Web-based training!!