1. Data Collection Strategies An Overview

2. How Do We Assess ?

3. Data Collection Strategies BEFORE collecting data, your first question should be.... Has someone already collected the information from my sample of interest?

4. Data Collection Strategies Personal interview Focus group Telephone poll Telephone interview Classroom assessment Paper/pencil survey Web survey Database/Warehouse Existing Reports Naturalistic observation Direct ContactIndirect Contact

5. Assessment Instruments Standardized Build Your Own When selecting assessment tools you have a choice …

6. Existing Standardized Instruments Proven item pool Reliability / Validity Clarity of directions Scoring Benchmark comparisons Data analysis Reporting formats Cost Processing Delay Item pool Inappropriate norms Pros Cons

7. Build Your Own Instruments Relevant item pool Stakeholder involvement Theory/Model based Lower Cost Processing control Data analysis control Report format Long development time Local technical expertise Unknown reliability/validity Length Poorly worded items Pros Cons

8. Evaluating Existing Instruments Does the title convey the content to student? Review content of EVERY item by team. Are items relevant to ALL students?

9. Evaluating Existing Instruments Does manual report reliability ? Does manual report validity evidence? How appropriate is normative/benchmark data?

10. Evaluating Existing Instruments Are results timely? Is the cost worth the benefit?

11. Sources for Assessment Instruments Instrument Sources