1. MONITORING FOR DESCRIPTION Performance Monitoring and Evaluation College of Public and Community Service University of Massachusetts at Boston ©2006 William Holmes

2. 2 DESCRIPTIVE ISSUES: 1 What do you need to describe? How do you want to describe? What are the units of analysis for your description?

3. 3 DESCRIPTIVE ISSUES: 2 What are the strengths and limitations of these procedures? How may the descriptive procedures be improved or changed?

4. 4 WHAT DO YOU NEED TO DESCRIBE? PART 1 Intake/Goal Status/Participant and Environmental Characteristics Services/Interventions/Resources Activities/Events/Processes Output/Goal Status/Participant Characteristics

5. 5 WHAT DO YOU NEED TO DESCRIBE? PART 2 Actions/Contacts/Tracking/Processing Compliance/Accountability/Standards Adherence/Requirement Documenting Quantitative Data Collection Qualitative Data Collection Knowledge Development

6. 6 HOW DO YOU WANT TO DESCRIBE? Numbers and Percentages Graphic charts Written narratives Media presentation

7. 7 WHAT ARE THE UNITS OF ANALYSIS FOR DESCRIPTION? Clients Transactions Events Services/activities Service workers Programs

8. 8 WHAT ARE THE STRENGTHS AND LIMITATIONS OF THIS DESCRIPTION? PART 1 Data related to logic model Elements missing from logic model Reliability, validity, completeness, and timeliness of data Purposes served by existing data Problems in data collection

9. 9 WHAT ARE THE STRENGTHS AND LIMITATIONS OF THIS DESCRIPTION? PART 2 Privacy protection issues Duplicate records Missing records Multiple records Matching records

10. 10 WHAT ARE THE STRENGTHS AND LIMITATIONS OF THIS DESCRIPTION? PART 3 Service data linkage to outcomes Moderating influences on outcomes Resource limitations Standards for comparison Problems of goal displacement

11. 11 DATA AND INFORMED CONSENT ISSUES Information to be provided Uses of information Access to information Client rights Risks and benefits Sunset provisions

12. 12 QUANTITATIVE AND QUALITATIVE DESCRIPTIVE ISSUES: 1 Characteristics as measures Standards for evaluating Tables, charts, & narrative Focus and consistency

13. 13 QUANTITATIVE AND QUALITATIVE DESCRIPTIVE ISSUES: 2 Good examples Deviant cases Telling a story Comparing and contrasting Summarizing

14. 14 IMPROVEMENTS/CHANGES: PART 1 Logic model measures refined Goal measures improved Unneeded data dropped New uses of existing data

15. 15 IMPROVEMENTS/CHANGES: PART 2 Procedures for more complete data Procedures for more timely data Procedures for better presentation of data Privacy and informed consent improvements

16. 16 IMPROVEMENTS/CHANGES: PART 3 Tying services to outcomes Linking risk factors and auxiliary services Better time sequencing of data Use of comparison standards

17. 17 DESCRIPTIVE EXERCISE: HOLLOWEEN PARTY Goals Value assumptions Measures When and how collected by whom How information used Limitations