1. UCEDD Response to DDPIE June 1, 2007
Double Tree Eleven
Purpose: to summarize the small group feedback on the DDPIE evaluation plan

2. UCEDD PM Work Group for DDPIE Responses--Purpose
Primary audience for this evaluation is OMB, secondary audience may be Congress, programs, public
Need to clarify what is needed for the PART process—keep this evaluation focused on that requirement
Evaluate DD Act programs—this feedback focuses on the UCEDD network

3. UCEDD PM Work Group for DDPIE Responses--Approach
Recommendations:
Base the evaluation on the DD Act; to the extent possible, be cognizant of the work on reauthorization
Align the evaluation with compliance with DD legislation (“are programs doing what they are funded to do?”)
Evaluate systems impact of the network (“is the legislative intent of the DD Act being accomplished?”)

4. UCEDD PM Work Group for DDPIE Responses--Approach
Clarify what qualifies as “independent” evaluation for OMB (e.g., recognize that independent evaluation may only require primary data collection to answer those questions for which you don’t already have reliable, valid data)
In keeping with OMB policy, minimize the burden of data collection on grantees—use existing data sources that we have already invested in (e.g., MTARS, NIRS, annual progress reports).

5. UCEDD PM Work Group for DDPIE Responses--Approach
The present proposal (scope and cost) seems disproportionate to the requirement
Use a criterion-based, stratified sampling method of approximately 20-30% of UCEDDs

6. UCEDD PM Work Group for DDPIE Responses--Methods
Add rigorous, qualitative evaluation to tell the story (e.g., multiple case studies, document analysis, focus groups, key informant interviews)
Ensure transparency of the process at all levels—communication on steps all along the way
Validation needs to occur sooner in the process

7. UCEDD PM Work Group for DDPIE Responses--Methods
Assessment of collaboration needs to include collaboration beyond other ADD-funded partners (i.e. leveraging and systems change requirements of the DD Act)

8. UCEDD PM Work Group for DDPIE Responses--Measures
Reduce the number of measures (benchmarks and performance standards); recommend two for each core function and one for collaboration—select those applicable to most UCEDDs
Eliminate measures of structure and process
Develop data definition for each measure and evaluate it in terms of applicability of the majority of the network

9. UCEDD PM Work Group for DDPIE Responses--Measures
In developing the benchmarks, use consistent language and framework (e.g., “the network of UCEDDs…”)
Focus now on design and benchmarks—wait until there is agreement on benchmarks before reviewing indicators

10. Next Steps
Share recommendations with UCEDD Directors and obtain consensus
Discuss recommendations with Westat and ADD
Support ongoing work on Benchmarks, Indicators and Performance Standards