1. Evaluating School Quality evidence based on test scores
Richard Lambert, Ph.D.

2. Standards for Accountability Systems
Setting Reasonable Targets
Symmetry
Fairness
Signal vs. Noise

3. Setting Reasonable Targets
Status
Growth
Value-Added
Progress Toward Full Proficiency (AYP)

4. Setting Reasonable Targets
These methods will not always identify the same schools as “high quality”
Each type of target provides different information
Ohio has moved to a balanced system of accountability that uses all four types

5. Symmetry Are both students and teachers held accountable?
Is there balance between student and teacher responsibility for learning?
Does the accountability system give incentives to both groups to perform?

6. Fairness
Is it fair to students and parents to judge schools as being effective if the students lack the skills and knowledge to be successful at the next level?
Is it fair to teachers and administrators to judge schools as ineffective when the students do not arrive ready to learn?

7. Fairness Opportunity to Learn Opportunity to Teach
Neither one is easy to evaluate

8. Signal vs. Noise
What is the quality of the information that the testing program provides?
Is it reliable and valid?
Is it reliable and valid given all the ways in which the scores are used?

9. Status vs. Improvement

10. Evaluating and Developing Teachers
Successfully developing the teacher workforce is more like which of the following:
A. Carefully crafting and aging fine wine
B. Mixing Kool-Aid
C. Buying your favorite beverage

11. Extending the Analogy
Variety
Vintage
Appellation
Blending

12. Extending the Analogy Preparing the soil Selecting the right grapes
Learning how to nurture the vines year after year
Making, aging and blending the wine

13. Extending the Analogy

14. Extending the Analogy

15. Bringing It All Together
Determining the Merit of Value of a Program
Judging a Program against its own Pretenses
Building an Argument for Specific Inferences about a Program based on Evidences

16. Bringing It All Together