1. Assessment, Accountability & Ultimate Learning Goals Joan L. Herman NEA Symposia Series Education Goals: Vision and Reality November 15, 2007 National Center for Research on Evaluation, Standards, & Student Testing Graduate School of Education & Information Studies UCLA | University of California, Los Angeles

2. Overview Role of assessment in improving student learning Research on accountability and assessment effects Quality of existing assessments Aiming for the right targets Implications

3. Assessment/Accountability Role in Learning Policy-symbolic Technical data Direct support

4. Socio-Political Role Communicate schools’ responsibilities for learning Motivate attention and improvement Promote data-based decision making Encourage continuous evaluation and improvement Public engagement Cheap fix for policymakers

5. Technical Evidence Role: Evaluate quality Examine changes in performance Analyze strengths and weaknesses/needs Individual students and subgroups Curriculum, schools,programs Identify outliers Evaluate personnel

6. Direct Support for Learning: Formative Assessment Communicate expectations Data for planning and decision making Before instruction During instruction After instruction Power of feedback Motivation

7. Direct Connection in Learning Theory Link to relevant background knowledge Scaffold learning Practice in new context Opportunity for explanation and new connections Meta-cognitive value Student responsibility for learning

8. Research Results: Effects of Accountability Testing Motivates attention Promotes focus on student performance, not process Encourages evidence-based planning Focuses teachers and schools on what’s tested Model what’s and how tested Ignores what’s not

9. Effects of Accountability Testing on At Risk Students Motivates attention to their needs Encourages alignment of curriculum and instruction Stimulates new courses and extra-school opportunities Possibility of unintended negative effects Drill and kill, devoid of rich curriculum Double (++) doses of what’s not working Focus on “bubble” kids? Increase in drop outs?

10. Effects of Accountability on Student Performance Prior NCLB: Strong accountability shows positive effects on NAEP Exit exams pose risk for most at risk Nature of exam may matter

11. Validity of Accountability Tests Strong by traditional psychometric indices Uneven in alignment with standards Tilt toward lower level rather than problem solving skills Sensitivity to instruction unknown Diagnostically gross Generalizability suspect: can students transfer knowledge and skills?

12. Getting the Targets Right Basic skills 21st century skills Respect and fuel individual interests, options, and responsibility

13. Proficiency Demonstrations Curriculum-based opportunities 21st century skills Rigorous application of academic skills Reflect high expectations: rigorously evaluated Customized to local and individual interests

14. Comparable, Academically Rigorous Demonstrations Not Performance Assessment redux Demonstration “templates” that can be embedded/instantiated in various contexts Task requirements Evaluation criteria Performances that serve by existing open to public inspection moderated to achieve common expectations not necessarily a number, aggregated up

15. Un-muddy the Message Assure that students engage in and develop 21st century skills Can they be: Diagnostic? A multiple measure? Used to evaluate students and schools? Provide formative information? Scaled, provide stable measure SURE, BUT that’s not the main point

16. Summary Accountability assessments are expected to serve multiple functions Research shows power in serving one of those functions Getting to ultimate goals for student learning requires that we concentrate on adequately serving that function.

17. www.cresst.orgwww.cresst.org herman@cse.ucla.eduherman@cse.ucla.edu