UCLA Graduate School of Education & Information Studies National Center for Research on Evaluation, Standards, and Student Testing Accountability and Assessment In the Service of Learning: Is Public Interest Being Served? Joan L. Herman The Future of Test-Based Educational Accountability: Festschrift in Honor of Robert L. Linn Los Angeles, CA - January 23, 2007 If you choose to use this title slide, simply delete the previous slide (the one-line title version). This will be slide 1 of your presentation.

2/20 Overview Public Interest and the Role of Accountability Effects on Teaching and OTL Effects on Performance Implications The section titles here should be reflected in the blue section dividers throughout your presentation.

Public Interest and the Role of Accountability The section title here should be reflected in the OVERVIEW slide of your presentation.

4/20 What is the Public Interest? Transcendent concerns for the good of society (Reich, 1988) Right to “life, liberty and the pursuit of justice” carries obligations and conflicts (Hochschild and Scovronick, 2004) Messy dilemmas: Whose values and strategies are transcendent (Stone, 2002)? What happens when values conflict? How many and who needs to be served? Are schools preparing students to serve the public interest? When pasting text from another document, do the following: 1.Highlight the text you want to replace 2.Go to the EDIT menu and select PASTE SPECIAL 3.Select “Paste as: UNFORMATTED TEXT”

5/20 The Accountability Model: Bureaucratic Establish standards Use assessments to communicate, motivate, provide feedback Add incentives/sanctions Educators listen and respond Students engage and learn more

6/20 Theory of Action Coordinate and align resources relative to specified goals Improve opportunities to learn Improve learning, reduce achievement gaps Accountability is a teaching and learning problem, NOT primarily a testing problem

Evidence on OTL Effects The section title here should be reflected in the OVERVIEW slide of your presentation.

8/20 The Good News on OTL State assessments focus instruction Educators work to align curriculum and instruction Educators more attuned to data and using it Growing interest in formative assessment At risk students get new, augmented opportunities

9/20 The Bad News on OTL Effects Instruction is aligned -- with test, not standards What is not tested can disappear Curriculum can narrow and B-O-R-I-N-G Practice can get distorted: focus on triage, “pushables” and “slippables” Rhetoric outstrips quality of practice

Effects on Performance The section title here should be reflected in the OVERVIEW slide of your presentation.

11/20 Performance Effects Prior 2001 NAEP data shows positive effects Effects of social promotion Overall positive Negative for most vulnerable High school exit exams show mixed effects Nature of assessment matters When pasting text from another document, do the following: 1.Highlight the text you want to replace 2.Go to the EDIT menu and select PASTE SPECIAL 3.Select “Paste as: UNFORMATTED TEXT”

12/20 Performance effects since 2001 Quality Counts shows modest, positive relationship between state accountability and assessment and NAEP NAEP data shows modest improvements generally for disadvantaged youth Students who quality for free lunch Students in large urban school districts

13/20 Effects on Special Populations Validity of data for EL’s and SWD a problem Both groups most likely to be adversely affected by high school exit exams Yet advocates cite benefits of visibility Effects of accountability on high ability students less visible A single test can’t motivate or well measure all parts of the distribution.

14/20 Effects on Teachers Survey results consistently show dissatisfaction Expectancy theory suggests that unrealistic goals are counter productive Empirical data show negative effects on teacher retention for low performing students

Is/Can Accountability Serving the Public Interest The section title here should be reflected in the OVERVIEW slide of your presentation.

16/20 YES, BUT…… Educators are trying to improve curriculum and instruction Educators are paying more attention to all students Modest effects on performance BUT, danger for our most vulnerable BUT, good intentions can yield curriculum deadening and discourage educators BUT, more assessment doesn’t mean good assessment When pasting text from another document, do the following: 1.Highlight the text you want to replace 2.Go to the EDIT menu and select PASTE SPECIAL 3.Select “Paste as: UNFORMATTED TEXT”

17/20 We Know How to Do Better Tests that support learning Integrated instructional and assessment systems Learning from existence proofs of “break the mold” teachers and schools Establishing realistically, aspirational targets Leaving our silos to work synthetically across the system

18/20 How Much Can Accountability Do? Only modest improvements in for low achieving students Accelerated trajectories needed -- but not generally occurring Faulty assumptions: That educators know how to or can find ways to accelerate learning for those who need it That existing R&D has the answers That the solution to any problem is another test

19/20 A Modest YES, but many needs Assure that accountability tests are measuring the right stuff: less is more Safeguards against curricular distortions and deadening Application/synthesis of existing knowledge rather than re-creating old mistakes Invest in new, cross-disciplinary knowledge to fuel transformative tools and strategies

20/20 All to be accountable and play their role

21/20 next presentation Joan Herman voice fax If you choose to use this end slide, simply delete the previous slide (with no contact information). ©2006 Regents of the University of California