Education Evaluation: Where Research and Policy Intersect Jonathan Plucker, Director April 8, 2009

2 Topics for Discussion Brief history of CEEP Some Recent Evaluation Projects Lessons Learned

3 CEEP History Formed in January 2004 by merging two smaller centers –Indiana Education Policy Center –Indiana Center for Evaluation Longest year and a half of my professional life

4 The Center Mission is to … Improve education by providing nonpartisan information, research, and evaluation on education issues to policymakers and other education stakeholders. Encourage rigorous program evaluation across a variety of settings by providing evaluation expertise and services to diverse agencies, organizations, and businesses. Expand knowledge of effective strategies in evaluation and policy research by developing, modeling, and disseminating innovative approaches to program evaluation and policy research.

5 Current Capabilities People –~100 people, 70 FTE (+ “highly involved” faculty) –25-30 graduate assistantships, dozens hourly Organization –Director, 2 assoc directors, asst director Facilities –Parts of three floors in Eigenmann

6 Externally-Funded Projects

7 Current and Recent Activities KY and IN 21 st CCLC projects Lev Tech US ED Technical Asssitance Alternative school evals and tech assistance –IDOE & GADOE charters, SyF ERC evaluation, alternative education, Cleveland voucher study, homeschooling IES evaluation

Some Current Projects

9 SYF ERC Evaluation SYF facilitates alternative high schools Comprehensive evaluation of most ERCs Site visits, analysis of existing data Report to BoD well-received, recommendations generally followed Bumpy ride at times …

10 Georgia Charter Evaluation Longshot RFP Great kickoff meeting … … but no follow-through Yearlong project compressed to a month Analysis of existing data Many GADOE staff changes Directly led to legislative changes

11 Indiana Charter Evaluation Read about it in state budget Bumpy kickoff … –Warning from professional association 2-year project compressed to 4 months Analysis of existing data Both foes and advocates deliberately distorted results … –… but that was expected (to a point)

12 SOE Principal Evaluation Study of principal assessments of SOE teacher education graduates Follow-up to graduate survey Phone, , and on-line components Internal politics considerable but manageable –However, they complicate the methodological challenges

What Have We Learned?

1. Evaluation is so much messier than you can possibly imagine …

15 How Research and Policy are Related (in reality) Previous governor, with record surplus, introduces FDK bill with state supt support Ed Roundtable supports FDK in P-16 plan Efforts fail R gov candidate supports FDK Time passes ( ) D gov proposes FDK Everyone supports FDK! Good news, right? Record deficit Public opinion FDK and funding Separated in D controlled House House ways and means hearing House education hearing Full House votes Action moves to heavily R senate Big, closely contested election later in year Several IU faculty testify I testify again (ouch) D gov and R IDOE get it worse Committee sends funding to study committee Education bills from both houses get killed Results: Big, closely contested election Record deficit PO: 60+ to 40- Lots of angry people Several good, stalled bills No FDK Conf comm to reconcile S and H actions Webcamgate House Ds aren’t very healthy FDK passes, funding bill doesn’t Lots of media coverage Public opinion Senate education hearing Everyone supports FDK! Good news, right? Things get ugly (or uglier) A few education groups decide to oppose bill for financial reasons

2. … but so much more important.

3. Ethics are a constant concern. 60+ grants and contracts self-funded projects + ~100 people + dozens of clients + clients’ internal and external political issues = lots of ethical quandries

4. Faculty, esp. senior faculty, are hard to get on board. I don’t blame them at all: Empire building, lack of credit, promotion concerns, etc.

5. Making valuable intellectual contributions in the current context is very difficult … … but hardly impossible.

6. Life would be simpler yet more exciting if everyone used logic models