Making self review work for you Julie Foley & Dave Shepherd Education Review Office October 2010

ERO self review presentation2 Talk outline Recapping concepts and activities from the 2009 presentation to the CPPA Schools’ self review and ERO’s 2010/11 reviews - how they can complement each other. Continuing to build evaluation capacity in the sector – ERO’s strategic theme

Evaluation Capacity Building ERO’s current thinking about internal and external evaluation Ko te Tamaiti te Putake o te Kaupapa

ERO’s conceptual framework Complementary evaluation takes key aspects of internal and external evaluation, and accountability and improvement, and adjusts these to the relevant context. It is informed by other evaluation theories and aims to resolve the conundrum raised by Nevo (2000, p.28) “Everybody seems to hate external evaluation while nobody trusts internal evaluation”. 4

ERO’s Reviews 2010 An evaluation approach that balances external ERO review with school self review according to each school’s circumstances. 5 ERO REVIEW SCHOOL SELF REVIEW Schools experiencing difficulties with very limited self review Schools sustaining high quality performance and continuous improvement through effective self review Schools operating well with established processes of self review

ERO self review presentation6 Self review is… Self review involves investigating evidence about student outcomes and current ways of doing things to find out where improvement is needed. Planning for school improvement requires schools to set goals and targets for better student outcomes and make changes that are necessary to bring about those improvements. (MoE Planning and Reporting, 2003)

ERO self review presentation7 Or… …the deliberate and on-going process of finding out how well our practice enhances children’s learning and development. Review allows us to see which aspects of our practice are working well and what we could do better. As a result we can make decisions about what to do to improve. Through review our practice is transformed, and ultimately, our children’s learning benefits. (MoE, ECE, 2006)

ERO self review presentation8 Or… Kura are most successful when there is an robust process of on-going internal review that leads to further improvement. Effective internal review focuses on the key goals for students and the evidence of the extent to which they are actually being achieved…[It is] rigorous, informed by excellence in evaluation practice, with findings based on evidence… (ERO, Te Aho Matua, 2008)

ERO self review presentation9 Or… Self evaluation will collect and analyse a combination of qualitative and quantitative data that helps leaders and teachers develop themselves, their practices, promote high performance, help support student progress and develop intervention programmes from a proactive perspective. (Paphitis, n.d.) Self evaluation is a dialogic process, a meeting of hearts and minds, a forging of new ways of doing, vital and continuing because it is the core of a school’s educational life. It is the essence of the learning community, the intelligent school, schools that learn. (MacBeath, 1994)

ERO self review presentation10 Summary. Self review is: Centred around giving all students the most successful educational experiences possible Set in a collaborative culture of continuous self reflection and constructive critique Self initiated by a desire to seek constant improvement – mostly planned but sometimes spontaneous Celebrating and sustaining what is working well while striving to improve what is not A willingness to embrace challenge and change Setting long term goals and mid-term targets and gathering rigorous and wide-ranging evidence to examine the extent to which these have been achieved

ERO self review presentation11 Different types of reviews but all linked to the big picture

ERO self review presentation12 Balancing accountability and improvement Complementary evaluation Eg Assurance statements Eg External audit Eg Self review Eg ERO education review INTERNAL EXTERNAL ACOUNTABILITYIMPROVEMENT

ERO self review presentation13 Turning data into evidence… Schools gather lots of data – ‘hard’ (quantitative/ numerical) and ‘soft’; (qualitative/narrative). The raw or aggregated data is only the first level of analysis. It describes ‘what is’ or ‘what is happening’. The second level of analysis takes data and turns it into information. It explains ‘what this means’. It allows schools to make key statements or comparisons. The third level uses this information as evidence to support judgments (how well/to what extent), to make decisions (if this so then we need to…) and to prioritise (the most compelling need is…).

ERO self review presentation14 On-going review cycles… On-going review cycles… Considering Informing Monitoring Implementing Organising

ERO self review presentation15 What are we learning from our workshops? Schools do it differently:

ERO self review presentation16 What are we learning from ERO’s good practice in self review case studies? Importance of leadership to drive the vision Importance of ‘buy-in” from all involved Belief that students can achieve Belief that teachers can make a difference Importance of the big picture Commitment to the long term Importance of prioritising Remembering to celebrate and build on successes Importance of gathering useful data and analysing it effectively Using strengths from in and outside the school There is no one way- each school and community is unique but we can learn from each other

Discussion Developing your own definition – purpose statement Mapping your review activities – quandrant diagramMapping your review activities – quandrant diagram Finding an overall structure that integrates your review activitiesFinding an overall structure that integrates your review activities What process works for your school?What process works for your school?

Recapping Thinking about self review Tools and strategies suggested Feedback – What did you use? How well did it work for your school? What changes have you made as a result? Share ideas ERO self review presentation18

What ERO is finding in schools Pivotal Questions for internal review What is so? Why is it so? So what? Now what? ERO self review presentation19

Planning reviews What do we need to review? Why? How? ERO self review presentation20

Focus of reviews Balance needed What is on paper / documented School practices Outcomes for students ERO self review presentation21

ERO’s reviews What is the place of school’s self review in 2010 and ahead? in 2010 and ahead?

Principles underpinning ERO’s reviews 23  Enquiry focused  Responsive  Transparent  Collaborative & constructive  Building evaluation capacity

ERO self review presentation24 What do we know about effective schools? National and international research, eg, BES ERO’s Evaluation at a Glance: Effective Schools: –A focus on the learner –Strong leadership within an inclusive culture –Effective teaching –Quality engagement with parents, whānau and communities –Coherent and consistent policies and practices within a cycle of continuous self review

ERO self review presentation25 ERO’s differentiated review cycle Rather than being based on a “one-size-fits-all” three-yearly cycle, the differentiated review cycle is based on ERO’s confidence that the school is providing a quality education for all students and maintaining an on-going process of continuous self improvement Self review is the tool through which schools monitor and sustain this continuous improvement and provide evidence of what they are doing and what is working for their students

Deciding on Future Action ERO intends to carry out another review over a period of one-to-two years: Building capacity ERO is likely to carry out the next review in three years: Developing capacity ERO is likely to carry out the next review in four- to-five years: Sustaining capacity –Criteria –Process Trends –Moderation ERO self review presentation26

ERO’s Reviews 2010 An evaluation approach that balances external ERO review with school self review according to each school’s circumstances. 27 ERO REVIEW SCHOOL SELF REVIEW Schools experiencing difficulties with very limited self review Schools sustaining high quality performance and continuous improvement through effective self review Schools operating well with established processes of self review

Meta evaluation and design 28

Schools operating well with established self review processes 29 ERO REVIEW Scoping & review design phase using self review to set the scene plan the evaluation, balancing the need for first hand evidence with evidence from self review On site external review phase Gathering evidence triangulate data ongoing & collaborative synthesis Reporting phase internal reporting external reporting use self review as the basis for external review use external review to test, affirm, strengthen & broaden self review Building & using school capacity to: account for progress & achievement of goals, & generate further priorities for development & review School self reviewing & reporting to school community

Schools performing very well with high quality self review processes 30 ERO REVIEW School builds & presents “a case” comprised of valid & significant examples of using self review to: maintain high levels of student progress & achievement make sound decisions & build continuous school improvement External review phase scope & design review using the school’s “case” meta-evaluation of school’s self review external reporting challenge their own assumptions about the quality & efficacy of self review processes & findings Using & building school capacity to: engage with & use external meta- evaluation to enhance self review School self reviewing & continuously improving & reporting to school community

Schools experiencing difficulties 31 ERO LONGITUDINAL REVIEW 1-2 YEARS Development & review planning phase planning the action, evaluation & support to improve student achievement Implementing phase implementing development adapting & modifying (praxis) Concluding phase internal reporting external reviewing external reporting identify purposes & goals, & plan what & how to evaluate systematically implement planning & use data to self review, adapt & improve Building school capacity to: account for progress & achievement of goals & generate further priorities for development & review

32

33

ERO self review presentation34 ERO’s updated evaluation indicators Six dimensions: –Student learning: engagement, achievement and progress –Effective teaching –Leading and managing the school –Governing the school –Safe and inclusive school culture –Engaging parents, whānau and communities

Focus on success for Māori Govt – MoE and ERO priority Based on clear evidence of disparity Expectations outlined in NAGs How well is your school promoting / achieving successful outcomes for Māori? What does your self review tell you about this evaluative question? ERO self review presentation35

ERO self review presentation36 Support available From ERO Framework for reviews Evaluation Indicators Good practice case studies (not yet available) Evaluation at a Glance: Effective Schools (not yet available) Board/Whānau Assurance Statement Checklist National Evaluations Getting the most out of your ERO review booklets Workshops and cluster meetings From other sources: MoE self review tools for National Standards MoE Kiwi Leadership material MoE governance resource Commercial tools Local advisors/consultants Each other!

ERO self review presentation37 To conclude Be assured that self review is not new and much of what you already do fits under that umbrella Make on-going self review, reflection and action a “state of mind” in your school Ensure smaller review cycles feed data into strategic self review in a coherent way ERO are not looking for particular ways of doing things but at the quality of the evidence and the robustness of the processes that you use for your decision making Ask yourself : “How do you know that you make a difference for every student in your school?”