Manaiakalani Evaluation Programme 2012 –2013 LCN Regional Networking Day | March 2014

1.A Short History of Manaiakalani – Pat Snedden 2.Key Questions for reflecting on this session 3.3 Key Findings from Manaiakalani research – Aaron Wilson 4.Engaging with these findings – Rebbecca Sweeney 5.Next steps in using the key findings This session

Introductions Rebbecca Sweeney (YouTube Mihi)YouTube Mihi Aaron Wilson Pat Snedden

Key questions for LCN participants today How do the Manaiakalani findings apply in your settings? What are your network’s priority challenges?

Nature of the evidence 1.In partnership with schools and Manaiakalani Education Trust we have collected evidence. 2.Our purposes have included (given our shared goals) 1.detailed profile (strengths and weaknesses) 2.contribution to further design and building 3.establish strong ongoing evidence base

Our methods and evidence include: 1.Achievement data 2.Classroom observations 3.Interviews and surveys

Achievement - evidence Groupings for which the questions can be asked:

Achievement - evidence Levels about which the questions can be asked:

Questions: What is coherence? Why do you need it? What if there isn’t coherence about network goals and vision across all participants? Key Finding 1: Coherence

Coherence - evidence 1.Teachers (interviews- early in 2013) 1.Had high enthusiasm for and fidelity to the goals (digital access to resources, curriculum and engagement) 2.Operationalised these goals, implementing digital pedagogies and engaging their students (global community and collaboration, future opportunities, accessibility of resources, and enhanced engagement 2.Students- similarly able to articulate goals

Questions: How is engagement defined in your network? Who defines it? Is one type of engagement more important than others or is there an order of importance? How do you measure engagement in a valid way? Key Finding 2: Engagement

Engagement – evidence (early 2013) Four Forms of Engagement: 1.“Behavioural” - indicated by being on task and participation in class  High: few intervals ‘off task’ 1.“Affective” - indicated by such things as shared goals and enjoyment  High: students aware, believe in goals 2.“Cognitive” - working on complex and challenging tasks  Variable: Learn-Create-Share & Practice activities (e.g. Googledoc worksheets, maths games) 3.“Active pursuit” – self report – at home  High: students engaged at home (e.g. ‘recreational’ writing, reading, maths)

Next Steps: Consider how to refine learning activities so that they maintain their engaging nature, but are also challenging and extending. This focus applies to both teacher-led and independent activities. Extend teaching for independence and autonomy, to disentangle the related but distinct concepts underlying notions of autonomy, self-regulation, cognitive engagement, and metacognition, and to identify the relationship of these to increased outcomes, of various sorts, for students. Engagement

Key Finding 3: Spreading ‘pockets of promise’ Manaiakalani has a focus on equity and access in connected communities. For this to be achieved, Manaiakalani pedagogies need to be implemented in all classes, equally well. Questions: Is consistency in implementation possible? How do you ensure/encourage this? How can networks capture/illustrate/promote effective practices? Should everyone be accountable for high quality practice?

Spreading ‘pockets of promise’ (end 2013) How? 1.Systematic and ongoing discussion of evidence and posing hypotheses (e.g. mid 2013) 2.Identifying examples and developing concepts of what's working (looking for variability) 3.Processes put in place - ‘Targeting resources’ (e.g. PLD) Next Steps: The cluster principals and staff have sharpened their focus with a specific goal to “Make all classrooms, classrooms of high cognitive engagement.” Alongside WFRC, the cluster of schools is working to identify, affirm and share the teaching approaches of their most effective teachers and work relentlessly to spread high teaching quality.

Actual comments mid Significant ‘pockets of promise’ 2.CHALLENGE: How to make these more consistent and spread the effectiveness? 3.Our ‘hypotheses’ from this evidence and from classroom observations and interviews:

Should your LCN engage with these key findings? Are the findings relevant to your network goals and your students’ needs? Adaptive Expertise Are network leaders and teachers able to identify whether or not their skills are adequate in these areas? Have network members identified that they require new information about these areas? What professional learning might need to occur to ensure that new practices are used to meet current students’ needs? How will your network check that the network programme/plan is being implemented consistently within and across schools?

Group discussion: Engage with the research 1. identify a current network goal/challenge related to at least one of the key findings. 2. consider the relevant key finding(s) and how the network might use the information presented today to address the goal/challenge. How do the findings apply in your setting? 3. use “questions to uncover assumptions” to guide your discussion.

Questions to uncover assumptions These types of questions help you to develop a better understanding of the challenge, and events/plans (NO advice-giving yet!)  what is the issue you want to address?  tell me more about your thinking…  what might be some examples of that?  what might be some other possibilities?  what might be some assumptions here?  what data do you/we have to support your/our thinking?  what evidence suggests that?  what additional information/evidence might be needed?  what other perspective might you need to consider?  what implications might there be for…?  what might happen if you didn’t take that pathway?  what might your next steps be? - Report assumptions identified - Explore/Give alternative versions of events/solutions - Give any advice Adapted from “Critical Conversations” (Brookfield) and “Learning Talk” (Dalton & Anderson)

Next Steps: Using the key findings What will you use and why/how? What will this look like in your network/school? What will the impact be on network plans, on infrastructure, on staffing, funding, quality assurance etc? Resources: - The “exploring” & “benefits” handouts - Dorothy Burt & Rebbecca Sweeney

Exploring Coherence Can all members of your network articulate your network goals? Is it clear that the network goals are related to student learning goals? Do all members of your network understand what implementation of the network programme looks like? Can all members of your network describe their role in achieving network goals and implementing the network programme? How will your network check that the network programme/plan is being implemented consistently within and across schools?

Benefits of looking at Engagement Current high levels of engagement can be exploited… Extend understanding and teaching for ‘independence’ and autonomy and the relationship of these to outcomes for students. More design for the activities that students are engaging with when not with the teacher; to increase levels of cognitive engagement, especially ‘practice’ activities. Might include an audit of ‘off the shelf programmes’ in literacy and in maths.

Do network members understand these four forms of engagement? Do network members know how and why their students are (or are not) engaged in learning? How will network members consider the importance of cognitive engagement, while not losing sight of the importance of behavioural engagement? Exploring Engagement

Benefits of ensuring the spread of effective practice Capitalise on efficiency…Foster consistency and coherence within AND between schools on what counts as effective practices. Digital pedagogies reduce transactional costs (to direct students or explain ‘what to do’) What is ‘best use of your time’? 1.learning conversations focused on extending students’ thinking, 2.less teacher time on ‘practice’, lower level activities. 3.preparation focused on resources and activities for higher level thinking, synthesis and evaluation

Exploring ways to spread effective practice What tools and resources can network members use to identify effective teachers and locate their practices in order to share and implement these more widely? What part does the Inquiry and Knowledge Building Cycle play in ensuring identification of effective practice and consistency in implementation? How will implementation integrity be assured across the network?