IPSP Outcomes Reporting Framework What you need to know and what you need to do

Measuring and monitoring outcomes: Why are we doing this? Measuring and monitoring outcomes enables us to identify: If/how the IPSP leads to changes in thinking and practice, and the extent of this change If/how program elements work together to produce outcomes The types of support and professional development that produce the best outcomes Opportunities for a nationally consistent approach, while retaining flexibility to be regionally responsive Most importantly, opportunities for the continuous improvement of the program.

Elements of the Framework The IPSP Service Process Logic (ie: the “program logic”) The service process logic is drawn from IPSP guidelines and summarises how IPSP is theorised to work in terms of: Inputs (forms of investment in IPSP) Processes (accountability and compliance; internal IPSP management; IPSP planning and delivery) Outputs (IPSP services delivered) Impacts (short-medium term changes) Outcomes (longer-term changes) NOTE: the logic also notes external variables that influence the achievement of outcomes

Elements of the Framework The IPSP Service Process Logic (ie: the “program logic”) At an overarching level, understanding the service process logic allows us to ask questions about IPSP in terms of: the implementation of IPSP (with a focus on continuous improvement in service delivery) the logic of IPSP (with a focus on continuous improvement in the design of IPSP in the future)

Elements of the Framework The IPSP Performance Indicator Framework The IPSP Performance Indicator Framework outlines the indicators of program equity, effectiveness and efficiency Indicators include: Output indicators (indicators of accessibility, appropriateness and quality of the services delivered) Impact indicators Outcome indicators NOTE: Not all indicators can be easily measured

Elements of the Framework Data Collection Instruments (across all IPSP elements – PSC, ISA, IPSU) LayerData Collection Instrument Timeline Planning Needs Analysis Survey (All program elements) April-May 2014 Needs Analysis Consultation (All program elements) Undertaken by IPSP providers Formative Feedback Reflective Conversations (All program elements) On-going July 2014 – June 2016 Initial Feedback Survey: Inclusion Support (ISA) On-going July 2014 – June 2016 Initial Feedback Survey: Professional Development (PSC & IPSU) On-going July 2014 – June 2016 Follow-up Feedback Survey (PSC & ISA) On-going July 2014 – June 2016 Summative Reporting Annual Outcomes Focus Groups (All program elements) March 2014 & 2015 Annual Outcomes Survey (All program elements) September 2014 & 2015

IPSU Data collection 1.Initial feedback survey 2.Reflective Conversations

Approach Our focus here is to approach this in ways that: are culturally appropriate (recognising that surveys are a mainstream cultural instrument) are practical and do not detract from service delivery capacity assist with building capacity for engaging with the National Quality Frameworks

Your Outcomes Reporting Toolkit Shift Consulting has developed a toolkit for providers in consultation with the IPSU Outcomes working group which includes: Guidelines for implementation This PowerPoint Information for educators fact sheet Reflective conversations resource for educators Hard copy survey Reflective conversation data sheets

IPSU Feedback Surveys Initial feedback is collected through a short survey. The survey can be administered: On-line Off-line (eg: via an IPSU ipad) Paper-based (and answers entered by the IPSU into the survey platform)

IPSU feedback survey Is to be undertaken as follows: With services on your first visit after July 2014 With Coordinators/educators when they are new Links to surveys Online version Offline version zmo.com/s3/ /IPSUFeedback&sLanguage=Auto

How to get the survey data into our system: Online mode Survey undertaken by educators in online mode They can do it alone of with your assistance Requires internet connection They are given a link by you They go online and fill out the survey Their completed survey is in the system

How to get the survey data into our system: Offline mode Survey undertaken by educators with your assistance in offline mode They can do it alone with your assistance Does not require internet connection but does require laptop, ipad or smartphone They are given a link by you or you sit together They follow the link and fill out the survey You later upload the survey into the system

How to get the survey data into our system: hard copy Survey undertaken by educators by filling out a hard copy form They can do it alone Is handed to you at the conclusion of the support visit Later, you enter the survey into the system (offline mode quickest) You provide hard copy survey They fill out the survey and hand it back to you You later enter the survey into the system

Why Reflective Conversations? The data gathered through the ‘reflective conversation’ instrument provides us (IPSP) with a means to build knowledge and understanding of the process of change in the IPSP context. Obviously, in the course of facilitating a conversation, “change” becomes contextualized with reference to the specific service environment, children, families and educators.

Reflective Conversations A reflective conversation is a guided conversation that encourages educators to: Reflect on change Identify barriers Identify strategies Plan for positive change Implement plans & strategies

Reflective Conversations Documenting reflections (targeted at educators) The key to conducting a good reflective conversation is active listening and responsively guiding the conversation in ways that encourage critical reflection, problem solving and planning. Documenting key reflections and ideas along the way provides the facilitator and the educator(s) with a framework for drawing links between ideas and critically reflecting on these over time. Documenting the conversation is modelling the same process as an educator might use to document children’s learning

Reflective Conversations Where educators begin How a conversation about change occurs and the depth of reflection that will occur is influenced by where educators begin. An educator’s “level of reflection” will influence how the reflective conversation will take place. The role of a professional facilitating a reflective conversation is to assist the educator to reflect more deeply and grow their reflection and understanding

Bain’s Critical Reflection Framework Level 1: REACTING Focuses on own feelings Offers straight forward description Seeks basic understanding Level 2: ELABORATING Compares self with others Considers alternatives Seeks deeper understanding Level 3: RECONSTRUCTING Draws new conclusions about practice Explores relationships between theory and practice Drives own learning (Miller, 2011, p. 6)

Reflective Conversations Levels of Reflection Level 1: REACTING Focuses on own feelings: Commenting on feelings related to an event Offers straight forward description: Providing a straight description with no added observation or insights Seeks basic understanding of what occurred and why

Reflective Conversations Levels of Reflection Level 2: ELABORATING Compares self with others: Comparing one’s reaction to other people’s responses and perspectives Considers alternatives: Analysing an event by asking questions and considering alternatives Seeks deeper understanding of an event by relating it to current literature and theories

Reflective Conversations Levels of Reflection Level 3: RECONSTRUCTING Draws new conclusions about practice: Drawing revised and new conclusions about own practices and the practices of others Explores relationships between theory and practice: Ongoing exploration of relationships between practice, literature and theory Drives own learning: Planning further learning on the basis of your reflections.

Reflective Conversations A frame of reference for facilitators Conversations flow… sometimes people offer up information without prompting… at other times they need significant support to draw out the details. There is a set of questions that can provide a facilitator with a frame of reference to support a holistic conversation. The key is that the facilitator must keep an ear out for the what information has been offered up and where they need to ask questions and encourage further reflection. The documentation tools provided can assist with this process.

Reflective Conversations Reference Questions – Initial Visit (from July 2104) What are the most significant learnings you have drawn from today? Are there any ways in which you have changed the way you think about [topic]? How do you think you will continue to use what you have learned from today?

Reflective Conversations Reference Questions – Follow-up visit What is the most significant change that has taken place as a result of receiving this support? Think about (encourage reflection on) The way you think about your role/approach/practice? Your relationships with others such as your team members, children and families? The practices that you use in relation to working with team members, children and families? Have you experienced any barriers to implementing change in your practice? How have or could these be overcome? How do you think you will continue to use what you have learned in the future?

How to get the Reflective conversations data into our system: Utilise existing methods of engaging in reflective conversations with educators. This is NOT a structured interview and the questions are there to guide your conversation. Document the conversation in terms of the reflective questions and themes outlined in the datasheet Gather data using existing reflective conversations strategies Transfer information to reflective conversations data sheet Forward to Shift Consulting as required (3 monthly)

Got any questions? Please forward any questions through the IPSU IPSP Outcomes working group – members are Chris, Kate and Peter.