1. IPSP Outcomes Reporting Framework
What you need to know and what you need to do

2. Measuring and monitoring outcomes: Why are we doing this?
Measuring and monitoring outcomes enables us to identify:
If/how the IPSP leads to changes in thinking and practice, and the extent of this change
If/how program elements work together to produce outcomes
The types of support and professional development that produce the best outcomes
Opportunities for a nationally consistent approach, while retaining flexibility to be regionally responsive
Most importantly, opportunities for the continuous improvement of the program.

3. Elements of the Framework The IPSP Service Process Logic (ie: the “program logic”)
The service process logic is drawn from IPSP guidelines and summarises how IPSP is theorised to work in terms of:
Inputs (forms of investment in IPSP)
Processes (accountability and compliance; internal IPSP management; IPSP planning and delivery)
Outputs (IPSP services delivered)
Impacts (short-medium term changes)
Outcomes (longer-term changes)
NOTE: the logic also notes external variables that influence the achievement of outcomes

4. Elements of the Framework The IPSP Service Process Logic (ie: the “program logic”)
At an overarching level, understanding the service process logic allows us to ask questions about IPSP in terms of:
the implementation of IPSP (with a focus on continuous improvement in service delivery)
the logic of IPSP (with a focus on continuous improvement in the design of IPSP in the future)

5. Elements of the Framework The IPSP Performance Indicator Framework
The IPSP Performance Indicator Framework outlines the indicators of program equity, effectiveness and efficiency
Indicators include:
Output indicators (indicators of accessibility, appropriateness and quality of the services delivered)
Impact indicators
Outcome indicators
NOTE: Not all indicators can be easily measured

6. Elements of the Framework Data Collection Instruments (across all IPSP elements – PSC, ISA, IPSU)
Layer
Data Collection Instrument
Timeline
Planning
Needs Analysis Survey (All program elements)
April-May 2014
Needs Analysis Consultation (All program elements)
Undertaken by IPSP providers
Formative Feedback
Reflective Conversations (All program elements)
On-going July 2014 – June 2016
Initial Feedback Survey: Inclusion Support (ISA)
Initial Feedback Survey: Professional Development (PSC & IPSU)
Follow-up Feedback Survey (PSC & ISA)
Summative Reporting
Annual Outcomes Focus Groups (All program elements)
March 2014 & 2015
Annual Outcomes Survey (All program elements)
September 2014 & 2015

7. IPSU Data collection
Initial feedback survey
Reflective Conversations

8. Approach Our focus here is to approach this in ways that:
are culturally appropriate (recognising that surveys are a mainstream cultural instrument)
are practical and do not detract from service delivery capacity
assist with building capacity for engaging with the National Quality Frameworks

9. Your Outcomes Reporting Toolkit
Shift Consulting has developed a toolkit for providers in consultation with the IPSU Outcomes working group which includes:
Guidelines for implementation
This PowerPoint
Information for educators fact sheet
Reflective conversations resource for educators
Hard copy survey
Reflective conversation data sheets

10. IPSU Feedback Surveys
Initial feedback is collected through a short survey.
The survey can be administered:
On-line
Off-line (eg: via an IPSU ipad)
Paper-based (and answers entered by the IPSU into the survey platform)

11. IPSU feedback survey Is to be undertaken as follows:
With services on your first visit after July 2014
With Coordinators/educators when they are new
Links to surveys
Online version
Offline version
zmo.com/s3/ /IPSUFeedback&sLanguage=Auto

12. How to get the survey data into our system: Online mode
Survey undertaken by educators in online mode
They can do it alone of with your assistance
Requires internet connection
They are given a link by you
They go online and fill out the survey
Their completed survey is in the system

13. How to get the survey data into our system: Offline mode
Survey undertaken by educators with your assistance in offline mode
They can do it alone with your assistance
Does not require internet connection but does require laptop, ipad or smartphone
They are given a link by you or you sit together
They follow the link and fill out the survey
You later upload the survey into the system

14. How to get the survey data into our system: hard copy
Survey undertaken by educators by filling out a hard copy form
They can do it alone
Is handed to you at the conclusion of the support visit
Later, you enter the survey into the system (offline mode quickest)
You provide hard copy survey
They fill out the survey and hand it back to you
You later enter the survey into the system

15. Why Reflective Conversations?
The data gathered through the ‘reflective conversation’ instrument provides us (IPSP) with a means to build knowledge and understanding of the process of change in the IPSP context.
Obviously, in the course of facilitating a conversation, “change” becomes contextualized with reference to the specific service environment, children, families and educators.

16. Reflective Conversations
A reflective conversation is a guided conversation that encourages educators to:
Reflect on change
Identify barriers
Identify strategies
Plan for positive change
Implement plans & strategies

17. Reflective Conversations Documenting reflections (targeted at educators)
The key to conducting a good reflective conversation is active listening and responsively guiding the conversation in ways that encourage critical reflection, problem solving and planning.
Documenting key reflections and ideas along the way provides the facilitator and the educator(s) with a framework for drawing links between ideas and critically reflecting on these over time.
 Documenting the conversation is modelling the same process as an educator might use to document children’s learning

18. Reflective Conversations Where educators begin
How a conversation about change occurs and the depth of reflection that will occur is influenced by where educators begin.
An educator’s “level of reflection” will influence how the reflective conversation will take place.
The role of a professional facilitating a reflective conversation is to assist the educator to reflect more deeply and grow their reflection and understanding

19. Bain’s Critical Reflection Framework
Level 1: REACTING
Focuses on own feelings
Offers straight forward description
Seeks basic understanding
Level 2: ELABORATING
Compares self with others
Considers alternatives
Seeks deeper understanding
Level 3: RECONSTRUCTING
Draws new conclusions about practice
Explores relationships between theory and practice
Drives own learning
(Miller, 2011, p. 6)

20. Reflective Conversations Levels of Reflection
Level 1: REACTING
Focuses on own feelings: Commenting on feelings related to an event
Offers straight forward description: Providing a straight description with no added observation or insights
Seeks basic understanding of what occurred and why

21. Reflective Conversations Levels of Reflection
Level 2: ELABORATING
Compares self with others: Comparing one’s reaction to other people’s responses and perspectives
Considers alternatives: Analysing an event by asking questions and considering alternatives
Seeks deeper understanding of an event by relating it to current literature and theories

22. Reflective Conversations Levels of Reflection
Level 3: RECONSTRUCTING
Draws new conclusions about practice: Drawing revised and new conclusions about own practices and the practices of others
Explores relationships between theory and practice: Ongoing exploration of relationships between practice, literature and theory
Drives own learning: Planning further learning on the basis of your reflections.

23. Reflective Conversations A frame of reference for facilitators
Conversations flow… sometimes people offer up information without prompting… at other times they need significant support to draw out the details.
There is a set of questions that can provide a facilitator with a frame of reference to support a holistic conversation.
The key is that the facilitator must keep an ear out for the what information has been offered up and where they need to ask questions and encourage further reflection.
The documentation tools provided can assist with this process.

24. Reflective Conversations Reference Questions – Initial Visit (from July 2104)
What are the most significant learnings you have drawn from today?
Are there any ways in which you have changed the way you think about [topic]?
How do you think you will continue to use what you have learned from today?

25. Reflective Conversations Reference Questions – Follow-up visit
What is the most significant change that has taken place as a result of receiving this support? Think about (encourage reflection on)
The way you think about your role/approach/practice?
Your relationships with others such as your team members, children and families?
The practices that you use in relation to working with team members, children and families?
Have you experienced any barriers to implementing change in your practice?
How have or could these be overcome?
How do you think you will continue to use what you have learned in the future?

26. How to get the Reflective conversations data into our system:
Utilise existing methods of engaging in reflective conversations with educators. This is NOT a structured interview and the questions are there to guide your conversation.
Document the conversation in terms of the reflective questions and themes outlined in the datasheet
Gather data using existing reflective conversations strategies
Transfer information to reflective conversations data sheet
Forward to Shift Consulting as required (3 monthly)

27. Got any questions?
Please forward any questions through the IPSU IPSP Outcomes working group – members are Chris, Kate and Peter.