1. Assumptions and Outcomes Manager Service Data Quality & Outcomes
Ross Bentong
Manager Service Data Quality & Outcomes

2. ASK: In this scenario the person falling is assuming that the people below will catch them, but is anyone else making an assumption here? (can the people catch the person? Will each person in the team below work together? Do they all understand what they need to do? Is it safe for them to carry out this activity?)
ASK: If we get assumptions wrong, what can happen?

3. TELL: We didn’t want to make assumptions about outcomes for our clients at AWA so we spent a lot of time with many of our services completing Program Logic workshops
TELL: We made the decision to do this with staff at all levels, not just managers/exec/board as we wanted staff to tell us what outcomes they want to see for their clients and to really engage staff in outcome measurement – our staff now really want to know if they make a difference.

4. TELL: Very simply put, this is how may people think about the things we do in our daily lives
Input = Resource
PutPut = activity
Output = The thing we produce (product) or something we have delivered/achieved (service)
TELL: I found that during PL workshops, staff often found it difficult to think beyond outputs, it wasn’t an obvious connection for them to link the service they delivered to outcomes.

5. TELL: Something as simple as training staff in the following process during PL Workshops made a huge difference to their understanding and engagement in the whole process
THIS CAN BE A BIG CULTURE CHANGE FOR MANY SERVICES/PEOPLE

6. Theme
Activity Group
Short Term Outcome
(Immediate)
Intermediate Outcome
(By End of Service)
Long Term Outcome
(Post Service)
Initial Contact
Referrals
Feel safe, heard & understood
Feel relieved & have hope
Have access to support during immediate crisis
Feel unjudged & that they are in a safe space
Have access to relevant information
Able to articulate what they need
Trust the Service
Access to basic human needs (food, water, shelter, clothing, first aid, identification)
Feel socially connected; not alone
Experience digital inclusion
Be aware of positive lifestyle choices & how to keep safe
Access to a support network & access to specialised service where required (e.g. AOD counselling)
Able to cope with feelings & emotions
Create awareness of issues facing street present young people
Feel able to actively engage with the Service
Able to reflect on life & progress made
Develop independent living skills
Develop plans to change / improve life
Have a sense of responsibility to maintain progress made
Build coping strategies to develop / sustain resilience
Reduction in risk taking behaviours
Able to transition from childhood to adulthood
Able to identify, understand & cope with trauma
Develop emotional intelligence with the ability to self-regulate emotions
Provide a conduit for other Services to better engage with street present young people
Work with other Services to provide a better & integrated approach to supporting street present young people
Ability to identify & communicate own needs effectively
Be a well-functioning adult & able to independently make responsible life choices
Able to participate actively & positively within the community
Be empowered to live a fulfilling life without external support
Be resilient
Able to act as a role model & support peers / others
Ability to accept, learn & move on from past trauma
Enable the community to support street present young people
Reduce existing & prevent future youth homelessness
Enabling the community to ensure the rights of young people are satisfied & protected
Outreach
Assessment & Planning
Case Management
Advocacy & Referrals
Risk
Client Admin
Case Management
Review
Client Review
Community
Community Development
Networking
TELL: This is one of the “outcomes” of a program logic workshop with a service – a list of short, intermediate and long term outcomes.
Staff are engaged, understand how their actions make a difference and are actively involved in measuring these outcomes – producing data to find out if they made a difference!
But when there are over 60 services, how do we know if the agency as a whole is making a difference?

7. 4. Fulfilment Growth & Contribution. 2. Sustaining Lives
4.1 Parenting Capacity
4.2 Community Capacity
2. Sustaining Lives
2.1 Youth Housing
2.2 Adult Housing
2.3 Financial Support
3. Social & Emotional Wellbeing
3.1 Strengthening Family Relationships
3.2 Mental Health & Disability Support
3.3 Adult Social & Emotional Wellbeing
3.4 Child & Youth Social & Emotional Wellbeing
1. Critical Response and Safety
1.1 Family & Domestic Violence
1.2 Suicide Prevention & Response
1.3 Support & Advocacy to Navigate the Justice/Legal System
Additional Outcomes
Organisational Outcomes
4.3 Employment Support
TELL: We considered the OUTCOMES of all of our services and grouped them into Micro & Macro Clusters
TELL: Within each cluster there are two types of outcomes
Org Outcomes – these had the potential to be agency wide
Additional Outcomes – these are specific to a service, or specific cluster and are not able to be applied agency wide
Once this clustering activity was complete we could consider a Theory of Change for the organisation

8. Outputs and Outcomes Impact Activity streams
Anglicare WA drives positive and lasting change by growing stronger communities and strong people within them.
As a result, Anglicare WA contributes towards a just and fair WA where all people thrive.
Access to appropriate information, services and supports through AWA and external referrals
Activity streams
Family &Domestic Violence
Strengthening Family Relationships
Financial Support
Adult Housing
Youth Housing
Mental Health and Disability Support
Adult Social & Emotional Wellbeing
Support & Advocacy to navigate the justice system
Suicide Prevention & Response
Child & Youth Social & Emotional Wellbeing
Parenting Capacity
Community Capacity
Children, Young People, Adults & Families
Acquire skills, knowledge & a network to manage own situation
Understand and more confident to make behavioural and attitudinal changes
Develop a trusted relationship with AWA
Have hope for better life
2. More people are HEALTHY & WELL
4. More people have opportunities to GROW & CONTRIBUTE
3. More people are CONNECTED
2a. Overall wellbeing
3a. Functioning families
3b. Increased social participation
4a. Increased meaningful engagement
4b. Learning & Developing
1a. Secure and maintain housing
1b. Free from neglect, abuse, violence
1. More people are SAFE
Aligned to the needs of children
TELL: This is the section of AWA Theory of Change that directly relates to outcomes from services
Employment Support

9. Measuring Outcomes with RBA
TELL: After completing program logics that identify outcomes which can be mapped to a organisation ToC, you then need to think about how you measure outcomes to see if you have made a difference.
At AWA Results Based Accountability is our chosen Monitoring and Evaluation framework

10. Is anyone better off? How well did we do it? How much did we do?
TELL: Very simply, there are 3 things that we aim to answer using RBA
How much did we do? These are our outputs
How well did we do it? What was the quality of our work
Is anyone better off? These are the outcomes we have achieved
ASK: Can anyone think of examples of the types of things we could measure to answer these questions?
TELL: These 3 questions directly link to the different sections of our Program Logics and Theory of Change
How much = outputs
How well = enablers (quality/satisfaction)
Better off = outcomes

11. Measuring how much we do is relatively easy – most service providers have been doing this for years and it is a contractual requirement from funders
Measuring how well we do it is a little bit harder but still relatively easy – should require minimal additional data collection
Measuring who is better can be complex

12. Things to think about when choosing outcomes measurement
What does your funder/stakeholder require?
What tools already exist?
Do your people understand it?
What are you going to do with the data?
What does your funder/stakeholder require?
Does your funder have an outcomes framework they want you to use
What do your contracts/grant agreements state regarding outcomes?
What tools already exist?
Do you need to use a validated tool?
Are they right for you?
If you design your own tool, have you involved your staff?
Don’t keep changing the baselines
Do your people understand it?
Have people been trained
Have you sought feedback
Do they have buy-in for the process
Do staff/clients want to use it?
What are you going to do with the data?
RBA has the Headline Measure Report and Turning the Curve exercise where services and staff can look at data and make decisions about improving the service
If you are not using RBA how will you use the data?
Will you set benchmarks?