1. Agcas Scotland Knowing your outcomes
Agcas Scotland Knowing your outcomes! The AMOSSHE Value and Impact model – Logic modelling Pam Crawford

2. How do you know that change took place?
Defining and measuring wider outcomes: what impact are you making?
DISCUSSION
Think of a strategy or project that you think has made a significant impact on students
What change took place?
How do you know that change took place?

3. Amosshe and Logic modelling
Amosshe value and impact toolkit
“Basically, a logic model is a systematic and visual way to present and share your understanding of the relationships among the resources you have to operate your program, the activities you plan, and the changes or results you hope to achieve.”
Kellogg Foundation p.1 (2006)

4. OBJECTIVES: What are the intended achievements of the strategy?
OUTPUTS: What are the service deliverables – what will we deliver: workshops; conferences; research; lobbying; fundraising; guides?
OUTCOMES: What change and impact will take place for our stakeholders? Will they be more knowledgeable, equipped, confident or solvent? Who are our stakeholders?
IMPACT INDICATORS: How can the service demonstrate that the objective has been met? What will tell us that the outcome has been achieved?
DATA: What tangible data sources will be used to provide evidence of change and success: surveys; interviews; external data and league tables; case studies; focus groups; web analytics (Qualitative and Qualitative)
Amosshe Logic Model

5. OBJECTIVES Mission Vision Objectives
Objectives are intended achievements. As your evaluation strategy emanates from your objectives, it’s essential to be absolutely clear about what you intend to achieve. Objectives need to be specific and time-limited. Confusion can occur between mission, vision and objectives. Let’s clarify the difference
Mission
Vision
Objectives
“Describes why the organisation exists and the broad effect it wants to have. It summarises the difference that an organisation wants to make.”
Vision statements tend to be inspirational and provide a mental picture of what an institution or organisation wants to achieve longitudinally
“Intended achievements” (AMOSSHE 2011 ).
“Google’s mission is to organise the world’s information and make it universally accessible and useful”.
Google
“A just world, without poverty” OXFAM
“AT IKEA, our vision is to create a better everyday life for people” IKEA
“Community – create a sense of common purpose so all AGCAS members feel that their profession/mission group/region is represented (2016/17)”
AGCAS
Pam Crawford Consulting

6. Objectives: worked example Big Lottery Fund sample: Community engagement
We intend to:
To reduce isolation and improve the quality of life for excluded communities on the two estates and increase engagement in local networks and activities by 2017.

7. Big Lottery Fund sample: Community engagement
OUTPUTS
“An output is a deliverable, i.e. it’s a physical thing. It’s basically a statement about what students receive from the service/activity”. (Amosshe)
“The activities, services and products provided by an organisation” (CES glossary)
Big Lottery Fund sample: Community engagement
Objective
We intend to:
To reduce isolation and improve the quality of life for excluded communities on the two estates and increase engagement in local networks and activities by 2017.
Outputs
We will deliver the following:
Hold three community meetings in each estate to support community in identifying needs and running projects. 
Carry out capacity-building work, providing six training sessions, and one to one support for 25 community members. 
Help residents set up five social groups in each estate. 
Carry out lobbying and advocacy activities and disseminate information between communities and the council. 
Hold three community events in each estate. 
Pam Crawford Consulting

8. Outcomes
Outcomes are, as stated previously, more immediate to medium term changes and benefits that occur, while impact reflects broader longitudinal change and improvement
“Evaluating outcomes is a specific type of evaluation aimed at gauging the impact of student services, programmes and facilities and their effect on learning, development, academic achievement, and other intended outcomes.”(AMOSSHE)
“Outcomes usually demonstrate growth or development in areas of ‘knowing, being or doing’.” (AMOSSHE)
What sort of changes are you looking for in your stakeholder or client group?
Outcome statements often contain with ‘improve’, ‘increase’.
Outcome statements often contain a sense of a changed state….”our students will be….”
Film

9. Big Lottery Fund sample: Community engagement
Outcomes
Big Lottery Fund sample: Community engagement
Objective
We intend to:
To reduce isolation and improve the quality of life for excluded communities on the two estates and increase engagement in local networks and activities by 2017.
Outputs
Hold three community meetings in each estate to support community in identifying needs and running projects. 
Carry out capacity-building work, providing six training sessions, and one to one support for 25 community members. 
Help residents set up five social groups in each estate. 
Carry out lobbying and advocacy activities and disseminate information between communities and the council. 
Hold three community events in each estate. 
Outcome
The following will change
Local residents feel less isolated as a result of participating in new social networks or activities. 
Local residents feel greater ownership of their local community and have a greater voice in issues that affect them. 
Services are more responsive and have a better understanding of local people's needs that will help them to plan and deliver more appropriate services on the estates.
Pam Crawford Consulting

10. Big lottery fund example
Outcome
Indicator – how will we know the outcome has been successful (or not)?
Services are more responsive and have a better understanding of local people's needs that will help them to plan and deliver more appropriate services on the estates
Increased numbers using services
Organisations reporting a better understanding of local people’s needs

11. Source of data and evidence
Identify the right data sources
Identify the right mix for a credible picture: USAGE; SATISFACTION and IMPACT
Each form of evaluation is suited to the activity or deliverable
It should address the question, ‘does it demonstrate whether this outcome been achieved?’
Counts
external surveys
web analytics
card sorts
Interviews
Case studies
Focus groups

12. What is a source of data?
It is the verification of change and evidence
Sources of data may be available to draw upon ( e.g. Student surveys; retention figures; module/course evaluation)
Sources of data may need to be developed and programmed (e.g. a focus group)
Baseline
(What’s in existence now?)
Develop new indicators
(What do we need to know?)

13. Big lottery fund example
Outcome
Indicator
Source of data
results
Timescale
Services are more responsive and have a better understanding of local people's needs that will help them to plan and deliver more appropriate services on the estates
Number using services
A sample of 50 local people report they have better access to local services
Annually
Organisations report a 20 per cent increase in take up of local services
By the end of the project
Organisations reporting a better understanding
18 local organisations report a better understanding of the needs of their users through feedback from forums and community champions

14. Baseline data New data Sources of data EXTERNAL INTERNAL
YOUR EXISTING DATA
GAPS
Baseline data
New data

15. OBJECTIVES: What are the intended achievements of the strategy?
OUTPUTS: What are the service deliverables – what will we deliver: workshops; conferences; research; lobbying; fundraising; guides?
OUTCOMES: What change and impact will take place for our stakeholders? Will they be more knowledgeable, equipped, confident or solvent? Who are our stakeholders?
IMPACT INDICATORS: How can the service demonstrate that the objective has been met? What will tell us that the outcome has been achieved?
DATA & EVIDENCE: What tangible data sources will be used to provide evidence of change and success: surveys; interviews; external data and league tables; case studies; focus groups; web analytics (Qualitative and Qualitative)
Amosshe Logic Model

16. Questions What change took place?
LETS REVISIT THE INITIAL DISCUSSION
Think of a strategy or project that you think has made a significant impact on students
What change took place?
How do you know that change took place?