1. Evaluating CLE Community Legal Education Masterclass
Rosslyn Monro & Carly Hanson
7 March 2018

2. Evaluation Fitness Test – Warm-up!
Stand up if you have delivered CLE
Stay standing if you have evaluated CLE
Raise your hands if you have used a feedback form as part of your evaluation
Hands on head if you have used a case study as part of your evaluation
Jazz Hands if you have used observation as part of your evaluation
Hands on hips is you have used focus groups as part of your evaluation
Keep standing if you thought about evaluation when you planned your CLE project or initiative
Chocolates as reward for those still standing at the end of the warm-up
Evaluation Fitness Test – Warm-up!

3. Aims of this workshop
Think about achievable evaluation methodology for CLE 1
When to think about evaluation in planning and delivering CLE 2
How to embed evaluation process in your CLE in a strategic and planned way 3
Share your evaluation experiences and build upon them 4
Workshop focus on getting the evaluation settings right in the planning – less focus on the execution and analysis – but don’t forget about these!
Evaluating CLE| 3
Evaluating CLE | 3

4. What is CLE
How to develop SMART objectives for CLE
Principles of adult learning & learning styles 
Community development models & CLE 
How to work with different groups / audiences 
Communication skills 
Working with different methodologies / media 
Using technology for CLE 
Working in partnerships with other stakeholders 
So far in the masterclass… This workshop will tie all of this together in considering whether the CLE has been effective
Evaluating CLE | 4

5. What is CLE evaluation?
Evaluation is the process of collecting the information that the centre needs in order to determine if the stated objective(s) of the CLE activity has been achieved 
CLE Made Easy Kit – Information Sheet 6
Evaluating CLE | 5

6. Why evaluate ? Inform the design and delivery of CLE activities
Determine the impact of your CLE
Test your SMART objectives 
Make improvements to your CLE practice 
Meet accreditation standards – C1: CLE activities are evaluated using stakeholder feedback
Funding agreement requirements 
Any other reasons?
Why evaluate ?
Any other reasons to evaluate CLE from the whole group?
Evaluating CLE | 6

7. How do we evaluate CLE? Audience Strategies Needs analysis
Video – Who’s Who in the District Court
Review the effectiveness of the CLE by reference to the CLE project outline areas, eg:
Audience
Strategies
Needs
analysis
Communication
Evaluating CLE | 7

8. Key Evaluation Questions When will we involve them?
CLE Evaluation Plan
SMART objective
Key Evaluation Questions
Data
Tools / Methods
Who will we involve?
When will we involve them?
The next part of the workshop is to work through the CLE evaluation plan (provide / refer to handouts)
Evaluating CLE | 8

9. Evaluation Plan
SMART objective example: 
By delivering CLE on Enduring Powers of Attorney to residents in the Restful Nursing Home, 25% of participants will check that their EPA is current in the next month
SMART objectives clarifies what the initiative is trying to achieve
Evaluating CLE | 9

10. Evaluation Plan: SMART objective
KEQ
Data
Tools / Methods
Who? / When?
By delivering CLE on EPAs to residents in the Restful Nursing Home, 25% of participants will check that their EPA is current in the next month
Allocate a SMART objective to each group from day 1 (put on white board / flip chart)
Each group writes their SMART objective into the Evaluation plan worksheet
This example is worked through as a group, not individually
Evaluating CLE | 10

11. Evaluation Plan: Key Evaluation Questions (KEQs)
KEQs are the questions we ask to tell us if something has been completed or a change has occurred
KEQs need to be formulated at the same time as the SMART objective so evaluation methodology can incorporated into CLE implementation
Evaluating CLE | 11

12. Key Evaluation Questions
Some suggested categories of questions:
Appropriateness
Was the initiative appropriate to the identified need?
Is this initiative appropriate for our CLC to undertake/ are we best placed to do this?
Does this initiative align with the goals of our CLC?
Effectiveness
Did the initiative achieve the intended outcomes?
Is the initiative producing worthwhile results or meeting objectives?
Is the situation or problem better or worse as a result of the initiative?
Evaluating CLE | 12

13. Key Evaluation Questions
Some suggested categories of questions:
Efficiency
Is this cost effective for us to do given the impact or outcomes?
Would another approach provided the same results for less cost?
Sustainability
What is the capacity to build on the initiative?
What needs to be put in place to sustain the benefits?
Evaluating CLE | 13

14. Evaluation Plan: KEQs SMART objective KEQ Data Tools / Methods
Who? / When?
By delivering CLE on EPAs to residents in the Restful Nursing Home, 25% of participants will check that their EPA is current in the next month
Did the CLE address an identified need? 
(Appropriate)
Did participants check that their EPA was current in the next month? (Effective)
Was this type of CLE the most effective use of resources for the outcome? (Efficient)
Should we deliver the CLE to the retirement village? (Sustainable)
Participants develop Key Evaluation Question
Groups share – lollies for participation
Evaluating CLE | 14

15. Sources of data can be quantitative and/or qualitative
Evaluation Plan: Data
Quantitative examples – number of participants, number of downloads, install numbers, webpage hits
Sources of data can be quantitative and/or qualitative
Qualitative examples – participant feedback, stakeholder / partner feedback, stories
Ideally a mix of both
Evaluating CLE | 15

16. Evaluation Plan: Data SMART objective KEQ Data Tools / Methods
Who? / When?
By delivering CLE on EPAs to residents in the Restful Nursing Home, 25% of participants will check that their EPA is current in the next month
Did the CLE address an identified need?
(Appropriate)
Did participants check that their EPA was current in the next month? (Effective)
Was this type of CLE the most effective use of resources for the outcome? (Efficient)
Should we deliver the CLE to the retirement village? (Sustainable)
Information about the residents' knowledge of EPAs & their legal needs
Number of participants booked into CLE; participant feedback about future intention
Number of hours to develop and provide the CLE; participant feedback about its usefulness
Participant numbers;
Retirement village legal need
Participants develop data requirements against KEQ in the evaluation plan
Groups share – lollies for participation
Evaluating CLE | 16

17. Evaluation Plan: Methodology Toolbox
Choice of methodology dependent upon:
The Key Evaluation Question you are trying to answer!
Existing capability
Availability and accessibility of data
Data collection capacity
Ethical consideration
Access to different clients, stakeholders, users
Evaluating CLE | 17

18. Evaluation Plan: Methodology Toolbox
Examples:
Surveys
Interviews
Focus Groups
Observations/site visits
Document review
Case studies
Ranking exercises
“I” statements
What else can we add to the methodology toolbox?
Question to the group – lollies for participation
Evaluating CLE | 18

19. Evaluation Plan: Methodology Toolbox
SMART objective
KEQ
Data
Tools / Methods
Who? / When?
By delivering CLE on EPAs to residents in the Restful Nursing Home, 25% of participants will check that their EPA is current in the next month
Did the CLE address an identified need?
(Appropriate)
Did participants check that their EPA was current in the next month? (Effective)
Was this type of CLE the most effective use of resources for the outcome? (Efficient)
Should we deliver the CLE to the retirement village? (Sustainable)
Information about the residents' knowledge of EPAs & their legal needs
Number of participants booked into CLE; participant feedback about future intention
Number of hours to develop and provide the CLE; participant feedback about its usefulness
Participant numbers;
Retirement village legal need
Interviews
CLASS data;
Survey 
Record of project hours;
Survey
Evaluating CLE | 19

20. Evaluation Plan: Who & When
SMART objective
KEQ
Data
Tools / Methods
Who? / When?
By delivering CLE on EPAs to residents in the Restful Nursing Home, 25% of participants will check that their EPA is current in the next month
Did the CLE address an identified need?
(Appropriate)
Did participants check that their EPA was current in the next month? (Effective)
Was this type of CLE the most effective use of resources for the outcome? (Efficient)
Should we deliver the CLE to the retirement village? (Sustainable)
Information about the residents' knowledge of EPAs & their legal needs
Number of participants booked into CLE; participant feedback about future intention
Number of hours to develop and provide the CLE; participant feedback about its usefulness
Participant numbers;
Retirement village legal need
Interviews
CLASS data;
Survey 
Record of project hours;
Survey
Key nursing home staff prior to CLE planning and delivery
Internal analysis of CLASS data up to 3 months after CLE; Participant survey questions about intended next steps after CLE
Internal analysis of project hours compared with outcome discussions at operational planning; Participant survey about CLE usefulness after CLE
CLE participants after delivery;  Retirement village residents committee prior to decision to deliver CLE
Participants to complete the Tools / Methods and Who and When columns for their SMART objective – groups share
Evaluating CLE | 20

21. Evaluation Plan: YOUR TURN!
Develop an evaluation plan for your CLE project:
SMART objective
Key evaluation questions
Data needed to answer KEQs
Method / tool
Who and when
Individual work – participant use the reverse of the worksheet to develop a plan for their own project
Evaluating CLE | 21

22. Evaluation Fitness Test – Warm-down
Participants stand up:
Hands on head for yes to the statement
Hands on hips for no to the statement
I feel have more skills to evaluate CLE
I feel more confident about evaluating CLE
I will use the evaluation plan for my next CLE
Evaluation Fitness Test – Warm-down

23. Evaluating CLE| 3