1. Formative Evaluation as Community Development Daniela Stehlik Alcoa Research Centre for Stronger Communities Curtin University of Technology Lesley Chenoweth School of Social Work & Applied Human Sciences The University of Queensland

2. Outline of presentation Key Learnings Program Context Formative Evaluation Philosophy Evaluation Dimensions Methods adopted Strengths & Challenges

3. Key Learnings -Building Communities through Evaluation Spatial communities Practice communities Human service communities

4. Program Context Rural/remote People with disabilities and their families Government staffed and funded Pilots Quest for better service delivery models

5. Formative Evaluation Philosophy Framework Social Justice Participatory Action research principles Ethical practice

6. Formative Evaluation Philosophy Framework - how? Evaluation as staff development Through Stakeholder Reference Group Involved in professional development activities Confidentiality/anonymity

7. Evaluation dimensions Complexities Multi-level interventions Cultural change agenda Searching for ‘best practice’ Pressure to get evidence ‘out’ Rural/remote Spectrum of disability service availability

8. What were the indicators used? From Program goals –At the level of …

9. What were the indicators used? From Program goals –At the level of … Government Policy Dept’l Program management Field Practice People with a disability and their families Community capacity building

10. What were the indicators used? From Program goals –At the level of … Government Policy (4) Dept’l Program management (3) Field Practice (6) People with a disability and their families (5) Community capacity building (7) 22 levels ….

11. Methods adopted …. Focus groups Questionnaires Indepth interviews Measurement of social relationships Participant observation Client records analysis Practioner as researcher

12. Evaluation dimensions Complexities Multi-level interventionsMulti-level interventions Cultural change agenda Searching for ‘best practice’ Pressure to get evidence ‘out’ Rural/remote Spectrum of disability service availability

13. Multi-level Interventions Community FamiliesIndividuals

14. Multi-level Interventions Community FamiliesIndividuals Program addressed each level and the interactions between

15. Multi-level Interventions Community FamiliesIndividuals Program addressed each level and the interactions between

16. Multi-level Interventions Community FamiliesIndividuals Program addressed each level and the interactions between

17. Multi-level Interventions Community FamiliesIndividuals Program addressed each level and the interactions between

18. Multi-level Interventions Community FamiliesIndividuals Program addressed each level and the interactions between

19. Key Learnings -Building Communities through Evaluation Spatial communities –Worker/families/ngos/govt etc. Practice communitiesPractice communities –Workers/ngos/govt/academe Human service communities –govt./ngos/academe

20. Building the Practice Community

22. CD in practice The role of the practitioner Capacity building doesn’t just ‘happen’ Facilitation & Intervention are required Building a 21st century model of human service practice

23. Strengths of Evaluation Approach Accommodated complexity Provided both quantifiable and illustrative evidence Gave value beyond the evaluation per se Enabled participation including people with a disability

24. Strengths cont. Built practitioner confidence Enhanced program profile within dept. Worked in longitudinal and short term situations Enabled timely feedback of findings Aided writing of report

25. Challenges Assumed high level evaluator knowledge of program, practice & issues Political sensitivity Distance across sites limited participant observation Demand for face to face contact Problematic access to client databases & records

26. Thank - you