Evaluating the Aboriginal Programs and Initiatives at the Department of Canadian Heritage Canadian Evaluation Society Conference 2003 Vancouver, British Columbia

Purpose of Presentation k Provide context to the evaluations within the Aboriginal Affairs Branch at PCH k Present two methodological approaches  Aboriginal Languages Initiative – project site visits  Aboriginal Women’s Program – Family Violence Initiative – learning circle k Discuss evaluation challenges and lessons learned k Discuss next steps

Aboriginal Affairs Branch - PCH k 13 programs and initiatives  Focussed mainly off-reserve  Pan-Aboriginal in nature k Developed over some thirty years  Multifaceted approach to complex Aboriginal issues  Facilitate participation in Canadian society  Strengthen Aboriginal cultural distinctiveness

Types of Program Support k Sustaining – established Aboriginal organizations  Representative, women’s, friendship centres, broadcasting  National, provincial/territorial, regional k Project k Partnering  Accords with territorial governments – mutual goals  Aboriginal organizations

Program Delivery – 3 approaches k Canadian Heritage administration  Aboriginal Affairs Branch  Regional offices k Aboriginal organizations k Six western cities – youth advisory councils

Comprehensive Review k Retrospective study k Environmental scan k Consultations – Aboriginal organizations/groups k Evaluation of 13 programs and initiatives

Evaluation Challenges k Complex approach to program delivery k Ensuring a collaborative approach (involvement of delivery partners) k Ensuring cultural sensitivity k Availability of baseline data and performance indicators

Evaluation of the Aboriginal Languages Initiative k Evaluation Methods  Document and File Review  Secondary Research and Statistical Review  Key Informant Interviews  Case studies of delivery organizations (AFN, ITK and MNC)  Project Site Visits – 10 k Working group and steering committee comprised of representatives from PCH (Aboriginal Affairs Branch and Corporate Review Branch) AFN, ITK, MNC)

ALI Project Site Visits k 2 Inuktitut sites, 2 Michif sites, 6 First Nations languages sites k Sites were selected to include all regions and geographic locations (near urban, rural, and isolated) and from each type of “project cluster”

Opportunities and Benefits k Opportunities  Conduct face-to fact interviews with those involved directly and indirectly with the project  Conduct focus groups/dialogue circles with project beneficiaries and other community people k Benefits  Observe projects and activities directly  Place project in context of the community within which it operates  Obtain information on program impact directly from beneficiaries - how they define success -  Obtain information in a culturally sensitive manner  Communities see themselves as part of evaluation process

Evaluation of the AWP Family Violence Initiative k Evaluation Methods Document & File Review Key Informant Interviews Learning Circle

What is a Learning Circle? k Gathering of people from different organizations k Opportunity for open sharing and exchange k Inclusion of range of projects and perspectives

Learning Circle Process Sharing knowledge What have we learned? Issues and challenges Stories and journeys Common threads Future direction

Learning Circle Opportunities and Benefits Opportunities to: Share experiences and insights Learn Explore ideas Reduce isolation Benefits: Connection at the community level Meaningful results Lessons learned Inspiration & momentum Future program directions

Disadvantage of methodological approaches k Costly and time consuming k Results from site visits or learning circles are not statistically representative of the results of the program as a whole – findings need to be balance with other lines of evidence k One time assessment rather then continuous measurement of results

k Importance of including Aboriginal communities in evaluation work – legitimizes results k Qualitative data should be balanced with quantitative information k Evaluations can be designed that meet both government and Aboriginal community information needs k Importance of information sharing Lessons Learned

k Analysis of information k Development of position paper k Consultation k Development of a new consolidated policy Aboriginal Affairs Branch Next Steps…

Evaluation reports available electronically Department of Canadian Heritage Website

Presenters k Audrey Greyeyes, Aboriginal Affairs Branch, PCH (819) k Dianne Lepa, Corporate Review Branch, PCH (819) , k Wanda Jamieson, JamiesonHartGraves Consulting (613)