Donna M. Mertens Independent Consultant Minnesota Evaluation Studies Institute, MESI Spring Training Minneapolis MN March 2015

 Name, position, organization (no acronyms please).  Where do you come from?  What is your experience in evaluation?  What has been your experience with social justice?  What do you want to get out of this workshop? Mertens MESI March /19/2015

PhilosophicalAssumptions AxiologyRespect for cultural norms; support for human rights and social justice; reciprocity OntologyIssues of power & critical interrogation of multiple realities: social, political, cultural, economic, race/ethnic, gender, age, religion and disability values to unmask those that sustain an oppressive status quo EpistemologyIssues of power & Interactive link; knowledge is socially and historically located; trusting relationship. Methodology Qualitative (dialogic)/ Quantitative / Mixed Methods; Context 2/19/2015 Mertens MESI March

2/19/2015Mertens MESI March Phase 1 Qual: Identify stakeholder s; build relationship s; analyze documents Quant: Identify available demographic and environmenta l data Phase 2 Qual: Interview s, focus groups, town meetings Quant: Establish baseline for pollutants and health status Phase 3 Qual: Develop interventi ons based on stakehold er input Quant/Qual: Pilot test intervention s & data collection Phase 4 Quant: process assessment of implementatio n; pretests Qual: observe and interview stakeholder groups during intervention Phase 5 Quant: Post- test measures Qual: Interviews, observations, policy planning

 Explicit goals?  Implicit goals?  Contradiction between explicit goals and behaviors?  Changes in stance over time; changes in personnel over time  Explicit: Deaf people in the courts  Implicit: I don’t really need to deal with these multicultural issues because we are all white and well educated/ of British extraction 2/19/2015Mertens MESI March

 “ism” present: depicting intended beneficiary as “less than” member of powerful majority  assuming problem is knowledge-based – not social/cultural/oppression  Problem: Self esteem; they just don’t care; have babies to get welfare $$  Problem: Ignorance (people with DA voting in Pakistan; HIVAIDS in Botswana) 2/19/2015Mertens MESI March

 Tell them what they need to know, e.g., you have a right to vote and this is how you do it; you can say no or use a condom  Acknowledge the complexity of the solution that is needed as compared to the services that are offered; develop partnerships to address complexity  Use prior research and evaluation and applicable theories (e.g., developmental psychology or change theory); best practices  Consult with community about potential solutions: Address the meaning of the experience in their lives (HIVAIDS), connect solution to experienced needs (dancing, healthy cooking) 2/19/2015Mertens MESI March

 Who to involve? How to involve?  Involvement throughout entire cycle of study: questions, methods, data collection, use  Notice: who is not here? Deaf people? Only people who I am comfortable with? 2/19/2015Mertens MESI March

 Use low tech radio shows to solicit community members to make contact via mobile phones; do follow up interviews  Use smartphones for short surveys  Web based recruitment; data collection and dissemination  GPS and mapping  An uncomfortable topic: sex with children; drug dealing; gang activity; sex with same sex partner 2/19/2015Mertens MESI March

 Appeal to shared desire to provide effective services addressing the needs of marginalized populations: process and impact evaluation  Argue for the primacy of the needs of the community and disadvantaged clients  Find a champion  Build a coalition  They don’t want this type of service; they won’t come…. 2/19/2015Mertens MESI March

 Finding higher ground  What are you going to do if you have negative findings? 2/19/2015Mertens MESI March

 Most Significant Change: Davies and Dart (2005)  ‘During the last month, in your opinion, what was the most significant change that took place for participants in the program?’ p. 10  Other PAR strategies (Robert Chambers)  Know the literacy level; comfort in speaking in groups  Inuit culture: Children would not speak to the evaluator. 2/19/2015Mertens MESI March

 How to keep people safe?  Sex with same sex partner is not accepted in culture, is against the law, could be punished by death 2/19/2015Mertens MESI March

 Wealth and status, gender, ethnicity, age group, sexual orientation, disability, deafness, refugee, immigrant, religion  Homogeneous groups; individual interviews  Supportive accommodations  Respect and meaningful involvement 2/19/2015Mertens MESI March

 Present actionable results; identify clear options for possible results and associated actions  Identify strategies to overcome barriers and support change  Share results in clear and understandable manners to full range of stakeholders  Share results with those who can advocate for change (Cook, 2014)  South Asia policy makers and program developers 2/19/2015Mertens MESI March

 Form coalitions  Press releases  Transparency  Present to multiple audiences in multiple formats 2/19/2015Mertens MESI March

 Advocate for good evaluation (standards); advocate for justice  Evaluators have to be objective; they can’t be advocates. 2/19/2015Mertens MESI March

 Mertens, D. M. & Wilson, A. (2012). Program Evaluation Theory and Practice: A Comprehensive Guide. NY: Guilford.  Mertens, D. M. (2015). Research and evaluation in education and psychology: Integrating diversity with qual, quant and mixed methods. 4th ed. Thousand Oaks, CA: Sage.  Mertens, D. M. (2009). Transformative research & evaluation. NY: Guilford.  Mertens, D. M. & Ginsberg, P. (2009).(Eds.) Handbook of Social Research Ethics. Thousand Oaks, CA: Sage.  American Evaluation Association (2011). Public Statement on Cultural Competence in Evaluation. AEA. Mertens MESI March /19/2015

Donna M. Mertens, PhD Independent Consultant Mertens MESI March /19/2015