1 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Culturally Appropriate Data Collection Methods How can we be respectful and culturally sensitive when collecting information?

2 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Evaluation is not culture-free! How can we be culturally sensitive, aware and respectful in our evaluation work? Cultural competence in evaluation is “a systematic, responsive inquiry that is actively cognizant, understanding, and appreciative of the cultural context in which the evaluation takes place.” (SenGupta, Hopson, Thompson-Robinson, “Cultural competence in evaluation: An overview”. New Directions for Evaluation, 102:5-19.

3 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Culture is an essential and integral part of working in communities and organizations. Communities and organizations are diverse in many ways, and these differences are a source of valuable perspectives and insights. Ignoring them can result in potentially invalid and harmful evaluations. Embracing them can result in credible findings and strong evaluations.

4 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation When we talk about culture we are talking about: Nationality Ethnicity Religious affiliation Regional contexts Gender Age Abilities Social class Economic status Language Sexual orientation Physical characteristics Organizational affiliation

5 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Take a few moments to reflect: What does culture have to do with the way we plan and implement an evaluation? How does culture affect how we collect data? The following slides provide some things to think about as you consider different data collection methods.

6 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Is a written questionnaire culturally appropriate? Things to consider: Literacy level Tradition of reading, writing Setting Not best choice for people with oral tradition Translation (more than just literal translation) How cultural traits affect response – response sets How to sequence the questions Pretest questionnaire may be viewed as intrusive

7 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Are interviews culturally appropriate? Things to consider: Preferred by people with an oral culture Language level proficiency; verbal skill proficiency Politeness – responding to authority (thinking it’s unacceptable to say “no”), nodding, smiling, agreeing Need to have someone present Relationship/position of interviewer May be seen as interrogation Direct questioning may be seen as impolite, threatening, or confrontational

8 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Are focus groups culturally appropriate? Things to consider: Issues of gender, age, class, clan differences Issues of pride, privacy, self-sufficiency, and traditions Relationship to facilitator as prerequisite to rapport Same considerations as for interview

9 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Is observation culturally appropriate? Things to consider: Discomfort, threat of being observed Issue of being an “outsider” Observer effect Possibilities for misinterpretations

10 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Cultural issues related to use of existing data/records Need careful translation of documents in another language May have been written/compiled using unknown standards or levels of aggregation May be difficult to get authorization to use Difficult to correct document errors if low literacy level

11 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Culturally appropriate informed consent How can we be culturally sensitive and respectful and ensure the protection of those involved in our evaluations? –Children –Marginalized, “less powerful” participants

12 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation Reflection time What is one thing you learned (or had reinforced) from going through this presentation that you had not thought of before?

13 © 2009 University of Wisconsin-Extension, Cooperative Extension, Program Development and Evaluation May we recommend reading: Culturally respectful evaluation by D. McDonald, P. Kutara, L. Richmond and S. Betts. In The Forum, Dec 2004, Vol. 9, No.3 at december/ar-1-cuturally.php december/ar-1-cuturally.php