1. Carolyn Sullins, Ph.D. Ladel Lewis, Ph.D. candidate The Kercher Center for Social Research Western Michigan University

2.  Comprehensive Community Mental Health Services for Children and Their Families Program: “Systems of care.”  incorporates a broad, flexible array of effective services and supports for a defined, multi-system population that is organized into a coordinated network… is culturally and linguistically competent, builds meaningful partnerships with families and youth at service delivery, management and policy levels, and has supportive policy and management infrastructure. (Pires, Lazear, & Conlan, 2008).

3.  144 sites have been or are in the process of being evaluated. Each 1-2 year cohort: +/- 30 sites.  Each SoC has distinct:  geographic location and scope (e.g., statewide, county wide, city-wide, tribal)  Ages of the youth served  Mental health issues facing the targeted youth  Racial, ethnic, and cultural factors

4.  Child c e ntered and family driven  Community based  Culturally competent

5.  Presenting issues of the youth  Youth’s level of functioning (strengths and weaknesses)  Family strengths and barriers  Types of services family and youth are receiving  Satisfaction with services  Cultural competence of services  Youth and family input into services

6.  “One size fits all” battery of questions  HSIRB mandates re language on consent forms  Ensuring an adequate sample size  Ensuring retention in a mobile population

7.  History of racist abuse by researchers (E.g., Tuskegee syphilis study)  Misinterpretation of data, or no access to results  Sensitive or stigmatized topics even more difficult  Families overwhelmed or embarrassed

8. Caucasian: 49.1% (including White Latino/a: < 5%) African-American/Multiracial: 50.9% Ages 7-17 Diagnosed with a Severe Emotional Disturbance

9.  Informed consent  Voluntary participation  Confidentiality and its exceptions

10. Parents, various social service workers, eval staff  Reviewed consent forms for clarity  Gave opinions to HSIRB re child abuse reporting  Parent input re: communication among clinicians, families, and interviewers  Interpretation of data  Reporting of results

11.  We came to their group  Help re local language  Helped us make it more comfortable for participants  Info that later helped us interpret data

12. Does confidentiality mean…  No interviews in public places, even if that’s what participants request?  Kicking Grandma out of the room?  Pretending you don’t see participant in public?

13.  Some found it emotionally draining  Too long and redundant  Questionnaires with overlapping questions  Categories of services – national vs. local terms  Keeping in touch with families every 6 months

14.  “Evaluating System of Care – not you”  Yet up front about sensitive, personal questions  Non-judgmental attitude for better rapport, retention, AND accuracy.  Balance – we can’t act as friends or counselors!  If SoC not working, or not working with all groups of people, we need to know.

15.  Offer breaks, gum, stress balls, etc. to participants  Coloring books, DVDs for young kids  Redundant questions from multiple surveys: propriety and accuracy trump methodological “letter of law.”

16.  Family address tracking form  Birthday and holiday cards with coupons  Incentives for families to contact us  Annual dinner as a “thank you”  Results in bimonthly newsletter, website, and other venues

18. Chi Square (1, N=224) =.466, p =.495

19. Chi Square= (1, N= 145) =.665, p =.415

20. Chi Square= (1, N=119) =.511, p =.561

21. Chi Square= (1, N=91) =.071, p =.834

22. Chi Square= (1, N=52) = 1.055, p =.402

23.  Fulfilling national evaluation requirements within a local context is a constant balancing act.  With enough input from local stakeholders at each stage, it can be achieved!

24.  Please contact carolyn.sullins@wmich.edu carolyn.sullins@wmich.edu  Or Ladel_lewis@yahoo.comLadel_lewis@yahoo.com  For more info, please see also  http://www.wmich.edu/sociology /kzoowraps.html http://www.wmich.edu/sociology /kzoowraps.html