Carolyn Sullins, Ph.D. Ladel Lewis, Ph.D. candidate The Kercher Center for Social Research Western Michigan University

 Comprehensive Community Mental Health Services for Children and Their Families Program: “Systems of care.”  incorporates a broad, flexible array of effective services and supports for a defined, multi-system population that is organized into a coordinated network… is culturally and linguistically competent, builds meaningful partnerships with families and youth at service delivery, management and policy levels, and has supportive policy and management infrastructure. (Pires, Lazear, & Conlan, 2008).

 144 sites have been or are in the process of being evaluated. Each 1-2 year cohort: +/- 30 sites.  Each SoC has distinct:  geographic location and scope (e.g., statewide, county wide, city-wide, tribal)  Ages of the youth served  Mental health issues facing the targeted youth  Racial, ethnic, and cultural factors

 Child c e ntered and family driven  Community based  Culturally competent

 Presenting issues of the youth  Youth’s level of functioning (strengths and weaknesses)  Family strengths and barriers  Types of services family and youth are receiving  Satisfaction with services  Cultural competence of services  Youth and family input into services

 “One size fits all” battery of questions  HSIRB mandates re language on consent forms  Ensuring an adequate sample size  Ensuring retention in a mobile population

 History of racist abuse by researchers (E.g., Tuskegee syphilis study)  Misinterpretation of data, or no access to results  Sensitive or stigmatized topics even more difficult  Families overwhelmed or embarrassed

Caucasian: 49.1% (including White Latino/a: < 5%) African-American/Multiracial: 50.9% Ages 7-17 Diagnosed with a Severe Emotional Disturbance

 Informed consent  Voluntary participation  Confidentiality and its exceptions

Parents, various social service workers, eval staff  Reviewed consent forms for clarity  Gave opinions to HSIRB re child abuse reporting  Parent input re: communication among clinicians, families, and interviewers  Interpretation of data  Reporting of results

 We came to their group  Help re local language  Helped us make it more comfortable for participants  Info that later helped us interpret data

Does confidentiality mean…  No interviews in public places, even if that’s what participants request?  Kicking Grandma out of the room?  Pretending you don’t see participant in public?

 Some found it emotionally draining  Too long and redundant  Questionnaires with overlapping questions  Categories of services – national vs. local terms  Keeping in touch with families every 6 months

 “Evaluating System of Care – not you”  Yet up front about sensitive, personal questions  Non-judgmental attitude for better rapport, retention, AND accuracy.  Balance – we can’t act as friends or counselors!  If SoC not working, or not working with all groups of people, we need to know.

 Offer breaks, gum, stress balls, etc. to participants  Coloring books, DVDs for young kids  Redundant questions from multiple surveys: propriety and accuracy trump methodological “letter of law.”

 Family address tracking form  Birthday and holiday cards with coupons  Incentives for families to contact us  Annual dinner as a “thank you”  Results in bimonthly newsletter, website, and other venues

Chi Square (1, N=224) =.466, p =.495

Chi Square= (1, N= 145) =.665, p =.415

Chi Square= (1, N=119) =.511, p =.561

Chi Square= (1, N=91) =.071, p =.834

Chi Square= (1, N=52) = 1.055, p =.402

 Fulfilling national evaluation requirements within a local context is a constant balancing act.  With enough input from local stakeholders at each stage, it can be achieved!

 Please contact  Or  For more info, please see also  /kzoowraps.html /kzoowraps.html