1. Workshop 1st International Society for Child Indicators Conference Chicago, IL June 26-28, 2007 Presenters: Brenda Moody, Amin Malik, Brad Bain, Yosi Derman & Mark Kartusch

2. Overview Context Methodology & challenges Small group activity Literature Lessons learned & discussion

3. Ontario, Canada

4. Child Welfare in Ontario Child and Family Services Act 53 independent, non-profit agencies Total budget $1.2 billion Caseload 82,346 investigations 26,378 open protection cases 18,497 children in care

5. Child Welfare in the Central Zone Eight diverse agencies in the Greater Toronto Area Represent approximately 1/3 of expenditures and caseloads in Ontario child welfare Central Zone Quality Assurance Committee

6. Accountability Levels National (CDN) Outcomes Indicator Matrix

7. Child Welfare National (CDN) Outcomes Indicator Matrix Outcome Outcome Indicators Child Safety  Recurrence of Maltreatment  Serious Injuries/Deaths Child Well-Being  School Performance (grade level/graduation)  Child Behaviour/Youth Criminal Justice Act Charges (YCJA) Permanence  Placement Rate  Moves in Care  Time to Achieving Permanent Placement Family and Community Support  Family Moves  Parenting Capacity  Ethno-Cultural Placement Matching

8. Approaches to Indicator Development DataPolicy TheoryField

9. Safety of Children - Indicators Serious occurrences New investigations on open cases Recurrence of maltreatment Consistent process for data: Defining Gathering Analyzing Reporting Interpreting

10. Methodology Determine how many of the total closed cases in a fiscal year were re-opened within 12 months. For cases that had multiple closings within the same fiscal year, use the first closing in the calculation. April 1 March 31 Closed Year 1 What happened 12 months forward? Year 2

11. Cases of Reopening Total number of unique families that had at least one reopening Calculate the number of cases reopened as a percentage of the number of cases closed

12. Cases of Reopening

13. Incidents of Reopening All incidents of reopening including more than one opening for the same family Calculate the number of incidents of re-openings as a percentage of the number of cases closed

14. Incidents of Reopening

15. Challenges Defining variables Data extraction Formulas Multiple databases Data analysis & interpretation Limitations Who does it? Inability to drill down into the data Data sharing Culture of secrecy - Provincially & organizationally What to do with the findings

16. A Tale of Two Counties… This tale transpires in the Sunshine State of Idyllia. You are the CEO of Donjon County Child Protection Service (CPS). Idyllia requires each CPS to evaluate the safety of children on its caseload─by measuring the recurrence of maltreatment according to the “Toronto” model. The child welfare authority of neighboring Paradise County finds that its recurrence rate is 25%. A week later, your IT manager informs you that she has found that YOUR recurrence of maltreatment rate is 45%. You have just received an e-mail from the Idyllia State Department of Children’s Services, wishing to discuss the performance of your Agency. Realizing that 45% looks quite bad compared with 25%, you have two choices: A. Bite down on your cyanide pill (secreted for just such an occasion in a handy wisdom tooth filling). B. Take a closer look at your data to see if you can understand—and help others understand—factors that may contribute to the measured differences between the agencies. If you choose B—what might you find that could potentially protect your Agency’s reputation (and your career)?

17. Identifying & Defining Key Variables Time frame Frequency Severity Type of maltreatment Referral source Verified / substantiated Child’s age Perpetrator Subject child Across & within jurisdiction Service type / level

18. Recurrence of maltreatment literature - Definitions No accepted standard definition Reliant on research objectives & availability of data Has been defined as: any subsequent report of maltreatment; any subsequent founded or verified report of maltreatment; any subsequent maltreatment of the same child, of another child within the family, or by the same perpetrator; recurrence of maltreatment without a prior report; and a combination of these

19. Recurrence of maltreatment literature - Factors Child age History/prior referrals Frequency of previous incidents of maltreatment Neglect or parent absence Parental conflict Parental mental health

20. Recurrence of maltreatment literature - Patterns Rate of maltreatment recurrence: 14.7% at 6 months 22.6% at 18 months Greatest risk soon after an index episode Intensity of intervention seems to reduce rate Risk of recurrent maltreatment increased after each maltreatment Time between episodes of maltreatment shortened as the number of episodes increased. Children with prior contacts tend to have > 2 contacts

21. Lessons Learned Developing outcome measures & indicators Defining indicators Gathering data Analyzing & reporting on indicators Interpretation

22. Where has our work led us? Pilot the data definitions Re-collect & analyze child indicators Complete Ministry Baseline Project Institutionalize in the Single Information System Adopt consistent approaches to indicator development Select additional indicators to measure

23. Thank you