1. Innovations in Measuring Outcomes in Children’s School Performance and Social Functioning Manitoba’s Child Well-Being Windows Presenters: Linda Burnside and Kathy Kristjanson

2. 2 Introduction Like other jurisdictions around the world, Manitoba has become increasingly interested in how we assess and measure the well-being of our children. This interest is particularly critical in the child welfare field, where the state has assumed parental responsibility for the care of children and therefore, responsibility for their well-being.

3. 3 Challenges The issues that bring children to the attention of child welfare agencies are multi-dimensional, which adds to the complexity of defining and measuring child well-being. Child welfare agencies may be reluctant to measure child well-being, given that the issues that result in many children coming to the attention of child welfare are issues that have likely already compromised their well-being.

4. 4 Purpose of Presentation To share Manitoba’s approach to measuring and tracking child well-being for children in care, in the following areas: –School performance Age/Grade Appropriateness School Type Literacy Learning Disabilities Speech and Language Development –Child Functioning Social Skills Problem behaviors Substance Abuse

5. 5 Acknowledgements The following individuals have been instrumental in advancing Manitoba’s work on the measurement of child well-being: –Nico Trocmé –Laurie Olafson –Rhonda Dagg –Dave McGregor

6. 6 Manitoba CFS Context 6629 (the number of children in care of Manitoba child and family services agencies at March 31, 2006) 7364 (the number of families receiving service from Manitoba child and family services agencies) 25 (the number of agencies operating under four child and family services Authorities) 133 (the number of child and family services offices operating in Manitoba)

7. 7 Manitoba CFS Context Winnipeg Brandon

8. 8 Methodology for Demonstration Project Looked at School Performance and Social Skills data for all children in care at one child welfare agency in Manitoba. Demonstration data only, to illustrate capacity of the windows, based on one agency.

9. 9 Demonstration Study Sample CFS Western Manitoba 2005/06

10. 10 Manitoba’s Approach

11. 11 National Outcome Measures Project The National Outcome Measures Project originated with the provincial Directors of Child Welfare across Canada. Intent of this Canadian initiative is to measure multi-dimensional child welfare outcomes in four domains (Trocmé, Nutter, MacLaurin, & Fallon, 1999): –Child safety –Child well being –Permanence –Family and community support

12. 12 Child Welfare Outcome Indicator Matrix* Child Safety –Recurrence of Maltreatment –Serious Injuries/Death Child Well Being –School Performance –Child Behavior Permanence –Moves in Care –Admission to Care –Time to Achieving Permanent Placement Family and Community Support –Family Moves –Aboriginal Placement Matching –Parenting Capacity * Trocmé, Nutter, MacLaurin, & Fallon, 1999

13. 13 Informed by Other Models Looking After Children (LAC) is a UK- based approach to working with children in care which measures a child’s developmental progress in seven dimensions: –health –education –identity –family and social relationships –social presentation skills –emotional and behavioral development –self-care

14. 14 Information Technology The Child and Family Services Information System (CFSIS) for Manitoba is a comprehensive case tracking system that was designed in 1993 to facilitate the tracking of children and families served, as well as reporting on the services being provided. The Intake Module, which began in 2004, functions in conjunction with CFSIS to provide standardized recording of intake information throughout the province as required by legislation.

15. 15 Child Well Being Windows

16. 16 Child Well Being Windows

17. 17 School Information

18. 18 Social Skills

19. 19 Data Collection and Measurement The applications create unique person records which may be attached to multiple intakes or CFSIS cases. CFSIS has a total of 436 tables which collect data from 3372 separate data fields. Any mix of this data can be produced in an ad hoc query.

20. 20 Data Collection and Measurement Information related to child well-being is updated annually. All well-being windows have 2 print functions. One will print a blank form that workers can print off to have filled out by foster parent, teacher, etc. The other print function shows all the data that has been entered.

21. 21 Benefits of Approach Child-specific data gathered by those who know the child best. Capacity to compare for each unique child with himself/herself over time (facilitates case planning and intervention for children who may need assistance in school performance or social skills development). Capacity for aggregate comparisons across agencies, regions, cultural groups, legal statuses, placement type, etc.

22. 22 Limitations of Approach Data may be affected by: –accurate data entry, –consistent definitions of terms, –how long the worker/teacher/foster parent has worked with the child, –how well the worker/teacher/foster parent knows the child, –turnover of staff, changes in placement, school changes, –behavioral/emotional issues that interfere with child’s expression of abilities.

23. 23 Limitations of Approach Measurement may be affected by changes in child-in-care population due to: –new admissions to care, –discharges from care, –children reaching age of majority.

24. 24 School Performance

25. 25 School Performance for Children in Care Several studies have found that youth in care are less likely to perform at age- appropriate grade level than their peers (e.g. Carey, Concannon & Goldschmidt, 1990). Research has also shown that children in care are less likely to graduate from high school (e.g. Barth, 2000; Westat, 1994; Roos, Brownell, & MacWilliam, 2007).

26. 26 Manitoba School Performance Measures Age/Grade Appropriateness Literacy Numeracy Speech and Language Learning Disabilities School Supports Type of Classroom Attendance School Changes

27. 27 School Performance A recent study of children in care involved with LAC found that 25% of youth were below an age appropriate grade level (Kufeldt, Simard, Tite & Vachon, 2003)

28. 28 Age-Grade Appropriateness 2005/06 CFS Western Manitoba Children in Care Attending School

29. 29 School Grade Range 2005/06

30. 30 Age-Grade Appropriateness and School Type

31. 31 Age-Grade Appropriateness and School Type/Grade Range

32. 32 Not Age-Grade Appropriate and School Type/Grade Range

33. 33 School Performance LAC study found that 40% of children ages 10 – 14 stated that they had a learning difficulty.* For youth ages 15 – 19, 27% stated that they had a learning difficulty.* *Based on youth self-reporting

34. 34 Learning Disability and Grade Range

35. 35 Learning Disability and School Supports LAC study found that only 9% of children who self-identified difficulty with learning were receiving extra help.

36. 36 Learning Disability and School Supports 26 (35%) of the total 74 children in school were identified with a learning disability. 24 (92%) were receiving additional supports in the school. 18 (24%) children were not identified as having a learning disability, but were still receiving additional supports in school.

37. 37 Learning Disability and School Supports For the 63 children who were in an age- appropriate grade: 20 were identified with a learning disability and 18 of them were receiving school supports (90%). 43 children were not identified with a learning disability. However, 16 of these children were receiving school supports.

38. 38 Learning Disability and School Supports For the 11 children who were not in an age- appropriate grade: 6 children were identified with a learning disability, and all 6 were receiving school supports (100%). 5 children were not identified with a learning disability. However, 2 of these children were receiving school supports.

39. 39 Learning Disability and School Supports Teaching assistant Resource support Counselling Reading assistance Occupational therapy Speech/language therapy Psychology Audiology Psychiatry

40. 40 Literacy and School Type Shin (2003) found that receiving special education was related to poorer reading skills.

41. 41 Literacy and School Type Of the 54 children attending regular class settings, 18 required ongoing assistance with literacy (33%). Of the 20 children attending special class settings, 13 required ongoing assistance with literacy (65%).

42. 42 Literacy and Placement Type Shin (2003) found that youth in kinship care had higher reading levels than those who were not in kinship care.

43. 43 Literacy and Placement Type Of the 74 children in care, 46 were in a foster home, with 19 (41%) of these children requiring assistance with literacy. Of the 15 children placed in kinship settings, 5 (33%) required assistance with literacy.

44. 44 School Changes Only 8% of the LAC study children had no school changes. 30% of the LAC sample had more than 6 school changes.

45. 45 School Changes 57 children of the 74 children in care attending school experienced no change in school (77%). 17 children did experience a change in schools (23%).

46. 46 School Changes and Age-Appropriate Grade For the 63 children in an age-appropriate grade, 14 experienced a change in school (22%). For the 11 children not in an age- appropriate grade, 3 experienced a change in school (27%).

47. 47 Speech and Language

48. 48 Speech and Language and Literacy Of the 74 children in school, 48 (65%) meet or exceed expectations in speech and language performance. Of those 48, 41 meet or exceed expectations in literacy (85%). Of the 26 who require ongoing help in speech and language, only 2 met expectations in literacy (8%) and 24 required ongoing help in literacy (92%).

49. 49 Child Functioning

50. 50 Manitoba Social Skills Measures Emotional Maturity Making Friends School Activities Problem Behaviors Substance Abuse

51. 51 Emotional Maturity

52. 52 Making Friends

53. 53 Extracurricular Activities/Educational Attainment Shin (2003) found that active participation in extracurricular activities has a strong association with educational attainment.

54. 54 School Activities/ Cognitive Development Of the 63 children in an age-appropriate grade, 23 were involved in school activities (37%). A total of 37 of the 63 children were meeting school expectations in terms of cognitive development.

55. 55 School Activities/ Cognitive Development Of the 11 children who were not in an age- appropriate grade, 4 were involved in school activities (36%). A total of 6 were meeting school expectations in terms of cognitive development.

56. 56 Behavioral Issues Inability to manage own behavior Inability to interact constructively with others Aggression Inability to adhere to structure and expectations Not aware of risk to self/others Out of control behavior Illegal activity Sexually inappropriate acting out

57. 57 Substance Abuse Alcohol Marijuana Substance not identified Cocaine Ecstasy Prescription drugs

58. 58 Substance Use and Literacy Shin (2003) found that youth who were involved in drug use had lower reading skills than youth who had not used drugs.

59. 59 Substance Use and Literacy Of the 74 children attending school, 11 were identified as having a substance abuse issues (15%). Of those 11 children, 6 required ongoing help with literacy (55%). Of the 63 children who did not have substance abuse issues, 24 required ongoing help with literacy (39%).

60. 60 Substance Abuse and Attendance School attendance is a concern for 8 of the 11 children with substance abuse issues.

61. 61 Substance Abuse/ Speech and Language Of the 74 children attending school, 26 require ongoing help with speech and language. 5 of these children have an identified substance abuse issue.

62. 62 School Change/ Substance Abuse The LAC study found that school disruption is highly correlated to the risk of substance use.

63. 63 School Change/ Substance Abuse A total of 17 children had a change in school. Of those, 6 had a substance abuse issue (35%).

64. 64 Implementation Challenges Manitoba is initiating a process to provide agencies with the necessary computer hardware and staff training to allow for the use of CFSIS throughout the province. Further, all agencies will use the same information system process for new intakes, which will allow for the uniform collection of data at the onset of service.

65. 65 Opportunities Valuable source of information for social workers, agencies, and the government. Strategy to further measurement of child well-being, both for National Outcome Measures Project and for the knowledge base of outcome measurement. Research opportunities that can be generated from the data collected.

66. 66 Questions? For further information, please contact the Manitoba Child Protection Branch, c/o: Linda Burnside Child Protection Branch Manitoba Department of Family Services and Housing 201 – 114 Garry Street Winnipeg, Manitoba R3C 4V5 Canada Linda.Burnside@gov.mb.ca