The challenges of being evidence-informed in out-of- home care policy and practice: exploring the potential of systematic maps Belinda Mayfield, BSW, PhD

Background/Contextual factors  AASW code of ethics – to maintain and expand levels of current knowledge, theory and skill  Barriers to accessibility of knowledge sources from the professional literature; time, oral culture, short timeframes for decisions, skill in searching, cost, gaps in research, debates about ‘evidence’  Audit of Australian out of home care research (Cashmore & Ainsworth, 2004)  Crime Misconduct Commission Inquiry (2005)  Out-of-home care: Messages from research (Bromfield,Higgins,Osborn,Panozzo & Richardson, 2005)  Emergence of systematic mapping to systematically and transparently describe the extent of research in a field, to identify gaps and to provide direct links to the evidence (Clapton et al., 2009)

Research aims Overall To investigate the issues of the accessibility of the out- of-home care literature as a source of knowledge to professionals in roles which influence policy and practice. Specifically, the aims were to: 1. Develop a systematic map of the out-of-home care literature 2. Provide a descriptive analysis of the composition of the available literature to identify gaps and areas of substantial knowledge 3. Gain insight about how different user roles view their knowledge needs and the barriers to accessibility 4. Gather feedback about the utility and limitations of the map

Why a map and not a systematic review  In social sciences interest in not only what works but for whom, in what circumstances, why and for what outcomes  Practitioners perspective – case management includes key decisions about removal, assessment of protective and care needs, placement matching, interventions etc. – broader scope than a review  Policy makers – often short timeframes eg post inquiry 12 weeks to prepare the Blueprint for reform; systematic reviews narrowly focused and 6 months + to undertake  Program development and management require different types of information, re cost effectiveness, feasibility, service and system level outcomes; new initiatives not yet evaluated  How theory, policy and practice literature may also inform….rather than directing what we do - to query/challenge current jurisdiction specific approaches; have we got it right or are there other ways eg permanency

Overview of study with 2 key phases: 1. A systematic map of out-of-home care literature Interviews with “users” to examine the utility of the map. User perspectives include: Researchers & knowledge ‘translators’ Researchers & knowledge ‘translators’ Policy makers, program managers & practitioners Policy makers, program managers & practitioners

Conceptual framework  Ecosystemic view of out-of-home care – built on Australian audit  Case management framework  Different types of knowledge, their interrelationships ( complementarities, inconsistencies and gaps), multiple uses  Models of knowledge utilisation

Model of Professional Knowledge (Drury-Hudson, 1997) Professional Knowledge Theoretical Knowledge Personal Knowledge Practice Knowledge Procedural Knowledge Empirical Knowledge

Accessibility - Exposure phase of Knowledge utilisation

Methodology of systematic mapping Initial planning, topic setting and preliminary work Establish inclusion/exclusion criteria Searching Screening and coding Analysis and development of map report Finalisation of the systematic map Dissemination

Methodology  Preliminary handsearch of a key UK journal and USA journal  Electronic searches to test individual search terms, prior to a ‘rolled up’ search  ( built on timeframes for Australian audit)  Searches in PsycInfo, ERIC, Social Services Abstracts,Medline,Family and Society Plus.

Search strategy Population or participant of interest termsCare leaver* State ward* Ward* of state Looked after child* Foster child* Foster adolescent* Emancipating youth Foster parent* Foster family* Foster mother* Foster father* Foster grandmother* Foster grandfather* Foster grandparent* Types of interventionFoster care* Foster home care Foster group care Out-of-home care Substitute care Alternative care State care Local authority care Care of local authority Kinship care* Non-kinship care* Relative care* Case management termsPlacement disruption* Placement break down* Placement stability Placement instability Failed placement* Disrupted placement* Placement failure* Placement move* Multiple placement* Concurrent plan* Permanency plan* Transition* from care Family reunification* Family restoration Emancipating from care Leaving care Ageing out of care

Findings  Content findings  Methodological findings  Knowledge needs of different users  Barriers and enablers to accessibility of the out-of-home care literature

Overall screening results Total reports identified n=12,939 (3,229 duplicates) Total abstracts screened n=9,710 Not on topic n=660 Not child protection population n=694 Not specific to out-of-home care n=2,986 Risk assessment n=146 Family preservation n=110 Prevention and early intervention n=137 Screened out n=5,285 Book reviews n=234 Program description n=387 Literature reviews n=207 Conceptual or theoretical n=981 Research studies n=2,616 Screened In N=4,425 Insufficient data to screen n = 945

Overall composition of the literature

Trends in publications reporting research Year Australian Research publications Total Research publications

Number of publications by jurisdiction

Trend in publications reporting literature reviews Year Number of reviews

Analysis of composition of content areas  Examined 6 overarching content areas: Families of children in care Families of children in care Children and young people in care Children and young people in care Foster families Foster families Kinship care Kinship care Residential care Residential care Quality of care Quality of care

Analysis  Composition of the literature  Gaps and areas for further investigation: Research gaps Research gaps Literature reviews Literature reviews

Example Content area or sub-themes Number of publications reporting Australian research Number of publications reporting international research studies Literature reviews Theory, policy and practice literature Program descriptions Needs of children and young people in care Adoption from care Participation by children and young people in decisions affecting their lives Young people leaving care Mentoring

Methological findings  Volume and complexity in locating relevant references (over 9,500 references= over 4,000 relevant publications and 2,615 research literature  Time consuming task – approx 50 hours to search; screening and coding (approx. 1 hour to screen 20 abstracts) Total approx. 500 hrs  62% of research found in PsychInfo  38% of research across other 4; 5% unique to Medline  Approx double number of Aust. studies identified by audit  Analysis phase 8-12 weeks

Informants views  Knowledge needs: Research Research Theory Theory Practice Practice  Strategies: Active Active Passive Passive Interactive Interactive  Barriers: Time, volume, access, readability, relevance, motivation and org culture A sense of being overwhelmed A sense of being overwhelmed

Utility of the map  Overview of body of literature  Responsive to different knowledge needs  Concise; a starting point  Easy to update and expand  New insights into the nature of the gaps eg policy debates that highlight need for research; emerging practice that requires evaluations

Conclusions  Time intensive processes not feasible  Importance of secondary analysis to support accessibility  Mapping offers potential to make explicit strengths and limitations of the knowledge bases; an accountable way to manage the uncertainty  Potential of maps to highlight gaps in primary research and areas of substantial knowledge for in-depth reviews.

Implications  Policy and practice: Need to go beyond established knowledge; Need to go beyond established knowledge; Basis for reasonable hypotheses Basis for reasonable hypotheses  Further research – primary and secondary analysis  Systematic mapping methodology: Boundaries of maps – content and types of knowledge Boundaries of maps – content and types of knowledge User involvement User involvement

Overall conclusions  To move beyond the rhetoric of being evidence- informed need to address gaps in both primary and secondary research  The need to engage users to identify questions and priorities