National Drugs Campaign Evaluation A Methodology for Telephone Research amongst Indigenous Youth (13-20 year olds) Presentation May 2006

2 NDC Evaluation A Telephone (CATI) Methodology for Research amongst Young Indigenous People

3 NDC Methodology - Background Background  Elements of National Drug Campaign were targetted specifically at Indigenous Youth and evaluation was required …  BUT a “large”, random sample of Indigenous Youth is DIFFICULT and EXPENSIVE to obtain  Need for a CATI-based methodology but not random sampling  The Social Research Centre and CIRCA developed an approach to using CATI amongst young Indigenous people  Centrepiece - a purposive sample frame (740 Indigenous young people aged 13 to 20 years from over 90 postcodes across Australia)  Incentives paid to recruiters and respondents

4 NDC Methodology – Building the Sample Frame Building the Sample Frame – The process List of urban, regional and remote locations developed Locations selected based on CIRCA’s experience Matched to Indigenous representation in each state (ABS Census data) Briefing notes for recruiters Ensures quality control – all recruiters “on the same page” Gives all recruiters assurance that process is legitimate and provides them with information on nature of study, payment of incentives, and how to capture parental consent Recruiting commenced CIRCA’s existing contacts in relevant areas approached New contacts established where necessary (e.g. If CIRCA’s existing contact/s had moved on) Follow-ups; Lists ed / faxed back CIRCA consultants following up with contacts to monitor recruitment progress Contact details ed / faxed back

5 NDC Methodology – Building the Sample Frame Building the Sample Frame - Types of contacts approached by CIRCA  CIRCA’s own contacts Facilitators and recruiters we work with on a fairly regular basis  Agencies that work with Indigenous youth eg: Youth Drop-in Centres, CDEPs  Indigenous-specific high schools or those with a high Indigenous student population via Aboriginal Education Assistants (“AEAs”)  Indigenous media organisations  Key university and youth websites eg: UNSW Careers website, Vibewire.net So large was the scope of recruitment that towards the final weeks of the project, CIRCA was approached by several Indigenous contacts who’d “heard” about the project and were well connected and keen enough to also act as recruiters for us!

6 NDC Methodology – Building the Sample Frame Building the Sample Frame - Much time and effort, but well worth it This approach yielded a sample of 740 potential respondents, indicative of a reasonably high level of co-operation from contacts and agencies approached  Sufficient time allowed for development of ‘database’ Mid August to early October 2005 Important as often much “chasing” involved!  Flexible approach Several deadlines staggered One contact number able to be used (in some cases) for an entire community  Win-win situation Recruiter/agency and young person both receive incentive to participate Making it workDifficulties/Limitations  Significant resources needed to make it happen Four CIRCA consultants utilised  Recruiter’s often under-resourced  Telephone methodology difficult in some communities  More cost-effective than alternatives but still not inexpensive Payment of incentives

7 NDC Methodology – Making Contact The Outcome  The required number of interviews (n=400) was easily achieved from this sample. [1] [1] ‘Population Characteristics, Aboriginal and Torres Strait Islander Australians’ (2001) Remote 5544 Regional 2430 Urban. %Location Sample Distribution Population Distribution [1] [1]  Very good gender distribution achieved (46% male / 54% female)  Age quotas met (75% 13 to 17 year / 25% 18 to 20 years)  Good geographical representation (97 metro respondents, 218 regional respondents and 85 remote) aligned with Indigenous population

8 NDC Methodology - Comparative Profile Match with “General Population Youth” Sample …  Good on sex and age  Slightly fewer in post-secondary education, more unemployed

9 NDC Methodology - Comparative Profile Match with “General Population Youth” Sample …  Geographic differences reflect non-metro targeting of Indigenous Youth sample

10 NDC Comparative Profile Match with “General Population Youth” Sample …  Some differences in education (recruitment) and living arrangements

11 NDC Methodology – Making Contact Methodology reached “hard to contact” respondents  Many respondents would not have been contactable using traditional telephone interviewing sampling methodologies  29% of interviews achieved on non-unique phone numbers (52% in remote areas)  43% of interviews from mobile phone numbers  85 interviews with respondents in remote locations (from Broome to Thursday Island to Port Lincoln)

12 NDC Methodology – Survey Execution Survey Implementation  Some tailoring of the questionnaire  Simplifying the wording  Changes of some response frames to yes/no rather than 5 point Likert scales  Making response frames (eg: living arrangements) more culturally appropriate.  Interviewer Selection and Training  Very small interviewing team (n=3) – one Indigenous interviewer  Cultural sensitivity training from CIRCA

13 NDC Methodology – Survey Execution  Nevertheless, interviewers found little difference between interviewing Indigenous and non-Indigenous Youth  Some feeling that ‘remotes’ differed in the nature of their responses. This is supported by the survey findings suggesting slightly greater reluctance among ‘remote’ respondents to offer an opinion about the impact of the NDC campaign  Possibly attributable to:  Slightly different media exposure (e.g. 9% don’t watch tv), and  Being marginally less comfortable / familiar with the interview experience Survey Implementation

14 NDC Methodology – Conclusion  A very successful methodology, which  Provided a cost effective way of enumerating a very diverse, low incidence, “hard to reach” population group.  However, the purposive nature of the sample means there are limits on the extent to which the survey results can be generalised to the population Overall this approach provided …