Using Behaviour Change Technique (BCT) analysis to improve fidelity to treatment manuals in smoking cessation: A case study Billie Bonevski, Laura Twyman, Fabiana Lorencatto, Susan Michie & Robert West

Newcastle, Australia 2

The Trial The primary aim is to evaluate the efficacy of a community social service case-worker delivered smoking cessation intervention for socially disadvantaged smokers The study design is a randomised controlled trial The primary outcome measure is CO verified continuous abstinence at 6 months follow-up 3

The Call it Quits smoker (n = 309) 4

The case-worker delivered intervention: Key components Delivered by trained volunteer case-workers Systematically (electronically) assessing and recording smoking status in client notes Tailored (based on print-out) Provision of advice to quit and behavioural counselling, motivational interviewing if required over two face-to-face sessions Free nicotine replacement therapy Buddy system Flexible follow-up through telephone calls, or drop-ins 5 But what are our intervention participants really receiving?

The aims of the current study 1.Assess volunteer case-worker fidelity to the Call it Quits intervention manual using BCT analysis 2.Assess the effectiveness of BCT performance feedback in improving fidelity to intervention manual 6

Methods Pre and post test study embedded in one intervention trial (Call it Quits) 6-week pre-test period of recording counselling sessions 6-week period post-intervention of recording counselling sessions Main outcome: –Proportion of each manual specified BCT delivered pre and post 7

Methods Intervention manual coded using BCT smoking cessation framework Pre-test: Counselling sessions audio recorded and transcribed Code transcriptions - BCTs Intervention: Volunteer case-workers provided with BCT performance feedback Post-test: Counselling sessions audio recorded and transcribed Code transcriptions - BCTs 8

Results: BCTs coded in the manual 9 Tailor interaction appropriately Facilitate goal setting Boost motivation and self efficacy Prompt review of set goals Provide feedback on current behaviour and progress Advise on stop smoking medication Provide rewards contingent on not smokingAdvise on/facilitate the use of social support Prompt commitment from the client there and thenAsk about experience of stop smoking medication that the smoker is currently using Provide rewards contingent on effort or progress Give options for additional and later support Facilitate identification of reasons for wanting and not wanting to stop smoking General communication approach Explain the importance of abrupt cessation Facilitate relapse prevention and coping Conduct motivational interviewing Promote cut-down-quit Facilitate barrier identification and problem solvingAdvise on management of mental health problems Information gathering and assessmentAssess alcohol or drug use disorder Facilitate action planning/develop a treatment planArrange follow-up session

Pre-test Results - Volunteer case-worker session recordings (n=23) Individual Volunteer BAPIGSKTotal 1 st F2F nd F2F235 1 st t/c112 2 nd t/c213 Final t/c11114 Total Duration (mins)

Inter-rater reliability 50% of pre-test coding Between two coders (BB & LT) Assessed using percentage agreement 79% (High - > 75%) 11

Pre-test Results - Number of face to face sessions each BCT delivered in (out of possible 14) 12 BCT labelN (%) Tailor interaction appropriately8 (57) Facilitate goal setting9 (64) Boost motivation and self efficacy3 (21) Prompt review of set goals0 Provide feedback on current behaviour and progress 10 (71) Advise on stop smoking medication14 (100) Provide rewards contingent on not smoking0Advise on/facilitate the use of social support13 (93) Prompt commitment from the client there and then 9 (64)Ask about experience of stop smoking medication that the smoker is currently using 12 (86) Provide rewards contingent on effort or progress6 (43) Give options for additional and later support3 (21) Facilitate identification of reasons for wanting and not wanting to stop smoking 5 (36) General communication approach14 (100) Explain the importance of abrupt cessation6 (43) Facilitate relapse prevention and coping5 (36) Conduct motivational interviewing3 (21) Promote cut-down-quit12 (86) Facilitate barrier identification and problem solving 6 (43) Advise on management of mental health problems1 (7) Information gathering and assessment13 (93)Assess alcohol or drug use disorder10 (71) Facilitate action planning/develop a treatment plan 14 (100) Arrange follow-up session14 (100)

Pre-test Results - Number of telephone sessions each BCT delivered in (out of possible 9) 13 BCT labelN (%) Tailor interaction appropriately3 (33) Facilitate goal setting0 Boost motivation and self efficacy8 (89) Prompt review of set goals2 (22) Provide feedback on current behaviour and progress 8 (89) Advise on stop smoking medication2 (22) Provide rewards contingent on not smoking1 (11)Advise on/facilitate the use of social support4 (44) Prompt commitment from the client there and then0Ask about experience of stop smoking medication that the smoker is currently using 5 (56) Provide rewards contingent on effort or progress4 (44) Give options for additional and later support8 (89) Facilitate identification of reasons for wanting and not wanting to stop smoking 2 (22) General communication approach7 (78) Explain the importance of abrupt cessation0 Facilitate relapse prevention and coping2 (22) Conduct motivational interviewing0 Promote cut-down-quit3 (33) Facilitate barrier identification and problem solving2 (22) Advise on management of mental health problems3 (33) Information gathering and assessment2 (22)Assess alcohol or drug use disorder3 (33) Facilitate action planning/develop a treatment plan0Arrange follow-up session5 (56)

BCT Performance feedback intervention 14 Group feedback session outlining: –Number of recordings per volunteer –BCTs specified in manual –BCTs delivered by each volunteer (anonymous) –Positive reinforcement –Discussion of BCTs, identification of aids to help boost BCT use Individual performance feedback letters including: –Length of sessions (in minutes) –List of BCTs identified and positive reinforcement –List of BCTs not coded –Goal setting – increase unused BCTs in 6 months

BCT Performance feedback intervention Desktop checklist with ‘top 5’ BCTs – goal for improvement 15 In face-to-face sessions 1.Explain the importance of abrupt cessation 2.Prompt commitment from the client there and then 3.Facilitate action planning/develop a treatment plan 4.Facilitate goal setting 5.Advise on stop smoking medication In telephone follow-up calls 1.Prompt review of set goals 2.Provide feedback on current behaviour and progress 3.Boost motivation and self-efficacy 4.Facilitate relapse prevention and coping 5.Provide rewards contingent on effort or progress

Next steps Continue coding post-test sessions Compare pre and post test use of manual specified BCTs Assess by –session type (face to face, telephone follow-up) –duration, –volunteer –BCT Assess inter-rater reliability for new BCTs 16

CRICOS Provider 00109J | THANK YOU Funding: National Health & Medical Research Council Cancer Institute NSW Hunter Medical Research Institute Anglicare Community Care Centre Wollongong clients, volunteers and staff Contact me on: or ph: