2. Monitoring and Evaluating Scotland’s Alcohol Strategy (MESAS)
The short-term impact of the Alcohol Act on alcohol-related health harms in Scotland: an interrupted time series analysis
Please make sure
Mark Robinson1
Janet Bouttell2, Jim Lewsey2, Danny Mackay2, Gerry McCartney1 and Clare Beeston1
1 NHS Health Scotland 2 University of Glasgow

3. of the prices at which each alcoholic
Multi-buy discount ban
‘A package containing two or more alcoholic products … may only be sold on the premises at a price equal to or greater than the sum
of the prices at which each alcoholic
product is for sale.’ x

4. Promotion intervention
Change in consumption
Change in outcomes or
Pricing intervention
Promotion intervention

5. Evaluating the Alcohol Act – phase 1
Quite a clean analysis
Not too many challenges
Biggest difficulty was the fact that the p-value was 0.07, obviously not below the holy grail of 0.05
Another challenge

6. Intervention Change in consumption Change in outcomes
Say a little about the data here – just mention comprability
Hospital admissions
Deaths

7. Analytical approach Alcohol Act introduced
General approach – before and after – interrupted time series analysis
Regression assuming negative binomial distribution
Include a binary variable for before/after
Need to model seasonality – use a ‘month’ variable
Adjust for all the usual suspects (age, sex, Carstairs)
Need to take account of trend
Difference in difference assumes parallel trends
Trends between Scotland and England not parallel
Strengthen findings by including England in Scotland analysis but allowing all the variables to interact (ie England and Scotland could have separate age, sex,seasonality, trend effects).
Results were not stable for regression method adopted when countries were combined
Used a panel data approach – where each subgroup gets its own ID – ie year old, men in Carstairs quintile 1
Single countries only
We wanted to exploit the individual level data
The same analytical framework was used for both deaths and hospital admissions

8. Analytical approach Control group Comparable data Long pre- trend
Alcohol Act
introduced
Analytical approach
General approach – before and after – interrupted time series analysis
Regression assuming negative binomial distribution
Include a binary variable for before/after
Need to model seasonality – use a ‘month’ variable
Adjust for all the usual suspects (age, sex, Carstairs)
Need to take account of trend
Difference in difference assumes parallel trends
Trends between Scotland and England not parallel
Strengthen findings by including England in Scotland analysis but allowing all the variables to interact (ie England and Scotland could have separate age, sex,seasonality, trend effects).
Results were not stable for regression method adopted when countries were combined
Used a panel data approach – where each subgroup gets its own ID – ie year old, men in Carstairs quintile 1
Single countries only
We wanted to exploit the individual level data
The same analytical framework was used for both deaths and hospital admissions
Control
group
Comparable
data
Long pre-
trend
Underlying
trends
Sensitivity analysis
Adjustment

9. Analytical approach Control group Comparable data Long pre- trend
Alcohol Act
introduced
Analytical approach
General approach – before and after – interrupted time series analysis
Regression assuming negative binomial distribution
Include a binary variable for before/after
Need to model seasonality – use a ‘month’ variable
Adjust for all the usual suspects (age, sex, Carstairs)
Need to take account of trend
Difference in difference assumes parallel trends
Trends between Scotland and England not parallel
Strengthen findings by including England in Scotland analysis but allowing all the variables to interact (ie England and Scotland could have separate age, sex,seasonality, trend effects).
Results were not stable for regression method adopted when countries were combined
Used a panel data approach – where each subgroup gets its own ID – ie year old, men in Carstairs quintile 1
Single countries only
We wanted to exploit the individual level data
The same analytical framework was used for both deaths and hospital admissions
Control
group
Comparable
data
Long pre-
trend
Underlying
trends
Sensitivity analysis
Adjustment

10. Analytical approach Control group Comparable data Long pre- trend
Alcohol Act
introduced
Analytical approach
General approach – before and after – interrupted time series analysis
Regression assuming negative binomial distribution
Include a binary variable for before/after
Need to model seasonality – use a ‘month’ variable
Adjust for all the usual suspects (age, sex, Carstairs)
Need to take account of trend
Difference in difference assumes parallel trends
Trends between Scotland and England not parallel
Strengthen findings by including England in Scotland analysis but allowing all the variables to interact (ie England and Scotland could have separate age, sex,seasonality, trend effects).
Results were not stable for regression method adopted when countries were combined
Used a panel data approach – where each subgroup gets its own ID – ie year old, men in Carstairs quintile 1
Single countries only
We wanted to exploit the individual level data
The same analytical framework was used for both deaths and hospital admissions
Control
group
Comparable
data
Long pre-
trend
Underlying
trends
Sensitivity analysis
Adjustment

11. Analytical approach Control group Comparable data Long pre- trend
Alcohol Act
introduced
Analytical approach
General approach – before and after – interrupted time series analysis
Regression assuming negative binomial distribution
Include a binary variable for before/after
Need to model seasonality – use a ‘month’ variable
Adjust for all the usual suspects (age, sex, Carstairs)
Need to take account of trend
Difference in difference assumes parallel trends
Trends between Scotland and England not parallel
Strengthen findings by including England in Scotland analysis but allowing all the variables to interact (ie England and Scotland could have separate age, sex,seasonality, trend effects).
Results were not stable for regression method adopted when countries were combined
Used a panel data approach – where each subgroup gets its own ID – ie year old, men in Carstairs quintile 1
Single countries only
We wanted to exploit the individual level data
The same analytical framework was used for both deaths and hospital admissions
Control
group
Comparable
data
Long pre-
trend
Underlying
trends
Sensitivity analysis
Adjustment

12. Analytical approach Control group Comparable data Long pre- trend
Alcohol Act
introduced
Analytical approach
General approach – before and after – interrupted time series analysis
Regression assuming negative binomial distribution
Include a binary variable for before/after
Need to model seasonality – use a ‘month’ variable
Adjust for all the usual suspects (age, sex, Carstairs)
Need to take account of trend
Difference in difference assumes parallel trends
Trends between Scotland and England not parallel
Strengthen findings by including England in Scotland analysis but allowing all the variables to interact (ie England and Scotland could have separate age, sex,seasonality, trend effects).
Results were not stable for regression method adopted when countries were combined
Used a panel data approach – where each subgroup gets its own ID – ie year old, men in Carstairs quintile 1
Single countries only
We wanted to exploit the individual level data
The same analytical framework was used for both deaths and hospital admissions
Control
group
Comparable
data
Long pre-
trend
Underlying
trends
Sensitivity analysis
Adjustment

13. Analytical approach Control group Comparable data Long pre- trend
Alcohol Act
introduced
Analytical approach
General approach – before and after – interrupted time series analysis
Regression assuming negative binomial distribution
Include a binary variable for before/after
Need to model seasonality – use a ‘month’ variable
Adjust for all the usual suspects (age, sex, Carstairs)
Need to take account of trend
Difference in difference assumes parallel trends
Trends between Scotland and England not parallel
Strengthen findings by including England in Scotland analysis but allowing all the variables to interact (ie England and Scotland could have separate age, sex,seasonality, trend effects).
Results were not stable for regression method adopted when countries were combined
Used a panel data approach – where each subgroup gets its own ID – ie year old, men in Carstairs quintile 1
Single countries only
We wanted to exploit the individual level data
The same analytical framework was used for both deaths and hospital admissions
Control
group
Comparable
data
Long pre-
trend
Underlying
trends
Sensitivity analysis
Adjustment

14. Analytical approach Control group Comparable data Long pre- trend
Alcohol Act
introduced
Analytical approach
General approach – before and after – interrupted time series analysis
Regression assuming negative binomial distribution
Include a binary variable for before/after
Need to model seasonality – use a ‘month’ variable
Adjust for all the usual suspects (age, sex, Carstairs)
Need to take account of trend
Difference in difference assumes parallel trends
Trends between Scotland and England not parallel
Strengthen findings by including England in Scotland analysis but allowing all the variables to interact (ie England and Scotland could have separate age, sex,seasonality, trend effects).
Results were not stable for regression method adopted when countries were combined
Used a panel data approach – where each subgroup gets its own ID – ie year old, men in Carstairs quintile 1
Single countries only
We wanted to exploit the individual level data
The same analytical framework was used for both deaths and hospital admissions
Control
group
Comparable
data
Long pre-
trend
Underlying
trends
Sensitivity analysis
Adjustment

15. Results – Deaths (seasonal component)
The overall crude rate in both of these outcomes over the study time period was plotted in charts. In addition, to ease visual interpretation of trends, the time series for each outcome was decomposed into trend and seasonal components using the stl command (seasonal decomposition by loess) in R Studio software (R Studio, Boston, USA).14 This decomposition is not adjusted for sex, age or socio-demographic factors and uses a different approach to trends and seasonality than the regression approach described below and so is included for descriptive purposes only.

16. Results – Deaths (trend component)
Alcohol Act
introduced
The overall crude rate in both of these outcomes over the study time period was plotted in charts. In addition, to ease visual interpretation of trends, the time series for each outcome was decomposed into trend and seasonal components using the stl command (seasonal decomposition by loess) in R Studio software (R Studio, Boston, USA).14 This decomposition is not adjusted for sex, age or socio-demographic factors and uses a different approach to trends and seasonality than the regression approach described below and so is included for descriptive purposes only.

17. Results – Deaths (regression)
Alcohol Act
introduced
IRR = 0.99 (0.91 to 1.07)
The overall crude rate in both of these outcomes over the study time period was plotted in charts. In addition, to ease visual interpretation of trends, the time series for each outcome was decomposed into trend and seasonal components using the stl command (seasonal decomposition by loess) in R Studio software (R Studio, Boston, USA).14 This decomposition is not adjusted for sex, age or socio-demographic factors and uses a different approach to trends and seasonality than the regression approach described below and so is included for descriptive purposes only.
IRR = 0.99 (0.95 to 1.02)

18. Results – Admissions (seasonal component)
The overall crude rate in both of these outcomes over the study time period was plotted in charts. In addition, to ease visual interpretation of trends, the time series for each outcome was decomposed into trend and seasonal components using the stl command (seasonal decomposition by loess) in R Studio software (R Studio, Boston, USA).14 This decomposition is not adjusted for sex, age or socio-demographic factors and uses a different approach to trends and seasonality than the regression approach described below and so is included for descriptive purposes only.

19. Results – Admissions (trend component)
Alcohol Act
introduced
The overall crude rate in both of these outcomes over the study time period was plotted in charts. In addition, to ease visual interpretation of trends, the time series for each outcome was decomposed into trend and seasonal components using the stl command (seasonal decomposition by loess) in R Studio software (R Studio, Boston, USA).14 This decomposition is not adjusted for sex, age or socio-demographic factors and uses a different approach to trends and seasonality than the regression approach described below and so is included for descriptive purposes only.

20. Results – Admissions (regression)
Alcohol Act
introduced
IRR = 0.98 (0.95 to 1.02)
IRR = 1.05 (1.03 to 1.07)
So, in other words

21. Main finding and interpretation
“The implementation of the Alcohol Act was not associated with any measurable, short-term changes in overall rates of alcohol-related deaths or hospital admissions in Scotland.”
So how do we interpret these results. Well…

22. Main finding and interpretation
“The implementation of the Alcohol Act was not associated with any measurable, short-term changes in overall rates of alcohol-related deaths or hospital admissions in Scotland.”
Intervention effect too small to be detected.
So how do we interpret these results. Well…

23. Main finding and interpretation
“The implementation of the Alcohol Act was not associated with any measurable, short-term changes in overall rates of alcohol-related deaths or hospital admissions in Scotland.”
Intervention effect too small to be detected.
Full effect of changes in consumption takes time.
So how do we interpret these results. Well…

24. Main finding and interpretation
“The implementation of the Alcohol Act was not associated with any measurable, short-term changes in overall rates of alcohol-related deaths or hospital admissions in Scotland.”
Intervention effect too small to be detected.
Full effect of changes in consumption takes time.
Narrow focus on 100% attributable conditions.
So how do we interpret these results. Well…

25. Main finding and interpretation
“The implementation of the Alcohol Act was not associated with any measurable, short-term changes in overall rates of alcohol-related deaths or hospital admissions in Scotland.”
Intervention effect too small to be detected.
Full effect of changes in consumption takes time.
Narrow focus on 100% attributable conditions.
Wine the drink type most affected.
So how do we interpret these results. Well…

26. Main finding and interpretation
“The implementation of the Alcohol Act was not associated with any measurable, short-term changes in overall rates of alcohol-related deaths or hospital admissions in Scotland.”
Intervention effect too small to be detected.
Full effect of changes in consumption takes time.
Narrow focus on 100% attributable conditions.
Wine the drink type most affected.
Rates of alcohol-related harm depend not only on average consumption levels.
So how do we interpret these results. Well…

27. Implications Policy evaluation Policy
Policy implications, but perhaps more important is evaluating policy that the analytical challenges presented.
Policy

28. Thanks to the study project team and to you for listening.
Report available at
Telephone

29. Strengths and limitations
Comparability
Coverage
Data
Patient level
Long pre-intervention trend
Study design
Analytical approach
Sensitivity analysis