Reitox Academy, June 2010 Evaluation of drug policy Activity 26.3 of Drug Action Plan

2 Object of evaluation Drug policy formulated in strategic documents: –National Drug Strategy –(second) Action plan

3 National Drug Strategy Specific goals: I. Stabilize / decrease the no. of problem drug users II. Stop the increase of experimental drug use III. Stabilize / decrease drug consumption IV. Decrease the risks of drug use V.Guarantee availability of drug services VI. Decrease the accessibility of drugs Technical-organizational goals: VII. Effective financing VIII. Coordination IX. Information to general public X. International cooperation XI. Evaluation of activities Decrease the level of any drug use prevalence and the potential risks To combat organized crime Main goals

4 Action plan 7 drug areas: Prevention Treatment Harm reduction Law enforcement Coordination and financing Information, research, evaluation International cooperation Action plan : 43 objectives 170 activities

5 Action plan

6 Aim of evaluation To learn : The extend to which strategic goals where achieved (How has the drug situation changed) What were the main achievements and failures of drug policy in 5 years

7 Organization 05/2009 – 03/2010 internal evaluation – NDC secretariat 1,5 employee no budget Established the Board of Drug Strategy Evaluation (5 experts): –methodological guidance of the evaluation process and evaluation tools –Supervision of final outcome/findings and final version of evaluation report

8 Concept Based on the (simplified) policy cycle model

9 Q1: Have changes occurred within the formulation process? Have strategic documents changed compared to their planed design? Q2: To what extend is the Action plan and Strategy cohesive/linkedup? How many activities really lead to strategic goals? Q3: What is the character/type of activities planed in Action plan? from the point of their formulation, potential to solve a problem, significance for drug situation Formulation

10 Q1: Have changes occurred within the formulation process? Comparison of official (and internal) documents (what was planed in 2004 what was approved in 2005) Semi-structure interview with the former head of secretariat NDC Formulation

11 Original purpose / structure and content of action plan has been affected/changed Formulation Abandoned area Use of legal drugs Changed area Public relation for Information, research, evaluation Alcohol and tobacco is not integrated part of drug policy – failure of drug policy no link exists between activities and one specific strategic goal ? ?

12 Q2: To what extend is the Action plan and Strategy cohesive/linkedup? Q3: What is the character/type of activities planed in Action plan? Analysis of documents based on the idea of logframe analysis helps to link and clarify mutual relationship among goals, objectives, outputs and activities Formulation

13 internal working group (5 people) assessed each activity of AP in 5 different criteria - clarity in formulation - factuality/concreteness in formulation - context of problem solving - significance for drug situation - relevance to strategic goals Formulation

14 Incidention matrix – all activities

15 Incidention matrix – key activities

16 Q1: How successful was the implementation of Action Plan ? Q2: What factors influenced the process of implementation (positively and negatively)? Q3: What are the conditions for implementation of drug policy on the regional level? Development in the area of drug policy in the last 5 years. Implementation

17 Q1: How successful was the implementation of Action Plan? Top-down approach Objectives set on the central level are assessed from the point of – how these have been achieved, what hindered their achievement, and what is the consistency of outputs and outcomes with set objectives. Implementation

18 Questionnaire survey –36 subject involved in drug policy implementation 16 directly responsible for activity implementation 6 collaborating subjects – professional organizations (new) 14 regions (new) 14 question for each activity, eg.: –Process of activity implementation (how) –Problems in implementation/reasons for not implementing –Outcome of activity implemented –Submission of outputs to secretariat NDC Implementation

19 Working group of the NDC secretariat (5 people) –Assessed the information submitted –Compared submitted outputs with objective/activities planed in Action plan This way we decreased the formality in fulfillment of activities (eg. in prevention declared implemented/fulfilled activities decreased from cca 45 % down to 35 %) Implementation

20 Q2: What factors influenced the process of implementation (positively and negatively)? on-line anonymous questionnaire (35 respondents) – state organizations, local government, professional organizations –Marked the level of influence of listed factors –Open questions Comparison of SWOT analysis in 2004 and 2009 (opportunities and threads) –expert groups for 7 drug areas (cca. 60 people) Implementation

21 Q3: What are the conditions for implementation of drug policy on the regional level? Development in the last 5 years? On-line questionnaire among regional drug coordinators (14 regions) –Extend to which goals of national drug strategy has been introduced into regional strategic documents –Cooperation with state organizations –Position of regional drug coordinator/possibilities to influence drug policy implementation –Development of conditions for implementation of regional drug policy (personal, political, financial, institutional) Implementation

22 Q1: What is the level of achievement of specific strategic goals? (focused on the change of drug situation) Analysis of quantitative data/indicators of drug situation (NFP) Outcome evaluation

23 Q2: Development in each drug area in the last 5 years? Expert group for each drug area (7 groups), 60 experts SWOT analysis Assessing each element of SWOT 2004 –Is the strength still strong point or has it become week point, have the week points been eliminated, have we used identified opportunities or have the threads occurred or not… –Identify new elements of SWOT for 2009 Outcome evaluation

24 Q3: What is the level of achievement of technical- organizational strategic goals? Combination of results and findings from expert working groups and results from evaluation of implementation Outcome evaluation

25 Findings

26 Findings The character and potential of planned activities (defined in Action plan ) and also the level of successful implementation corresponds with the success in achieving strategic goals and areas

27 eg. - prevention High level of experimental use – cannabis, alcohol … The consumption of illegal drugs increased, of legal drug stabilized on a high level Prevention in Action plan No activity leaded directly to specific strategic goals High % (70) of activities focused on organizational-coordination frame – only 30 % of activities were focused on solving a problem Fulfilled/implemented 58 % of activities in , and 35 % in Implemented 1 activity out of 6 characterized as those with higher potential of changing a drug situation No improvement was noticed/declared in this area; what more number of strong points from 2004 were weakened (SWOT)

28 eg. – harm reduction Infection diseases and other health consequences on a low level Stabilization of problem drug users Relatively stable network of drug services Harm-reduction in Action plan Had the highest no. of activities leading directly to achieving most of strategic goals Highest % (65) activities, which were specifically focused on problem solving, Fulfilled/implemented 78 % of activities (most from all 4 pillars), 41 % in Implemented 8 key activities (from 16) important for strategic goals achievement

Thank you

30 Findings According to how activities were formulated in action plan we can say that: Drug policy was mainly focused on the coordination, legislative, organizational frame and on the evaluation of interventions Area of information, research, evaluation – has most activities of strategic implication – important for key decisions Area of harm-reduction – had the greatest potential of changing drug situation Area of prevention – had the lowest potential of changing drug situation

31 Recommendations- Challenges Better linking up of activities and strategic goals Limit the no. of objectives, clearly state the priorities Prevention – its scope/intention, the way of defining activities and support of their implementation Dealing with alcohol and tobacco use – coordination, availability of information, services Law enforcement especially concerning legal drugs Maintain the network of drug services and necessary interventions

32 Findings - formulation Incidention matrix. Typical problem of vague and unclear definition of activities or goals Activity unclearly formulated – variety of interpretation, it complicated implementation and evaluation Positive correlation was find between clear formulation of activity and the level of achievement (this does not apply for concreteness of activity) The use (the role) of indicators for activities was not clear Overlapping of strategic goals - goal III. Stabilize / decrease drug consumption … was seemed to be redundant

33 I. Stabilizovat / snížit počet problémových uživatelů stabilizace, O.K. II. Zastavit nárůst experimentálního užívání mírný nárůst. stabilizace na vysoké úrovni III. Stabilizovat / snížit spotřebu drog stabilizace na vysoké úrovni (legální drogy), nárůst (nelegální drogy) IV. Snížit rizika užívání drogstabilizace, O.K. V. Dostupnosti služeb léčby a resocializace Stabilizace (kromě legálních), ale v ohrožení VI. Snížit dostupnost drogvysoká dostupnost Přehled – specifické cíle

34 V II. Efektivní financováníčástečně V III. KoordinaceČástečně (vedlo k němu nejvíc aktivit) IX. Informování veřejnostinehodnotitelné, oblast - X. Mezinárodní spolupráceO.K. XI. Evaluace aktivitnehodnocené Přehled – technicko-org. cíle

35 Incidention matrix – all activities

36 Výsledky - implementace Akční plán – splněno 73 % aktivit Akční plán – splněno 52 % aktivit

37 Výsledky - implementace Pozitivní faktory: Odborní potenciál expertů Dostupnost relevantních informací Institucionální zabezpečení protidrogové politiky Úroveň koordinace a vzájemné spolupráce – pozitivní i negativní faktor implementace Negativní faktory: Rozdílné výchozí přístupy resortů k protidrogové politice / prohlubování resortizmu Formálnost ve stanovení i plnění aktivit Nedostatek finančních zdrojů Snižující se zájmem o problematiku ze strany politiků Vysoká společenská tolerance legálních drog Podceňování rizik návykových látek