1. Defining and measuring results
EHRN Interim and final evaluation approach

2. The Regional Program evaluation
Baseline assessment (conducted in 2014 in 6 countries by external agency);
Program implementation monitoring (program and financial reporting from EHRN staff, SSR and SRs conducted mostly quarterly throughout implementation of the Regional Program);
Interim evaluation (beginning of 2016, conducted by EHRN and external interviewer)
Final evaluation (beginning of 2017, will be conducted by external agency)

3. Measuring impact by objectives
Changes in HR funding
OB 1: Sustainable HR funding
OB2: community capacity
policy
Baseline assessment
HR funding and procurements
Financing through NGO
Program sustainability
Community involvement
Regional influence
4 elements of influence:
Evidence base on HR
Collaborative environment
Media environment
Advocacy capacity building
Milestones
Processes indicators
Baseline assessment

4. Measuring impact by objectives
Results – changes in HR funding stimulated by activities under the Regional Program
Objectives - progress in developing sustainable HR policy environment for HR funding (objective 1) and progress in increasing community capacity for involvement in HR advocacy (objective 2).
The Interim evaluation M&E framework:
Scales for impact evaluation on national level
Summary of regional influence for impact evaluation on regional level
Four elements of advocacy efforts/activities influence for outcome evaluation on national level
Milestones for outcome evaluation on regional level
Aggregation of data from program reporting from EHRN and staff

5. Evaluation approach for HR funding
4 points of Data collection: 1. Impact evaluation of policies: HR funding situation in a country through measuring progress by 3 scales (by stakeholders and partners). Comparison of progress with baseline 2. Outcome evaluation: Milestones and interview with stakeholders and partners to evaluate changes achieved by the project by 4 elements. Comparison of progress with baseline 3. Output monitoring: Annually aggregated process indicators 4. Process monitoring: Detailed quarterly reports with process indicators provided by partners

6. Measuring impact: Advocacy for funding harm reduction and community capacity. National level
To insure sustainable funding of harm reduction services, 4 basic directions of institutionalization of the services are:
Budgeting for harm reduction
Mechanisms of contracting civil society and community based organizations to provide HR services
HR program sustainability
Civil society and PWUD community involvement in decisions on HR funding

7. Progress scales for country evaluation
Baseline assessment:
Setting starting stages at the beginning of the Regional Program
The progress of budget advocacy is measured by:
Moving from stage to stage on each direction
Stage completion is defined by development and establishment of a normative document that institutionalizes changes
A stage can be completed partly implying that a country will get back in future to adjust and implement changes
Impact of the Regional Program is changes on one or more stages of any direction
Verification of the stage completion:
Qualitative evaluation by different national stakeholders
Comparison of progress made with the baseline

8. Sustainable HR funding based on community needs
Final
Stage 4
Stage 3
Stage 2
Stage 1
Final
Stage 6
Stage 5
Stage 4
Stage 3
Stage 2
Stage 1
Final
Stage 6
Stage 5
Stage 4
Stage 3
Stage 2
Stage 1
Final
Stage 5
Stage 4
Stage 3
Stage 2
Stage 1
HR funding and procurements
Financing through NGO
Program sustainability
Community involvement
Baseline for the Regional Program
Country’s progress on the direction

9. Questions for interview with stakeholders by four directions
S.1. Stages: Indicate 3 points on each of the scales – where country was at the beginning of the Regional Program; where it is at the moment; and where it is planned to be at the end of the Regional program
S.2. Factors of changes: what factors did influence the progress? What events/activities of the Regional Program did make it possible to achieve the progress?
S.3. Regional influence: How achievement of results was supported at the regional level?

10. Scale 1. Funding harm reduction from state budget
Services are provided (coverage in comparison with GF level)
Expenses for HR are monitored. Price reduction for procurement is done
Final
Mechanisms of procurement of supply medical and non-medical goods is developed (budget code and classificatory of medical goods)
Stage 6
Separate budget line with planned budget for HR (NSP, OST) services is defined in state/municipal budgets
Stage 5
Harm reduction cost to be covered from domestic budget is included into the National HIV Program or other relevant national health program
Stage 4
The state agency or ministry on national and municipal level responsible for HR services coordination/provision is defined
Stage 3
Model/scenario of HR services provision and implementation is defined
Stage 2
Availability of evidence-base on HR funding including determination of all legal and policy barriers
Stage 1

11. Scale 2. Mechanisms of HR services funding through NGO
Coverage of services provided by NGO using NGO contracting mechanism is sufficient to respond on epidemic
Final
PUD community involvement in services provision and/or quality assuring or watch-dogging is defined and budgeted
Stage 6
Mechanisms of NGO contracting for HR are piloted and adjusted to meet community needs and quality standards
Stage 5
NGO can legally be contracted for provision HR services (implacable and transparent mechanisms of registering, licenses, using money, no pre-payment)
Stage 4
NGO contracting/supporting mechanisms could be used for harm reduction services
Stage 3
Selection procedures and requirements to NGO and CSO services providers are defined and transparent
Stage 2
NGO contracting/supporting mechanisms exist in the country (for provision of social and medical services, youth and patriotism)
Stage 1

12. Scale 3. Program sustainability
System of education and capacity building for HR services providers is developed, institutionalized and budgeted
Final
Monitoring of services quality with community participation is included in the national monitoring and evaluation strategy and budgeted
Stage 4
The system of HR quality monitoring and evaluation with community participation developed, institutionalized and budgeted
Stage 3
List and quality of NSP and OST service is regulated by state normative document (instruction, method guidance etc) based on needs of community
Stage 2
Outreach, NSP and OST services are legal in the country (there no legal barriers for its provision)
Stage 1

13. Scale 4. Civil society and PWUD community involvement in decisions on HR funding
CSO and PWUD make impact in state/local decisions on HR sustainability
Final
CSO and PWUD demonstrate:
Capacity in HR budget advocacy
Capacity in budget monitoring and watchdog functions
Stage 5
CSO and/or PWUD community is represented in state/sub-national decision-making bodies
Representation supported by open feedback and regular communication with PWUD community
Stage 4
Joint vision and targets on advocacy is defined and agreed by CS and PWUD community
Stage 3
Visibility of CS and PWUD community for decision makers and general public
Stage 2
Stable existence of community and civil society systems (readiness to expand, structured leadership, regularity and openness of activities)
Capacity to mobilize resources for sustainable work of CSO and PWUD community groups
Stage 1

14. HR funding and procurements
Final
Stage 6
Stage 5
Stage 4
Stage 3
Stage 2
Stage 1
Goal:
To define separate line for PWUD in the national AIDS program and the state budget
Progress:
1. HR budgeted in the draft of the national AIDS program based on IMG and SMG data
2. Centre for Hygiene and Epidemiology became in charge for HR
Baseline for the Regional Program
Country’s progress on the direction

15. HR funding and procurements
Прогресс:
1. Национальная программа на 2016 – 2020 гг. включает расходы на ПОШ, рассчитанные на основании стоимости оптимального пакета услуг на одного клиента программы
2. Центр гигиены и эпидемиологии отвечает за СВ
Факторы успеха:
1. Переговоры и многосекторное участие (ВИЧ, полиция, наркополиция, сообщество, гос. СМИ) в Тбилисском Региональном Диалоге
2. Доказательная база на основе исследований IMG/SMG и с привлечением экспертов из Минфина
Цель:
1. Определить отдельную линию для ЛУН в рамках Национальной программы по СПИДу и в государственном бюджете
2. Мониторинг предполагаемых и фактических расходов из средств ГФ и Государственной программы, анализ пробелов
Stage Stage Stage Stage Stage Stage Final
Baseline for the Regional Program
Country’s progress on the direction

16. Measuring impact: Regional level
Focus on strategic response on HR funding:
Key overview points:
Systematic advocacy
Resourced advocacy
Evidenced-based advocacy
Targeted advocacy
Broadened partnerships
Insights of regional influence from national outcome findings
Interviews with EHRN and regional stakeholders

17. Measuring outcome: 4 Evaluation elements
Element A: Evidence base
Use of evidence base for national strategic and policy documents
Use of evidence to support HR advocacy
Element B: Collaborative environment for funding HR
Influence of stakeholders’ collaboration on sufficient, strategic and sustainable investments in HR
Development of collaboration between international, regional and national stakeholders (PWUD community, civil society, health sector, governmental bodies)
Element C: Media environment for HR
Improvement of image of harm reduction
Visibility of the HR topic in national/local environment
Element D: Advocacy capacity building
Visibility and involvement of PWUD community in national/local policy processes on HR
Development of HR advocacy capacity of community leaders
Development of budget advocacy capacity of civil society

18. Measuring outcome: 4 Evaluation elements
Comparison of baseline and achieved progress:
Summary of baseline evaluation by 4 elements and their sub-elements (if applicable)
Interviews with SRs and national stakeholders
National level of influence
Milestones:
Regional level of influence
Set of the principal outcomes
Prepared by EHRN

19. Questions for interview with stakeholders by elements
Element A: Evidence base A1. How has evidence been used for national strategic and policy documents? A2. Has the collected evidence supported advocacy for sufficient, strategic and sustainable investments in harm reduction? Element B: Collaborative environment for funding HR B1. How did stakeholders’ collaboration influence sufficient, strategic and sustainable investments in harm reduction? B2. Has collaboration between international, regional and national stakeholders (PWUD community, civil society, health sector, governmental bodies) become broader and stronger? Element C: Media environment for HR C1. Has the image of harm reduction improved in national and regional (Russian-language) mass media and social media? C2. Has the visibility of HR improved in mass media? Element D: Advocacy capacity building D1. Has community become more visible and involved in national/local policy processes on HR? D2. Has HR advocacy capacity of community leaders grown? D3. Has budget advocacy capacity of civil society grown

20. Milestones by elements (as foreseen in the grant application)
Evidence base
Collaborative environment for funding HR
Media environment for HR
Advocacy capacity building
Develop regional methodology for harm reduction investment assessment
Publish regional report "Harm reduction: the case for strategic investment“
Produce Evaluation Report
Publish national reports "Harm reduction: the case for strategic investment“
Develop national advocacy strategies and plans
Establish High-level Regional Harm Reduction Council
Conduct First Drug Users' Forum
Conduct High-level Policy Meeting
Conduct Donors' Coordination Meeting (Year 3)
Conduct EECA Regional Harm Reduction Conference (Year 3)
Prepare detailed arguments sets and video materials & simple infographics
Develop training module for community driven assessment: whether harm reduction services meet drug users' needs
Needs assessment implemented by PWUD groups, initiatives and/or CBOs and needs assessment data included in national reports
Develop training module on community-led advocacy and representation in decision-making bodies engagement
Develop training module on documentation of good practices and peer to peer education Meeting (Year 3)

21. Monitoring output: 12 Aggregated output indicators
Element A: Evidence base
Number and name of statements/strategic acts/ policy documents where HR evidences were used
Milestones on evidence base development (number of developed documents on evidence base and their names)
Number of policy briefs based on evidence base documents
Element B: Collaborative environment for funding HR
Numbers of national and regional advocacy meetings
Number of unique participants (decisions makers, CCM, community and other partners) in advocacy meetings
New partners (organizations) and events that brought the partnership (through joint-statement/joint-action)
Partners with increased level of collaboration (to the high level) and events that brought the partnership (through joint-statement/joint-action)
Element C: Media environment for HR
The maximum audience reached (number of audience and percentage)
Element D: Advocacy capacity building
Number of national advocacy trainings of PWUD and civil society
Number of unique PWUD participants in advocacy activities
Number of unique trained PWUD
Number of unique trained civil society participants

22. Monitoring process: detailed process indicators
Process indicators are organized by 4 elements
Two levels of process are distinguished: regional and national
The status of activity completion is included

23. Regional and national level by processes
Regional component: Strategic response on HR funding
National component: Actions on HR funding
Development of methodologies and tools
Baseline assessment of current situation
Development of training modules
Development of regional advocacy strategy, plan and campaign
Conduct of high-level events with broad participation
Establishment of pool and participation of Opinion Leaders and Donors
Provision of tech. support to decision-makers
Development of media campaign
Implementation of methodologies and tools
National reports
Conduct of trainings
Development of two-year national advocacy plans and strategies
Conduct of working advocacy meetings and national dialogues
Participation of national decision-makers, partners and CCM
Provision of technical support to community
Media actions

24. Element A: Evidence base (1)
Phase
Level
Process
Indicator
Status
Forming evidence base for HR
Regional
1. Development of the methodology and tools for assessment of investments in harm reduction
Developed regional methodology for harm reduction investment assessment
Developed and published report in Y1
2. Baseline assessment of current situation in HR
Developed methodology
Conducted assessment
Developed and published report in Y2 (currently in progress)
3. Development of the methodology for assessment and documentation of best practices of HR financing:
Analysis of existing practices of financing and HGO contracting: advantages and disadvantages
Developed review
In progress Y3
National
4. Documentation of good practices and peer to peer education
Number of documented best practices
5. Provision of technical support for implementation of projects on good practice documentation

25. Element A: Evidence base (2)
Phase
Level
Process
Indicator
Status
Forming evidence base for HR
National and regional
6. Development of a training module for community driven assessment: whether harm reduction services meet drug users' needs:
National training
Development of a training module (regional)
Developed methodology and instruments for assessment of harm reduction services
Number of trained participants
Developed and published report in Y1
Conducted training for community in Y1
National
7. Implementation of drug users' needs assessment by PWUD groups, initiatives and/or CBOs:
Provision of national technical support by EHRN on data collection and analysis
Conduct of data collection and data analysis by SRs and SSRs
Number of participants in focus groups
Number of participants in questionnaires
Data included in national report
Provided tech. support in Y1
Assessment in Y1
8. Publication of national reports "Harm reduction: the case for strategic investment“:
Current level of funding HR
Needs and gaps in HR funding
Prioritization of HR services and recommendations
Published national reports
Published report in Y1/Y2
Regional
9. Publication of a regional report "Harm Reduction: The Case for Strategic Investment“
Published regional report
Published report in Y2

26. Element A: Evidence base (3)
Phase
Level
Process
Indicator
Status
Planning
National
Development of two-year national advocacy plans and strategies:
Development of national advocacy plans
Development of country positions
Development of technical support requests to donors
National advocacy plans
National country position
Number of request for tech. support
Provision of technical support by EHRN to SRs
Developed plans in Y1
Country position Y2
Tech. request Y3
In progress Y1-Y3
Regional
2. Development of a regional advocacy strategy, plan and a regional campaign "Harm Reduction Works: Fund It“:
Development of regional advocacy strategy
Development of regional advocacy plan
Development of regional campaign
Regional advocacy strategy
Regional advocacy plan
Regional campaign outline
Developed strategy, plan and campaign in Y2
Development of advocacy statements at the national and local levels
Conducted meetings with focus on investments in harm reduction/ or harm reduction barriers
Number of OFFICIAL* advocacy meetings;
Number of participants
Number of new national advocacy statements prepared and sent to national decision-making institutions in the form of written letters
 In progress Y2/Y3

27. Element B: Collaboration between International, Regional and National Stakeholders (1)
Stakeholders groups
Level
 Process
 Indicator
Status
High level harm reduction advocates (Opinion leaders)
Regional
1. Conduct of national dialogues
2. Conduct of advocacy meetings with participation of regional advocates
Number of round tables
Number of Opinion Leaders participants
Meeting results (statement, follow-up, if applicable)
In progress Y2/Y3
3. Establishment of pool of High level harm reduction advocates
Number of participants
Number of meetings
In progress Y1-Y3
National decision-makers
National
1. Conduct of working advocacy meetings at the national level with participation of national decision makers
Number of OFFICIAL advocacy meetings
Number of participants decision makers
2. Conduct of national dialogues
Partners
Conduct of advocacy meetings with participation of partners
Number of participants partners

28. Element B: Collaboration between International, Regional and National Stakeholders (2)
Stakeholders groups
Level
 Process
 Indicator
Status
Donors
Regional
Conducted meetings with participation of different stakeholders groups:
High-level Policy Meeting (Tbilisi Dialogue)
Conducted Donors’ Meeting
Conducted EECA Regional Harm Reduction Conference
Conducted Drug Users' Forum (Moldova)
Number of participants
Number of donors participants
Number of public (SRs and SSRs) participants
Number of reports
Resolutions
Conducted in Y2
In progress Y3
Conducted Y1 and Y2
National
Other conducted advocacy meetings with Donors and public participation
Number of meetings
Meeting results (statement, follow-up, if applicable)

29. Element B: Collaboration between International, Regional and National Stakeholders (3)
Stakeholders groups
Level
 Process
 Indicator
Status
Community
Regional and national
Participation in national and regional meeting (and other events)
Number of meetings
Number of SSRs participants
Number of regional meetings
Number of national meetings
Meeting results (statement, follow-up, if applicable)
In progress Y1-Y3
CCM
National
1. Conducted trainings on transitional processes and participation in the processes
Number of trained participants
Reports
In progress Y2-Y3
2. Participation in CCM events by EHRN
Presentations
In progress Y2/Y3
Regional
3. CCM members’ participation in EHRN conducted events
Number of CCM participants

30. Element B: Influence of Collaboration Influence on Investments in Harm Reduction (1)
Allies
Target stakeholders
Phase
Level
Process
Indicator
Status
Opinion leaders, community partners
National decision makers, CCM, donors
Support of national advocacy activities on the regional level
Regional
Conducted high-level events to support national advocacy of investments:
Drug users’ forum in Moldova
Regional Dialogue in Tbilisi
Donors meeting
Number of meetings
Resolutions
Reports
Number of participants
Conducted in Y1
Conducted in Y2
In progress Y3
Opinion leaders, community, partners
Relevant national bodies’ written statements/ decisions/adopted legal acts to provide financial or other kinds of support to HR programs, including the appearance of harm reduction interventions in strategies, programs, plans, budgets
National
1. Conduct of working advocacy meetings at the national level
Number of OFFICIAL advocacy meetings;
Meeting results (statement, follow-up, if applicable)
In progress Y2/Y3

31. Element B: Influence of Collaboration Influence on Investments in Harm Reduction (2)
Allies
Target stakeholders
Phase
Level
Process
Indicator
Status
Opinion leaders, community, partners
National decision makers, CCM, donors
Relevant national bodies’ written statements/ decisions/adopted legal acts to provide financial or other kinds of support to HR programs, including the appearance of harm reduction interventions in strategies, programs, plans, budgets
National
2. Conduct of national dialogues in 6 countries
Number of dialogues;
Number of requests for tech. support
Reports
Number of national decision-makers participants
Number of donors participants
In progress Y2/Y3
Regional
3. Provision of tech. support to decision-makers on harm reduction:
Training
Study visits
Site visits
Number of trainings
Number of decision-makers participants
Number of study visits
Number of site visits
In progress Y3

32. Element C: Media environment for HR
Stakeholder
Process
Indicator
Status
EHRN
Development of media campaign as a part of advocacy campaign
Advocacy campaign outline (process)
Developed in Y2
Media actions
Number of information products:
Fact sheets
Press-releases
Media briefings
Posts in social media (facebook, blog posts)
In progress Y1-Y3
SRs and SSRs
Journalists (media representatives)
Conducted events with participation of journalists
Number of contacts with journalists
Number of journalists participated in events
Number of produced information products on harm reduction of participated journalists

33. Element D: Advocacy capacity building
Level
Process
Indicators
Status
National
Trainings:
On community driven assessment: whether harm reduction services meet drug users' needs
On community-led advocacy and representation in decision-making bodies engagement
On documentation of good practices and peer to peer education
Number of participated SSRs
Conducted Y1
Conducted in Y2
In progress Y3
Provision of tech. support
Number of participated SRs
Reports
In progress Y1-Y3
Needs assessment implemented by PWUD groups, initiatives and/or CBOs
Number of participants in focus groups
Number of participants in questionnaires
Data included in national report
Conducted in Y1
Documentation of good practices and peer to peer education
Number of SSRs participants
Developed review
Number of documented best practices
and regional
Participation in advocacy meetings/events:
Regional
Number of meetings
Number of participants SSRs