GENEVA EVALUATION NETWORK WORKSHOP CONFERENCE EVALUATION Organized by Laetitia Lienart & Glenn O’Neil Geneva, 16 March 2011

WHY IS IT IMPORTANT TO EVALUATE CONFERENCES? ESSENTIAL FOR… Institutional Memory Continuous Learning/ Improvement Accountability

WHAT ARE THE OBJECTIVES OF CONFERENCE EVALUATIONS? ASSESS PROCESS Governance Programme Logistics & onsite support Information/ Communication IMMEDIATE OUTCOMES Reactions Learnings/ Benefits Applications IMPACTS (lasting changes) At individual level At organizational level At country level

RELEVANT DATA COLLECTION METHODS Face-to-face or phone individual interviews (structured & semi-structured) Focus group interviews Online surveys Printed surveys Structured observations of key sessions and conference areas Review of conference programme and online resources Review of statistical data on conference registration, scholarship recipients, abstracts, etc Review of statistical data and evaluation findings from previous conference to allow comparison over time

Use of rapporteurs to follow sessions addressing key topics. Their feedback can be also used to measure some indicators (e.g. number of sessions presenting new findings). Use of conference “instant” feedback systems. Use of network analysis and mapping. Analysis of the conference media coverage. Review of posts and comments left by delegates and non-attendees on the conference blog, Facebook page and Twitter. RELEVANT DATA COLLECTION METHODS (cont.)

Focus on IMPACT ASSESSMENT Assessing conference impact(s) is feasible but needs to be planned and budgeted for at the planning stage (incl. in ToRs) Methods: follow-up survey (online/face-to-face), action plans  1,195 AIDS 2008 delegates completed the survey  About 2/3 had learnt something new and had changed some aspects of their work practice thanks to the new knowledge gained at the conference  Almost half reported that AIDS 2008 had directly influenced their organizations’ HIV work  Almost 4 in 10 were aware of AIDS 2008’s influences on HIV work, policies or advocacy in their countries  75% had kept in contact with at least 1 person met at AIDS 2008, mainly to exchange knowledge, lessons learnt and/or suggested solutions (86%) Ex: AIDS 2008 follow-up survey (1,5 year after)

USE OF EVALUATION FINDINGS Evaluation findings should be “very usable” as conferences are often repeated annually or bi-annually. Importance of “buy-in” of conference organizers. Sharing of evaluation plan with conference organizers and committees/working groups*. Evaluation reports: the quality of content and format is crucial to attract readers and convince them that evaluation results are reliable and useable. Dissemination of evaluation results: timely, use a variety of channels depending on the target audience. Use of follow-up mechanisms** with conference organizers and relevant stakeholders. Review progress on evaluation findings in the lead-up to the next conference.

KEY LESSONS LEARNT 1.Over-positive feedback (new strategy to be tested in 2011). 2.Evaluation report more used as an accountability & marketing tool rather than for learning purposes. 3.Unwillingness of conference organizers to devote adequate human & financial resources to evaluation. 4.Impact assessment remains a challenge (difficult to measure the extent to which changes are attributable to the conference, especially policies, norms & guidelines). 5.Data disaggregation is important to make evaluation results more accurate and useful*. 6.Conference evaluation provides unique opportunity to see how findings are integrated (or not) into future conferences.

Further information Proceedings (slides & handouts) of a 1-day workshop on conference evaluation held in Nov 2010 are available on request ( Feel free to join the Conference Evaluation Google Group: Glenn’s blog has more resources on conference evaluation, see category “event evaluation: