Measuring Outreach Effectiveness Web4Dev – November 2006 Alex McKenzie - Knowledge & Evaluation Capacity Development Independent.

Slides:

Advertisements

Similar presentations

Program Evaluation Alternative Approaches and Practical Guidelines

Advertisements

1 Evaluating Communication Plans Cvetina Yocheva Evaluation Unit DG REGIO 02/12/2009.

Building capacity for assessment leadership via professional development and mentoring of course coordinators Merrilyn Goos.

The Student VU Learnscope 2001 Project Tracey-ann Reynolds School of Business, TAFE Division.

Developing an Evaluation Strategy – experience in DFID Nick York Director – Country, Corporate and Global Evaluations, World Bank IEG Former Chief Professional.

Valuing evaluation beyond programme boundaries: Communicating evaluations to enhance development effectiveness globally Anna Downie

Screen 1 of 24 Reporting Food Security Information Understanding the User’s Information Needs At the end of this lesson you will be able to: define the.

Output Costing 11 th November 2010 Toby Spanier

Second Legislated Review of Community Treatment Orders Ministry of Health and Long-Term Care November 9, 2012.

Anna Drummond, UMV Chief Compliance Officer Robert F. Roach, NYU Chief Compliance Officer 2B. How’s Your Compliance Communication Service?

Nadine Drew Lynn Goldman Merrie Meyers Charles Webster.

Student Assessment Inventory for School Districts Inventory Planning Training.

Construction Management Practice Implications of the Theory of Construction Management.

15 April Fostering Entrepreneurship among young people through education: a EU perspective Simone Baldassarri Unit “Entrepreneurship” Forum “Delivering.

The Irish Association of Supported Employment (IASE) Job Shadow Initiative: A National Success Story EITHNE JARRETT IASE Roscommon Supported Employment.

Communication strategy and techniques to launch InnovMed Dr. Fatma H. Sayed Vienna meeting 25 June 2007.

The Operational Mandate for the Technical Centre for Agricultural and Rural Cooperation (CTA) André Vugayabagabo Senior Programme Coordinator, CTA.

Strengthening the quality of research for policy engagement in the African context – achievements and aspirations Tebogo B. Seleka Botswana Institute for.

From Evidence to Action: Addressing Challenges to Knowledge Translation in RHAs The Need to Know Team Meeting May 30, 2005.

EQARF Applying EQARF Framework and Guidelines to the Development and Testing of Eduplan.

JOINT STRATEGIC NEEDS ASSESSMENT Rebecca Cohen Policy Specialist, Chief Executive’s.

Skills Online: Building Practitioner Competence in an Inter-professional, Virtual Classroom Canadian Public Health Association 2008 Annual Conference.

Toolkit for Mainstreaming HIV and AIDS in the Education Sector Guidelines for Development Cooperation Agencies.

Rwanda MCC Threshold Program CIVIL SOCIETY STRENGTHENING PROJECT Cross-Cutting Advocacy Issues Data Collection Monitoring and Evaluation.

Welcome! Please join us via teleconference: Phone: Code:

1 WERT: WP 5 RG EVANS ASSOCIATES November 2010 Aim To pilot and evaluate the content and context of the course material with target groups To help women.

Strengthening Constituencies for Effective Competition Regimes in Select West African Countries (7Up4 Project) PROJECT OVERVIEW Presentation at NRG-I Meeting.

1 Analysing the contributions of fellowships to industrial development November 2010 Johannes Dobinger, UNIDO Evaluation Group.

ROLE OF INFORMATION IN MANAGING EDUCATION Ensuring appropriate and relevant information is available when needed.

Report on Puente in the Caribbean Caribbean Conference on Horizontal Cooperation in Social Protection January 19-20, 2011 Barbados Julie Nurse, Specialist.

Slide 1 Monitoring & Evaluation of e-Government Projects, Performance Metrics for CM People Process Technology Outputs Outcomes Impact.

Paul Griffiths and Roland Simon Wrap-up presentation What has the EMCDDA learned ?

Key Stakeholder Interviews Assessing Effectiveness of Washington State Board of Education Communications with Key Stakeholders.

Evaluation of EU Structural Funds information and publicity activities in Lithuania in Implementing recommendations for Dr. Klaudijus.

Our Journey with Family Engagement An Overview of North Penn School District’s experiences with Family Friendly Schools As part of our strategy to Engage.

Outreach Retreat 2009 UCS Website Taking Stock Web Stats Popular Pages Search Engine Traffic Stats and How Google Finds Us Status of Consulting.

Continuing the work of the Bill & Melinda Gates Foundation Presented by: Jeff Stauffer WebJunction Service Manager Date: 3 February 2005.

Office of International Research, Education, and Development, Virginia Tech SANREM CRSP Communications: Getting the message out Keith M. Moore Associate.

Corporate Social Responsibility LECTURE 25: Corporate Social Responsibility MGT

BEYOND MKUKUTA FRAMEWORK: Monitoring and Evaluation, Communication and Implementation Guide Presentation to the DPG Meeting 18 th January, 2011.

European Commission Joint Evaluation Unit common to EuropeAid, Relex and Development Methodology for Evaluation of Budget support operations at Country.

Effective Learning Support: The key to quality and success Enhancement of Learning Support.

Communication in Governance PREM Core Course Public Sector Governance April 2007 Paul Mitchell Development Communication Division The World Bank.

GENEVA EVALUATION NETWORK WORKSHOP CONFERENCE EVALUATION Organized by Laetitia Lienart & Glenn O’Neil Geneva, 16 March 2011.

Effectively Prioritising Key Outcomes for Tackling Poverty and Community Regeneration Thursday 25th November City Halls, Merchant City, Glasgow.

1 Fit for the Future Selvin Brown MBE Programme Director, GCS Improvement Programme November 2015.

Catholic Charities Performance and Quality Improvement (PQI)

Barcelona Declaration of Measurement Principles Presented June 17, 2010 Revised June 20, 2010 Final July 19, 2010 Global Alliance ICCO Institute for Public.

Regional Centers for Evaluation Capacity Development Regional Centers for Evaluation Capacity Development A Multilateral Initiative June 2009.

CT TEFT 1 November 5, Agenda Introduction Goal of Pilot Tier Piloting Activity to Pilot Role of Connecticut in the pilot Standards and Technologies.

Knowledge Exchange to Policy Action The Active Healthy Kids Canada Report Card on Physical Activity for Children and Youth Building Capacity for Use in.

Dimension 1: Define and Monitor Social Goals Today’s speakers: Margaret Namazzi, Opportunity Bank, Uganda (OBUL) Anton Simanowitz, Director, The Business.

C4EO – Ways of Working Heather Rushton, Planning and Performance 1.

1 MID TERM REVIEW MEETING RESA MEETING RESA PROJECT: KEY OUTPUTS 1 st April, 2009 RAJESH KUMAR, CUTS INTERNATIONAL.

World Bank Group Impact Evaluations: Relevance and Effectiveness Jeff Tanner Task Team Leader for Impact Evaluations Independent Evaluation Group Based.

European Union COMMUNICATION AND VISIBILITY. Importance of Visibility EU taxpayer money Need for awareness, accountability and transparency EU should.

Organizational Communications and Its Importance to Company Growth. Presented by: Kenneth Martinez Organizational Communications Manager.

ACF Office of Community Services (OCS) Community Services Block Grant (CSBG) Survey of Grantees Satisfaction with OCS Survey of Eligible Entities Satisfaction.

INNOVATIVE DIGITAL SOLUTIONS HOW WE DO BUSINESS….

Chapter Objectives Understand steps of effective advertising management. Recognize when to use in-house advertising and when to employ an external agency.

“Evaluation Follow-Up: Challenges and Lessons” AEA 2011 Conference Think-Tank Session Organizer: Scott Chaplowe Presenters: Osvaldo Feinstein Bidjan Nashat.

European Agency for Development in Special Needs Education Project updates Marcella Turner-Cmuchal.

Professional Social Networking Career Advising Career Advising Southern Connecticut State University Southern Connecticut State University.

0 Proposed AMI and Dynamic Pricing Customer Education Plan for Pepco’s Maryland Service Area November 15, 2010.

Leadership Development at Bruce Power

Scoping the CILIP Ethics Review

ACE II Communication Strategy

Enacting Multiple Strategies and Limiting Potential Successes: Reflections on Advocacy Evaluation, Competing Objectives, and Pathways to Policy Change.

The IAASB’s Future Strategy

Public engagement strategy

Presentation transcript:

Measuring Outreach Effectiveness Web4Dev – November 2006 Alex McKenzie - Knowledge & Evaluation Capacity Development Independent Evaluation Group – World Bank

Key Points: ►Share IEGWB’s approach for aligning web/online practice with mandate and business strategy ►Share our experience running several outreach campaigns, discuss metrics ►Measuring effectiveness and impact ►Would appreciate hearing about your own experience and/or feedback

Business Strategy ►IEG is an independent unit that assesses the effectiveness of World Bank Group’s interventions ►Well regarded for its evaluation capabilities and reports ►Combines Accountability & Learning ►“To be effective, IEG want’s to be an influential and reliable source of knowledge” ►New Outreach and Communications Strategy: –Target key audiences –Embed outreach throughout the evaluation process –Accelerate learning, encourage utilization –Measure results ►Decided on a campaign-based approach, 10 to 12 major evaluation products per year. ►Multi-pronged outreach campaigns: media, print, online, in-person, translations, etc. ►Piloted many new ideas, mainstreamed what worked

Key Audiences ►Systematic effort to identify audiences has led to two sets: “core” & campaign-specific (specialists) Identify decision-makers & influencers Identify key network nodes ►Evolving audience criteria: from “static” classifications, to others based on opportunity or time-sensitive demand ►Customize products to audience whenever economically feasible (e.g. conference CDROMs, translations, etc.)

Embed in Evaluation Process Web support for:Design PhaseResearch PhaseOutreach Phase Communications- Release approach paper - Interim data and research - Audience identification - Outreach campaign* - Campaign metrics Process support- Virtual team workspace - Research surveys - Feedback, discussion, comments Value Created>>Transparency, InclusionTransparency, EfficiencyKey messages for relevant audiences * Initial role for web

Encourage Utilization

Measuring Results ►Results-Based Management causal chain

Measuring Results – Causal Chain Source: “Strengthening EAP’s Results Orientation”, April 2004

Measuring Results - Outputs

Measuring Results - Outcomes ►Intermediate Outcomes: perception survey* of IEG evaluation products, covering: –Readership –Quality –Influence on Bank processes and policies (similar for external clients) –Utilization ►Annual Review of Operations Evaluation (AROE) 2006, to be released in November/December. *Source: IEG’s Annual Review of Operations Evaluation (AROE). See

Measuring Results - Outcomes ►Survey targeted individuals defined as intended audience: 4285 internal (WBG) and 2759 external contacts. Results are indicative for respondents (22%) but are not generalizable to population Readership InternalExternal Direct awareness:42%55% Overall awareness:56%76% heard via 64%69% visited web site12%28% received hard-copy20%15%

Perception of Quality – Staff/EDs (E4)

Perception of Quality – External Clients (E13)

Influence – Staff (E8)

Influence – External Clients (E15)

Staff Use of Evaluations (E9)

External Clients Use of Evaluations (E16)

Measuring Results – Survey Implications IEG provides good services to its main client, the Board. External audiences are also highly satisfied with the quality of IEG evaluations and considered them to be influential. However external audiences would like IEG to broaden its external consultation during evaluations. Bank staff see IEG products useful with deepening their understanding of a given subject. However they don’t see them as useful in day-to-day work. IEG needs to focus on influencing ongoing and future operations.

Measuring Results – Survey Implications ►All audiences suggest: High value placed on timeliness to provide information when decisions need to be made Deepening analysis through more consultation and attention to context Focus on operational value of findings ►A call for action for IEG’s outreach program 4-page briefs Customized “findings” brief, upstream, via More use of non-print media Balance “lecturer” approach with conversations

Thoughts Going Forward ►"What information consumes is rather obvious; it consumes the attention of its recipients. Hence, a wealth of information creates a poverty of attention." Herbert Simon, Nobel laureate

Your Experience/Feedback ►What has worked in your organization? (audience analysis; relevance/timeliness; measuring outputs, outcomes, impacts, etc.) ►Choices related to channels? High quality content vs. open/collaborative models ►Outreach models used? Worthwhile consultants & advice? ►What challenges do you see ahead?