Introduction to ROI Methodology for Meeting Professionals Elling Hamso European Event ROI Institute ©MPI Foundation/Fusion Productions 2005

Understand the basic ROI methodology Understand the different evaluation levels Isolate the effects of the meeting Convert the effects of the meeting into $ values Today’s Meeting Objectives

What is ROI? Today’s Meeting Objectives

What is ROI? % Return on Investment

X % ROI = What is ROI? Sales Price€ Purchase Price€ = Profit€ € €

Why do you need to know about ROI? Today’s Meeting Objectives

Why ROI? Sales Argument Defend or get new budgets Selling events through ROI Talking the language of customer Sponsor ROI Competitive Advantage Design events to improve ROI More targeted events Compare Improve Agency offer ROI services to clients To other events To other marketing tools Benchmarking Show Competence Transparency Accountability Financial

“I need my people to think like business people and focus on the ‘business of meetings’ – not the meeting business.” Chief Executive Officer Benefits of Meeting & Event ROI

MPI Pathways to Excellence E I O ducation nfluence pportunity Provide education and recognition to the meeting planning profession Make senior managers understand the value of meetings Provide business opportunities for members ♫♫♫ Old MacDonald had a farm, ei, eio...

Can we measure everything in monetary values? Questions about ROI

Is it difficult? Is it expensive? Questions about ROI

Should we measure ROI for all meetings? Questions about ROI

What is ’acceptable’ ROI? Questions about ROI

Is it a well proven method? Questions about ROI

Refined over a 20-year period Thousands of ROI studies conducted every year 100 case studies published 2,500 individuals certified 30 books developed to support the model Adopted by hundreds of organizations in more than 40 countries The Phillips ROI Model

Some Users of the Phillips ROI Model  AT&T  Coca Cola  General Motors  Hewlett Packard  Bristol-Myers Squibb  Motorola  PricewaterhouseCoopers  Shell Oil  Singapore Airlines  US Department of Defense  NASA  Government of Italy  Government Canada

The Phillips ROI Model Develop Evaluation Plans Develop Evaluation Plans Evaluation Planning Data Collection Collect Data Before and During Meeting Collect Data Before and During Meeting Collect Data After Meeting Collect Data After Meeting Develop Meeting Objectives Develop Meeting Objectives © Copyright 2004 ROI Institute 1234

Calculate Costs of Meeting Calculate Costs of Meeting Isolate the Effects of the Meeting Isolate the Effects of the Meeting Convert Data to Monetary Value Convert Data to Monetary Value Calculate the Return On Investment Calculate the Return On Investment Communicate results Data Analysis Identify Intangible Benefits Identify Intangible Benefits Reporting © Copyright 2004 ROI Institute

Evaluation Planning Stakeholders’ needs A stakeholder will say: ”It is important to me that this meeting is sucessful and meets its objectives”

Evaluation Planning 5 levels of measurement

Evaluation Planning The Chain of Impact © Copyright 2004 ROI Institute 1.Participants react favorably to the meeting 2.Participants learn new skills or information 3.Participants apply new skills or information 4.A business impact is achieved 5.A return on investment is generated

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: 90% of attendees will indicate an intention to implement new sales strategies within two months after the meeting Answer: Level 1 - Satisfaction, reaction, planned action (intentions are always level 1)

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Decrease the amount of time required to complete projects by 5% within 6 months of the meeting Answer: Level 4 - Impact (time may be converted to money)

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Participants rate the relevance of the meeting to success in their jobs as 4.5 out of 5 Answer: Level 1 - Satisfaction, reaction, planned action

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Use the new customer interaction skills in 90% of situations where they are needed. Answer: Level 3 - Application

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Score 75 out of 100 or better on the new strategy quiz Answer: Level 2 - Learning

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Rate the meeting as a good investment for the company with an average of 4.3 out of 5. Answer: Level 1 - Satisfaction, reaction, planned action

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Increase sales from existing customers by 5% within 9 months of the meeting Answer: Level 4 - Impact

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Achieve a 25% return on investment within the 12 months of the meeting Answer: Level 5 - ROI

Evaluation Planning Assigning objectives to levels of measurement 1. Satisfaction, reaction, planned action 2. Learning 3. Application 4. Impact 5. ROI Objective: Successfully demonstrate new selling techniques during classroom role play Answer: Level 2 - Learning

Surveys (“smile sheets”) The Tools Level 1 Reaction, Satisfaction, Planned Action Data Collection © Copyright 2004 ROI Institute 80% of attendees would recommend the conference to others. Receive a speaker rating from participants of at least 4.5 out of % of attendees will indicate an intention to submit an RFP to one or more trade show vendors within 3 months of the meeting.

Tests, exams, audience response systems Level 2 Learning Data Collection © Copyright 2004 ROI Institute Name two major sponsors of the event 50% of booth attendees will list three product features The Tools

Follow up surveys, focus groups, observation The Tools Level 3 Application / Implementation Data Collection © Copyright 2004 ROI Institute Conduct a meeting with committee members to implement new association policy within one month of chapter leadership conference 50% of attendees will make an inquiry for the software within 6 months of product launch event

Follow up questionnaires, performance monitoring, scorecards The Tools Level 4 Business Impact Data Collection © Copyright 2004 ROI Institute Decrease customer complaints by 30% within 9 months Increase membership satisfaction level by 10% in 6 months Generate 1 million in signed contracts within 9 months of trade show

Net Meeting Benefit Total Cost The Tools Level 5 Return on Investment Data Collection x 100 = ROI © Copyright 2004 ROI Institute Achieve a 5% return on investment using first year benefits.

Control groups Forecasting methods Trend analysis Participants’ estimates Supervisors’ estimates Outside experts The Solution: The Problem: Other Influences Meeting Impact Total measured Impact Data Analysis Isolating the effects of the meeting © Copyright 2004 ROI Institute

Data Analysis Using control groups to isolate the effects of the meeting © Copyright 2004 ROI Institute Retail Merchandise Company – Interactive Selling Skills Programme Two groups of 3 stores matched on: Store size Store location (median houshold income in the area) Customer traffic levels Previous store perfomance Weekly sales 3 months after training ($) Trained group12,075 Control group10,449 Increase due to training1,626 (ROI was in this case calculated to be 118%)

Data Analysis Using participants’ estimates to isolate the effects of the meeting Elements influencing sales increase: Influence factor Sales kickoff meeting Advertizing campaign New incentive programme Product improvements Other factors Total 20% 15% 30% 25% 10% 100% Participants’ estimates of factors influencing sales increase following a sales kick-off meeting Confidence of estimate Adjusted influence factor 50% (i.e. 10 – 30%)10%

Profit from sales Converting employee’s time Historical costs Internal and external experts Participants’ estimates Supervisors’ and managers’ estimates Staff estimates The Solution: The Problem: Data Analysis Converting data to monetary value © Copyright 2004 ROI Institute

Data Analysis Converting data to monetary value using participants’ estimates Basis for ValueEstimated Value Confi- dence (%) Adjusted Value ($) Improvement in efficiency of group. $1,500 month x 12 (group estimate) $18,00085%$15,300 Turnover reduction: Two employees retained per year. Base salary x 1.5 $36,00090%$32,400 15% improvement in customer response time. Estimated value $1.100/month $13,20060%$7,920 Absenteeism reduction (50 absences per year x $150 standard value) $7,50085%$6,375 Examples of business impact converted to money after leadership training seminar With permission from Jack Phillips, ’Measuring Return on Investment’ Vol. 2. ASTD Press, 1997

Data Analysis Assigning costs Marketing Costs Developments Costs Program Materials Facilitator/Speaker Costs Facilities Costs Travel/Hotel/Meals Participants Salaries & Benefits Administrative/Overhead Costs Evaluation Costs Taxes © Copyright 2004 ROI Institute

€ Net Meeting Benefit € Meeting Costs x 100 = ROI Benefits of meeting (1st year)€750,000 Costs of meeting€500,000 € 250,000 € 500,000 x 100 = 50% Data Analysis Calculating ROI

Improved Image Improved Customer Service Reduced Complaints Reduced Conflicts Reduced Stress Data Analysis Potential intangible benefits © Copyright 2004 ROI Institute

The Phillips ROI Model Develop Evaluation Plans Develop Evaluation Plans Evaluation Planning Data Collection Collect Data Before and During Meeting Collect Data Before and During Meeting Collect Data After Meeting Collect Data After Meeting Develop Meeting Objectives Develop Meeting Objectives Level 1 – Reaction & Satisfaction Level 2 – Learning Level 3 – Application Level 4 – Impact Reporting 1234 © Copyright 2004 ROI Institute

Calculate Costs of Meeting Calculate Costs of Meeting Isolate the Effects of the Meeting Isolate the Effects of the Meeting Convert Data to Monetary Value Convert Data to Monetary Value Calculate the Return On Investment Calculate the Return On Investment Communicate results Data Analysis Identify Intangible Benefits Identify Intangible Benefits Reporting Level 5 - ROI © Copyright 2004 ROI Institute

Want to know more?

The Bottom Line on ROI. Patricia P. Phillips. Atlanta: CEP Press, Return in Investment in Training and Performance Improvement Programs (2nd Edition). Jack J. Phillips. Boston: Butterworth-Heinemann, Measuring Return on Investment, Volume 3. In In Action series, Patricia P. Phillips (editor) Jack J. Phillips (series editor). Alexandria: ASTD, *Available from Books on ROI

14 – 15 December 2005, Copenhagen (in English) 20 – 21 December 2005, London (in English) 11 – 12 January 2006, Frankfurt (in German) 18 – 19 January 2006, Stockholm (in Swedish) 8 – 9 February 2006, Amsterdam (in English) More information and registration on 2-day ROI Training Seminars

Thank you!

European Event ROI Institute Elling Hamso Vagsgt. 43 Postboks 171 NO-4302 Sandnes Norway Telephone: Bernhard Aggeler Flurstrasse 12 CH-6332 Hagendorn ZG Switzerland Telephone: