EVENT EVALUATION: Powerpoint Figures from the book (Getz, 2018) Figure 3.1 CIPP Applied to a Charity Event Figure 3.2 Evaluation Complexity Model Figure 3.3 A Systems Model Figure 4.1: Major Considerations in the Evaluation Process Figure 5.4 Logic Model Illustrated Figure 6.1 Modified Balanced Scorecard Concept for Event Management and Event Tourism Figure 6.2 Generic Strategy Map Structure Figure 7.1 Overcoming Tunnel Vision Figure 7.2 The Event Compass - Concept Figure 7.3 Sample Event Compass Radar Graph Figure 8.1 A Framework for Evaluating the Event Organisation Figure 8.12 Root-Cause and Impact Forecasting Combined Figure 11.5 Importance-Performance Illustration Figure 12.4 A Hierarchical Model For Evaluating Training Effectiveness © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

GOALS PLANS ACTIONS OUTCOMES CONTEXT EVALUATION INPUT EVALUATION CIPP Applied to a Charity Event ACTIONS Produce a 1-day food event Generate sponsorship revenue Market to special-interest food tourists OUTCOMES X money raised Satisfied customers Enhanced community self-reliance PLANS Stakeholder input Resources available Market intelligence Past experience The concept; theme GOALS Raise money for charity Foster community development Attract tourists CONTEXT EVALUATION INPUT EVALUATION PROCESS EVALUATION PRODUCT EVALUATION CORE VALUES © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Evaluation Complexity Model Evaluating Long-term, Cumulative Impacts & Sustainability Determining The Worth Of Events, Portfolios, Policies Proving Cause & Effect POLITICAL COMPLEXITY Full Cost / Benefit Evaluation Comprehensive Impact Assessment Forecasting Impacts Evaluating Organisational Effectiveness and Efficiency Evaluating Goal Attainment; Summative evaluation Evaluating Experience & Quality From Multiple Stakeholder Perspectives Evaluating Performance (HR) Discrepancy Identification and Correction; Process Evaluation Problem Solving & Supporting Decisions With Evidence; Formative Evaluation Permanent Data Collection & Monitoring THEORETICAL & TECHNICAL COMPLEXITY © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

A Systems Model TRANSFORMING PROCESSES OUTPUTS INPUTS ENVIRONMENT -producing an event -managing event portfolios INTERNAL EVALUATIONS Internal functions OUTPUTS INPUTS A Systems Model ENVIRONMENT COMMUNITY CONTEXT EXTERNAL EVALUATION BY STAKEHOLDERS © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

EVALUATION CONTEXT; WHY EVALUATE? PARADIGMS & THEORIES EVALUATION CONTEXT; WHY EVALUATE? WHAT TO EVALUATE METHODS AND MEASURES USES OF EVALUATION Major Considerations in the Evaluation Process -Positivism -Interpretive -Critical/Emancipatory e.g., formal vs informal -Internal vs external -One-time vs periodic -Decision support or determining worth -Formative, -Process, -Summative -Qualitative and Quantitative -Indicators and KPIs -The TOOLBOX -Uses and Misuses -Ensuring relevance and utility © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Logic Model Illustrated for goal-attainment evaluation: single event & event portfolios INPUTS -mandate -resources -venues -information PURPOSE & GOALS PLANNED ACTIONS (the event as a transforming process) EXPECTED OUTPUTS -e.g. attendance, tourists, money raised, satisfaction DESIRED IMPACTS; CHANGE Single events do not usually pursue enduring change Process for Single Events Special Concerns for Event-Tourism Portfolios Portfolio Strategy; Long-term Investment -Overall ROI -Sustainability -Integration with other policy fields -Bidding (one-time events) -Owning/producing -Supporting events -Growth in tourism yield -Leveraging effects -Economic and community growth -Competitive advantages © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

ORGANISATIONAL CULTURE CUSTOMER & STAKEHOLDER ORIENTATION Modified Balanced Scorecard Concept for Event Management and Event Tourism ORGANISATIONAL CULTURE -enhancing capacity to learn, adapt, grow -evaluation system -continuous improvement CUSTOMER & STAKEHOLDER ORIENTATION -satisfying guests/customers -satisfying regulators, suppliers, facilitators, partners and residents SUSTAINABILITY -financial/economic -social/cultural -environmental INTERNAL OR TRANSFORMING PROCESSES -all management functions -event production -managed event portfolios MANDATE, VISION, STRATEGY, GOALS, KPIs, EVALUATIONS © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Generic Strategy Map Structure DESTINATION STATEMENT: Mandate, Vision, Core Values OUTCOMES/IMPACTS with Key Performance Indicators VALUE to be created for residents and other stakeholders: Economic, Ecological, Built Environment, Social, Cultural ACTION PLAN: The transforming processes that create value, each linked to desired outcomes ENABLERS: Organisational Culture (Learning and Growth);Internal Processes; Customers/Stakeholders; Finances/Stewardship/Sustainability © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Did we attain our goals? Overcoming Tunnel Vision Use theory Identify unintended and undesired consequences What EXTERNALITIES can be attributed to our event/actions? Overcoming Tunnel Vision Use theory or logic models Systematic analysis of inputs, processes, outputs Engage stakeholders on: -Pollution -Costs borne by others? -Cumulative impacts © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

The Event Compass - Concept © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Sample Event Compass Radar Graph ORGANISATION PLANNING MARKETING RISK DESIGN In this radar (‘spider web’) graph the organisation has been evaluated as being: Very strong on economic impact 90 out of 100 (100 being the benchmark for best practice) Strong on management of the event organisation 75 Good on residents 60 Good on environmental impacts 65 Good on socio-cultural impacts 65 Good on cultural value 60 Very weak on event brand and marketing 30 Weak on planning, design and quality 55 SOCIO-CULTURAL OUTCOMES ENVIRONMENTAL OUTOMES ECONOMIC OUTCOMES © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

SOCIAL RESPONSIBILITY INFORMATION & KNOWLEDGE A Framework for Evaluating The Event Organisation EVALUATING THE EVENT ORGANISATION OWNERSHIP MANDATE GOVERNANCE BUSINESS MODEL PLANNING Project, Strategic, Business, Marketing, Site, Operations DESIGN MARKETING COMMUNICATIONS SUSTAINABILITY GREEN OPERATIONS SOCIAL RESPONSIBILITY FINANCES CONTROL RISK AND SECURITY HUMAN RESOURCES PROFESSIONALISM ETHICS ADMINISTRATION Conformity To Standards Adequacy Of Support Services INFORMATION & KNOWLEDGE © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

PROBLEM Root-Cause and Impact Forecasting Combined CONSEQUENCES? CAUSES? CONSEQUENCES? Root-Cause and Impact Forecasting Combined SOCIAL IMPACTS ECONOMIC IMPACTS ENVIRONMENTAL IMPACTS PEOPLE PLACE PRODUCT PROGRAMMING Training PROBLEM Lack of Sponsors PRICE PACKAGING; DISTRIBUTION PROMOTION; COMMUNICATIONS FINANCIAL (LOSSES?) REPUTATION (DAMAGED?) PARTNERS (COMMITMENT?) PARTNERSHIPS © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK

Importance-Performance Illustration KEEP UP THE GOOD WORK CONCENTRATE HERE Alternative to grand means Grand Means of Scores LOW PRIORITY POSSIBLE OVERKILL Increasing Importance © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK Increasing Performance

GOAL: MEASURE POSITIVE RESULTS (ROI; ROO; ROE) GOAL: TRANSFER LEARNING TO THE EVENT; NEW BEHAVIOURS GOAL: LEARN SKILLS & KNOWLEDGE; CHANGE ATTITUDES GOAL: POSITIVE REACTIONS TO THE TRAINING EXPERIENCE A HIERARCHICAL MODEL FOR EVALUATING TRAINING EFFECTIVENESS Source: Adapted from Kirkpatrick and Kirkpatrick (2007) © Donald Getz 2018 Event Evaluation: Theory and Methods for event management and tourism, Goodfellow Publishers Ltd, Oxford, UK