MLLSM01 EVENTS POLICY LECTURE 8 Evaluating Events

Lecture content This lecture will discuss why monitoring and evaluation are so important to event policy makers and practitioners It will also review aspects of the evaluation process It will then reflect upon the principal techniques of evaluation utilised within the events arena Finally, it will consider why net impacts are the most important facet of evaluating the success or failure of an event policy intervention

The politics of evaluation Moore & Sykes (2000) argue that “monitoring and evaluation is closely linked with policy development at a strategic level and when specific projects are being designed and implemented” (p203) Related to the development of event aims and objectives – often derived from political aims So, the “approach to evaluation, the choice of what is measured and the judgement as to what has been achieved cannot be divorced from the wider political or cultural context” (Moore & Sykes, 2000: p203) Political actors ‘set the agenda’ (Lukes’ theory of power) and monitoring and evaluation legitimates it and produces transparency and accountability

Evaluating Events Despite the predominance of economic logic in the evaluation of event impacts until recently, this is changing as the socio-cultural and image impacts rise in importance (Richard and Wilson, 2004) This does not mean that the economic is now defunct; rather that it has been augmented by a wider understanding of social, cultural and environmental impacts – the economic is not enough As politics is about the allocation of scare resources, investment in an events-led urban strategy needs to be justified/legitimated on the basis of objective outcome and output measures – does it work (e.g. Commonwealth Games) Especially as we see “significant public sector expenditure to develop, underwrite and promote event-based tourism development” (Connell & Page, 2005: p64):  LA’s and LECs provide seed funding to encourage events growth and need to be satisfied of a ROI – EventScotland also invests in international/regional and local events on the basis of a return – tourist visits, regional identity, local importance.

The purpose of evaluation To judge whether there is a rationale for policy intervention/are interventions producing the desired outcomes (Moore & Spires, 2000). But also to:  Monitor progress of projects against target – has hosting the Special Olympics increased overseas visitation to Glasgow/Has the investment in EventScotland been worth it  Review and revision of strategy in light of new information/challenges – policy adjustment (e.g. new approach to extending visitor stay required)  Overall judgement on success or failure of strategy/policy – performance related and identifying causes of failure Evaluation in both public and private sector event contexts now ever more connected to KPIs – numbers, type of visitors, media profile, reduced reliance on public purse

Internal and External Evaluation Internal (organisational):  Team analysis and performance review  What worked, what can be improved upon, which systems need to be revised, which org structure is most effective  Development of KPIs to quantify objective performance of team:  Customer complaints, resident complaints, deadlines met, media exposure External (environmental/macro factors):  Reinforcing customer-focus  Seeking quantitative/qualitative responses on the customer ‘experience’  Visitor numbers, impression management

When to evaluate Three principal windows of opportunity for evaluating events: Pre-event:  Expectations, economics, ownership, ticket purchases During event:  Reliant on dynamic methodologies and support of organisers/funding partners Post-event:  Legacy (traffic, housing, pollution, community pride)  Online presence, building relationship databases

Methods of event evaluation Quantitative measures:  Visitor numbers (how many, where from: pre-booked tickets, pay as you go)  Spend per head (how much spent and on what element of the event) – economic additionalities  Customer surveys, business surveys (Connell & Page, 2005), online mechanisms Qualitative measures:  How did customers find out about the event (Advert, flyer, radio, press, word of mouth)  Expectations and satisfaction (rate satisfaction with event cost, value, product, impressions of destination)  Visual methods (video, camera), interviews, workshops, focus groups New tools: online digital media can now facilitate customer/visitor interaction and ongoing evaluation

Evaluating the social, cultural & environmental Waitts (1999; 2003) one of the first to consider socio- cultural dimensions of events:  He concluded that Sydney 2000 had brought about a shift from welfare to entrepreneurial goals; a functional transformation of the cityscapes to generate cultural capital and, the growing importance of media images However, politically, the economic still garners more support But, social models are being developed and cultural capital can be as much about image, perception and impressions as about immediate economic return The environmental is increasingly important as sustainable events become the target of organisers

Evaluating Images Hiller (1989;1998) and Ritchie & Smith (1991) the first to seriously consider the image effects of major events Now, as Hall (1992: 155) remarks, “hallmark events may be the regarded as the image makers of modern tourism” Started out with narrow definition of ‘image’ and one-off studies – failing to consider the multi-dimensional aspects of image and the benefits of longitudinal studies Richard & Wilson (2004) used designative (informational) and appraisive (feelings, values and meanings) elements of images to evaluate the impact of European Cultural Capital on Rotterdam:  Considered residents and tourists and cultural sector stakeholders  Focused on images attributes including: modern architecture (designative), water (designative), multicultural, working city, international, dynamic, culture and art, lots to discover, events (designative), shopping (designative), nightlife (designative), cozy, unsafe

From mega to community evaluation Evaluation strategies need to be specific to the nature and scope of the event and to be able to identifying potential spatial inequity (Connell & Page, 2005) In justifying the ‘benefits’ of attracting a large scale event to your city/country, organisers are expected to measure ROI, additional tourist spend, increased dwell time, infrastructural legacy etc. In contrast, the success (or failure) of a local community events strategy will be justified on the basis of building social capital (networks, sense of belonging), pride in place, sense of community, reductions in crime, quality of life The challenge for policy makers is that benefits of the mega/hallmark event are easier to document (economic impact assessments) than the longer term and more subjective social and cultural indicators Policy makers need to develop social impact tools to make the intangible tangible – only then will the social and cultural be taken seriously

Gross and net impacts Difference between what would have happened had the policy intervention not taken place and what additional benefits have accrued because of an intervention Links back to Carlsen’s (2004) arguments about multiplier effect – what added value does winning the rights to host the UEFA Cup Final have on a city – that wouldn’t have accrued anyway To reach net additional impact, value has to be assigned to:  Substitution – policy targeted investment takes the place of other investments  Displacement – additional activity from some sectors counterbalanced by less output from others (tourism displacement as a result of Olympic Games hosting)  Indirect impacts – additional spend in the local economy as a result of increased business for some firms The political issue is whether public sector investment leverages additional benefits and provides a healthy ROI for public stakeholders (e.g. London 2012 regeneration)

Strategic Objectives ECONOMICPHYSICALSOCIAL Tourism employment Environment/ infrastructure QOL, community cohesion Inputs (expenditure) Scot Exec funds Other public sector Private sector Activity measures Business assTrainingNeighbourhood renewal Participation in arts Reduction in crime Output and Outcome measures Start –upsEvent jobsExt of cultural activity No of performances No of volunteers Gross impacts Adapted from PACEC (1999) Gross additionality – displacement plus linkages/multipliers = Net Impacts

References Carlsen, J, Getz, D & Soutar, G (2000) ‘Event Evaluation Research’, Event Management, 6 (4): Richards, G & Wilson, J (2004) The Impact of Cultural Events on City Image: Rotterdam, Cultural Capital of Europe 2001, Urban Studies, 41 (10): Connell, J. & Page, J.S. (2005) Evaluating the Economic and Spatial Effects of an Event: The Case of the World Medical and Health Games. Tourism Geographies, Vol 7 (1), pp Moore, B & Spires, R (2000) Monitoring and Evaluation. In: Roberts, P & Sykes, H (eds) Urban Regeneration: A Handbook, London. Sage