EVALUATION An introduction: What is Audience Data and Where does it come from? Kate Pontin Visitor Studies and Learning

What is Evaluation? “Evaluation is the systematic collection of data and information about the characteristics, activities and outcomes of an exhibition or public programme, that is useful in making decisions about the programme’s improvement and continuation” (Ref. Korn) Photo: Interviewing is a common approach

Evaluation is a Cyclic Process for improving what we do Planning Evaluation objectives Collect evidence From different approaches Analyse and conclude findings Feed into practice Kate Pontin

Data can tell us? Through evaluation we can find out: who our visitors are, what they want what they think about your museum and gallery We can also find out who isn’t coming and why!

Data can also Provide information for future planning Help find out how effective existing services are Develop new projects Measure change Provide evidence for learning Develop a dialogue with visitors Share good practice Resolve differences of opinion And act as an advocacy tool

To make evaluation effective we need to know: What it is we want to find out Why you we to find it out How we are going to find it out Who we are going to ask Who the information is for How we are going to tell them And what we are going to do as a result!

Choosing approaches (tools) When thinking about tools we need to consider What it is we want to find out Who we are asking How we are going to analyse the data! Tools can either be Quantitative i.e. numbers Qualitative e.g. in-depth study of ideas and knowledge Using several tools can help make data more valid

Evaluation aims: Bruce Castle pack Choosing the right Tool Was the teachers’ pack helpful? If yes what was most useful? Would teachers have liked more support (and if so, what type)? Were instructions for preparation clear? Was the museum welcoming? If not why not? What were the pupils’ favourite aspects? What were pupils’ least favourite aspects? Were post-project ideas suitable? Did the sessions support the National Curriculum? And if so which aspects?

Case Study: Across the board Kate Pontin

Case Study: Across the board The exhibition key objectives were to: stimulate creativity, social integration, and develop problem-solving skills support areas across the National Curriculum including literacy, numeracy, design, history and citizenship develop long term partnerships between the British Museum and regional museums embed evaluation in the project and develop a model for wider dissemination

Case Study: Across the board What I want to find out? What are the students learning from the exhibition – using generic learning outcomes to structure this Why I want to find it out? To report back to funding bodies and the British Museum How am I going to find it out? Contact students and teachers Kate Pontin

Case Study: Across the board Who will I ask? Sample school visits for observation but questionnaires with all school visits Who the information is for/How am I going to tell them? My written report will go to Head of Learning at Hampshire Museum Service Kate Pontin

Case Study: Across the board Tools Used: Observations of some class sessions at the museum Survey feedback from teachers Discussion with pupils (in groups) Discussion with staff running the sessions

Case Study: Across the board Issues Observing children was quite difficult with such large groups in a small space The surveys from teachers were really useful but some data about pupil progress would have been useful too!

Case Study: Across the board I really, really enjoyed the African game. I played it 3 or 4 times and loved it! I wouldn’t swap my slim Play station 2 or TV for one, but I would buy one if they were for sale! I would definitely recommend it to people who are always bored (child) The children are full of motivation to create illustrations for their story this afternoon They have asked for more of the games in the classroom

Ethics Act professionally at all times.   Act professionally at all times. Inform participants about the purpose of the research, the reasons for their participation and how the information will be used. Be honest about the constraints influencing decision-making (eg. space, time, staff, budget) so that suggestions made are realistic and do not raise false expectations. Ask permission before recording proceedings by camera, tape or video recorder. Avoid references to particular individuals in final report. Ensure the methods used are open-ended so that you do not lead people to respond in a particular way. Use a range of methods so that you do not rely too heavily on one set of data. Do not make unjustifiable claims for the effectiveness of the methods or their results. Ensure that the visitor study is part of a longer term strategy, supported by management and that findings will be acted upon.