1003 Evaluation—Measuring the Impact of Food Safety and Nutrition Policy Plaza Building, Concourse Level, Plaza Court 3 FSN Section Track Session This session takes stock of what we “know” about what works in influencing food safety and nutrition and provides a forum to share information on what is being done on evaluation of new and existing policy initiatives. Participants from federal and state governments will share their experiences in program evaluation and discuss common themes in defining objectives and measuring outcomes. The role of economics in the process will be highlighted in the presentations. Organizer: Victoria Salin, Texas A&M University Moderator: Victoria Salin, Texas A&M University Presentations: GAO’s Approach to Evaluating Federal Programs in Food Safety Lisa Shames, Alfredo Gomez and, David Bennett, U.S. Government Accountability Office Evaluation of School Nutrition Policy Phillip Gleason, Mathematica Policy Research Using a Balanced Scorecard to Evaluate a State Food Quality and Safety Program Timothy Herrman, Office of the Texas State Chemist; Peggy Wantwadi, Texas A&M University Program Evaluation Components of the United Kingdom Food Standards Agency’s 5-Year Strategic Plan Derrick Jones, U.K. Food Standards Agency SLIDE FOR INFO- not part of presentation

Program Evaluation Components of the United Kingdom Food Standards Agency's 5-Year Strategic Plan Abstract: The Food Standards Agency is an independent UK government department set up by an Act of Parliament in 2000 to protect the public's health and consumer interests in relation to food. In 2006 the Agency introduced its first annual review of its Strategic Plan That review measured and reported progress against strategic targets to ensure that the strategic plan is an evolving, dynamic document suitable for the Agency’s multiple purposes. A new Strategic Plan for has now been published and sets out a clear direction towards a vision of 'Safe food and healthy eating for all.' The updated plan does not signal a change in direction for the Agency. The key aims of food safety, eating for health and choice and the supporting strategies for delivery, remain the same. However, the approaches to measure impact are changing. This presentation will report on the innovations in evaluation that are under way to judge the impact of the new strategic plan. SLIDE FOR INFO- not part of presentation

Programme Evaluation Components of the United Kingdom Food Standards Agency's 5- Year Strategic Plan Derrick Jones Head of Analysis and Research Division Food Standards Agency AAEA Annual Meeting FSN Evaluation Track Session 25 July 2010

This session takes stock of what we “know” about what works in influencing food safety and nutrition and provides a forum to share information on what is being done on evaluation of new and existing policy initiatives. Participants from federal and state governments will share their experiences in program evaluation and discuss common themes in defining objectives and measuring outcomes. The role of economics in the process will be highlighted in the presentations. (From Concurrent Session planner, highlights added) Evaluation—Measuring the Impact of Food Safety and Nutrition Policy

Overview UK Food Standards Agency- Role and remit Development of FSA’s Strategic plan – Food chain analysis Evaluation work to support strategic plan delivery (establishing what we know, what is being done, objectives, outcomes) Some emerging lessons / issues NOTE – Change of UK Government May 2010 – change in “home” of nutrition policy and some labelling

About the Food Standards Agency Independent Government department set up in 2000 to protect consumers’ interests in relation to food A UK wide remit covering both Food Safety and Nutritional Health Vision: – “Safe food and healthy eating for all” Three core values: – Putting the consumer first – Openness and independence – Being science- and evidence-based

Improve food safety and the balance of people’s diets 3. Food products and catering meals are healthier The main priorities: 3.1 continue to achieve reductions in levels of saturated fat, salt and calories in food products 3.2 encourage the development, promotion and availability of healthier options when shopping and eating out 3.3 make sure that portion sizes appropriate for a healthy diet are available and promoted The main priorities: 3.1 continue to achieve reductions in levels of saturated fat, salt and calories in food products 3.2 encourage the development, promotion and availability of healthier options when shopping and eating out 3.3 make sure that portion sizes appropriate for a healthy diet are available and promoted 2. Imported food is safe to eat The main priorities: 2.1 work internationally to reduce risks from food and feed originating in non- EU countries 2.2 ensure risk-based, targeted checks at ports and local authority monitoring of imports throughout the food chain The main priorities: 2.1 work internationally to reduce risks from food and feed originating in non- EU countries 2.2 ensure risk-based, targeted checks at ports and local authority monitoring of imports throughout the food chain 4. Consumers understand about safe food and healthy eating, and have the information they need to make informed choices The main priorities: 4.1 improve public awareness and use of messages about healthy eating and good food hygiene practice at home 4.2 increase provision of information to consumers on the hygiene standards of food premises when they choose where to eat 4.3 increase the availability of information on calories in meals in catering establishments 4.4 promote the adoption of a single, simple and effective front-of-pack labelling approach 4.5 develop and promote integrated Government advice for consumers on food issues The main priorities: 4.1 improve public awareness and use of messages about healthy eating and good food hygiene practice at home 4.2 increase provision of information to consumers on the hygiene standards of food premises when they choose where to eat 4.3 increase the availability of information on calories in meals in catering establishments 4.4 promote the adoption of a single, simple and effective front-of-pack labelling approach 4.5 develop and promote integrated Government advice for consumers on food issues 1. Food produced or sold in the UK is safe to eat The main priorities: 1.1 reduce foodborne disease using a targeted approach – tackling campylobacter in chicken as a priority 1.2 increase horizon scanning and improve forensic knowledge of, and intelligence on, global food chains to identify and reduce the impact of potential new and re-emerging risks – particularly chemical contamination The main priorities: 1.1 reduce foodborne disease using a targeted approach – tackling campylobacter in chicken as a priority 1.2 increase horizon scanning and improve forensic knowledge of, and intelligence on, global food chains to identify and reduce the impact of potential new and re-emerging risks – particularly chemical contamination

Food Safety-Estimates of the main public health risks Death pa.Number Cases pa.Number of incidents pa. (2007) FSA Consumer tracker - % concerned Dec 2008 Foodborne Disease (FBD) (microbiological) , % Allergens/ intolerance~5-104, % Chemical contamination Not Known 66032% 3 TSEs-vCJD new cases % Radiological contamination less than 10 5 less than % 6 1 England and Wales for 2006, Health Protection Agency 2 Estimated new cases with peanut allergy for 2005 taken from ‘Primary care epidemiology of allergic disorders’ applied to UK population ( 3 Figure is for consumer concern about pesticides, as a proxy figures taken from ‘Incidence of variant Creutzfeldt-Jakob disease diagnoses and deaths in the UK January 1994 – December 2007’ ( 5 FSA estimates for UK population based on data on exposure from radiological discharges into the environment 6 June 2006 figure for consumer concern about irradiated food, used as a proxy. Irradiated food is no longer tracked in the consumer survey.

Summary: Key Pathogens 2007 Campylobacter is clearly the highest priority pathogen in terms of public health impacts Salmonellas and Listeria monocytogenes are also significant Contribution of VTEC O157 to deaths and costs are low by comparison Food related Norovirus cases are estimated to have trebled between 2003 and 2007 however, there are large uncertainties around the role of food, which make this trend far from certain – new IID study will provide a clearer picture. Pathogens Total CostDeathHospitalisationCases Key Sources of risk from the UK food chain %Rank % % % Campylobacter spp 33%118%383%136%1 Poultry meat & environmental contamination by farm animals Salmonellas non- typhoidal 15%221%26%23%5 Varied - but eggs important sources VTEC O157 4%55%52%50%10 Beef, lamb & environmental contamination by farm animals Listeria monocytogenes 12%337%12%30%11 Ready to eat foods (environment), vulnerable consumers Norovirus 6%47%41%819%2 Molluscs, food handlers Total (Based on 2007 HPA Data) £1,520m , ,766

Evaluation “Learning from what we do”, evaluation covers processes and outcomes. Need to consider evaluation from the outset Part of “ROAMEF” cycle, also fits with Impact Assessment (Post Implementation Review- i.e. did the policy achieve its aims?) Evaluation key to delivery of Strategic Plan

Forms of Evaluation

Evaluation Programme Body of work to support delivery of Strategic plan objectives Not all parts of Strategic Plan are at same stage Cannot evaluate everything to same degree What is appropriate? Where should we focus? Evaluation of individual projects and impact of overall programmes of activity

Evaluation Programme work Looking at Strategy and components Role of Evaluation Advisory Studies- looking at specific policy areas to help clarify issues and how best to evaluate. Recent studies in both nutrition(front of pack labelling, calorie labelling) and safety (national Scores on the Doors scheme) FSA currently developing its Foodborne Disease Strategy-involving many parts.

Evaluation Issues Importance of clear objectives and outcomes in determining appropriate evaluation Policy environment- legal framework (EU law), voluntary action, Better regulation agenda Multiple policy interventions (defining counterfactual?) Importance of understanding behaviour and incentives, needs multi-disciplinary approach. Focus – what can be evaluated? -How best to go about it?

Wider lessons-Evaluation themes Needs to be seen as part of policy cycle Think about evaluation at outset clear about objectives Issues of counterfactual in a changing, multi- policy environment Appropriate / proportionate Share findings

Further info… FSA website: