EASY–ECO 2010 Sustainable Development Evaluations in Europe: From a Decade of Practices, Politics and Science to Emerging Demands Brussels, November 2010 The limits of indicators in public policy evaluation: The case of e-waste. Cédric Gossart

The Holy Grail of policy evaluation « Is my policy evaluation going to be of any use? page 1

The e-waste problem WEEE = Waste of Electrical and Electronic Equipment Hazardous waste tonnes (world) 300% growth rate in developing countries Illegal exports page 2

E-waste policies 1. Europe: WEEE Directive 2002/96/EC 2. USA: State patchwork (NCER: 20% collection rate) 3. China: China WEEE effective in Switzerland: since 1994 (>80% collection rate) 5. France: since 2006 (25% collection rate) page 3

4 Best E-waste Policies project Initial aim: To comparatively evaluate different national policies and legislation on e- waste and, from this, to come up with legal and policy recommendations for “best e-waste policies”...

5 Aim: To explore how indicators can be used to identify best e-waste policies. Step 1: Collect indicators to build the e-waste profile of different countries, Step 2: Compare these profiles and identify key factors conducive to best e-waste policies. The indicators project

« Best » e-waste policy? Policy which outcomes meet the objectives defined in the official ewaste policy of a given country. No one-size-fits-all: The definition of “good” results varies across countries, A policy producing good results in country A may not produce the same results in country B. The criteria upon which e-waste policies are to be evaluated is the outcome of a democratic choice that cannot be determined by experts evaluating these policies (e.g. costs of the take back system, or collection and treatment rates).

7 Method 1. Collect indicators: Explore several cases (European countries: to find data more easily => mostly early movers: Switzerland, Belgium, Netherlands, France), Build framework for e-waste profile based on collected indicators, Test framework on one early moving country (Switzerland). 2. Identify key factors supporting best e-waste policies: Questionnaire to stakeholders (data analysis not enough)

Results 1) E-waste profile of Switzerland 2) Mapping available indicators (.../…) 3) Objective data analysis difficult to identify factors => Questionnaire for subjective analysis of factors conducive to best e-waste policies.

1) E-waste profile of Switzerland

2) Mapping of available indicators

Conclusions (1/2) Andrew Jordan (UEA) -Implementation lies in the hands of member states -Need common evaluation criteria that go beyond mere economic efficiency John Hontelez (EEB) -Simple indicators may have more impact on the policy- making process than complex indicators sets. page 11

1. The Holy Grail of policy evaluation does not exist. 2. My evaluation has been useful if somebody has learned something along the way. 3. I learned how to reach the saturated ears of policy-makers and citizens: The E-waste Solutions Index (ESI) …/… page 12 Conclusions (2/2)

page 13 Source: project internal document, please do not quote.

page 14 Thank you Cédric Gossart Associate Professor, Telecom Institute ETOS research group 9, rue Charles Fourier , Evry Cedex – France Tel. : +33 (0) Fax : +33 (0)

page 15 The E-waste Solutions Index (ESI).

page 16

page 17