Measurement Systems for Sustainability Arrow’10 Inclusive wealth – one particular metric Parris & Kates Review 12 indicator initiatives  How do we choose between different measurement systems? What are the relevant issues in evaluating such measurement systems? Can we design a metric for metrics?

Outline Important dimensions to report when proposing a measurement system for sustainability: 1. Choices 2. Assumptions 3. Measurement Unit (Resolution) 4. Time Scale 5. Uncertainty

Choices made designing indicators Inclusion and Exclusion of certain dimension or type of indicator e.g. exclusion of ecosystem services e.g. health capital (Arrow’10 vs Hamilton’99) Do there exist alternative indicators for the same dimension? e.g. Health: Life Expectancy vs. Percent Malnourished Justification for choices: Which choices were normative Which choices were empirically-based Aggregation or not?

Aggregation across indicators

Assumptions made in metric design Essential to both specify and justify assumptions Does the indicator rely on a key assumption Are indicator values robust to alternative assumptions

Examples of some assumptions social worth of marginal increase in life expectancy Value of statistical life = difference in pay between risky and riskless jobs times the difference in probability of death due to the job Assumption: wages represent competitive equilibrium between the markets for such jobs; all workers have the same risk preference Might not hold for countries with rigid labor markets or state interventions Arrow ‘10: U(t) – the structure of the underlying utility function remains constant over time Arrow ‘10: To obtain shadow prices assume marginal rate of substitution is equal to the marginal rate of transformation. Assumption: V(t) is maximized by economy (on optimal path)

Measurement unit Does the indicator apply to a single scale or multiple scales (e.g. world, country, region, village, household) what are the scales at which the indicator can be evaluated? what is the range of scales for which it can be computed with reasonable certainty? What is the sensitivity of the indicator to the choice of measurement unit? what is the variance of the indicator value when evaluated on finer granularity versus coarser granularity (can the aggregated indicator value hide huge distributional differences?)

Biodiversity as an indicator of ecosystem productivity At which scale should the indicator be computed? Evaluate correlation with desired sustainability property at different resolutions Does the indicator capture the desired sustainability property? (which alternative to choose) Western China (Yue et al. 2004)

Biodiversity Intactness Index at the three levels of environmental decision-making in South Africa (Biggs et al 2004) Distributional effects can vary greatly by scale Variance of indicator value at sub-scales

Time Scales Can the indicator be used for out-of-sample prediction, i.e. forecasting? How far into the future can the indicator be predicted? Can precise error-bounds be computed as a function of time into the future? Example: IPCC indices

Time Scales Does the indicator incorporate the well-being of future generations? Not at all Millennium Development Goals Index Yes, under the assumption of identical preferences Inclusive Wealth (Arrow 2010): Shadow prices depend on entire future of the economy Yes, taking into account changing preferences “Responsibility approach” (HDRP 2010/34): Acknowledges uncertainty about the true value of natural systems in the future

Time scales What is the operating time scale of the process that the indicator measures? Source: IPCC Millennium Ecosystem Assessment

Uncertainty Data/Measurement Uncertainty: What is the uncertainty in the data used to compute the index? Confidence intervals or variance analysis Example: Biodiversity Intactness Variance (BIV) as a formal measure of uncertainty to accompany the recently developed Biodiversity Intactness Index (BII) (Hui et al. 2008) Example: (Parris & Kates ’04) report on 12 initiatives none of which report on sensitivity to measurement unit

Uncertainty Structural Uncertainty How well does the indicator capture the fundamental relationships between the quantities represented? How does the model aggregate different entities? Is there a way to quantify it? Comprehensive reporting of sensitivity analysis Example: Millennium Development Goals Indicators Dashboard acknowledge uncertainty, don’t quantify it Example: Arrow’10 sensitivity analysis of comprehensive wealth

Uncertainty While there is more relevance as a metric moves in the impacts direction, there are also more explicit uncertainties involved and often less consistency among models.

Uncertainty How to report uncertainty? How to aggregate uncertainty in order to report a single uncertainty value for an aggregate indicator? Consolidating risk and uncertainty, one approach:

Conclusions