The NPF & Scotland Performs: Analytical Underpinning and Challenges Mairi Spowage Office of the Chief Statistician 9 th June 2009

An outcome based approach This “New Approach” presented many opportunities for analysts We could now bring to bear the whole package of evidence to say something meaningful about change A way to go, particularly in evaluation

Introduced since May 2007

National Performance Framework

Summary of National Performance Framework

How were the indicators chosen? There was a need to be able to measure progress against the 5 strategic objectives and the 15 national outcomes

How were the indicators chosen? A selection of indicators were chosen to act as a representative set, so when taken all together they may be able to tell us something about progress on the outcome

Longer, Healthier Lives

Scotland Performs Designed to show how the government is performing against its key indicators and targets New and innovative approach Big difference to previous administrations

The Indicators Mixture of many types of targets and indicators, from existing targets to those which were set down by legislation to some which were not currently measureable Many are direction of travel indicators

Existing Targets/ Those from other frameworks HEAT – Health improvement, Efficiency, Access and Treatment E.g. Achieve annual milestones for reducing inpatient or day case waiting times culminating in the delivery of an 18 week referral to treatment time from December 2011

Those contained in legislation All unintentionally homeless households will be entitled to settled accommodation by 2012 Reduce the number of Scottish public bodies by 25% by 2011

Those which could not be/were not measured Improve knowledge transfer from research activity in universities Increase the average score of adults on the Warwick-Edinburgh Mental Wellbeing Scale by 2011 Reduce overall ecological footprint

Challenges Huge range of data sources –Survey vs. Administrative Data –Composite measures –Some SG sources, some UK Departments, some ONS, some from partner agencies Many are National Statistics, but many are not, at least at the moment Differing frequencies of publication and lags

Recent Change Arrows Each indicator has an arrow assigned to it Arrows comment on the change between the last two data points –Simple, easy to understand, can be universally applied and is transparent

Indicator Page

Recent Change Arrow Performance Improving Performance Maintaining Performance Worsening

Thresholds Thresholds are used to ensure that small fluctuations are not claimed as worsening or improvement, e.g. for smoking, NI 22; any difference within +/- 0.5 percentage points of last year's figure suggests that the position is more likely to be maintaining than showing any change. A decrease of 0.5 percentage points or more suggests the position is improving; whereas an increase of 0.5 percentage points or more suggests the position is worsening.

Thresholds Thresholds take into account: –Known Variance –Past trends –Significance of change –To some extent, the change required by the target

Threshold Examples GDP –0.1 percentage points Social Economy –£10M Housing Supply –1,000 Houses

Who sets the thresholds? Scotland Performs Technical Assessment Group (SPTAG) –Chief Statistician (chair) –Chief Researcher –Head of OCEA –Chief Scientist SPTAG make all analytical decisions to do with Scotland Performs

When data need to be updated… Each indicator has a Lead Analyst assigned to it from within Government We have built in the SP updates to the standard publication process The analyst submits a recommendation to the SPTAG The group make comments on the presentation of the information, to improve accessibility They approve or reject recommendation

SPAN Made up of all of the analysts who are involved across Government Network meets regularly to debate technical issues which underpin Scotland Performs Support to SPTAG Acts as a “peer-review” function

Future Challenges Reflecting the contribution of the wider public sector – our partners Evaluation of outcomes – how do we know we have achieved them? What does success look like?

Questions?