Advice on Data Used to Measure Outcomes Friday 20 th March 2009

Today’s Workshop Introduce myself & background 2008/09 SOA – SBC experience 2009/10 SOA – Context How we developed the 2009/10 SOA –Consultation Linking activities to SOA – the 4 areas PED leads How to measure success –The outcome –What SBC needs to measure – a worked example Issues in what to measure –Primary Some conclusions Discussion

Introduction Planning & Economic Development Business Information Unit has led development of Outcome Agreements Own experience of developing evaluation frameworks contributing to PSA in England Local Authorities Process of Logical Frameworks, Logic Chains etc

Area of activity Regeneration – town centre, retail, events & tourism expertise Property – economic development, infrastructure and property management expertise Economic Development – tourism, inward investment, business support expertise Rural Development – economic development, key sectors, funding, regeneration expertise Business Gateway – key sectors, business support expertise

Our Experience Scottish Government’s National Outcomes Development for 2008/09 Scottish Borders Council - Observations –New way of working –Community Planning Partnership = 20 partners –c 48 Outcomes –c 15+ under ‘economic’ –Mix of outcome, indicators, output, activity –Difficulties in performance management Different times

2009/10 SOA – Context NO - We realise our full economic potential with more and better employment opportunities for our people NO - We are better educated, more skilled and more successful, renowned for our research and innovation NO - Our young people are successful learners, confident individuals, effective contributors and responsible citizens NO - We live in well-designed, sustainable places where we are able to access the amenities and services we need

9 Outcomes that relate to the ‘economy’ 2.1 – supporting & growing business activity in key sectors 2.2 – promoting self- employment & creating sustainable businesses 2.3 – maximising employment opportunities 10.3 – improving the socio- economic performance of our town centres & growing stronger communities 3.1 – maximising participation in education, training & employment amongst economically inactive 3.2 – existing workforce is highly skilled and responsive to the needs of employers 3.3 – the Scottish Borders Campus is a World Class Centre of Learning 4.2 – all young people will be in education, training or employment 4.3 – young people demonstrate achievement & attainment

How did we decide? Examined where SBC PED is responsible & accountable? –Probably 4 key areas –Looked at the nature of our contribution Where SBC contributes with other Stakeholders? –Through Competitive Borders Where SBC’s other departments are responsible & accountable –Skills, education, learning

Consultation Internal consultation – SBC –Logic Chain Approach External consultation –Partners eg SEn, VisitScotland, SDS, Sectoral Groups –Education sector Business Planning

Objective of exercise To understand the activities, projects we do How these link to strategy, policy, objectives And how these need to link to Single and National Outcomes What is needed to deliver activities => And this is what we have come up with

Linking activities to SOA

But how do we measure success? Outcome = result from outputs – why we should be spending ££ (cumulative = impact) Output = The services delivered as a result of activities – measurable Activities = The way(s) in which we deliver/ implement Inputs = The resources used to deliver or produce activities (& outputs) eg human and financial resources

How to Measure the Outcome

And what is SBC measuring? Outcome? Outputs? Effectiveness in meeting Objectives? Inputs - Project Managers must consider the causal link between activities & ultimate performance So we use ‘if/ then’ methodology - at each level we can develop targets and indicators => For example, the data we need to collect

Example: Business Start Up Support

Regardless of ‘who measures’, there may be issues What to monitor – hard & soft outcomes? –Baseline Tracking - a data capture strategy; –Primary – Business or Beneficiary – survey? –Secondary – spatial availability? Sampling issues? Delay Frequency – over what time period –Longitudinal Attribution –Reference Case? Control Groups Collection & Ownership –Who and at what cost? –Single point of capture? Shared assessment? Benefits?

Pros & Cons - Primary MethodologyBenefitDrawback In project/ programme – data collection Vital in assessing baseline position against which to measure any change Requires coordination & resource ControlInformed view of benefits that are attributable to an intervention vs deadweight ie additionality Complicated, may be costly to recruit and challenge to ensure it is representative LongitudinalImportant to assess longer term effects of intervention; for labour market, allows tracking of changes Challenge to maintain reliability as non- response rates increase over time; cost

Advice on data used will depend on Why it needs to be collected; at which level of the ‘Logic Tree’; and by who The audience – is it –Internal (eg Economic Development, budget holders, project managers) –External (eg partners, funders, public, policy makers) Their different levels of interest –Achievements against objectives –Monitoring –Outcomes & Impacts –Value & Lessons –Strategic Added Value

Close Richard Sweetnam