1. Keeping Track in Complicated and Complex Situations 1
Keeping Track in Complicated and Complex Situations
The ‚Process Monitoring of Impacts‘ approach
Anaheim
November 5th 2011
Richard Hummelbrunner
OEAR Regionalberatung Graz, Austria

2. The challenges of tracking achievement of effects in larger programs
Key program characteristics
Multiple objectives pursued at different levels or sites
Activities implemented by several (semi-autonomous) actors
Contribution to objectives through high numbers of outputs
Difficult to establish clear relationships between outputs and further effects (results, impacts)
Limited utility of many monitoring systems
Predominant focus on inputs or outputs (easy to capture)
Predominant use of quantitative indicators (easy to measure)
Information on result / impact indicators comes often (too) late to change course during implementation

3. The ‚Process Monitoring of Impacts‘ approach
Theory-based monitoring approach making use of logic models (‘impact diagrams’)
Focus on processes, which should lead to results / impacts
Logic models are considered as hypotheses (to be modified during implementation), not as ‘blue-prints for development
Perspectives and observations of various stakeholders are captured and reflected, making use of systemic methods
Core rationale of the approach
Provide information for programme actors as early as possible on the likeliness of achieving results/impacts
Particular emphasis on domains that can be influenced by them or for which they are responsible.

4. Monitoring of change processes
Basic assumptions for change along a result chain
Inputs are used to produce outputs
(= projects)
Outputs are used
(by someone, in a specific manner)
to reach results
Results will lead to (expected) impacts in a plausible manner
Inputs
Outputs
Results
Impacts 4

5. qualitative and quantitative information
OUTPUT (TYPES)
USE of OUTPUTS
RESULT
IMPACT
Indicator
e.g. Nr. of projects
Indicator
qualitative and quantitative information
Indicator
e.g. Nr. of business links I I I
Enterprises (esp. SME) collaborate in networks (also with large enterprises)
Increased linkages between enterprises at regional level/ scale
Advise for co-operation
Enterprises (esp. SME) carry out reorganisation processes
Enterprises (esp. SME) carry out reorganisation processes
Sustainable stabilisation of enterprises
Sustainable stabilisation of enterprises
Adaption to international competition
Adaption to international competition
Enterprises (esp. SME) carry out product finding processes
Enterprises (esp. SME) carry out product finding processes
New/ improved services, products and production processes
New/ improved services, products and production processes
External expertise,
Process consulting
External expertise,
Process consulting
Increase in employment/ new jobs
Increase in employment/ new jobs
Enterprises (esp. SME) introduce new technologies
Enterprises (esp. SME) introduce new technologies
Enterprises (esp. SME) gain new markets
Enterprises (esp. SME) gain new markets
New contacts with clients, new orders
New contacts with clients, new orders
Enterprises (esp. SME) conceive innovation-/ investment projects
Enterprises (esp. SME) conceive innovation-/ investment projects
Services to sensitize for innovations
Services to sensitize for innovations
AF6

6. Main advantages, based on experience gained so far (in Austria and beyond)
Particularly suited for interventions
with long effect chains, where results are produced at the end of the implementation period - or even later
implemented through a large number of projects and independent actors, process assumptions function as joint rules for maintaining course towards intended effects.
Advantages for specific stakeholder groups:
Program actors: Clarifying program logic, joint orientation for impact-led project assessment and selection, evidence base for joint learning
Project owners: more flexibility during implementation, because the monitoring focus is not on original plans, but the achievement of objectives
Evaluators: guidance for identifying evaluation questions and for analyzing information gathered at project level (e.g. reports, surveys).

7. Process Monitoring of Impacts Further Reading
“Process Monitoring of Impacts - Proposal for a new approach to monitor the implementation of Structural Fund Programmes” (R. Hummelbrunner, W. Huber, R. Arbter, 2005)
Download under:
Process Monitoring of Impacts - Proposals for monitoring Territorial Cooperation Programmes (R. Hummelbrunner, 2006)
Process Monitoring of Impacts - Applied Study for Territorial Cooperation Programmes ” (R. Hummelbrunner, 2008)
download both documents under :
Process Monitoring of Impacts, in Williams B., Hummelbrunner R.: Systems Concepts in Action - A Practitioners Toolkit, Stanford University Press 2011
Process Monitoring of Impacts Further Reading

8. Richard Hummelbrunner
ÖAR - Regionalberatung
Alberstrasse 10,
8010 Graz, AUSTRIA
T: +43/
Thank you for your attention