1. Numerical Advantage Pty. Ltd. Measuring performance measurement American Evaluation Association 2012 Conference Graham Smith

2. Numerical Advantage Pty. Ltd. Agenda Introduction Context Costs Benefits accountability improvement decision-making Program Logic Conclusion Interactive!

3. Numerical Advantage Pty. Ltd. INTRODUCTION Benefits: what gets measured gets managed Aid to evaluation Context: A contested and political arena

4. Numerical Advantage Pty. Ltd. Measurement quotes Measure what is measurable, and make measurable what is not so. (Galileo Galilei) If you can not measure it, you can not improve it. (Lord Kelvin)

5. Numerical Advantage Pty. Ltd. Distortion Campbell's Law: "The more any quantitative social indicator is used for social decision making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

6. Numerical Advantage Pty. Ltd. Costs Planning and development IT Data entry QA Revision of measures Analysis and reporting Management and review Senior management attention

7. Numerical Advantage Pty. Ltd. Benefits Intended Objectives Accountability Improvement Decision-making Better Outcomes

8. Numerical Advantage Pty. Ltd. Accountability ‘The obligation of an individual or organization to account for its activities, accept responsibility for them, and to disclose the results in a transparent manner.’ Agency theory: relationship between a principal and an agent

9. Numerical Advantage Pty. Ltd. Measuring Accountability Quality of reporting Frequency of reporting Perception of being held responsible Action taken to address errors or failures Increased confidence of stakeholders

10. Numerical Advantage Pty. Ltd. Measuring improvement Performance measures trend Customer perception Reforms implemented Management perception Case studies

11. Numerical Advantage Pty. Ltd. Measuring Decision making Management perceptions Level of appeal or complaint Customer satisfaction Overall success level of projects Case studies

12. Numerical Advantage Pty. Ltd. Logic diagram Program success Better decisions improvements Improved accountability Use of performance measures Quality of performance measurement system Design DataStaff support Management support Review Resources

13. Numerical Advantage Pty. Ltd. Discussion further discussion welcome: graham.smith@numericaladvantage.com.au