1. 1 Agency Effectiveness – the New Zealand Experience Commerce Commission Roundtable on Agency Effectiveness 16 August 2012 at 10am Dr Mark Berry, Chair Commerce Commission

2. 2 Outline Why ask questions about agency effectiveness? What general principles ought to apply in assessing effectiveness? The four pillars: o objectives/goals o selection and flexible application of correct “tools” o ex post assessments of performance o capability and capability enhancement (This is a condensed version of the eight Kovacic measures: see (2009) 16 Geo Mason L Rev 903,923)

3. 3 How Well Do We Do? All organisations need to set goals and measure performance against those goals. The assessment of antitrust agency performance is complex. Regrettably, there is no easy meaningful and consistent way to measure performance.

4. 4 How Well Do We Do? (continued) This does not mean that we should not assess performance (we must). It simply means the assessment is more complex: o quantification assessments are limited in scope and may be unreliable (and create wrong incentives), and o some of the more important assessment measures are essentially subjective and involve long-term perspectives.

5. 5 What Matters? Step 1, The Culture Shift In the past we have measured performance against activity counts (eg numbers of cases and press releases, penalties achieved etc). These measures matter, but only a little bit (ie they may reveal an agency “asleep at the wheel”). The shift from this “safe” way to measure performance has involved a culture change – and certain risks. The shift from the “old” to the “new” culture is ongoing. The search for what matters requires constant reassessment.

6. 6 What Matters? Step 2, The “New” Culture Our current approach to setting our performance goals followed strategic planning in 2009/2010 (and included stakeholder engagement). Our approach to assessing our effectiveness is based upon four central pillars: (1)The setting and achievement of objectives/goals. (2)The selection and flexible application of correct “tools” (ie advocacy, settlement or enforcement – and the policy agenda issue). (3)Ex post assessment of performance, measured against the stated objectives/goals. (4)Capability and capability enhancement.

7. 7 Objectives/Goals: Why? Objectives/goals need to be identified ex ante for a number of reasons: o they provide a basis upon which to measure progress and outcomes o they assist in allocation and prioritisation of scarce agency resources o they become a point of reference for the development of work programmes o they facilitate communication and accountability (and assist stakeholder dialogue), and o they motivate and provide guidance to staff. (See Kovacic et al (2011) European Competition Journal 25,31)

8. 8 What Are Our Objectives/Goals? “To achieve the best possible outcomes in competitive and regulated markets for the long-term benefit of New Zealanders.” Current governmental policy aims to build a more competitive and productive economy (will any given policy make the economy grow?) The outcomes that we seek, in order to achieve this goal, are as follows: o markets are more competitive, and o consumers are better informed (leading to more competitive markets)..

9. 9 What Approach to Achieving the Objectives/Goals? In order to assess if we are realising our objectives/goals, it is necessary to assess the impacts we may have. Impacts depend in large part on the selection of the range of “tools” that are available. We think there are three main measureable impacts: (1)improved levels of awareness and understanding of competition and consumer law (2)improved levels of business compliance with competition and consumer laws over time, and (3)detection and appropriate actions in response to non-compliant conduct.

10. 10 Strategic Framework We achieve the best possible outcomes in competitive and regulated markets for the long-term benefit of New Zealanders Markets are more competitive Businesses undertake fewer anti- competitive mergers and trade practices, allowing markets to function more competitively Consumers are better informed Businesses represent goods and services more accurately, allowing consumers to make better informed purchasing decisions Businesses and consumers are aware of and understand competition and consumer laws and the benefits of competition Improved levels of business compliance with competition and consumer law over time Business conduct that does not comply with competition or consumer laws is detected and responded to appropriately

11. 11 How Do We Measure These Three Impacts? (1)Awareness and understanding: o we conduct general surveys of awareness o the first survey in 2009/2010 reflected a high level of recognition, confidence and trust in us – but low levels of understanding of our competition laws o the challenge for us is one of communication/advocacy o sector specific targets for awareness/advocacy have also been identified (eg construction sector, given international cartel experience in this sector) and o our success depends on future surveys showing greater awareness and understanding.

12. 12 How Do We Measure These Three Impacts? (continued) (2)Improved levels of business compliance: o should be enhanced if we succeed with communication and advocacy, and o not easily measureable. (3)Detection and actions: o each case is assessed on its facts – no annual targets, and o actions can be based on: ⁻compensation or refunds for affected consumers ⁻penalties as a deterrent ⁻test cases (to create new law).

13. 13 An Aside – The Policy Issue The Commerce Commission is an independent Crown entity. It is a quasi-judicial body. Another agency, the Ministry of Business Innovation and Employment (MBIE) reports to the Minister of Commerce and is responsible for policy advice. As a general rule, we work closely with the MBIE on policy initiatives, and make operational submissions.

14. 14 An Aside – The Policy Issue (continued) But there are times where the achievement of our objectives/goals requires more than this. In these cases, we endeavour to lead policy change (ie monopolisation and interview powers). Success in such policy initiatives is consistent with the achievement of our objectives/goals.

15. 15 Ex Post Assessment of Performance Some have already been noted in the impacts discussion above. In broad terms we are endeavouring to measure our performance against the following criteria: o survey awareness of competition and consumer laws o percentage of businesses that have an active compliance programme

16. 16 Ex Post Assessment of Performance (continued) o levels of market competition not substantially lessened where we have granted clearance/authorisation or declined to take action (including OFT framework for ex post merger and market conduct reviews), and o penalties, compensation, refunds as an indication of appropriate outcomes (but no targets set because of perverse incentives). (For further discussion, see pages 13-14 of our Statement of Intent 2012-2015, www.comcom.govt.nz/accountability/).

17. 17 Capability and Capability Enhancement In terms of organisational effectiveness, the capability of staff is a “top of the list” consideration. The Commerce Commission is now 26 years old. Organisational capability has grown over time. A recent organisational restructure has been part of this process, recognising the need for enhanced capability of staff. The talent pool in New Zealand is small: accordingly international outreach for staff, co-ordination with other agencies, and participation in international fora and academic research centres is all the more important for us.

18. 18 Capability and Capability Enhancement (continued) This is a long-term game: investments made today (eg our proposed summer clerk programme) may not have a pay-back for some time yet. Anecdotally, capability is measureable (eg feed-back from stakeholders as to quality of work), and there is also the prospect of judicial scrutiny in some cases (eg Phase 1 of Air Cargo). Doc 1413385