Health and Consumers Health and Consumers NAS Network feedback on reference document "Objectives and Indicators" MANCP Network meeting FVO, Grange November 2014

Health and Consumers Health and Consumers In plenary, members analysed the document "Objectives and Indicators", Version 1, Revision 5 of 15/07/2014 Replied to three questions: 1.Is the document useful for NAS auditors? 2.Is the approach followed for the structure of the document adequate? 3.What can be improved? NAS Network meeting of /09/2014

Health and Consumers Health and Consumers Auditors don’t write the MANCP but if people who write the MANCP follow the reference document then it will be more easily audited. Yes, it is useful but if it is to be used in an audit context it needs an annex with specific examples (e.g. validation of indicators). Provides important background information on “objectives and indicators” in the MANCP context. Is the document useful for NAS auditors?

Health and Consumers Health and Consumers Helps an auditor understand the auditees’ intention and direction of their control systems and how they define objectives and indicators (common terminology), but is not a tool for audit performance. Encourages MS to have objectives and indicators and is a good reference for CAs that have to design systems. Not relevant for audits directly but allows understanding of how indicators are developed/created. Is the document useful for NAS auditors?

Health and Consumers Health and Consumers Outcome of the use of the reference document is useful for the auditors (useful for improving the MANCP). More useful for management level, but still useful for auditors as a background. It needs to be completed with national audit instructions and translated into MS languages. Is the document useful for NAS auditors?

Health and Consumers Health and Consumers In general yes. The parts “purpose and use” and “characteristics” (although the acronym RACER was new for some participants) are useful. The part “principles” is confusing. Does the scope include “fraud” too? (Example on Annex 1.2). More effective to have objectives and indicators together in one reference document. Is the approach followed for the structure of the document adequate?

Health and Consumers Health and Consumers Impact of indicators in terms of behaviour/risk management needs to be highlighted. Needs to be included in the wider context of the purpose of the MANCP. Structure and logics are adequate but the diagrams need more explanations (for instance diagram 2 and its legend). Is the approach followed for the structure of the document adequate?

Health and Consumers Health and Consumers Strategic objectives cannot be SMART. On the other hand, operational objectives are easier to fit in the SMART characteristics. A hierarchy of objectives should be defined, where indicators are linked specifically with operational objectives. Part “Types and Examples” could be clearer and include more examples. A procedure for acceptance of objectives and indicators is missing (top-down, down-up). Better examples linked with diagram 2. What can be improved?

Health and Consumers Health and Consumers More diverse examples (quantitative and qualitative indicators). In the definition of indicators change “tool” to “feature”. Title of the document doesn’t reflect content. Reference to “Effectiveness” reference document – link the two documents. Examples in annexes could be further developed (bearing in mind that examples for each sector could undermine a final agreement on the reference document). What can be improved?

Health and Consumers Health and Consumers Keep the categorisation of indicators in the definitions - “output”, “outcome” and “impact” indicators. Clarify the part “principles”. Include a methodology to build a good indicator (use a diagram?) Change title of the document – “Principles for objectives and indicators and good practice”. Add examples of key indicators. Prescriptive indicators – list of indicators that could be used by all MS? What can be improved?

Health and Consumers Health and Consumers Thank you! Unit F7 Operational and Administrative services European Commission Food and Veterinary Office Grange, Dunsany, Co. Meath, Ireland