SAACTA 13 th Annual Conference October 2010 Performance auditing: Where are we? By Dr. Shalini Singh Department Operations and Quality Management Durban University of Technology

Overview of presentation Way forward Conclusions & recommendations Results & discussion Methodology Objectives Aim Statement of the problem Introduction

Introduction - Challenges – Changing economic landscapes – Fierce competition – Recessions – Changing customer needs

Suggestion of Experts Change audit style Move away from traditional auditing To performance auditing Improve orgn. effectiveness

More valuable Insight effectiveness of QMS Opportunity for Continuous improvement

Explore the current thinking of compliance and performance auditing AIM

OBJECTIVES Investigate organizations’ readiness for change Literature to highlight current practice

Research design & methodology Inductive reasoning (observer audits) Critical thinking (conferences & literature) Theoretical strategy (review of literature) Empirical evidence (questionnaires) Phenomenologica l QualitativeSamplingPurposive

Current thinking of experts- literature review Documented system measured against std No improvement Measuring up to objectives Sample of QMS audited Assumptions based on above on compliance or non- compliance of entire system No. and intensity of customer complaints – (conclude why failure occurred) Awareness and adoption of policy – (if policy was driving output or just existing) QMS designed accordance with what auditor is looking for Assess NC – product or customer oriented Focus attention and CA

Current thinking of experts- literature review Audit process Auditor must establish - distinction between actual functioning and doc. QMS Uncover minimalist implementation Avert ceremonial auditing Increase value of audit

Current thinking of experts- literature review Auditors should focus on Is orgn. customer focused Process monitoring Improving performance Focus fundamental process Facilitating CI Understand quality at each functional level Competency of employees Simplifying documents

Update of ISO version Determine – rather than identify – understanding, relevant functions\ levels (process approach) Effectiveness – measurable objectives Business environment – holistic Customer satisfaction – eliminate unnecessary, delight Applicable – value add, robustness Refocus pitch

Science of discipline with quality Can not be isolated Not a one size fits all Experts of discipline to maintain Update of ISO version cont.. Integration

Results & discussion – orgn.’s readiness. Population Manufacturing & service type +500 questionnaires Sample = 37 Manufacturing = 27Service = 8Unknown = 2 Mainly internal auditors

?Status (instruction to change) of where we are? – Strongly Agree (SA) = 9 – Unsure (U) = 2 – Strongly Disagree (SD)= 16 – Majority of orgns. have not changed – Changes in std, expert opinion, case studies – not changed – Why? Too much trouble, time, lack of expertise Q “My orgn. has informed us to move from compliance to performance audits”

?If internal auditors were aware of a change? – SA = 20 – U = 2 – SD = 5 – Majority auditors aware of performance audits, so…. – What is preventing this change? Top mgt, resistance to change (conventional wisdom), priority Q “As an auditor I understand the difference between compliance and performance audits”

?Org. were serious about performance audits & feedback effectiveness of QMS – SA = 7 – U = 11 – SD = 9 – Majority – unsure, possible ambiguity in question? – 9 haven’t been retrained – mindset? – Lack urgency, lack of information or guidance, disbelief Q “All the internal auditors have been retrained on the processes they audit”

?Expect optimisation, C I, etc – SA = 21 – U = 3 – SD = 3 – Majority understand – Infer – expect - CI, effectiveness, optimisation Q “All internal auditors understand the processes in the areas which they audit”

? Confidence in external auditor & moved to performance audits? – SA = 10 – U = 11 – SD = 6 – From results respondents are confident with external audits – Outcomes beneficial to them Q “All external auditors understand the processes in the areas which they audit”

?Multiskilling, level of preparedness, process approach, systems approach – holistic (WCM) – SA = 19 – U = 3 – SD = 4 – Possible contradiction – above auditors trained on pre & post processes – Multiskilled, implement systems thinking Q “Employees are trained only on the processes and activities in his/her job description”

?Extent of knowledge – understand the science and interconnections of processes? ?Process approach, optimisation, bottle-necks – SA = 13 – U = 8 – SD = 6 – SA = 12 – U = 9 – SD = 5 Q “Employees are trained on the process preceding their process and activities” Q “Employees are trained on the process proceeding their process and activities”

Third of industry not sure Third not trained Interconnections, PA, optimisation ISO 9001:2008 – “determine & manage numerous interlinked activities”

Effectiveness? Interlinked activities Domino effect Communication Process Approach Systems thinking

? Has change been made? – SA = 5 – U = 5 – SD = 16 – Why? Q “Our organization has changed the we way we audit”

? Committed was there, but implemented?? – SA = 13 – U = 9 – SD = 5 – Why? – Urgency? Q “Management is committed to change to performance audits”

?Feelings about compliance audits for future intention of performance audits – SA = 13 – U = 7 – SD = 7 – Warning to avoid - perception performance audits Q “Compliance audits are cumbersome”

? Usefulness of compliance audits – SA = 17 – U = 5 – SD = 5 – Generally – useful – Perhaps the auditor knows more than the orgn.? Q “Compliance audits yield improvement for the organization”

?Optimisation expected? Internal report comprehensive for external auditor – SA = 19 – U = 3 – SD = 4 – SA = 14 – U = 8 – SD = 4 Q “Internal auditors are well informed about processes” Q “External auditors are well informed about processes”

? How beneficial ? – SA = 16 – U = 12 – SD = 9 – SA = 17 – U = 2 – SD = 7 Q “Internal auditors noticed opportunities for improvement during an audit” Q “External auditors noticed opportunities for improvement during an audit”

– SA = 19 – U = 1 – SD = 6 – SA = 17 – U = 2 – SD = 6 Q “Internal auditors noticed non-conformances during an audit” Q “External auditors noticed non-conformances during an audit”

Both internal & external audits were found to be beneficial Consistent results KnowledgeImprovementNC

General comments from respondents! Updates come from American bodies not SA Use of certification for training No formal communication about change in auditing style External auditors unaware of performance audits Far from effectiveness when still struggling with compliance Compliance audits not conducted in all areas Internal audits focus on current issues- not representative General lack of findings

Conclusions & recommendations Has poor implementation prevented orgn. from optimisation? ISO – effectiveness of QMS to meet objectives – relooked at for alignment – Content with current situation Mgt. aware of performance audits Experts resolute about the change

Future work is being conducted with lead auditors Determine their perceptions & challenges with adoption Way forward

Thank you!