1. Dr. Alexia Nalewaik and Prof. Anthony Mills

2. The Three E’s of Performance Audit
A performance audit examines the process by which an entity is achieving its objective using;
Economy – undertaking the work with least wastage of physical and financial resources (inputs)
Efficiency – performing work productively, with a high ratio of inputs to outputs
Effectiveness – extent to which business and stakeholder objectives are met (outputs)

3. QA and QC Quality Assurance Quality Control Processes Product
Prevention
Evaluation
Compliance
Performance

4. Nalewaik - Mills Performance Review method

5. Module 4 - Compliance Sample findings Retention not held consistently
Total equipment charge exceeds equipment Fair Market Value
Labor costs calculated incorrectly
Unallowable charges & reimbursable expenditures
Failure to give credit for deductive change orders
Change orders calculated using different unit rates
Color of money issues
Reconciliation issues

6. Module 1 – Project Planning
Sample findings
Gap between project scope and available funding, can’t afford the project!
Not enough contingency allocated – P50? P80?
Contract language that does not protect the owner from some risks
Disconnect between organization’s strategic plan and capital plan
Extreme low bid

7. Module 2 – Stakeholder Identification
Sample findings
Measures of “Effectiveness” are unclear or ambiguous
Stakeholders increased scope that was not beneficial to the project
Contractor failed to declared conflict of interest with other firms employed on the contract
Stakeholder turnover
Power, influence, and level of interest
Stakeholders held project permits hostage
Quotable book quote
Failure to understand stakeholders could very well mean failure of the project. (Nalewaik & Mills 2017, p51)

8. Module 3 – Risk Assessment
Sample findings
Risk modeled, but not managed
Fraud protection or Inspector General function not in place
Intentional understatement of risk
Lessons learned and risks not applied from previous projects
Extreme focus on only technical and financial risks
Quotable book quote
“one of the objectives of the performance review is to understand both project and enterprise vulnerabilities and opportunities …..strengthening related controls, examination of risk assessment is a key element of the performance review. (Nalewaik & Mills 2017, p56)

9. Module 5 – Resource Analysis
Sample findings
Equipment utilization is inconsistent
Redundancy, overburdening, & gaps in staffing
Failure to document the poor performance of contractors and consultants
Effective use of technology
Poor management resulting frequent turnover of staff
Duplicate IT systems
Materials and vendor management
Contingency spent before project broke ground
Quotable book quote
The concept of resources becomes broadened … to any that may impact project outcomes. (Nalewaik & Mills 2017, p65)

10. Module 6 – Management Controls
Sample findings
Project procedures that are burdensome or duplicative
Failure to update written procedures
“Independent PMO” had a conflict of interest
Lack of controls over who can sign invoices and contracts
Line item budget exceeded without explanation
Organizational structure isn’t functional
Lack of process QA/QC
Missing deductive change order
Line item transfers
Lack of change management

11. Module 7 – Post-Project Concerns
Sample findings
Projects that have not been closed out (for years)
Assets missing from asset management system
Funding remains in system after projects were closed out
Failure to capture lessons learned
Poor quality of historic data
Less than smooth transition to operations
Warranty / maintenance system incompatible with BIM
Missing closeout documentation
Quotable book quote
Much of what is reviewed during project closeout and post-project is related to effectiveness, and capturing information for success of …future projects (eg cost, (Nalewaik & Mills 2017, p73)

12. Recap - Performance Audit Objectives
Facilitate continuous improvement
Enable organizational maturity
Support a learning culture
Improve performance and professionalism
Provide assurance to stakeholders
Contribute to good governance
Empower critical questioning

13. Project Performance Review: Capturing the Value of Audit, Oversight, and Compliance for Project Success
Nalewaik & Mills 2017
Routledge / Taylor & Francis website
Hardcover & e-book
Amazon.com