Auditing and Assessing the PMS – Government and Contractor Requirements Presented by: Dr. Douglas Goetz, CPPM, CF NPMA Rocky Mountain Chapter 19 May 2010

TASKING Over the past six months design develop and write a DOD Property Guide…. What Professor Chuck Waszczak referred to as “The Property Book of Wisdom.” Its foundation – the DOD Property Manual Rewrite team: – Team Members were from the Air Force, Army, DCMA, Navy, and NASA The original rewrite died an ignominious death….

TASKING OSD tasked DAU to create six guides: – Property Administrators – Plant Clearance Officers – Program Managers – Procuring Contracting Officers – Administrative Contracting Officers – CORs/COTRs/TRCOs

Thinking!?!?! One Chapter of the PA Guide got me thinking! – In the words of Lyle Hestermann, it caused me to go, “Hmmmmm?” – What were the FOUNDATIONAL ELEMENTS of an ASSESSMENT? – What were the FOUNDATION ELEMENTS of an AUDIT? – And were/are we RIGOROUSLY applying those ELEMENTS?

FAR REQUIREMENTS GP CLAUSE – Requirement for a “SYSTEM” (b) Property management. – (1) The Contractor shall have a system to manage (control, use, preserve, protect, repair and maintain) Government property in its possession. – Requirement for “ASSESSMENT” (f) Contractor Plans and Systems – (3) The Contractor shall establish and maintain procedures necessary to ASSESS its property management system effectiveness, and shall perform periodic internal REVIEWS and AUDITS. – Allowance to “AUDIT” (g) SYSTEM ANALYSIS – (1) The Government shall have access to the contractor's premises and all Government property, at reasonable times, for the purposes of reviewing, inspecting and evaluating the Contractor's property management plan, systems, procedures, records

Contractor Requirement ASSESSMENT of their PMS – What methods can the contractor use… what exists out there today? E Standard Practice for Equipment Management Process Maturity Model Property Management E-Tools

Contractor Requirement There is no specific contractual requirement as to HOW the contractor is to “ASSESS,” “REVIEW” or “AUDIT” their PMS. – So long as they “assess” the outcomes/processes set forth in the GP Clause or other requirements. Contractors should determine the APPROPRIATE methodology(ies) for their “environment.” Ensure reliability, validity, and the ability to replicate the findings.

As our profession has matured the underlying concepts – and their accompanying semantics become more critical. The current perspective from DC (That 20 square miles of land surrounded by reality) is that our work, i.e., our reviews need more rigor and consistency. Just listen to the Congressional Hearings on DCAA! What does this mean? Next Slide!!! Government System “?????” What… Is it a survey or analysis or ???

Government System “?” ASPR Sup. 3 FOR DOD Ye’ Really Olden Days!Ye’ Recent Olden Days! DCMA GUIDEBOOK Current Guidance For DCMAers Property Control SYSTEM SURVEY Property Control SYSTEM ANALYSIS Property Management SYSTEM ANALYSIS DoD GUIDE For the Performance Of Contract Property Administration Proposed Guidance For All of DOD Property Management SYSTEM AUDIT

AUDIT REQUIREMENTS If Property, within the Government, is going to use the term AUDIT – they open themselves to the requirements of the AUDIT PROFESSION! AICPA IIA GAGAS - (Generally Accepted Government Auditing Standards)

AUDIT PROTOCOLS PROCESS – Since the FAR has moved to a PROCESS orientation it would be wise to view “Auditing” as a PROCESS. – And, since I am trying to move NPMA to a more RESEARCH based body… Thought it would be good to support my contentions based upon RESEARCH and not opinion. – So, think for a minute, looking at every reference I cited in earlier slides, are they not all based upon AUDIT PRINCIPLES and PRACTICES?

AUDIT PROTOCOLS The answer is yes! So, what does that mean to every contractor and every Government Employee involved with either ASSESSING their PMS or ANALYZING a PMS? The Answer… – We had best be well versed in AUDIT PROTOCOLS, METHODOLOGIES, and… WAIT FOR IT…. STANDARDS!!!

AUDIT PROTOCOLS (Regardless of which methodology you use) AUDIT PROCESS – what are the steps? Audit Objectives Define the attributes or transactions or characteristics Define the Population Specify the acceptance rejection rate Sampling Methodology Select Sample Perform the Audit Analyze Sample Determine Status Issue report Source: National Property Management Association, Intermediate Studies Manual, Clearwater, FL Pg Please note that there are TONS of Texts On the Auditing Process – encompassing VOLUMES!

So, how do I apply this? Let’s see – Some Examples! ASTM E 2452, 2005 Paragraph 8. Equipment Life-Cycle Phases Receiving – Identify and resolve shipping discrepancies – Record and post receipts DoD Property Manual M (1991) App. A Function 3 – Receiving – Discrepancies 2. Causes of discrepancies are investigated and documented. II 3. Contractor initiates appropriate discrepancy reports…. – Receiving Process 3. Receiving reports are promptly prepared that document items and quantities received, etc., etc., etc. Note: Neither of these documents Tell you the specific data elements Within each PROCESS. That is where Your PROCESS REQUIREMENTS and PROCEDURES are critical!

Both of these items may apply steps within the AUDIT PROCESS. For example: – What is your population? – What is your sample size? – What specific DATA ELEMENTS will you evaluate/test under this process: Timeliness Accuracy of data Etc. – How will you EVALUATE the “maturity” or “adequacy” of the sample/population? The EMPM uses maturity levels (1-5) The DoD Property Manual uses Confidence rates, i.e., acceptance and rejection rates. Both take into consideration issues of Materiality & Significance.

THE ISSUE We in the property community, at least under DOD, have grown “comfortable” in doing our work ONE WAY, i.e., using the DoD Property Manual. In today’s environment we need: – The ability to adapt and change! – The ability to think and analyze – creatively! – Most importantly – the ability to LEARN!

We need to be well versed in A WIDE VARIETY of AUDIT and ASSESSMENT PROTOCOLS!

An Interesting Quote “There are known knowns. There are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.” Donald Rumsfeld So, the concern is, what is the depth of your knowledge to properly apply an assessment or audit program/process?

Are you as comfortable with the Language of Auditing and Assessment as you are with Property?

“Well, wait a minute Doug – this is all new, it’s different – and it ain’t easy!” What to do?

Now, if I were “King of the Forest” and was to design a curriculum for the Field… Foundations in Statistics Statistical Process Control (SPC) – Including Metrics Auditing – With ALL of its embedded requirements Just take a look at the AICPA SAS or the DCAA Manuals Property Related Tools Maturity Model Construct AIA’s E-tools and Metrics Auditing and Assessment -- Relationship(s) to VCSes A few Tiny Questions - Why is it that DOD has just one two week on-line Property Audit Course yet DCAA and QA have extensive training & education requirements in auditing? And other Government agencies with Property responsibilities have NO training requirements…. And contractors, you’re not innocent either. What does YOUR company require/support?

What’s the old adage? “Jack of all trades and master of none.” – Anonymous BREADTH YES!!! We must have BREADTH to our knowledge and experience DEPTH But… we must also have DEPTH to our knowledge!

Our Challenge here… Again, go out and learn – all that you can, such that you acquire and develop the knowledge, skills, and abilities to perform exceptional work in the area of PROPERTY Audits and Assessments.

THANK YOU! Dr. Douglas N. Goetz, CPPM, CF OLD DAU PROFESSOR Recently Retired – now Rehired. President of the newly incorporated GP Associates. Hopefully a Good NPMA Instructor. Chief cook and bottle washer at Home, responsible for all “honeydue” requirements – though my wife has told me to stay out of HER house! 233 N. Maple Ave Fairborn, OH