1. LESSON 4 Quality System Audits
15November2013
Lesson 4: Safety Stock

2. Lesson Introduction
Given a FAR higher-level Quality Management System (QMS) requirement, the student will recognize the process for conducting a quality system audit.

3. Lesson Objectives
Upon completion of this lesson, you should be able to:
Relate the importance of QMS audits to Defense Contract Management Agency (DCMA) Government Contract Quality Assurance (GCQA) surveillance.
Distinguish between the three types of QMS audits: First Party, Second Party, and Third Party.
Recognize the five principles of auditing.
Identify factors that affect the decision to conduct a system audit.
Outline the steps for initiating an onsite system audit.
Lesson 4: Safety Stock

4. Lesson Objectives (cont.)
Upon completion of this lesson, you should be able to:
Relate the importance of the document review to the onsite system audit.
Outline the system audit activities.
Find commonly used system audit checklists and other system audit support documents.
Lesson 4: Safety Stock

5. Lesson Topics Importance of QMS Audits Audit Types
Five Principles of Auditing
System Audit Decision Factors
Initiating an Onsite System Audit
System Audit Activities
System Audit Checklists and Support
Documents

6. WIIFM This lesson will help you to:
Understand the DCMA Quality System Audit Policy, a Core Plus Quality Assurance Specialist (QAS) skill
Recognize the elements of an adequate quality system audit, which includes:
Auditing principles
Audit requirements
Steps for initiating and conducting a system audit
Data to evaluate and analyze

7. Overview

8. Policy Requirements – FAR 46.104
DFARS
DCMA Policy
Federal Acquisition Regulation (FAR) Contract Administration Office Responsibilities
Develop and apply efficient procedures for performing Government contract quality assurance actions under the contract.

9. Policy Requirements – DFARS 246.102
DCMA Policy
Defense Federal Acquisition Regulation Supplement (DFARS) Policy
Develop and manage a systematic, cost-effective Government contract quality assurance program.
Formal audits are one of DCMA’s cost-effective programs.

10. Policy Requirements – Quality System Audit
FAR
DFARS
DCMA Policy
Quality System Audit
System Audits shall be conducted on the QMS standard identified in existing contracts or Letters of Delegation (LOD).

11. Definitions
Management System – set of interrelated or interacting elements of an organization to establish policies and objectives, and processes to achieve those objectives
Note 1: A management system can address a single discipline or several disciplines, e.g. quality management, financial management or environmental management.
Note 2: The management system elements establish the organization’s structure, roles and responsibilities, planning, operation, policies, practices, rules, beliefs, objectives and processes to achieve those objectives.
Note 3: The scope of a management system can include the whole of the organization, specific and identified functions of the organization, specific and identified sections of the organization, or one or more functions across a group of organizations.
(ISO )

12. Definitions (cont.)
Quality Management System (QMS) – Part of a Management system to direct and control an organization with regard to quality.
(ISO )
Audit – A systematic, independent, and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which audit criteria are fulfilled.

13. Definitions (cont.)
Internal Audit – Also referred to as first party audits; conducted by, or on behalf of, the organization itself for management review and other internal purposes, and can form the basis for an organization’s declaration of conformity.
External Audits - Generally called second and third-party audits.
Second party audits are conducted by parties having an interest in the organization, such as customers, or by other persons on their behalf.
Third-party audits are conducted by external, independent auditing organizations such as those providing certification/registration of conformity.
(ISO )

14. Importance of QMS Audits
Lesson Topics:
Importance of QMS Audits
Audit Types
Five Principles of Auditing
System Audit Decision Factors
Initiating an Onsite System Audit
System Audit Activities
System Audit Checklists and Support Documents

15. Topic 1: Importance of QMS Audits
Plan, Do, Check, Act (PDCA) procedures
Continual QMS improvement
Plan Do
Plan Do
Act
Check
Act
Check

16. Plan Do Act Check Shewhart Cycle
Created by Walter A. Shewhart ( ), a physicist known as the father of Statistical Quality Control.
Four-step model for carrying out change.
Repeat PDCA cycle again and again for continuous improvement.
Plan Do
Plan Do
Act
Check
Act
Check

17. Plan Do Act Check PDCA Procedure
Plan: Recognize the need to plan. Do: Take action, put the plan in motion. Check: Measure and analyze the results. Act: Implement change based on results.
Plan Do
Plan Do
Act
Check
Act
Check

18. Continual Improvement of the QMS
Figure: ISO 9001:2008
Process-based Quality Management System
Continual improvement of the Quality Management System
Customers
Requirements
Satisfaction
Management responsibility
Resource management
Product realization
Product
Measurement, analysis, and improvement
Input
Output
Key
Value-adding activities
Information Flow

19. Question and Answer
What is one of the goals of a Quality Management System?
To recognize an opportunity.
To perform audits on suppliers.
To take action on problems found.
Continuous improvement.

20. Audit Types Lesson Topics: Importance of QMS Audits Audit Types
Five Principles of Auditing
System Audit Decision Factors
Initiating an Onsite System Audit
System Audit Activities
System Audit Checklists and Support Documents

21. Three types of audits: Topic 2: Audit Types First Party Second Party
Third Party

22. First Party Audit Internal Audits
Organizations (suppliers) conduct audits on themselves.
Internal auditing is required by most higher level QMS standards.
Identify?
Compare?
Confirm?
Recommend?
Eliminate?

23. Second Party Audits External audits Customer audits
Conducted by the Second Party to the contract.
DCMA represents the customer when delegated GCQA.
Joint DCMA/customer audits satisfy DCMA Policy requirement
Identify?
Ensure?
Determine?
Monitor?
Implement?
DCMA Quality System Audit is a 2nd Party Audit focused on the supplier’s QMS compliance to customer requirements.

24. Third Party Audits External audits by an objective party.
Conducted by organization or contractor not party to the Government contract or contract being administrated by DCMA Quality Assurance (QA).
Does NOT satisfy DCMA Policy requirement
Usually paid for by supplier for QMS certification purposes.

25. Five Principles of Auditing
Lesson Topics:
Importance of QMS Audits
Audit Types
Five Principles of Auditing
System Audit Decision Factors
Initiating an Onsite System Audit
System Audit Activities
System Audit Checklists and Support Documents

26. Topic 3: Five Principles of Auditing
American Society for Quality (ASQ) Quality Engineering (QE) 19011S Principles
Ethical Conduct
Fair Presentation
Due Professional Care
Independence
Evidence-based Approach

27. Ethical Conduct The foundation of professionalism.
The following personal characteristics are essential to auditors:
Trust
Integrity
Confidentiality
Discretion

28. Fair Presentation
Report the following materials truthfully and accurately:
Audit findings
Audit conclusions
Audit reports

29. Due Professional Care Apply diligence and judgment in auditing.
Do not wash over issues, but do not be rigid in approach either.
Diligence
Judgment 29

30. Independence Impartiality Objectivity
Forms the basis for the impartiality of the audit and objectivity of the audit conclusions.
Impartiality
Objectivity

31. Evidence-Based Approach
The rational method for reaching reliable and reproducible audit conclusions in a systematic audit process.

32. SYSTEM Audit Decision Factors
Lesson Topics:
Importance of QMS Audits
Audit Types
Five Principles of Auditing
System Audit Decision Factors
Initiating an Onsite System Audit
System Audit Activities
System Audit Checklists and Support Documents

33. Topic 4: System Audit Decision Factors
Higher-level quality requirements
System audit:
Surveillance
Frequency
Scope
Standards
ISO 19011
ASQ/ANSI/ISO 19011S
SAE AS9101

34. QMS Requirements (1 of 4)
QMS Review Cycle
Higher-Level Quality Requirement (FAR )
Results of Risk Assessment
Customer Required

35. QMS Requirements (2 of 4)
Higher-Level Quality Requirement (FAR )
QMS Review Cycle
Initial Quality System Audit as soon as possible on new contractor or contractor with new requirement, NTE 2 yrs from contract date.
Baseline to assess adequacy & compliance of QMS.
Subsequent audits based on risk. Update according to supplier performance history.
Escalate to Full system audit if systemic issues identified via GCQA surveillance.
Results of Risk Assessment
Customer Required

36. QMS Requirements (3 of 4)
Higher-Level Quality Requirement (FAR )
QMS Review Cycle
Results of Risk Assessment
“Yes” to Risk Statement Generator question 17, “Does supplier have a higher-level quality requirement (FAR )?”
Performance Factors and DC&A
Scope of audit based on Risk Impact.
Frequency & intensity based on Risk Likelihood.
Customer Required

37. QMS Requirements (4 of 4)
Higher-Level Quality Requirement (FAR )
QMS Review Cycle
Results of Risk Assessment
Customer Required
Customer requires DCMA system audit.
Customer requests DCMA participation in system audit.
Customer requests to participate in DCMA system audit.
NASA - Every 3 years (may be extended based on certain criteria).

38. Question and Answer
Does the QAS’s participation in a QMS audit conducted by the customer satisfy the DCMA policy requirement for system audits?
Yes No

39. QMS Audit Surveillance
Formal notification to supplier
Coordination with other Government parties

40. Question and Answer
Where in the Risk Profile and Plan Tool would the QAS identify the contractual requirement for a higher-level QMS?
Data Collection & Analysis tab
Performance Factors -11 tab
Risk Statement Generator tab
Facility Process List

41. Audit Frequency Initial (Baseline) audit within 2 years.
Subsequent audits based on Risk.
QMS elements risk assessed separately …..Frequency & Intensity of ongoing audits based on Risk.
Focus on elements of high and moderate Risk Likelihood.

42. System Audit Scope Full Partial
All factors/clauses addressed during a single event
Baseline QSA performed as Full audit
Full system audit if systemic issues identified via GCQA surveillance.
Partial
May audit one or more QMS elements
Based on risk when a QMS element is determined to be a potential risk cause
To facilitate planning when time is a constraint (audit of individual elements scheduled/performed based on risk)

43. Standard Used
Audits are conducted using the following common and established methods:
ISO 19011
ASQ/ANSI/ISO 19011S
SAE AS9101
Information Handling Services (IHS) Global website: ma_OC/haystack/index.cfm

44. Initiating an Onsite SYSTEM Audit
Lesson Topics:
Importance of QMS Audits
Audit Types
Five Principles of Auditing
System Audit Decision Factors
Initiating an Onsite System Audit
System Audit Activities
System Audit Checklists and Support Documents

45. Topic 5: Initiating an Onsite System Audit
Five Steps for Initiating an Audit
Appoint Team Leader
Define objectives, scope, and criteria
Determine feasibility of the audit
Select the Audit Team
Establish contact with the auditee

46. Step 1: Appoint Team Leader
Arranges for the audit.
Sets up the schedule.
Divides the work among the team.
Conducts meetings.
Collects feedback.
Produces the report.

47. Step 2: Define Objectives, Scope, and Criteria
Develop Audit Plan defining conduct of the audit.
From the contract quality requirements, determine audit:
Objectives
Scope
Full
Partial
Criteria, compliance to quality standard
Sample System Audit

48. Step 3: Determine Feasibility of the Audit
Consider the following questions:
Is there sufficient and appropriate information for planning the audit?
Is there adequate cooperation from the auditee?
Is there adequate time and resources to conduct the audit?

49. Step 4: Select the Audit Team
Once the Team Leader determines the audit is feasible, he/she solicits the First Line Supervisor (FLS) for assistance to select the audit team. QA personnel & technical specialists must possess the necessary competencies to support the QSA.

50. Step 5: Establish Contact with the Auditee
Initial contact should be made by the Team Leader.
Contact can be formal or informal.

51. System Audit Activities
Lesson Topics:
Importance of QMS Audits
Audit Types
Five Principles of Auditing
System Audit Decision Factors
Initiating an Onsite System Audit
System Audit Activities
System Audit Checklists and Support Documents

52. Topic 6: System Audit Activities
System audit activities include:
Review documents prior to onsite visit (Adequacy)
Prepare for the onsite visit
Conduct an onsite audit (Compliance)
Document audit findings
Conclude the audit

53. Document Review Prior to Onsite Visit
Prior to onsite visit, review the following:
Relevant QMS documents
Records
Previous audit reports

54. Preparing for the Onsite Audit
Prepare the Audit Plan
Assign work to the team
Prepare work documents

55. Conducting an Onsite Audit
Conduct the opening meeting
Communicate during the audit
Explain roles of guides and observers
Collect and verify information

56. Documenting Audit Findings
Generate audit findings
Prepare audit conclusions
Conduct the closing meeting

57. Concluding the Audit Complete the audit
Conduct audit follow-up, if required

58. SYSTEM Audit Checklists and Support Documents
Lesson Topics:
Importance of QMS Audits
Audit Types
Five Principles of Auditing
System Audit Decision Factors
Initiating an Onsite System Audit
System Audit Activities
System Audit Checklists and Support Documents

59. Tools and Additional Guidance (1 of 3)
AS9100 Rev C ISO Checklist
ISO-9001 Internal Audit Checklist PDF
AS9100C Electronic Checklists

60. Tools and Additional Guidance (2 of 3)
AS9100C Handcarry Checklists
Audit Checklist for the Evaluation of Calibration Systems
ISO Check List Rev 2
ISO modified Check List

61. Tools and Additional Guidance (3 of 3)
ISO Audit- Check List
ISO cl
MIL-I-45208A Check List
MIL-Q-9858A Check List

62. Summary Having completed this lesson, you should now be able to:
Relate the importance of QMS audits to DCMA GCQA surveillance.
Distinguish between the three types of QMS audits: First Party, Second Party, and Third Party.
Recognize the five principles of auditing.
Identify factors that affect the decision to conduct an audit.
Outline the steps for initiating an onsite audit.
Lesson 4: Safety Stock

63. Summary (cont.)
Having completed this lesson, you should now be able to:
Relate the importance of the document review to the onsite audit.
Outline the audit activities.
Describe how GCQA audit data is analyzed.
Find commonly used audit checklists and other audit support documents.
Lesson 4: Safety Stock

64. Questions

65. Review Question 1
What policy establishes the Government’s responsibility to plan and develop procedures for conducting Quality Assurance surveillance of its procurements?
FAR
FAR
FAR
FAR

66. Review Question 2
What is the maximum interval to conduct an initial QMS audit according to DCMA policy?
2 years
6 years, 3 months
3 years
Never

67. Review Question 3
What is one main reason driving the decision to audit a Quality Management System?
When performance history warrants.
When requested by the supplier.
When requested by DCMA.
When there is a change of leadership at the customer site.