Eric Fassett July 26, 2011

Background DoD prescribed a Single Sampling Plan (SSP) in DAR Supp. 3 Annex II in October 1975 that was replaced by the DoD Property Manual (4161) Double Sampling Plan (DSP) in December 1991 The FAR revisi0n of June 2007 required that contractors shall manage property consistent with voluntary consensus standards/industry leading practices and have a process to report self assessments Many contractors adopted metrics as replacements for traditional 4161 type assessments DCMA has questioned the same statistical validity of these self assessments against the 4161 provisions of “90% confidence of rejecting lots with 10% or more defects” DAU AAP 9/22/2010

Background (Cont.) Normally using a sampling plan, the Government's risk shall not exceed 10% (a 90% confidence level) excepting slight variations due to changes in population sizes.” DCMA Guidebook DL RISK. A measure of those instances where a sample will not be an accurate representation of the lot. For example, a sampling plan having a 90 percent confidence level means that a sample will not be an accurate representation of the lot 10 out of 100 times. DoD Manual M DLA Research Report “Statistical Sampling for Property Management” represents a 90% confidence of rejecting lots with 10% or more defects

Issues with 4161 DSP Methodology The sample sizes from the 4161 Double Sampling Plan (DSP) do not coincide with Table I of Mil Std 105E (e.g. Lot sizes 51-90, , and are the only matches) The Acceptance Quality Limit (AQL) from the Mil Std 105E for Double Sampling (Normal, Reduced, or Tightened) do not coincide with the 4161 DSP defect criteria. The 4161 DSP actual AQL runs between 1.5 and 2.5% The DSP is based on DLA Research Report “Statistical Sampling for Property Management” DLA-XX-P20258, August 1992 MIL-STD-105E is no longer supported by the Government

Issues with 4161 DSP Methodology (Cont.) ASTM has incorporated it into E2234 via the E11 committee on Quality and Statistics Met with E11 Vice-chairman Dean Neubauer who is co-author of “Acceptance Sampling in Quality Control” He stated “The sampling plan that you sent is not a military sampling plan I have ever seen. For example, the lot ranges you show are not what is given in the sample size code letter table for General Inspection Level II (the default level to use) in Mil-Std-105E.” Confidence level is not really the issue as much as using an accepted sampling plan

Key Concepts Confidence level - means that the sample will be an accurate representation of the lot Probability of acceptance (rejection) - percentage of inspection lots estimated to be of acceptable quality when subjected to a specific sampling routine Process average - the average percent of defective or average number of defects per hundred units of product submitted by the supplier for original inspection

OC Curve for 4161 DSP

DSP Simulation Sample Results – 9% Defect Rate

Sampling Plan Typical AQL levels are 10%, 6.5%, 4%, 2.5%, 1.5% We have proposed using the Mil-Std-105E criteria for doing Contractor Self Assessments and establishing an Acceptance Quality Limit (AQL) of 6.5% E2234 charts are statistically sound ASTM has incorporated Mil-Std-105E into E2234 via the E11 committee on Quality and Standards Sample rates are similar to what is being currently performed

Self Assessment Variables to Consider Organizational Level of Sample (Sector, Division, Site) Sampling Plan (Single, Double) Frequency (Yearly, Quarterly, Monthly) Each sample must stand on it’s own

Status Briefed DCMA/HQ on They met to evaluate our proposal and a technical follow up was held Recognized in most recent DCMA Standard Operating Procedure (SOP) Contractor’s Internal Assessment or Self Audit of their Property Management System

ASTM E2234 SSP – AQL 6.5% " Adapted, with permission, from E Standard Practice for Sampling a Stream of Product by Attributes Indexed by AQL, copyright ASTM International, 100 Barr Harbor Drive, West Conshohocken, PA "

ASTM E2234 DSP – AQL 6.5% " Adapted, with permission, from E Standard Practice for Sampling a Stream of Product by Attributes Indexed by AQL, copyright ASTM International, 100 Barr Harbor Drive, West Conshohocken, PA "