Registration Management Committee (RMC) Other Party (OP) Assessor AS Oversight Training January 24, 2008 Mesa, Arizona Body Of Knowledge Aerospace Standards Will Tate Vought Aircraft

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Body of Knowledge Purpose of the training module:Purpose of the training module: –Training on the criteria and processes that OP assessors will be implementing when performing oversight on behalf of the AAQG/RMC Objective:Objective: –Provide OP assessors with knowledge and tools to perform effective oversight of ABs, CBs, AABs and TPABs. Accreditation Bodies, Certification Bodies, assessorAuthentication Bodies, Training Provider Authentication Bodies

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor OPMT Assessors will assess: Accreditation BodiesAccreditation Bodies –Office assessment –Witness assessment Certification/Registration BodiesCertification/Registration Bodies –Office assessment –Witness assessment assessorAuthentication BodiesassessorAuthentication Bodies Training Provider Approval BodiesTraining Provider Approval Bodies

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Assessment can be conducted As part of a Joint AssessmentAs part of a Joint Assessment –CB office assessment –AB and CB Witness assessment Or aloneOr alone –AB office assessment –AAB office assessment –TPAB office assessment

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor AS & AS9014AS & AS9014 AQMS Standards AS9100/9110/9120AQMS Standards AS9100/9110/9120 AS9104-2AS AS9104-3AS ISO 19011ISO ISO 17011ISO ISO Guide 62 and IAF Guidance to ISO Guide 62ISO Guide 62 and IAF Guidance to ISO Guide 62 ISO 17021(deadline for implementation is 9/08)ISO 17021(deadline for implementation is 9/08) ISO 17024ISO Documents comprising BOK

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Additional relevant documents Official ISO 9001 InterpretationsOfficial ISO 9001 Interpretations ISO 9001 auditing Practices Group papersISO 9001 auditing Practices Group papers Accreditation auditing Practices Group papersAccreditation auditing Practices Group papers AB specific documents,AB specific documents, IAQG Resolutions logIAQG Resolutions log

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Where to obtain the documents All AS documents and ISO standards are copyrightedAll AS documents and ISO standards are copyrighted AS documents can be purchased from documents can be purchased from ISO Standards can be purchased from several places, such as the ASQ website, ANSI webstore or Standards can be purchased from several places, such as the ASQ website, ANSI webstore or The IAF Guidance documents can be downloaded for free from IAF Guidance documents can be downloaded for free from OP Assessors MUST bring the relevant documents to the assessmentOP Assessors MUST bring the relevant documents to the assessment

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) AssessorRemember It is the OEMs responsibility to gain access to these standards; standards are not provided by the AB or CB.It is the OEMs responsibility to gain access to these standards; standards are not provided by the AB or CB. OP Assessors must read and familiarize themselves with the applicable parts of the Standards prior to the oversight assessment.OP Assessors must read and familiarize themselves with the applicable parts of the Standards prior to the oversight assessment.

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Links to the available documents. IAF Guidance documentsIAF Guidance documentsIAF Guidance documentsIAF Guidance documents ISO auditing Practices GroupISO auditing Practices GroupISO auditing Practices GroupISO auditing Practices Group Accreditation auditing Practices GroupAccreditation auditing Practices GroupAccreditation auditing Practices GroupAccreditation auditing Practices Group IAQG Resolutions logIAQG Resolutions logIAQG Resolutions logIAQG Resolutions log ANAB Accreditation Manual, MA 6000,ANAB Accreditation Manual, MA 6000,ANAB Accreditation Manual, MA 6000ANAB Accreditation Manual, MA 6000

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) AssessorApplicability

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB assessment Office assessmentOffice assessment –A critical aspect is the quoting process, in terms of minimum number of audit days –OPMT assessors must understand AS9104 and Annex 2 of the IAF Guidance to ISO/IEC Guide 62 and the auditor-days estimation process –Justification is needed for departures from the guidelines.

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Estimation process

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Estimation process

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Additional time per AS9104

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Upgrade assessment

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Another critical issue Multi-site certificationsMulti-site certifications AS9104, paragraph 8.2.3AS9104, paragraph IAQG Resolution # 36IAQG Resolution # 36 Available at OASIS Website Guidance

Registration Management Committee (RMC) Other Party (OP) Assessor AS Oversight Training January 24, 2008 Mesa, Arizona Assessment Performed Without AB No Joint Team Assessment

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Reminder: Rules and Responsibilities of Assessors ISO 19011:2002 Section 4ISO 19011:2002 Section 4 –Ethical Conduct - the foundation of professionalism; –Fair presentation - the obligation to report truthfully and accurately; –Due professional care - the application of diligence and judgement in auditing; –Independence - the basis for impartiality of the assessment and objectivity of the assessment conclusions; –Evidence based approach - the rational method for reaching reliable and reproducible assessment conclusions in systematic assessment process.

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Accreditation Body Office Assessments / Witness assessment Conducted by a few OP AssessorsConducted by a few OP Assessors Review presentation attached prior to conducting an AB assessmentReview presentation attached prior to conducting an AB assessment Most AB office assessments are conducted by a team that includes one OP assessor whom has assessed an AB previouslyMost AB office assessments are conducted by a team that includes one OP assessor whom has assessed an AB previously

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Joint Team Assessments of Certification Bodies

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment & Witnessed assessment Scheduling These assessments may be performed alone (by an OP Assessor) if a joint assessment can not be scheduledThese assessments may be performed alone (by an OP Assessor) if a joint assessment can not be scheduled If performed alone; the OP assessor is responsible forIf performed alone; the OP assessor is responsible for –contacting the CB –Arranging an opening meeting –Planning the assessment (if office) –Writing NCRs –Handling corrective action reviews –All processes apply; including not interfering in the witnessed assessment, etc.

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment & Witnessed assessment Scheduling Note: other AB processes may vary AB confirmation letter sent to OEM primary contactAB confirmation letter sent to OEM primary contact OEM notify AB of assigned assessorOEM notify AB of assigned assessor –AB to update the Oversight Spreadsheet with OEM assessor information (for OEM & AB use) OEM primary contact to notify their assessorOEM primary contact to notify their assessor –Assessor shall already be trained and have access to all applicable requirement documents (to be brought to assessments) AB notifies their assessment team with the OEM assessor contact infoAB notifies their assessment team with the OEM assessor contact info AB team lead will contact the OEM assessor directlyAB team lead will contact the OEM assessor directly –May not be right away; may be closer to the assessment for open meeting planning.

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment Preparation Documents needed to perform assessment: Requirement DocumentsRequirement Documents CB Procedures related to AS programCB Procedures related to AS program AS Appendix F (CB Office Assessment Checklist)AS Appendix F (CB Office Assessment Checklist) AS Appendix K (Oversight Assessment Summary Report)AS Appendix K (Oversight Assessment Summary Report) assessment Planassessment Plan Document Review (if applicable)Document Review (if applicable)

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment: Document Review AB may perform document reviewAB may perform document review –Depending on the type of office assessment OP assessor to review document review report to;OP assessor to review document review report to; –Gain knowledge/understanding of documentation and any gaps (i.e. AB assessment trails) ISO (7.6)

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment Planning CB documents are received by the AB (from the CB) approx. 4 weeks prior to the assessmentCB documents are received by the AB (from the CB) approx. 4 weeks prior to the assessment AB sends the info to assigned OEM assessor (NOT including standards / requirement documents)AB sends the info to assigned OEM assessor (NOT including standards / requirement documents) –If the OEM assessor has yet to be assigned; AB sends the info to the OEM main contact Information may include:Information may include: –Logistic info (hotel, directions) –Assessment Plan - ISO (6.4) –CB documented processes for assessment –Other documentation as required by the AB

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment Planning Joint Team Assessment Prep (by OEM assessor) ISO (6.4) Review AB assessment plan to gain an understanding of what will be assessed and to provide feedback to the AB team leadReview AB assessment plan to gain an understanding of what will be assessed and to provide feedback to the AB team lead Review all documentation to gain a knowledge and an understanding of the CBs processes.Review all documentation to gain a knowledge and an understanding of the CBs processes. Review appropriate parts of the standards to re-fresh your memory.Review appropriate parts of the standards to re-fresh your memory. Ask any questions to the AB team lead. ANAB team lead to schedule a pre-assessment meeting with OEM assessor to meet and discuss plan and remind OEM assessor of basic processANAB team lead to schedule a pre-assessment meeting with OEM assessor to meet and discuss plan and remind OEM assessor of basic process

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment ~ OP Assessor Role OP Assessor serves as team memberOP Assessor serves as team member Roles & Responsibilities covered in previous presentationRoles & Responsibilities covered in previous presentation AS (9)

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment Conduct Conduct opening meetingConduct opening meeting AB Team LeadAB Team Lead –Confirm requirement documentsrequirement documents assessment plan including scope and objectives - ISO (6)assessment plan including scope and objectives - ISO (6) Summarize how activities will be undertakenSummarize how activities will be undertaken ISO ( 7.7)

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment Conduct Gather objective evidence that the CB is competent and conforms to the relevant standard(s) and other requirements for accreditation.Gather objective evidence that the CB is competent and conforms to the relevant standard(s) and other requirements for accreditation. ISO (7.7.2) Focus on AS specific elements as they relate to:Focus on AS specific elements as they relate to: –Impartiality - ISO (5.2) –Management - ISO (5, 6, 7) –Inclusion of AS in internal assessment & management review- ISO 17021, AS9104 –Review AS complaints - ISO 17021, AS9104 –Effective closure of previously issued aerospace related findings –File Review

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment Conduct Main activity by OP Assessor – File ReviewMain activity by OP Assessor – File Review –Registration, surveillance, re-certification process - ISO (9) –Competency (qualifications) of assessor- AS9104 (7, 8.1), AS9104-3, ISO (7) –Citation of findings (by CB) – AS9104 (8.3), ISO (9) –Appropriate resolution of findings – AS9104 (8.3), ISO (9) –Reporting – AS9104 (8.4), ISO (9) –Certification decisions (including competency of reviewers) – AS9104 (8.6), ISO (7, 9) –Oasis – AS9104 –Multi-site – AS9104 (8.2.3), IAF Guidance to Guide 62 (Annex 3) –Competency of staff – AS9104, ISO (7)

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment Conduct Final steps:Final steps: –Identify Findings and notify the AB team lead - AS (8.6.2) If AB lead is not in agreement; notify the AB & RMC via AS (Appendix G)If AB lead is not in agreement; notify the AB & RMC via AS (Appendix G) –AB (with OP assessor) to establish conclusions & prepare NCRs - ISO (7.8) –AB to conduct closing meeting - ISO (7.8)

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Office Assessment : Post Activities AB to generate assessment report – ISO (7.8)AB to generate assessment report – ISO (7.8) OP Assessor to complete AS (Appendix F & K) and provide to the AB and OEM main contactOP Assessor to complete AS (Appendix F & K) and provide to the AB and OEM main contact AB to manage the NCR / corrective action process with CB – ISO (7.8)AB to manage the NCR / corrective action process with CB – ISO (7.8) AB ensures effective cause, correction, corrective actionAB ensures effective cause, correction, corrective action OEM may request a copy of ABs assessment reportOEM may request a copy of ABs assessment report –If requested AB sends the information to the OEM main contact

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment Preparation Documents needed to perform the Witnessed assessmentDocuments needed to perform the Witnessed assessment Requirement DocumentsRequirement Documents CB Procedures related to AS assessmentCB Procedures related to AS assessment AS Appendix G (CB Witnessed assessment Checklist)AS Appendix G (CB Witnessed assessment Checklist) AS Appendix K (Oversight Assessment Summary Report)AS Appendix K (Oversight Assessment Summary Report) CB assessment PlanCB assessment Plan CB document review (if applicable)CB document review (if applicable)

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment: Document Review CB may perform document reviewCB may perform document review –Depending on the type of assessment OP assessor to review document review report to;OP assessor to review document review report to; –Gain knowledge/understanding of documentation and any gaps

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment Planning CB documents are received by the AB (from the CB) approx. 4 weeks prior to the witnessed assessmentCB documents are received by the AB (from the CB) approx. 4 weeks prior to the witnessed assessment AB sends the info to assigned OEM assessor (NOT including standards / requirement documents)AB sends the info to assigned OEM assessor (NOT including standards / requirement documents) –If the OEM assessor has yet to be assigned; AB sends the info to the OEM main contact Information may include:Information may include: –Logistic info (hotel, airport) –CB assessment Plan - ISO (6.4) –CB assessment teams credentials (e.g. resume) –CB documented processes for assessment –Other documentation as required by the AB

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment Planning Joint Assessment Prep (by OEM assessor) ISO (6.4) Allow approx. 2 hours AFTER the CBs end time to allow for the AB closing meetingAllow approx. 2 hours AFTER the CBs end time to allow for the AB closing meeting Review CB assessment plan - ISO (6.4) to gain an understanding of what will be assessedReview CB assessment plan - ISO (6.4) to gain an understanding of what will be assessed Review all documentation to gain a knowledge and an understanding of the CBs processes.Review all documentation to gain a knowledge and an understanding of the CBs processes. Review CB assessment teams credentials to confirm acceptabilityReview CB assessment teams credentials to confirm acceptability Review organizations information to understand scope, etc.Review organizations information to understand scope, etc. Review OASIS (if organization is certified)Review OASIS (if organization is certified) Review appropriate parts of the standards to re-fresh your memory.Review appropriate parts of the standards to re-fresh your memory. Review existing initial audit/surveillance Client audit reports (AS9101).Review existing initial audit/surveillance Client audit reports (AS9101). Ask any questions to the AB team lead.Ask any questions to the AB team lead. ANAB team lead to schedule a pre-assessment meeting with OEM assessor to meet and discuss plan and remind OEM assessor of basic processANAB team lead to schedule a pre-assessment meeting with OEM assessor to meet and discuss plan and remind OEM assessor of basic process

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment – OP Assessor Role OP Assessor serves as an observer (as does AB)OP Assessor serves as an observer (as does AB) Roles & Responsibilities covered in previous presentation; however,Roles & Responsibilities covered in previous presentation; however, During witnessed assessmentDuring witnessed assessment –Silently observe –Do not influence outcome of assessment! –Do not talk –Do not make body motions to show approval or disapproval –Do not answer questions of organization (or CB) –Do not conduct a OEM 2 nd Party Audit AS (8.2, 9)

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment Conduct Conduct opening meeting with CBConduct opening meeting with CB AB Team LeadAB Team Lead PRIOR to CBs opening meeting with organizationPRIOR to CBs opening meeting with organization –Confirm requirement documentsrequirement documents scope and objectivesscope and objectives Summarize AB team involvementSummarize AB team involvement Timing of closing meetingTiming of closing meeting ISO 17011, section 7.7

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment Conduct Gather objective evidence that the CB is competent and conforms to the relevant standard(s) and other requirements for accreditation.Gather objective evidence that the CB is competent and conforms to the relevant standard(s) and other requirements for accreditation. ISO (7.7.2) Focus on:Focus on: –Opening and closing meeting (and team caucuses) – ISO (9) –assessment days and assessment efficiency, effectiveness – AS9104 (8.2), ISO (6.5), IAF Guidance to Guide 62, Annex 2 –CBs assessment teams competency to perform assessment - AS9104 (7, 8), AS , ISO (7) Including knowledge of standards, industry, regulatory requirements and interpretationsIncluding knowledge of standards, industry, regulatory requirements and interpretations Including CBs process for assigning assessment teamIncluding CBs process for assigning assessment team –Certificate accuracy – ISO (8) –assessment depth and trails – AS9104 (8), ISO (6.5), IAF Guidance to Guide 62

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment Conduct Final steps:Final steps: –Identify Findings and notify the AB team lead - AS (8.2.5) If AB lead is not in agreement; notify the AB & RMC - AS9104-2If AB lead is not in agreement; notify the AB & RMC - AS –AB (with OP assessor) to establish conclusions & prepare NCRs - ISO (7.8) –AB to conduct closing meeting - ISO (7.8) AFTER CBs closing meeting with organizationAFTER CBs closing meeting with organization

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor CB Witnessed assessment : Post Activities AB to generate assessment report – ISO (7.8)AB to generate assessment report – ISO (7.8) OP Assessor to complete AS (appendix K) and provide to the AB and OEM main contactOP Assessor to complete AS (appendix K) and provide to the AB and OEM main contact AB to manage the NCR / corrective action process with CB – ISO (7.8)AB to manage the NCR / corrective action process with CB – ISO (7.8) AB ensures effective cause, correction, corrective actionAB ensures effective cause, correction, corrective action OEM may request a copy of ABs assessment reportOEM may request a copy of ABs assessment report –If requested AB sends the information to the OEM main contact

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Lessons Learned over the years on joint Team Assessments

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Lessons Learned Scheduling Joint Activity OEM to identify assessor to conduct assessments ASAP (and notify AB)OEM to identify assessor to conduct assessments ASAP (and notify AB) OEM to notify AB of any assessor changes ASAPOEM to notify AB of any assessor changes ASAP –ANAB to notify all parties of assessor change –Make sure replacement is qualified.

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Lessons Learned Preparation for Joint Activity OP assessor mustOP assessor must –Be qualified/trained –Have knowledge of the standards –Have a copy of the standards –Have post-activity forms to complete All to be provided by OEM. AB nor CB can provide this information !

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Lessons Learned CB Office Assessment Conduct OP Assessor active participant in assessment – assist where AB needs assistance WITHOUT a lot of AB assistance OP Assessor active participant in assessment – assist where AB needs assistance WITHOUT a lot of AB assistance Understanding that for many CBs, the aerospace component of the CBs business is a [very] small fraction of their overall business.Understanding that for many CBs, the aerospace component of the CBs business is a [very] small fraction of their overall business. –Sensitivity to the ENTIRE scope of the assessment. Not conducting 2 nd party assessmentNot conducting 2 nd party assessment Caution with conflict of interest and confidentiality.Caution with conflict of interest and confidentiality. –Acquiring information for OEM procurement process is not acceptable.

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Lessons Learned CB Witnessed assessment Conduct Organization may ask questions about the oversight; OP assessor may have to explain the oversight process (during opening meeting, breaks, etc)Organization may ask questions about the oversight; OP assessor may have to explain the oversight process (during opening meeting, breaks, etc) Asking simple questions and/or making simple comments during witnessed assessment is considered interfering and is NOT allowed.Asking simple questions and/or making simple comments during witnessed assessment is considered interfering and is NOT allowed. Talking to the AB assessor team is distracting during the assessment.Talking to the AB assessor team is distracting during the assessment. Not conducting 2 nd party assessmentNot conducting 2 nd party assessment

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Lessons Learned Joint Assessment Post-Activity Document findings against REAL requirements.Document findings against REAL requirements. –What an OEM wants (or their philosophy) v. requirements and wants are not NCRs Categorize findings appropriately.Categorize findings appropriately. OEM assessors to complete AS appendices in a timely manner.OEM assessors to complete AS appendices in a timely manner. –Provide feedback on the AB process too!! If a decision is appealed by CB; be available to participate in appeal hearing.If a decision is appealed by CB; be available to participate in appeal hearing.

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) AssessorRemember

Registration Management Committee (RMC) Workshop – January 24, 2008 – Phoenix, AZ Other Party (OP) Assessor Burning questions