Dec. 6, 2010 Gum-Ho Choe Accreditation of Software Testing Working Group, APLAC Technical Committee

· Ⅰ. Background Ⅱ. Activities on Software Testing Ⅲ. Supplementary Requirements for SW testing Ⅳ. Future Work

1/9 An Accreditation of Software Testing has emerged as a key points to improve product competiveness. * Necessary to improve the product and process such as smart phone, etc. Automotive Electrical/Electronic System (AEES) additionally requires the Security Management System (SMS)

To verify the conformity of product, process, and system (quality & security) To verify the conformity of product, process, and system (quality & security) Classification for Assessment for SW testing Certification Product Process System Conformity Assessment Conformance Testing ISO/IEC ISO/IEC (SPICE) CMMI ISO 9001 TL FDIS (Security)

Scope References KOLAS-SR-015 is applicable to accredit testing labs. embedded which conduct Testing related to security, embedded, bio-recognition, game, and mobile software, etc.. ISO/IEC 17025, ISO/IEC , -2, ISO/IEC 25001, ISO/IEC 25020, ISO/IEC 25030, ISO/IEC 25051, ISO/IEC For specific test for Software, IEC , ISO/IEC , -2, etc. Supplementary Requirements for SW Testing in KOLAS

Assessment Checklist (1)Assessment Checklist for Documentation - Product Manual : Basic information, Functionality, Reliability, Usability, Efficiency, Maintainability, Portability - Document for Users : Integrity, Accuracy for Explanation, Consistency, Understanding, Outline (2) Assessment Checklist for Quality Properties - Specified based on Quality model for external and internal quality, ISO/IEC : Functionality, Reliability, Usability, Efficiency, Maintainability, Portability (3) Assessment Checklist for Accreditation of Testing Lab. - Specified based on the Checklist of ISO/IEC 17025

Structure The structure of KOLAS-SR-015 follows ISO/IEC Structure of KOLAS-SR Technical Requirements 5.1 General 5.2 Personnel 5.3 Accommodation and environmental conditions 5.4 Test methods and method validation 5.5 Equipment 5.6 Measurement traceability 5.7 Sampling 5.8 Handling of test items 5.9 Test reports

5.1 General This document describes supplementary requirements for accreditation in software field This document is based on ISO/IEC

5.2 Personnel The technical manager shall have the following knowledge, education & training and the ability to monitor and order properly. ① testing activities including preparation for test reports ② knowledge of software ③ knowledge of software test and evaluation Testing labs shall have at least five testing personnel including one technical manager

5.3 Accommodation and Environmental conditions Testing labs shall have the policy and procedure to guarantee that the confidential information and customers’ ownership are protected. Transmission path shall be secure in order to protect the confidential test information. Testing Labs shall give the minimal right to access the testing room

5.4 Test Methods and method validation Test labs shall use test methods in international standards as follows - ISO/IEC ISO/IEC 25051:2006 Test labs shall perform the risk-based analysis through reviewing the test cases. Defect severity, status, and so on shall be managed, and defect handling procedure shall be established. For software field, test labs shall not apply estimation of uncertainty of measurement

5.5 Equipment Equipment may include the software and other machines used in evaluation and tests Test labs have manuals that describe how to operate all equipment. 5.6 Measurement traceability Measurement traceability shall not be applied in software tests * Test equipment --  Validation

5.7 Sampling Sampling in software tests refers to the selection of test cases The examples of sampling are the following: (1) the selection of the test cases to test the combinations of different conditions and variables (2) the selection of the re-execution test cases for regression tests

5.8 Handling of test items The configuration management of customer's submissions and test documents shall be performed 5.9 Test reports Test results shall be reported to a technical manager.

Current Status KOLAS has accredited 7 testing laboratories for SW Accredited Laboratories No.Lab. NameAccreditation Date Accreditation Scope (Sub Major Discipline) KT 005Korea Testing Certification June 10, 2008Embedded SW for Smartcard KT 009Korea Testing Laboratory September 30, 2010 SQuaRE, COTS, Common Criteria for IT Security Evaluation

Accredited Laboratories No.Lab. NameAccreditation DateAccreditation Scope (Sub Major Discipline) KT 167Telecommunications Technology Association May 14, 2009Common Criteria for IT Security Evaluation KT 327Korea System Assurance June 27, 2007Common Criteria for IT Security Evaluation KT 402Korea Security Evaluation Laboratory Co. Ltd. April 24, 2009Common Criteria for IT Security Evaluation KT 448ICT Korea Ltd.April 26, 2010ID Card; ISO/IEC , , ISO/IEC ~-4 KT 463KISANovember 16, 2010ISO/IEC (IT-Conformance testing for the biometric application programming interface

Focusing on Product Quality and Security Extending to Security Management System Extending to ISO FIDS for SMS in AEES * Ready to certify the SMS by CBs such as SGS-TUV, UL, etc Future Work Based on CC, ISO/IEC 9126, 25000, and IEC 61508, etc