1. Dec. 6, 2010 Gum-Ho Choe Accreditation of Software Testing Working Group, APLAC Technical Committee

2. · Ⅰ. Background Ⅱ. Activities on Software Testing Ⅲ. Supplementary Requirements for SW testing Ⅳ. Future Work

3. 1/9 An Accreditation of Software Testing has emerged as a key points to improve product competiveness. * Necessary to improve the product and process such as smart phone, etc. Automotive Electrical/Electronic System (AEES) additionally requires the Security Management System (SMS)

4. To verify the conformity of product, process, and system (quality & security) To verify the conformity of product, process, and system (quality & security) Classification for Assessment for SW testing Certification Product Process System Conformity Assessment Conformance Testing ISO/IEC 9126 12119 ISO/IEC 15408 15504 (SPICE) CMMI ISO 9001 TL 9000 26262 FDIS (Security)

5. Scope References KOLAS-SR-015 is applicable to accredit testing labs. embedded which conduct Testing related to security, embedded, bio-recognition, game, and mobile software, etc.. ISO/IEC 17025, ISO/IEC 9126-1, -2, ISO/IEC 25001, ISO/IEC 25020, ISO/IEC 25030, ISO/IEC 25051, ISO/IEC 25062 For specific test for Software, IEC 61508-3, ISO/IEC 24709-1, -2, etc. Supplementary Requirements for SW Testing in KOLAS

6. Assessment Checklist (1)Assessment Checklist for Documentation - Product Manual : Basic information, Functionality, Reliability, Usability, Efficiency, Maintainability, Portability - Document for Users : Integrity, Accuracy for Explanation, Consistency, Understanding, Outline (2) Assessment Checklist for Quality Properties - Specified based on Quality model for external and internal quality, ISO/IEC 9126-1 : Functionality, Reliability, Usability, Efficiency, Maintainability, Portability (3) Assessment Checklist for Accreditation of Testing Lab. - Specified based on the Checklist of ISO/IEC 17025

7. Structure The structure of KOLAS-SR-015 follows ISO/IEC 17025 Structure of KOLAS-SR-015 5. Technical Requirements 5.1 General 5.2 Personnel 5.3 Accommodation and environmental conditions 5.4 Test methods and method validation 5.5 Equipment 5.6 Measurement traceability 5.7 Sampling 5.8 Handling of test items 5.9 Test reports

8. 5.1 General This document describes supplementary requirements for accreditation in software field This document is based on ISO/IEC 17025.

9. 5.2 Personnel The technical manager shall have the following knowledge, education & training and the ability to monitor and order properly. ① testing activities including preparation for test reports ② knowledge of software ③ knowledge of software test and evaluation Testing labs shall have at least five testing personnel including one technical manager

10. 5.3 Accommodation and Environmental conditions Testing labs shall have the policy and procedure to guarantee that the confidential information and customers’ ownership are protected. Transmission path shall be secure in order to protect the confidential test information. Testing Labs shall give the minimal right to access the testing room

11. 5.4 Test Methods and method validation Test labs shall use test methods in international standards as follows - ISO/IEC 9126-2 - ISO/IEC 25051:2006 Test labs shall perform the risk-based analysis through reviewing the test cases. Defect severity, status, and so on shall be managed, and defect handling procedure shall be established. For software field, test labs shall not apply estimation of uncertainty of measurement

12. 5.5 Equipment Equipment may include the software and other machines used in evaluation and tests Test labs have manuals that describe how to operate all equipment. 5.6 Measurement traceability Measurement traceability shall not be applied in software tests * Test equipment --  Validation

13. 5.7 Sampling Sampling in software tests refers to the selection of test cases The examples of sampling are the following: (1) the selection of the test cases to test the combinations of different conditions and variables (2) the selection of the re-execution test cases for regression tests

14. 5.8 Handling of test items The configuration management of customer's submissions and test documents shall be performed 5.9 Test reports Test results shall be reported to a technical manager.

15. Current Status KOLAS has accredited 7 testing laboratories for SW Accredited Laboratories No.Lab. NameAccreditation Date Accreditation Scope (Sub Major Discipline) KT 005Korea Testing Certification June 10, 2008Embedded SW for Smartcard KT 009Korea Testing Laboratory September 30, 2010 SQuaRE, COTS, Common Criteria for IT Security Evaluation

16. Accredited Laboratories No.Lab. NameAccreditation DateAccreditation Scope (Sub Major Discipline) KT 167Telecommunications Technology Association May 14, 2009Common Criteria for IT Security Evaluation KT 327Korea System Assurance June 27, 2007Common Criteria for IT Security Evaluation KT 402Korea Security Evaluation Laboratory Co. Ltd. April 24, 2009Common Criteria for IT Security Evaluation KT 448ICT Korea Ltd.April 26, 2010ID Card; ISO/IEC 7816-3, 10373-3, ISO/IEC14443-2~-4 KT 463KISANovember 16, 2010ISO/IEC 24709-1(IT-Conformance testing for the biometric application programming interface

17. Focusing on Product Quality and Security Extending to Security Management System Extending to ISO 26262 FIDS for SMS in AEES * Ready to certify the SMS by CBs such as SGS-TUV, UL, etc Future Work Based on CC, ISO/IEC 9126, 25000, and IEC 61508, etc