1. Copyright atsec information security, IBM, 2007 How To Eat A Mammoth Experiences With the Evaluation of Complex Software Products Under the Common Criteria Gerald Krummeck (atsec), Bill Penny (IBM)

2. Copyright atsec information security, IBM, 2007 Agenda  Our Experience  Challenges from complex systems  Evaluations under the Common Criteria  The influence of complexity  Strategies in mastering complexity  Summary

3. Copyright atsec information security, IBM, 2007 atsec‘s Experience  Evaluation Labs in Germany, USA, Sweden  More than half of all OS evaluations performed world-wide z/OS (IBM Mainframes) z/VM (IBM Mainframes) Linux (SuSE, Red Hat, Oracle) AIX Cray PR/SM, AIX LPAR  Databases IBM DB2 Oracle DB  Tivoli System Management Products

4. Copyright atsec information security, IBM, 2007 IBM‘s experience  ISO 9001 Certified since 1993  WW development organization US, Canada, Germany, Australia, US Mexico, Russia, China  Historically Independent  Long History of IT Management Project Management System Management Process Control  Large Complex Systems HW, SW New Function and Service Models  Support Largest WW Business Requirements High availability, security, integrity

5. Copyright atsec information security, IBM, 2007 Challenges from complex systems Dimensions of complexity in evaluations  Size of the product  Size of the TOE (what part will be evaluated)  Amount of security functions Protection Profiles  Depth of evaluation (EAL)  Global distribution of development Multi-national Large number of organisational units

6. Copyright atsec information security, IBM, 2007 Evaluation under Common Criteria Security Target Functional Specification High-Level Design Low-Level Design Implemen- tation Tests Vulnerability Analysis Guidance documentation Development Process (Life Cycle) Delivery and Operation Configuration Management Product Processes Security Policy Model Design Correspondence

7. Copyright atsec information security, IBM, 2007 Influence of Complexity  Simple Systems „Isolated“ evaluation possible Without knowledge of its origin and heritage Emphasis on design, test, guidance, vulnerability analysis  Complex Systems Cannot be fully investigated Need to find additional ways to establish assurance/trustworthiness Establish trust in the development process

8. Copyright atsec information security, IBM, 2007 Example: IBM z/OS Version 1Release 8  Size Several Millions LOC (Assembler, PL/X, C, Java) Over 30 years development history Over 300 Manuals (120.000 pages) Over 630 Claims on security functions in the ST 10 development sites distributed globally  10 CM systems  Common Corporate Standards and Processes Toute la Gaule est occupée… Toute?

9. Copyright atsec information security, IBM, 2007 Interim Result  You cannot look at everything  But you don‘t need to Security functions can be located quite accurately and can be tested thoroughly Requires sufficient experience and product know-how of the evaluators  Development processes become very important  Build trust in the developer to comply with his duties for every piece that has not been scrutinized by the evaluators  Again: Evaluators need experience and product know-how: It is an illusion to assume that everybody can perform a good evaluation just by applying the CC methodology (not everybody can eat the mammoth without choking on it) Customers need to identify the right laboratory for them with evaluators skilled in their type of product

10. Copyright atsec information security, IBM, 2007 Strategies to master complexity  Not everything at once  How to eat the mammoth  Assistance  Site Certification

11. Copyright atsec information security, IBM, 2007 Not everything at once  Start modest Focus on core functionality Start with lower assurance level (EAL2 oder EAL3) Pro: Get your first certificate in due time Con: lower assurance level than competition  Example Linux: Start with EAL2, restrictive configuration Now EAL4, CAPP/LSPP, almost all packages included In between: write low-level design, add audit functions

12. Copyright atsec information security, IBM, 2007 Example z/OS  MVS: Orange Book B1 (in the mist of times…)  V1R6 – 2005 EAL3, CAPP+LSPP (multilevel security) Core functions: RACF, BCP, JES2, CS390, …  V1R7 – 2006 EAL4 Additional security functions  V1R8 – 2007 Major expansion of security functionality  V1R9 …

13. Copyright atsec information security, IBM, 2007 How to eat a Mammoth?  Bite by bite, of course!  Don‘t become intimidated by the size  Don‘t try to swallow it in one piece, either  Important factors: Experience Confidence Perseverance

14. Copyright atsec information security, IBM, 2007 Assistance  2 Teams from evaluation lab  Evaluators Working on-site with developers is beneficial Additional testers with product know-how  Consultants Help developer to gather evidence, prepare required documents Do not influence product itself or developer‘s decisions  Experienced certifiers help, too

15. Copyright atsec information security, IBM, 2007 Developer committment  Multi-year committment  Strong project management to coordinate all participating organizations  Strong technical leadership  „Divide and Conquer“ Strong leaders at distributed locations Educate, track, report Focus by area (ST, CM,HLD, Test)  Communicate with Evaluation Team Open, early and frequent discussions

16. Copyright atsec information security, IBM, 2007 Site Certification  Reduce complexity of the evaluation by reference to certification of sites  Idea Certify development process for one site Re-use certificate in all applicable evaluations  BSI tasked with development of site certification methodology  Since 2005 development and test of certification process  2006 first pilot certification  Acceptance in CC community  Still more experience needed.

17. Copyright atsec information security, IBM, 2007 Conclusion  Evaluation of complex products fits well in CC scheme  Medium to long term strategy (and committment!) Start modest Increase assurance level and functionality  Processes must fit  Find the right partner with experience and product know-how ITSEF and certification body

18. Copyright atsec information security, IBM, 2007 Questions, Comments