The 7 Year Itch - Time To Commit Or Time To Move On? Shaun Lee Security Evaluations Manager, Global Product Security

Agenda Setting the Scene Discussions and Dilemmas Conclusions

Why we perform Evaluations Benefits to the Consumer Evaluations are standard and independent measures of assurance Fulfils procurement requirements Improved product security Benefits to Oracle Better products Confirms the security of the development process Contributes to culture of security Competitive advantage

The Drawbacks Costly Typically several hundred thousand $ to evaluate a complex product to EAL4 Time consuming Can be 2-3 years first time through, for EAL months once proficient Traditional security analysis, e.g.: Benign network assumptions permitted * Hacker style penetration testing can be avoided ‘Unrealistic’ configurations permitted * Lots of ‘academic’ proof documents to produce * Some schemes are tightening up on these

Oracle and Common Criteria Evaluating under 3 different national schemes Each national scheme has a perceived image / reputation Differences in policies, interpretation, oversight Different labs for different schemes

Agenda Setting the Scene Discussions and Dilemmas Conclusions

“Under attack: Common Criteria has loads of critics, but is it getting a bum rap?” (GCN, 13-Aug-2007) “Common Criteria is dead.” Colin Williams, SBL, UK (ICCC, Berlin, 2004)

CC and Criticism Vendors issue updates to certified products soon after a product has achieved certification. The CC does not credit vendors for investing in and using tools & methods for identifying insecure coding errors, like static source code scanners. The length of time and the cost of evaluations Certification does not mean vulnerability free Unrealistic Protection Profiles being written and evaluated. CC not prepared to acknowledge other official international standards Certifications are for benign environments. CC being updated without vendors input or buy in. Just a paperwork exercise

Do we still agree with this? “This is not to say that vendors should be allowed to "get away with anything". They should not. But evaluation processes should take into account differences among vendors, the need to repair flaws, and the likely impossibility of preventing them totally. They should also allow for process improvement -- a key ingredient in the quest for improved product quality that will yield better security. ”

Countering a Criticism The CC paradigm is not at fault, rather the national Schemes for allowing: Benign environments The exclusion of Buffer Overflows, SQL injections (Media driven customer concerns) The abuse of the CC’s flexible language Various national scheme interpretations Possible Reason: Trying to address the criticism on length of time and cost for an evaluation CC flexibility makes it difficult not to allow

Realistic Environments Realistic Environments require: Realistic Protection Profile environments not just Department of Defence environments Vendors performing more penetration testing as a result of the change to the Threat environment Evaluators to perform more realistic penetration testing Perception / Danger More realistic environment will cost more money and extend the length of time for an evaluation further. Real penetration testers better value for money.

Static source code scanners Currently Pinpoint insecure coding errors Tools are not endorsed by national schemes Should reduce vendors’ evaluation costs and time, but actually slows evaluation down because the activity is additional work Future CC Scope should take into account the use of static source code scanners Schemes should force evaluators to look at these tools CC officially approved tool ‘collections’

Agenda Setting the Scene Discussions and Dilemmas Conclusions

ICCC (2000) – ICCC (2007): 7 Years of Progress? Major revision of CC, now at V3.1 CCRA now at 23 participants Little take-up away from government arena Continued criticism

2000: Why Common Criteria? Mutual Recognition Evaluate once, recognised everywhere Saves repeating work Saves time and money We wanted To show our commitment to evaluations A world first To set a high standard for DBMS vendors

2000: What was our conclusion? Delta ITSEC to CC evaluations Solid foundation of ITSEC and TCSEC More work for vendor, less for evaluator CEM introduces criteria creep Other sources (National Scheme rules, Interpretations) introducing criteria creep Mutual recognition makes it all worthwhile!

What is the position in 2007? Criteria Creep less of a problem But raising the bar is done through the mechanism of PPs, apparently by people with insufficient product set knowledge, and with little consultation. We have the criticisms and other issues to concern us CCRA Alternative assurance schemes Management of the CC Mutual recognition still makes it all worthwhile – for now!

The Seven Year Itch – Answering the Question It’s not time to move on (abandon), but: the stakeholders need to work harder and together to get the commitment to move on (make progress). However failure to engage sufficiently with vendors and raising the bar too high too quickly especially above the level of CCRA may cause vendors to experience “negative cost recovery”, which could be an issue if the market for evaluated product remains restricted

For More Information Oracle Security Evaluations: General Oracle Security information:

A Q &

Do we still agree with this? IFIP-SEC ’91 Keynote: Criteria, Evaluation and the International Environment: where have we been, where are we going? Steve Lipner, Digital Equipment Corporation