SwA Co-Chair and Task Lead Strategy Session Agenda Technology, Tools and Product Evaluation Working Group Status Briefing Co-Chair(s) Michael Kass (NIST), Larry Wagoner (NSA) March 31, 2008

Common Weakness Enumeration (CWE) Common Attack Pattern Enumeration (CAPEC) Software Assurance Metrics and Tool Evaluation (SAMATE) Project OMG Software Assurance Framework and Tool Test Generation TT&PE Working Group Projects

CWE Draft 8 (30 Jan 08) Added 22 CWEs

Formalizing a Schema for Weaknesses Identifying Information CWE ID Name Describing Information Description Extended Description Alternate Terms Demonstrative Examples Observed Examples Context Notes Source Taxonomy References Whitebox Definition Blackbox Definition Formal Definition Scoping & Delimiting Information Type Functional Area Likelihood of Exploit Common Consequences Enabling Factors for Exploitation Common Methods of Exploitation Applicable Platforms Time of Introduction Prescribing Information Potential Mitigations Enhancing Information Weakness Ordinality Causal Nature Affected Resource Related Attacks Detection Factors Node Relationships Research Gaps

Department of Homeland Security’s National Vulnerability Database (NVD) tags Vulnerabilities with CWEs NVD Now Maps to CWE! nvd.nist.gov

CAPEC Status

New CAPEC Status Attack Pattern multi-level abstraction tagging –Levels Meta Standard Detailed –All current authored patterns (101) as well as all potential patterns in the attack taxonomy have been tagged CAPEC description initial schema formalization –Targeted to support security test case identification –Updated schema complete –25 of the authored patterns have been fleshed-out to the new schema

The SAMATE Project

Testing the Tools SAMATE Reference Dataset (SRD) –Online repository of tool tests –Thousands of source code samples containing examples of CWE’s Discrete tests – developed by NIST, contributed by tool developers, academia and public Tests are based upon interpretation of a particular weakness definition (currently no formal white-box definitions) Tests are freely available at

Automated Test Case Generation (TCG) Funded by DHS Part of SAMATE effort to expand SRD to cover as many CWE’s as possible Based upon OMG MDA Technology (MOF, UML, XMI) –Uses formalized CWE definitions (SBVR) Contractual Formalization that is based on OMG standard, Semantics of Business Vocabulary and Rules (SBVR) and Technical Formalization that is based on OMG standard, Knowledge Discovery Metamodel (KDM) Formal CWE Definitions (SBVR/KDM) Tool Tests (code)Code Analysis Tool KDM

CWE Formalization White Box Definitions : Focus on the structure patterns of the inner components and their interactions (that determine certain observable behavior) –Provide “compliance points” that: Describe patterns of code (as they can be directly identified in code) Identify discernable properties of patterns of code Enable automation Enable direct step-by-step comparisons of the decision procedures implemented within tool

SAMATE and CWE Effectiveness Program Long-term goal : To auto-generate tool tests using formal CWE definitions in collaboration with MITRE’s CWE Effectiveness program –Provide tests “ad hoc” to tool developers –Developers run tests against their tool –Developers can publish test results

TCG: Where are we now? TCG Status: –Can generate tests for 3 CWE’s –26 CWE white-box definitions for “high priority” CWE’s are complete based upon their: –Long term, TGC will cover as many CWEs as possible With coding complexities

Other SAMATE Projects Ongoing work –Developing tests for web application scanners –Adding to existing tests for source code security analyzers –Performing tool effectiveness studies New areas –Testing binary analyzers –The static analyzer tool exposition (SATE) –Software transparency/pedigree information

NIST will be hosting SwA Forum in October 2008 Opportunity to showcase NIST’s work in SwA –NVD –SAMATE –SCADA –Trustworthy Systems Project –NVLAP (CC labs, Crypto Testing, Voting System Testing Laboratory Accredidation) –NIST Special Pubs (FIPS, SP 500 and 800 series) –Voting System Testing Project