03/06/18 1 Software Security for Open- Source Systems Crispin Cowan, Ph.D. Chief Scientist, Immunix Inc.

03/06/18 2 Gratuitous Plug: USENIX Security Panel on TCPA/Palladium –Lucky Green: radical hippy :-) –Bill Arbaugh, U.MD: dissertation on secure bootstrap –David Safford, IBM: group released a GPL’d Linux driver for IBM’s TCPA hardware –Peter Biddle, Microsoft

03/06/18 3 Sharing Source and Power Source code is power –To defend and attack Sharing source code shares power –With both attackers and defenders Opening source and doing nothing else just degrades security Conversely, opening source enables defenders and others to enhance security to the degree that they care to

03/06/18 4 Secure Software Reliable software does what it is supposed to do Secure software does what it is supposed to do, and nothing else –It’s those surprising “something else”’s that get you So to be secure, only run perfect software :-) Or, do something to mitigate the “something else”’s

03/06/18 5 Doing Something Code Auditing: static or dynamic analysis of programs to detect flaws, e.g. ITS4 and friends Vulnerability Mitigation: compiled in defense that block vulnerability exploitation at run-time, e.g. StackGuard and friends Behavior Management: OS features to control the behavior of programs Classic: mandatory access controls Behavior blockers: block known pathologies

03/06/18 6 Software Auditing Audit your code to try to eliminate vulnerabilities Problems –Tedious & error prone –Requires expertise to be effective –Defender needs to find all the vulnerabilities while attacker need only find one Solutions –Encourage auditing despite the challenge –Tools to make bug- finding easier

03/06/18 7 Sardonix Security Auditing Portal Vision Repository of auditing resources & tools Leverage the open source “karma whore” effect by providing a mechanism to get famous for your security auditing skilz –Rate auditors according to their auditing success –Rate programs according to who has audited them Reality Lots of talk, little action Conjecture: finding one bug and making a lot of noise about it on Bugtraq is easier & more rewarding than doing the hard work of finding many bugs

03/06/18 8 Tools

03/06/18 9 Vulnerability Mitigation StackGuard: compiled-in protection against “stack smashing” buffer overflows –ProPolice: from IBM Research Japan Adds variable sorting FormatGuard: compiled-in protection against printf format string vulnerabilities

03/06/18 10 Behavior Management Kernel or OS enforcement on the behavior of applications Classically: access controls –Many ways to model access controls Behavior blocking: –Characterize “bad” behavior –Stop that behavior when you see it

03/06/18 11 LSM: Linux Security Modules Too many access control models for Linus to just choose one Instead: build a module interface to enable pluggable access control modules Before LSM: –Each access control group busy forward porting –Advanced security hard for users to get After LSM: –Shared infrastructure maintained by collective –Users can choose one and plug it into a standard kernel

03/06/18 12 Open Source Access Control Modules Type enforcement, DTE: “new” way to model access control, 1986 SELinux: provides TE and RBAC Immunix SubDomain: TE-style MAC specialized for server appliances LIDS: another popular open source access control system, unclear model

03/06/18 13 Open Source Behavior Blockers Openwall: –Non-executable stack segment –Restrictions on symlinks and hard links –Restrictions on file descriptors across fork/exec libsafe: libc with plausibility checks on arguments to prevent stack smashing attacks

03/06/18 14 Open Source Behavior Blockers RaceGuard: kernel detects & blocks non- atomic temp file creation Systrace: hybrid system controlling access to system calls –Classical file access control by controlling arguments to open syscall –Behavior blocking by not permitting e.g. mount system call

03/06/18 15 Closed Source Microsoft /gs –Very similar to StackGuard –Dispute about whether it was “independent innovation” Okena, Entercept: use very similar models to the Systrace system, controlling system call access

03/06/18 16 But how well does this stuff work? Measurement makes it science

03/06/18 17 Assessing the Assurance of Retro-Fit Security Commodity systems (UNIX, Linux, Windows) are all highly vulnerable –Have to retrofit them to enhance security But there are lots of retrofit solutions –Are any of them effective? –Which one is best? –For my situation?

03/06/18 18 What New Capability Would Result? Instead of “How much security is enough for this purpose?” We get “Among the systems I can actually deploy, which is most secure?” –Tech transfer experience: customer says “We are only considering solutions on FooOS and BarOS” Relative figure of merit helps customer make informed, realistic choice

03/06/18 19 Why Now? Old Stove pipe systems, made to order Orange book/Common Criteria lets customer order a custom system that is “this” secure The question is “Is this secure enough?” New Reliance on COTS Customer must choose among an available/viable array of COTS systems –And possibly an array of security enhancements The question is “Which is best?”

03/06/18 20 State of the Art Common Criteria High barrier to entry: –At least $1M for initial assessment Hard to interpret result –Only a particular configuration is certified, and it may not relate to real deployments 3-bit answer: EAL0-7 –Several of which are meaningless (0-2 useless) –Others are infeasible (6 & 7 are too hard for most systems) –Really 2-bit answer: none, 3, 4, 5 ICSA Lower barrier to entry –But still high enough that most retrofit mechanisms are not certified Hard to interpret result –ICSA certifies that whatever claims the vendor makes are true –Not whether those claims are meaningful 1-bit answer: certified/not

03/06/18 21 Proposed Benchmark: Relative Vulnerability Compare a “base” system against a system protected with retrofits –E.g. Red Hat enhaced Immunix, SELinux, etc. –Windows enhanced with Entercept, Okena, etc. Count the number of known vulnerabilities stopped by the technology “Relative Invulnerability”: % of vulnerabilities stopped

03/06/18 22 Can You Test Security? Traditionally: no –Trying to test the negative proposition that “this software won’t do anything funny under arbitrary input”, I.e. no surprising “something else’s” Relative Vulnerability transforms this into a positive proposition: –Candidate security enhancing software stops at least foo% of unanticipated vulnerabilities over time

03/06/18 23 Immunix Relative Vulnerability Immunix OS 7.0: –Based on Red Hat 7.0 –Compare Immunix vulnerability to Red Hat’s Errata page (plus a few they don’t talk about :-) Data analyzed so far: 10/2/ /31/2002 –135 vulnerabilities total

03/06/18 24 Vulnerability Categories Local/remote: whether the attacker can attack from the network, or has to have a login shell first Impact: using classic integrity/privacy/availability Penetration: raise privilege, or obtain a shell from the network Disclosure: reveal information that should not be revealed DoS: degrade or destroy service

03/06/18 25 Immunix Relative Vulnerability

03/06/18 26 Version Churn Previous data compared Red Hat 7.0 to Immunix 7.0 –2 year old technology –Notably did not include SubDomain Defcon 2002 system: Immunix 7+ –Mutant love child of Red Hat 7.0 and 7.3 –No valid basis for RV comparison Next up: Red Hat 7.3 vs. Immunix 7.3

03/06/18 27 Impact Lower barriers to entry –Anyone can play -> more systems certified Real-valued result –Instead of boolean certified/not-certified Easy to interpret –Can partially or totally order systems

03/06/18 28 RV Database Built a PostgreSQL database of RV findings Allows relational queries to answer statistical questions

03/06/18 29 RV Summary

03/06/18 30 Issues Does not measure vulnerabilities introduced by the enhancing technology –Actually happened to Sun/Cobalt when they applied StackGuard poorly Counting vulnerabilities: –When l33t d00d reports “th1s proggie has zilli0ns of bugs” and supplies a patch, is that one vulnerability, or many?

03/06/18 31 Issues Dependence on exploits –Many vulnerabilities are revealed without exploits –Should the RV test lab create exploits? –Should the RV test lab fix broken exploits? Exploit success criteria –Depends on the test model –Defcon “capture the flag” would not regard Slammer as a successful exploit because payload was not very malicious

03/06/18 32 Issues What is the goal? –Access control can keep an attacker from exploiting a bad web app to control the machine –But cannot prevent the attacker from exploiting a bad app to corrupt that app’s data Idea: RV for applications –Consider the RV of an application vs. that application defended by an enhancement –E.g. web site defended by in-line intrusion prevention

03/06/18 33 Technology Transfer ICSA Labs –traditionally certify security products (firewalls, AV, IDS, etc.) –no history of certifying secure operating systems –interested in RV for evaluating OS security ICSA issues –ICSA needs a pass/fail criteria –ICSA will not create exploits

03/06/18 34 Questions? Open source survey: IEEE Security&Privacy Magazine, February 2003 – urvey.pdf LSM: RV: so far unpublished