1. Intrusion Tolerant Distributed Object Systems 2002 OASIS Winter PI Meeting Hilton Head, SC March 12, 2002 Gregg Tally Gregg Tally Brent Whitmore Brent Whitmore

2. March 12, 2002 2 Agenda Overview Technical Challenges and Solutions Testing Plan Summary

3. March 12, 2002 3 Overview

4. March 12, 2002 4 Technical Objectives Provide intrusion tolerance for CORBA applications System level approach – Middleware Eliminate reliance on any single server – secure, reliable group communication directly between clients and replicated servers Detect Byzantine (arbitrary) faults in servers Support heterogeneity (diversity of implementation) – Boundary controllers (firewalls) Protocol inspection Prevent flooding attacks

5. March 12, 2002 5 Current Technical Approach Active replication of clients and servers with voting Integrate voter and secure, reliable multicast transport into open-source ORB Leverage prior work on fault tolerant CORBA; secure, reliable, authenticated multicast; total ordering; Byzantine fault detection – Augment original C-L to fit CORBA model, eject faulty replicas to restore confidentiality Protect client and server enclaves with limited application proxy firewall

6. March 12, 2002 6 Expected Achievements At least one implementation of an ORB on two or more heterogeneous platforms that tolerates Byzantine faults Integrated firewall support to protect COTS client and server hosts from multicast denial of service attacks Understand trade-off between performance and degrees of intrusion tolerance Apply Validation Framework to identify assumptions and residual risks

7. March 12, 2002 7 Challenges and Solutions

8. March 12, 2002 8 Heterogeneity Goal: – Permit heterogeneous client and server implementations to gain survivability through diversity of implementation – Includes heterogeneous hardware, operating systems – Adhere to the interoperability CORBA goals Approach: – Vote on unmarshalled data, not bitwise comparison of messages – Voter must be integrated into the ORB to unmarshall the data reliably - GIOP protocol doesn’t provide enough information for an external process to unmarshall the data

9. March 12, 2002 9 Heterogeneity Issue #1 - Voter Integration Non-portable integration of voter into the ORB. Requires modification to ORB source code. Solution: Causes engineering difficulties, but solvable. Some optimizations may be required to avoid duplicate unmarshalling.

10. March 12, 2002 10 Voter Integration

11. March 12, 2002 11 Heterogeneity Issue #2 - Voting on Inexact Values Must determine equivalence across inexact values (e.g., floating point). Difficult to find good equivalence set among values where equivalence is not transitive. Solution: VVM techniques from WSU permit flexible voting mechanisms with epsilon to permit inexact matches. Enhanced with additional techniques to deterministically find equivalence set without transitivity.

12. March 12, 2002 12 Inexact Voting Protocol needs deterministic behavior – Cannot modify values to compute results of the vote (e.g., mean / average not usable) – Varying representations on different hosts – Values appear equivalent on one host but not on another Extends “Voting in Middleware” work at Washington State University (St Louis) Uses equality within є distance (є-equivalence) Problem voting over multiple inexact values – Equivalence not transitive (a = b) and (b = c), but (a  c)

13. March 12, 2002 13 Enhancements to WSU Voter Incremental – Always votes as soon as a potential quorum develops Polymorphic – Relies on equality defined by the message type – Extends easily to new message types Handles multiple inexact values Handles lack of transitive equivalence differently – Incrementally develops a message comparison matrix – Uses matrix to find a subset of voters that all compare equal Search limited to comparisons with message just received

14. March 12, 2002 14 Remaining Voter Problem Є-equivalence reduces, but does not eliminate non- determinism – Some hosts may still consider messages to be equivalent when other hosts do not – “Fuzzyness at є” Opinion: Probably not a problem in practice – Configure for sufficiently large є for all platforms & hosted applications, but no larger Potential solutions (speculative) – Should be based upon application-declared needs, platform- declared capabilities – Introduce a super-є that must compare greater than є on all hosts – Negotiate either є or equivalence among replicas

15. March 12, 2002 15 Heterogeneity Issue #3 - Threading Thread scheduling algorithms may vary across replicas, causing potentially inconsistent results Example: – Request R1 and R2 both modify ObjectA. – R1 properly delivered before R2 on all replicas. – R1 starts executing in Thread1; R2 in Thread2 – Thread1 and Thread2 must block and resume identically on all replicas or replicas may compute different results Temporary Solution: Single-threaded servers only Long-term Solution: TBD

16. March 12, 2002 16 Firewall Proxy Goal: – Provide standard application proxy firewall protection for ITDOS clients and servers – Protect ITDOS enclaves at least as well as standard CORBA enclaves (e.g., prevent network attacks from entering the enclave) Approach: – Application proxy firewall inspects multicast messages and passes only valid messages from permitted sources

17. March 12, 2002 17 Typical Application Proxy Protocol Inspection (GIOP) SSL Packet Filtering TCP/IP Application (ORB) SSL TCP/IP TCP Connection SSL Security Association Application (ORB) SSL TCP/IP TCP Connection SSL Security Association Client Server

18. March 12, 2002 18 Firewall Issue #1 - Maintain Consistent Delivery ITDOS assumption: If one correct process delivers a message, all will eventually deliver the message BFT transport provides total ordering, determines when messages are delivered to the ORB Firewall must not cause inconsistent delivery because of GIOP inspection failures (above BFT protocol layer) Short-term Solution: Inspect only the BFT protocol. GIOP inspection not useful, so decryption not required. Long-term Solution: Change ORB implementation to deliver only valid GIOP messages

19. March 12, 2002 19 Firewall Conceptual Design C-L Protocol Inspection DoS Packet Blocking IP Virtual Connection Client Application Code IT ORB Voter Marshalling C-L BFT Protocol IP Multicast Server Application Code IT ORB Voter Marshalling C-L BFT Protocol IP Multicast Connection-less IP Multicast

20. March 12, 2002 20 Firewall Issue #2 - Flooding Attacks Blocking non-permitted sources not feasible: – C-L’s multi-MAC inadequate for source authentication – Source digital signatures within encrypted SMIOP payload (not visible to firewall) Replay could be used for flooding attacks Short-term Solution: Limit DoS flooding attack effects within enclave by probabilitisticly dropping some redundant messages Long-term Solution: Modify packet format to permit source determination, replay prevention

21. March 12, 2002 21 BFT Packets C-LSMIOPSMIOPDigitalMulti HeaderHeaderDataSignatureMAC Current Packet Layout C-LSMIOPSMIOPDigital HeaderHeaderDataSignature Optimized Packet Layout Encrypted Data Unencrypted Data Signature invisible to firewall Multi-MAC is redundant Signature visible to firewall Signature incorporated into C-L

22. March 12, 2002 22 Instantiation of New Replicas Goal: Replace faulty replicas as needed Required Steps: – Copy object state from running replicas to new replica – Add new replica to Replication Domain through key generation, distribution

23. March 12, 2002 23 Instantiation of New Replicas Issue: Potentially very large object space to replicate across hosts Total object space could be many gigabytes of data; might not be completely held in replica memory at any time Requires time, CPU, bandwidth on replicas Maintain referential integrity and object identity Possible Solution: Application-supported replication (only copy what the application requires) Alternate Solution: Very fast data movement

24. March 12, 2002 24 Other Potential Enhancements Voter – Solve є-equivalence problem Integrate secure IGMP when available Integrate with access control mechanism, such as OO- DTE Integrate Virtual Voting Machine after enhanced by WSU Support multiple group manager domains – Group manager identifier in IOR – Client able to connect to independent server replication domains

25. March 12, 2002 25 Testing Plan

26. March 12, 2002 26 Hypotheses H1: The system will not fail if it maintains n replicas of each application object, where n  3f + 1, and f or fewer fail. H2: The system rejects information from unauthorized entities. The act of rejecting the information does not excessively reduce the system’s performance.

27. March 12, 2002 27 Performance Variables V1: How the number of faults that the system can tolerate affects transaction times. V2: How fault detection by singleton objects compares to fault detection in replica groups. V3: How the number of replicas affects initialization times. V4: How voting with floating point values affects performance when compared to voting with discrete values.

28. March 12, 2002 28 Performance Variables (cont’d) V5: How message size affects system performance. V6: How servant implementation overhead compares with IT CORBA overhead. V7: How interposing an ITDOS application proxy firewall between a replicated server and its attacker reduces the impact of the attacks.

29. March 12, 2002 29 Testing Methodology Total of 6 experiments to support or refute hypotheses, measure performance variables. Each experiment employs a set of custom developed applications. –The applications record their own execution times and activities. –Applications log at the transaction level, liberally describing critical decisions and timing such as authentication activities, voting, and data communication. –Log selectively in order to determine and adjust for logging overhead in our results.

30. March 12, 2002 30 Status & Schedule

31. March 12, 2002 31 Progress to Date Completed draft Intrusion Tolerant CORBA Architecture, Firewall Architecture, Testing Plan Applied Validation Framework to ITDOS architecture Presented IT CORBA architecture at DOCSec Workshop and TAO Workshop Submitted paper on ITDOS architecture for DSN 2002 conference (accepted) Attending all domestic OMG meetings Completing IT CORBA prototype for TAO / C++ / Linux

32. March 12, 2002 32 Near-term Plans Complete initial IT CORBA prototype for Linux; port to Solaris Implement Firewall Proxy Present project update at DOC Sec 2002 in March Update Architecture documents to ‘as built’

33. March 12, 2002 33 Summary Usability requires: – Multi-threading support in servers Requires thread scheduling synchronization across hosts – Full protocol inspection in firewall Change multi-MAC to digital signature; remove signature from SMIOP Modify sequence number so that retries from a correct source have unique identifier in C-L header Decrypt and inspect entire message – Instantiation of new replicas Very fast data movement across hosts

34. March 12, 2002 34 Questions?