1. 11/9/041 Bridging the gap between specification and implementation Insup Lee Department of Computer and Information Science University of Pennsylvania November 9, 2004

2. 11/9/04 2 The gap between specification and implementation ► Problem –gap between an abstract model and the implementation –scalability challenge (software size and complexity) ► Approaches –Software model checking –Model-based code generation –Test generation from specification –Run-time verification/checking ► Model checking –Formal, Complete –Does not scale well –Checks design, not implementation ► Testing –Tests an implementation directly –Informal, Incomplete Requirements Design specification Implementation

3. 11/9/04 3 Run-time Verification and its Application to Security ► Run-time verification ► The MaC framework –Java-MaC –Steering ► Model-based security checking –Security automata –Model carrying code ► Current work

4. 11/9/04 4 Runtime Verification ProgramVerifier Execution Information Check Sat / Unsat Feedback User

5. 11/9/04 5 Run-time verification ► Run-time monitoring and checking w.r.t. formal specification ► Ensures the runtime compliance of the current execution of a system with its formal requirement ► Steps 1.Specify formal requirements 2.Extract information from current executing program 3.Check the execution against formal requirements 4.Steer the computation to a safe state ► Complementary methodology to formal verification and program testing –Validate implementation –Not complete: guarantee for current execution –Prevention, avoidance, and detection & recovery ► Joint work with S. Kannan, M. Kim, U. Sammapun, O. Sokolsky, M. Viswanathan

6. 11/9/04 6 The MaC Framework Program Static Phase Run-time Phase low-level behavior high-level behavior Program Filter Automatic Instrumentation Human Monitoring Scripts Low-level Specification High-level Specification Event Recognizer Event Recognizer Automatic Translation Run-time Checker Run-time Checker Automatic Translation Input Informal Requirement Spec [Kim et al, ECRTS 99]

7. 11/9/04 7 Design of the MaC Languages ► Must be able to reason about both time instants and information that holds for a duration of time in a program execution. –Events and conditions are a natural division, which is also found in other formalisms such as SCR. –Conditions, which are true or false for a finite duration of time (e.g., is variable x >5?) –Events, which are either present or absent at some instant of time (e.g., is the control right now at the end of method f?). ► Need temporal operators combining events and conditions in order to reason about traces. start(position==100)end(position==100) 1:00:101:00:301:00:15 raiseGate Time position == 100

8. 11/9/04 8 Logical Foundation ► Conditions interpreted over 3 values: true, false and undefined. ► [.,.) pairs a couple of events to define an interval. ► start and end define the events corresponding to the instant when conditions change their value. [Lee et al, ICPDP 99] e1e1 e2e2 [e 1,e 2 )

9. 11/9/04 9 Meta Event Definition Language (MEDL) ReqSpec /* Import section */ import event ; import condition ; /*Auxiliary variable */ var int ; /*Event and condition */ event =...; condition =...; /*Property and violation */ property =...; alarm =...; /*Auxiliary variable update*/ -> { :=... ; } End ► Expresses requirements using the events and conditions, sent by event recognizer. ► Expresses the subset of safety properties. ► Describes the safety requirements of a system, in terms of conditions that must always be true, and alarms (events) that must never be raised. –property safeRRC = IC -> GD; –alarm violation = start (!safeRRC); ► Auxilliary variables may be used to store history. –endIC-> { num_train_pass’ = num_train_pass + 1; }

10. 11/9/04 10 The MaC languages Run-time state: method call object state local variables Abstract state: events conditions MEDL PEDL SADL ► PEDL: abstraction ► MEDL: abstract transformation ► SADL: feedback

11. 11/9/04 11 PEDL (Primitive Event Definition Language) ► Primitive Event Definition Language (PEDL) –Maps the low-level state information of the system to high-level events. –Depends on target program implementation –Design Issues What can be observed? Passive and active probes Software probe vs. hardware probe Programming language-based vs. systems API-based ► Java-PEDL –Provides primitives to refer to values of variables and to certain points in the execution of the program. –PEDL is defined so that events can be recognized in time linear to the size of the PEDL specification [Kim et al, FMSD 04]

12. 11/9/04 12 Java-MaC Static Phase Java Bytecode Instrumented Bytecode MaC Compilers PEDLMEDL MaC Specifications Event Recognizer Checker MaC Verifiers SADL Steerer

13. 11/9/04 13 Instrumented Java ProgramMaC Verifier Execution Information Sat / UnsatFeedback User Event Recognizer (PEDL) Steerer Checker (MEDL) Java-MaC Dynamic Phase

14. 11/9/04 14 MaC Language - PEDL Java ProgramPEDL Abstraction - When train position is between 30 and 50 - When gate starts/ends being down export event startGD, endGD; export condition cross; // specify what to monitor monobj Train.position; monmeth Gate.up(); monmeth Gate.down(); // specify abstraction condition cross = (30 < Train.position) && (Train.position < 50); event startGD = endM(Gate.down()); event endGD = startM(Gate.up()); Railroad Crossing Property: - If train is crossing, then gate must be down - Train is crossing when position is between 30 and 50 position = 0 position = 20 position = 40 Gate.down() position = 55 Gate.up() position = 60 startGD endGD cross cross = true cross = false

15. 11/9/04 15 MaC Language - MEDL PEDLMEDL import event startGD, endGD; import conditions cross; // specify abstraction condition gateDown = [startGD, endGD); // specify property property safeRRC = cross -> gateDown; Violation gateDown cross Abstraction - When gate is down Property - If train is crossing, then gate must be down Railroad Crossing Property: - If train is crossing, then gate must be down - Train is crossing when position is between 30 and 50 startGD endGD cross cross = true cross = false

16. 11/9/04 16 Instrumentation class Train { int position; main() { position = 0; position = 20; position = 40; position = 55; } monobj Train.position; class Train { int position; main() { position = 0; send(x,0); position = 20; send(x,20); position = 40; send(x,40); position = 55; send(x,55); } + = Sent to Event Recognizer: [ (position,0), (position,20), (position,40), (position,55) ]

17. 11/9/04 17 MaC Language - Components Java ProgramPEDLMEDL Abstraction - When train position is between 30 and 50 - When gate starts/ends being down Railroad Crossing Property: - If train is crossing, then gate must be down - Train is crossing when position is between 30 and 50 Abstraction - When gate is down Property - If train is crossing, then gate must be down position = 0 position = 20 position = 40 Gate.down() position = 55 Gate.up() position = 60 Violation gateDown cross startGD endGD cross cross = true cross = false

18. 11/9/04 18 Evaluation Overview ► Static Phase: Each property is represented as a tree –The most basic events/conditions/aux vars are at the leaf level For PEDL, variable updates, start/end method events are leaves For MEDL, imported events from PEDL and auxiliary variables are leaves –Composition of events and conditions link to events/conditions that are composed of –PEDL and MEDL are forests ► Dynamic Phase: In both Event Recognizer (ER) and Checker, evaluation starts at the leaves and traverses up to the root –Evaluation starts only at leaves representing occurred events or changed conditions –Otherwise, no evaluation is done –Efficient

19. 11/9/04 19 PEDL Graph export event startGD, endGD; export condition cross; monobj Train.position; monmeth Gate.up(); monmeth Gate.down(); condition cross = (30 < Train.position) && (Train.position < 50); event startGD = endM(Gate.down()); event endGD = startM(Gate.up());

20. 11/9/04 20 MEDL Graph import event startGD, endGD; import conditions cross; condition gateDown = [startGD, endGD); property safeRRC = cross -> gateDown;

21. 11/9/04 21 Algorithm ► Static Phase: –Create PEDL and MEDL graphs –Assign a height to each node in both graphs ► Dynamic Phase: –Maintain an evaluation list sorted by height –Add all occurred primitive events and changed conditions to the evaluation list at height 0 –For each event/condition in the evaluation list, Call evaluate() method Add its parent in the evaluation list (if not already in) Repeat until the list if empty –Finishing ER sends occurred events/changed condition to checker for each exported events/condition Checker notifies user for each event in the alarm list. ► Complexity –The size of the PEDL or MEDL graph is linear in the size of the formula. –Evaluation of a MEDL or PEDL formula on a single observation is linear in the size of the graph.

22. 11/9/04 22 Instrumented Java ProgramMaC Verifier Execution Information Sat / UnsatFeedback User Event Recognizer (PEDL) Steerer (SADL) Checker (MEDL) Java-MaC Dynamic Phase

23. 11/9/04 23 Steering process system checker violation action invoked action executed detection steering condition satisfied action invocation received event received

24. 11/9/04 24 Simplex Architecture Safety Experimental Decision Module Physical System usus ueue xu SC EX x0x0 Equilibrium state ► Experimental controllers provide improved performance but uncertain stability properties –Can be dynamically added or replaced ► Safety controller has the largest stability region [L. Sha]

25. 11/9/04 25 Inverted Pendulum (IP) Example Device Drivers angle, track volts Decision Module Switching logic Written in C Safety Controller Experimental Controller Experimental Controller Experimental Controller [L. Sha] m l  x g M ufuf

26. 11/9/04 26 Steering Action Definition Language ► SADL script –identifies object used in steering // the target of steering is the object dm of type DecisionModule // located in the class IP DecisionModule IP:dm; –defines steering actions // setSC() method of dm is invoked steering action change2SC = { call (IP:dm).setSC(); } –specifies steering location locations in the code where the actions can be executed before read DecisionModule:volts;

27. 11/9/04 27 IP and MaC Device Drivers angle, track volts Decision Module Written in Java Safety Controller Experimental Controller Experimental Controller Experimental Controller JNIJNI monitor steer MaC: Switching logic [Kim et al, RV 02]

28. 11/9/04 28 Applications ► Where can we get specifications? –During the design specification and analysis phase, many properties may be identified and verified. Reuse properties model checked during the design phase E.g., more than 1000 properties during designing of flight control systems –Extract from (informal) requirements and specification documents –Security Policy –Extract from the target program To ensure that the program has not been tampered Model Carrying Code ► Other application areas –Network routing simulation –Hardware design –Adaptable sensor network systems –Cheat detection in distributed game ► Annual run-time verification workshop (01, 02, 03, 04, …)

29. 11/9/0429 Monitoring and Checking for Security Properties

30. 11/9/04 30 Similar techniques, different purposes ► Check security policy –Security automata, edit automata –Model-Carrying Code (MCC) ► Intrusion detection –Extract from the target program to ensure that the program has not been tampered –Signature-based approach

31. 11/9/04 31 Security Policy in Security/Edit Automata Example (modified from [BLW02]): Limit the amount of memory that an application can allocate for itself Property: application must not allocate memory more than n a = (malloc(q), q’=q’+q, q’< n) a a a = (malloc(q), q’=q’+q, q’ >= n) halt a = (malloc(q), q’=q’+q, q’< n)

32. 11/9/04 32 Must not allocate more than n export event mallocCall; monmeth int malloc(int); event mallocCall = startM(malloc(int)); PEDL import event mallocCall; import action halt; var int memory; alarm violateMemoryPolicy = end(memory < 1000); mallocCall -> { // value(mallocCall,0) returns arg of malloc() memory’ = memory + value(mallocCall,0); } violateMemoryPolicy -> { invoke(halt); } MEDL steering action halt = // exit before next malloc() call { call System.exit(); } before call malloc(int); SADL

33. 11/9/04 33 Model-Carrying Code (MCC) ► How can we run untrusted code on our machine? –Untrusted code comes with a model of its security-relevant behavior –Users have their own security policies –Employ two types of checking ► Static checking: to ensure that untrusted program’s model respects user’s security policy –Use model checking to check that Beh(Model) are in Beh(Policy) ► Run-time checking: to ensure that program behaves as specified by model –Use runtime checking with Model is a specification (Automata) Events are system calls

34. 11/9/04 34 MCC Framework [SVB+03]

35. 11/9/04 35 Current Work on MaC ► Java-MaC available for download –www.cis.upenn.edu/~rtg/mac ► Minimum trace of run-time verification ► MEDL-RE: MEDL with regular expressions ► MaC with probability ► Hierarchical IDS ► Using MaC to detect failed/malicious sensor nodes

36. 11/9/04 36 Minimal Trace for run-time verification ► Large amounts of monitored data hampers communication between the system and the checker –How to extract minimum information necessary to check a given property? ► Redundant information in traces –alarm dropReq = fault when responding condition responding = [acceptReq,response) event acceptReq = request when ready==true event fault = start(abort == true) –Trace with redundant data: request ready(t) abort(t) request abort(f) request abort(t) –Trace without redundant data: ready(t) request abort(t) ► Approach: define minimally adequate trace, and MEDL automata

37. 11/9/04 37 MEDL-RE: MaC with regular expression ► Regular expression over events –Statement: RE R {Ē} =, –Grammar of R: R ::= e | R.R | R+R | R* –Alphabet of R contains events used in R and events in its relevant set {Ē} ► Regular expressions are neither events nor conditions and cannot be used alone –Events associated with RE R : startRE(R), success(R), fail(R) ► Example –Three components of a media must start in the following order: video, caption, audio –RE media {} = –alarm notOrdered = fail(media) ► Challenges –(Possibly infinite) multiple instances of (possibly overlapping) regular expressions

38. 11/9/04 38 ► Definition of Probability –P(E) = |E| / |S| Ex. Sample Space S = {H,T} and Event E = {H}, P(E) = 0.5 –Long run frequency: P(E) = lim n(E) n   n where n = the number of experiments performed, n(E) = the number of outcomes belonging to the event E Ex. experiment is performing coin flipping, P(H) is the number of times the outcome is H in relative to the number of times the coin flipping is performed ► Challenges –How to detect experiments from execution trace –Accuracy of probability calculated from execution trace ► Possible approach –Detecting experiment using events or regular expressions –Calculate probability from execution trace with confidence interval –Possible syntax: E ~ [  p, Exp] and C ~ [  p, Exp] where  = | =, p is probability, Exp is an experiment MaC with probability

39. 11/9/04 39 Extend MaC Architecture for IDS

40. 11/9/0440 Thank You!