1. Model Checking Software Artifacts http://www.cis.ksu.edu/bogor SAnToS Laboratory, Kansas State University, USA Matt Dwyer John Hatcliff Gurdip Singh Principal Investigators Support US National Science Foundation (NSF) US National Aeronautics and Space Agency (NASA) US Department of Defense Advanced Research Projects Agency (DARPA) US Army Research Office (ARO) Rockwell-Collins ATC Honeywell Technology Center and NASA Langley Sun Microsystems Intel Students William Deng Georg Jung Oksana Tkachuk Robby Venkatesh Ranganath Jesse Greenwald Todd Wallentine http://www.cis.ksu.edu/bandera http://www.cis.ksu.edu/cadena

2. For the past decade … We’ve been developing program analysis frameworks Standard tensions Scalable versus Precise How semantic is the analysis? Property-specific versus Language-based How rich are the properties? Push-button versus Configurable How usable is the technology?

3. Analyzing Source Code Worked on a broad range of case studies SPIN, SMV, … Extracting models by hand Developed a series of tool frameworks for analyzing safety properties of concurrent programs FLAVERS (Ada) INCA Translators (Ada) Bandera (Java)

4. Succesful? Tools are widely used For education As a basis for further work by us and others Tools have been used to find bugs in real systems 1000-10000 LOC <10 threads Bugs that eluded significant testing efforts

5. Whole Program Analysis will never scale to large code bases even for highly abstract analyses (e.g., control flow) even for simple properties (e.g., def-use) Must perform modular analyses Hard to do for truly global properties? Hard to do in presence of concurrency? What are the natural module boundaries? How big can a module be?

6. A Solution … Target the full range of software artifacts Requirements models Architectural descriptions Designs (at various levels of refinement) Code Use semantic analyses within artifacts (properties) across different artifacts (conformance)

7. Features of our Vision Early and varied semantic modeling structural modeling is useful as well Analysis driven feedback and refinement Artifact generating analyses Proofs, reachable modes, … Synthesize code wherever possible Aspects of an agile process continuous delivery of working artifacts Exploit "domain information" throughout ultimately meta-tools may be useful

8. Development Flow Requirements Model User’s informal requirements Query checker, Visualization tools Requirements Model Requirements Model Requirements Model Consistency, Completeness, … checker

9. Development Flow User’s informal requirements Model-specific analysis Inter-model consistency, completeness, … checking Functional Model ’ Performance Model Functional Model Functional Model Functional Model ’ Performance Model Functional Model Functional Model ’ Performance Model Functional Model Functional Model ’ Performance Model

10. Development Flow Functional Model Functional Model ’ Performance Model … Design Model Conformance checker(s) Design Model

11. Development Flow Functional Model Functional Model ’ Performance Model … Structural Design Model Synchronization Policy Spec Quality of Service Spec Abstract Behavioral Model … Multi-layer conformance checking

12. Development Flow Functional Model Functional Model ’ Performance Model … Structural Design Model Synchronization Policy Spec

13. Development Flow Functional Model Functional Model ’ Performance Model … Structural Design Model Synchronization Policy Spec Quality of Service Spec Abstract Behavioral Model … Structural Design Model Synchronization Policy Spec Quality of Service Spec Abstract Behavioral Model … Structural Design Model Synchronization Policy Spec Quality of Service Spec Abstract Behavioral Model … Structural Design Model Synchronization Policy Spec Quality of Service Spec Abstract Behavioral Model …

14. Development Flow … Structural Design Model Synchronization Policy Spec Quality of Service Spec Abstract Behavioral Model … Code Conformance checker(s)

15. Development Flow … Structural Design Model Synchronization Policy Spec Quality of Service Spec Abstract Behavioral Model … Model/spec dependent synthesis procedures (proof generating) Domain-appropriate Implementation Framework

16. Lessons Adapt methods to developers Ease of use, leverage domain abstractions Use layered, incremental methods Low entry barrier, early and focused feedback Focus technology on the hard part Synchronization, timing, global properties Synthesize as much code as possible Developer buyin, reduce code-level reasoning Developers won’t write specs, so tell them they are writing code

17. and now for Bogor …

18. Model Checking in Cadena Steps toward our vision Hard problems here are not component coding (localized) Inter-component coordination (sequencing, synchronization, timing, …) Theme exploit domain semantics exploit implementation infra-structures

19. An Overview of … Component modeling Middle-ware modeling Develop an abstract model that captures semantics of actual middle-ware Environment modeling Exploit environment information to reduce state space Property specification Structural reductions Exploit structure of state space of periodic RT systems

20. Modal SP

21. Component Behavior component BMModal { uses ReadData dataIn; consumes DataAvailable inDataAvailable; publishes DataAvailable outDataAvailable; provides ReadData dataOut; provides ChangeMode modeChange; enum Modes {enabled,disabled}; Modes m; behavior { handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } … input ports

22. Component Behavior component BMModal { uses ReadData dataIn; consumes DataAvailable inDataAvailable; publishes DataAvailable outDataAvailable; provides ReadData dataOut; provides ChangeMode modeChange; enum Modes (enabled,disabled); Modes m; behavior { handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } … output ports

23. Component Behavior component BMModal { uses ReadData dataIn; consumes DataAvailable inDataAvailable; publishes DataAvailable outDataAvailable; provides ReadData dataOut; provides ChangeMode modeChange; enum Modes (enabled,disabled); Modes m; behavior { handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } … mode declaration using CORBA IDL

24. Component Behavior component BMModal { uses ReadData dataIn; consumes DataAvailable inDataAvailable; publishes DataAvailable outDataAvailable; provides ReadData dataOut; provides ChangeMode modeChange; enum Modes (enabled,disabled); Modes m; behavior { handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } … behavior for events on dataInReady port

25. Component Behavior component BMModal { uses ReadData dataIn; consumes DataAvailable inDataAvailable; publishes DataAvailable outDataAvailable; provides ReadData dataOut; provides ChangeMode modeChange; enum Modes (enabled,disabled); Modes m; behavior { handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } … behavior mode cases

26. Component Behavior component BMModal { uses ReadData dataIn; consumes DataAvailable inDataAvailable; publishes DataAvailable outDataAvailable; provides ReadData dataOut; provides ChangeMode modeChange; enum Modes (enabled,disabled); Modes m; behavior { handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } … data flow specification

27. Component Behavior component BMModal { uses ReadData dataIn; consumes DataAvailable inDataAvailable; publishes DataAvailable outDataAvailable; provides ReadData dataOut; provides ChangeMode modeChange; enum Modes (enabled,disabled); Modes m; behavior { handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } … publish event

28. Towards a Complete Model We have transition semantics for intra-component behavior. How should we model communication layer? ?

29. Middleware/Service Semantics Weak CCM and Event Services Specs (OMG) Informal : English and examples Intentionally under-specified to allow implementor freedom Looked at implemented semantics of existing ORBs and Event Services ACE/TAO, OpenCCM, K-State Developed a family of semantic models that captured their behavior

30. Outline of Real System Event channel with internal thread pool … Thread Pool … 60Hz20Hz5Hz1Hz ……… … passive components proxy consumer holds list of consumer references dispatch queues for each rate group publish correlation & filtering consumer refs Threads run call-backs associated with event consumer ports getData

31. Lots of details … What events are users interested in reasoning about? publish, dispatch, timeouts, … What state information? modes, dispatch queues, correlation automata, … Minimize detail, but retain … ability to express properties of interest correspondance with implementation semantics

32. System Observations Event channel with internal thread pool … Thread Pool … 60Hz20Hz5Hz1Hz ……… … invoke[m,c] publish[e,c] dispatch[e] publish correlation & filtering consumer refs accept[e,s] eof[r] getData c.m == v

33. Parts of Model Components Modes and attributes defined over simple types Handler/method state machines Method calls Middleware Subscriber lists, correlators, dispatch queues Scheduler, thread pool Environment Time-triggering of events Data from devices, etc.

34. Modeling Strategy Event channel with internal thread pool … Thread Pool … 60Hz20Hz5Hz1Hz ……… … correlation & filtering

35. Modeling Strategy Component Models Event Channel Model Component Models Environment Model Connection Models

36. Model Checker Support Ability to define efficient building blocks Sets, queues (with symmetry support) Flexible atomicity control Programmable scheduler Static data definition State compaction Flexibility in search strategies Full-state space Bounded/heuristic search State-less search BOGOR has all this and more …

37. Modeling of Components function tacticalSteering_push_inDataAvailable(CAD.Event event){ Data d; loc loc0: live {} when tacticalSteeringMode do {} goto loc1; when !tacticalSteeringMode do {} return; loc loc1: live{d} when true do { d := CAD.getField (AirFrame, "ReadData.data"); } goto loc2; loc loc2: live{} when true do { CAD.setField (TacticalSteering, “dataOut.data", d); } goto loc3; loc loc3: live {} invoke pushOfProxy(TacticalSteering, “dataOutReady“) return; } handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } handles dataInReady (DataAvailable e) { case m of enabled { dataOut::data <- dataIn.getData(); push {} dataOutReady; } disabled {} } Structure follows component behavior spec and connection representation closely

38. Modeling of Connections CAD.connectEvent(GPS, “dataCurrent", AirFrame,"inDataAvailable", 20, false); instance AirFrame of BMLazyActive on l2 { connect dataAvailable to GPS.dataCurrent atRate 20 connect dataIn to GPS.dataOut Modeled very directly in BOGOR

39. Modeling Middleware (Threads) thread threadgroup5() { Pair.type pair; EventHandlerEnum handler; CAD.Event event; loc loc0: live { handler, event } when Queue.size >(Q5) > 0 do invisible { pair := Queue. getFront >(Q5); Queue.dequeue >(Q5); handler := Pair. first (pair); event := Pair.second (pair); } goto loc1; loc loc1: live {} invoke virtual f(handler, event) goto loc0; } … Thread Pool 60Hz20Hz5Hz1Hz Dispatch queue polling

40. Extend model checker types Polymorphic extension Modeling Middleware (Queues) extension Queue for edu.ksu.cis.cadena.bogor.ext.Queue { typedef type ; expdef int size (Queue.type ); expdef int capacity (Queue.type ); expdef boolean isFull (Queue.type ); expdef boolean isEmpty (Queue.type ); expdef Queue.type create (int); actiondef enqueue (Queue.type, 'a); expdef 'a getFront (Queue.type ); actiondef dequeue (Queue.type ); expdef boolean containsPair (Queue.type,'a); } ……… Data in state space, operations implemented as Java code

41. Modeling Middleware (Scheduling) … Thread Pool 60Hz20Hz5Hz1Hz ……… … Bold Stroke Systems are scheduled based on RMA run highest-priority (i.e., rate) enabled action many fewer schedules, contains all real schedules BOGOR allows encoding specific schedules Java plugin filters enabled actions in state exploration algorithm Typically model checkers use non- deterministic scheduling i.e., choose from set of enabled transitions in a state set of all such schedules contains all real schedules

42. Modeling of Environment Model time directly expensive (state space becomes acyclic) hard to get accurate timing info (platform specific) Boeing isn’t very interested in real-time properties other than schedulability (?) Abstract time modeling strategies Timeouts can happen at any time Bound number of timeouts in hyper-period Bound relative number of timeouts in adjacent rate groups Approximate passage of time System behavior is driven by periodic time-triggered events

43. Modeling of Environment The maximum rate timeout count (tc) keeps track of the number of timeouts that have occurred during a hyper-period Assume a group of harmonic rates (R 1, …, R max ), where R max is the maximum rate Let p(R i ) be the normalized period of R i with respect to R max e.g., {5,10,20} => p(5) = 200ms/50ms, p(10) = 100ms/50ms, p(20) = 50ms/50ms For rate R i, timeouts are issued whenever tc % p(R i ) == 0 Increment tc between 0 and p(R 1 ) after every timeout of rate R max is issued

44. Relative Timeout Counts R1R2R3R1R2R3 R1R1 R1R1 R 1, R 2 R 1, R 2, R 3 Assume that worst case execution time of R i work can be performed in the period of R i There is a pattern to the number of timeout counts in a frame e.g., in frame of R i there are two timeouts of R i-1

45. Relative Timeout Counts Enforce only the relative # of timeouts for adjacent rates Timeout for R i is enabled after work for R i is complete proper number of timeouts for R i-1 are performed R1R2R1R2 R1R1 R 1, R 2 Problem: Don’t know how long R 2 work takes?

46. Relative Timeout Counts Enforce only the relative # of timeouts for adjacent rates Timeout for R i is enabled after work for R i is complete proper number of timeouts for R i-1 are performed R1R2R1R2 R 1, R 2 Problem: Don’t know how long R 2 work takes? Next R 1 timeout could fall in the middle of R 2 work R1R1 Must consider all interleavings of R 1 timeout and actions performed in R 2 work (or R 3 work, …)

47. Relative Timeout Counts R1R2R1R2 R 1, R 2 R1R1 R1R2R1R2 R1R1 R1R2R1R2 R1R1 R1R2R1R2 R1R1

48. Modeling of Environment Previous model does not relate component execution with passage of time Assume we have worst-case execution bounds for event handlers e.g., from schedulability analysis Keep track of intra-hyper-period time (ihp) normalized by duration of shortest action Increment ihp by duration bounds for handlers as they are executed One tricky issue …

49. Lazily-Timed Components ihp 12 t handler ihp 2 t handler Handler duration fits in before next frame boundary: 1.ihp += t handler 2.execute handler to completion 1.ihp += t handler 2.execute handler to completion ihp 1 t handler - Handler duration overruns next frame boundary: 1.ihp += 2.choose prefix of handler to execute 3.assign residual duration to handler suffix 1.ihp += 2.choose prefix of handler to execute 3.assign residual duration to handler suffix

50. Preliminary Results (Full Search) Basic 1 rate, 3 components, 2 events per hyper-period Multi-Rate 2 rates, 6 components, 6 events per hyper-period Modal 3 rates, 8 components, 125 events per hyper-period Medium 2 rates, 50 components, 820 events per hyper-period System ND Priority Lazily-Timed 20,.12s12,.11s14,.11s 120k, 5m100,.38s33,.19s 3M+, ?9.1k, 8.6s900, 1.3s 13M+, ?740k, 29m4k, 8.6s

51. Functional Properties Property II: If navSteering is enabled when 20Hz timeout occurs, then airFrame should fetch navSteering data [before end of frame] Property II: If navSteering is enabled when 20Hz timeout occurs, then airFrame should fetch navSteering data [before end of frame] Property I: System never reaches a state where TacticalSteering and NavSteering are both disabled Property I: System never reaches a state where TacticalSteering and NavSteering are both disabled

52. Lack of Model Analysis Boeing OEP Challenge Problems If component 1 is in mode A when component 2 produces event E, then component 3 will consume event F (Section 4.1.5.3.6) A temporal property well-suited for model-checking!

53. Event-based Specifications Many properties of interest in this domain are event oriented Some require access to state information A state qualified event (written e + f ) defines the occurrence of an observable event, e, in a state that satisfies a given formula, f. For example, If component c1 is in mode A when component c2 produces event E, them component c3 will consume event F.*; publish[E, c2] + c1.mode == A; [- invoke[F handler, c3]]* No trace of form

54. Exploiting System Structure Symmetry reductions Equivalence between states Partial order reductions Commutativity between transitions Collapse compression Sharing between state components Can we exploit the time-triggered nature of real-time systems?

55. A Simple Transition System l1: y = 0; goto l2; l2: x = 0; goto l3; l3: true -> x = 2; goto l4; true -> x = 3; goto l4; l4: y = y + x; goto l5; l5: y>5 -> skip; goto end; y skip; goto l2; end:

56. System State Space

57. State Space Decomposition

58. Synopsis A system is quasi-cyclic if A subset of its state variables repeatedly reach certain fixed values (e.g., initial values) The rest of the variables can vary freely Decompose DFS of quasi-cyclic system BFS of quasi-cyclic regions DFS within regions Memory bounded by region DFS Time penalty due to redundant state visits

59. Cadena models are quasi-cyclic

60. Parallel Quasi-cyclic Search Region DFS are completely independent Embarassingly parallel Naïve implementation on 4 processors overcomes overhead This summer check realistic scenario ~400 components, ~130 modal components

61. Ultimate Modeling View CCM IDL Model Layer Check mode behaviors, temporal properties, timing constraints Code Layer Check that implementations satisfy/refinement high-level specs – leverage the fact that code skeletons are automatically generated Generate code, fill- in skeletons, check for refinement We don’t do all of this yet!

62. Some Ongoing Work Integration of specification support into Cadena Model generation from Cadena Counter-example display Refinement checking (e.g., dependences against state machines) Incorporating synchronization info Modeling distributed nodes Incorporating time into spec language