Tool-support for Invariant-based Specification, Synthesis, and Verification of Synchronization in Concurrent Java Programs M.S. Defense William Deng Department of Computing and Information Sciences Kansas State University

Goals of the Project II. Automatic derivation and weaving of synchronization code … multiple language and synchronization targets (Java, C++, monitors, semaphores, etc.) … weaving & optimization via abstract interpretation and program specialization techniques III. Automatic verification of critical safety and liveness properties of woven embedded code … domain-specific model-checking engines … built on previous DARPA work – Bandera environment I. Provide high-level, modular specification of global synchronization aspects … integrated with UML/RUP … formal specification via global invariants … language of composable invariant patterns … powerful, yet easy to use IV. Evaluation using Boeing BOLDSTROKE platform and military networking target vehicle electronics (CDA101)

My Contributions III. Automatic verification of critical safety and liveness properties of woven embedded code … Designed and implemented bounded counter version (coarse & fine grain) I. Provide high-level, modular specification of global synchronization aspects … Implemented formula & pattern-based specification language … Designed web pages to guide users in synchronization specification II. Automatic derivation and weaving of synchronization code … Designed and implemented translation from specifications to coarse-grain solution … Implemented translation to Java fine-grain solution … Implemented language- independent weaving process … Designed and implemented an approach for specific synchronization

SyncGen Tool Architecture Both coarse-grain and fine-grain solutions are synthesized automatically.java + Invariant Fine-grain Java Representation Generator Intermediate Representation Generator PVS.java + guarded commands.java + Core code Synchronization aspect

SyncGen Context UML Tools Synchronization Aspect Specification Tool Intermediate Representation Generator Solver/ Prover Course-grain solution Synchronization Aspect Back-end Bandera Analysis & Transformation Fine-grain solution Specialization Engine Bandera Safety Properties Liveness Properties Code Weaver Optimized Woven Code Invariant & Region tags Functional Core Code Templates (Java, C++, …) Template Instantiation Traditional Development Environment Functional Core Code (Java, C++, …) Finite State Models

Outline Coarse-grain solution generation –Guards Formula-based Pattern-based –Notification Formula/Pattern-based Specific synchronization Bounded Counter Version Open Issues Conclusion

Gyroscope/Rudder Synchronization Gyroscope Controller Buffer ProduceConsume Produce Rudder Controller RGRG R Gyroscope Value Resource(R G,1,R R,1,0) Empty Buffer Slot Resource(R R,1,R G,1,1) Exclusion(R G,R R ) Resource(R G,1,R R,1,0) + Resource(R R,1,R G,1,1) + Exclusion(R G,R R ) Invariant:

Coarse-grain Solution for Guards --- Formula-based Generation Resource(R G,1,R R,1,0) + Resource(R R,1,R G,1,1) + Exclusion(R G,R R ) Invariant: Resource(R G,1,R R,1,0) R_in <= G_out (R_in <= G_out) && (G_in <= R_out + 1) && ((G_in == G_out) || (R_in == R_out)) Desugared Invariant: Resource(R R,1,R G,1,1) Exclusion(R G,R R ) G_in <= R_out + 1 (G_in == G_out) || (R_in == R_out) …producer out …consumer in

Generating Coarse-grain Solution G_in++ …invariant I holds here …want I to hold here Task: generate a condition B that ensures that I holds after counter increment. Step 1: generate weakest-precondition(G_in++,I) (R_in <= G_out) && (G_in <= R_out + 1) && ((G_in == G_out) || (R_in == R_out)) (R_in <= G_out) && (G_in+1 <= R_out + 1) && ((G_in+1 == G_out) || (R_in == R_out)) Substitute G_in+1 for G_in G_in++>

Generating Coarse-grain Solution Step 2: simplify using decision procedures 1.Convert to disjunctive normal form 2.Eliminate disjuncts that are can never be satisfied using decision procedures 3.Minimize remaining conjuncts using decision procedures Example: (R_in <= G_out) && (G_in+1 <= R_out+1) && (G_in+1 == G_out) || (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out) 1. (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out) 2. (G_in+1 <= R_out+1) && (R_in == R_out) 3. Main point: fine a smaller B’ such that B’ && I && A B && I && A.

Generating Coarse-grain Solution Step 2.1: convert to disjunctive normal form (R_in <= G_out) && (G_in+1 <= R_out + 1) && ((G_in+1 == G_out) || (R_in == R_out)) (R_in <= G_out) && (G_in+1 <= R_out+1) && (G_in+1 == G_out) || (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out)

Generating Coarse-grain Solution Step 2.2: eliminate unsatisfiable disjuncts (R_in <= G_out) && (G_in+1 <= R_out+1) && (G_in+1 == G_out) || (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out) Intuition: before entrance to region, we know I holds and we know that A holds where A represents some basic properties on counters A = R_in >= 0 && R_out >= 0 && R_in >= R_out && G_in >= 0 && G_out >= 0 && G_in >= G_out (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out) For each disjunct D_k, ask theorem prover to check not(I && A && D_k). If return is true then D_k can be eliminated.

Generating Coarse-grain Solution Step 2.3: minimize remaining disjuncts (R_in <= G_out) && (G_in+1 <= R_out+1) && (R_in == R_out) Some of conjuncts in the remaining disjuncts are redundant. For each C_k in D_j, if (I && A && (D_j – C_k) => C_k) then C_k can be removed. (G_in+1 <= R_out+1) && (R_in == R_out)

Generating Coarse-grain Solution Pattern Based Approach Apply the process for each pattern: For pattern P(R_1,R_2,…,R_n) SubregionGuard R_i entrywp(In_i++,I) R_i exitwp(Out_i++,I)

Generating Coarse-grain Solution Bound Apply the process for Bound pattern: Bound(R,n): I = In – Out <= n SubregionGuard R entry wp(In++, In – Out <= n) = (In + 1) – Out <= n R exit wp(Out++, In – Out <= n) = In – (Out + 1) <= n ´ True

Generating Coarse-grain Solution Exclusion Apply the process for Exclusion pattern: Exclusion(R_1,R_2,…,R_n): I = Ç C 2 Comb(n, n-1) ( Æ i 2 C (In_i - Out_i == 0)) SubregionGuard R_i entry wp(In_i++,I) = Æ 1 · j · n Æ j  i (In_j – Out_j == 0) R_i exit wp(Out_i++,I) = true

Generating Coarse-grain Solution Resource Apply the process for Resource pattern: Resource(R_p,N_p,R_c,N_c,n): I = In_c*N_c – Out_p*N_p <= n SubregionGuard R_p entry wp(In_p++,I) = true R_p exit wp(Out_p++,I) = true R_c entry wp(In_c++,I) = In_c*N_c – Out_p*N_p <= n – N_c R_c exit wp(Out_c++,I) = true

Generating Coarse-grain Solution --- Building Guards From Relevant Patterns P_1(……) + P_2(……) P_n(……) Invariant : Each pattern instance contributes a portion of the guard R Counter++ Counter++> Intuitively, if R does not appear in P_i, then B_i is true Why this is correct? wp(S,I_1 &&…&&I_n) = wp(S,I_1) && … wp(S,I_n)

Generating Coarse-grain Solution Pattern Based Approach Compared with result generated by formula G_in++ G_in++> Resource(R G,1,R R,1,1,0) B_1 = true Resource(R R,1,R G,1,1) B_2 = G_in-R_out <=0 Exclusion(R G,R R ) B_3 = R_in == R_out B = B_1 && B_2 && B_3 = G_in – R_out <= 0 && R_in == R_out Example: the entry of gyroscope region B’ = (G_in+1 <= R_out+1) && (R_in == R_out)

Assessment In essence, we have performed the formula- based calculation statically to create guard schemas which are then instantiated when a pattern is used Efficient no need to call DP, convert to DNF, etc. Easy to optimize for example: for Bound(R,1), the guard of the entry of R should be in +1 <= out +1 generated by formula; in == out by pattern.

Notification Information If a counter C_1 increment can cause an await statement A transition from false to true, then there should be a notify/notifyall issue to A. A: C_2++> B: {P}C_1++{Q} Observation: we can calculate Q as the strong postcondition: sp(C_1++,P); P can be I&&G (G is the guard of B). Thus if Q implies B_2, we should issue notify/notifyall.

Notification Information Example: Readers/Writers Problem In_r ++> Reader Entry Reader Exit { In_w - Out_w == 0 && In_r - Out_r >= 1 } { In_w - Out_w == 0 && In_r - Out_r >= 0 } Writer Entry In_w ++> Notify reader entry? Writer Exit { In_w - Out_w == 1 && In_r - Out_r == 0 } { In_w - Out_w == 0 && In_r - Out_r == 0 } Notify writer entry Notifyall reader enter Notify writer entry

Notification Information P_1 Not (P_2 implies P_1) If Q_2 implies P_1 Then S_2 needs to notify S_1 S_2 P_2 Q_1 Q_2 S_1 Q_2 = sp(S_2, P_2) Generalize sp to sp n (S_2,P_2) If exists n s.t. sp n (S_2,P_2) implies P_1 Then S_2 should notify/notifyall S_1. We want to decide the notification information from S_2 to S_1

Notifyall vs. Notify 1. Resource(R_p,3,R_c,1,0): one enables three. 2. Resource(R_p,3,R_c,2,0): two enables three. 3. Exclusion(R_1,…,R_n): when (In_i – Out_i == 0)… Three cases for Notifyall: Cases 1 and 2 are normal: Case 3 is sort of burst: Three cases for Notify: 1. Writer exit to writer entry in Readers/writers: one enables one. 2. Resource(R_p,1,R_c,3,0): three enables one. 3. Group(R1,1,…,Rn,Nn): Ni enables one for R1… Formalize notifyall (from S_2 to S_1):

Notification Information Generation for Bound P_1 = In_B–Out_B<=n-1 Not (P_2 implies P_1) Q_2 implies P_1 Out_B++ P_2 = In_B–Out_B<=n Q_1 = In_B–Out_B<=n Q_2 = In_B–Out_B<=n-1 In_B++ Entry Exit …belongs to the case 1 of notify (one out let one enter). Thus a notify from the exit to the entry.

Notification Information for Exclusion Pattern Exclusion(R_1,R_2,…,R_n): SubregionNotification Information R_i entryNone R_i exitNotifyall R_j entry where i != j

Notification Information for Resource Pattern For pattern Resource(R_p,N_p,R_c,N_c,n): SubregionNotification Information R_p entryNone R_p exitNotifyall case 1: N_p/N_c >= 2; case 2: if 2>N_p/N_c>1. The exit of R_p to the entry of R_c. Notify case 1: if N_p/N_c =1; case 2, if N_p/N_c < 1. R_c entryNone R_c exitNone

Notification Information for Other Patterns-- Summary Exclusion(R1,R2,..,Rn) Notifyall case 3: from the exit of Ri to the entry of Rj (i != j) Resource(R_p,N_p,R_c,N_c,N) Notifyall case 1: N_p/N_c >= 2; case 2: if 2>N_p/N_c>1. The exit of R_p to the entry of R_c. Notify case 1: if N_p/N_c =1; case 2, if N_p/N_c < 1. Barrier(R1,R2) Notify case 1, the entry of R1 notify the exit of R2. Same for R2. Relay(R1,R2) Notify case 1, the entry R1 notify the exit of R2. Group(R1,N1,..,Rn,Nn) Notifyall case 3: if Nj>1, the entry of Ri notifyall the exit of Rj. Notify case 3, if Nj=1.

Notification Information for Gyroscope/Rudder Resource(R G,1,R R,1,1,0) The exit of R_G notify the entry of R_R Resource(R R,1,R G,1,1) Exclusion(R G,R R ) The exit of R_R notifyall the entry of R_G; The exit of R_G notifyall the entry of R_R. The exit of R_R notify the entry of R_G Overall effect The exit of R_R notify the entry of R_G; The exit of R_G notify the entry of R_R.

Specific Synchronization R_B1 R_B2 R_B3 R_C1 R_C2 R_C3 R_C4 R_C5 Barber Thread Customer Thread fetch inform leave B1 B2 C1C2 Specific groups: 0: B1 and C1; 1: B2 and C2. Problem: monitor can not differentiate B1 and B2. 1.Strength monitor to keep track of the relation? 2.Use object (multiple copies) instead of static method (one copy) to keep the relation. Relation: threads -> specific group Solution:

Implementation of Specific Synchronization Main (init) Component Threads SynManager Cluster Objects initializes call calls Component Threads Cluster Classes call Current Implementation: Proposed approach: use a central controller, SynManager

Implementation of Specific Synchronization Cont. Forming Cluster synchronization transition: code 1.builds an array of cluster names; 2.specifies synchronization (anonymous, forming, specific, or dissolving) for each cluster; 3.defines actors (the types of threads); 4.specifies specific group; 5.provides the maximum number of specific groups. User has to provide in main Specific Inform Anonymous Fetch Dissolving leave 0 1 B1 C1 B2 C2 Specific group -> cluster instances Threads -> specific group index Cluster Instances

Bounded Counter Version Problem: unbounded In/Out counters…partial state space check Bounded counter solution to check state space exhaustively. PatternBounded Counter BoundB = In - Out ExclusionE_i = In_i – Out_i ResourceR_pc = In_c*N_c - Out_p*N_p

Example: Readers/Writers Problem CLUSTER: RW STATE SPACE VARIABLES: E1_0, E1_1, B1; LOCAL VARIABLES: ; REGION: Reader ENTER: E1_0++;> NOTIFY: ; NOTIFYALL: ; EXIT: NOTIFY: ; NOTIFYALL: Writer_in; REGION: Writer ENTER: E1_1++;B1++;> NOTIFY: ; NOTIFYALL: ; EXIT: NOTIFY: Writer_in; NOTIFYALL: Reader_in;

Open Issues Exception handling Deadlock checking of SyncGen output Problem: nested or overlapping regions. Suggested solution: for nested or overlapping regions check B_1 && B_2… && B_n != false. Scheduling issues: priorities Responses: balking and timeout …

Conclusion Successfully implemented Mizuno’s approach Developed the bounded counter version solution for model checking Proposed an implementation for specific synchronization

To Do List Web-pages, manual, and tutorial Tutorial lecture for CIS720 Clean up code for release

Questions? Comments?