1. Modular Verification of Concurrent Assembly Code with Dynamic Thread Creation and Termination Xinyu Feng Yale University Joint work with Zhong Shao

2. 2005-9-16NJPLS@Stevens Motivation Proof-carrying code (PCC) In principle: verify any property on any code Real binaries & no loss of efficiency Embedded OS, device drivers… All safety & liveness properties… Formal, machine-checkable proofs In reality: only works for sequential code Can concurrent codeever be supported by the PCC framework ?

3. 2005-9-16NJPLS@Stevens Challenges Challenges for Proof-carrying concur. code A general framework for concurrent assembly code verification Lack of structures (e.g. cobegin/coend blocks) Specification/proof generation Spec inference, proof assistant, theorem prover Concurrent assembly code verification No directly applicable logic Traditional Hoare-logic: only sequential code Type Systems: no Concurrent Typed Assembly Language (TAL)

4. 2005-9-16NJPLS@Stevens Previous work Rely-Guarantee (R-G) Method Shared memory concurrency Thread modular verification Only for higher-level code: cobegin/coend CCAP [Yu&Shao, ICFP’04] The first PCC framework supporting concurrent assembly code R-G method Only support static threads P 1 || … || P n

5. 2005-9-16NJPLS@Stevens Concurrency Programming cobegin / coend S::=…| cobegin P1 || P2 codend | … Higher-level, well-structured Only support properly nested concurrent code fork/join S::=…| tid := fork f(a) | join tid | … More flexible: improperly nested code OSes/Java/…

6. 2005-9-16NJPLS@Stevens Our Contributions A new PCC framework: CMAP Verification of general properties Dynamic thread creation/termination Generalize the Rely-Guarantee method Modular verification Realistic features Multiple instantiations of thread code Thread argument passing, thread-local data

7. 2005-9-16NJPLS@Stevens Outline of This Talk Background: the Rely-Guarantee Method Challenges for Dynamic Thread Creation/Termination Our Approach The CMAP Framework Conclusion and Future Work

8. 2005-9-16NJPLS@Stevens The Rely-Guarantee Method Thread 1 Thread 2 (A 1,G 1 ) (A 2,G 2 ) Shared Memory S1S1 S2S2 S3S3 S4S4 S5S5 A 1 : S 2 – S 3, S 4 – S 5,… G 1 : S 1 – S 2, S 3 – S 4,… A 2 : S 1 – S 2, S 3 – S 4,… G 2 : S 2 – S 3, S 4 – S 5,… G 1  A 2 G 2  A 1

9. 2005-9-16NJPLS@Stevens The Rely-Guarantee Method Thread + Thread Environment Rely and Guarantee A, G: State  State  Prop Thread Modularity Non-Interference (interface compatibility):  i,j. i  j  G i  A j Safety of each thread T i : (A i, G i )

10. 2005-9-16NJPLS@Stevens GCD Example [Yu&Shao’04] Thread1: while(a<>b){ if(a > b) a := a-b; } Thread2: while(a<>b){ if(b > a) b := b-a; }

11. 2005-9-16NJPLS@Stevens Outline of This Talk Background: the Rely-Guarantee Method Challenges for Dynamic Thread Creation/Termination Our Approach The CMAP Framework Conclusion and Future Work

12. 2005-9-16NJPLS@Stevens Concurrency Programming cobegin / coend S::=…| cobegin P1 || P2 codend | … Higher-level, well-structured Only support properly nested concurrent code fork/join S::=…| tid := fork f(a) | join tid | … More flexible: improperly nested code OSes/Java/…

13. 2005-9-16NJPLS@Stevens Static and Dynamic Threads f(a). fork f(a 1 ) fork f(a 2 ) fork f(a n ) … “Static Threads” “Dynamic Threads”

14. 2005-9-16NJPLS@Stevens Challenges First attempt Check NI between all static threads T i : (A i, G i )  i,j. i  j  G i  A j  Too rigid to handle changing env.

15. 2005-9-16NJPLS@Stevens Challenges: Changing Env. I A-B: initialize data d no other threads will change d A : d = d’ B-C: collaborate with T 3 to process d T 3 may change d Still do not allow other threads change d C-D: T 3 terminates No other threads can change d T1T1 T2T2 A B T3T3 C D Use pc to mark stages?

16. 2005-9-16NJPLS@Stevens Challenges: Changing Env. I main: int i:=0; while (i<100){ data[i]:=f(i); fork child(i); i++; } Global data: int data[100] T1T1 T2T2 A B T3T3 C D …

17. 2005-9-16NJPLS@Stevens Challenges: Changing Env. II T 2 and T 3 have no overlap in their lifetime non-interference between all threads? Only check those that overlap? How to specify the overlapping? T1T1 T2T2 T3T3

18. 2005-9-16NJPLS@Stevens Challenges: multiple instantiations f(a). (A a, G a ) (A a1, G a1 ) fork f(a 1 ) fork f(a 2 ) fork f(a n ) (A a2, G a2 )(A an, G an ) G ai  A aj GaAa?GaAa?

19. 2005-9-16NJPLS@Stevens Challenges: Modularity T 1:. jmp f f:. exit T 2:. jmp f (A 1, G 1 )(A 2, G 2 ) Certify once, use everywhere?

20. 2005-9-16NJPLS@Stevens Outline of This Talk Background: the Rely-Guarantee Method Challenges for Dynamic Thread Creation/Termination Our Approach The CMAP Framework Conclusion and Future Work

21. 2005-9-16NJPLS@Stevens Our Approach (1) Problems for checking NI of static threads Changing environment Multiple instantiations Modularity issues CMAP: “lazy checking” At each step, all live (dynamic) threads do not interfere

22. 2005-9-16NJPLS@Stevens Our Approach (2) … t0t0 tntn Q (A 0, G 0 ) … (A n, G n )  How to track the changing thread queue? WF(Q,  ): each t i satisfies (A i, G i )

23. 2005-9-16NJPLS@Stevens Our Approach (3) Q' ''  WF Q   Initial condition:  0. WF(Q 0,  0 )  ::= add | sub | jd f |… | exit | fork | yield Borrow ideas from typechecking data heaps (as in TAL):

24. 2005-9-16NJPLS@Stevens Our Approach (4) Thread Termination: exit Q t  (A,G)  \{(A i, G i )} (A i,G i )  WF WF! exit Q\{t i } titi Q

25. 2005-9-16NJPLS@Stevens Our Approach (5) Thread Creation: fork f(a) Q t (A,G) t ? WF fork (1) t' does not interfere with Q (2) t does not interfere with the new env.  Q t’

26. 2005-9-16NJPLS@Stevens Our Approach (6) t (A,G) Q  {t’} t fork G   i A i  i G i  A  ? (A’,G’)  {A’’,G’’} G'  (  i A i )  A'' (  i G i )  G''  A' WF WF? Q  G''   i A i  i G i  A'' G G A A 

27. 2005-9-16NJPLS@Stevens Our Approach (7) Queue Extension WF(Q  {t},  {(A, G)}) WF(Q  {t',t},  {(A’’, G’’), (A  G’’, G  A’’)}) fork f(a) A  A’’, G’’  G

28. 2005-9-16NJPLS@Stevens Our Approach (8) Queue Update WF(Q  {t},  {(A, G)}) WF(Q  {t},  {(A’, G’)}) A  A’, G’  G; t: (A ’, G ’ )

29. 2005-9-16NJPLS@Stevens Our Approach (9) T 1:. jmp f f:. exit T 2:. jmp f (A 1, G 1 )(A 2, G 2 ) Certify once, use everywhere? (A, G) A i  A, G  G i

30. 2005-9-16NJPLS@Stevens Our Approach (10) Check static threads  Lazy Check Changing Env.  Changing (A, G) Multiple instantiation  Not care Modularity  Certify only once General Enough Language (higher-level/assembly) Thread Model (preemptive/non-preemptive)

31. 2005-9-16NJPLS@Stevens Example – Unbounded Thread Creation main: int i:=0; while (i<100){ data[i]:=f(i); fork child(i); i++; } void child(x:int){ data[x] = g(x, data[x]) } Global data: int data[100]

32. 2005-9-16NJPLS@Stevens Example – Unbounded Thread Creation Specification of Child: A x : G x : Non-interference between children:

33. 2005-9-16NJPLS@Stevens Example – Unbounded Thread Creation How to specify the main thread? main: int i:=0; while (i<100){ data[i]:=0; fork(child, i); i++ } Do we need a G such that: But main cannot satisfy such a G!

34. 2005-9-16NJPLS@Stevens main: int i:=0; while (i<100){ data[i]:=0; fork(child, i); i++ } (A’, G’) (A, G)

35. 2005-9-16NJPLS@Stevens Outline of This Talk Background: the Rely-Guarantee Method Challenges for Dynamic Thread Creation/Termination Our Approach The CMAP Framework Conclusion and Future Work

36. 2005-9-16NJPLS@Stevens The CMAP Framework The abstract machine The verification logic Specification language Inference rules Soundness proof Example programs Unbounded dynamic thread creation Readers/Writers problem Lock-free program All implemented in Coq!

37. 2005-9-16NJPLS@Stevens The CMAP Framework - Machine I1I1 f1:f1: I2I2 f2:f2: … (code heap) C (program) P::=(C,T,S,Q,I) 0 r1r1 12… r2r2 r3r3 …rnrn (data heap) H (register file) R (state) S::=(H,R) I1I1 h1:h1: I2I2 h2:h2: … (thrd entries) T add … fork h yield exit (instr. seq.) I I R I R I R … (dyn. queue) Q

38. 2005-9-16NJPLS@Stevens The CMAP Framework The paper on CMAP (Feng&Shao ICFP’05) : http:// flint.cs.yale.edu/publications/cmap.html

39. 2005-9-16NJPLS@Stevens Conclusion Problems for unbounded dynamic thread creation Changing environment (fork/exit) Multiple instantiation of thread code No previously known modular verification method Our approach INV: active threads in the system do not interfere Combine the type-based proof technique with R-G method Unify thread’s assumption/guarantee with env.’s guarantee/assumption Thread modularity + code/proof reuse The CMAP framework and its Coq implementation

40. 2005-9-16NJPLS@Stevens Future Work Certified Thread Libraries fork, yield, exit join, lock, monitors Surface language Higher-level specifications Partially infer A and G Certifying compilation to CMAP Where is the threads ? User-level thread + thread lib.

41. 2005-9-16NJPLS@Stevens Thank you!