Verifying Transaction Ordering Properties in Unbounded Multi-Bus Networks Michael D. Jones, Ganesh Gopalakrishnan University of Utah, School of Computing FMCAD’00 Austin, Texas

Single-Bus ...

Multi-bus ... ... ... ... ... ... Verifying safety properties of networks when the network topology matters. Reorderings, deletions happen in internal nodes. ... ...

Case Study Abstraction, theorem proving and model checking applied to reasoning about multi-bus PCI. HOL theorem proving too hard (for us...) Finite state model checking impossible

Motivation Difficult, interesting problem, but few published solutions Application to shared memory systems, multi-bus IO

Related Work PCI Verification Parameterized Branching Networks Shimizu:FMCAD’00,Clarke:Charme’99, Corella:CHDL’97 Parameterized Branching Networks Bhargavan:TPHOLs’00, Kesten:CAV’97

How PCI works (in our model) Bus Posted d p c Delayed completion this is a definitional slide. What is an agent, a bridge, how do you hook them up? an agent is at the end of a network of busses. Every agent can talk to every other agent. No cycles. What kinds of transactions are there? Posted and delayed. d Agent Bridge

Posted transactions Posted transaction, P, from A to B. A puts p on “the rest of the network” and forgets about it. B receives P and that’s it. big animation of a posted transaction. Highlighting that a P trans is complete at the master when it leaves. The Rest of the network p B A

Delayed transactions Delayed trans., d, from A to B. A puts d on “the rest of the network” and waits for a completion. B receives d and sends a completion,c. even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. The Rest of the network d B A

Delayed transactions 2 bridges between A and B Other transactions as shown. d tries to latch to bridge 1. d is now committed (called d’). even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d c p’ B A

Delayed transactions Eventually, d’ latches to bridge 1. bridge 1 has an uncommitted copy of d d can pass the other d entry already in bridge 1. even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d d c p’ B A

Delayed transactions d can attempt to latch to bridge 2. d will then be committed at bridge 1. even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d d c p’ B A

Delayed transactions Eventually, d’ latches to bridge 2. d’ d’ d c p’ even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d c p’ B A

Delayed transactions d can pass completion entry c. d’ d’ d d c p’ B A even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d d c p’ B A

Delayed transactions But, uncommitted d entries can be dropped at any time... even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d d c p’ B A

Delayed transactions bridge 1 has to resend d’ to bridge 2 d’ can not be deleted even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d c p’ B A

Delayed transactions d can be dropped again... pretend it passes C again. d can not pass posted transactions. d waits till p’ completes. even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d d c p’ B A

Delayed transactions d commits then latches to agent B. B creates a completion entry C. even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d d c B A

Delayed transactions d’ in bridge 2 can complete with the completion in B. d’ will be deleted from bridge 2. c will move into into bridge 2. even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d’ d’ d c c B A

Delayed transactions d is now complete at bridge 2. d’ in bridge 1 can complete with c in bridge 2. c can be deleted too... even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d’ d c c B A

Delayed transactions d is now complete at bridge 1. finally, d’ in agent A completes with c in bridge 1. even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. d’ d’ d c c B A

Delayed transactions d is now complete at A. c d’ d c B A even bigger annimation showing a delayed transaction building its path of committed entries, an uncommitted transaction getting deleted. Then show a completion getting issued. Show the completion coming back through and deleting the stack of committed entries. Show a commit getting dropped. c d’ d c B A

Reordering and deletion P can pass anything except P. D and C can pass either D or C. uncommitted D can be dropped. oldest C in a queue can be dropped. P and committed D never dropped. Have already shown some deletion rules. Maybe reproduce the passing table from the FMSD paper?

Producer/Consumer property if a producer agent writes a data item and the producer sets a flag and if the consumer reads the flag then the consumer will read the new data item.  in any PCI network, during any execution sort of reproduce the definition from the paper. Do an example in the next slide. move the example to be a running example over several slides fake animated. maybe no running example? Pretty simple concept. then move the formal definition to the next slide.

Solution C = acyclic multi-bus PCI networks = Producer/Consumer property L = Labelings assigned by Producer/Consumer

Solution C = acyclic multi-bus PCI networks = Producer/Consumer property L = Labelings assigned by Producer/Consumer = Project a finite state model out of n v = Add non-determinism to PCI on n

State Projection  

State Transitions    

Unreachable states     

Adding Non-determinism     

What is actually modeled     

Despite the spurious behaviors in PCI’, PCI’ can still be used to prove useful properties of PCI.     

Refinement Proof s t post(t,s)   (s) t’ post(t’, (s))

Proof Metrics ~1,500 lines to model transitions and abstraction ~1,000 proof commands in final proof ~1 month of effort to build models and do the proof.

Checking the Reduced Model States CPU time (sec) P D C F 2,690 51.20 P C D F 1,614 35.35 P F D C 914 18.68 P F D C 648 12.56 Total 5,866 117.79

Solution Summary PCI is a refinement of PCI’ PVS proof All traces of PCI on all configs satisfy PC. Four network topologies in n All traces of PCI’ on all topologies satisfy PC. Murphi model check

Hierarchical caching networks wr(A,2) rd(A,-) rd(A,1) rd(A,0) E wr(A,0) wr(A,1) M M2 M0 M1 P Q Model Checker

Model Checking Results States CPU time (sec) P P Q 110,995 87.57 P Q P 151,598 65.51 P P Q * 618,874 282.40 Total 881,467 435.48

Discussion and Future Work Abstraction technique that yields a finite state model which preserves enough information to reason about useful properties in networks where the behavior and arrangement of the intermediate nodes matters. General refinement proof and tool. www.cs.utah.edu/formal_verification

State Transitions

State Transitions

Producer/Consumer for PCI ... p c d f ...for all networks and all executions.

Abstracting PCI Networks D F

Abstracting PCI Networks D F P C D F

Abstracting PCI Networks D C F P C D F P C D F P F D C P F D C

Abstracting PCI Messages dwc c p d dwc c dc d dwc d dw d d p fw P ... ... p p c cdw dwc p c dwc dc c ... dwc d dw d p fw P d d d c ... p p c cdw

Abstracting PCI Messages dwc p c dwc dc c dwc d dw d d p fw P ... ... d d d c p p c cdw dwc dwc ... dwc dw fw P cdw

Abstracting PCI Messages dwc p c dwc dc c dwc d dw d d p fw P ... ... d d d c p p c cdw dwc dw fw P cdw

Solution #1 Proofs of obvious lemmas hard: All traces, all configs. satisfy P/C PVS proof PCI model Proofs of obvious lemmas hard: “if a message is present in a queue, then it was created previously”

Solution #2 1 SPIN some traces in these Model Check configs. satisfy P/C. 2 3

Posted transactions Pretend there are 2 bridges between A and B With the other transaction shown. Here’s how P gets from A to B... big animation of a posted transaction. Highlighting that a P trans is complete at the master when it leaves. p d c p’ B A

Posted transactions P goes to bridge 1. P is now complete at A. P can pass delayed transaction d big animation of a posted transaction. Highlighting that a P trans is complete at the master when it leaves. p d c p’ B A

Posted transactions Next, P completes to bridge 2. p d c p’ B A big animation of a posted transaction. Highlighting that a P trans is complete at the master when it leaves. p d c p’ B A

Posted transactions P is now complete at bridge 1. P can pass the completion trans. C. P can not pass the other posted trans. big animation of a posted transaction. Highlighting that a P trans is complete at the master when it leaves. p d c p’ B A

Posted transactions P waits until P’ completes on bridge 2 p d c p’ B big animation of a posted transaction. Highlighting that a P trans is complete at the master when it leaves. p d c p’ B A

Posted transactions Pretend that P’ went to another bridge (not shown). P can now complete to destination B. big animation of a posted transaction. Highlighting that a P trans is complete at the master when it leaves. p d c B A

Posted transactions No acknowledgement is sent to A. P is now complete at B. big animation of a posted transaction. Highlighting that a P trans is complete at the master when it leaves. p d c B A