1 Reducing Verification Complexity of a Multicore Coherence Protocol Using Assume/Guarantee Xiaofang Chen 1, Yu Yang 1, Ganesh Gopalakrishnan 1, Ching-Tsun Chou 2 1 University of Utah, 2 Intel Corporation

2 Hierarchical Cache Coherence Protocols Chip-level protocols Inter-cluster protocols Intra-cluster protocols dir mem dir mem …

3 Verification Challenges More complicated –More corner cases –More state space No public domain benchmarks

4 A Multicore Coherence Protocol RAC L2 Cache+Local Dir L1 Cache L1 Cache Global Dir Main Memory Home ClusterRemote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir L1 Cache L1 Cache

5 Protocol Features Modeling one address Both levels use MESI protocols –Level-1: Flash –Level-2: Dash Silent drop on non-Modified cache lines Network channels are non-FIFO

6 An Example Scenario Excl: 1 Home Cluster Remote Cluster 1Remote Cluster 2 Excl Invld dst Req_Ex 2 Req_Ex 3 Fwd_ReqEx 4.1 Fwd_ReqEx 4.2 Silent-drop 5 NACK

7 Complexity of the Protocol Multiplicative effect of four protocols running concurrently Model check failed after 161,876,000 of states

8 Intuitively, We Want to … Compositional approach –Split a large protocol into several smaller ones –Verify correctness of smaller protocols –Using compositional methods to prove the correctness of the large protocol

9 For the 2-Level Protocol Build two simpler protocols M 1 and M 2 from M By model checking M 1 and M 2, we can conclude if M is coherent

10 Abstracted Protocol #1 RAC L2 Cache+Local Dir’ Global Dir Main Memory Home Cluster Remote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir’

11 Abstracted Protocol #2 RAC L2 Cache+Local Dir’ Global Dir Main Memory Home Cluster Remote Cluster 1 Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir’

12 Methodology Abstraction –Simple projection Fixing real bugs in M Refinement –Counter-example guided refinement –Adding new verification obligations (VO)

13 Abstraction on States: Projection Intra-cluster details Inter-cluster details

14 Abstracting Transitions Rule-based system: guard  action; Abstraction = overapproximation –Relaxing guards –Relaxing expr values –Remove stmt –…

15 Example of Abstracting Transitions Procs[p].WbMsg.Cmd = WB_Wb → Procs[p].L2.Data := Procs[p].WbMsg.Data; Procs[p].L2.HeadPtr := L2; … true → Procs[p].L2.Data := d; …

16 Detecting Bugs in M When a real error is found in M i –Fix bug in M –Regenerate Mi’s –Iterate the process

17 Refinement When a bogus error found in M i –Analyze and find out problem rule g → a –Locate original rule in M G → A –Add new lemma in one abstracted protocol G => I –Strengthen rule into g Λ I → a

18 Home Cluster Remote Cluster 1 Remote Cluster 2 1 M1M1 1. False alarm found –Remote cluster-1 can modify its L2 line arbitrarily true → … Details of Refinement (I)

19 Home Cluster Remote Cluster 1 Remote Cluster 2 1 M1M1 2. Locate the original rule in M before abstraction –It is only enabled when the local dir receives a WB from an L1 cache Procs[p].WbMsg.Cmd = WB → … Details of Refinement (II)

20 Home Cluster Remote Cluster 1 Remote Cluster 2 1 M1M1 3. Strengthen problematic rule in 1. –Only when local dir is exclusive, could L2 modify its line 3 Procs[p].L2.State = Excl → … Details of Refinement (III)

21 Home Cluster Remote Cluster 1 Remote Cluster 2 1 M1M1 4. Why strengthening is sound? –Need to prove whenever local dir receives WB, L2 cache is exclusive –But how? 3 Details of Refinement (IV)

22 Home Cluster Remote Cluster 1 Remote Cluster 2 Home Cluster Remote Cluster 1 Remote Cluster 2 1 M1M1 M2M2 4. We can add a new VO (lemma) in M 2, Procs[p].WbMsg.Cmd = WB Procs[p].L2.State = Excl 3 4 Details of Refinement (V)

23 Experiment Results A real bug found 10 iterations of refinements –The size of each error trace is < 12 –One human/day of work

24 Soundness of the Approach Theorem –If M 1 and M 2 can be model checked correct w.r.t. the coherence property Ф in M, M must also be correct w.r.t Ф

25 Soundness Proof Temporal Induction –Initial states Each var has the same value in M, M 1 and M 2 Each newly added VO is checked in M 1 and M 2 Refinement is sound –Suppose soundness in state s –For every next state s’ of s, easy to prove Each var has the same value Refinement is sound

26 Reduction ProtocolNumber of states M > 161,876,000 M1M1 31,919,219 M2M2 78,689, bit Murphi, with 16GB of memory

27 More Reduction Another 2-level hierarchical cache coherence protocol ProtocolNumber of states M> 1,521,900,000 M1M1 234,478,105 M2M2 283,124,383

28 Conclusion Developed a 2-level hierarchical protocol Proposed a compositional approach –Abstraction –Bug fixing –Refinement Proved the soundness

29 Thank you!

30 Outline Background Benchmarks Abstraction Counter-example guided refinement Experiments Soundness Conclusion