1 A Compositional Approach to Verifying Hierarchical Cache Coherence Protocols Xiaofang Chen 1 Yu Yang 1 Ganesh Gopalakrishnan 1 Ching-Tsun Chou 2 1 University.

Slides:



Advertisements
Similar presentations
Functional Decompositions for Hardware Verification With a few speculations on formal methods for embedded systems Ken McMillan.
Advertisements

Copyright 2000 Cadence Design Systems. Permission is granted to reproduce without modification. Introduction An overview of formal methods for hardware.
Implementation and Verification of a Cache Coherence protocol using Spin Steven Farago.
Tintu David Joy. Agenda Motivation Better Verification Through Symmetry-basic idea Structural Symmetry and Multiprocessor Systems Mur ϕ verification system.
Modular and Verified Automatic Program Repair Francesco Logozzo, Thomas Ball RiSE - Microsoft Research Redmond.
1 Lecture 6: Directory Protocols Topics: directory-based cache coherence implementations (wrap-up of SGI Origin and Sequent NUMA case study)
Hierarchical Cache Coherence Protocol Verification One Level at a Time through Assume Guarantee Xiaofang Chen, Yu Yang, Michael Delisi, Ganesh Gopalakrishnan.
Department of Computer Sciences Revisiting the Complexity of Hardware Cache Coherence and Some Implications Rakesh Komuravelli Sarita Adve, Ching-Tsun.
Model-based reasoning meets code verification Michael Butler 21 May 2014 WG 2.3 Meeting 55, Orlando.
Transaction Based Modeling and Verification of Hardware Protocols Xiaofang Chen, Steven M. German and Ganesh Gopalakrishnan Supported in part by SRC Contract.
Transaction Based Modeling and Verification of Hardware Protocols Xiaofang Chen, Steven M. German and Ganesh Gopalakrishnan Supported in part by Intel.
Compositional reasoning for Parameterized Verification Murali Talupur Joint work with Sava Krstic, John O’leary, Mark Tuttle.
Model Checking : Making Automatic Formal Verification Scale Shaz Qadeer EECS Department University of California at Berkeley.
6/14/991 Symbolic verification of systems with state machines David L. Dill Jeffrey Su Jens Skakkebaek Computer System Laboratory Stanford University.
Background information Formal verification methods based on theorem proving techniques and model­checking –to prove the absence of errors (in the formal.
CIS 540 Principles of Embedded Computation Spring Instructor: Rajeev Alur
The Stanford Directory Architecture for Shared Memory (DASH)* Presented by: Michael Bauer ECE 259/CPS 221 Spring Semester 2008 Dr. Lebeck * Based on “The.
Using Statically Computed Invariants Inside the Predicate Abstraction and Refinement Loop Himanshu Jain Franjo Ivančić Aarti Gupta Ilya Shlyakhter Chao.
Slide 0 FMCAD 2004 How To Specify and Verify Cache Coherence Protocols: An Elementary Tutorial Ching-Tsun Chou Microprocessor Technology Lab Corporate.
1 Scaling Formal Methods toward Hierarchical Protocols in Shared Memory Processors: Annual Review Presentation – April 2007 Presenters: Ganesh Gopalakrishnan.
STARI: A Case Study in Compositional and Hierarchical Timing Verification Serdar Tasiran, Prof. Robert K. Brayton Department of Electrical Engineering.
Scaling Formal Methods Toward Hierarchical Protocols in Shared Memory Processors Presenters: Ganesh Gopalakrishnan and Xiaofang Chen School of Computing,
Abstractions. Outline Informal intuition Why do we need abstraction? What is an abstraction and what is not an abstraction A framework for abstractions.
Thread-modular Abstraction Refinement Tom Henzinger Ranjit Jhala Rupak Majumdar [UC Berkeley] Shaz Qadeer [Microsoft Research]
Verification of Hierarchical Cache Coherence Protocols for Future Processors Student: Xiaofang Chen Advisor: Ganesh Gopalakrishnan.
Scaling Formal Methods toward Hierarchical Protocols in Shared Memory Processors Intel SRC Customization Award 2005-TJ-1318 Ganesh Gopalakrishnan* School.
1 Scaling Formal Methods Toward Hierarchical Protocols in Shared Memory Processors Joint work with Xiaofang Chen (PhD student) Ching-Tsun Chou (Intel Corporation,
Scaling Formal Methods Toward Hierarchical Protocols in Shared Memory Processors Presenters: Ganesh Gopalakrishnan and Xiaofang Chen School of Computing,
Utah Verifier Group Research Overview Robert Palmer.
Computing Over­Approximations with Bounded Model Checking Daniel Kroening ETH Zürich.
Operational Semantics Semantics with Applications Chapter 2 H. Nielson and F. Nielson
1 Lecture 3: Directory-Based Coherence Basic operations, memory-based and cache-based directories.
Counterexample Guided Invariant Discovery for Parameterized Cache Coherence Verification Sudhindra Pandav Konrad Slind Ganesh Gopalakrishnan.
1 Reducing Verification Complexity of a Multicore Coherence Protocol Using Assume/Guarantee Xiaofang Chen 1, Yu Yang 1, Ganesh Gopalakrishnan 1, Ching-Tsun.
Transaction Based Modeling and Verification of Hardware Protocols Xiaofang Chen, Steven M. German and Ganesh Gopalakrishnan Supported in part by SRC Contract.
Slide 0 FMCAD 2004 A Simple Method for Parameterized Verification of Cache Coherence Protocols Ching-Tsun Chou Phanindra K. Mannava Seungjoon Park Microprocessor.
By D. Beyer et. al. Simon Fraser University (Spring 09) Presentation By: Pashootan Vaezipoor.
Thread-modular Abstraction Refinement Thomas A. Henzinger, et al. CAV 2003 Seonggun Kim KAIST CS750b.
Automatic Abstraction Refinement for GSTE Yan Chen, Yujing He, and Fei Xie Portland State University Jin Yang Intel Nov 13, 2007.
A Simple Method for Extracting Models from Protocol Code David Lie, Andy Chou, Dawson Engler and David Dill Computer Systems Laboratory Stanford University.
1 Automatic Non-interference Lemmas for Parameterized Model Checking Jesse Bingham, Intel DEG FMCAD 2008.
B. Fernández, D. Darvas, E. Blanco Formal methods appliedto PLC code verification Automation seminar CERN – IFAC (CEA) 02/06/2014.
Yang Liu, Jun Sun and Jin Song Dong School of Computing National University of Singapore.
Race Checking by Context Inference Tom Henzinger Ranjit Jhala Rupak Majumdar UC Berkeley.
Ronny Krashinsky Erik Machnicki Software Cache Coherent Shared Memory under Split-C.
Going with the Flow Parameterized Verification using Message Flows Murali Talupur & Mark Tuttle SCL, Intel Corporation.
Lecture 5 1 CSP tools for verification of Sec Prot Overview of the lecture The Casper interface Refinement checking and FDR Model checking Theorem proving.
11 Counter-Example Based Predicate Discovery in Predicate Abstraction Satyaki Das and David L. Dill Computer Systems Lab Stanford University
HARD: Hardware-Assisted lockset- based Race Detection P.Zhou, R.Teodorescu, Y.Zhou. HPCA’07 Shimin Chen LBA Reading Group Presentation.
SAT-Based Model Checking Without Unrolling Aaron R. Bradley.
Formal Verification. Background Information Formal verification methods based on theorem proving techniques and model­checking –To prove the absence of.
Synergy: A New Algorithm for Property Checking Bhargav S. Gulavani (IIT Bombay)‏ Yamini Kannan (Microsoft Research India)‏ Thomas A. Henzinger (EPFL)‏
Parosh Aziz Abdulla 1, Mohamed Faouzi Atig 1, Zeinab Ganjei 2, Ahmed Rezine 2 and Yunyun Zhu 1 1. Uppsala University, Sweden 2. Linköping University, Sweden.
CS357 Lecture 13: Symbolic model checking without BDDs Alex Aiken David Dill 1.
/ PSWLAB Thread Modular Model Checking by Cormac Flanagan and Shaz Qadeer (published in Spin’03) Hong,Shin Thread Modular Model.
A Calculus of Atomic Actions Tayfun Elmas, Shaz Qadeer and Serdar Tasiran POPL ‘ – Seminar in Distributed Algorithms Cynthia Disenfeld 27/05/2013.
September 1999Compaq Computer CorporationSlide 1 of 16 Verification of cache-coherence protocols with TLA+ Homayoon Akhiani, Damien Doligez, Paul Harter,
Gauss Students’ Views on Multicore Processors Group members: Yu Yang (presenter), Xiaofang Chen, Subodh Sharma, Sarvani Vakkalanka, Anh Vo, Michael DeLisi,
Compositional Verification for System-on-Chip Designs SRC Student Symposium Paper 16.5 Nishant Sinha Edmund Clarke Carnegie Mellon University.
Verifying Component Substitutability Nishant Sinha Sagar Chaki Edmund Clarke Natasha Sharygina Carnegie Mellon University.
CIS 540 Principles of Embedded Computation Spring Instructor: Rajeev Alur
1 Lecture 8: Snooping and Directory Protocols Topics: 4/5-state snooping protocols, split-transaction implementation details, directory implementations.
Lecture 8: Snooping and Directory Protocols
Xiaofang Chen1 Yu Yang1 Ganesh Gopalakrishnan1 Ching-Tsun Chou2
Opeoluwa Matthews, Jesse Bingham, Daniel Sorin
Michael D. Jones, Ganesh Gopalakrishnan
Cache Coherence (controllers snoop on bus transactions)
Property Directed Reachability with Word-Level Abstraction
Over-Approximating Boolean Programs with Unbounded Thread Creation
Predicate Abstraction
Presentation transcript:

1 A Compositional Approach to Verifying Hierarchical Cache Coherence Protocols Xiaofang Chen 1 Yu Yang 1 Ganesh Gopalakrishnan 1 Ching-Tsun Chou 2 1 University of Utah 2 Intel Corporation * Supported in part by Intel SRC Customization Award 2005-TJ-1318

FMCAD Hierarchical Cache Coherence Protocols Chip-level protocols Inter-cluster protocols Intra-cluster protocols dir mem dir mem …

FMCAD Verification Challenges  No public domain benchmarks  More complicated with more Corner cases State space

FMCAD Outline  Two hierarchical protocols Inclusive Non-inclusive  A compositional approach Abstraction Counter-example guided refinement Soundness

FMCAD A Multicore Coherence Protocol RAC L2 Cache+Local Dir L1 Cache L1 Cache Global Dir Main Memory Home ClusterRemote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir L1 Cache L1 Cache

FMCAD Protocol Features  Both levels use MESI protocols Level-1: FLASH Level-2: DASH  Silent drop on non-Modified cache lines  Network channels are non-FIFO

FMCAD Livelock Problem Dir Agent1Agent2 1. Req_E 2. Grant_E 4. Req_S 3. Silent-drop 5. Fwd_Req6. NACK Invld Excl

FMCAD Blocking WB + NACK_SD Dir A1A2 Req_E Gnt_E Req_S Modify WB Fwd_S WB_Ack NAck_SD NAck (I) (E) (M) (I)

FMCAD Complexity of the Protocol  Multiplicative effect of four protocols running concurrently  Model check failed after 161,876,000 of states

FMCAD Outline  Two hierarchical protocols Inclusive Non-inclusive  A compositional approach Abstraction Counter-example guided refinement Soundness

FMCAD A Compositional Approach Constraining Original protocol Abstraction … Abstracted protocol

FMCAD Non-Circular Assume/Guarantee  We can’t Verify: h ║ r1 ║ r2 ╞ Coh  Instead Check-1: h ║ R1 ║ R2 ╞ Coh1 Λ Constrains1 Check-2: H ║ r1 ║ R2 ╞ Coh2 Λ Constrains2

FMCAD Verification Methodology  Abstraction Two abstracted protocols  Fixing real bugs in M  Refinement

FMCAD Abstracted Protocol #1 RAC L2 Cache+Local Dir’ Global Dir Main Memory Home Cluster Remote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir’

FMCAD Abstracted Protocol #2 RAC L2 Cache+Local Dir’ Global Dir Main Memory Home Cluster Remote Cluster 1 Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir’

FMCAD Abstraction  States Projection  Transitions Overapproximation

FMCAD Abstraction on States Intra-cluster details Inter-cluster details

FMCAD Abstracting Transitions  Rule-based system: guard  action; Relaxing guards Relaxing expr values Remove stmt Procs[p].WbMsg.Cmd = WB_Wb → Procs[p].L2.Data := Procs[p].WbMsg.Data; Procs[p].L2.HeadPtr := L2; … true → Procs[p].L2.Data := d; …

FMCAD Detecting Bugs in M  When a real error is found in M i Fix bug in M Regenerate M i ’s Iterate the process

FMCAD Refinement  When a bogus error found in M i Analyze and find out problematic rule g → a Locate original rule in M G → A Add a new lemma in one abstracted protocol G => P Strengthen rule into g Λ P → a

FMCAD M1M1 1. False alarm found Remote cluster-1 can modify its L2 line arbitrarily Details of Refinement (I) true → …

FMCAD Locate the original rule in M before abstraction Guard: when the local dir receives a WB from an L1 cache Details of Refinement (II) 1 M1M1 Procs[p].WbMsg.Cmd = WB → …

FMCAD Strengthen problematic rule in 1. Only when local dir is exclusive, could L2 modify its line Details of Refinement (III) 1 M1M1 3 true & Procs[p].L2.State = Excl → …

FMCAD Why strengthening is sound? Details of Refinement (IV) 1 M1M1 3

FMCAD We can add a new lemma in M 2 Details of Refinement (V) M1M1 1 3 M2M2 4 Procs[p].WbMsg.Cmd = WB => Procs[p].L2.State = Excl

FMCAD One Detail Excl: 1 Home Cluster Remote Cluster 1Remote Cluster 2 Excl Invld Req_E2 Req_E3 Fwd_ReqE 4 Fwd_ReqE5 Gnt_E

FMCAD Original Transitions (I) GUniMsg[src].Cmd = RDX_RAC & GUniMsg[src].Cluster = r & Procs[r].L2.Gblock_WB = false & Procs[r].L2.State = Excl & Procs[r].L2.HeadPtr != L2  … undefine GUniMsg[src]; GUniMsg[src].Cmd := GUNI_None;

FMCAD Original Transitions (II) Procs[r].ShWbMsg.Cmd = SHWB_FAck & src_node = L2  … true & A BSProcs[r].L2.State = Excl & ABSProcs[r].RAC.State = Inval & ABSProcs[r].L2.Gblock_WB = false & GUniMsg[src].Cmd = RDX_RAC & GUniMsg[src].Cluster = p  …

FMCAD Adding A Variable Excl: 1 Home Cluster Remote Cluster 1Remote Cluster 2 Excl Invld ifKeepMsg: boolean

FMCAD Soundness of the Approach  Goal If M 1 and M 2 can be model checked correct w.r.t. the coherence property Ф in M, M must also be correct w.r.t Ф

FMCAD Soundness Proof  Temporal Induction Initial states  Each var has the same value in M, M 1 and M 2  Each newly added lemma is checked in M 1 and M 2  Each property is checked Suppose soundness in state s

FMCAD Soundness Proof (II) h1, h2, r11, r12, r21, r22 h1, h2, r12, r22 h1, r11, r12, r22 h1’, h2’, r11’, r12’, r21’, r22’ g  a g 1 & p 1  a 1 h1’, h2’, r12’, r22’ g 2 & p 2  a 2 h2’, r11’, r12’, r22’ M M1M1 M2M2

FMCAD Experiment Results  A real bug found  10 iterations of refinements The size of each error trace is < 12 One person-day of work

FMCAD ProtocolNumber of states M> 161,876,000 M1M1 31,919,219 M2M2 78,689,678 Reduction  64-bit Murphi  IA-64 with 20GB of memory

FMCAD Outline  Two hierarchical protocols Inclusive Non-inclusive A compositional approach Abstraction Counter-example guided refinement Soundness

FMCAD Caching Hierarchy  Inclusive  Exclusive  Non-inclusive

FMCAD A Non-Inclusive Hierarchical Protocol RAC L2 Cache+Local Dir L1 Cache L1 Cache Global Dir Main Memory Home ClusterRemote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir L1 Cache L1 Cache

FMCAD Protocol Differences  Broadcasting channels RAC L2 Cache+Local Dir L1 Cache L1 Cache SnoopMsg[]

FMCAD Imprecise Local Directory LDir L1-1 GDir Req_S (S) S: L1-1 L1-2 (I) Swap Broadcast NAck Fwd_Req Gnt_S S: L1-2 Imprecision!

FMCAD Verification Difficulty  Coherence properties Can involve multiple L1 caches  Refinement Noninterference lemmas cannot infer L2 cache line states, from local behaviors

FMCAD An Example Excl Invld Excl Invld WB L2: (Excl, data1)  (Excl, data2) L2: (Invld, *)  (Excl, data2)

FMCAD Two Approaches of Refinement  Inferring “exclusive” from Outside the cluster Inside the cluster

FMCAD Infer exclusive From Outside Invld Excl Invld WB L2: (Invld, *)  (Excl, data2) IsExcl(p) Ξ Dir.State = Excl & GUniMsg[p].Cmd != (ACK || IACK || ImACK) & GUniMsg[h].Cmd != (ACK || IACK || ImACK) & GWbMsg.Cmd = GWB_None & ( (GShWbMsg.Cmd = GSHWB_None & Dir.Headptr = p) || (GShWbMsg.Cmd = DXFER & GShWbMsg.Cluster = p)) Cluster p

FMCAD Refinement Example Invld Excl Invld WB L2: (Invld, *)  (Excl, data2) Cluster p p.WbMsg.Cmd = WB => IsExcl(p) (Invld & IsExcl(p), *)  (Excl, data2)

FMCAD Infer exclusive From Inside M1M1 M2M2

FMCAD Definition of IE IE(p): exists i: L1_caches (p.L1(i).state = Excl or p.SnoopMsg(i).Cmd = (Put or PutX) or p.UniMsg(i).Cmd = PutX) or p.WbMsg.Cmd = WB or p.ShWbMsg.Cmd = ShWb or p.ShWbMsg.Cmd = FAck

FMCAD Refinement Invld Excl Invld WB L2: (Invld, *)  (Excl, data2) Cluster p Procs[p].WbMsg.Cmd = WB & Procs[p].L2.Stae = Invld => IE(p) (Invld & IE(p), *)  (Excl, data2)

FMCAD Soundness  Still holds by adding the extra bits “IE”

FMCAD Experiment Results  17 iterations of refinements  Size of each error trace is < 8 ProtocolNumber of states M> 1,521,900,000 M1M1 234,478,105 M2M2 283,124,383

FMCAD Outline Two hierarchical protocols Inclusive Non-inclusive A compositional approach Abstraction Counter-example guided refinement Soundness

FMCAD Conclusion  Developed 2-level hierarchical protocols  Proposed a compositional approach Abstraction Bug fixing Refinement  Proved the soundness

FMCAD Related Work  FMCAD’04 Chou et. al., A simple method for parameterized verification of cache coherence protocols  CHARME’99 McMillan, Verification of infinite state systems by compositional model checking

FMCAD For Details

FMCAD A Multicore Coherence Protocol RAC L2 Cache+Local Dir L1 Cache L1 Cache Global Dir Main Memory Home ClusterRemote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir L1 Cache L1 Cache

FMCAD About the Bug IACK

FMCAD Another Decomposing Approach  Split protocols hierarchically Intra-cluster protocol Inter-cluster protocol

FMCAD Intra-cluster Protocol RAC L2 Cache+Local Dir L1 Cache L1 Cache Cluster Environment

FMCAD Inter-cluster Protocol RAC L2 Cache+Local Dir’ Global Dir Main Memory Home ClusterRemote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir’ RAC L2 Cache+Local Dir’

FMCAD Verification Difficulty Environment RAC L2 Cache+Local Dir L1 Cache L1 Cache Global Dir Main Memory Home ClusterRemote Cluster 1Remote Cluster 2 RAC L2 Cache+Local Dir L1 Cache L1 Cache RAC L2 Cache+Local Dir L1 Cache L1 Cache

FMCAD An Example Scenario Excl: 1 Home Cluster Remote Cluster 1Remote Cluster 2 Excl Invld NACK 1 Req_E2 Req_E3 Fwd_ReqE 4 Swap5 Req_E6 Fwd_ReqE 7