Stop Writing Assertions! Efficient Verification Methodology Dave Whipp – DVClub SV, August 2008

NVIDIA Confidential Todays Talk An Overview of Methodology Creation Methodology as User Interface Making ESL work as a Hardware Design Flow Evolution of a Interface Definition Language Transaction Level Assertions Transaction Level Debug

NVIDIA Confidential A HW Development Flow Big Paper Spec Design Verification RTL Testbench Checkers Tests Scoreboard Assertions C Model Synthesis Directed Random BFMs, TLMs Clocks, Resets Assertions ISS Model Coverage Debug Formal

NVIDIA Confidential User Model for Running Tests Build Testbench Generate Test Run Predictor Run DUT Compare Behaviors *

Part 1 Methodology Design

NVIDIA Confidential Purpose of a Flow Conceptual Framework Where do things live? How do things work? Keep out of the way Don’t make life difficult for people Define Metaphors Subtly influence future directions Anchor for Variation Give people something to complain about

NVIDIA Confidential Two Philosophies Conformity Emphasize Standardization Diversity Emphasize Innovation Synergy or Conflict?

NVIDIA Confidential Understanding Variation To understand what to standardize: you need to understand what not to standardize Personal Preferences Technical Aspects of the Designs Supporting Legacy Seeking the Next Big Thing

NVIDIA Confidential Personal Preferences Emacs! vi

NVIDIA Confidential Personal Preferences Choice of editor doesn’t affect others At least, not much Choice of scripting language has greater impact But is encapsulated A script’s users don’t see the implementation language Choice of HVL affects whole team Can’t write “E” tests for a “Vera” testbench! But a unit testbench isn’t seen by other units A good flow will allow encapsulation of preferences I can go to any unit and build & run its tests Enables rapid localization of infrastructure issues

NVIDIA Confidential Technological Differences

NVIDIA Confidential Technical Characteristics PCIEPCIE Video Frame Buffer Off-chip Memory Graphics Pipe

NVIDIA Confidential Reuse Vs Stagnation Reuse considered Good Avoid reinventing the wheel Build on the shoulders of giants Reuse invites Inertia Reuse can propagate dependencies Dependencies make things harder to change Resistance to change is known as inertia Inertia can lead to Stagnation Improper reuse accumulates dependencies Reused code that is not understood will bit-rot To avoid stagnation, inject agitation

NVIDIA Confidential Are Single Paradigm Projects Possible? Project 1 Unit A Unit B Unit C Project 2Project 3 Unit D Unit B Unit C Unit B Paradigm 1 Paradigm 2 Paradigm 3 time

NVIDIA Confidential Watch some Real Users Image courtesy of DAZ Productions

NVIDIA Confidential How to “Watch” Users Meetings and Discussions Coffee-Break Grousing Bug Reports Keep Track of Support Requests create FAQs VNC (Remote Desktop) Instrumentation

NVIDIA Confidential Build Time Distribution: 10,000 per sample

NVIDIA Confidential Build Time Distribution: 1000 per sample

Part 2 ESL for Transaction Design

NVIDIA Confidential A HW Development Flow Big Paper Spec Design Verification RTL Testbench Checkers Tests Scoreboard Assertions C Model Synthesis Directed Randoms BFMs, TLMs Clocks, Resets Assertions ISS Model Coverage Debug (BAD)

NVIDIA Confidential A HW Development Flow Big Paper Spec Design Verification RTL Testbench Checkers Tests Scoreboard Assertions C Model Synthesis Directed Randoms BFMs, TLMs Clocks, Resets Assertions ISS Model Coverage Debug (BAD)

NVIDIA Confidential A HW Development Flow (Better) RTL Scoreboards Synthesis Randoms Clocks, Resets Design Verification Testbench Small Paper Spec ISS ModelESL C Model Interfaces Assertions Directed Tests BFMs TLMs Validation Assertions Triage Debug Coverage

NVIDIA Confidential Who Writes Assertions? Designers Bottom Up Assumptions Verification Engineers Top-down Intent

NVIDIA Confidential Who Writes Assertions? Designers Bottom Up Assumptions Verification Engineers Top-down Intent Architects The Specification Top Down Assumptions Bottom Up Intent

NVIDIA Confidential Where to Write Assertions The RTL Inline Bound The Testbench Scoreboard Environment E.g. Post Process Log file

NVIDIA Confidential Where to Write Assertions The RTL Inline Bound The Testbench Scoreboard Environment Post Process Log files The Specification C Models (?)

NVIDIA Confidential Where To Write Specification Assertions Functionality Model Performance Model RTL Design

NVIDIA Confidential Where To Write Specification Assertions Transaction Model Performance Model RTL Design Correlation Model ISS Model Debug/Triage Model

NVIDIA Confidential Where To Write Specification Assertions Performance Model RTL Design Structural Model Transaction Model

NVIDIA Confidential Interfaces Vs State Two approaches to comparing models: Compare “Architectural State” Registers/flops within the design whose existence is required by the specification Compare externally visible behavior Compare interface traffic B. F. Skinner?

NVIDIA Confidential Birth of an IDL Interface Description Language Initially, a language just to define signals Interface a2b clock clk down U valid 1 up U busy 1 down U cmd 24 down U data 32

NVIDIA Confidential Evolution of an IDL Quickly added flow-control protocol abstraction Interface a2b clock clk flow valid_busy down U cmd 24 down U data 32 From this we can generate: Testbench components (BFMs: producers, consumers) Protocol Assertions …

NVIDIA Confidential Continued Evolution of an IDL Separation of packet structure from interface group SOP down U cmd 24 group MOP down U data 32 group EOP down U checksum 32 Interface a2b clock clk flow valid_busy packet SOP, MOP, EOP

NVIDIA Confidential Transaction Assertions in YACC valid_interface_traffic: | valid_interface_traffic transaction; transaction: begin middle end; begin: SOP; middle: | middle MOP; end: EOP

NVIDIA Confidential Cycle Level Assertions in SVA sequence valid_trans; (cmd==SOP) (##1 cmd != SOP && cmd != EOP) [*0:$] ##1 cmd == EOP endsequence a_well_formed_transaction: clk) cmd == SOP |-> sequence (valid_trans) clk cmd SOPEOP MOP

NVIDIA Confidential Transaction Level Assertions in SVA sequence valid_transaction; (cmd==SOP) (##1 cmd != SOP && cmd != EOP) [*0:$] ##1 cmd == EOP endsequence clk valid cmd SOPEOP MOP

NVIDIA Confidential Transaction Level Assertions in SVA event sample; clk) if (valid && ! busy) -> sample cmd == SOP |-> sequence (valid_transaction) clk valid busy cmd SOPEOP MOP

NVIDIA Confidential Transaction Level Assertions in SVA event sample; clk) if (valid && ! busy) -> sample cmd == BEGIN |-> sequence (valid_transaction) clk valid busy cmd SOPEOP MOP

NVIDIA Confidential Grammer using IDL assert “temporal representation” data SOP => past( EOP ) data MOP => past( SOP | MOP ) data EOP => past( SOP | MOP ) assert “bnf representation” bnf SOP -> MOP ** past( SOP.length-1 :sample( SOP ) ) -> EOP

NVIDIA Confidential Multi-Unit Assemblies AEBCDFG a2bb2cc2dd2ee2ff2g A simple pipeline

NVIDIA Confidential Multi-Unit Assemblies A E BC D F G a2bb2cc2d d2e e2f f2g Simple rearrangement

NVIDIA Confidential Multi-Unit Assemblies A E BC D F G a2bb2cc2d d2e e2f f2g Identify units with similar behaviors

NVIDIA Confidential Multi-Unit Assemblies BEBE CFCF D a2be be2cf cf2d d2be cf2g G A Extract common behavior into unified components be2cf === b2c + e2f

NVIDIA Confidential Reusing Interface Definitions AEBCDFG D G A BEBE CFCF How to maximize reuse between these two architectures?

NVIDIA Confidential Packets as Traffic Streams group b2c down U data 32 group e2f down U data 32 Interface be2cf clock clk flow valid_credit packet b2c, e2f

NVIDIA Confidential Time Units of Temporal Expressions Group b2c down U value 4 assert value != past( value ) Group e2f down U value 4 assert ( value == 0 ) => ( past( value ) != 0 ) Interface be2cf packet b2c, e2f assert b2c => ( b2c.value != past( b2c.value :sample(b2c) ) ) assert past( e2f && e2f.value == 0 ) => (b2c && b2c.value != 0)

NVIDIA Confidential Example RendererMemory

NVIDIA Confidential The Traffic group mem_write down U address 16 down U data 1 group sync down U shape 2 enum SQUARE, CIRCLE, TRIANGE, BLANK down U radius 3

NVIDIA Confidential Accumulate Memory State group mem_write down U address 16 down U data 1 assign mem[ x = ][ y = ] = past( data :sample( address == {x,y} ))

NVIDIA Confidential Add Predicates group_more mem_write assign is_circle[ r = 0..7 ] = “&&”( [ x = ] [ y = ] mem[ x+8 ][ y+8 ] == ( x**2 + y**2 <= r**2 ) )

NVIDIA Confidential Define the interface interface render2memory clock mem_clk flow valid_busy packet sync, mem_write assert “correct sync shape” sync && sync.shape == CIRCLE => past( mem_write.is_circle[ sync.radius ] )

NVIDIA Confidential Example RendererMemory assert “correct sync shape” sync && sync.shape == CIRCLE => past( mem_write.is_circle[ sync.radius ] )

NVIDIA Confidential Summary Efficient Verification Understand Variation Encourage Diversity avoid anarchy Encourage Standardization avoid stagnation Frontload The Schedule Provide Verification with Executable Spec (ESL) C Model, Correlation Model (pre validated) Interfaces, Transactions (definitions for code generators) Assertions, Testpoints (neutral language, pre validated)

Questions

NVIDIA Confidential Summary Architects should write assertions Validated assertions are input to Verification Assertions must be directly reusable across models Manual recoding invites errors Explicitly model the structure that is common to architectural models and to design Tie assertions to these common points

NVIDIA Confidential Threading Models Thread Follows Resource Traditional SystemC approach Thread Follows Transaction Think “Multithreaded ISS” Cycle-based Model The “Old” way of doing things A non-threaded version of “thread follows resource” Different models are convenient at various times But SystemC forces you to commit early to one approach Therefore need extra code for transaction level debug

NVIDIA Confidential 70% 70% of schedule is Verification What is correct figure 0% synthesize architectural models, constraints Correct by construction! 100% synthesize verification models, constraints no design needed!

NVIDIA Confidential Transactions Vs Cycles Data min_val (Addr a1, Addr a2) { Data d1 = mem_read(a1); Data d2 = mem_read(a2); if (d1 < d2) return d1; else return d2; } Address Data a1 d1 Pipelined Bus a2 d2 t1 t2 t3 t4 t5 t6

NVIDIA Confidential Unit Level System Testing Stage N-1 Stage N+1 Stage N Graphics Pipeline (Transaction Model) RTL Unit DIFF

NVIDIA Confidential New Challenges New Tools New Platforms New People New Ideas Refactoring Testability D.R.Y. Avoiding Stagnation