Pre-verified IP? Metrics and methods to determine if your IP has been pre-verified’ Moshik Rubin

Agenda The end customer profile and the resulting challenges Introduction to Metric Driven Verification with OVM Metric Driven Verification IPs Summary 2 September 19, 2018 dsfdsfdsafaf 2

Our Customer’s Customer And the Resulting Design Challenge 2010 25 Years 1985 Design a 2 lb. radio… Over a 10 year period… that will run for ½ hour… and sell for $4000… for use by New York stock brokers… to raise their social status. Design a 4 oz. radio/computer/camera/TV/stereo/GPS… Over a 9 month period… that will run for 2 days… and sell for $0 (after rebate)… to connect every human on the planet… to instantly exchange every form of information: voice, text, web, photo, video,… dsfdsfdsafaf

Back then…Analog Piece Parts What’s Inside? Back then…Analog Piece Parts 2010 1985 dsfdsfdsafaf

…Every Conceivable Interface What’s Inside? Today… Everything You Got …Every Conceivable Interface DSI Digital Serial Interface DSI Host DSI Device Display Storage Speaker Keyboard SLIMbus Serial Low-power Inter-chip Media bus SLIMbus Host/Master SLIMbus Device CSI Camera Serial CSI Host CSI Device Camera Microphone UniPro UniPro Host UniPro Device Application Processor dsfdsfdsafaf

Really…More than You Got …All Your Partner’s IP too What’s Inside? Really…More than You Got …All Your Partner’s IP too Interfaces Make or Break: Block Realization SoC Realization System Realization DSI Digital Serial Interface DSI Host DSI Device Display Storage Speaker Keyboard SLIMbus Serial Low-power Inter-chip Media bus SLIMbus Host/Master SLIMbus Device CSI Camera Serial CSI Host CSI Device Camera Microphone UniPro UniPro Host UniPro Device Application Processor And you no longer have control over the design of these interfaces dsfdsfdsafaf

Which Brings Us Back to the Verification Challenge… How to Verify IP Protocol Compliance and Manage SoC-Level complexity? Interfaces Make or Break: Block Realization SoC Realization System Realization Engine X And you no longer have control over the design of these interfaces Cadence Metric-Driven Verification Flow Management requires Measurement Measurement requires Metrics Done Signoff? Measure & Analyze Execute Construct Plan Spec Metric Definitions OVM VIP Simulation, Formal, Acceleration Failure and Metric Analysis The Fuel that feeds the MDV Engine Is Verification IP including: - vPlans - Test Suites - Coverage Maps - Protocol Checks Creating all the above requires lots of: - Protocol Expertise - Time “A 1 line change in the spec cost me 2 engineers for 6 months” - Major company Verification Manager regarding SATA protocol dsfdsfdsafaf

Agenda The end customer profile and the resulting challenges Introduction to Metric Driven Verification with OVM Metric Driven Verification IPs Summary 8 September 19, 2018 dsfdsfdsafaf 8

Traditional Approach: Directed Testing Verification engineer writes a directed test for each item in Test Plan: Rework when design changes DUT Point out that the traditional Verif Planning process usually indicates what test cases need to be written; but the underlying reason for said test cases, should be due to design analysis. Usually the classical test plan will then just document what directed tests need to be written. The test plan should be the set of Goals to achieve, but most plans document the means by which to achieve those goals. Since manual testing is tedious and labor-prone, the typical method of copy-pasting snippets in test cases results in relatively low variability from 1 test case to the next. Automation Significant manual effort to write all the tests Automation Manual effort needed to verify each goal was reached Completeness No coverage of non-goal scenarios …especially problematic for unanticipated cases Visibility No measures of functional coverage completeness dsfdsfdsafaf

Metric-Driven Verification (MDV) Planning with unified verification metrics Functional specification . . metric-based plan Yes No Done Signoff? Metric definitions reuse checks assertions coverage OVM VIP Plan Failure and metric analysis Measure and analyze Construct Execute [ Animation requires 5 manual clicks to fill out the regions around the circle.] (Discuss the metric-driven verification story.) (Emphasize the impact of coordinating “across” design teams and the improved ability to manage the project. Next slide will emphasize impact of consistent metrics from “specification to closure.”) (Emphasize ability to include hardware/software co-simulation metrics as one way to help close the gaps of the traditional verification flow seen earlier.) Simulation, formal, acceleration dsfdsfdsafaf 10 10

Even for non- goal states! Coverage Driven Verification Defining Coverage “Goals” Enables Automation Focus moves to reaching goal areas (versus execution of test lists): Add constraints to target a specific corner case Simply changing seeds generates new stimulus DUT Constrained-random stimulus generation explores goal areas (& beyond) Coverage shows which goals have been exercised and which need attention Self-Checking ensures proper DUT response. Automation – Constrained-random stimulus accelerates hitting coverage goals and exposing bugs. Coverage and checking results indicate effectiveness of each simulation, which enables scaling many parallel runs. Even for non- goal states! dsfdsfdsafaf

Agenda The end customer profile and the resulting challenges Introduction to Metric Driven Verification with OVM Metric Driven Verification IPs Summary 12 September 19, 2018 dsfdsfdsafaf 12

From Specification to Coverage Metrics Through Verification Plan Specification => Verification Plan => Coverage and checks Metrics Synchronizes spec changes with verification plan Functional & Design Specs Synchronizes verification plan changes with coverage metrics Verification Plan Coverage Metrics Tracks which verification plan features have coverage metrics implemented Copyright © 2009, PCI-SIG, All Rights Reserved

Standard Specification => Verification Plan (PCIe gen3 example) Associated Checks Associated Coverage Associate Verification plan items with the spec annotations Annotate the spec with areas to monitor with coverage, checks and specific testcases Capture relevant meta data and location in spec to track changes Copyright © 2009, PCI-SIG, All Rights Reserved

Verification Plan => Coverage Metrics (AMBA AXI example) Bucket Analysis shows test detail & results Graphical view of constrained-random test sequences Protocol tests performed (in this case, channel transitions) Directly correlated to protocol specification Identifies gaps in simulation regressions Metrics grade verification progress © 2009 Cadence Design Systems, Inc. All rights reserved.

Module => System with multiple vPlans SoC top vPlan view AXI Master vPlan AXI Slave vPlan PCIe vPlan User specific vPlan September 19, 2018 Cadence Confidential: Cadence Internal Use Only

Summary Today’s SoC complexity requires integration of multiple Standard Interfaces IPs Compliance to the standard and Interoperability testing requires advance protocol knowledge and high investment Effective verification management requires metrics and measurable goals MDV Verification IP reduces the protocol knowledge barrier and increase predictability and productivity dsfdsfdsafaf