1. - Verifying “Golden” reused IPs The Evil’s in the Edits William C Wallace Texas Instruments Nitin Jayaram Texas Instruments Nitin Mhaske Atrenta Inc Vijay Kanumuri Texas Instruments

2. Author Details William C Wallace: Senior Digital Designer, Texas Instruments 20+ years of experience, focused on using new methods to reduce risk and increase productivity when making modification on legacy IP Nitin Jayaram: Lead DV engineer, Texas Instruments 7+ years of experience, focused on new methods to increase DV productivity Nitin Mhaske: Sr Manager, Atrenta Inc nitinm@atrenta.com 13+ years of experience, leading verification for Semis and EDA

3. Abstract Time To Market makes IP reuse imperative in TI’s SoCs with expectations that reused IP development should be less efforts and less risks. However, most of the IPs need to go through some edits to meet current marketing requirement or meet constraints such as power, and timing. Lack of micro- architecture knowledge, poor documentation and rudimentary testing for reused IPs makes verification of new/modified RTL challenging and time consuming. Traditional quality control techniques fails to capture dependencies of these IP edits, risking the SoC. Assertion Synthesis techniques that learns design behavior based on simulation tests and RTL was used to solve the problem. These techniques automate process of finding functional coverage holes for the IP edits and guided test development to better coverage. With limited design knowledge, these coverage holes are harder to find using traditional coverage techniques. Assertions generated out of these techniques made sure that design edits did not affect the un- modified RTL and respected the SoC requirements.

4. Backdrop

5. SOCs are Heterogeneous TI - SoC Typical Internally Reused IP Typical Third Party Reused IP

6. The Problem Statement

7. Challenges With Reused IPs Almost never satisfies “current” SoC marketing requirements Supported features, configurations SoC specific Power Intent Other constraints such as timing, area etc Original (Micro-architecture) design Intent lost over time Original developers not available Changed hand multiple times at multiple locations Poorly maintained documents How much verification good enough? (Any design edits may affect many features) Existing Env often directed; so cannot trust quality No standardization like UVM; makes reuse harder Lacks functional Coverage, assertions

8. Verification Efforts For Reused IPs Expectations: Reused IP should consume less time/resources “Many unknowns” makes test development daunting Modification of existing Env is challenging and time consuming Modification of existing Env is challenging and time consuming Regressions of un-modified feature tests may not work as-is (due to latency changes etc) Regressions of un-modified feature tests may not work as-is (due to latency changes etc) Poor documentation, lack of localized assertion makes debugging tougher Summary: Verification of reused IPs, if not planned properly, may take as much efforts as new IP

9. Standard Quality Metrics Not Effective IP sign-off to SoC primarily based on two metrics Code coverage Code coverage Functional coverage Functional coverage Code coverage consume lot of time for Reused IPs Deprecated Features Deprecated Features Waivers Modification Waivers Modification Functional coverage monitors are subjective, error prone DV writes high level coverage monitors for modified/new features DV writes high level coverage monitors for modified/new features Important cross coverage due to micro-arch. dependencies of IP edits is difficult to plan and implement Important cross coverage due to micro-arch. dependencies of IP edits is difficult to plan and implement

10. Need For Improving The Methodology Quality: Need automated checkers and coverage to find bugs at IP level; otherwise may be found during late stage in SoC verification. Productivity: Need guidance to test development Verifying reused IP from scratch is not practical Verifying reused IP from scratch is not practical Coverage associated to IP edits and their dependencies highly important Coverage associated to IP edits and their dependencies highly important Other verification techniques do not work Formal Model checking difficult to setup and runtime too long Formal Model checking difficult to setup and runtime too long Sequential equivalence does not understand functional impact Sequential equivalence does not understand functional impact

11. Assertion Synthesis techniques and how to apply them

12. Solution: Assertion Synthesis (Generates assertions and coverage automatically from tests + RTL) Assertion Synthesis Un-modified Modules New/Modified Modules IP RTL Legacy Tests New Tests IP Verification Env 1 to 2 cycle properties generated by analyzing all tests specific to control modules Trivial un-reachable properties removed automatically Redundant properties from others removed automatically Properties divided into different levels L0 (single signal), L1 (Two signal) and L2(rest) “Feature_A == 1’b0” Tests never covers feature_A set “!(stall && wr) Test never asserts stall and wr same cycle “stall |->@cs == IDLE” In all Tests, when stall == 1, next cycle cs is always idle If signal “stall” is tied low in RTL, property “stall == 1’b0” or “!(stall && wr)” removed automatically If “feature_A == 1’b0”, redundant low imp properties such as “!(feature_a && feature_b) removed autmatically L0 Properties: onehot0(sel), cnt < 4’ha L1 properties: !(rd && wr) Properties capturing design assumption/intent categorized as assert; otherwise as cover If design does not support rd & wr in the same cycle “!(rd && w)” is an assertion. Otherwise captures TB limitations; so a coverage

13. Methodology on Reused IPs SoC M1 M4 IP1 – RTL (Reused) M2 M3 IP2-RTL IP3-RTL M5 M6 Module M4 being new/modified Rest of the modules in IP1 unmodified IP1 has legacy verification env+tests Before modification in IP1 starts, run all legacy tests and generate properties for unmodified modules M1, M2.. All properties converted as assertions automatically w/o review. Assumption is that for unmodified RTL legacy tests has complete coverage; no coverage holes Integrated the properties back to IP simulations. So any assertion that fires indicates changes in module M4 that violation assumption of unmodified RTL; hence find bugs Once sufficient tests are written for module M4, generated properties using all tests for M4 and review Classified assertions are added back into simulation. Coverage holes targeted w/ additional tests. When IP coverage matures(holes taken care), integrate all IP assertions into SoC simulations SoC Phase # 1 Any assertion firing can indicate design bug or limitation of IP tests (did not cover fully as SoC intends too) Phase # 2 Phase # 3

14. Example Property Stats From 2 IPs 10-15 Lines/Property for control intensive modules 20-50 Lines/Property for datapath intensive modules Performance Numbers PLI overhead during simulation when collecting properties 10-15% PLI overhead during simulation when collecting properties 10-15% When assertions are added back into simulation, overhead 5-10% When assertions are added back into simulation, overhead 5-10% Classification of properties : Avg. 1 min per property ModuleRTL Lines PropertiesLines/ Property L0 (1 signal) L1 (2 signals) L2 (>2 signals) sch_rd~1800195550~14 sch_wr~1500156062~11 galois_eng_pool~300062912~50 rtc_vbusp~100018227~20 rtc_func~250432~25

15. Assertions Found and Impact Property: mrd_aes_data_in_index != mwr_aes_data_in_index Captures micro-architecture design intent: rd and wr index generated in the RTL based on algorithm; Should never be the same. Property: mrd_aes_data_valid == 1’h1 |-> !isunknown(mrd_aes_data) Captures X propagation: If data driven unknown in SoC, it would localize failure Property:!(rd_mac_start && rd_mac_data_out_pop) Captures code coverage waivers: Known coverage waiver; but an assertion will catch that it is truly not hit in any IP tests or SoC in any corner scenario Assertion Synthesis captures design usage assumption and intent. Assertion Synthesis captures design usage assumption and intent. Integrated Assertion in IP simulation and SoC simulation Integrated Assertion in IP simulation and SoC simulation

16. Coverage Holes Found and impact Property: interrupts <= 3'h5 Finds holes missed by code coverage: Property indicates that higher values of interrupts possible; but missed. This hole could not be found by toggle coverage as all bits toggled Property: mutex(cache_enable, roll_2k_page_active) Helps to understand & code micro-arch dependent functional coverage: Cache enable and roll_rk_page_active can be high in the same cycle; but DV missed important dependency Property: emif_SDataAceept |-> n_mac_seq != MAC_WRT_CMD_STALL Finds BFM limitation: Data accept high and command accept low can happen in stall; but BFM did not support this functionality Assertion Synthesis discovered individual and cross coverage holes missed Assertion Synthesis discovered individual and cross coverage holes missed Analysis of coverage holes was efficient due to removal of un-reachable and redundant coverage holes Analysis of coverage holes was efficient due to removal of un-reachable and redundant coverage holes Guided our future test development based on the coverage holes found Guided our future test development based on the coverage holes found

17. Risk mitigation by Reusing Properties in SoC Assertions generated using IP tests are directly ported/bound to SoC After review for modified RTL and W/O review for unmodified RTL After review for modified RTL and W/O review for unmodified RTL When assertions fire in SoC tests, it indicates either of following Design bug: IP assumption/intent violated in SoC Design bug: IP assumption/intent violated in SoC Misconfiguration: IP is not used correctly in SoC; saves debug time Misconfiguration: IP is not used correctly in SoC; saves debug time IP Coverage holes: IP is missing important coverage; important to particular SoC IP Coverage holes: IP is missing important coverage; important to particular SoCFindings When properties from IP used in SoC Context and no assertions fired implies: When properties from IP used in SoC Context and no assertions fired implies: SOC did not violate an IP requirement and/or SOC did not violate an IP requirement and/or SOC did not exercises a unique state-space that IP DV did not cover SOC did not exercises a unique state-space that IP DV did not cover Performance overhead of running assertions in SoC was comparable as IP (5-10%) Performance overhead of running assertions in SoC was comparable as IP (5-10%) IP Tests Simulation + Assertion Synthesis Assertion Properties (Reviewed + non-reviewed) SoC Tests Simulation Assertion Fires Assertion Holds True

18. Backup

19. Conclusions & Next Steps Assertion synthesis guides test development for reused IPs by finding important functional coverage holes Integrating assertions generated using Assertion Synthesis back to IP and SoC simulations give extra quality assurance To-Do Assertions generated from IP need to be ported to SoC Emulation Env Assertions generated from IP need to be ported to SoC Emulation Env Find out how effective flow can be for even third party IP Find out how effective flow can be for even third party IP

20. Lessons Learned Some learning curve involved in understanding properties Some learning curve involved in understanding properties Aggravated due to less knowledge of RTL Aggravated due to less knowledge of RTL Some properties need couple of minutes to review; improves over time Some properties need couple of minutes to review; improves over time Review & target properties based on Level (Lx) to reduce property review time and prioritize test development Deliver properties to SOC after coverage matures at IP; not early DV should still code high level functional coverage for modified/new features Co-relates to spec better & complement micro-architecture ones from Assertion Synthesis Co-relates to spec better & complement micro-architecture ones from Assertion Synthesis