Using Formal Verification to Replace Mainstream Simulation Erik Seligman Intel Brandon Smith Intel

Slides:



Advertisements
Similar presentations
Providing Feedback to Employees
Advertisements

1 General-Purpose Languages, High-Level Synthesis John Sanguinetti High-Level Modeling.
April 30, A New Tool for Designer-Level Verification: From Concept to Reality April 30, 2014 Ziv Nevo IBM Haifa Research Lab.
Lecture 5: Requirements Engineering
Prescriptive Process models
Putting It All Together: Using Formal Verification In Real Life Erik Seligman CS 510, Lecture 19, March 2009.
Clock Domain Crossing (CDC)
Cut Points On Steroids: Handling Complexity for FPV
FPV For Design Exploration Erik Seligman CS 510, Lecture 11, February 2009.
Automated Method Eliminates X Bugs in RTL and Gates Kai-hui Chang, Yen-ting Liu and Chris Browy.
Timing Override Verification (TOV) Erik Seligman CS 510, Lecture 18, March 2009.
Annoucements  Next labs 9 and 10 are paired for everyone. So don’t miss the lab.  There is a review session for the quiz on Monday, November 4, at 8:00.
Xiushan Feng* ASIC Verification Nvidia Corporation Assertion-Based Design Partition 1 TM Jayanta Bhadra, Ross Patterson.
Alternate Software Development Methodologies
Presenter: PCLee – This paper outlines the MBAC tool for the generation of assertion checkers in hardware. We begin with a high-level presentation.
The Future of Formal: Academic, IC, EDA, and Software Perspectives Ziyad Hanna VP of Research and Chief Architect Jasper Design Automation Ziyad Hanna.
PowerPoint Presentation for Dennis, Wixom & Tegarden Systems Analysis and Design Copyright 2001 © John Wiley & Sons, Inc. All rights reserved. Slide 1.
EE694v-Verification-Lect5-1- Lecture 5 - Verification Tools Automation improves the efficiency and reliability of the verification process Some tools,
SE 555 Software Requirements & Specification 1 SE 555 Software Requirements & Specification Prototyping.
Formal verification Marco A. Peña Universitat Politècnica de Catalunya.
Principle of Functional Verification Chapter 1~3 Presenter : Fu-Ching Yang.
Introduction to Software Testing
Software Engineering Lecture 13 Software Testing Strategies 1.
Approaches to ---Testing Software Some of us “hope” that our software works as opposed to “ensuring” that our software works? Why? Just foolish Lazy Believe.
Test and Verification Solutions116 th April 2010 Silicon South West, “Testing Times” The Economics of Verification Mike Bartley, TVS.
Software Integration and Documenting
Analysis of Simulation Results Andy Wang CIS Computer Systems Performance Analysis.
S/W Project Management Software Process Models. Objectives To understand  Software process and process models, including the main characteristics of.
1 Validation & Verification Chapter VALIDATION & VERIFICATION Very Difficult Very Important Conceptually distinct, but performed simultaneously.
Software Testing.
A Taxonomy of Evaluation Approaches in Software Engineering A. Chatzigeorgiou, T. Chaikalis, G. Paschalidou, N. Vesyropoulos, C. K. Georgiadis, E. Stiakakis.
Chapter 2 소프트웨어공학 Software Engineering 임현승 강원대학교
1 Integration Verification: Re-Create or Re-Use? Nick Gatherer Trident Digital Systems.
Object-Oriented Software Engineering Practical Software Development using UML and Java Chapter 1: Software and Software Engineering.
Presenter : Ching-Hua Huang 2013/7/15 A Unified Methodology for Pre-Silicon Verification and Post-Silicon Validation Citation : 15 Adir, A., Copty, S.
Using Formal Verification to Exhaustively Verify SoC Assemblies by Mark Handover Kenny Ranerup Applications Engineer ASIC Consultant Mentor Graphics Corp.
Slide 1 Construction (Testing) Chapter 15 Alan Dennis, Barbara Wixom, and David Tegarden John Wiley & Sons, Inc. Slides by Fred Niederman Edited by Solomon.
The First in GPON Verification Classic Mistakes Verification Leadership Seminar Racheli Ganot FlexLight Networks.
(A radical interpretation) Tomo Lennox Bow Tie computer services Why Agile Works.
West Virginia University Towards Practical Software Reliability Assessment for IV&V Projects B. Cukic, E. Gunel, H. Singh, V. Cortellessa Department of.
1 Hybrid-Formal Coverage Convergence Dan Benua Synopsys Verification Group January 18, 2010.
© 2012 xtUML.org Bill Chown – Mentor Graphics Model Driven Engineering.
Introduction to Software Testing. Types of Software Testing Unit Testing Strategies – Equivalence Class Testing – Boundary Value Testing – Output Testing.
Functional Verification Figure 1.1 p 6 Detection of errors in the design Before fab for design errors, after fab for physical errors.
From Quality Control to Quality Assurance…and Beyond Alan Page Microsoft.
Jan. 29, 2002Grand Challenges in Simulation Issues in Enhancing Model Reuse C. Michael Overstreet Richard E. Nance Osman Balci.
Assessing the influence on processes when evolving the software architecture By Larsson S, Wall A, Wallin P Parul Patel.
Unique Methodology. Highest Coverage. Fastest Time to Market. Formal Verification in the Industry: a 2020 Vision VIGYAN SINGHAL Oski Technology.
Functional Verification of Dynamically Reconfigurable Systems Mr. Lingkan (George) Gong, Dr. Oliver Diessel The University of New South Wales, Australia.
Software Engineering Project.  Why User involvement?  Requirements Gathering statistics.  Ways of Gathering user requirements.  One-on-One Interviews.
Verification – The importance
Scientific Debugging. Errors in Software Errors are unexpected behaviors or outputs in programs As long as software is developed by humans, it will contain.
CSCI1600: Embedded and Real Time Software Lecture 28: Verification I Steven Reiss, Fall 2015.
Winter 2011SEG Chapter 11 Chapter 1 (Part 1) Review from previous courses Subject 1: The Software Development Process.
The Power of YET! The power of believing that you can improve. So when you can’t do something now – it is a can’t do it ‘YET’!
1 Software Testing Strategies: Approaches, Issues, Testing Tools.
Software Quality Assurance and Testing Fazal Rehman Shamil.
Biological Model Engineering Peter Saffrey, Department of Medicine Cakes Talk Monday, October 20, 2008.
1 Requirements Engineering for Agile Methods Lecture # 41.
WELCOME TO OUR PRESENTATION UNIFIED MODELING LANGUAGE (UML)
The PLA Model: On the Combination of Product-Line Analyses 강태준.
Types for Programs and Proofs
Matching Logic An Alternative to Hoare/Floyd Logic
The Systems Engineering Context
Chapter 18 Software Testing Strategies
Software Project Planning &
Script-less Automation: An Approach to Shift-Left.
Alon Flaisher Alon Gluska Eli Singerman Intel Corporation
Formal Verification of Partial Good Self-Test Fencing Structures
Issues in Enhancing Model Reuse
Presentation transcript:

Using Formal Verification to Replace Mainstream Simulation Erik Seligman Intel Brandon Smith Intel

For Submission: Author Slide Erik Seligman, Intel, , Brandon Smith, Intel, , 2

For Submission: Abstract Formal Verification (FV), the use of EDA software to mathematically prove that all possible behaviors of a Register Transfer Level (RTL) model will be correct, has been a successful and growing technique in design validation. While there is general agreement that FV is useful, it has most often been seen as an additional task to ensure extra safety, rather than a more efficient way to do mainstream validation of a design in progress. We believe that FV technology can now be used to replace a significant amount of the simulation that is currently done in our major core designs. There is clearly a need to improve our validation techniques, as our simulation environments have seemed to grow without bound, and now contain as much code, and as many bugs, as our actual RTL. Thus, the team decided to launch a pioneering effort to demonstrate that FV really could replace simulation as our main validation technique. In order to test this hypothesis, the team enlisted engineers on 11 major units of one of our core designs, and asked them to attempt to validate their units through formal verification. This group included engineers with little or no formal verification experience, as well as a few experts. As the team developed formal proofs on these units, they learned about the challenges of bringing FV to new units with inexperienced engineers, and report on our key takeaways from this effort. As a result of this pioneering, we are confident that FV can indeed be used as a primary validation strategy for applicable units on nearly any future Intel project. We conclude with a set of recommendations for managing the mix of formal and dynamic techniques during product development. 3

Acknowledgements This talk summarizes work by a large team, not just the authors! –M. Arifin, A. Bunker, V. Frolov, M. Lifshits, K. Natarajan, T. Schubert, F. Tabesh, A. Thatcher, C. Wall, R. Yan, C. Yan 4

Outline Current Validation Methodology New Vision for Validation Pioneering The New Vision Example Validation Plan Results and Future Plans 5

Current RTL Validation Full Chip Simulation Cluster Level Simulation Formal Verification– by FV Experts EXE ClusterN Cluster1 Full Chip …

Why More Formal Verification (FV)? 7 Early exercise independent of Test Env progress Instant Testbenches using coverpoint FV on new RTL Early block exercise before combining into unit Quickly find basic bugs Faster integrations by getting healthy RTL earlier Other Benefits of Formal Verification: Validate hard-to-hit conditions with relative ease Quicker and easier debug due to very short traces Instant validation of late-binding changes Complete validation sooner with excellent quality

New Vision for Validation Full Chip Simulation Cluster Level Simulation- Gone? Formal Verification– by FV Experts EXE ClusterN Cluster1 Full Chip Formal Verification– by anyone …

Pioneering the New Vision 11 core units chosen for pioneering –Varying levels of FV expertise among owners –3-month targeted effort (part time) Pioneering goals –Develop validation plans With path to simulation replacement –Build FV environment & wiggle models –Observe real traffic comparable with simulation tests 9

Pioneering Challenges & Solutions 10 ChallengeSolution Hard for non-experts to get started: general FV training overwhelming. Pair each non-expert with expert for initial bring-up. Local reset poorly understood.Use simulation reset initially. Experts help to understand. Conceptual change from active BFMs to passive assumptions. Emphasize interface assumes in FV plan. Also auto-generate simple constraints. Size of some units (300K+ flops) too large for formal. FV plan includes reducing structures & memories. Also need to improve RTL parameterization. Complex interfaces dont enable simple assumptions. Encourage reference modeling on interfaces when applicable, + reusable interface property sets

Sample Formal Verification Plan 11 Instruction Fetch Unit Page Miss Handler interface Memory interface Tracking FSM interface Backend Queue interface = Verification Focus = Abstract/Reduce or Blackbox = Reference Model with Assumptions

Results of Pioneering All 11 units are reasonable FV targets –Assuming expert help to get started –With proper abstraction & reduction –Best when full cluster can be built for FV FV very useful for design exercise –Wiggling waveforms in early/partial proofs –Enables quick sanity check in modified RTL FV assumption creation effort comparable to sim env development –But low ROI if good sim env already done BUT cant completely eliminate cluster simulation –Complex interfaces can be difficult to model passively –Some subset of non-fv-friendly unit types –Inherited units with lots of tests dont want to redo effort 12

Proposed POR: FV Where We Can, Simulate Where We Must 13 Simulation Formal Full Simulation Full FV Sim + Formal Design Exercise Eval FV results

Conclusions Formal Verification IS feasible for mainstream validators –But need experts to help with initial setup FV can replace lots of simulation –Some effort to bring up FV environment But current simulation envs effort-intensive & buggy –Not 100% of units, but major subset We should be doing more FV –Current efforts need to measure & report results –Focus on developing reusable FPV collateral –Demonstrate success to engineers and managers 14