Download presentation
Presentation is loading. Please wait.
Published byRichard Baldwin Modified over 6 years ago
1
Cool FPV Tricks: Reaching Deep Bounds With Not-Quite-Formal Methods
Erik Seligman CS 510, Lecture 13, February 2009
2
Agenda Motivation: FPV and Deep Bounds 0in’s ‘Dynamic Formal’ FPV
Synopsys ‘Hybrid Formal’ FPV When FPV Is Not FPV Some Meta-Comments
3
Agenda Motivation: FPV and Deep Bounds 0in’s ‘Dynamic Formal’ FPV
Synopsys ‘Hybrid Formal’ FPV When FPV Is Not FPV Some Meta-Comments
4
Bounded FPV Majority of FPV work done at low bound
Around cycles Many modules testable this way Limits of engines: exponential blowup Good in many (most?) cases Verification of isolated modules Tricky bugs often don’t need much depth with right input & constraints Cases where deep queues/counters not critical Double-check with coverage points!
5
Problem: FPV Bounds Bounded FPV is ignoring possibilities
Known industry cases of “deep” errors Deep bugs suspected in some cases Example: 2 independent 13-bit ctrs How many combinations possible? How many cycles needed? Bugs may be lurking beyond FPV bounds Maybe too much logic to solve with pruning Strategies desired for hi-bound FPV
6
Compromise: Abstraction
Counters, state machines: free outputs As we saw in memory controller example All values possible Useful in many cases But at significant cost Realism of cases: values jump around No coverage for abstracted logic
7
Compromise: Initial State
Set initial state to something ‘interesting’ Fill queues, set high counters, etc. FPV no longer exhaustive Errors from different initial state not covered Focus is bug hunting in suspicious area Can we do better?
8
Limits of Pruning Pruning often helps complexity issues
In best case, bounded full proof Or can deepen bounds But it can’t help in worst cases Maybe long, complex interactions of parts Think about 2-counter example again Lots of logic may be involved
9
Key Insight: Leverage Simulation
Simulation env has design knowledge Usually set up to create interesting states Wide variety of states possible in sim Can we leverage simulation for FPV? Simple example: Snapshot a simulation state, use as FPV input Are there bigger opportunities?
10
Agenda Motivation: FPV and Deep Bounds 0in’s ‘Dynamic Formal’ FPV
Synopsys ‘Hybrid Formal’ FPV When FPV Is Not FPV Some Meta-Comments
11
0in Dynamic Formal FPV Monitor simulation runs
Tool runs as plugin to simulator Identify “interesting” states Hit cover point Transition counter, state machine, etc. Launch many low-bound FPV runs
12
Ordinary FPV & State Space
rst Small proof radius covered after reset
13
Dynamic Formal Approach
rst Many mini-FPV runs launched from sim Smaller proof radius for each FPV run
14
Advantages/Disadvantages
Based on real tests likely good examples Produce realistic counterexamples since most cycles are from simulation Finds deep bugs “almost” tested in sim Disadvantages?
15
Advantages/Disadvantages
Based on real tests likely good examples Produce realistic counterexamples since most cycles are from simulation Finds deep bugs “almost” tested in sim Disadvantages? Depends on setting up simulation env Not usable really early or arbitrary hierarchy Can’t find counterexamples distant from tests Misses major selling point of FPV: corner cases not conceived by designer Tool pain: FPV & sim tools interact? Harder than it sounds!
16
Agenda Motivation: FPV and Deep Bounds 0in’s ‘Dynamic Formal’ FPV
Synopsys ‘Hybrid Formal’ FPV When FPV Is Not FPV Some Meta-Comments
17
Synopsys Magellan Approach
Similar motivations to 0in Recognize problem: simulation env Few real designers use ‘raw’ sim tool Many layers of scripts & wrappers Really painful to enable 0in-like method Solution: Random Simulation Still uses simulation engine Randomly tries to simulate to get cover pts No use of real tests like 0in
18
Hybrid Formal Approach
rst Lots of random simulation paths Many mini-FPV runs launched from sim Smaller proof radius for each FPV run
19
Advantages/Disadvantages
Not limited by tests from validation team Can theoretically find very deep bugs Independent of simulation environment Disadvantages?
20
Advantages/Disadvantages
Not limited by tests from validation team Can theoretically find very deep bugs Independent of simulation environment Disadvantages? Very random: need to get lucky for bug Loses both comprehensiveness & test-guidance Good cover points may mitigate Still have sim/fpv tool integration issues Many “simulation-synthesis mismatches”: time delays, system functions, non-det behavior, X/Z vals, etc.
21
Agenda Motivation: FPV and Deep Bounds 0in’s ‘Dynamic Formal’ FPV
Synopsys ‘Hybrid Formal’ FPV When FPV Is Not FPV Some Meta-Comments
22
Can We Use A Non-Simulation Approach?
FPV user suspects deep error But proof bounds not enough Standard complexity techniques not working What does FPV user know? Probably has likely scenarios in mind But FPV tool isn’t getting there So why not use user-guided FPV? Try to manually get FPV tool close to error Then launch bounded, comprehensive run
23
User-Guided FPV Create cover points for reset state
Suspected intermediate state on way to err Ask FPV tool to reach cover point CEX at cover == reset state for next Save state manually Nicer if tool help, but not seen in current tools After cover pt, use real bounded FPV Run isn’t full formal proof of properties But is comprehensive bug hunt from there
24
User Guided Approach rst Use FPV engine to visit cover point
Launch bounded FPV from targeted pt
25
Advantages/Disadvantages
Completely eliminates sim engine issues Can theoretically find very deep bugs Takes advantage of user knowledge Disadvantages?
26
Advantages/Disadvantages
Completely eliminates sim engine issues Can theoretically find very deep bugs Takes advantage of user knowledge Disadvantages? Highly dependent on user intuition Loses ‘automatic search’ aspect of 0in/Synopsys Problem areas must be specifically targeted
27
Agenda Motivation: FPV and Deep Bounds 0in’s ‘Dynamic Formal’ FPV
Synopsys ‘Hybrid Formal’ FPV When FPV Is Not FPV Some Meta-Comments
28
Deep Bounds: Why ‘Sexy’?
Much effort in industry/academia Recognition of limits of bounded FPV Potential for fame and fortune Really cool to claim find of cycle-deep error nobody noticed! Common view: One such error justifies cost of full deployment + BMW for the validator But is this really the key FPV problem?
29
Where Do FPV users Spend Their Time?
Intuitive explanation, take with grain of salt 70% “Wiggling” Early attempts with quick counterexamples Root-causing CEXs, adding assumptions, & finding occasional shallow bugs Many FPV efforts never get beyond this stage 20% “Solidifying” Have bounded proofs on most assertions Incrementally deepening bounds, checking covers 10% “Heroics” Extra, expert efforts to get full proofs or deep proofs in suspected problem areas Trying exotic FPV technologies
30
Where Should FPV Research Be Concentrated?
Vast majority of FPV effort is “wiggling” Assuming previous slide roughly accurate! Then next largest chunk is “solidifying” Understaning, improving low bounds Basic complexity issues What are FPVers actually doing? Running low-bound proof attempts Root-causing failures to add assumptions Attacking basic complexity issues (IMHO) The real problem is Usability
31
What Would An FPV Breakthrough Look Like?
Need quantum leaps in productivity of “wiggling” stage Secondary priority: addressing complexity Key functionality I want to see Viewing and understanding counterexamples More & better live what-if experiments Intuitive user interaction Really fast incremental responsiveness Support for interactive complexity analysis
32
My Advice To Academia Better engines & deeper proofs are nice
But only leveraged by small group of engineers who progress to “heroic” FPV stage Best impact: where engineers spend time Wiggling stage: quickly understanding cex Solidifying stage: root-causing & fixing complexity Focus on usability, interactivity, and incremental resposiveness!
33
References / Further Reading
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.