The First in GPON Verification Classic Mistakes Verification Leadership Seminar Racheli Ganot FlexLight Networks

The First in GPON Introduction It’s easy to make mistakes when verifying a DUT or planning a testing effort. Some mistakes are made so often, so repeatedly, by so many different people, that they deserve the label Classic Mistakes.

The First in GPON The presentation contents is divided to three main topics:  Verification team / personnel issues  Planning the testing efforts  The verificator at work

The First in GPON Mistake Declarations: Suggesting solution.

The First in GPON Verification team Personnel Issues Who should test?

The First in GPON Hire one verificator per two or more designers. Ideal verification team includes at least equal amount of engineers as a design team, In particular if the designers don’t write tests at all.

The First in GPON Using new and inexperienced engineers as verificators. A bozo in verification is dangerous like a bozo in design ( and even more ).

The First in GPON Recruiting verificators from the ranks of failed designers. A bad designer likely has some work habits that will make him a bad verificator too. For example, someone who makes lots of bugs because he is inattentive to details will miss lots of bugs from the same reason.

The First in GPON Relying on designer explanations about his updates. Suspicion is a good quality for the verificator.

The First in GPON Designers can’t test their own code. Designers test their own code all the time and they do find bugs, just not enough of them, which is why we need independent verification. Designers can do a perfectly fine job, finding basic functional bugs, but the verificators are the ones to perform the most thorough tests ( different & complex scenarios, special cases, error cases, interaction with other blocks etc.).

The First in GPON Planning the testing effort How should the whole team’s work be organized?

The First in GPON Starting verification too late. Starting verification design when starting HDL design. It doesn’t only improve the environment and give time to develop friendly user interface, but also can prevent designer’s bugs (because he knows the cases that will be checked).

The First in GPON Working with no methodologies. Defining set of verification rules for the team:  Enable each one to understand and support the others’ code.  Enable smooth interaction between blocks in the full chip level.  Very helpful for new employees.

The First in GPON Design block level environment with no preparation / matching for full-chip. These plans are never perfect, but they still decrease the full-chip conversions time.

The First in GPON Test plans are biased toward too specific functional testing. Test plans should include real scenarios, i.e. a set of functions in reasonable order.

The First in GPON Verification plan review is done by the verificator and the block design owner Including more verificators and designers in these reviews can save a lot of time. It is done by learning from others’ experience, sharing similar features and parts of code.

The First in GPON It is preferable to verify more features for basic functions, than checking deeply less features. Complete verification of a single module before starting the next one.

The First in GPON Implementing stress tests on the last minute. For example: DUT that works with several channels and has been checked with only a single one, is almost the same as if this DUT hasn’t been checked at all.

The First in GPON The Verificator at work Designing, writing, debugging, and maintaining tests.

The First in GPON Writing / updating specs at the end of the project (“when we will have time”). Keep your docs open on your desktop during coding and update changes immediately.

The First in GPON Writing tests without enough knowledge about the DUT. It’s better to waste two more days of learning and understanding the DUT, instead of ten days of confusion and uncertainty debugging.

The First in GPON Effort to find bugs for rare scenarios. Important bug is the one that important to customers.

The First in GPON Tests that are understandable only by their owners. Printing clear information and detailed error messages during the test.

The First in GPON Paying more attention to running tests than to designing them.  Test review meetings  Adding coverage items  Increase / decrease randomization in tests according to the coverage results

The First in GPON Relying on the generation in the checker. Less is better mainly for the modularity in full- chip.

The First in GPON Checking that the product does what it’s supposed to do, but not that it doesn’t do what it isn’t suppose to do. Examples:  The output signals need to be checked all the time, not only when a change is expected.  All registers need to be checked after updating a single register (particular test).

The First in GPON Repeat same mistakes at the same team. Sharing knowledge and insights can save a lot of the team’s spent time.

The First in GPON Failing to take notes for the next project After each project is ended, it is mandatory to take notes and come up with conclusions, and then follow them in the next projects.

The First in GPON The End! "If I had to live my life again, I'd make the same mistakes, only sooner". -- Tallulah Bankhead