Embed Size (px)
Citation preview
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Verification Classic Mistakes
Verification Leadership Seminar
Racheli GanotFlexLight Networks
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Introduction
It’s easy to make mistakes when verifying a DUT or planning a testing effort.
Some mistakes are made so often, so repeatedly, by so many different people, that they deserve the label Classic Mistakes.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
The presentation contents is divided to three main topics:
Verification team / personnel issues Planning the testing efforts The verificator at work
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Mistake
Declarations:
Suggesting solution.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Verification team Personnel Issues
Who should test?
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Hire one verificator per two or more designers.
Ideal verification team includes at least equal amount of engineers as a design team, In particular if the designers don’t write tests at all.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Using new and inexperienced engineers as verificators.
A bozo in verification is dangerous like a bozo in design ( and even more ).
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Recruiting verificators from the ranks of failed designers.
A bad designer likely has some work habits that will make him a bad verificator too.
For example, someone who makes lots of bugs because he is inattentive to details will miss lots of bugs from the same reason.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Relying on designer explanations about his updates.
Suspicion is a good quality for the verificator.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Designers can’t test their own code.
Designers test their own code all the time and they do find bugs, just not enough of them, which is why we need independent verification. Designers can do a perfectly fine job, finding basic functional bugs, but the verificators are the ones to perform the most thorough tests ( different & complex scenarios, special cases, error cases, interaction with other blocks etc.).
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Planning the testing effort
How should the whole team’s work be organized?
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Starting verification too late.
Starting verification design when starting HDL design.
It doesn’t only improve the environment and give time to develop friendly user interface, but also can prevent designer’s bugs
(because he knows the cases that will be checked).
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Working with no methodologies.
Defining set of verification rules for the team: Enable each one to understand and support
the others’ code. Enable smooth interaction between blocks in
the full chip level. Very helpful for new employees.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Design block level environment with no preparation / matching for full-chip.
These plans are never perfect, but they still decrease the full-chip conversions time.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Test plans are biased toward too specific functional testing.
Test plans should include real scenarios, i.e. a set of functions in reasonable order.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Verification plan review is done by the verificator and the block design owner
Including more verificators and designers in these reviews can save a lot of time. It is done by learning from others’ experience, sharing similar features and parts of code.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON It is preferable to verify more features for basic
functions, than checking deeply less features.
Complete verification of a single module before starting the next one.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Implementing stress tests on the last minute.
For example:
DUT that works with several channels and has been checked with only a single one, is almost the same as if this DUT hasn’t been checked at all.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
The Verificator at work
Designing, writing, debugging, and maintaining
tests.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Writing / updating specs at the end of the project (“when we will have time”).
Keep your docs open on your desktop during coding and update changes immediately.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Writing tests without enough knowledge about the DUT.
It’s better to waste two more days of learning and understanding the DUT, instead of ten days of confusion and uncertainty debugging.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Effort to find bugs for rare scenarios.
Important bug is the one that important to customers.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Tests that are understandable only by their owners.
Printing clear information and detailed error messages during the test.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Paying more attention to running tests than to designing them.
Test review meetings Adding coverage items Increase / decrease randomization in tests
according to the coverage results
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Relying on the generation in the checker.
Less is better mainly for the modularity in full-chip.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Checking that the product does what it’s supposed to do, but not that it doesn’t do what it isn’t suppose to do.
Examples: The output signals need to be checked all the
time, not only when a change is expected. All registers need to be checked after updating
a single register (particular test).
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Repeat same mistakes at the same team.
Sharing knowledge and insights can save a lot of the team’s spent time.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
Failing to take notes for the next project
After each project is ended, it is mandatory to take notes and come up with conclusions, and then follow them in the next projects.
Th
e F
irst
in G
PO
NT
he
Fir
st in
GP
ON
The End!
"If I had to live my life again, I'd make the same mistakes, only
sooner". -- Tallulah Bankhead
![UNIVERSAL VERIFICATION METHODOLOGY BASED VERIFICATION ... · UNIVERSAL VERIFICATION METHODOLOGY BASED VERIFICATION ENVIRONMENT FOR PCIE DATA LINK LAYER Dr.T.C.Thanuja[1] , Akshata[2]](https://img.pdfslide.us/doc/110x75/5e769b33e234ad4a136bda09/universal-verification-methodology-based-verification-universal-verification.jpg)
