VEasy: a Tool Suite for Teaching VLSI Functional Verification

Samuel Nascimento Pagliarini and Fernanda Lima KastensmidtProgramas de Pos-Graduacao em Microeletronica (PGMICRO) e Computacao (PPGC)

Instituto de Informatica - UFRGSPorto Alegre, Brasil

{snpagliarini, fglima}@inf.ufrgs.br

Abstract—This paper describes a tool suite aimed at Func-tional Verification (FV) teaching in the context of VLSI circuits.FV is considered a major bottleneck in design cycles and one ofthe reasons is the lack of proper training. Therefore teachingit at the undergraduate or graduate levels is an importantissue. This paper presents VEasy and describes the featuresthat allow for lint analysis, simulation, data generation by aGraphical User Interface, checking and coverage collection andanalysis. All these features merge together to create a completeverification environment, which is appropriated for teachingseveral concepts of FV.

Keywords-Functional Verification, dynamic verification,VLSI teaching, coverage analysis, testbench creation.

I. INTRODUCTION

The primary goal of Functional Verification (FV) is toestablish confidence that the design intent was capturedcorrectly and preserved by the implementation [1]. In orderto do that, FV often uses a combination of simple logicsimulation and test cases (i.e. sequences of inputs) generatedfor asserting specific features of the design. In the contextof this paper, FV is performed on the Register TransferLevel (RTL) representation of the design. On top of thesimulation, FV applies specific and specialized constructs,like constrained randomness [2][3], assertions [4][5] andcoverage metrics [6]. Some of these constructs are addressedwithin the VEasy environment and will be discussed in thenext sections.

Verification as a whole is a necessary step in the de-velopment of today’s complex digital designs. Hardwarecomplexity continues to grow and that obviously impactsthe verification complexity. In fact, it has been shown thatthe verification complexity theoretically rises exponentiallywith hardware complexity [7]. The Application SpecificIntegrated Circuit (ASIC) industry already acknowledgesthat the verification process is extremely necessary andhard to accomplish. Verification itself occupies a fair shareof the design cycle time, some say even 70% [8]. Yet,functional/logic flaws are still the main reason for siliconre-spins [9]. Therefore there is a need to improve the currentverification practices through (better) teaching.

By considering the Brazilian actual context, teaching FVto students has become of great concern. Due to govern-mental incentives, several new design houses have been

created in the last three years. Yet, finding human resourcesthat understand the proper role of verification in the designcycle is not easy since such topic is generally not addressedby undergraduate courses. Also, the current methodologiesbeing used by the industry are so complex that require theunderstanding of several key-concepts of verification, suchas:

• Verification IPs and reuse• Constrained data generation• Assertions• Functional and structural coverage metrics• Transaction level modeling• Verification-specific planning and managementIn order to diminish the implied complexity of these

topics, a tool suite was developed. This tool suite is referredas VEasy [10] and it is freely available for undergraduateand graduate courses usage. The tool suite described in thispaper deals with the most fundamental concepts of FV. First,it is important to realize that FV is strictly simulation based,therefore also referred as dynamic verification. Althoughsimulation itself is an intensive task that contributes forthe bottleneck behavior of FV, it is rather appropriatedfor teaching purposes. Most students, at some point, havealready simulated some RTL models. What VEasy is ableto do is to give a better meaning to what once was a simplesimulation environment, transforming it into an information-rich analysis system.

This paper is organized as follows: Section II explains thegeneralities of the tool suite and later the testbench creationmethodology is explained in Section III. The checkingcapabilities of the tool are explored in Section IV while thecoverage collection and analysis is explored in Section V.Section VI contains the concluding remarks.

II. VEASY - A FUNCTIONAL VERIFICATION TOOL SUITE

The tool suite comprehends four main modules:• The Verilog RTL linting• The Verilog RTL simulation• The testbench creation methodology• The coverage collection and analysisAlso, the tool suite has two distinct work-flows: the

assisted flow and the simulation flow. The Verilog linting

978-1-61284-639-2/11/$26.00 ©2011 IEEE 94

[11] is performed only in the assisted flow of the tool,which starts when the Verilog [12] description of the DesignUnder Test (DUT) is parsed and analyzed. The simulationflow is only enabled when the description complies withthe linting rules. Linting guarantees that the input is writtenusing strictly RTL constructions, i.e. it is synthesizable.These checks are important because they provide an easyway to analyze and debug the code being developed by thestudents even before synthesis is performed.

One example of a linting rule is the single assignment,which ensures that a reg or wire is only assigned in a singleprocedural block (typically an always block). The illustrationof Fig 1 contains one example of such rule being analyzedin the linting environment Graphical User Interface (GUI),which also serves as a code editor with syntax highlightingcapabilities. Using the environment the student is able toperform changes in the source code, save it and lint againuntil it is free of errors.

Once the code of the DUT is linted, the simulation work-flow may begin. The input of the simulation flow is nolonger a hardware description. Instead, it is a verificationplan file. The verification plan file used by VEasy is actuallya complete view of the verification effort. It includes thetraditional lists of features and associated test cases but italso contains simulation and coverage data. This approachmakes it an unified database of the current verificationprogress, therefore simplifying the storage of such data.Students are not required to understand industry standardsthat are used to store such data.

Another useful feature of VEasy is the fact that it is able toextract the design’s interface and special signals (clock andreset) automatically. This feature makes it easier for studentsto create testbenches without being required to control thetiming of the simulation, given that VEasy will automaticallygenerate the special signals. Generating the clock signal istrivial and no user input is required. Yet, for the reset signal,the student is only required to choose one of the availablereset modes:

• No reset, for designs that are not required to reset.• Reset at time zero, i.e. a single reset pulse is thrown at

time zero.

Figure 1: Linting environment GUI reporting an error.

• Probabilistic reset, i.e. reset pulses will be thrown witha given probability.

• Ranged reset, i.e. one reset pulse will happen betweena given time range (e.g. every [100:200] cycles).

After these initial steps the student is ready to startcreating stimulus for the simulation, which is the subjectof the next section.

III. TESTBENCH CREATION USING THE GUI

The main feature of the tool is the testbench creationcapability using the GUI, in which the student is able todrag-and-drop sequence items to build larger sequences. Themethodology that allows the automation is entirely based onlayers. A layer is just a container used to build sequencesprogressively. For instance, a layer1 sequence contains itemsfrom layer0 while a layerN sequence contains items fromlayerN-1 and below.

The possibility of combining more and more layers allowsthe construction of a sequence library. This library then holdssophisticated test cases that are of particular interest for theFV of a design. There are only a few rules that must beobserved: only sequences of layer0 are able to interfacewith the design. Also, the communication between layersis achieved through the creation of logical members (i.e.,members that are not physical ones, that are not connectedto the interfaces). Each member, either physical or logical,is entitled to a list of rules (i.e. constraints).

One example of a simple testbench being constructed us-ing the GUI is shown on Fig. 2. The illustration shows threesequences of layer0, referred as Education, Microelectronicand Systems. It also shows that, when executed, the layer1sequence MSE will simulate the other three in the properorder a few times (check the Sequence items box). It isalso possible to see that the MSE sequence has one logicalmember, which has a constraint being applied. The ruleeditor is shown on top of the image and it is being usedto defined that the member YEAR will always be generatedequal to 2011.

There are more complex scenarios that the layeredmethodology is capable of creating, by using multiple rules,expressions and logical members manipulation. So far, theyare aimed at more complex design scenarios and they willnot be detailed in this paper. Also, all the features availablethrough the GUI are later exported to a wiki-like formatthat allows a more experienced user to produce testbenchswithout using the GUI.

Once the stimuli is set it is necessary to perform someform of checking for evaluating the DUT responses (out-puts). What follows in the next section is the checkingconvenience that using the layered methodology allows.

95

Figure 2: Testbench automation Graphical User Interface.

IV. CHECKING OUTPUTS BY USING THE LAYEREDMETHODOLOGY

When performing FV, it is important to check if the designis responding properly to the simulation inputs. Typicallythis is done by a reference model connected in parallel withthe design, as shown on Fig. 3. Yet, this type of model rarelyis built using the same level of abstraction as in the actualdesign, which makes the student responsible for creatingstimuli that is able to feed both the DUT and the model.

The traditional approach is to convert the low-level stimuliof the DUT to fit for the reference model. Instead of adaptingthe low-level stimuli, by using the layered methodology, thestudent is able to attach the reference model in any levelof the hierarchy, as shown on Fig. 4. The actual simulationof both representations of the design is handled by the toolusing a FIFO structure that stores data until it is ready to becompared.

V. COVERAGE DATA COLLECTION AND ANALYSIS

The quality of the verification relies on coverage metrics,either functional or structural ones [6]. Functional coverage,either data-oriented or control-oriented is possible within

Figure 3: Connecting the reference model using a traditionalmethodology.

VEasy, which has also integrated three different metrics thatare based on structural coverage:

• Block coverage• Expression coverage• Toggle coverage

Block coverage measures if each and every block hasbeen simulated at least once. A block of code is definedas a set of statements that will always execute together.Usually decision statements like ifs and cases are consideredblock starters. Expression coverage measures how much anexpression has been exercised and therefore complementsblock coverage. Finally, toggle coverage measures if everybit of every signal has been properly stimulated by observingif a given bit has toggled from zero to one and vice-versa.

When using VEasy the coverage collection is fully auto-mated. The student is only required to choose which metricsof interested should be collected. After choosing the metricsand performing the simulation, the tool allows for coverageanalysis, as shown on Fig. 5. The student then might checkfor coverage holes and create new scenarios that will mostlikely hit that feature or code area.

Figure 4: Reference model connection using the layeredmethodology.

96

Figure 5: Hole analysis environment for block coverage.

The illustration of Fig. 5 shows the hole analysis envi-ronment of block coverage. Each block that has not beensimulated is highlighted. In the example of Fig. 5, the codebeing analyzed is a single D type flip-flop. The simulationof such flip-flop was performed without asserting the resetsignal, therefore a portion of the code was left unexcited(line 7).

VI. CONCLUSION

This paper presented a tool that is suitable for FV teachingin a VLSI context. Most of the key-concepts of FV areaddressed in a simplified way so that students from bothgraduate and undergraduate levels might use it. Our beliefis that, although new methodologies that combine formal,semi-formal and functional solutions have been proposedand adopted by the industry, these methodologies are limited[13]. FV is the de-facto industry verification method and itis mainly simulation based and coverage dependent.

In that context, this paper described a tool suite thatallows FV teaching using these two concepts. For simulationpurposes it allows some level of testbench automation usingthe GUI while for coverage it allows automatic collectionfollowed by hole analysis. It is also worth mentioning thatchecking might be simplified by the use of the layeredmethodology. All these features together make of VEasy asuitable tool for teaching FV.

REFERENCES

[1] A. Piziali, Functional Verification Coverage Measurementand Analysis. Kluwer Academic, 2004, ch. 2.

[2] J. Yuan, K. Shultz, C. Pixley, H. Miller, and A. Aziz,“Modeling design constraints and biasing in simulation usingbdds,” in ICCAD ’99: Proceedings of the 1999 IEEE/ACMinternational conference on Computer-Aided Design. Pis-cataway, NJ, USA: IEEE Press, 1999, pp. 584–590.

[3] N. Kitchen and A. Kuehlmann, “Stimulus generation for con-strained random simulation,” in ICCAD ’07: Proceedings ofthe 2007 IEEE/ACM international conference on Computer-Aided Design, Nov. 2007, pp. 258 –265.

[4] Standard for the Property Specification Language (PSL),IEEE Std. 1850, 2005.

[5] Standard for SystemVerilog - Unified Hardware Design, Spec-ification, and Verification Language, IEEE Std. 1800, 2009.

[6] R. Grinwald et al., “User defined coverage - a tool supportedmethodology for design verication,” in Proc. 35th annualDesign Automation Conference, San Francisco, United States,June 15–19, 1998, pp. 158–163.

[7] D. Dempster, M. Stuart, and C. Moses, Verification Methodol-ogy Manual: Techniques for Verifying HDL Designs, 2nd ed.Teamwork International, 2001.

[8] S. Fine and A. Ziv, “Coverage directed test generation forfunctional verification using bayesian networks,” in DAC’03: Proceedings of the 40th annual Design AutomationConference. New York, NY, USA: ACM, 2003, pp. 286–291.

[9] R. Schutten and T. Fitzpatrick. (2003) Design for verificationmethodology allows silicon success. [Online]. Available:http://www.eetimes.com/story/OEG20030418S0043

[10] S. N. Pagliarini. (2010) VEasy - a Func-tional Verification Tool Suite. [Online]. Available:http://www.inf.ufrgs.br/∼snpagliarini/veasy

[11] L. Bening and H. Foster, Principles of verifiable RTL design:a functional coding style supporting verification processes inVerilog. Springer.

[12] Standard for the Verilog Hardware Description Language,IEEE Std. 1364, 2001.

[13] A. Aziz, F. Balarin, R. K. Brayton, S. Cheng, R. Hojati, S. C.Krishnan, R. K. Ranjan, A. L. Sangiovanni-vincentelli, T. R.Shiple, V. Singhal, S. Tasiran, and H. Y. Wang, “Hsis: A bdd-based environment for formal verification,” in In Proc. of theDesign Automation Conf, 1994, pp. 454–459.

97