CoveragePkg Doc

CoveragePkg_doc rev 2.2 documentation rel 0.1

Copyright © 2011 by SynthWorks Design Inc. All rights reserved. 1Verbatim copies of this document may be used and distributed without restriction.

Functional Coverage and CoveragePkg

Functional coverage measures of the verification coverage of design requirements. Assuch, it is code written that tracks whether important values, sets of values, and designfeatures have been exercised.

1 Intelligent Coverage: Better than SystemVerilog and 'e' Coverage

VHDL does not have language syntax to model functional coverage, instead weimplement it in a data structure. The package, CoveragePkg, implements the datastructure within a protected type named CovPType. It uses method (procedures andfunctions) to model (describe) coverage, accumulate coverage, interact with thecoverage data structure, report coverage, and save and restore the coverage datastructure. Using these methods we are able to not only implement most of what isprovided by other verification languages such as SystemVerilog or "e", we are able togo beyond what they currently do.

One of the issues with randomized testbenches is that they take approximately"n * logn" randomizations to generate n unique test cases. Ideally we would like to dothis with only n randomizations. Other verification methodologies offer you thiscapability by using either a separate intelligent testbench tool or tool features which willcost you $$$$$$.

What differentiates CoveragePkg from other methodologies (syntax based or other) isthat this feature, which we call Intelligent Coverage, is built into the package. Theprocess is simple. Use a method from CoveragePkg to randomly select a coveragehole. Pass that coverage hole to the stimulus generator as a basis to generate the nextstimulus. Ideally it will only take n passes through stimulus generation to generate allof the stimulus that is needed to test the design. The complexity in this case is inassociating the observed coverage with the input stimulus. In more complex cases, weuse procedural code with additional randomizations to steer the stimulus generation.Since this capability is a fundamental part of CoveragePkg, if a simulator can compileCoveragePkg, then you can use Intelligent Coverage.

Since these features are implemented in a package rather than language syntax, theycan be updated on a regular basis and we don't even need to wait for a vendor toimplement the new language feature.


2 About CoveragePkg, STANDARD revision 2.2

Revision 2.2 adds AtLeast and Weights to the coverage database. The AtLeast valueallows individual bins to have a specific coverage goal. A conjunction of the AtLeastand Weight (depending on the WeightMode) are used to weight the random selectionof coverage holes. These features are at the heart of intelligent coverage.

The STANDARD revision of this package requires VHDL-2008. It will work with a VHDL-2002 compliant simulator by uncommenting the VHDL-2008 compatibility packages.

This package is updated from time to time and is freely available athttp://www.synthworks.com/downloads.

CoveragePkg 2.2 represents the coverage database in a protected type rather thansignals as was done in the versions 1.X. The signal based coverage methodology hasbeen moved to the package, CoverageSigPkg. Version 1.3 of this package willinteroperate with CoveragePkg.

3 What About Code Coverage?

VHDL simulation tools can automatically calculate code coverage (assuming you havelicenses for this feature). Code coverage indicates what in an implementation (thedesign code) has been executed or whether a value has changed (such as from 0 to 1).Code coverage can also track operation of a statemachine.

Code coverage works well on clocked processes and statemachines. For combinationallogic, code coverage does not work so well. To understand this, consider the followingprocess. If SelA, SelB, and SelC all are 1, then the line coverage is 100%. Since thisprocess runs due to any change, glitches due to either delta cycles or propagationdelays (in gate simulations) will be counted as line coverage.

PrioritySel : process (SelA, SelB, SelC, A, B, C)begin Y <= "00000000" ; if (SelC = '1') then Y <= C ; end if ; if (SelB = '1') then Y <= B ; end if ; if (SelA = '1') then Y <= A ; end if ;end process ;

Code coverage only tracks things in an implementation. As a result, if a design can hasa missing feature, it can achieve 100% code coverage without the design being done.


On a similar note, we cannot detect that worst case boundary conditions have occurredunless there is a term for these. Some boundary conditions when different blocks arein particular states and there are no terms that capture this.

What we do know about code coverage is that if we have less than 100% coverageafter marking the items that cannot be covered (because they cover things like casestatement others conditions that do not represent a 0 or 1), then we have work to do.On the other hand, if we reach 100% coverage, there still may be work to do. This iswhere we need functional coverage to help.

4 What is Functional Coverage?

Functional coverage is simply code that measures requirements, features, andboundary conditions of a design or test. As such it can be written either manually, withlanguage syntax (like SystemVerilog or 'e') do, or with CoveragePkg.

With functional coverage we generally measure operations on a higher level than codecoverage. We write code to examine relationships between different objects and countwhen appropriate operations occur. These measurements may be made as a result ofan event (such as the rising edge of clock) or when a transaction completes. Coveragemay examine the values within a single object (aka point or item coverage) such asdifferent transfer sizes across a packet based bus. Coverage may also examine therelationships between different objects (cross coverage) such as an ALU has done all ofits supported operations with every different input pair of registers. A functionalcoverage hole (missing coverage) indicates that either the feature is not tested or is notimplemented.

Functional coverage terms are written for all design features and interface operationsthat must be tested. If we write a high fidelity coverage model, with 100% functionalcoverage and 100% code coverage, then testing is done.

5 A First Example: Manually Tracking Transfer Sizes

Functional coverage can be captured with any code. CoveragePkg and language syntaxare solely intended to simplify this effort. As a first example, lets just write the code.

In a packet based transfer (such as across an ethernet port), most interesting thingshappen when the transfer size is at or near either the minimum or maximum sizedtransfers. It is important that a number of medium sized transfers occur, but we donot need to see as many of them. For this example, lets assume that we areinterested in tracking transfers that are either the following size or range: 1, 2, 3, 4 to127, 128 to 252, 253, 254, or 255. The sizes we look for are specified by our test plan.


We also must decide when to capture (aka sample) the coverage. In the followingcode, we will use a rising edge of clock where the flag TransactionDone is 1.

signal Bin : integer_vector(1 to 8) ;. . .processbegin wait until rising_edge(Clk) and TransactionDone = '1' ; case to_integer(unsigned(ActualData)) is when 1 => Bin(1) <= Bin(1) + 1 ; when 2 => Bin(2) <= Bin(2) + 1 ; when 3 => Bin(3) <= Bin(3) + 1 ; when 4 to 127 => Bin(4) <= Bin(4) + 1 ; when 128 to 252 => Bin(5) <= Bin(5) + 1 ; when 253 => Bin(6) <= Bin(6) + 1 ; when 254 => Bin(7) <= Bin(7) + 1 ; when 255 => Bin(8) <= Bin(8) + 1 ; when others => end case ;end process ;

Any coverage can be written this way. However, this is too much work and too specificto the problem at hand. We could make a small improvement to this by capturing thecode in a procedure. This would help with local reuse, but there are still no built-inoperations (such as reporting or save database).

6 Basic Point Coverage (aka Item Coverage) with CoveragePkg

Point coverage examines values within a single object, such as we did in the example inthe previous section. The basic steps done when collecting functional coverage aredeclare the coverage object, model (describe) the coverage, accumulate the coverage,interact with the coverage data structure, and report the coverage. In this section wewill explore how to do this with CoveragePkg.

Item or point coverage (single object) is a collection of one or more bins, such as the 8bins from the previous example: 1, 2, 3, 4 to 127, 128 to 252, 253, 254, and 255.Internal to CoveragePkg, each bin is represented by a minimum and maximum value(effectively a range). Bins that have only one value, such as 1 are represented by thepair 1, 1 (meaning 1 to 1). Internally, the minimum and maximum values are storedin a record with other bin information.

The data structure for coverage is hidden within the protected type, CovPType. Thefirst step to modeling a coverage item is to create a shared variable to hold the datastructure, such as CovBin1 shown below.

architecture Test1 of tb is shared variable CovBin1 : CovPType ;begin


The next step is to call the function GenBin to create the bins and then call the methodAddBins to add the bins to the protected type. The version of GenBin shown below hasthree parameters: min value, max value, and number of bins. The call, GenBin(1,3,3),breaks the range 1 to 3 into the 3 separate bins with ranges 1 to 1, 2 to 2, 3 to 3.

TestProc : process begin -- min, max, #bins CovBin1.AddBins(GenBin(1, 3, 3)); -- bins 1 to 1, 2 to 2, 3 to 3 . . .

Additional calls to AddBins will append additional bins to the data structure. As a result,the call, GenBin(4, 252, 2), adds two bins with the ranges 4 to 127 and 128 to 252respectively.

CovBin1.AddBins(GenBin( 4, 252, 2)) ; -- bins 4 to 127 and 128 to 252

Since creating one bin per value in the range is common, there is also a version ofGenBin that has two parameters: min value and max value which creates one bin pervalue. As a result, the call GenBin(253, 255) adds three bins with the ranges 253 to253, 254 to 254, and 255 to 255.

CovBin1.AddBins(GenBin(253, 255)) ; -- bins 253, 254, 255

The next step is to call the method ICover to accumulate coverage. This methodologysupports either clock based sampling (shown below) or transaction based sampling (bycalling ICover after a transaction that returns a value - shown in later examples).

-- Accumulating coverage using clock based sampling loop wait until rising_edge(Clk) and nReset = '1' ; CovBin1.ICover(to_integer(unsigned(RxData_slv))) ; end loop ; end process ;

When we are done with our test, we want to print out a report on the coverage. Oneway to decide we are done is to call the method IsCovered. IsCovered returns a truewhen all count bins in the coverage data structure has reached its goal. Just likeICover, IsCovered is called at a sample point (either an event or a transaction). A callto the method WriteBin prints the coverage results to OUTPUT (generally the transcriptwindow when running interactively). The following code prints the coverage resultsafter complete coverage is reached.

ReportCov : processbegin wait until rising_edge(Clk) and Bin1.IsCovered ; CovBin1.WriteBin ; wait ;end process ;

Putting the entire example together, we end up with the following. Note that when weare working with coverage, we primarily work with integer values. All of the inputs to


GenBin and ICover are integers, WriteBin reports results in terms of integers. Each ofthe methods have additional overloading that we will explore later.

architecture Test1 of tb is shared variable CovBin1 : CovPType ; -- Coverage Objectbegin TestProc : process begin -- Model the coverage CovBin1.AddBins(GenBin(1, 3 )); -- bins 1, 2, 3 CovBin1.AddBins(GenBin( 4, 252, 2)) ; -- bins 4 to 127 and 128 to 252 CovBin1.AddBins(GenBin(253, 255 )) ; -- 253, 254, 255

-- Accumulating coverage using clock based sampling loop wait until rising_edge(Clk) and nReset = '1' ; CovBin1.ICover(to_integer(unsigned(RxData_slv))) ; end loop ; end process ;

ReportCov : process begin wait until rising_edge(Clk) and Bin1.IsCovered ; CovBin1.WriteBin ; wait ; end process ;

7 Cross Coverage

Cross coverage examines the relationships between different objects, such as an ALUhas done all of its supported operations with every different input pair of registers.The hardware we are working with is as shown below.

Mux8:1

Mux8:1

Q0

Q7

D0

D7...... ...

...

SRC1

SRC2

For this problem, the only supported operation is addition. The cross coverage needsto see every register used with every other register. This results in the following matrixof desired coverage points.


R7R6R5R4R3R2R1R0

R7R6R5R4R3R2R1R0SRC2

SRC1

When collecting cross coverage, we follow the same steps used to create pointcoverage: declare, model, accumulate, interact, and report. These steps are shown inthe code below. The first step is to declare the coverage object, ACov as a sharedvariable. Next we model the coverage using AddCross. For cross coverage, AddCrossreplaces AddBins. More on this shortly. Next we loop generating one transaction untilwe have complete coverage. The first step in the loop is to check IsCovered to see ifwe are done. Next we use RandInt to randomize the register addresses and get avalue that is in the range of 0 to 7. There is more on RandInt in the RandomPkgdocumentation (available at http://www.SynthWorks.com/downloads). These registeraddresses are used by the procedure DoAluOp to generate transactions. After thetransaction completes, we accumulate the coverage using ICover. After the loopcompletes, write out the coverage results and end the testbench (using the procedure,EndStatus).

architecture Test2 of tb is shared variable ACov : CovPType ; -- Declare Cov Objectbegin TestProc : process variable RV : RandomPType ; variable RegIn1, RegIn2 : integer ; begin ACov.AddCross( GenBin(0,7), GenBin(0,7) ); -- Model

while not ACov.IsCovered loop -- Interact -- Randomize register addresses -- see RandomPkg documentation RegIn1 := RV.RandInt(0, 7) ; RegIn2 := RV.RandInt(0, 7) ;

DoAluOp(TRec, RegIn1, RegIn2) ; -- Do a transaction ACov.ICover( (RegIn1, RegIn2) ) ; -- Accumulate end loop ;

ACov.WriteBin ; -- Report EndStatus(. . . ) ; end process ;

Collecting cross coverage is almost identical to collecting point coverage, except thatthe model and accumulate steps have changed. AddCross creates the cross product ofthe point bins (created by GenBin) on its inputs. Each call to GenBin(0,7) creates the 8


bins: 0, 1, 2, 3, 4, 5, 6, 7. As a result then, AddCross creates 64 bins with the pairs:(0,0), (0,1), (0,2), (0,3), (0,4), (0,5), (0,6), (0,7), (1,0), (1,1), (1,2), (1,3), … , (7,0),(7,1), (7,2), (7,3), (7,4), (7,5), (7,6), (7,7).

In the accumulate step, we still call ICover, however, with cross coverage, ICover hasan integer_vector input with one value for each point in the cross cover. As a result,the call requires a second set of parentheses to denote an integer_vector aggregatewith RegIn1 and RegIn2 as its elements.

While this model only does a cross product of two items, AddCross supports crossing ofup to 20 items.

8 The Problem with Constrained Random

Earlier it was noted that a constrained random testbench takes O(n * Log n) togenerate n unique test cases. Running the above code, we get the following coveragematrix when the code completes. It takes 315 ≈ 64 * log2(64) randomizations togenerate this. By changing the seed value, the exact number of randomizations mayincrease or decrease (but this would be a silly way to try to reduce the number ofiterations a test runs).

R7R6R5R4R3R2R1R0

R7R6R5R4R3R2R1R0SRC2

SRC1

566419664559634364323514644336557710910554835363646417436354566437

9 Intelligent Coverage

Intelligent coverage is what differentiates CoveragePkg from other coveragemethodologies (syntax based or other). The process is simple. We add coverage goalsand/or weights to the data structure and randomly select holes in the coverage to bethe next item generated. Using this, we can generate each item exactly the number oftimes specified in its goal. The code is shown below.

architecture Test3 of tb is shared variable ACov : CovPType ; -- Declare Cov Objectbegin TestProc : process


variable RegIn1, RegIn2 : integer ; begin -- AtLeast, Bin1, Bin2 ACov.AddCross( 2, GenBin(0,7,8), GenBin(0,7,8) ); -- Model

while not ACov.IsCovered loop -- Interact -- Randomize register addresses -- see RandomPkg documentation (RegIn1, RegIn2) := ACov.RandCovPoint ;



In this approach we add coverage goals to the modeling steps (AddCross, AddBins,and/or GenBin) and use RandCovPoint to randomize. In the code above, the AtLeastparameter of AddCross is 2. This means that the bin is considered a coverage holeuntil it has been seen twice. The method RandCovPoint randomly selects a coveragebin that is not covered and then randomly selects a value that is within that coveragebin. RandCovPoint always returns an integer_vector value.

10 Incremental Coverage Construction

Not all coverage is a cross product of a set of points. Instead it can exclude eitherpoints, rows of points, or columns of points. While AddCross and AddBins allows allinformation to be entered in one call, they also allow incremental addition to thecoverage model. Rather thinking about what to exclude, with CoveragePkg we thinkabout what we want in the model and incrementally construct the model bin by bin ifnecessary. Hence, creating a high fidelity cross coverage model is relatively straightforward.

A high fidelity cross coverage model is important. Without one, reaching 100%coverage can be impossible. In addition, the Intelligent Coverage methodology will notwork without it. Incremental construction of the coverage also facilitates givingdifferent bins different weights.

As an example, consider the ALU, except disallow a register to be used with itself. Inaddition give each row a different coverage goal, as shown in the table below. Whilethis is not a real example, it does demonstrate the CoveragePkg coverage modelingcapability.

Coverage Goal Bin1 Bin21 0 1, 2, 3, 4, 5, 6, 72 1 0, 2, 3, 4, 5, 6, 73 2 0, 1, 3, 4, 5, 6, 7


Coverage Goal Bin1 Bin24 3 0, 1, 2, 4, 5, 6, 75 4 0, 1, 2, 3, 5, 6, 76 5 0, 1, 2, 3, 4, 6, 77 6 0, 1, 2, 3, 4, 5, 78 7 0, 1, 2, 3, 4, 5, 6

To model the above coverage, we can do a separate call for each different coveragegoal.

architecture Test4 of tb is shared variable ACov : CovPType ; -- Declare Cov Objectbegin TestProc : process variable RegIn1, RegIn2 : integer ; begin -- Capture coverage model ACov.AddCross( 1, GenBin (0), GenBin(1,7)) ; ACov.AddCross( 2, GenBin (1), GenBin(0) & GenBin(2,7)) ; ACov.AddCross( 3, GenBin (2), GenBin(0,1) & GenBin(3,7)) ; ACov.AddCross( 4, GenBin (3), GenBin(0,2) & GenBin(4,7)) ; ACov.AddCross( 5, GenBin (4), GenBin(0,3) & GenBin(5,7)) ; ACov.AddCross( 6, GenBin (5), GenBin(0,4) & GenBin(6,7)) ; ACov.AddCross( 7, GenBin (6), GenBin(0,5) & GenBin(7)) ; ACov.AddCross( 8, GenBin (7), GenBin(0,6) ) ;




11 GenBin, IllegalBin, and IgnoreBin

GenBin, IllegalBin, and IgnoreBin are used to create bins of type CovBinType. Usingthe functions replaces the need to know the details of the types.

CovBinType is an array of the record type, CovBinBaseType. These are shown below.

type CovBinBaseType is record . . .end record ;type CovBinType is array (natural range <>) of CovBinBaseType ;


11.1 GenBin

The following are five variations of GenBin. The ones with AtLeast and Weightparameters are mainly intended to for use with constants.

function GenBin(Min, Max, NumBin : integer ) return CovBinType ; function GenBin(Min, Max : integer) return CovBinType ; function GenBin(A : integer) return CovBinType ;

-- Only intended for constants function GenBin(AtLeast, Weight, Min, Max, NumBin : in integer) return CovBinType ; function GenBin(AtLeast, Min, Max, NumBin : integer ) return CovBinType ;

The version of GenBin shown below has three parameters: min value, max value, andnumber of bins. The call, GenBin(1, 3, 3), breaks the range 1 to 3 into the 3 separatebins with ranges1 to 1, 2 to 2, 3 to 3.

-- min, max, #bins CovBin1.AddBins(GenBin(1, 3, 3)); -- bins 1 to 1, 2 to 2, 3 to 3

If there are less values (between max and min) than bins, then only "max - min + 1"bins will be created. As a result, the call GenBin(1,3,20), will still create the three bins:1 to 1, 2 to 2 and 3 to 3.

CovBin2.AddBins( GenBin(1, 3, 20) ) ; -- bins 1 to 1, 2 to 2, and 3 to 3

If there are more values (between max and min) than bins and the range does notdivide evenly among bins, then each bin with have on average (max - min + 1)/bins.Later bins will have one more value than earlier bins. The exact formula used is(number of values remaining)/(number of bins remaining). As a result, the callGenBin(1, 14, 4) creates four bins with ranges 1 to 3, 4 to 6, 7 to 10, and 11 to 14.

CovBin2.AddBins( GenBin(1, 14, 20) ) ; -- 1 to 3, 4 to 6, 7 to 10, 11 to 14

Since creating one bin per value in the range is common, there is also a version ofGenBin that has two parameters: min value and max value which creates one bin pervalue. As a result, the first call to AddBins/GenBin can be shortened to the following.

-- min, maxCovBin1.AddBins(GenBin(1, 3)); -- bins 1 to 1, 2 to 2, and 3 to 3

GenBin can also be called with one parameter, the one value that is contained in thebin. Hence the call, GenBin(5) creates a single bin with the range 5 to 5. The followingtwo calls are equivalent.

CovBin3.AddBins( GenBin(5) ) ;CovBin3.AddBins( GenBin(5,5,1) ) ; -- equivalent call

11.2 Illegal and Ignore Bins

When creating bins, at times we need to mark bins as illegal and flag errors or asignored actions and not to count them.


The functions IllegalBin and IgnoreBin are used to create illegal and ignore bins. Oneversion of IllegalBin and IgnoreBin has three parameters: min value, max value, andnumber of bins (just like GenBin).

-- min, max, NumBinIllegalBin( 1, 9, 3) -- creates 3 illegal bins: 1-3, 4-6, 7-9IllegalBin( 1, 9, 1) -- creates one illegal bin with range 1-9IgnoreBin ( 1, 3, 3) -- creates 3 ignore bins: 1, 2, 3

There are also two parameter versions of IgnoreBin and IllegalBin that creates a singlebin. Some examples of this are illustrated below. While this is different from the actionof the two parameter GenBin calls, it matches the common behavior of creating illegaland ignore bins.

-- min, maxIllegalBin( 1, 9) -- creates one illegal bin with range 1-9IgnoreBin ( 1, 3) -- creates one ignore bin with range 1-3

There are also one parameter versions of IgnoreBin and IllegalBin that creates a singlebin with a single value. Some examples of this are illustrated below.

-- AValIllegalBin( 5 ) -- creates one illegal bin with range 5-5IgnoreBin ( 7 ) -- creates one ignore bin with range 7-7

11.3 Predefined Bins

The following are predefined bins.

constant ALL_BIN : CovBinType := GenBin(integer'left, integer'right, 1) ;constant ALL_COUNT : CovBinType := GenBin(integer'left, integer'right, 1) ;constant ALL_ILLEGAL : CovBinType := IllegalBin(integer'left, integer'right, 1) ;constant ALL_IGNORE : CovBinType := IgnoreBin(integer'left, integer'right, 1) ;constant ZERO_BIN : CovBinType := GenBin(0) ;constant ONE_BIN : CovBinType := GenBin(1) ;

11.4 Combining Bins

Since GenBin, IllegalBin, and IgnoreBin all return CovBinType, their results can beconcatenated together. As a result, the following calls to GenBin creates the bins: 1to 1, 2 to 2, 3 to 3, 2 to 127, 128 to 252, 253 to 253, 254 to 254, and 255 to 255.

CovBin1.AddBins(GenBin(0, 2) & GenBin(3, 252, 2) & GenBin(253, 255));

Calls to GenBin, IllegalBin, and IgnoreBin can also be combined. As a result thefollowing creates the four separate legal bins (1, 2, 5, and 6), a single ignore bin (3 to4), and everything else falls into an illegal bin.

CovBin2.AddBins( GenBin(1,2) & IgnoreBin(3,4) & GenBin(5,6) & ALL_ILLEGAL ) ;


12 Data Structure Construction

The methods AddBins and AddCross are used to create the coverage data structure.

12.1 AddBins

The method AddBins is used to add point coverage bins to the coverage data structure.Each time it is called new bins are appended after any existing bins. AddBins hasadditional parameters to allow specification of coverage goal (AtLeast) andrandomization weight (Weight). By using separate calls to AddBins, each bin can havea different coverage goal and/or randomization weight.

procedure AddBins (CovBin : CovBinType) ;procedure AddBins (AtLeast : integer ; CovBin : CovBinType) ;procedure AddBins (AtLeast, Weight : integer ; CovBin : CovBinType) ;

12.2 AddCross

The method AddCross is used to add cross coverage bins to the coverage datastructure. Each time it is called new bins are appended after any existing bins.AddCross has additional parameters to allow specification of coverage goal (AtLeast)and randomization weight (Weight). By using separate calls to AddCross, each bin canhave a different coverage goal and/or randomization weight.

procedure AddCross( Bin1, Bin2 : CovBinType ; Bin3, Bin4, Bin5, Bin6, Bin7, Bin8, Bin9, Bin10, Bin11, Bin12, Bin13, Bin14, Bin15, Bin16, Bin17, Bin18, Bin19, Bin20 : CovBinType := NULL_BIN) ;procedure AddCross( AtLeast : integer ; Bin1, Bin2 : CovBinType ; Bin3, Bin4, Bin5, Bin6, Bin7, Bin8, Bin9, Bin10, Bin11, Bin12, Bin13, Bin14, Bin15, Bin16, Bin17, Bin18, Bin19, Bin20 : CovBinType := NULL_BIN) ;procedure AddCross( AtLeast : integer ; Weight : integer ; Bin1, Bin2 : CovBinType ; Bin3, Bin4, Bin5, Bin6, Bin7, Bin8, Bin9, Bin10, Bin11, Bin12, Bin13, Bin14, Bin15, Bin16, Bin17, Bin18, Bin19, Bin20 : CovBinType := NULL_BIN) ;

12.3 Multiple Matches within the Coverage Data Structure

When coverage has more than one bin, bins are processed in order. By default, if binsoverlap, only the first matching bin is considered. This behavior is controlled by theCountMode variable. The default value of the variable is COUNT_FIRST. Setting thecount mode to COUNT_ALL, as shown below allows all matching bins to be counted.

type CountModeType is (COUNT_FIRST, COUNT_ALL) ;


CovBin4.SetCountMode(COUNT_ALL) ; -- Count all matching binsCovBin4.SetCountMode(COUNT_FIRST) ; -- default. Only count first matching bin

12.4 Controlling Reporting for Illegal Bins

By default, illegal bins both count and flag an error. This behavior is controlled by theIllegalMode variable. The default value of the variable is ILLEGAL_ON. Setting thecount mode to ILLEGAL_OFF, as shown below suppresses printing of messages on anerror.

type IllegalModeType is (ILLEGAL_ON, ILLEGAL_OFF) ;CovBin4.SetIllegalMode(ILLEGAL_OFF) ; -- Illegal printing offCovBin4.SetIllegalMode(ILLEGAL_ON) ; -- Default: Illegal printing on

12.5 SetBinSize

SetBinSize predeclares the number of bins to be created in the coverage data structure.Use this for small bins to save space or for large bins to suppress the resize and copythat occurs when the bins resize.

procedure SetBinSize (NewNumBins : integer) ;

13 Accumulating Coverage

The method ICover is used to accumulate coverage. For point coverage, ICoveraccepts an integer value. For cross coverage, ICover accepts an integer_vector. Theprocedure interfaces are shown below. Since the coverage accumulation is writtenprocedurally, ICover will support either clock based sampling or transaction basedsampling (examples of both shown previously).

procedure ICover( CovPoint : in integer ) ;procedure ICover( CovPoint : in integer_vector ) ;

Since the inputs must be either type integer or integer_vector, conversions must beused. To convert from std_logic_vector to integer, numeric_std_unsigned andnumeric_std provide the following conversions.

CovBin3.ICover( to_integer(RxData_slv) ) ; -- using numeric_std_unsigned (2008)CovBin3.ICover( to_integer(unsigned(RxData_slv)) ) ; -- using numeric_std

To convert either std_logic or boolean to integer, CoveragePkg provides overloading forto_integer.

CovBin3.ICover( to_integer(Empty) ) ; -- std_logicCovBin3.ICover( to_integer(Empty = '1') ) ; -- boolean

To convert either std_logic_vector or boolean_vector to integer_vector (bitwise),CoveragePkg provides to_integer_vector functions.

CrossBin.ICover( to_integer_vector(CtrlReg_slv) ) ; -- std_logic_vectorCrossBin.ICover( to_integer_vector((Empty='1')&(Rdy='1')) ) ; -- boolean_vector


Since the language does not do introspection of aggregate values when determiningthe type of an expression, the boolean vector expression needs to be constructed usingconcatenation (as shown above) rather than aggregates (as shown below).

--! CrossBin.ICover( to_integer_vector( ((Empty='1'),(Rdy='1')) )); -- ambiguous

14 Reporting Coverage

The procedures WriteBin and WriteCovHoles are used to report coverage. Thefollowing overloading for these methods are supported. String typed parameters areused since the language does not allow file typed parameters with protected typemethods.

procedure WriteBin ;procedure WriteBin (FileName : string; OpenKind : File_Open_Kind := APPEND_MODE) ;procedure WriteCovHoles ( PercentCov : real := 100.0 ) ;procedure WriteCovHoles ( FileName : string; PercentCov : real := 100.0 ;OpenKind : File_Open_Kind := APPEND_MODE ) ;

The procedure method WriteBin prints out the coverage results with one bin printed perline. There are two versions. The first has no arguments and prints to OUTPUT. Thisis shown below. Note bins marked as either ignore are not printed by WriteBin and binsmarked as illegal are only printed if they have a non-zero count.

ReportCov : processbegin wait until rising_edge(Clk) and Bin1.IsCovered ; CovBin1.WriteBin ; wait ;end process ;

The other version accepts two arguments. The first argument, FileName specifies thefile name as a string. The second argument specifies the OpenKind argument (tofile_open) and accepts either WRITE_MODE or APPEND_MODE. The OpenKindargument is initialized to APPEND_MODE.

-- FileName, OpenKindCovBin1.WriteBin ("Test1.txt", WRITE_MODE);

The procedure method WriteCovHoles prints out coverage results that are below thePercentCov parameter. Note bins marked as either illegal or ignore are not printed byWriteCovHoles. PercentCov is initialized to 100.0 and it is common to leave it off.

CovBin1.WriteCovHoles ;

Another version of WriteCovHoles specifies FileName, PercentCov, and OpenKind in asimilar fashion to WriteBin. The OpenKind argument is initialized to APPEND_MODE.This is shown below.

-- FileName, PercentCov OpenKindCovBin1.WriteCovHoles("Test1.txt", 100.0, APPEND_MODE);


The methods SetName and SetItemName are used to print a first and second lineheading for WriteBin and WriteCovHoles. SetName is intended to reference the nameor intent of the coverage bin. SetItemName is intended to print column headings forthe coverage bins. Each of these also uses its string parameter to initialize the internalrandomization seed.

CovBin1.SetName("DMA") ; -- Group name - prints firstCovBin1.SetItemName("Stat, WordCnt") ; -- Item Names

15 Coverage Goals, Weights, and Randomization

Coverage goals, weights, and randomization are the core of the Intelligent Coveragemethodology.

15.1 Coverage Goals and Weights

A coverage goal specifies how many times a value must land in a bin before the bin isconsidered covered. A randomization weight determines the relative number of times abin will be selected in randomization. Any of the following can be specified to be usedas the randomization weight: coverage goal, weight, or remaining coverage. Each bincan have a different coverage goal and weight.

Selection of the weight mode is done using SetWeightMode. The following table liststhe current set of supported modes and how the randomization weight is calculated.

Mode WeightAt_Least AtLeastWeight WeightRemain Scale*AtLeast - CountScale is used to adjust the weight when the desired percent coverageis above or below 100%

The interface for procedure SetWeightMode is shown below. Note that the Scaleparameter to SetWeightMode and modes REMAIN_AT_LEAST and REMAIN_WEIGHTare experimental and may not be in the next revision.

type WeightModeType is (AT_LEAST, WEIGHT, REMAIN, REMAIN_AT_LEAST,REMAIN_WEIGHT);procedure SetWeightMode (A : WeightModeType; Scale : real := 1.5) ;


15.2 Adding Coverage Goals and Weights to the Coverage Model

All of the bin construction methods and functions support an extra parameter thatallows specification of AtLeast (coverage goal) and Weight (a potential choice forrandomization weight). Primarily these are specified in AddBins and AddCover asshown previously.

We will see later that GenBin, IllegalBin, IgnoreBin, and GenCross also supportspecification of AtLeast and Weight, but this is primarily intended to allow theirspecification in a constant.

15.3 Basic Randomization

Randomization is handled by either RandCovPoint and RandCovHole. RandCovPointreturns a randomly selected value within the randomly selected bin and is typeinteger_vector. RandCovHole returns the randomly selected bin and is typeRangeArrayType. Only bins that have not reached the specified percentage of theircoverage goal (100.0 by default) are considered for randomization. The typeRangeArrayType and the function definitions are shown below. Note it is recommendedto use RandCovPoint if possible as RangeArrayType may change.

type RangeType is record min, max : integer ;end record ;type RangeArrayType is array (integer range <>) of RangeType;impure function RandCovHole (PercentCov : real := 100.0) return RangeArrayType ;impure function RandCovPoint (PercentCov : real := 100.0) return integer_vector ;

15.4 Randomization, Illegal, and Ignore Bins

RandCovPoint and RandCovHole will never select a bin marked as illegal or ignore.However, if count bin overlaps with an illegal or ignore bin then the illegal or ignorevalue may be generated by randomization. Currently the method around this is tocarefully construct your bins so this will not happen.

15.5 Randomization Thresholds

Each call to RandCovPoint and RandCovHole allows the specification of PercentCov. Formost tests the value is 100.0. RandCovPoint and RandCovHole randomly select anybin that has less than PercentCov of coverage.

Setting PercentCov below 100.0 allows certain thresholds within the coverage bins tobe met. Ideally this threshold would be some value that is within a certain percentageof the current minimum coverage in the coverage model and would increase as theminimum coverage increases. By setting PercentCov to a value less than the currentminimum coverage value, to say 0.0, then automatic thresholding is used. Theautomatic threshold is set to be the current minimum coverage plus the internal


CovThresholdPercent variable value. The intenternal CovThresholdPercent variable isset by the procedure SetCovThreshold. When using automatic thresholding, it isrecommended to set PercentCov to 0.0.

procedure SetCovThreshold (Percent : real) ;

Automatic thresholding is useful when trying to get a balanced solution across all binswhen the bins have a coverage goal that is greater than 1. Automatic thresholdingextends the notion of cyclic randomization to work across a cross coverage set ofvalues where each value is to occur more than once.

15.6 Setting the Seeds

The internal seed for randomization can be initialized using the procedure methods,InitSeed [string] and InitSeed [integer]. Their overloading and an example call isshown below.

procedure InitSeed (S : string ) ;procedure InitSeed (I : integer ) ;. . .CovBin1.InitSeed( CovBin1'path_name ) ; -- stringprocedure SetSeed (RandomSeedIn : RandomSeedType ) ;impure function GetSeed return RandomSeedType ;

In addition, the procedure methods SetName and SetItemName (used with reportingcoverage) also call InitSeed with their parameter. Their overloading and an examplecall is shown below. As a result, for most tests, usage of SetName or SetItemName issufficient to ensure that each coverage model has a unique randomization seed.

procedure SetName (NameIn : String) ;procedure SetItemName (ItemNameIn : String) ;. . .CovBin1.SetName("DMA: Stat, WordCnt") ; -- also calls InitSeed

The methods GetSeed and SetSeed are intended for saving and restoring the seeds.In this case the seed value is of type RandomSeedType, which is defined inRandomBasePkg. RandomBasePkg also defines procedures for reading and writingRandomSeedType values.

16 Interacting with the Coverage Data Structure

In addition to randomization, the following methods are provided for interaction withthe coverage data structure.

impure function IsCovered (PercentCov : real := 100.0) return boolean ;impure function CovBinErrCnt return integer ;impure function GetMinCov return real ;impure function GetMaxCov return real ;impure function CountCovHoles (PercentCov : real := 100.0) return integer ;impure function GetCovHole(ReqHoleNum : integer := 1 ; PercentCov : real := 100.0)


return RangeArrayType ;

The function method, IsCovered, returns a true when all count bins in the coveragedata structure has reached its goal. Just like ICover, IsCovered is called either at asampling point of either the clock or a transaction.

The function method CovBinErrCnt sums up the count in each of the error bins andreturns the resulting value. Generally CovBinErrCnt is called at the end of a testbenchfor coverage models that have bins marked as illegal.

TestErrCount := CovBin1.CovBinErrCnt + (Other_Error_Sources) ;

The function method GetMinCov returns the smallest number in any count field coverbins. The function method GetMaxCov returns the largest number in any count field ofthe cover bins. Both functions are called as follows.

MinCov := CovBin1.GetMinCov ;MaxCov := CovBin1.GetMaxCov ;

The function method CountCovHoles returns the number of holes that are below thePercentCov parameter value (generally allowed to default to 100.0).

-- PercentCovNumHoles := CovBin1.CountCovHoles( 100.0 ) ;

GetCovHole gets the ReqHoleNum bin with a coverage value less than the PercentCovvalue. The following call to GetCovHole gets the 5th bin that has less than 100%coverage. Note that ReqHoleNum must be between 1 and CountCovHoles. The valuereturned by GetCovHole is of type RangeArrayType (same type returned byRandCovHole).

-- ReqHoleNum, PercentCovTestData := CovBin1.GetCovHole( 5, 100.0 ) ;

17 Coverage Database Operations

Coverage can be accumulated by writing out coverage results from one test and thenreading the results back in before the next test. The methods used for this are shownbelow. String typed parameters are used since the language does not allow file typedparameters with protected type methods.

procedure ReadCovDb (FileName : in string) ;procedure WriteCovDb (FileName : in string; OpenKind : File_Open_Kind :=APPEND_MODE ) ;


The procedure method WriteCovDb saves coverage results into a file. It has twoarguments. The first argument, FileName specifies the file name as a string. Thesecond argument specifies the OpenKind argument (to file_open) and accepts eitherWRITE_MODE or APPEND_MODE. The OpenKind argument is initialized toAPPEND_MODE. An example is shown below.

-- FileName, OpenKindCovBin1.WriteCovDb( "CovDb.txt", WRITE_MODE ) ;

The procedure method ReadCovDb reads the values from a file. It has a single stringtyped FileName parameter. An example is shown below.

-- FileNameCovBin1.ReadCovDb( "CovDb.txt" );

The procedure method SetCovZero sets all the coverage counts in a coverage bin tozero. This allows the counts to be set to zero after reading in a coverage database. Asimple call to it is shown below.

CovBin1.SetCovZero ; -- set all counts to 0

The procedure method Deallocate deallocates the entire database structure.

CovBin1.Deallocate ;

18 Creating Bin Constants

Constants are used for two purposes. The first is to create a short hand name for apoint bin (normal constant stuff) and then use that name later in composing thecoverage model. The second is to create the entire coverage model in the constant tofacilitate reuse of the model.

18.1 Point Bin constants

In a previous model, we constructed a cross coverage model using the following call toAddCross.

ACov.AddCross( GenBin(0,7), GenBin(0,7) ); -- Model

On step of refinement is to create a point bin constant for the register addresses, suchas REG_ADDR shown below. The type of REG_ADDR is CovBinType. Since constantscan extract their range based on the object assigned to them, it is easiest to leaveCovBinType unconstrained.

constant REG_ADDR : CovBinType := GenBin(0, 7) ;

Once creating the constant, it can be used in for further composition, such as shownbelow. Just like normal constants, this increases both the readability andmaintainability of the code.

ACov.AddCross(REG_ADDR, REG_ADDR); -- Model


Since each element in a point bin may require different coverage goals or weights,additional overloading of GenBin were added. These are shown below.

function GenBin(AtLeast, Weight, Min, Max, NumBin : integer ) return CovBinType ;function GenBin(AtLeast, Min, Max, NumBin : integer ) return CovBinType ;

As demonstrated earlier, point bins can be composed using concatenation. Thefollowing example creates two bins: 0 to 31 with coverage goal of 5, and 32 to 63 withcoverage goal of 10.

constant A_BIN : CovBinType := GenBin(5, 0, 31, 1) & GenBin(10, 32, 63, 1) ;

18.2 Writing an Cross Coverage Model as a Constant

To capture a cross coverage model in a constant requires some additional types andfunctions. The following methodology is based the language prior to VHDL-2008 andrequires a separate type definition for each size of cross coverage model. Currently upto a cross product of 9 separate items are supported by the following type. In VHDL-2008 where composites are allowed to have unconstrained elements, this will bereduced to a single type (and cross products of greater than 9 can be easilysupported).

type CovMatrix2Type is array (natural range <>) of CovMatrix2BaseType;type CovMatrix3Type is array (natural range <>) of CovMatrix3BaseType;type CovMatrix4Type is array (natural range <>) of CovMatrix4BaseType;type CovMatrix5Type is array (natural range <>) of CovMatrix5BaseType;type CovMatrix6Type is array (natural range <>) of CovMatrix6BaseType;type CovMatrix7Type is array (natural range <>) of CovMatrix7BaseType;type CovMatrix8Type is array (natural range <>) of CovMatrix8BaseType;type CovMatrix9Type is array (natural range <>) of CovMatrix9BaseType;

The function GenCross is used to generate these cross products. We need a separateoverloaded function for each of these types. The interface that generatesCovMatrix2Type and CovMatrix9Type are shown below.

function GenCross( -- cross 2 point bins - see method AddCross constant AtLeast : integer ; constant Weight : integer ; constant Bin1, Bin2 : in CovBinType) return CovMatrix2Type ;

function GenCross(AtLeast : integer ; Bin1, Bin2 : CovBinType) return CovMatrix2Type ;function GenCross(Bin1, Bin2 : CovBinType) return CovMatrix2Type ;

function GenCross( -- cross 9 point bins - intended only for constants constant AtLeast : integer ; constant Weight : integer ; constant Bin1, Bin2, Bin3, Bin4, Bin5, Bin6, Bin7, Bin8, Bin9 : in CovBinType) return CovMatrix9Type ;function GenCross( AtLeast : integer ; Bin1, Bin2, Bin3, Bin4, Bin5, Bin6, Bin7, Bin8, Bin9 : CovBinType


) return CovMatrix9Type ;function GenCross( Bin1, Bin2, Bin3, Bin4, Bin5, Bin6, Bin7, Bin8, Bin9 : CovBinType) return CovMatrix9Type ;

Now we can write our constant for our simple ALU coverage model.

constant ALU_COV_MODEL : CovMatrix2Type := GenCross(REG_ADDR, REG_ADDR);

When we want to add this to our coverage data structure, we need methods thathandle types CovMatrix2Type through CovMatrix9Type. This is handled by theoverloaded versions of AddBins shown below.

procedure AddBins (CovBin : CovMatrix2Type) ;procedure AddBins (CovBin : CovMatrix3Type) ;procedure AddBins (CovBin : CovMatrix4Type) ;procedure AddBins (CovBin : CovMatrix5Type) ;procedure AddBins (CovBin : CovMatrix6Type) ;procedure AddBins (CovBin : CovMatrix7Type) ;procedure AddBins (CovBin : CovMatrix8Type) ;procedure AddBins (CovBin : CovMatrix9Type) ;

To create the coverage data structure for the simple ALU coverage model, call AddBinsas shown below.

ACov.AddBins( ALU_COV_MODEL ); -- Model

This capability is here mostly due to evolution of the package. Keep in mind, the intentis to create readable and perhaps reusable coverage models.

GenCross also allows specification of weights. In a similar manner to AddCross, we canbuild up our coverage model incrementally using constants and concatenation. This isshown in the following example.

architecture Test4 of tb is shared variable ACov : CovPType ; -- Declare Cov Object constant ALU_BIN_CONST : CovMatrix2Type := GenCross(1, GenBin (0), GenBin(1,7)) & GenCross(2, GenBin (1), GenBin(0) & GenBin(2,7)) & GenCross(3, GenBin (2), GenBin(0,1) & GenBin(3,7)) & GenCross(4, GenBin (3), GenBin(0,2) & GenBin(4,7)) & GenCross(5, GenBin (4), GenBin(0,3) & GenBin(5,7)) & GenCross(6, GenBin (5), GenBin(0,4) & GenBin(6,7)) & GenCross(7, GenBin (6), GenBin(0,5) & GenBin(7)) & GenCross(8, GenBin (7), GenBin(0,6) ) ;begin TestProc : process variable RegIn1, RegIn2 : integer ; begin -- Capture coverage model ACov.AddBins( ALU_BIN_CONST ) ;





19 Reuse of Coverage

There are a couple of ways to reuse a coverage model. If the intent is to reuse andaccumulate coverage across tests, then the only way to accomplish this is to useWriteCovDb and ReadCovDb. If the intent is to just reuse the coverage model itself,then either a constant or a subprogram can be used. The calls to ICover generally aresimple enough that we do not try to abstract them.

20 Compiling

We compile all of our packages into a library named SynthWorks. CurrentlyCoveragePkg does not use any VHDL-2008 features, so there is only one version of thepackage. Your programs will need to reference CoveragePkg.

library SynthWorks ; use SynthWorks.CoveragePkg.all ;

21 CoveragePkg vs. Language Syntax

The basic level of item coverage (aka: point coverage) that can be captured withCoveragePkg is similar to when can be captured with IEEE 1647, 'e'. CoveragePkg and'e' allow a item bin to consist of either a single value or a single range. SystemVerilogextends this to allow a value, a range, or a collection of values and ranges. While thisadditional capability of SystemVerilog is interesting, it did not seem to offer anycompelling advantage that would justify the additional complexity required to specify itto the coverage model.

For cross coverage, both SystemVerilog and 'e' focus on first capturing item coverageand then doing a cross of the items. There is some capability to modify the binscontents within the cross, but at best it is awkward. On the other hand, CoveragePkgallows one to directly capture cross coverage, bin by bin and incrementally if necessary.Helper functions are provided to simplify the process. This means for simple things,such as making sure every register pair of an ALU is used, the coverage is captured in avery concise syntax, however, when more complex things need to be done, such asmodeling the coverage for a CPU, the cross coverage can be captured on a line by linebasis.


As a result, with CoveragePkg it is easier to capture high fidelity coverage within asingle coverage object. A high fidelity coverage model in a single coverage object isrequired to do Intelligent Coverage.

22 Deprecated Methods

In the original design of the coverage feedback and randomization functions, there wasno coverage goal or weight. Instead, each bin has a weight of 1 and the coverage goalis determined by the AtLeast parameter in the function calls. These functions areshown below. In this implementation, all bins had the same coverage goal. Usage ofthe AtLeast parameter has been subsumed by the real valued PercentCov parameter.In addition, each bin now has the capability to have a different coverage goal andweight. With different coverage goal values, PercentCov has replaced the AtLeastparameter. The functionality of the AtLeast parameter has been subsumed by hasbeen subsumed by the PercentCov parameter. A PercentCov parameter of 200.0 isequivalent to an AtLeast parameter of 2.

impure function GetMinCov return integer ;impure function GetMaxCov return integer ;impure function CountCovHoles ( AtLeast : integer ) return integer ;impure function IsCovered ( AtLeast : integer ) return boolean ;impure function GetCovHole ( ReqHoleNum : integer := 1 ; AtLeast : integer ) return RangeArrayType ;impure function RandCovHole ( AtLeast : in integer ) return RangeArrayType ;impure function RandCovPoint (AtLeast : in integer ) return integer_vector ;

procedure WriteCovHoles ( AtLeast : in integer ) ;procedure WriteCovHoles ( FileName : string; AtLeast : in integer ; OpenKind :File_Open_Kind := APPEND_MODE ) ;

23 Future Work

CoveragePkg.vhd is a work in progress and will be updated from time to time.

Some of the plans for the next revision are:

• Add a compress method that removes normal bins that are entirely contained ina previously defined ignore or illegal bin.

• Add a global coverage settings protected type.• Add capability for global on/off for coverage collection.• Set defaults for CountMode, IllegalMode, WeightMode, CovThresholdPercent

in the global coverage model.• Remove OrderCount (was for development purposes only).• Consider overloading AddBins to subsume AddCross - that way AddBins is the

only method needed to create the coverage data structure.


If you have ideas that you would like to see, please contact me [email protected].

24 Other Packages

In addition to the CoveragePkg, we also are freely distributing our randomizationpackages (RandomPkg, RandomBasePkg, SortListPkg_int). Seehttp://www.SynthWorks.com/downloads. Over time we will also be releasing otherpackages and hope to convince simulation vendors to distribute our libraries with theirtools.

25 About the Author

Jim Lewis, the founder of SynthWorks, has twenty-six years of design, teaching, andproblem solving experience. In addition to working as a Principal Trainer forSynthWorks, Mr Lewis has done ASIC and FPGA design, custom model development,and consulting. Mr Lewis is an active member of the VHDL standards effort and is thecurrent IEEE VHDL Study Group chair.

I am passionate about the use of VHDL for verification. If you find bugs with any ofSynthWorks' packages or would like to request enhancements, you can reach me [email protected].

26 References

[1] Jim Lewis, VHDL Testbenches and Verification, student manual for SynthWorks' class.

[2] Andrew Piziali, Functional Verification Coverage Measurement and Analysis, Kluwer AcademicPublishers 2004, ISBN 1-4020-8025-5

[3] IEEE Standard for System Verilog, 2005, IEEE, ISBN 0-7381-4811-3

[4] IEEE 1647, Standard for the Functional Verification Language 'e', 2006

[5] A Fitch, D Smith, Functional Coverage - without SystemVerilog!, DVCON 2010

Documents

CoveragePkg Doc