IC04 Incisive

< 1 >

How to Use Coverage Analysis to Increase Testbench Performance

Graser

-

Rolls

< 2 >

• Overview• Multi-Language Simulation• Debugging Environment • Coverage Analysis• New Update Information

Agenda

< 3 >

• Overview• Multi-Language Simulation• Debugging Environment • Coverage Analysis• New Update Information

Agenda

< 4 >

CoverageIUS

Incisive Platform RelationOverview

< 5 >

• SimVision- GUI for unified debug and analysis- Multi-language debug with schematic tracing

+

IUS

• Incisive Unified Simulator- Industry leading, multi-language simulator with ABV- Event-driven native compiled code in a single kernel

architecture for highest-performance- Code ,functional coverage and static analysis

included

Incisive Enterprise SimulatorCore Capabilities

< 6 >

Unified Debugand Analysis

Dynamic Engines

VerificationResults

Static Engines

Compile

VHDL

Compile Compile Compile Compile Compile

Verilog VHDL SystemC PSL/OVL

System Verilog

VHDL –AMS

Compile

Verilog –AMSe

Compile

Comprehensive Coverage

Debug and Analysis

Assertion Analysis

Incisive Simulation Core

•Single kernel architecture- Mixed languages in a single instruction stream:

Verilog, VHDL.- One simulator for all levels of abstraction:

behavioral, digital, analog

• Key benefits- Native compile for high performance- Optimized for small memory footprint- Integrated coverage analysis- One debugging environment across all design

disciplines: architecture, HW, SW, mixedlanguage/signal

- Open standard interfaces assures design portability

< 7 >

• Overview• Multi-Language Simulation• Debugging Environment • Coverage Analysis• New Update Information

Agenda

< 8 >

Gate-level

Signal-levelRTL

Transactions

TA

RTL

Standard RTL

Concise Language

Static Data Types

Clocking Domains

Assertions

Constraint Randomization

TA Recording & Analysis

Port Interfaces

Dynamic Data Types

Coverage

Object Oriented Extensions

C Interfaces

System-level

SystemC 2.1

SystemC 1.0

SystemC 2.1

OSCI SystemC p1666

IEEE p1647

VHDL IEEE 1076

Verilog IEEE 1364

VHDL 2000

OVL/PSL

VHDL 2000&

VHDL 200x

Verilog 2001

OVL/PSL, SVA

SystemVerilog

OVL/PSL

SystemVerilog

VHDL 200x

OVL/PSL

High Abstraction Modeling

“Infinity Minus” Built-In

e

Scenarios & Sequences

Aspect Oriented Prog.

Multi-Language Details

< 9 >

Simulator-X

Verilog

VHDL

Compile

Compile

Compile

….

VS T

AS T

Ela

bora

te

Ver

ilog

VH

DL

sim

ulat

orX

Upd

ate

Ver

ilog

VH

DL

sim

ulat

orX

Single contiguous code stream

How Native Compiled Code is Interleaved

< 10 >

Single Step Execution% irun [Sources] –top [lib.]cell(.view) [Other_Switches]

New “irun” Front EndSingle line activation of IES

< 11 >

• Overview• Multi-Language Simulation• Debugging Environment • Coverage Analysis• New Update Information

Agenda

< 12 >

File Browser Library Browser

Console

Editor

Refresh

Running Simulation on NClaunchNcvhdl Ncvlog Ncelab Irun

< 13 >

Coverage Information Schematic TracingAssertion Browser

SimVision Does it All!

Multi-Language Debugging

HDL WaveformsSignal/TransactionsAnalog/Digital

SimVision Advanced Analysis Debug GUIIntegrated Debugging Environment

< 14 >

• Overview• Multi-Language Simulation• Debugging Environment • Coverage Analysis• New Update Information

Agenda

< 15 >

The goal of coverage is to measure how well the testbenchverifies the design:• To identify design areas in which to focus verification efforts• To estimate the remaining verification effort

Coverage data helps answer these kinds of questions:• What do I test next?

Aim tests for area with lowest coverage• Did my last change break the design?

Run a quick regression (knowing what tests cover the changed area)• How can I be absolutely sure?

Run a complete regression (knowing they have high coverage)• When will I be done?

Track your progress over time• Am I done now?

Do the functional and code coverage meet your requirements?

Introduce the Coverage Basic

< 16 >

1. Instrument the design• Automatic: tools add coverage monitors to inferred coverage points• Manual: user adds coverage monitors (PSL, SVA, covergroups)2. Collect the coverage data• Coverage point monitors count occurrences of coverage point hits

during simulation3. Reduce the coverage data• Tools bin and cross coverage data to make it more manageable• Tools record data to coverage databases for later analysis4. Analyze the coverage data• Tools merge coverage from multiple runs• Tools report coverage statistics using user-selected formats• User diagnoses reports to determine additional test requirements

Introduce the Coverage Process

< 17 >

Coverage typesComprehensive coverage include:

1. Code Coverage• Analyze HDL code structure – which blocks of design code execute.• Determine how fully code structure is exercised.

2. Functional coverage• Analyze design functionality – which functions such as Add and

Queue executed. • Determine how fully design behavior is exercised.

< 18 >

View Coverage SummaryStart the reporting tool in the graphical mode (iccr -gui) and with the File menu, open the coverage test(s).

< 19 >

View Code Coverage DetailsSelect the “Code/Data” tab. Select an instance-based display and expand the instance hierarchy. Double-click an instance to view further details...

< 20 >

• What is a block?Normally a set of statements (may be only one) that always execute together.

• What does block coverage do?Monitor and count block entry. Calculate and report block entry statistics.

• Why bother with block coverage?To measure how completely your testbench exercises the design code.It is the most effective (and thus usually the first) code coverage type employed.

Briefly Describe Block Coverage

< 21 >

Construct VHDL Verilogbegin (explicit or implicit) begin begin, forkbranch statements if, elsif, else

case itemif, elsecase item

event and timing control wait wait, @, #loop statements for/while loop

next, exitfor, forever, repeat, whiledisable

A block is a sequence of statements that always executes together. Either none or all of the statements in a block execute. A block can contain a single statement.Any construct that breaks execution flow creates a new block, for example:

Block coverage = Line coverage

Define a Code Block

< 22 >

VHDL

begin<block>

end block ;

if (a=b) then<block>

else<block>

end if ;

case (x) iswhen 0 ->

<block>when others ->

<block>end case ;

while (a=b) loop<block>

end loop ;wait for 10 ns ;<block>

Verilog

begin<block>

end

if (a==b)<block>

else<block>

case (x)0 :

<block>default :

<block>endcase

while (a==b)<block>

# 10 ;<block>

Example Code Blocks

< 23 >

View Block Coverage DetailsSelect to view the block coverage details...

< 24 >

• What is an expression?An arithmetic, boolean, or relational expression.

• What does expression coverage do?Monitor and count operand states. Calculate and report termvalue coverage.

• Why bother with expression coverage?Expression coverage helps explain why the execution threadentered a block.It is the most natural next verification step after you attain highblock coverage.

• Expression coverage involves a multitude of dimensions:- Which statements and operators to include?- How to score the expressions?

To cover insufficiency of block coverage

Briefly Describe Expression Coverage

< 25 >

View Expression Coverage DetailsSelect to view the expression coverage details...

< 26 >

• What does toggle coverage do?Collect and report design signal toggle activity

• What is a design signal toggle?Normally a binary transition (and return after a finite delay) of aDUT signal. Signals may transit through (but not terminate at) an unknown state.

• Why bother with toggle coverage?It’s the only “code” coverage available for a gate-level netlistVerify that design interconnect is connected and “wiggles”

Netlist code coverage

Briefly Describe Toggle Coverage

< 27 >

View Toggle Coverage DetailsSelect to view the toggle coverage details...

< 28 >

• Users generally look at:- State coverage — what states visited- Transition coverage — what state transitions occurred- Arc coverage — why each state transition occurred

• FSM coverage needs to address several issues:- Recognize FSMs coded using different styles- Recognize FSMs surrounded by other code- Allow users to tag FSMs

Briefly Describe FSM Coverage

< 29 >

View FSM Coverage DetailsThis FSM coverage details window is displaying uncovered state information.

< 30 >

View FSM Coverage DetailsThis FSM coverage details window is displaying uncovered transitions with the input conditions that would have caused them.

< 31 >

Insert coverage configuration by giving optionsElaborate the Design

ncelab top_level_unit [coverage options] [other_options]

Coverage Option Description-coverage <coverage_types>ICC does block, expression, FSM, toggle, and functional coverage

Coverage types {b e f t u a}To select all coverage use b:e:f:t:u or all.

-covfile <coverage_configuration_file> Coverage configuration fileProvide this option once for each configuration file

-covdut <DUT_module_or_entity> Restrict coverage hierarchiesProvide this option once for each hierarchy.

Code Coverage Collection ProcessTo collect code coverage data, you compile, elaborate, and simulate.

< 32 >

Simulate the Designirun source_files(s) [coverage_options] [other_options]

ncsim snapshot [coverage_options] [other_options]

Coverage Option Description Default-covworkdir directory Specify coverage work directory cov_work/

-covdesign directory Specify coverage design directory Design/

-covtest directory Specify coverage test directory Test/

-covoverwrite Overwite existing test directory

-covnomodeldump Suppress dumping coverage model

-input tcl_script Input Tcl command script

Code Coverage Collection ProcessTo collect code coverage data, you compile, elaborate, and simulate.

< 33 >

Step Analyzer Command1. Launch iccr

2.Load load_test <test_spec(s)> test_spec ::= [[workdir/]design/]test

3.List list_coverage_fileslist_coverage [-instance |-module |-entity] <list>

4.Operate merge [-code] [-toggle] [-fsm] [-functional] {<test_specs> | -testfile <filename>} – output <test> [message]

5.Select (de)select_coverage [-betsa] [-instance |-module |-entity] <list>

6.Mark mark {-ignore | -covered} {-instance |-module |-entity}{-b [-recursive] <name> | <indices> -e <name> <indices> <indices>}Indices ::= { n[,] | n1-n2[,] | * }

7.Report report_summary [-instance |-module |-entity] [-hidezero] [-nocgopt] [-betsafd] <list>report_detail [-instance | -module |-entity ] [-covered | -uncoverd | -both | - marked ] [ -nocgopt ] [-nosource] [-nocompact] [-betdafd ] <list>report_html [-instance | -module |entity] [-output <rptname>] [-covered |- uncovered | -both | -marked] [-nocgopt ] [-nocgopt ] [-nosource ] [nocompact ] [- betsafd] <list>

Code Coverage Reporting ProcessTo report code coverage, you:

< 34 >

Compare Functional Coverage to Code Coverage

Code Coverage Functional Coverage

What -Line , statement , Block , Expression , Path-FSM state, Arcs

-Data: Value, Range -Control: Conditions, Sequences, Protocols

Pros -Automated-Good first-level metric

-Closely correlated to desired functionality

Cons -No correlation to desired functionality

-Additional effort

Conclusion -Necessary but not sufficient -Necessary to achieve efficient functional closure

< 35 >

Testbench

DUT

Code Coverage

Code Coverage Functional Coverage

Testbench

DUT

Code Coverage SVA,PSL

• Stimulus generation• Protocol checking• Data checking • Protocol coverage• Test plan coverage

• Stimulus generation• Protocol checking• Data checking

• Protocol coverage• Test plan coverage

Compare Functional Coverage to Code Coverage

< 36 >

Functional Coverage introFunctional coverage focuses on the functional aspects of your design:

• User-specified scenarios, corner cases, protocols, etc.

• As apposed to code coverage (which blindly focuses on individual

items)

• User must be familiar with the design!Functional coverage is in two flavors:

• Control-oriented functional coverage

• Data-oriented functional coverage

< 37 >

Briefly Describe Control-Oriented Functional CoverageControl-oriented functional coverage focuses on design behavior:

• Sequences of actions

• Design protocols

Control-oriented functional coverage slightly overlaps but greatly extends code coverage capabilities, with:

• Arbitrarily complex sequences

• Temporal logic

< 38 >

Briefly Describe Data-OrientedFunctional Coverage

Data-oriented functional coverage focuses on design data:

• Variable values and range

• FSM states and transitions

Data-oriented functional coverage somewhat overlaps (but also extends) code coverage capabilities, with:

• Conditional scoring

• Cross-products

< 39 >

Insert coverage configuration by giving optionsElaborate the Design

ncelab top_level_unit [coverage options] [other_options]

Coverage Option Description-coverage <coverage_types>ICC does block, expression, FSM, toggle, and functional coverage

Coverage types {b e f t u a}To select all coverage use b:e:f:t:u or all.

-covfile <coverage_configuration_file> Coverage configuration fileProvide this option once for each configuration file

-covdut <DUT_module_or_entity> Restrict coverage hierarchiesProvide this option once for each hierarchy.

Code Coverage Collection ProcessTo collect code coverage data, you compile, elaborate, and simulate.

< 40 >

Simulate the Designirun source_files(s) [coverage_options] [other_options]

ncsim snapshot [coverage_options] [other_options]

Coverage Option Description Default-covworkdir directory Specify coverage work directory cov_work/

-covdesign directory Specify coverage design directory Design/

-covtest directory Specify coverage test directory Test/

-covoverwrite Overwite existing test directory

-covnomodeldump Suppress dumping coverage model

-input tcl_script Input Tcl command script

Code Coverage Collection ProcessTo collect code coverage data, you compile, elaborate, and simulate.

< 41 >

View Functional Coverage SummaryStart the reporting tool in the graphical mode (iccr -gui) and with the File menu, open the coverage test(s).

< 42 >

View Data-Oriented CoverageSelect the “Functional” tab and expand the Data-oriented Coverage...

< 43 >

View Control-Oriented CoverageSelect the “Functional” tab and expand the Control- oriented Coverage...

< 44 >

• Overview• Multi-Language Simulation• Debugging Environment• Coverage Analysis• New Update Information

Agenda

< 45 >

• We can effectively exclude our model by typing file name following by the command, which means if you want to exclude the CPU in your design then just type the CPU’s file name behind and do not have to alter each single module and merge into it.

• This utility is not yet supported for the functional coverage.

• This new utility is released in the newest IUS82 s17 version.

Configuration command :

deselect_coverage [-betf] [[-module] list | - instance list | -file list]

Deselect Coverage with File

< 46 >

With deselect coverage command it could ignore certain files as the blackbox to increase the total coverage rate

Deselect Coverage with File

< 47 >

• In a design, an unverified block with MORE statements is likely to have MORE ERRORS than the one with a single statement.

• A block coverage report, by default, does not include informationon the number of statements within a block

Command: set_statement_score

With this command, scoring of statements is enabled, and thestatement coverage information is also included in the blockcoverage report.

Statement Coverage

< 48 >

• The following report is generated from the code after enabling branch and statement coverage.

• In the above report, column #Stmts shows the number of statements in the corresponding block.

Statement Coverage

< 49 >

Thank You !!!