ftv07 9 concl T.ppt - ENSIMAG · • Verification Navigator 6.2 + NCSim 3.2 + Ultra 10 Statement Branch Condition Path Trigger Signal 25% 25% 12000%! 25% 25% ~2000% ... Introduction

Code CoverageDesign Verification Techniques

TestBench / Code Coverage / Cycle Based Simulation …

RTL Verification

Circuit Validation

Equivalence Checking

Acceleration

Emulation FunctionalVerification

TemporalVerification

Static Timing Analysis

RTL Verification

TestbenchCode Coverage

Cycle-based simulationProperty Checking

Functional Verification

2


Code Coverage

Circuit Validation


Acceleration




RTL Verification




3


Motivation…

Key points about simulation:

How much of my design has been

simulated?

How can I reduce my test

development time ?

How can increase my test efficiency ?


4


…Motivation…

• Definition• Code coverage reports the way the design has been exercised

during a given simulation

DUT

TestbenchIf (a = ‘1’ or b = ‘0’) then

z <= data(7 downto 0) + base

else

z <= base << 2;

endif

Source file

Never exercised!Exercised


5


…Motivation

• Goal• Measure the test cases completeness• Indicate whether the test exercises any construct or combination of

construct• Report holes in verification to refine existing test cases

• Coverage indicates the maturity of test cases

Code coverage does not prove that a design is functionally correct!


6


Where in the Flow

RTL = main usagerich variety of metrics

Gate level: very restrictiveSimple metrics only(e.g. toggle coverage)

Synthesis

Floorplan

Place & Route

Extraction

RTL Entry

Gate Level Netlist(Pre layout)

Gate Level Netlist(Post layout)

+ Parasitics

Code Coverage


7


Inputs

• The complete simulation environment• Design Under Test (DUT)• Testbench + components• Testcases

• Metrics


8


Technique

• Code coverage is a dynamic technique that is applied during a functional simulation

• Usually coming with a testbench in order to validate the tb coverage on the design under test

• The code coverage itself does not perform any functional verification on the design


9


Technique…

1. Instrument the design• To add checkpoints at strategic locations to record whether a

particular construct has been exercised

• Note: testbench itself is not instrumented


10


…Technique…

2. Run the simulation• Using existing testcases• Cumulative traces are collected

DUT

TestbenchTest 1

Test n

…If (a = ‘1’ or b = ‘0’) then

z <= data(7 downto 0) + base

else

z <= base << 2;

endif

Source file

0

4747


11


…Technique…

3. Report coverage (metrics)• Coverage metrics are then displayed


12


…Technique…

• Statement coverage• A counter is put on each statement• Counter values are annotated in the HDL code

These lines are not covered by any of the testcases


13


…Technique…

• Signal coverage• Counts and reports the number of time a signal changes values

• e.g.: to check a signal goes through all possible values• e.g.: to check a signal makes all possible transitions

Count c_state

12 0

150 1

0 2

17 3


14


…Technique…

• Branch coverage• Very close to statement coverage• Determines if all possibilities from a control flow are executed

Assume test case forces b = a• 100% statement coverage• 50% branch coverage

if (b = a) thenc := 1;

end if;d := c;

b = a ?

c := 1;

d := c;

TRUEFALSE


15


…Technique…

• Path coverage• Counts the paths in a process.

a = 1 ?

b = 1 ?

Op = 0 Op = data

result = op result = 1.0 / op

A=1 B = 1T TF F

100% branch coverage

A=1 B = 1T TT FF FF T

50% path coverage

F

F T

T


16


…Technique…

• Finite State Machine coverage• State• Transition• Path Cond0-1

Cond1-2

Cond0-2

State2

State0

State1

State3

Cond1-3

Cond2-3

Cond2-1

State2

State0

State1

State3


17


…Technique…

• Condition coverage• Shows which combinations in an expression are responsible for

entering (or not) a branch• Hard to reach 100% for this criteria!

if ((a = 1) or (b = 2)) thenz <= d;

Assume test case sets a and b either to 0 or 1• 100% statement coverage• 100% branch coverage• 50% condition coverage

b=2

False True

a=1F Covered Not Covered

T Covered Not Covered


18


…Technique…

• Trigger coverage• Counter on the signals of any waiting list.

Line Text122 p_fsm: process(a, b, c)

Count Signal Transition200 only ‘a’ changed

0 only ‘b’ changed14 only ‘c’ changed

Construct entered 214 timesCovered 2/3 cases: 66% coverage


19


…Technique

• The Coverage completes the TB :

• Refine the test case to reach higher coverage=> Functional Verification coverage!

• Define the minimum number of test cases to get the higher coverage (regression issue)

=> Simulation run time!

• Spot the most frequently called routines=> Help optimizing the source code to reduce simulation time!

• Help analyzing the code quality=> Remove unused code!• e.g. if (A and not(A)) : no branch condition flagged• e.g. if (…) A <= D(0 to 3) … case A when “11000”


20


Limitations…

• Restriction on the design style• Better coverage if no functions nor procedures• Loops should be used with caution

• Simulation runtime impact (in %)• e.g. small block (1300 gates)• Verification Navigator 6.2 + NCSim 3.2 + Ultra 10

Statement Branch Condition Path Trigger Signal25% 25% 12000%! 25% 25% ~2000%


21


Strategy

• RTL Level: Metrics are dependent• No need to look for 100% path coverage if statement and branch

covers less than 100%1. Start with Statement and Branch coverage 2. Increase test cases to reach 100%)3. Go on with Condition only

• Gate level coverage• No more construct exist at gate level!

• Statement, branch, conditions,… : all useless!• Only remaining metric

• Toggle (simplified metric from signal coverage where signal toggling (1 to 0 and 0 to 1 transitions) is counted)

• Usually done on flip-flops only


22


Tools

• Plug-in tools• Benefit: rich variety of metrics and report capabilities• Drawback: simulation speed! (run through an API)

• Simulator native capability• Reverse benefit and drawback!

SimulatorCoverage tool

API link


23


Deliverables

• Coverage reports• Sorted by metric, indicates completeness of test• Warning: Reminder… coverage is not function!

• Minimum set of test cases to get the higher coverage• Suppress redundant test cases• Result is to reduce simulation run time with the best coverage!


24


And what’s more on tools?


25


Verification Closure Flow

VerificationPlan

vManager

VerificationClosure?

Done!

Yes

No

ProjectProgress

FailureTriage

CoverageAnalysis

Tests

Specman Elite+

SimulationSpecman Elite

+SimulationSpecman Elite

+Simulation

Verification Environment

DUTVerification Environment

DUTVerification Environment

DUT

DeployAnalyzeControl


26


Simulation Failure Triage 27


• Rapid identification of failures• Multiple simulations analyzed

together• Sort and group by error message,

time to failure,

• Eliminate extraneous common failures

• Select shortest failing run• Easier, faster debugging

• Explore coverage contribution common to failing runs

• May point to the functionality causing the failure

Analyze


Track and Report Coverage 28


• Executable coverage plan • Coverage levels are

annotated in the plan

• Total coverage measurement

• Tracks progress against milestones

• Allows aspects/phases, with different coverage goals

• HTML reports • For email, project website

Analyze


Advanced Coverage Analysis

• Correlate coverage and scenarios• Run more of the most effective

• Group coverage holes• Clear picture of uncovered areas• Easy to characterize required new

scenarios

• Correlate various coverage metrics types

• Discrepancies between coverage levels in various metrics


29

Analyze


Controlling Closure

• Refine/enhance scenarios• To target coverage holes revealed during analysis

• Refine coverage model• Based on failure analysis and the correlation between

different coverage metrics

• Create next session description• Daily/weekly/…• To target all or part of the verification plan goals eSIF

Session Runner

Control


30


Verification ManagementVerification Environment

Simulation

Failure Analysis Coverage Analysis

Spec Deploy Analyze

Project Progress

DUT

Verification Closure

Yes No

VerificationPlan

SimulationSession

SimulationSession


31

Cycle Based SimulationDesign Techniques

Code Coverage / Cycle Based Simulation / Property Checking - ABV…

RTL Verification 2

Circuit Validation


Acceleration




RTL Verification




Cycle-based simulation 3

Circuit Validation


Acceleration




RTL Verification




Motivation… 4

How can I run faster simulations ?


…Motivation…

• Concept:• “traditional software simulation” = event driven

• Cycle-based concept reduce the number of calculation

• Cycle-based simulation is no more a specific technique, it is automatically integrated in every simulation tool when usable

• Used at RTL level only

5

INPUT

CLOCK

OUTPUT


Deliverables

• Cycle-based simulation in the verification flow

6

Event driven

Cycle-basedSanity check (triggers, constructs,

Combinational loops)

Faster simulationFirst trials

Acceleration Emulation

Event drivengate levelsimulation

Logic Synthesis

Universal Verification MethodologyDesign Verification Techniques

Verification Techniques / UVM / Property Checking - ABV…

Overview 2

Introduction

RTL Verification

Circuit Validation

Verification Platform



A Global Methodology 3

Circuit Validation


Acceleration




RTL Verification




Motivation… 4

How can I put in place a reusable

solution of verification?


Motivation

• What is simulation versus functional verification?

• Simulation is the process of adding test vectors into a model of the design-under-test then observing how that model behaves

• A traditional test bench reads test vectors or commands from a file to change the values of the signals in the DUT over time, then dumps output from the DUT to compare versus expected. This process is limited and cannot support the reliable verification of very complex systems.

• An effective Verification Methodology is necessary!

5


…Motivation…

• Definition:• UVM is a methodology for the functional verification of digital

hardware• UVM is explicitly simulation-oriented, but UVM can also be

used alongside assertion-based verification, hardware acceleration or emulation

• The hardware to be verified would be described using Verilog, SystemVerilog, VHDL or SystemC

• This could be behavioral, register transfer level, or gate level

6


…Motivation…

• Objective:• A good verification methodology starts with a statement of

the function the DUT is intended to perform. From this is derived a Verification Plan

• UVM test benches are more than traditional test benches• UVM test benches are complete verification environments

composed of reusable verification components, and used as part of an overarching methodology of constrained random, coverage-driven, verification

7

Ref. Doulos web introduction on UVM

(http://www.doulos.com/knowhow/sysverilog/uvm/)


…Environment• Objective:

• UVM is using a library of components with defined role to describe the DUT environment

8

Uvm_subscriber

Uvm_agent

Uvm_sequencer

Uvm_driver Uvm_monitor

Uvm_envt

DUT

Uvm_subscriber

Uvm_agent

Uvm_sequencer

Uvm_driverUvm_monitor


Where in the Flow

• RTL = main usage

• RTL simulation = mandatory

• Gate level: reuse same verification environment

9

Synthesis

Floorplan

Place & Route

Extraction

RTL Entry



+ Parasitics

TestbenchWith UVM


Technique

• UVM drives a dynamic technique that is based on a functional simulation

• This methodology includes code coverage in order to check the ‘design under test’ is well covered

• The UVM must reflect the functional design environment of the DUT and can be re-used at any step of the design

10


Turning Simulation into Verification

• A good verification methodology starts with a statement of the function the DUT is intended to perform. From this is derived a verification plan

• Verification of complex systems must be automated. Along with the verification plan, automated checking, functional coverage and analysis are explicitly addressed by SystemVerilog and UVM.

• Checkers and functional coverage model take time to create but result in better quality of verification.

11


Verification Process1. Start with simple directed (non-random) tests to bring up the

design,

2. Move to fully random tests to explore the state space and flush out as many bugs as possible with minimum human effort devoted to test writing. This will typically achieve much less than 100% functional coverage, \

3. Define a series of tests, where constrains can shape the random stimulus to push the design into interesting corner cases. • random stimulus alone is not enough to explore all the key use cases• directed or highly constrained tests can be too narrow to give good overall

coverage• Constrained random stimulus is a compromise between the two extremes

12


Checkers, …

• Constrained random verification relies on Checkers, Coverage and Constraints

• Automatic checkers ensure functional correctness

• Checkers can be implemented using SystemVerilogassertions or using regular procedural code.

• Assertions can be embedded within the design-under-test, placed on the external interfaces, or can be part of the verification environment (supported in UVM)

13


…, Coverage ...

• Coverage provides a measure of the functional completeness of the testing, and tells you when the goals defined in the verification plan are met.

• SystemVerilog offers two mechanisms for coverage:• property-based coverage (cover directives) • sample-based coverage (covergroups). • The specification and execution of the coverage model is tied to

the verification plan, annotated by simulation tools

14


… and Constraints

• Constraints provide the means to reach coverage goals by shaping the random stimulus to push the DUT into interesting corner cases.

• Constrained random stimulus is still random, but the statistical distribution of the vectors is shaped to ensure the DUT is pushed into interesting corner cases.

• SystemVerilog offers dedicated features for expressing constraints, when UVM provides mechanisms that allow constraints to be part of a test rather then embedded within verification components.

15


Verification Reuse• UVM facilitates the construction of verification environments (VE), by

providing reusable components within a library of SystemVerilogclasses, and by providing a set of guidelines for best practice

• Verification productivity is enhanced by reuse. UVM enable it by:• Having a modular VE where each component is clearly defined, • Allowing flexibility since each components are configured and used, • Allowing imported components to be customized to the application,• Having well-defined coding guidelines to ensure consistency.

• Verification components at all layers can be reused in different environments. Low-level driver and monitor components can be reused across multiple DUT. The whole verification environment can be configured top-down by multiple tests

16


Test sequence• UVM is based on OVM (Open-source Verif Method), using system-

verilog language (structure close to verilog)

• A Verification Environment sequence:

17

Define the different steps in systemV

- Interface to the design-under-test - Design-under-test (or DUT) - Verification environment (or test bench) - Transaction - Sequencer (stimulus generator) - Driver - Top-level of verification environment - Instantiation of sequencer and driver - Top-level module - Instantiation of interface, DUT, Test (which

instantiates the verification environment)- Process to run the test


Inputs

• Design Under Test (DUT)

• Key signal drivers (clocks, resets)

• Components (models)

• Test (=stimuli vectors)

18


Inputs: DUT…

• Design Under Test (DUT)

• RTL/gate description (VHDL , Verilog)

• Same code as for synthesis (except for hard blocks where a model for simulation should be provided)

• The hardware to be verified can be described using Verilog, SystemVerilog, VHDL or SystemC

• It can be behavioral, register transfer level, or gate level. • UVM is based on system-verilog languages

19


Limitations…

• Same as TB with less restriction

• Exhaustive tests do not exist! …• UVM includes random test to increase coverage

20


Automated Verification 21

Verification Environment

AutomaticStimulus

GenerationData and AssertionCheckers

Physical Layer

Device

CoverageMonitor

StimulusScenarios

stimulusScenarios

StimulusScenarios

Generation Self Checking

Coverage


Tools…

• Same as TestBench!

• Classical functional simulators (Ncsim, Questa, Vcs…)

• Language: systemVerilog• Object language• Mixed of classes (software, dynamic object) and module

(hardware component, hierarchy)

22


Deliverables

• Same as TB!

• Result of the test• Should clearly state the passing test cases and the failing ones (if

any and why!)• Coverage report to deliver as an evidence for test completeness

• Sign-off criterion• Execution of test cases on RTL and gate with no difference

• Good verification = high coverage with no test failing

23

Property Checking - ABVDesign Verification Techniques

Cycle Based Simulation / Property Checking – ABV / Equivalence Checking …

RTL Verification 2

Circuit Validation


Acceleration




RTL Verification




Assertion Based Verification 3

Circuit Validation


Acceleration




RTL Verification


Property CheckingCycle-based simulation


Motivation… 4

of How can I ensure properties of my design are valid under all

circumstances ?


…Motivation…

• Property

• Expected behavior of a circuit based on existing specifications

• Safety: a condition that is always true or false• Liveness: a condition that must happen regardless of timing

• A property sometimes is hard to prove with simulation• e.g. bus arbiter managing many masters -> heavy TB!

5


…Motivation…

• Under all circumstances

• The verification should ensure the property would never violate, regardless of any stimulus

• Assertion technique is use to describe the property then check this property is correctly applied in the circuit

• Property Checking uses the Assertion Based Verification technique (ABV)

6


Why assertions ?

• Assertions can find ‘hard to find’ bugs (deep in the design)

• Reduce debug time: assertions are very close to the root of bugs and stop on error

• Improve re-use by communicating design intent (embedded in IP)

• Travel with the design (executable specification)

• Improve verification efficiency• Increase debugging productivity• assertions are “alive” all the time• Could be tool independent : both static and dynamic

7


…Motivation

• Protocol compliancy• “A grant must be generated at max 3 cycles after a request is

asserted”

• Operation• “CPU_READ and CPU_WRITE must never be asserted at the

same time”

• Design quality• “all flip-flop can be reset”• “no flip-flop get stuck”• “an enumerated signal goes through all values”

8


Where in the Flow 9

Micro-architecture:to check the behavior(mainly protocols)RTL = main target(operation or design quality)

Synthesis

Floorplan

Place & Route

Extraction

RTL Entry

Micro-Architecture


Inputs…

• Design• model for micro-architecture (usually C language)• RTL (written in hdl)

• Assumptions about the design environment• To limit the set of admissible sequences, the strategy is to

specify valid operation mode:• “X and Y are never asserted at the same time”• “once a request has been asserter, no other request is

asserted before a grand is asserted”

10


…Properties

• Properties to check• Signals in property are those seen in the architecture document or

at the block boundary• The verification plan should include:

• The properties & environment in natural language• Property: “Req persists until ack is asserted”• Environment: “Ack is asserted one cycle after req”

• The properties & environment in the tool language• PSL (Property Specification Language)• SVA (System Verilog Assertion)

11


Assertion Based Verification Technique

• ABV can be a Dynamic verification• Property is verified during a simulation• Overhead during simulation

• Or a Static verification = model checking• Exhaustive and semi-exhaustive formal property checking• No need to provide stimuli

12


Technique

• Property checking process: Assertion• Describe the environment and the properties to be checked

(can/should be done before the DUT!)• The tool determines whether a property holds under the adopted

assumptions• If a property is violated, a counterexample is automatically

generated by the tool• Different languages are involved for DUT, environment, property

and tool

13


Technique

• How to define an Assertion• If constraints are too strong, a misbehaviour can be missed (false

positive)• If constraints are too weak, non-significant misbehaviours can be

spotted out (false negative)

14


Who should write assertions ?

• Architect

• Verification Engineer• “Black-box” assertions: interface• Use of separate assertions to verify protocols or “interface”

assertions

• Designer• “White-Box” assertions, prefer inline assertions• Comment assumptions on the interface and ensure they will be

fulfilled when integrated• Verify some implementation invariants

15


Assertion Languages• Many languages exist today:

• VHDL – assert statement• Only combinational

• OVL (Open Verification Library)• Very simple, but not powerful

• PSL (IEEE1800 - Property Specification Language)• OVA• SVA• HVL like e

• No static tools, running during TB simulation• Others proprietary languages

• Tools specific (PEC, HPL, ITL…)

16


Using existing HDL functionality• VHDL – assert statement

assert a<breport “a must be smaller than b”severity 0;

• Verilog – no assert, but:always (a or b) beginif (a >= b) begin$display(“error: a must be smaller than b”;$finish;end

end

• Pros• Use of existing language and tools

• Cons• No temporal expressions ; limited to boolean logic only

17


Limitations• RTL coding style for property checking

• No unstable combinational loops• Handling tri-state signals can be a problem• Deep FIFO makes issues:

• Strategy is to use shallow FIFO for debug purpose• An easy way to do that: declare FIFO size as a top level constant• The FIFO can be verified separately and then replaced by a black-box

• Design size limit:• Block-level• Property checking is CPU (fixed point) and Memory intensive task

18


Tool

• RuleBase (IBM)• Historical tool on the market • Still leading edge technology…

• Many EDA tools today• Magellan (Snps), IFV (Cadence Incisive), 0-in (Mentor)• A lot of start-up

19


Deliverables

• The implementation report should state• Property has been fully checked and passes• Property has been checked under certain restrictions and passes• Property is known to fail under specified conditions

• Note:• design may be correct for a given implementation. Violation

conditions may be used in other systems

20

Equivalence CheckingDesign Verification Techniques

Property Checking – ABV / Equivalence Checking / Acceleration …

Overview 2

Introduction

RTL Verification

Circuit Validation




Circuit Validation 3

Verification PlatformAcceleration




RTL VerificationTestbenchCoverageProperty Checking

Circuit Validation



Equivalence Checking 4






Circuit Validation



Motivation… 5

How can I prove that my two netlists have the same

functionality ?


…Motivation• Goal

• To check a new implementation step preserves the functionality, fastest than simulation

• Advantage:• Equivalence is ensured whatever the stimuli are• Equivalence is an exhaustive technique

• Example• Compare two different implementations of the same architecture• Compare two successive releases of the same design

6


Where in the Flow… 7

Equivalence Checking (EC) can be performed between:

RTL vs RTLRTL vs gateGate vs gate(the fastest and most used verification)

Synthesis

Floorplan

Place & Route

Extraction

RTL Entry



+ Parasitics


…Where in the Flow…

• RTL sign-off flow• Effort is put on the RTL simulation (the reference)• EC is then used to prove the netlist after synthesis, place and

route behaves the same

8

RTL Gate1(hierarchical)

Gate2(flat)

Equivalence Checking Equivalence Checking


…Where in the Flow

• Validation after every design modification• RTL optimization, scan chain or clock tree insertion, timing

optimization: EC can prove the function remain unchanged

• Design Migration• EC is a fast way to compare two designs mapped on two different

libraries

9


Inputs• Design (HDL)

• Recommendation: keep the reference structure (hierarchy, naming scheme) for implementation to speed up the process

• Assumptions• To limit the exploration space and speed up the process (e.g test

mode disabled to avoid discrepancies on the scan chain)

• Name mapping• If name changed for the same functionality, provide a mapping file

that allows EC to retrieve the points to compare

10


Technique

• Equivalence Checking is a static mode verification

• No stimuli are required

• That make an exhaustive verification of the whole design

• EC does not check the functionality versus the specification

• The goal is to insure the functionality does not change between to design descriptions

11


Technique…

• Comparison• Made on memory points (flip-flop, latches, memories) and the

output of both designs• The logic cones ending to these points are compared

12

Netlist A

Netlist B Logic equivalenceNot Gate equivalence!


…Technique

• Checking result• Pass implementation matches the reference• Fail implementation is different (function)

• Tool provides vectors to prove the difference• Abort runtime or memory capacity overflow or too difficult to

compare

• Check is:• successful if all compare points passes• Partially successful if pass or abort only

13


Limitations…

• RTL coding style for equivalence checking• No combinational loops• No bus keeper for maximum efficiency

• Design complexity• Strategy: to perform hierarchical verification (check low level

blocks, black-box, go to upper level…)

14

Run TimeMemory


…Limitations…

• Retiming• EC is a combinational process. Thus the designs must match from

a sequential point of view• Some basic retiming is admissible

15

+

+ x

- + -

x+VS

Clock cycle boundary


…Limitations

• Netlist sign-off• EC must be completed with Static Timing Analysis

• In some cases (reset, combinational loops, non synchronism), the gate level simulation remains mandatory

16

Netlist A

Netlist B +Clock period = 5 ns

Arc1Arc2

Equivalence Checking Static Timing Analysis

=Simulation

~


Tools

• Cadence Conformal LEC

• Synopsys Formality

• Some other EDA tools

17


Deliverables• Documentation

• Views checked (RTL, gate) and the result• Views partially checked

• e.g memory, transistor view,• e.g assumptions used to achieve EC and methods to prove those

assumptions (e.g. test enable)• Memory/CPU usage: to reproduce the run

• Scripting:• List of compare points + compare points removed because known

to be different

18

AccelerationDesign Verification Techniques

Equivalence Checking / Acceleration / Emulation …

Overview 2

Introduction

RTL Verification

Circuit Validation

Verification Platform Static Timing Analysis


Verification Platform 3

Circuit Validation


FunctionalVerification





AccelerationEmulation


Acceleration 4

Circuit Validation









Motivation… 5

Man!This simulation lasts forever…

How can I simulate faster ?


…Motivation• Goal

• Improve Software simulation run time which is too slow for today chips

• reach higher simulation cycles/s using Acceleration

• Performances• Software simulation: ~100 cycle/s• Accelerated simulation: ~3000 cycle/s

Note: cycle/s = simulation cycle/clock wall second

6


Where in the Flow 7

Anytime the design has reached a minimum maturity(netlist is roughly stable)

Targets:– RTL– Gate level

Synthesis

Floorplan

Place & Route

Extraction

RTL Entry




Inputs• Design:

• RTL/gate level• A sub-system or the whole DUT is mapped onto the acceleration

box • Memories are usually mapped onto physical on-accelerator

memories

• TestBench and test• The same TB and stimuli as for classical simulation• If TB is synthesizable can be mapped too (emulation)

8


Technique: Principle…

• Acceleration implements the software simulation kernel on dedicated hardware (CPU + memories, FPGA,…)

• Simulation is then run on a workstation connected to hardware boxes

9

Software simulation

Acceleration

Triggers:-process P_1 (a,b)=> z-Process clock (every 10ns)-…-Process alu (opcond, z)=> accu


…Technique…

• Accelerating the design

10

MA

PP

ING

If (a=‘1’) thenZ <= c + d;

DUT

Test

DUT: port map…Clock <= not(clock) after 5 ns

Testbench

Accelerator


…Technique: Usage…

• Simulation speed-up (2X – 20X)

• Non regression tests• Acceleration is used to prove that a new netlist behaves as the

previous release throughout long test sequences

• Toggle count• Acceleration is used to count the flip-flop toggle (hence the

coverage at gate level)

• Manufacturing Tests• Accelerating ATPG vectors (very long sequences)

11



• Interesting tests with acceleration• Reset and clock generator

• To ensure the validity of these critical blocks that cannot be verified outside of simulation

• Debug port verification• To ensure the TAP controller

can control and observe everything required for test

12

TAP



• Timing verification with acceleration• Not exactly functional verification…• Acceleration provides a way to run back-annotated gate level

simulation (wires + cells)• A way to check timing violations and bus contention

13

Place & Route

Extraction Gate Level Netlist(Post layout)

+ Parasitics


Limitations…

• RTL coding for acceleration• X, U and Z states are supported• Metastability should be handled properly at design level

(synchronization stage)• Asynchronous designs simulate poorly: gated clock may introduce

skew problems hard to solve

14

data

clock

enable


…Limitations…

• RTL coding for acceleration• Behavioral RTL cannot be accelerated (not synthesizable)• Testbench should be synthesizable for maximum efficiency

(emulation)

15

Accelerator

Testbench Triggers:

-Process clock (every 10ns)-Process reset(after 100 ns)-Process tb (read file) => a API link


…Limitations…• Debug cycle

• Any change in the design requires a re-mapping on the accelerator• Debugging is not so easy as in software simulation• Acceleration should be used whenever the design has reached a

minimum maturity

• Acceleration box capabilities• Number of I/O• On-board Memory Size• Gate count

16


Tools

• Example: AXIS (without back-annotation)

• Capacity• Up to 8 boards of 448k cells + 32MByte memory

17

300k300k300k300k300k300k300k300k448k

+ 32 Mbyte memory

1 single huge design(2.5 Mcells)

OR

448k 448k 448k

448k

448k448k448k

448k

8 smaller designs in parallel

+ 32 Mbmemory


Deliverables

• The same as for simulation

• Report on test completeness:

• Table of failing/passing test cases

• Toggle count for flip-flops (optional)

18

EmulationDesign Verification Techniques

Acceleration / Emulation / Stating Timing Analysis…

Verification Platform 2

Circuit Validation









Emulation 3

Circuit Validation









Motivation… 4

How can I ensure the design will be working in the context

of the real system ?


…Motivation…

Goals

• Hardware checks: functional verification• to ensure the hardware is interacting correctly with the actual

application• Accuracy: the testbench is the actual application!

• Software checks• To offer a software development platform before the silicon is

ready

• Run Time: Fast!• Up to 1 M cycle/s

5


…Motivation

• Platform example: a video coding/decodingsystem

6

Display

User interface:-Host program loading-Stimuli provider (data in)-Checker (data out)

Host board:-Host CPU tocontrol the chip

Emulator:-The design ismapped here

Camera

Interface:-Actual devices



RTL Level: design synthesis is handled by emulation tool

Gate level: a “translation” library must be provided GTECH (Synopsys) CORELIB (ST) Others

Synthesis

Floorplan

Place & Route

Extraction

RTL Entry



+ Parasitics


Inputs

• Design• A minimum maturity is required

• fine debug on the emulator is hard to conduct• Mapping onto the emulator is not an easy task: it does takes time

(several hours for a typical design)

• Planning• Long and complex task to map a design onto the emulator.

Verification engineer is required!• Some physical environment must be built

8


Technique…• Principle

• Dynamique technique as Simulation• Emulator = sea of programmable devices (FPGA)

• Evolution: mapping of the gate on processors (Palladium)

9

If (a=‘1’) thenZ <= c + d;

RTL + TB

Synthesis

Gates

Mapping


…Technique

• Modes of operation• In-circuit: good coverage

• emulator connected to the actual application platform• Non-regression: easy debug

• “Accelerated platform” Everything (including TB) is mapped onto the emulator. Speed is higher but the TB must be often simplified (less coverage)

• Co-emulation: fast simulation• When the TB is not synthesizable, emulator (as accelerator with a

simulator) can co-emulate with an external TB. It offers a very fast simulation

10


Limitations…

• Cost• Price: up to 1 M US$• Less shareable than a SW simulator (but under continuous

improvement)• Time: expertise to map and drive the emulator

• Signals• 2-state (0,1) behavior only

11


…Limitations

• RTL coding for emulation• Memories are mapped to their emulation model

• Need to write specific memory models• Non-synthesizable constructs are not supported

• e.g. absolute time construct (pulse, delay)• Signals crossing clock domains

• must always be re-synchronized• Clock gating limited to “clean” gating (and-like)• No X,U propagation in emulation• Z is supported (must be resolved)• All flip-flops must be controllable by reset

12

Acceleration / Emulation / Stating Timing Analysis…Tool

• Old: Mentor Celaro (until 2002)

• Up to 192 slots for 6M gates• 128Mbyte memory (SRAM)• Shareable: up to 4 designs in

parallel• Speed: up to 1 M cycle/s• Size: up to 8 m3, 2.5 Tons

• Cadence Palladium• 16 M gates on 8 Gbytes

memory• Shareable: up to 16 designs

in parallel• Speed up to 1.8 M cycles/s• Size: 1.4 m3 , 500 kg

• Mentor Veloce (2007) • 16 M gates on 1 Gbytes

memory per user• Shareable: up to 4 users in

parallel (128MG/8GB)• Speed up to 1.5 M cycles/s• Size: 1.3 m3 , ~450 kg

13


Deliverables

• The same as for simulation!

14


Prototyping

Goal:

• MAIN: early SW development - Validation• To offer a SW development platform before the silicon is ready in

order to debug the final application• The testbench can be the actual application!

• HW/SW• HW debugging is still possible (minor refinement)

• Speed:• Up to 30 M cycle/s

15


…Prototyping

• Platform example: an ARM7/Entry Level Navigation Project

16

User interface:-Host program loading-RTOS (Nucleus)-SW Drivers

Prototyping Platform

ARM Integrator(ARM7TDMI)

The ASIC ismapped onto FPGAs

I2S Audio

CDROM Drive

LCD Display

SRAM and SDRAMonto custom boards


Limitations…

• Cost• Price: ~$US 400K

• Need for external equipments: LSA, Oscilloscope, Clock and Pattern Generator, …

• Not sharable as a SW simulator• Time: expertise to map and drive the tool

• Debug Environment• Poor visibility of internal nodes• Observation nodes are decided at mapping time• Limit = number of I/O, resources to make a node observable

17


Tool

• Aptix MP3/MP4• Up to 8 Mgates (10 Xilinx Virtex2V8000)• Up to 10k visible nets at a time• Custom Modules

• Aptix: SRAM, SDRAM, DPRAM, Integrator I/F, …• Replicas available, Speed: up to 30 M cycle/s

• ProDesign (Platinium)

• Eve (Zebu)

• …

18


Emulation vs Prototyping

• Emulation• Easy to map a design (1 to 3 weeks) but request a good expertise

of the tool• Easy for debug• Available for huge design

• Prototyping• High Speed simulation• Long to map (1 to 3 months)• More for early SW development than design validation

19

Static Timing AnalysisDesign Verification Techniques

Emulation / Stating Timing Analysis / Conclusion …

Overview 2

Introduction

RTL Verification

Circuit Validation




Temporal Verification 3

Circuit Validation




Temporal Verification




Motivation… 4

How can I ensure my design will work

at the target frequency under all

circumstances ?


Motivation• Goal:

• To check the design will reach the expected timing performances

• How:• By ensuring any timing path meets the timing requirements• Warning: STA does not check functionality!

5


Motivation

• Concept• Check the data are available at the right time around the clock

edge signal through static timing calculation

6

INPUT

CLOCK

OUTPUT

CLOCK

INPUT

SETUPHOLD



Pre layout gate level:to set up the timing environment

Post layout gate level:main target

Synthesis

Floorplan

Place & Route

Extraction

RTL Entry



+ Parasitics


…Where in the Flow

• Pre layout accuracy

• Post layout accuracy

8

Statistics based(Wire Load Model)

R,C = f(Area)

R,C = extractedfrom layout


Inputs…

• Design• Gate level (Verilog, EDIF, VHDL)• Analog blocks, memories have their own timing model

• Timing Annotation• Gate and/or wire annotation

• As extracted from Place & Route• Or as estimated in pre-layout analysis

9

R1 C1 R2 C2


…Inputs…

• Design constraints• Environment

10

Driving cell

Input transition

Load

Operating conditions

DSPFSDF


…Inputs…

• Design constraints• About environment

11

Process

Quantifies the manufacturing dispersion(oxide & metal thickness, …)

Voltage

Temperature

Timing model for devices (transistors,…) is available for a range of voltage supply

Timing model for devices (transistors,…) is available for a range of temperature


…Inputs

• Design Constraints• Timing Constraints

12

Combo Logic

Combo Logic Combo Logic Combo Logic

Clock period, waveform and phase shift

Arrival Time Departure Time


Technique

• Static Timing Analysis is a static technique, no stimuli are required

• STA insures the timing performances are reached on the functional design

• Note: The goal is NOT to check the functionality of the design: it must be associated with a verification technique to insure the full verification

13

STA + EQ Check ~ TB simulation(static) (static) (dynamique)


Technique: Delay Calculation…

• For gate level designs

14

Input transitionOutput capacitance

0.4 ns

A Z

" 0.04046, 0.15501, 0.40826, 0.53434",

"0.04465, 0.15978, 0.41281, 0.67763",

"0.10748, 0.26067, 0.51005, 0.76346",

"0.16294, 0.42466, 0.73709, 1.04356",

"0.17673, 0.43920, 0.87670, 1.23678

library.lib (STF)" 0.04046, 0.15501, 0.40826, 0.78816",

"0.04465, 0.15978, 0.41281, 0.79198",

"0.10748, 0.26067, 0.51005, 0.88492",

"0.16294, 0.42466, 0.73709, 1.10670",

"0.20836, 0.55820, 0.99390, 1.42270

Library (characterization)

0.09 pF

Inpu

t tra

nsiti

on

Output capacitance

Transition = 0.78773 ns

Delay = 0.51005 ns


…Technique: Delay Calculation…

• For custom blocks (analog, memories)• Using timing models

15

Min, Max timing

CLOCK

Setup, hold time Access time


…Technique: Block Level…

• Goal: • check any timing path is within the constraints (min < TPath < max)

• Timing path

16

Combo Logic

Input port Output port

Clock pin of asequential device

Data pin of asequential device

Start Point End Point


…Technique: Block Level…

• Clock management

17

Clock pin

Tdelay

skewFFx

FFy

Transition at the end points


…Technique: Issuing reports…

• DRC issues• Timing values reported by the tool are reliable if transition and

capacitance are within the maximum used for library characterization

18

library.lib (STF)" 0.04046, 0.15501, 0.40826, 0.78816",

"0.04465, 0.15978, 0.41281, 0.79198",

"0.10748, 0.26067, 0.51005, 0.88492",

"0.16294, 0.42466, 0.73709, 1.10670",

"0.20836, 0.55820, 0.99390, 1.42270

Library (characterization)

Inpu

t tra

nsiti

on

Output capacitance

0.93709

Max capacitance violation

1.00274 Max transition violation

Manufacturing yield


…Technique: Issuing Reports…

• Coverage

Type of Check Total Met Violated Untested---------------------------------------------------------------------setup 3119 1176 ( 38%) 9 ( 1%) 1934 ( 62%)hold 3120 1168 ( 37%) 17 ( 3%) 1935 ( 62%)

...--------------------------------------------------------------------All Checks 11379 6753 ( 59%) 40 ( 7%) 4586 ( 34%)

19

May be due to:

- Environment assumption (e.g. report done in functional mode)- Design constructs (e.g. combinational loops cannot be verified)


…Technique:Top level…

• Hierarchical analysis is based on timing models for blocks• Timing model may be extracted from the netlist• Or “carved” netlist may be used

20

Combo Logic

Combo Logic Combo Logic

Internal paths are removed


…Technique: Top Level

• Clock tree impact at top level• Clock tree balancing

between blocks, IPscan be checkedonly at top level

21

Block 1Clock tree depth = 1.3 ns

IPClock tree depth = 0.7 ns

PAD


Limitations…

• RTL coding for Static Timing Analysis• Asynchronous constructs cannot be verified

• Asynchronous inputs• Paths between asynchronous clock domains• Combinational loops

22


…Limitations

• Accuracy of timing models• Need a continuous improvement in the accuracy of timing

models at standard cell level! (from linear -> NLDM -> CCS)• Need a strong verification of timing model at block or IP level

(ILM models)

• Runtime• More than 1 day for flat chip level analysis• Can be done in a hierarchical mode (if accurate models!)

• Note: always fastest than a simulation!

23


Tools

• Primetime (Synopsys)• Accuracy proven versus transistor level timing analyzers• Chip level capability (timing models management)• Runtime: more than 1 day for large chips

24


Deliverables

• Reports• Max capacitance, transition• Coverage: including details about the untested nodes• Clock skews all over the chip

25

ConclusionDesign Verification Techniques

Stating Timing Analysis / Conclusion …

Training Objective

• At the end of this class you are able to :

• Understand what is a Functional Verification• What is the basic taxonomy of functional verification• Discover the different Verification Techniques with a list of tools

and languages• Understand how to apply Vtechniques on a design

• Understand what is a temporal verification and the link with the functional verification

2


Verification in the Design Flow 3

Acceleration

Emulation

PrototypingSynthesis

Floorplan

Place & Route

Extraction

RTL Entry



+ Parasitics

Layout

Architecture

Transistor

Testbench Coverage

Property Checking




-Architecture


People involved in Verification 4

Architect- chip analysis- block partitioning- detailed specifications

Designer (HW, SW)- block description (C, HDL)- block synthesis (Ass, Gate)- block validation (Simulation,Timing)- block implementation (P&R)- block verification (LVS, DRC)

Block & IP Integrator-chip assembly- chip verification vs spec

Verification Engineer- coverage driven verification- emulation- acceleration- equivalence checking- ABV (property checking, ..)

CAD Support- CAD tools, flow- Training

- simulation, acceleration- ABV - equivalence checking- Static Timing Analysis


5

Documents

ftv07 9 concl T.ppt - ENSIMAG · • Verification Navigator 6.2 + NCSim 3.2 + Ultra 10 Statement Branch Condition Path Trigger Signal 25% 25% 12000%! 25% 25% ~2000% ... Introduction