1

SoC Verification Strategies for Embedded Systems Design

November 5-6, 2003/ Seoul

Chong-Min Kyung,

KAIST

SOC Design Conference

2

Various Embedded Mobile Systems

Desktop PC

Data Processing Consumer

Communication Automotive & etc.

DTV

MediaCenter

GameConsole

DSC

DVC

Car Navigation

DVD

Telematics

MP3 Player

Credit Card

Cellular Phone

Notebook PC

Internet

Smart Phone PDA

Bluetooth

Audio

3

Difficulties in Embedded Systems Design(What‟s special in ES design?)

Real operating environment of ES is difficult to reproduce. -> In-system verification is necessary.

Total design flow of ES until the implementation is long and complex. -> Verification must be started early.

Design turn-around time must be short as the life-time of ES itself is quite short. -> Short verification cycle

4

Three Most Important Issues in Verification

1. Verify Right :

Always make sure you have correct specifications to start with. (Frequent interaction with SPECIFIER, customer, marketing, etc.)

In-System Verification

Check Properties in Formal Techniques.

2. Verify Early

System-level, Heterogeneous Models, SW-HW

3. Verify Appropriately

HW-SW Co-simulation

4. Verify Fast

Hardware Acceleration, Emulation

5

Strongly Required Features of Verification

Accommodate Multiple Levels of Design Representation

Exploit Hardware, Software and Interfacing mechanisms as Verification Tools

6

Virtual Chip : “Verify Early, In-System”

Virtual Chip: Making Functional Models Work on Real Target System [DAC98]

Simulating ISS of a Processor Chip along with real target environment

Target board

Host computeras Virtual Chip cable

Pin Signal Generator with Buffers

Chip Model;ISS

Chip Socket

7

PC

running

ISS

Without

CPU

Picture taken in 1993 or 1994

8

Reducing TTM using Virtual Chip

idle

Conventional design flow

Architectural

model

RTL

model

Gate-level

model

H/W

Emulation

Verification w/ H/W

H/W prototype

(H/W emulation)

Board

design

H/W System

Application

S/W

idle

design

Virtual Chip design flow; EARLY ,IN-SYSTEM

Architectural

model

H/W prototype

(Virtual Chip)

Board

design

H/W System

design

RTL

model

Gate-level

model

H/W

Emulation

Verification w/ H/W

Application

S/W

Design time is

drastically reduced

9

Microprocessor Design Verification Methodology

Instruction

Behavior

In C

(Polaris)

Micro-

architecture

in C

RTL

Verilog

Gate-Level

Verilog

Real

Mother-board

H/WVirtual PC in C (VPC)

C Language

Target Environment

HDL

MCV : Microcode Verifier PLI : Programming Language Interface

more refined model

MCVFlexPCVirtual Chip PLI

CPU Model

10

HK386 (1995)

Design Specification Instruction level, Pin-to-Pin compatible with

i386 Operation speed : 40 MHz 0.8 m DLM CMOS ASIC

Test Programs MS DOS 6.0, Windows 3.1, Office 4.0 CAD tools, games, etc..

MS Win. 3.1 MS Office MaxPlus II

11

What about Emulation?

Swallowed a lot of Money

Required a separate, air-conditioned room

Needed a lot of time for Compile

Needed an expensive slowed-down PC ($20,000 for 500kHz clocking)

However…worked at the last minute

Finding bugs is not easy

12

x86 Emulation Configuration

HardwareEmulator

500kHz Slowed-Down PCProbeModule

Target Interface

Board

13

Booting Windows 20M instructions on Marcia

0

2000

4000

6000

8000

10000

12000

14000

16000

18000

20000

0 5 10 15 20 25 31 36 41

Time (weeks)

Instr

uctions (

thousand)

HDL saver

Attached

Windows

DOS

HDL Simulation Hardware

Emulation

setup

version

update 1

version

update 2

version

update 3

Simulation debugs,

Emulation approves.

14

Agenda

Verify Early and Altogether (In-System)

Why Verification ?

Verification Alternatives

Verification with Progressive Refinement

SoC Verification

Concluding Remarks

15

Trend of Verification Effort in the Design

Verification portion of design increases to anywhere from 50 to 80% of total development effort for the design.

Code Verify (30 ~ 40%)Synthesis P&R

Code Verify (50 ~ 80%) Synthesis P&R

1996300K gates

20001M SoC

Verification methodology manual, 2000-TransEDA

16

Percentage of Total Flaws

About 50% of flaws are functional flaws.

Need verification method to fix logical & functional flaws

From Mentor presentation material, 2003

Clocking

5%

Race

5%

Power

4%

Other

9%

Yield

7%

Noise

12%Slow Path

13%

Logical/

Functional

45%

17

Another recent independent study showed that more than half of all chips require one or more re-spins,

and that functional errors were found in 74% of these

re-spins.

With increasing chip complexity, this situation could worsen.

Who can afford that with >= 1M Dollar NRE cost?

18

Bug Fixing Cost in Time

Cost of fixing a bug/problem increases as design progresses.

Need verification method at early design stage

Verification methodology manual, 2000 – TransEDA

BehavioralDesign

RTLDesign

Gate LevelDesign

DeviceProduction

Cost ofFixing

a Problem

19

Verification Performance Gap; more serious than the design productivity gap

Growing gap between the demand for verification and the simulation technology offered by the various options.

System-on-a-chip verification, 2001 – P.RashinkarDesign complexity

Sim

ula

tio

n p

erf

orm

an

ce

Ve

rifi

ca

tio

n c

om

ple

xit

y

Verification Performance Gap

SmallASIC

MediumASIC

ComplexASIC

SOC

20

Completion Metrics; How do we know when the verification is done?

Emotionally, or Intuitively;

Out of money? Exhausted?

Competition‟s product is there.

Software people are happy with your hardware.

There have been no bugs reported for two weeks.

More rigorous criteria;

All tests passed

Test Plan Coverage

Functional Coverage

Code Coverage

Bug Rates have flattened toward bottom.

21

Verification Challenges

Specification and/or Operating Environment is Incomplete/Undetermined. (Just like last-minute ECO!)

Cannot avoid using Yesterday‟s tool for Today‟s Design.

Design productivity grows faster than Verification productivity. (Verification productivity Gap)

Besides Functional Verification, Estimation on Performance, Power Consumption becomes crucial issues in SoC design.

22

Agenda

Verify early and in-system

Why Verification ?

Verification Alternatives Simulation

Hardware-accelerated simulation

Emulation

Prototyping

Formal verification

Semi-Formal (Dynamic Formal) verification

Verification with Progressive Refinement

SoC Verification

Concluding Remarks

23

Overview of Verification Methodologies

Simulation

Hardware

Accelerated

Simulation

Emulation

Formal

Verification

Semi-formal

Verification

Prototyping

Basicverificationtool

24

Software Simulation

Pros

The design size is limited only by the computing resource.

Simulation can be started as soon as the RTL description is finished.

Set-up cost is minimal.

Cons

Slow (~100 cycles/sec) ; Speed gap between the speed of software simulation and real silicon widens. (Simulation speed = size of the circuit simulated / speed of the simulation engine)

The designer does not exactly know how much percentage of the design have been tested.

25

Hardware-Accelerated Simulation

Simulation performance is improved by moving the time-consuming part of the design to hardware.

Usually, the software simulation communicates with FPGA-based hardware accelerator.

Simulation environment

Testbench

Module

0

Module

1

Module 2

Hardware

Accelerator

Module 2 is

synthesized &

compiled into

FPGAs

26

Hardware-Accelerated Simulation

Pros

Fast (100K cycles/sec)

Cheaper than hardware emulation

Debugging is easier as the circuit structure is unchanged.

Not an Overhead : Deployed as a step stone in the gradual refinement

Cons (Obstacles to overcome)

Set-up time overhead to map RTL design into the hardware can be substantial.

SW-HW communication speed can degrade the performance.

Debugging of signals within the hardware can be difficult.

27

Emulation

Imitating the function of another system to achieve the same results as the imitated system.

Usually, the emulation hardware comprises an array of FPGA‟s (or special-type processors) and interconnection scheme among them.

About 1000 times faster than simulation.

Simulation

Hardware

Accelerated

Simulation

Emulation

Prototyping

28

Emulation

Pros

Fast (500K cycles/sec)

Verification on real target system.

Cons

Setup time overhead to map RTL design into hardware is very high.

Many FPGA‟s + resources for debugging high cost

Circuit partitioning algorithm and interconnectionarchitecture limit the usable gate count.

29

Emulation

Challenges

Efficient interconnection architecture and Hardware Mapping efficiency for Speed and Cost

RTL debugging facility with reasonable amount of resource

Efficient partitioning algorithm for any given interconnection architecture

Reducing development time (to take advantage of more recent FPGA‟s)

30

Prototyping

Special (more dedicated and customized) hardware architecture made to fit a specific application.

Simulation

Hardware

Accelerated

Simulation

Emulation

Prototyping

31

Prototyping

Pros

10X higher clocking than emulation

Components as well as the wiring can be customized.

Can be carried and dispatched for demo or customer evaluation

Cons

Not flexible for design change

(Even a small change requires a new PCB.)

32

Overview of Verification Methodologies

Formal verification

Application of logical reasoning to the development of digital system

Both design and its specification are described by a language in which semantics are based on mathematical rigor.

Semi-formal verification

Combination of simulation and formal verification.

Formal verification cannot fully cover large designs, and simulation can come to aid in verifying the large design.

SimulationFormal

Verification

Semi-formal

Verification

More complete verification

33

Formal Verification

Classes Model Checking : verifies that the design satisfies properties

specified using temporal logic

Equivalence Checking

Symbolic Simulation

Theorem Proving

Pros Assures 100% coverage.

Run fast.

Cons Works only for small-size finite state systems due to memory

explosion. (so far, <500 flip-flops)

Uncomfortable, i.e., need to learn temporal logic for “property” description in Model Checking

34

Efforts by Formal People to reduce the Complexity

Reachability analysis

Design state abstraction

Design decomposition

35

Semi-Formal Verification - Assertion

Assertion-based verification (ABV)

“Assertion” is a statement on the intended behavior of a design.

The purpose of assertion is to ensure consistency between the designer‟s intention and the implementation.

36

Semi-Formal Verification - Assertion

Simulation Quality of assertion-based verificationN

um

be

r o

f b

ug

s f

ou

nd

Time, EffortSetup

testbenchDescribe

assertions

Formal verificationSimulation

Simulation with assertionsEfficiency of

assertion

By IBM in “Computer-Aided Verification” 2000

37

Semi-Formal Verification - Coverage

Coverage-directed verification

Increase the probability of bug detection by checking the „quality‟ of stimulus

Used as a guide for the generation of input stimulus

Test Plan

(Coverage

Definition)Directives

Random

Test

GeneratorTest Vectors

SimulationCoverage

analysis

Coverage

Reports

38

Semi-Formal Verification

Pros

Designer can measure the coverage of the test environment as the formal properties are checked during simulation.

Cons

Simulation speed is degraded as the properties are checked during simulation.

Challenges

Guiding the direction of test vectors to increase the coverage of the design.

Development of more efficient coverage metric to represent the behavior of the design.

39

Speed Comparison

0 kHz

Software

Simulation

10 kHz

1MHz

Hardware-

Accelerated

Simulation

(from

Quickturn/Dynalith

Presentation)

Hardware

emulation

(from Quickturn

presentation)

100kHz

500KHz

100Hz

100 kHz

Speed (Cycles/sec, log scale)

10MHz1~10MHz

Prototyping Semi-formal

(Assertion-

based

verification)

50-70Hz

40

Verification Time vs. Coverage (not to exact scale): Which best fits your application?

Coverage

Verification Time

Simulation

Semi-formal

Prototyping

Emulation/Simulation Acceleration

Simulation setup

Semi-formal setup

Emulation/SA setup

Prototyping setup

41

Agenda

Verify early and in-system

Why Verification ?

Verification Alternatives

Verification with Progressive Refinement

Flexible SoC verification environment

Debugging features

Cycle vs. transaction mode verification

SoC Verification

Concluding Remarks

42

Criteria for Good SoC Verification Environment

Support various abstraction levels

Support heterogeneous design languages

Trade-off between verification speed and debugging features

Co-work with existing tools

Progressive refinement

Platform-based design

43

Core model

ISS

Transaction-Level Modeling

Model the bus system in transaction-level

No notion of exact time.

But precedence relation of each functional block is properly modeled.

Rough estimation on performance is possible.

Used as the fastest reference model by each block designer

Memory IP IP

Transaction-level Bus model

Rough

Performance

Estimation

...

...

Behavior

model

---

---

---

---

---

---

Software design Hardware design

transaction

44

Core model

ISS

Cycle-Accurate Bus Modeling

For more accurate modeling

Build a cycle-accurate system model in C or SystemC

Replace the transaction-level bus model with a cycle-accurate bus model

ARM released a “Cycle-Level Interface Specification” for this abstraction level.

Memory

IP

IP

Cycle-accurate Bus model

Accurate

Performance

Estimation

...

...

Behavior

model

---

---

---

---

---

---

transaction

BFM BFM BFM

RTL

model

45

AMBA AHB CLI Specification

AMBA AHB Cycle Level Interface (CLI) Specification

CLI spec defines guidelines for TLM of AHB with SystemC.

Interface methods

Data structures

Header files for SystemC models

CLI spec leaves the detailed implementation of the AHB bus model to the reader.

master

slave

master

HADDR

HWRITEHRDATA

HTRANS

slave

Read 10 data from slaveX startingAddress 0x10

Cycle-level modeling Transaction-level modeling

46

AMBA AHB CLI Specification

Example master implementation Transactions are represented as methods in transaction-level modeling.

Abstraction levels of each method can be decided as needed such as cycle-count accurate, cycle-accurate, transaction accurate, etc.

void masterA::run(){bus_request();has_grant();init_transaction();read_from_slave();...

}

Header Body

The granularity of transactions are decided according to the verification needs.

SC_MODULE(masterA) {......void run();

};

masterA.h masterA.cpp

47

Flexible SoC Verification Environment

Build C reference model for the target application.

Setup of platform-level verification environment as early as possible

JPEG decoding

HEAD VLD IDCT DISP

Verific

atio

n E

nviro

nm

ent S

etu

p

Functional block model

TRS TRS TRS TRS

HEAD VLD IDCT DISP

Platform-level model

Algorithm

Memory Timer INTC

SW Model

HW Model

48

Flexible SoC Verification Environment

Transactor connects various types of SW block models with HW bus system.

Several types of transactors must be prepared, for example

AHB C model

AHB HDL model

OPB Testbuilder

PLB SystemC

JPEG decoding

HEAD VLD IDCT DISP

Verific

atio

n E

nviro

nm

ent S

etu

p

Functional block model

TRS TRS TRS TRS

HEAD VLD IDCT DISP

Platform-level model

Algorithm

Memory Timer INTC

49

Flexible SoC Verification Environment

Socketize IP representation

HW: C HDL EDIF

SW: native C ISS Processor Core

C

Transactor

HDL

AHB

C to HDL

EDIF

Synthesis

Transactor

C

AHB

ISS

AHB

Processor

Cross-compiler

Transactor

Transactor

Transactor

SW partmodel

HW partmodel

50

Cycle-Level Transactor

Generate stimulus at every clock cycle

Check the result of DUT at every clock cycle

PCIController

DUTPCIChannel

Testbench

Testbench DUT

FPGA part

DeviceDriver

S/W simulation part

Cycle-leveltransactor

51

Transaction-Level Transactor

Only information to generate transaction is transferred to DUT, i.e., address and data

No need to synchronize at every clock cycle

PCIController

DUTDMAChannel

Testbench

Testbench DUT

FPGA part

DeviceDriver

S/W simulation part

Tra

nsacto

r

MainMemory

52

Cycle vs. Transaction-level Transactor

Cycle-level transactor

Synchronized at every clock cycle.

Operating speed depends on the number of signals to be transferred.

Transaction-level transactor

Synchronized at the end of each transaction.

Transactor must be designed for each interface standard

ex) AHB transactor, SDRAM transactor, IIS transactor

53

DFV (Design for verification) : OpenVera (OV) verification IP

Reusable verification modules, i.e.,

1) bus functional models,

2) traffic generators,

3) protocol monitors, and

4) functional coverage blocks.

54

RTL Debugging Feature

Gate-level netlist generated from the synthesis tools is difficult to trace manually.

Techniques to resolve RTL symbol names from the gate-level symbol names and to provide debugging environment in RTL name spaces is required.

Insert RTL instrumentation IP for debugging

Design flow

Read RTL design (Verilog, VHDL)

Generate instrumented RTL design (spiced with triggering and dump logic)

Synthesis

Compile (mapping & PAR)

DiaLite (Temento), Identify (Synplicity)

55

Synthesizable Testbench

Prepare a set of programmable test bench module which can be also synthesized into hardware.

To verify a DUT, build a test bench using the programmable test bench.

The test bench is applicable to both simulation and emulation in the same fashion.

from Duolog TechnologGeneric Mini-Controller(GMC)

gmc_cfg_datasize 3 # Set to 32 Bits

gmc_cfg_msmode 0xF # Set to Master TX

gmc_cfg_datapath_0x00 # Read/Write to Mem

gmc_cmd_send_data 100 # Send 100 Bytes

uart_cfg_odd_parity 1 # Set ODD parity

56

Agenda

Verify early and in-system

Why Verification ?

Verification Alternatives

Verification with Progressive Refinement

SoC Verification

Co-simulation

Co-emulation

Concluding Remarks

57

SoC Verification

Co-simulation

Connecting ISS with HDL simulation environment

Seamless, N2C

Co-emulation

Emulation/rapid-prototyping equipments supporting co-emulation

ARM Integrator, Aptix System Explorer,

AXIS XoC, and Dynalith iPROVE

58

What‟s the point in SoC Verification?

Mixture of SW and HW

Let the HW model to cooperate with Processor Model such as ISS or BFM (Bus functional model)

Mixture of pre-verified, unverified components

Utilize legacy IPs already verified

Mixture of different language, different abstraction levels

Provide common interface structure between SoC components

59

Canonical SoC design flow

SystemSpec.

SystemDesign

HW/SWPartitioning

HWDevelopment

SWDevelopment

HW refinement(UT->T->RTL)

Gate

HW IP

SW IP

SoftwareVerification

FunctionalVerification

Gate-LevelVerification

HW-SWCo-Design

HW-SWCo-

Verification

SW refinement(RTOS

mapping)

Final code Emulator

In-system emulator

HW-SW co-debugging

60

Tools for HW-SW Co-Verification

HW-SW co-simulation

ISS

RTOS simulator

HW/SWPartitioning

HWDevelopment

SWDevelopment

HW refinement(UT->T->RTL)

SoftwareVerification

FunctionalVerification

Co-Verification

SW refinement(RTOS

mapping)

HW-SW

High-level synthesis

Testbench automation

IP accelerator

SystemSpec.

SystemDesign

HW/SW

HW IP

SW IP

61

Tools for System-level Verification

System-level design (Performance analysis tools)

Hot-spot analyzer

High-level cycle count estimation

High-level power analysis

High-level chip area estimation

On-chip-bus traffic estimation

SystemSpec.

SystemDesign

HW/SWPartitioning

HW IP

SW IP

HW-SWCo-Design

62

Co-Simulation Tools

Software debugging in ISS and hardware verification in HDL simulator are done in parallel way.

Co-simulation stub manages the communication between HDL simulator and ISS.

The most accurate solution albeit very slow

ISSHDL

Simulator

Co-simulationStubDebugger

RemoteDebuggingInterface

63

Co-Emulation Tools

Link hardware emulators with a processor model running in host machine or an actual processor core module

Most emulation and rapid prototyping products support linkage with ISS running in host machine

As the emulator runs very fast, speed of ISS including memory access as well as synchronization between emulation and ISS rise as new bottlenecks in verification

64

Example) ARM ETM (Embedded Trace Macrocell)

Real-time trace module capable of instruction and data tracing dedicated to the ARM core family

Triggering enables to focuscollection around the region of interest

Trace module controls theamount of trace data by filtering instructions or data

Only applicable to the ARMcore debugging

ARMcore

Memories Peripherals

Oth

er

com

ponents

ETM

Target system with ARM based ASIC

PC-based debugger

Trace

Trigger

Traceport

analyzer

JTAGinterface

Conf.

65

Typical Co-Emulation Environment

Connect ARM ISS or ARM board to the emulation system.

ARM ISSEmulationSystem

ARMDevelopment

Board

EmulationSystem

ARM ISS can be configured to user‟s target SoC architecture.

SW debugger can be fully utilized.

Faster than ISS.

Ready-made ARM development boards has fixed architecture which may be different from the user architecture.

66

Example 1: ARM Integrator

ARM prototyping platform

Composed of

Platform : AMBA backbone + system infrastructure

Core Module : various ARM core modules (up to four)

Logic Module : FPGA boards for AMBA IP‟s

Allows fast prototypingof ARM-based SoC

Enables co-emulationof both software in ARMprocessor core andhardware in the FPGA

Difficult to debug hardwarelogic

from ARM

67

FPGA

Hardware Sessions

C/SystemC/HDL sessions

HardwareProcessor

Logic Designin EDIF/FPGA

Transactor Transactor

Target board

I/F protocol I/F protocol

Inter-Lingual Communication

Designin C

Example 2 : iSAVE/iPROVE (Dynalith)

All-in-one verification :

Heterogeneous models including ARM ISS, C hardware model, HDL hardware description

SW models run in Linux-based PC

HW models run in FPGA‟s, or

as a processor chip

HW/SW Co-debugging with

Pin Signal Analyzer

On-the-fly probing of FPGA

internal signals to SRAM

Communicate with C modelusing PCI DMA

ISS sessions

ISSDesignin SC

Designin HDL

68

Agenda

Verify early and in-system

Why Verification ?

Verification Alternatives

Verification with Progressive Refinement

SoC Verification

Concluding Remarks

69

So, in Summary…

Verification is challenging; It needs strategy!

Strategy is to apply each method when appropriate.

Verify with Correct Specification

In-System Verification

Verify as early as possible; Kill the bug when it is small and still isolated in a smaller region. (Kill it before it grows big and kills you(r design and your company).)

Accelerate the verification cycle using

Efficient debugging tools

Hardware Accelerator

70

A Tactical Sequence of Applying Various Verification Tools

1st step: Apply formal methods

Static formal verification

Assertion-based verification

2nd step: Simulate IP

with transaction level test-bench

with cycle-accurate test-bench

3rd step: Emulate design

Emulate IP operation in FPGA

In-system IP verification

Cycle-level vs. transaction level test-bench

71

Concluding…

Main differences of SoC with ASIC design are

Planned IP-reuse

Reuse of pre-verified platform

Focus on co-verification with software

Newly added IP‟s must be thoroughly verified before its application to SoC.

Well-established verification platform allows progressive verification/refinement of each IP, i.e., each SoC component while assuring correct implementation.

Powerful HW/SW co-simulation and co-debugging features are required.

72

Thank you for your kind attention!