Mentor Graphics TVS Presentation 130319a...© 2013 Mentor Graphics Corp. 2 HF -February 2013, DVCon Lunch Conducted by Wilson Research Group — Commissioned by Mentor Graphics —

SoC Design VerificationVerification Futures Conference

Mark OlenVerification Intrapreneur

Mentor Graphics Corp

March 19, 2013

© 2013 Mentor Graphics Corp.

www.mentor.comHF - February 2013, DVCon Lunch2

� Conducted by Wilson Research Group— Commissioned by Mentor Graphics— Format followed 2002, 2004 Collett studies for trend analysis, as

well as the 2007 FarWest Research Study

� Worldwide study— Overall confidence of 95% plus/minus 4.05%

� This was a blind study! — To eliminate any bias in the results

� This was a balanced study! — No single vendor dominated responses Wilson Research Group

Study Overview

www.mentor.com© 2013 Mentor Graphics Corp. Company Confidential

Time Design Engineers Spend on Verification vs. Design

54%

2007

46%

2007

47%

2012

53%

2012

Doing Design Doing Verification

Design Engineer Project Time2007 - 2012

WCR DVCon February 20133


Verification Consumes Majority of Project Time

0%

5%

10%

15%

20%

25%

1%-20% 21%-30% 31%-40% 41%-50% 51%-60% 61%-70% 71%-80% >80%

Non-FPGA Study Participants

Time (Percent)

Total Project Time Spent in Verification

2007

2010

2012

2007: Mean 49%2010: Mean 56%2012: Mean 56%

Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study



More and More Verification Engineers Average Peak Number of Design vs. Verification Engineers

7.8 8.1 8.5

4.8

7.68.4

2007 2010 2012

~ 1-to-1 ratio of peak design and verification

engineers

58% 11%

4% 5%

Verification Engineers

Design Engineers

Source: Wilson Research Group and Mentor Graphics, 2012 Functional Verification Study



SOC Verification Challenges Broaden

Verification Futures Conference

Infineon

nVidia

STMicro

Broadcom

ST Ericsson

TI

Wolfson

Dialog

Productivity/Performance � � � � � � � �

Debug � � � �

Verification Complexity � � � �

Coverage & Metrics � � � � �

Low Power Verification � � � � �

SoC & HW/SW Verification � � � � �

Verification IP & Reuse � � � �

Recruiting & Resources � � � �

Source: Mentor Analysis



On-Time Completion Remains ConstantNon-FPGA Project’s Schedule Completion Trends

0%

5%

10%

15%

20%

25%

30%

35%

More than 10%EARLY

ON-SCHEDULE 20% 40% >50% BEHINDSCHEDULE


2007

2010

2012

2007: 67% behind schedule2010: 66% behind schedule2012: 67% behind schedule

Behind ScheduleAhead of schedule




It’s an SoC World~80% of Designs Contain One or More Embedded Processors

21%22%

28%

6%

10% 13%

0%

10%

20%

30%

40%

50%

60%

NONE 1 2 3 4 5 or MORE


Number of Embedded Processors for Non-FPGA Designs

2004

2007

2010

2012




Embedded Processors per Design Size

0.91

1.86

2.67

0

1

2

3

< 5M 5 - 20M > 20M

Average Number of Embedded

Processors

2012 Design Size (Gate Count Excluding Memories)




MACRO TREND #1:

Standardization of Verification Languages

0%

20%

40%

60%

80%

VHDL Verilog Synopsys Vera System C SystemVerilog Specman e C/C++ OTHERTestbench


Languages Used for Verification (testbenches)

2007

2010

2012

SystemVerilog grew 8.3% between 2010 and 2012




SystemVerilog Adoption by Design Size

59%

71%

89%

0%

20%

40%

60%

80%

100%

< 5M 5 - 20M > 20M


SystemVerilog Adoption by Design Size (Gate Count Excluding Memories)




Standardization in Base Class Libraries

0%

10%

20%

30%

40%

AccelleraUVM

OVM MentorAVM

SynopsysVMM

SynopsysRVM

CadenceeRM

CadenceURM

Other


Testbench Methodologies and Base-Class Libraries

2010

2012

486% UVM growth between 2010 and 201246% UVM projected growth in the next twelve months




MACRO TREND #2:

Standardization of the SoC Verification Flow

� Emerging from ad hoc approaches to systematic processes

Unit Level ConnectivityIP

Integration /Data Path

System Level



Aspects of SoC Verification

Block-Level

Verification

Interconnect

Verification

Datapath

Verification

Application/SW

Verification

•Interface-centric•Test under all conditions

•Interconnect is a block unto itself•1000s of connections to check

•SW-driven•Register access•Ensure memory map is correct

•Basic power control

•Couple SW with external stimuli

•CDC•Power control adds levels of complexity

Avoid the “HW works, SW works,

System doesn’t” problem

DVCon 2013 Tutorial - We've Got You Covered14


The Verification Paradox

� A good verification process lets you get the most out of best-in-class verification tools

Start ToolsAd Hoc

Processes6-9%

Cost Increase

Start Process Tools20-30%

Cost Savings

Source: Cisco Momentum Research Group



Unit-Level Checking

� Goal: Verify the functionality is correct for each IP while achieving high coverage

� Traditional techniques used include



System Level

— Simulation with directed, constrained-random, graph-based testbenches— Lots of verification IP (VIP)— Coverage techniques— Assertion-based verification— Formal property checking



� GOAL: Verify the functionality is correct for each IP while achieving high coverage

� Goal: Ensure IP blocks are connected correctly

Connectivity Checking



System Level

— Verify full connectivity, not simply point-to-point connections— Validate numerous configurations and operation modes



IP Integration and Data Path Checking



System Level

� Goal: Ensure IP blocks are integrated correctly

— Access all memories— Access all registers, such as control— Verify multiple clock domain crossings



� Goal: Performance, power analysis and SoC functionality

System-Level Checking

� SOC ‘features’ that need to be verified



System Level

— Verify functional interaction between multiple IP blocks—firmware, middleware, application focused testing

— Requires all IPs to be integrated at this level, including software

— Power control — Functional use-cases to stress interacting IP components— System performance testing— Firmware and other software for signoff



MACRO TREND #3: Coverage and Power across All Aspects of Verification

� What gets measured, gets done

� What gets measured, gets improved

� What gets measured, gets managed

– Peter Drucker



Coverage IntroductionWhat are the Types of Coverage?

Coverage Closure with the Questa Platform21

� Origin of Source

— Specification

— Implementation

� Method of Creation

— Explicit

— Implicit

Explicit

Specification

Implicit

Implementation

Functional

Code

Assertion

Input Output

Protocol

FSM

Power

State

Clock

Domain

Toggle

Branch

Statement

Line


Coverage IntroductionFundamental Metric Across the Entire Verification Process

22

� Verification Planning— Requirements Mapping— Coverage Planning

� Testbench Creation— Coverage Modeling— Stimulus & Environment Creation

� Simulation-Based Techniques— Dynamic Verification

� Formal-Based Techniques— Static Verification

� Analysis & Reporting— Analyzing— Ranking & Merging— Reporting

ANALYSIS &REPORTING

SIMULATIONBASEDTECHNIQUES

FORMALBASED

TECHNIQUES

TESTBENCHCREATION

VERIFICATIONPLANNING

PLANNING

MODELING

TARGETING

ACHIEVING MEASURING

ANALYZING

RANKING MERGING

REPORTING


Unified Coverage Interoperability Standard New Accellera UCIS Standard Announced at DAC 2012

UCIS API

Coverage Database

Simulation

Formal

Emulation

Analysis Testplan



Low Power Design Requires Multiple Verification Approaches

Design Functionality

Power Management Architecture

Power Domain Behavior

Power Domain Interactions

Power State Transitions

Power Control Hardware

Power Control Software

System with Power Management

Verify that design functions correctly with power always on

Verify that power management architecture is correct

Verify power up/down and reset/restore on power up for each block/power domains

Verify that interfaces are correctly isolated and level shifted

Verify that HW generates correct control signals in correct sequence

Verify SW Power Control interfaces with HW is correct

Verify full system with HW and SWfor power control

Verify that all transitions are covered and behave correctly with power control



Trends in Power Management Verification

0% 20% 40% 60% 80%

Power domain powerdown/power up

Power domain statereset/restoration

Transitions between systempower states

Hardware power controlsequence generation

Interactions between power domains

Operation in each system power state

Application-level power management

Hypervisor/OS control of powermanagement

Aspects of Non-FPGA Power Managed Design That Are Verified

67% of the industry actively manages power




Macro Enablers in VerificationDelivering Order-of-Magnitude Improvements

Intelligent test bench

Multi-engine verification platforms

Application-specific formal









Time

Coverage

Functional Coverage Achieved vs Project Time

� Accelerates Coverage Closure— No wasted stimulus

— Faster ramp to coverage goals

— Fewer simulation resources

� Multiplies Existing Tests— Re-use stimulus model

— Expand coverage model

— Generate even more tests

— Added tests will cover more

CRTDT

iTBA

10X Faster

Questa UltraIntelligent Testbench Automation Benefits

28 Questa Platform - 100X Faster Functional Coverage Closure

CRTDT

iTBA

Time

Coverage

Functional Coverage Achieved vs Project Time

10X Faster

100X More Tests


Intelligent Test Bench Technology Accelerates Coverage Closure

OfficeOfficeProducts 3 Days

36 hours on 6 CPUs36 hours on 6 CPUs8 weeks on 6 CPUs8 weeks on 6 CPUs100% coverage100% coverage60% coverage

37 X faster37 X faster+ 40% coverage+ 40% coverage

�� ==

WirelessWirelessNetworking 35 Hours

48 CPU hours48 CPU hours3175 CPU hours3175 CPU hours97% coverage97% coverage95% coverage

66 X faster66 X faster+ 2 % coverage+ 2 % coverage

�� ==

ConsumerConsumerElectronics 1 Day

118 minutes118 minutes>18 hours100% coverage100% coverage100% coverage

�� ==

Storage & Networking 1 Week196,000 tests196,000 tests26,315,000 tests26,315,000 tests100% coverage100% coverage79% coverage

�� ==

BasestationBasestationTelecom unknown

75,000 vectors75,000 vectors825,000 vectors825,000 vectors100% coverage100% coverage100% coverage

10 X faster10 X fasterEqual coverageEqual coverage

�� ==

IndustryConversion

TimeConversion

TimePre iTB Results Results Benefits

SwitchingSwitchingSubsystems 2 Days

400,000 tests400,000 tests10,000 tests 40 X more tests40 X more tests�� ==

WirelessWirelessTelecom 6 Days

45 minutes45 minutes3 days3 days100% coverage100% coverage100% coverage

27 X faster27 X fasterEqual coverageEqual coverage

�� ==

Processors 1 Day30 minutes30 minutes5 hours5 hours

100% coverage100% coverage100% coverage10 X faster10 X faster

Equal coverageEqual coverage�� ==

37 X faster

10 X faster

66 X faster

+ 40% coverage

40 X more tests

27 X faster

10 X faster

Average customer adopting inFactincreased performance by more

than 40X

9.5 X faster9.5 X fasterEqual coverageEqual coverage9.5 X faster

170 X faster170 X faster+ 21% coverage+ 21% coverage170 X faster

+ 21% coverage









System-Level Checking

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Hardware/software

co-verification

HW Acceleration &

Emulation

FPGA prototyping Prototyping with

special test chips

No

n-F

PG

A S

tud

y P

art

icip

an

ts

2007

2010

2012




As Design Sizes IncreaseEmulation Up, FPGA Prototyping Down

18%

32%

50%

45%

57%

28%

0%

10%

20%

30%

40%

50%

60%

HW Acceleration/Emulation FPGA Prototyping


Adoption by design size forHW acceleration/emulation and FPGA Prototyping

< 5M

5 - 20M

> 20M




Design under Test

OVM/UVMSystemC/C++

Intelligent Test

TestbenchAcceleration

Assertions & Checkers

Integrated Verification EnvironmentsSimulation/Formal/Emulation/Software Verification Environments Emerge


Clocks, Connectivity,Coverage

Formal Verification

Virtual Prototype

Offline

JTAG

SW Debug

System Level

Virtual Devices & Transactors

Protocol Solutions








The Evolution of Formal TechnologyNot Just for Experts Anymore

FormalPropertyChecking

FullyAutomaticFormal

AutomatedApplications

LowEffort

HighEffort

FormalExperts

Everyone

1990s Today2000s



Larger Designs Use More Formal

20%

26%

41%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

< 5M 5 - 20M > 20M


Formal Property Checking Adoption by Design Size (Gate Count Excluding Memories)




Improve Simulation Coverage by Identifying

Unreachable Code

Identify Full ChipSoC IP Connectivity

Errors

Identify Deadlockand FSM Design

Issues

Today’s Automated Push-Button Formal

Automatic Formal

Applications

Automatic Formal

Applications

Automatic Formal

Applications

IdentifyX-Propagation

RTL Issues

ApplicationSpecificFormal



Property Checking

Automated Applications

Fully Automatic

Application-Specific Formal Includes Checking Clock Domain Crossings

� Identify metastability issues for signals crossing multiple

clock domains due to improper synchronization

� Now scales to the full chip level

Clock Domain A Clock Domain B

Synchronizer



Average Number of Clock Domains by Design Size

5.2

9.4

11.5

0

2

4

6

8

10

12

14

< 5M 5 - 20M > 20M

Average Number of Clock Domains

Average Number of Clock Domains by Design Size (Gates Excluding Memories)




Accelerating Innovation

Macro trends in verification

Macro enablers in verification

— Standardization of verification languages— Verification flow standardization— Coverage and power across all aspects of verification

— Intelligent test bench— Multi-engine verification platforms— Application-specific formal



www.mentor.com

The Verification Academy

HF - February 2013, DVCon Lunch

10,000 Member Milestone in May 2012!

Today Over 15,000 Members Registered!


www.mentor.com

Verification Academy(1) Online Verification Courses

� AMS

� Assertion-Based Verification

� CDC Verification

� Evolving Capabilities

� FPGA Verification

� OVM Basic & Advanced

� Testbench Acceleration

� Intelligent Testbenches

� UVM Basic & Advanced

� UVM Express & Connect

� Verification Planning

� VHDL 2008HF - February 2013, DVCon Lunch42


www.mentor.com

Verification Academy(3) Methodology Cookbooks

� The UVM Cookbook is the most comprehensive resource

available on UVM today!

� The Coverage Cookbook is

the first of its kind!

UVM Cookbook

Coverage Cookbook


www.mentor.com

Verification Academy

HF - February 2013, DVCon Lunch

www.verificationacademy.com

http://blogs.mentor.com/verificationhorizons/


Documents

Mentor Graphics TVS Presentation 130319a...© 2013 Mentor Graphics Corp. 2 HF -February 2013, DVCon Lunch Conducted by Wilson Research Group — Commissioned by Mentor Graphics —