Upload
dvclub
View
660
Download
5
Tags:
Embed Size (px)
Citation preview
Introducing Obsidian Software and RAVEN-GCS for PowerPC
2
Obsidian Software
Founded in 1997 by 3 ex-AMDers
Located in Austin, TX
Bootstrapped Private Company
Focus: Processors & Verification
Made Inc.500’s List of Fastest Growing Companies
3
Understanding Your Verification Methodology
Describe your processor development activities.
• Type/schedule
What is your methodology for stimulus generation?
Are you using any verification languages?•
e
•
Vera
4
Do your current test generation tools have any
limitations?
• Limits on randomness?
• Coverage holes?
• Ease of use?
How many processor verification engineers will be on
the project?
Level of experience
Understanding Your Verification Methodology
5
Obsidian’s Mission
Random Test Generators• Standard Architectures
• Custom Architectures
• Simple Processors
Verification Services
6
Resource ExpenditureTask Tool/Solution Required Effort
Correctness Checking ISS Low (Reuse ISS from other projects)
Testbench Development Memory Model Low
Coverage Measurements Architectural
Implementation
Code
Moderate
Moderate
Moderate
Stimulus Generation Directed Tests
Random Test Generators
High
High
Debug Effort Engineers Looking at Waveforms
High
7
Common Customer Challenges
Scaling existing test generators for future designs
Challenge of staffing and maintaining in-house expertise
Test generators often need rewrites for new architecture features
Test generation development is often in the critical path
Focus should be on processor, not tool generation
Understanding the cost of generating an internal test generator
The last 10% is the most difficult and highest risk
8
What do Random Test Generators Do?
Reduce the Need for Directed Tests
Tests Full Range of the Architecture
Generate Directed-Random Tests
Allow Verification Engineers to Focus on
Failing Tests
9
Model of Operation
10
The RAVEN Solution
RAVEN® is not:• A test bench language (e, vera, perl) to write a Random Test
Generator• Used to wiggle pins on buses
RAVEN® is:• A tool to intelligently generate assembly instructions to provide
test stimulus for a microprocessor• A full featured processor specific random test generator• Under complete user control: intelligently direct tests into
difficult to reach areas• Extensible – users can add new instructions
11
Architecture
Architecture DescriptionExtracted from Headers orProcessor Documentation
ISSDynamically linked
shared object
Architectural LayerLinked in
Template File
generated by
GUI or text editor
XML
C API
C++ API(function overrides)
XMLRAVEN Core
C++ libraries
CustomerCustomer or Obsidian
ObsidianGenerated
GUIXML
Legend
12
Business ModelStandard Processors Proprietary Processors
• ARMv(4-7), MIPS 32/64, X86
• Full featured generators for above cores licensed as run time licenses
• Obsidian maintains architecture layer
• Obsidian provides technical support thru tape-out including RAVEN bug fixes and work-arounds
• CISC, RISC, VLIW, SIMD, DSP
• Obsidian develops the architecture layer and delivers it as source code to customer
• Core technology licensed to customer as run time licenses
• On-going support of architecture layer can be done by customer or Obsidian
• Obsidian provides technical support for both RAVEN® core and development effort.
Verification Services : Obsidian can provide verification services at additional cost to increase verification productivity and reduce overall project risk. These services may include, training, test creation services, RAVEN customization, coverage services, and other RAVEN related activities.
13
Pricing ConsiderationsProduct or Service Description
RAVEN licensing• One year time based licensing• Quantity affects total price• Standard software support included in license price
Development services• Engineering resource billed as a result of RAVEN®
customization• Engineering resource billed as a result of custom
architecture layer development in RAVEN®-GCS
Verification Services • Test creation services, training, coverage related services, methodology integration, test planning
Intellectual Property • Instruction set simulators• Can be sold as time based license or source code
14
Relationship Solution Structure Engagement Activities Impact on Success
TrustedAdvisor
Same as below plus:Onsite RAVEN verification
engineerHigh level Methodology
consultant
Same as below plus:Work closely with team onsite to
maximize productivityEarly stage verification
planning to identify areas of focus for you and Obsidian
BestProactive services insure
success due to identification of key areas of focus and
dedication to maximum verification productivity
Strategic Partner
Same as below plus:Full time dedicated
development engineer for ongoing enhancement requests and feature development
Same as below plus:Interactive development support
for continued RAVEN enhancement in synch with your architecture changes and RAVEN productivity enhancement requests
BetterEnhanced development
support to bolster productivity at all phases in the design cycle
TechnologyProvider
RAVEN LicensingCPU Arch layer
developmentStd. Software SupportRAVEN training
Arch layer developmentISS IntegrationPhased product deliveryArch layer and RAVEN core
support
GoodHigh-end solution focused
specifically on your processor feature set
15
Engagement Model
Customer and Obsidian agree in principle to move forward together
Obsidian provides on-site 1-2 day tutorial for detailed | technical evaluation with following objectives:
• Provide in depth technical view of RAVEN
• Determine architecture challenges
• Gather requirements for test generator
Customer and Obsidian agree on proposal and SOW
16
Engagement Model
Phased delivery (3-9 months) • Integrate core ISS from customer
• Deliver architecture layer in multiple phases
Provide upgrades and maintenance for core test generator as well as guidance for customer architecture development
Provide sustaining or peak development help for customer architecture layer as an option
17
In Summary
Collett International Research 2002 Data:• 39% First SI success
• 60% of initial failures caused by logical / functional problems•
82% -
Design errors•
47% -
Specification errors•
14% -
Imported IP problems• 50% - 70% of resources consumed in verification
18
In Summary
Verification is Usually on the Critical Path to SI
Failure Costs Ranges From Bad (lots of money) to Very Bad (lost market)
• Spins
• A few months time to market
• Delay of next project
• Loss of confidence in the market
19
RTG Methodology
20
Overview of Methodology
Use random or mostly random templates to generate initial coverage
Define coverage points based on design knowledge
Write directed-random templates to hit missed coverage points
Review all templates and relax restrictions where possible
21
Step 1: Random Templates
There’s random and then there’s “random”• In an immature design some features will be disabled
• Some features have enables rather than sliders•
Enable exceptions•
Coprocessor present• Some control register bits need to be configured at reset
• Initial test may be single CPU/thread
• RAVEN architectural layer may need enhancement to configure some behaviors
22
Coverage Using Random Only Verification
Space
Expect about 80% coverage
Coveredbehaviors
23
Step 2: Define Coverage Points
Black-box• Coverage defined based on architectural definition• Coverage based on architectural specification from
block designers• Coverage of block interfaces• Should not be sensitive to implementation• Points are defined by verification engineers
White-box• Areas of greatest change or risk• Can be sensitive to implementation• Points are defined by designers
24
Distribution of Coverage Points Coverage
Points
Designknowledge andrisk assessmentdeterminesplacementand densityof coveragepoints
25
Randomtemplate coverage iscombined with directed testcoverage
Intersection of Coverage Points and Pure Random
26
Step 4: Review Templates
An engineer should examine all existing templates• Find any areas where the template is artificially restricted and
remove restrictions
• Increase the number of instructions per template where appropriate
• Make sure MP/MT behavior is enabled where appropriate
• Allow the template to be used as a random macro where appropriate
Engineer should not be the same as the person who originally created the template, if possible
27
RAVEN-GCS Introduction
28
RAVEN-GCS Features
Random test generator• Fully-random, semi-directed, and fully-directed
• Generator knows full state of the processor at each instruction boundary
•
Test files contain all register and memory updates•
Full definition of the initial register and memory image•
Full report of the final register and memory image• Never generates an invalid test
29
RAVEN-GCS Features
Easy to use• Intuitive GUI for template creation
• Templates can be run from the GUI or the command line
• User and developer manuals
• Simple and well-documented interface to simulator
• Can automatically place comments in test output
• Supports user-defined test sequences that can be used as macros
30
Expandable to all architectures• VLIW – any packet size
• Multi-processing and/or multi-threading
• Paging
• Exceptions
• CISC or RISC
• Cache support
• Floating point
• Predicated instruction execution
• Branch-not-taken code insertion
RAVEN-GCS Features
31
Biases or constraints for all supported features• Instruction
• Operands
• Addresses
• Number generation
• Operating modes
• Register values
RAVEN-GCS Features
32
Data files are used to define architecture• Much of the architecture can be modified without recompiling
• Features can be added as they become available in the RTL
• Easy to work around known RTL or simulator bugs
Test output in XML format• Easy to parse
• Source for converter for test output to customer specific format provided
•
Example provided for conversion to user-readable form
RAVEN-GCS Features
33
RAVEN – ISS interface
InstructionSet
Simulator
RAVEN Test
Generator
Register WriteMemory Write
Register Update
Memory Update
Step Simulator
Test Output
File
Template File
RAVEN generator reads template file• Populate internal event queue with requested behaviors
Generator picks instruction or event from top of event queue
• For example if the queue element is InstrTree, a specific instruction such as an “ADD register immediate” is picked
Generator picks operands/data values/memory addresses/etc based on current environment (settings and overrides)
34
ISS-Interface
Generator writes values to uninitialized registers and data memory
Generator determines exact opcode of instruction and writes it to the simulator’s memory
Generator calls simulator Step()• Simulator returns memory and register updates caused by instruction
execution
Repeats steps 2 through 7 until end of event queue
Generator writes test output file including final memory and register image
35
RAVEN-GCS GUI
36
RAVEN-MP GUI
37
Instruction Overrides
Values for instruction fields may be generated randomly or may be overridden with specific values.
Double-clicking on a canvas element invokes the appropriate override pop-up window.
38
Test Output Files
Test output files are in XML formatConsist of the following parts:
• Initial state •
memory and registers•
completely defines the state of the processor before any instructions in the test have generated
•
register state can be used for hotloading
if supported by the testbench• Initialization sequence
•
used if hotloading
is not supported•
instructions from reset to create the state needed to execute the first instruction in the test
39
Test Output Files
Consist of the following parts• Instructions
•
sequence created from the canvas•
all memory and registers updated as a result of instruction execution are reported
•
the full state of the system is known at every instruction boundary• Final state
•
fully defines the state of memory and registers after completion
of the test
40
Architecture definition
41
Architecture defintion
Most of the architecture can be specified in data files• these are all xml files• many can be parsed directly from architecture definition files such as
manuals or simulator header files
Types of files• definition of instructions and instruction trees• definition of operands and operand groups• definition of special conditions
exceptionspipelining issuespacketization rules for VLIW
• display information for the GUI
42
Instruction tree definition
43
Instruction definition
44
Operand class is AddressingModeOperand Formula specifies how to determine address
Formula: NIA+(LI << 2)NIA = next instruction addressLI = value of the LI operand (an immediate) < < = “<<“
syntax due to xml restrictions=> New address = next instruction address + (immediate << 2)
Instruction definition
45
Handling non-determinism
46
Using RAVEN Register Output
RAVEN test output files contain a complete list of expected values for all registers that affect the behavior of the test.
This example shows a simple case with no unpredictable register fields
GP1 GP2 CR1
Initial State 0x01234567 0xabcdef0 0x35742278
Instr. 1 0x55327a8b --- ---
Instr. 4 --- --- 0x35743378
Instr. 22 --- 0xabcd7a1f ---
Instr. 55 0x00000000 0x55327a8f ---
Instr. 56 --- --- 0x25743378
Instr. 121 0x00000001 --- ---
Instr. 139 0x00000002 0x55327a90 ---
Final State 0x00000002 0x55327a90 0x25743378
47
Basic Register Checking
GP1
Initial State 0x01234567
Instr. 1 0x55327a8b
Instr. 4 ---
Instr. 22 ---
Instr. 55 0x00000000
Instr. 56 ---
Instr. 121 0x00000001
Instr. 139 0x00000002
Final State 0x00000002
List of expectedvalues from testoutput file
0x55327a8b
=
0x00000000
0x01234567
0x000000a1
pass
pass
pass
0x000000020x000000010x000000000x55327a8b0x01234567
fail
Actual valuesfrom RTL
=
=
=
48
Register Checking With Unpredictable Fields
With each reported register update, RAVEN also reports a valid mask
Compare logic needs to examine only those bits whose corresponding valid mask bit is set
The valid mask may change over the course of the test to reflect processor state
CR1 value CR1 mask
Initial State 0x35742278 0xffffffff
Instr. 1 --- ---
Instr. 4 0x35743378 0xfffffcff
Instr. 22 --- ---
Instr. 55 --- ---
Instr. 56 0x25743378 0xfffffcff
Instr. 121 --- ---
Instr. 139 --- ---
Final State 0x25743378 0xfffffcff
49
Using RAVEN Memory Output in an MP/MT Test
Each CPU reports all memory writes
Between zone semaphores, multiple writes to the same address results in non- deterministic memory
The memory final state is reported as bytes with non- deterministic bytes shown as Xs
Writes to Addr X CPU1 CPU2
Initial State 0x01234567
Instr. 1 0x01327a8b ---
Instr. 22 --- 0xabcd7a1f
Instr. 55 0x00000000 0x39327a8f
Instr. 56 --- ---
Instr. 121 0x55abf790 ---
Instr. 139 0x55327a90
Final State 0x55,0xXX (0xab/0x32),0xXX (0xf7/0x7a),
0x90
50
Memory Checking in an MP/MT Test
0x55abf7900x000000000x01327a8b0x01234567
0x55327a900xabcd7a1f0x01234567
0x01327a8b
0x00000000
0x01234567
0xabcd7a1f
Actual valuesfrom RTL
pass
pass
pass
Always compare againstthe last unmatched write
for each CPU
=
=
=
=
0x1234567
0x1327a8b or0xabcd7a1f
0x00000000 or0xabcd7a1f
0x55abf790 or0xabcd7a1f
0x55abf7900x000000000x01327a8b
0x55327a900xabcd7a1f
0x55abf7900x00000000
0x55327a900xabcd7a1f
0x55abf790 0x55327a900xabcd7a1f
CPU1 CPU2
passfail
51
Using Zone Semaphores
Zone semaphores divide instruction execution into known time fragments
• A shared memory location is used as a counter
• No CPU that shares the semaphore can advance to the next zone before the counter reaches zero
• Execution of all instructions within the zone must complete before advancing to the next zone
This may require synchronizing instructionsDoes not imply cache flush
Not all CPUs need to use all semaphores
52
Constraining non-determinism using SemaphoresCPU1 CPU2 CPU3 CPU4
0xaabbccdd 0xaabbccdd 0xaabbccdd 0xaabbccdd
0x9867a430 0x20080325 0x03195725 0x0f152b35
0xa9fc20d4 0xa9441901 0x2298afc7 0xffffff34
0x00001003 0x70073447 0x7e9a98e0 0x99a9c0d8
0x109000ff 0x70073448 0xacde3f54 0xa5aa0977
0xdcb9444a 0x70073449 0x00000000 0x00000000
0x8987564a 0x7007344a 0x7007344c 0x00000001
0xf0cd7aad 0x7007344b 0x10001000
0x32457699 0x7007344c 0xefffefff
0x75913052 0x7007344d 0x7091304d
= initial state= zone 1
= zone 2= zone 3
= zone 4
CPU1 implements all 4 semaphoresCPU2 implements all 4 semaphoresCPU3 implements 2 &4 onlyCPU4 implements 1, 2 & 4 only
CPU3 does no writes to this addressin zone 4
All 4 CPUS read and write the same semaphore here
53
Constraining Non-determinism using semaphores
CPU1 CPU2 CPU3 CPU40xaabbccdd 0xaabbccdd 0xaabbccdd 0xaabbccdd
0x9867a430 0x20080325 0x03195725 0x0f152b35
0xa9fc20d4 0xa9441901 0x2298afc7 0xffffff34
0x00001003 0x70073447 0x7e9a98e0 0x99a9c0d8
0x109000ff 0x70073448 0xacde3f54 0xa5aa0977
0xdcb9444a 0x70073449 0x00000000 0x00000000
0x8987564a 0x7007344a 0x7007344c 0x00000001
0xf0cd7aad 0x7007344b 0x10001000
0x32457699 0x7007344c 0xefffefff
0x75913052 0x7007344d 0x7091304c
At the end of zone 1:
9 possible values
At the end of zone 2:
0x8987564a or0x7007344b or0x7007344c or0x00000000
At the end of zone 3:6 possible values
At the end of zone 4:0x75913052 or0x7007344d or0x7091304d
Final memory reported as:0x75 or 0x700x91 or 0x070x30 or 0x340x52 or 0x4d or 0x4c