Upload
kylee-shaw
View
60
Download
1
Embed Size (px)
DESCRIPTION
RAS. ISP. MicroTESK : Automation of Test Program Generation for Microprocessors. Alexander Kamkin [email protected]. Institute for System Programming of the Russian Academy of Sciences (ISPRAS) http://hardware.ispras.ru. RAS. ISP. What does “MicroTESK” stand for?. - PowerPoint PPT Presentation
Citation preview
MicroTESK: Automation ofTest Program Generation for Microprocessors
Alexander [email protected]
Institute for System Programming of the Russian Academy of Sciences (ISPRAS)http://hardware.ispras.ru
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 2 of 23
What does “MicroTESK” stand for?
Micro = Microprocessor Microprocessor = programmable device
Programmable = program-controlled device
TESK = TEsting and Specification ToolKit Testing = simulation-based verification Specification = specification-driven approach Toolkit = computer-aided automation
MicroTESK is a specification-driven (model-based) approach that automates simulation-based verification of programmable devices by means of test programs
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 3 of 23
Levels of microprocessor verification
lui s1, 0xdead
ori s1, s1, 0x0
lui s3, 0xbeef
ori s3, s3, 0xf
add v0, a0, a2
sub t1, t3, t5
add t7, s1, s3
Core-level verificationUnit-level verification
Model of a microprocessor is tested as a whole
with the help of test programs
Model of a particular unit is tested
via inputs and outputs signals
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 4 of 23
Test programs-based verification
Target Design(RTL, FPGA)
lui s1, 0xdeadori s1, s1, 0x0lui s3, 0xbeefori s3, s3, 0xfadd v0, a0, a2sub t1, t3, t5add t7, s1, s3
Test programs(assembler, C)
Reference simulator (C/C++, ADL)
Execution traces(formatted text)
0x2000: lui ...0x2004: ori ...0x2008: ori ...0x200c: lui ...0x2010: add ...0x2014: sub ...0x2018: add ...
Trace comparator (Perl, C/C++)
0x2000: lui ...0x2004: ori ...0x2008: ori ...0x200c: lui ...0x2010: add ...0x2014: sub ...0x2018: add ...
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 5 of 23
Approaches to test program construction
IF ID EX WB
IF ID EX WB
Requirements
Design
Unformalized knowledge
Formalized knowledge
Test programsManual Development
Random Generation
Template-Based Generation
Model-Based Generation
?
lui s1, 0x2779ori s1, s1, 0x0lui s3, 0x4eeori s3, s3, 0xfadd v0, a0, a2sub t1, t3, t5add t7, s1, s3
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 6 of 23
Advantages of model-based testing
Formalization helps to find problems in a design Deeper comprehension of requirements Formal checking of model consistency
Models automate verification of a design Correctness estimation Test sequence generation Testing adequacy criterion
Models simplify tests maintenance One model, many tests Reuse of models and testing knowledge
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 7 of 23
Approaches to design modeling
ADD x, y, z @ z ≥ 0LD u, x, z @ Cache_HitDIV y, u, x @ Division_By_Zero...DIV.S f, g, h @ Hard_To_RoundADD.S i, f, h @ InexactMUL.S g, i, f @ Invalid
Instruction-level modelsCycle-accurate models
Test templates are used for verificationTo generate tests, FSM models are used
ABS.fmt (fd:FPR, fs:FPR) {...}
ADD (rd:GPR, rs:GPR, rt:GPR) {...}
ADD.fmt (fd:FPR, fs:FPR, ft:FPR) {...}
...XOR (rd:GPR, rs:GPR, rt:GPR) {...}
XORI (rd:GPR, rs:GPR, rt:IMM) {...}
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 8 of 23
Combinatorial test program generation
Tests are constructed by combining Sequences of instructions Test situations for instructions Dependencies between instructions Execution traces, if branches are used
Volume of tests is reduced by heuristics Equivalence of instructions Length of sequences Depth of dependencies Length of execution traces
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 9 of 23
Structure of test programs
initialization
test case
…
test action
…
load a, (b)add c, a, dload e, (f)
move c, 1
store t, (b)move t, 0move b, 0x4
move f, 0x4
store t, (f)move t, 2
move g, 0xd
...bne c, g, error test oracle
address dependencyregister dependency
test situation
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 10 of 23
Instruction specification
Instruction interface Instruction operands Instruction precondition
Execution function Calculates outputs Updates a model state
Assembler format Textual representation
temp (GPR[rs]31||GPR[rs]31..0) + (GPR[rt]31||GPR[rt]31..0)if temp32 temp31 then SignalException(IntegerOverflow)else GPR[rd] sign_extend(temp31..0)endif
if NotWordValue(GPR[rs]) or NotWordValue(GPR[rt]) then UNPREDICTABLEendif
output rd: GPR, Wordinput rs: GPR, Wordinput rt: GPR, Word
ADD rd, rs, rt
instruction ADD(rd, rs, rt)
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 11 of 23
Test situations and dependencies
Parameters Specifies family of test situations / dependencies
Constructor / constraint Constructs / constrains values of the operands
Preparator Creates initialization program of the instruction
if NotWordValue(GPR[rs]) or NotWordValue(GPR[rt]) then UNPREDICTABLEendiftemp (GPR[rs]31||GPR[rs]31..0) + (GPR[rt]31||GPR[rt]31..0)if temp32 temp31 then SignalException(IntegerOverflow)else GPR[rd] sign_extend(temp31..0)endif
0x0af1036f0x1
0xf47298b90x3
0xffff0a310x00xdeadbeef0xd
datatag
0x19d02c340xa
0x7420ba9c0x2......
Cache (L1/L2) TLB
0xa0000x0000 0xb0200x0001 ...... 0xa0100x003e 0xb0000x003f
PFNVPN
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 12 of 23
load ?, (?)
add ?, ?, ?load ?, (?)
Test program generation scheme
load a, (b)
add c, a, dload e, (f)
cache miss
cache hit
normal
move c, 1
store t, (b)move t, 0
move b, 0x4
move f, 0x4
store t, (b)move t, 2
Test programadd
...
...
...
sub
storeload
cache hitcache miss
normaloverflow
register dependencyaddress dependency
address dependencyregister dependency
generation parameters
lui s1, 0xdeadori s1, s1, 0x0lui s3, 0xbeefori s3, s3, 0xfadd v0, a0, a2sub t1, t3, t5add t7, s1, s3
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 13 of 23
MicroTESK tool support
The approach is supported by the MicroTESK test program generator
Test programs are generated according to the specified goals
Technology and tool provide high level of automation
Test programs might include checks of microprocessor state
Specifications are developed in Java language
There are ready-to-use test data generators
Generator has graphical user interface
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 14 of 23
Generator development process
// Test program N // Dependencies: div[0]-add[1] div.s $1, $5, $3 // Underflow add.d $2, $1, $8 // Overflow mul.s $7, $4, $9 // Inexact
// Test program 1 // Dependencies: sw[0]-lw[1] sw v0, (t0) // L1 Miss lw v1, (t1) // L1 Hit dw v2, (t2) // TLB Miss
// Test program 0 // Dependencies: add[0]-sub[1] add v0, a0, a2 // Overflow sub t1, v0, t5 // Normal div t7, s1 // Div by Zero
...
Set of Test Programs:Delivered to Microprocessor Designers
Architecture Modeler(s) /Test Developer(s)
MicroTESKTest Program
Generator
Documentation(Instruction Set Manual,
Standards, Implementation Features, etc.)
Verification Analyst(s)
Verification Engineer(s)
Verification Plan (Generalized Goals and Tasks of Test Program
Generation)
Documentation: Provided byMicroprocessor Designers
Formal Specification of MicroprocessorInstruction Set
Instruction-Level Test Coverage (Test Situations
and Dependencies)
Configuration / Parameters for MicroTESK Test Program Generator
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 15 of 23
How a MicroTESK generator looks like
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 16 of 23
Case studies
Memory subsystem MIPS64 microprocessor DSP co-processor Floating-point co-processors DMA controller Branch prediction unit
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 17 of 23
Some statistics
Characteristic MMU MIPS64 DSP CP1 CP2
Number of instructions 4 221 109 114 75
Efforts, man-months 2 9 4 1 1
Volume of specifications and tests, lines of code 5560 47190 20610 6450 11550
Number of found bugs in a reference simulator 0 6 > 10 >10 >10
Number of found bugs in an RTL model 1 9 > 10 >10 >10
Many critical bugs were found in the designs, which had been verified by hand-written, random-generated, and template-based test programs
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 18 of 23
Future directions
Automatic extraction of test coverage
Advanced test template generation
Integration with ADL (ISE, nML, LISA, etc.)
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 19 of 23
Extraction of test situations
L1
TLB
Hit, Valid Hit, Invalid
Exception: TLBInvalid
Hit Miss
Exception: TLBRefill
Miss
Normal1 Normal2
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 20 of 23
Extraction of dependencies
associative buffer TLB<TLB_ENTRY, 64> { VIRTUAL_PAGE_NUMBER tag( VIRTUAL_ADDRESS va) { ... } ... INTEGER index(VIRTUAL_ADDRESS va) { ... } ...}
structure TLB_ENTRY { VIRTUAL_PAGE_NUMBER vpn; PHYSICAL_PAGE_NUMBER pfn; ...}
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 21 of 23
MicroTESK project
Download the MicroTESK demohttp://hardware.ispras.ru
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 22 of 23
Contacts
Institute for System Programming of RAS (ISPRAS)http://www.ispras.ru
Hardware Verification R&D @ ISPRAShttp://hardware.ispras.ru
Alexander [email protected]
IEEE East-West Design and Test Symposium - September 18-21, 2009 - Moscow, Russia 23 of 23
Thank You!Questions?