Upload
rickey-ashford
View
244
Download
6
Tags:
Embed Size (px)
Citation preview
Making a Making a Simple, Structured and Efficient TestbenchSimple, Structured and Efficient Testbench
Step-by-stepStep-by-step
www.bitvis.noEspen Tallaksen
Making a simple, structured and efficient Testbench, Step-by-step
1
About BitvisAbout Bitvis
Leading Vendor Independent Design Centre in NorwayLeading Vendor Independent Design Centre in Norway FPGA and Embedded SW services for customersFPGA and Embedded SW services for customers From specification to final product – or any phase in betweenFrom specification to final product – or any phase in between Good overview of pitfalls, time wasters and risksGood overview of pitfalls, time wasters and risks Focus on methodology, quality, efficiency and customersFocus on methodology, quality, efficiency and customers Located in Asker outside OsloLocated in Asker outside Oslo
Making a simple, structured and efficient Testbench, Step-by-step
2
Products and courses from BitvisProducts and courses from Bitvis
Products from BitvisProducts from Bitvis 'Bitvis Utility Library' (Free and Open source, Directly downloadable)'Bitvis Utility Library' (Free and Open source, Directly downloadable)
- Currently being used world wide- Currently being used world wide 'UVVM' (Universal VHDL Verification Methodology) (UVL for VHDL)'UVVM' (Universal VHDL Verification Methodology) (UVL for VHDL)
To be released 2014, Q4To be released 2014, Q4 'RegisterWizard', 'RegisterWizard',
For generation of SW, Doc. and VHDL (bus IF, regs, etc.)For generation of SW, Doc. and VHDL (bus IF, regs, etc.)To be released 2014, Q4To be released 2014, Q4
Courses from BitvisCourses from Bitvis 'FPGA Development Best Practices' - A two day course'FPGA Development Best Practices' - A two day course
- A pragmatic approach to improving quality and efficiency.- A pragmatic approach to improving quality and efficiency.- So far Denmark, Sweden and Norway. May be held anywhere on request- So far Denmark, Sweden and Norway. May be held anywhere on request
See our website for more offersSee our website for more offershttp://bitvis.no/services/fpga-courses/
Making a simple, structured and efficient Testbench, Step-by-step
3
Making a simple, structured and efficient Testbench, Step-by-step
4
Why Testbenches and Simulation
Far more control and observability Far faster iterations May have a structured bottom- up
verification. Detect bugs that cannot or most
probably will not be detected in a lab-test
Most bugs can be found with short simulations. Cost of
corrections
Design stage
Spec.
Volum/Field
Ease of correction &
debugging
Verification: Verification: State of the FPGA communityState of the FPGA community
Even for quite simple modules Even for quite simple modules (e.g. IRQC, SPI, I2C, UART):(e.g. IRQC, SPI, I2C, UART):
Average verification workload: 3 days to 3 weeksAverage verification workload: 3 days to 3 weeks Often inadequate coverageOften inadequate coverage Multiple Design, Synthesis, P&R iterations on full FPGAMultiple Design, Synthesis, P&R iterations on full FPGA A low quality testbench A low quality testbench
Hopeless to understand for anyone else Hopeless to understand for anyone else Difficult to extendDifficult to extend Terrible to modifyTerrible to modify Tedious debuggingTedious debugging
Making a simple, structured and efficient Testbench, Step-by-step
5
Why?How can we improve?
Making a simple, structured and efficient Testbench, Step-by-step
6
The Verification Paradox
For proper simulation of FPGAs: More lines of verification code than for design source More time consuming than design
Yet - a far lower effort is spent on verification Lower effort on partitioning Lower effort on structuring Lower effort on documentation
A good TB is key to success:
- Simple 500 MH project: Saves 50-100 MH - Complex 5000 MH project: Saves 500-1500 MH- Significantly improves TTM, LCC, Quality
Making a simple, structured and efficient Testbench, Step-by-step
7
Different TB requirements (1)
The required level of verification differs a lot Low verification complexity
» single data path FPGA with some additional functionality» modules with no or very simple corner casesE.g. GPIO, simple IRQC, CRC, FEC, decoder...
Higher verification complexity: Still needs basic infrastructure – as a platform Need more advanced verification solutions on top
Low verification complexity Simple Testbench But – how simple?
Making a simple, structured and efficient Testbench, Step-by-step
8
How simple should a TB be?
Understanding somebody’s design module is often very hard Understanding somebody’s testbench is normally impossible
And modifying it… ?
Testbench Issues: Purpose: To verify DUT requirements Focus: Sufficient functional coverage with a minimum effort Test Sequencer Requirements:
Simple to write
Simple to understand and modify - by anyone
Simple to execute, debug and understand reports & results
Making a simple, structured and efficient Testbench, Step-by-step
9
The development stages
Ideally: - Specify Design Simulate Synthesize Test- Verification should start as early as possibleVerification should start as early as possible
But: Specification changes all the time… Design must be extended or modified Testbench or test cases must be extended or modified Simulations must be re-executed and the result must be checked again
TB Readability, Flexibility and Extendibility is important
Simulation result checking and readability is important
Making a simple, structured and efficient Testbench, Step-by-step
10
Verification stages
1. Make a verification specification – but avoid overhead and aim for single source
2. Define your TB architecture and concept- but consider what is already available
3. Implement TB architectureIf available – start TB implementation from a template
4. Implement test cases- In sensible steps and starting ASAP
But also focus on:
- Simulation execution, Reporting, Debugging, Results
- Iterations
- Will go through, stage-by-stage
Making a simple, structured and efficient Testbench, Step-by-step
11
The IRQC
irq_source(n)
IRQC
/n
clk
irq2cpu_ack
bus interface
arst
irq2cpu
irq_source(0)S Q
R
IER(0)
IRR(0)
ITR(0)
ICR(0)
D Q
IPR(0)
IPR(1..n)
(Handling of one single irq_source.To be repeated for all sources.)
irq2cpu
irq2cpu_ack (from CPU)
(to CPU)
igr
D Q
R
IRQ2CPU_ENA IRQ2CPU_ALLOWEDNote: Uppercase names indicate software accessible registers (or dummy registers)
irq_source(n)
IRQC
/n
clk
irq2cpu_ack
bus interface
arst
irq2cpu
Businterface
Making a simple, structured and efficient Testbench, Step-by-step
12
1: Make a verification specification
A verification specification is always needed, but The verification spec. should not be too extensive/detailed Required details could be added in a later spec. iteration
A separate verification spec. document is normally not needed
irq_source(n)S Q
R
IER(0)
IRR(n)
ITR(n)
ICR(n)D Q
IPR(n)
irq2cpu
irq2cpu_ack
igr
D Q
R
IRQ2CPU_ENAIRQ2CPU_ALLOWED
Verification specification IRQC:
• Check defaults on output ports• Check register defaults and access (write + read)• Check register trigger/clear mechanism• Check interrupt sources, IER, IPR and irq2cpu• Check autonomy for all interrupts• Check irq acknowledge and re-enable
Making a simple, structured and efficient Testbench, Step-by-step
13
2: Define your TB architecture
Two main categories for almost all modules- Defined by degree of interaction on interfaces 1. Basic TB infrastructure only 2. TB for simultaneous handling of multiple interfaces
IRQC has very simple interfaces Very little and no critical interaction No contention issues Still – A bad design may always fail for corner cases
Hence IRQC TB Simple Verification Simple TB arch./concept Basic TB infrastructure
Making a simple, structured and efficient Testbench, Step-by-step
14
Simple TB architecture
Avoid all complex issues No support for handling simultaneous interfaces No queuing of commands Use a single, simple, understandable test sequencer
Required support Logging Alert handling & reports Continuous actions
» Actions not handled by sequencer» Min. interaction with sequencer
Repeated actions» Avoid unstructured copying» Provide more info for multi-usage
TB_IRQC
DUT(IRQC)
Testsequencer
Supportprocesses
Supportprocedures & functions
Making a simple, structured and efficient Testbench, Step-by-step
15
3: Implement TB architecture
Assuming no template is available…
Generate TB entity with DUT instantiated Push-button in several tools (MsD, Emacs, etc.)
Add support process for clock generation Allow enable/disable from seq.
Add test sequencer process Main process in TB Controls everything!
» From TB initialization » To termination of simulation
TB_IRQC
DUT(IRQC)
Testsequencer
Supportprocesses
Supportprocedures & functions
*** Inside TB architecture: ------------------------------------------------ -- Main process and test sequencer ------------------------------------------------ p_main: process begin log(ID_LOG_HDR, "Start simulation TB_IRQC"); clock_ena <= true; -- start clock generator
*** *** Actual test sequence. To be filled in ***
--========================================================== log(ID_LOG_HDR, "SIMULATION COMPLETED"); clock_ena <= false; -- to gracefully stop the simulation assert false report "End of simulation. (***Ignore this provoked failure.)" severity failure; wait; -- to stop completely end process p_main;
Ready to implement the first testsTB workload so far: 10-30 min.
Making a simple, structured and efficient Testbench, Step-by-step
16
4a: Implement First tests
First tests are important To verify compilation and elaboration OK To see that our Testbench is up and running To actually verify our first tests of the DUT
First test: Output defaults Very simple test Allows module to be integrated First example on active support
» Using support procedures
TB_IRQC
DUT(IRQC)
Testsequencer
Supportprocesses
Supportprocedures & functions
set_inputs_passive(VOID); apply_reset(VOID);
log(ID_LOG_HDR, "Check defaults on output ports"); --------------------------------------------------------------- check_value(irq2cpu, '0', ERROR, "irq2cpu must be default inactive"); check_value(dout, x"00", ERROR, “dout must be default inactive");
The above code will result in the following log/transcript:
BV: 0 ns irqc_tb All inputs set passiveBV: 60 ns irqc_tb Pulsed reset-signal - active for 10TBV:BV: 60 ns irqc_tb Check defaults on output portsBV:----------------------------------------------------------------------BV: 60 ns irqc_tb check_value(sl 0)=> OK. irq2cpu must be default inactiveBV: 60 ns irqc_tb check_value(slv x00)=> OK. dout must be default inactive
Progress information
Detailed progress information. Only interesting initially and for debug…
Inside p_main process test sequencer process)
Making a simple, structured and efficient Testbench, Step-by-step
17
Verbosity controlVerbosity control
Method to control amount of informationMethod to control amount of information To allow only selected groups of messagesTo allow only selected groups of messages Allows reduction/increase of information – without Allows reduction/increase of information – without
modifying the code (comment/uncomment)modifying the code (comment/uncomment) Makes it possible to get a different set of messages Makes it possible to get a different set of messages
depending on current simulation focus - e.g.:depending on current simulation focus - e.g.:» Debugging testcase or verification componentDebugging testcase or verification component» Debugging a specific interface on DUTDebugging a specific interface on DUT» Debugging a data flow through DUTDebugging a data flow through DUT» General simulation progress report.General simulation progress report.
E.g. Modelsim vsim options:E.g. Modelsim vsim options:-version -version Print the version of the simulatorPrint the version of the simulator-quiet -quiet Do not report 'Loading...' messagesDo not report 'Loading...' messages+sdf_verbose +sdf_verbose Display SDF annotator status messagesDisplay SDF annotator status messages
verbosity switches turning on or off verbosity switches turning on or off a given set of messages a given set of messages
Priority based verbosity controlPriority based verbosity control
Every message has a given priority.Every message has a given priority. Typically 1 for a high priority message and 6 for a low priority Typically 1 for a high priority message and 6 for a low priority
message.message. Every possible message from any part of the testbench has a Every possible message from any part of the testbench has a
given priority - normally defined when writing the message.given priority - normally defined when writing the message. E.g. log(2, "Packet header received”) E.g. log(2, "Packet header received”)
Verbosity level will determine which messages to showVerbosity level will determine which messages to show E.g. a verbosity level of 2 means E.g. a verbosity level of 2 means
- Only priority 1 and priority 2 messages are let through - Only priority 1 and priority 2 messages are let through (i.e. shown in the transcript or log); (i.e. shown in the transcript or log); - whereas all other messages are blocked.- whereas all other messages are blocked.
Messages could be defined anywhereMessages could be defined anywhere In Sequencer, BFMs, verification components, other processes, In Sequencer, BFMs, verification components, other processes,
etc..etc..
SystemVerilog’s verbosity control is basically priority basedSystemVerilog’s verbosity control is basically priority based
Making a simple, structured and efficient Testbench, Step-by-step
18
Making a simple, structured and efficient Testbench, Step-by-step
19
ID based verbosity controlID based verbosity control
ID based verbosity allows log messages with different ID based verbosity allows log messages with different IDs to be shown or blocked depending on whether a IDs to be shown or blocked depending on whether a given ID is enabled or disabled.given ID is enabled or disabled.
Every message has a given IDEvery message has a given ID Could be a string, a number, an enumerated, etc….Could be a string, a number, an enumerated, etc…. There are no prioritiesThere are no priorities
E.g. ID=58 and ID=216 are not prioritised in any wayE.g. ID=58 and ID=216 are not prioritised in any way Example: Example: log(216, "Packet header received”)log(216, "Packet header received”)
Verbosity control will determine which messages to Verbosity control will determine which messages to showshow Test sequencer may enable or disable message IDs dynamically Test sequencer may enable or disable message IDs dynamically
depending on which messages are of interest. depending on which messages are of interest.
SystemC’s verbosity control is ID basedSystemC’s verbosity control is ID based
Verbosity Control – Using EnumeratedVerbosity Control – Using Enumerated
ID based verbosity control.ID based verbosity control.
Enumerated IDsEnumerated IDs A set of predefined IDs. E.g. ID_LOG_HDR or ID_BFMA set of predefined IDs. E.g. ID_LOG_HDR or ID_BFM Example: Example: log(ID_BFM, "UART byte received")log(ID_BFM, "UART byte received")
Verbosity controlVerbosity control Enable an ID using Enable an ID using enable_log_msg(ID_BFM)enable_log_msg(ID_BFM) Disable an ID using Disable an ID using disable_log_msg(ID_BFM)disable_log_msg(ID_BFM) Only enable or disable from the test sequencer!Only enable or disable from the test sequencer!
20Making a simple, structured and efficient Testbench, Step-by-step
Making a simple, structured and efficient Testbench, Step-by-step
21
Why ID based verbosityWhy ID based verbosity
Far more flexible and controllableFar more flexible and controllable
Real priorities are dependent on the situationReal priorities are dependent on the situation applicationapplication development stagedevelopment stage problem at handproblem at hand
ID’s allows full control of verbosityID’s allows full control of verbosity
May still define numeric priorities locallyMay still define numeric priorities locally E.g. to indicate your assumed prioritiesE.g. to indicate your assumed priorities May then choose to use numeric priority or IDMay then choose to use numeric priority or ID ID-based verbosity is a superset of priority-based.ID-based verbosity is a superset of priority-based.
Making a simple, structured and efficient Testbench, Step-by-step
22
The need for verbosity controlThe need for verbosity control
Verbosity control is not required for the simplest TBs, but...Verbosity control is not required for the simplest TBs, but...
Provides an advantage - even for most Provides an advantage - even for most reallyreally simple TBs simple TBs
More efficient to use the same methodology for simple, More efficient to use the same methodology for simple, medium and advanced TBsmedium and advanced TBs
Most TBs normally end up more advanced than assumedMost TBs normally end up more advanced than assumed
Yields no overhead in even the simplest test sequencerYields no overhead in even the simplest test sequencer
Now – back to simple TB
Making a simple, structured and efficient Testbench, Step-by-step
23
Back to TB: Do not waste time
Most designers are unstructured wrt. verification Do not wait with your testbench and verification Do not dive into your first test case with no structure Do not use stupid testbench generators Do not write stimuli sequences by pure assignments Do not check results by checking outputs in the wave view
Identify the need for subprograms – ASAP Immediately write subprogram when detecting repeated code
Write a “self checking” testbench Requires good check subprograms
Good Comments/documentation is NEVER wasted Provide in sequencer and in log/transcript
Making a simple, structured and efficient Testbench, Step-by-step
24
Identify the need for subprograms
Obvious subprogram candidates for IRQC Register Access Signal checkers Interrupt source pulsing? Interrupt acknowledge pulsing? (Report/log method) (Alert-handling) (reset, set_passive, …)
Split in two categories Local – IRQC-dedicated
» Declare locally Common – non dedicated
» Declare in common package» Share with others
irq_source(n)S Q
R
IER(0)
IRR(n)
ITR(n)
ICR(n)D Q
IPR(n)
irq2cpu
irq2cpu_ack
igr
D Q
R
IRQ2CPU_ENAIRQ2CPU_ALLOWED
*** Inside p_main process (test sequencer process)
log_hdr(“Reset"); --------------------------------------------------------------- set_inputs_passive(VOID); apply_reset(VOID);
log_hdr("Check defaults on output ports"); --------------------------------------------------------------- check_value(irq2cpu, '0', ERROR, "irq2cpu must be default inactive"); check_value(pif_irqc_dout, X"00", ERROR, “pif_irqc_dout must be default inactive");
Local procedure – dedicated to inputs
Common procedures.
Logging and checking is required in all testbenches.
Evaluate whether this should be a common, slightly more general procedure…
Making a simple, structured and efficient Testbench, Step-by-step
25
4b: Next test: Register Access
First check register defaults
Then general access (Write + Read) Provide Check-procedure – using Read
Make dedicated procedures Will be common for many register interfaces
Bus access procedures are used to set up signal sequences to access internal registers
BFM : Bus Functional Model
Making a simple, structured and efficient Testbench, Step-by-step
26
BFM / TLM Purpose
Handle transactions at a high level E.g. Read, Write, Send packet, Config, etc More understandable for anyone Simpler code & Improved overview Uniform style, method, sequence, result Easy to add several very useful features
Example: BFM for a CPU access to a module's register
E.g. write 0xF0 (“11110000”) into a register at address 0x22 (“100010”)
Replaced by
write(x”22”, x”F0”);
cs <= ’1’;we <= ’1’;addr <= ”00100010”;data <= ”11110000”;wait until rising_edge(clk);wait until falling_edge(clk);cs <= ’0’;we <= ’0’;
Making a simple, structured and efficient Testbench, Step-by-step
27
Major BFM quality differences
Inside a ”normal” BFM» Pure Read, Write or Check transaction
Additionally - Inside some BFMs» Syncronization of access to the relevant clock
Additionally - Inside good BFMs» Normalisation of inputs » Sanity-check on inputs» Configuration of behaviour» Logging of all accesses – with parameters and result» Severity control and alert handling» Verbosity control to potentially suppress log
Making a simple, structured and efficient Testbench, Step-by-step
28
Making common, general BFMs
Value and Signal widths are initially unknown Hence use unconstrained input vector for addr & data But unconstrained vectors do not define direction/range
e.g constant data_exp : in std_logic_vector;
Unconstrained data must often be “interpreted” Comparisons, Bit-manipulation, etc…
Must Normalise
Must Check for sanity – for early problem detection
function normalise( *** parameters (incl. implicitly constrained vectors) ) is *** Constants and Variablesbegin *** checks for legal constrained ranges *** check if (restriction = ALLOW_WIDER_SOURCE) then check_value(***Complex check and reporting***); *** corresponding checks for ALLOW_SHORTER_SOURCE, etc. *** Adapt value to target signal range and returnend;
Making a simple, structured and efficient Testbench, Step-by-step
29
Make and run first BFM - Summary
As soon as register interface is implemented: Make BFMs for write, read, check – in a common package
Then make simplified BFMs First check defaults, and then write followed by check
procedure write ( constant addr : in unsigned; constant data_in : in std_logic_vector; constant msg : in string := "" ) is begin sbi_write(addr, data_in, pif_irqc_cs, pif_addr, pif_wr, pif_rd, pif_din, pif_rdy, clk, C_T_CLK, msg , C_SCOPE, ***more***); end;
write(x"1", x"0F", "IER"); check(x"1", x"0F", error, "Pure readback");
BV: 172 ns. irqc_tb SBI write(x1, x00000F) completed. IERBV: 192 ns. irqc_tb SBI check(x1, ==> x00000F) completed. BV: Pure readback
BV: 172 ns. irqc_tb SBI write(x1, x00000F) completed. IERBV:BV:==============================================================BV: ERROR:BV: 192 ns. irqc_tb BV: value was: 'x00000E'. expected 'x00000F'.BV: (From SBI check(x1, x0F, Pure readback))BV:==============================================================
ifcheck ok
ifchecknot ok
procedure sbi_write ( constant addr : in unsigned; constant data_in : in std_logic_vector; signal cs : inout std_logic; signal addr : inout unsigned; signal wr : inout std_logic; signal rd : inout std_logic; signal din : inout std_logic_vector; signal rdy : in std_logic; signal clk : in std_logic; constant clk_period : in time; constant msg : in string; constant scope : in string; *** more ***) is begin normalise, sanity check, adapt, synch, write, wait?, alert?, log? end;
Only if allowed by verbosity control
Will break simulation if stop limit is reachedNot shown (or counted) if set to ignore Error
Making a simple, structured and efficient Testbench, Step-by-step
30
TB: Workload so far?
Note: Assuming a TB infrastructure for log/alert/checksOtherwise only slightly more, but far less log/report/debug)
Initial architecture 10-30 min
First tests: Incl. simple local procedures 10-30 min
Simple BFM set for a known protocol - More for protocol-oriented BFMs
1-4 h
Overhead for Good BFM (Normalise, Log, Checks, etc.)- Given a good BFM template
30-60 min
Reg. access tests and reg. interaction 10-30 min
Total so far: 2-7 h
4c: Continuing with TB for IRQC
Checking register interactions Just a series of Write and Check procedures
Checking signal stability E.g. interrupt to CPU stable on or off
Waiting for an event- inside a “window” E.g. interrupt to occur wrt. activated source
log(ID_LOG_HDR, "Check register trigger/clear mechanism", C_SCOPE);
------------------------------------------------------------
write(C_ADDR_ITR, x"AA", "ITR : Set interrupts");
check(C_ADDR_IRR, x"AA", ERROR, "IRR");
write(C_ADDR_ITR, x"55", "ITR : Set more interrupts");
check(C_ADDR_IRR, x"FF", ERROR, "IRR");
write(C_ADDR_ICR, x"71", "ICR : Clear interrupts");
check(C_ADDR_IRR, x"8E", ERROR, "IRR");
write(C_ADDR_ICR, x"85", "ICR : Clear interrupts");
check(C_ADDR_IRR, x"0A", ERROR, "IRR");
write(C_ADDR_ITR, x"55", "ITR : Set more interrupts");
check(C_ADDR_IRR, x"5F", ERROR, "IRR");
write(C_ADDR_ICR, x"5F", "ICR : Clear interrupts");
check(C_ADDR_IRR, x"00", ERROR, "IRR");
***Etc…
*** time_tmp is set at a time from which the interrupt should be stable
check_stable(irq2cpu, (now - time_tmp), error, "No previous active irq2cpu");
await_value(irq2cpu, '1', 0 ns, C_T_CLK, ERROR, "IRQ2CPU expected by now");
31Making a simple, structured and efficient Testbench, Step-by-step
Making local procedures
Local procedures should be made for any repeated actions for which a common (central) procedure does not make sense E.g. pulsing the interrupt acknowledge: procedure pulse_irq2cpu_ack(
constant dummy : in t_void
) is
variable initial_value : std_logic := irq2cpu_ack;
begin
check_value(std_match(irq2cpu_ack, '0'), tb_warning, “Test seq.”,
"irq2cpu_ack='1' when pulse_irq2cpu_ack() called");
wait until falling_edge(clk);
irq2cpu_ack <= '1';
wait until rising_edge(clk);
wait for (C_T_CLK / 4);
irq2cpu_ack <= '0';
log(ID_SEQUENCER_SUB, “Test seq.”, "Pulsed irq2cpu_ack from 0 to 1 to 0");
end;
In addition to the actual signal toggling – several tasks should be added.
Check that ack is not already active
Synchronise pulse on and off
Report the pulse to the log
Dedicated msg ID
Then evaluate whether this should be a common, slightly more general procedure…
32Making a simple, structured and efficient Testbench, Step-by-step
Making a simple, structured and efficient Testbench, Step-by-step
33
4d: Completing the Test case
Add missing sections acc. to verif. spec.
Fill inn all sections Section headers Log headers Details inside sections Std. log messages
End simulation by reporting all alerts
Making a simple, structured and efficient Testbench, Step-by-step
34
Report summaries
report_alert_counters() Reports all alert counters
=========================================================================== BV: *** SUMMARY OF ALL ALERTS *** BV: ====================================================================== BV: REGARDED EXPECTED IGNORED Comment? BV: NOTE : 0 0 0 ok BV: TB_NOTE : 0 0 0 ok BV: WARNING : 0 0 0 ok BV: TB_WARNING : 0 0 0 ok BV: MANUAL_CHECK : 0 0 0 ok BV: ERROR : 0 0 0 ok BV: TB_ERROR : 0 0 0 ok BV: FAILURE : 0 0 0 ok BV: TB_FAILURE : 0 0 0 ok BV: ====================================================================== BV: >> No mismatch between counted and expected serious alerts BV: ======================================================================
Verbosity control – revisitedVerbosity control – revisited
Log-result for TB for IRQCLog-result for TB for IRQC Only ID_LOG_HDR enabled: Only ID_LOG_HDR enabled:
» 32 lines32 lines» Log headers onlyLog headers only
All IDs enabled:All IDs enabled:» 500 lines500 lines» All detailsAll details
IRQCIRQC is an extremely simple module… is an extremely simple module…
Bitvis: 0.0 ns TB seq. Start Simulation of TB for IRQCBitvis: --------------------------------------------------------------------------Bitvis: Bitvis: 110.0 ns TB seq. Check defaults on output portsBitvis: --------------------------------------------------------------------------Bitvis: Bitvis: 110.0 ns TB seq. Check register defaults and access (write + read)Bitvis: --------------------------------------------------------------------------Bitvis: Bitvis: 217.5 ns TB seq. Check register trigger/clear mechanismBitvis: --------------------------------------------------------------------------Bitvis: Bitvis: 337.5 ns TB seq. Check interrupt sources, IER, IPR and irq2cpuBitvis: --------------------------------------------------------------------------Bitvis: Bitvis: 637.5 ns TB seq. Check autonomy for all interruptsBitvis: --------------------------------------------------------------------------Bitvis: Bitvis: 2167.5 ns TB seq. Check irq acknowledge and re-enableBitvis: --------------------------------------------------------------------------Bitvis: Bitvis: 2367.5 ns TB seq. Check ResetBitvis: --------------------------------------------------------------------------Bitvis: Bitvis: =====================================================================Bitvis: *** SUMMARY OF ALL ALERTS ***etc……
35Making a simple, structured and efficient Testbench, Step-by-step
Making a simple, structured and efficient Testbench, Step-by-step
36
Reduced design time
Early structured TB & test cases improve efficiency Forces you to think about the spec. in a different way Avoids wasting time on “simple/stupid” simulations
» Do NOT make initial “force/check” verification Yields a good TB & Test case structure - right away
» and provides a “shell” for filling in tests as needed Early verification of selected issues is useful
» May add verification tasks as IRQC is being designed» No time wasted – as all verification is useful
Continuous TB update on spec/design-changes Late changes are far simpler to handle
Making a simple, structured and efficient Testbench, Step-by-step
37
Reduced debug time
Far better simulation debug support Logging sequences and every single action
» Possibility to reduce amount of logging Good alert messages with mismatch report
» And preceeding progress report
Far better quality on lab-releases Minimised need for lab debug
Easier to make new test cases to trace lab problems
38
Documentation
Required doc++ Examples from IRQC testbench
Verification specification Check register trigger/clear mechanism 1
Code sectioning for multiple test areas
Check register trigger/clear mechanism 2
Code commenting ITR should set IRR
Simulation status when completed (No alerts reported) 4
Intermediate progress report “Check register trigger/clear mechanism” 3
Alerts “ERROR: ITR should set IRR”
Check-reports “Checked: ITR should set IRR”
Debug support “ERROR: ITR should set IRR; Was ‘00110011’, expected ‘11111111’”
Progress report and debug support “pif_write(x3, x00A5A5) completed. ITR”
Single source for all documentation/commenting May easily extract required documentation
Making a simple, structured and efficient Testbench, Step-by-step
Bitvis Utility LibraryBitvis Utility Library
Open source VHDL library – Released April 2013Open source VHDL library – Released April 2013 Initial version VHDL 2002/08 compliant onlyInitial version VHDL 2002/08 compliant only VHDL '93 version released August 2013VHDL '93 version released August 2013
Important featuresImportant features Logging and Verbosity control Logging and Verbosity control Alert handling and reportingAlert handling and reporting Simple randomisation Simple randomisation Basic testbench checking and await proceduresBasic testbench checking and await procedures
Simplicity is keySimplicity is key Strong focus on allowing real simple infrastructure usageStrong focus on allowing real simple infrastructure usage Project adaptable behaviour and log layoutProject adaptable behaviour and log layout Quick Reference for all methods is providedQuick Reference for all methods is provided
Advanced method versions allow added complexityAdvanced method versions allow added complexity
39Making a simple, structured and efficient Testbench, Step-by-step
Verification effort for a single module/FPGA
Coverage
Time (MH)
100%
Approved
No “wasted” time up front
Straight on coding
Structured approachTypical Project
Structured approachusing an already available infrastructure
Total of 4 hours for complete TB for IRQC...
40Making a simple, structured and efficient Testbench, Step-by-step
Making a simple, structured and efficient Testbench, Step-by-step
41
Conclusions for my TB for IRQC
A uniform and structured methodology More compact and understandable code Single source for spec, code comments and logging Far easier to write, read, modify and extend a test case Easy to execute and understand output/status Debugging is much faster with a good progress report
Total of 4 hours for complete TB for IRQC
Very efficient TB implementation and design debug
Making a simple, structured and efficient Testbench, Step-by-step
42
The end.
Making a Making a Simple, Structured and Simple, Structured and
Efficient TestbenchEfficient TestbenchStep-by-stepStep-by-step