Implications of Code Coverage Verification Techniques for Designs Adhering to DO-254

David Clift

Application Specialist

FirstEDA Limited

2

Abstract

In DO-254, elemental analysis provides metrics on how much of the design was covered through the requirements-based verification of the associated functional elements - the goal is to show that there is no redundant or dead code in the design. Redundant code not only consumes device resources, it can also be a vector for a cyberattack or device operation outside of the design requirements, leading to potential system failure. In this presentation we will look at different code coverage techniques and discuss their potential limitations as well as introducing strategies to improve the verification and quality of your designs.

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

3

Elemental Analysis in DO-254

• Levels A and B airborne hardware are critical to the safety of the aircraft and its passengers

• For both Levels A and B hardware the FAA recommends that designers follow an appropriate design assurance method described in RTCA/DO-254 Appendix B

• Elemental Analysis is one of these and it is the most commonly used method of design assurance for FPGAs/PLDs

• It’s Code Coverage

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

4

TAP State Machine

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

5

Code Coverage Results

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

Question: Do you think these code coverage results are sufficient?

6

What is Happening?

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

TCK <= ‘1’ after 5 ns;

105 NS 110 NS + 0

STATE <= NEXT_STATE; TMS <= ‘1’;

TCK <= ‘0’ after 5 ns;

TCK = ‘1’ STATE = SHIFT_DR TMS = ‘1’

110 NS + 1

NEXT_STATE <= SHIFT_DR; TMS_SIG <= ‘1’;65

110 NS + 2

NEXT_STATE = SHIFT_DR TMS_SIG = ‘1’

NEXT_STATE <= EXIT1_DR;

110 NS + 3

NEXT_STATE = EXIT1_DR

Evaluate

Update

115 NS + 0

TCK = ‘0’

7

Code Coverage – Lessons Learnt

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

• Code coverage records execution

• Code Coverage does not record function

• Code Coverage results are affected by simulation Delta Cycles

• Code Coverage only tells you what you have NOT tested!

8

OSVVM

• The Open Source VHDL Verification Methodology

• Based on VHDL 2008 - can work with VHDL 2002

• Provides advanced capabilities for random value generation and functional coverage

• Allows definition of normal, illegal and ignore bins for regular coverage and cross-coverage

• Equipped with flexible coverage reporting procedures

• Works perfectly with Transaction Level Modeling

• Enables intelligent randomisation based on the functional coverage holes (bins that are not covered)

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

9

Coverage Model – State Register

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

TAP_FSM_CP.AddBins(GenBin(TapStateType'pos(TapStateType'left), TapStateType'pos(TapStateType'right)));

shared variable TAP_FSM_CP : CovPType; -- Object for TAP FSM states coverpoint

library osvvm; use osvvm.CoveragePkg.all; use osvvm.RandomPkg.all;

10

The Cross Coverage Model – State Transitions

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

11

State Transition Coverage Model

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

12

Functional Coverage Results

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

13

Adding Constrained Random Stimulus Generation

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

14

Function Coverage with Constrained Random Stimulus

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

15

Formal Verification

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

16

Formal Verification

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

17

Summary

• Code Coverage provides important metrics, highlighting untested areas of code which are possible areas of concern

• It only shows what you haven’t tested

• Functional Coverage shows which functions have been covered

• Random stimulus generation can produce large amounts of stimulus. But must be used in combination with a coverage model

• Formal tools can provide stimulus free verification

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

18

Enabling Design…Engineering Change…

• FirstEDA Limited • Technical distributor of EDA solutions for FPGA & ASIC

• Specification, design, implementation, verification & certification

• Represent best-in-class EDA suppliers • Aldec, Concept Engineering, OneSpin Solutions, Sigasi

• Develop, partner & deliver complementary training • Language, methodology and product • Jim Lewis (SynthWorks) partnership for VHDL excellence

Eastlands II

London Road

BASINGSTOKE

RG21 4AW

Implications of Code Coverage Verification Techniques | CONFIDENTIAL | © FirstEDA Limited | 19-May-16

Fax: +44 (1295) 201252

Tel: +44 (1295) 201250

www.firsteda.com

Thank you!

www.firsteda.com