Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
© LDRA 2017 LDRA tool suite v9.7.0 August 2017
SimplifyingFunctional Safety
Certificationwith the ARM® DS-5 Development Studio
and the LDRA tool suite®
1
2
Goals
▪Starting with some simple requirements written in Word & Excel, show how a project can be developed using the ARM® DS-5 IDE with the Cortex-M3 VFP
▪Verify the traceability between all the requirements and the code
▪Verify every high level and low level requirement by running tests on the target
▪Verify that the code is MISRA C:2012/AMD1 compliant and also clear, maintainable and testable
▪Verify that 100% structural coverage has been attained
3
Pre-requisites
▪ LDRA tool suite V9.7.0 or above
▪ARM DS-5 IDE (Tested with v5.27.1)
Preparation
4
5
ARM DS-5 Development Studio
▪ It is assumed that ARM DS-5 Development Studio is installed and a license obtained
6
LDRA tool suite
▪ It is assumed that the LDRA tool suite is installed and licensed
7
Environment Variables
▪ In order to use ARM DS-5 via the command line, it may be necessary to configure the following Environment Variables:
▪ This can be done by running a batch file similar to the following:
setx ARM_PRODUCT_PATH "C:\Program Files\DS-5 v5.27.1"
setx ARM_TOOL_VARIANT ult
setx ARMLMD_LICENSE_FILE L:\LDRA_Flexlm\Ltype\DS-5_license.dat
8
Install the TLP (Target License Package)
▪ The TLP allows the LDRA tool suite to work with ARM DS-5 Development Studio
▪Run the ARM DS-5 TLP installer
▪Select the following options
9
ARM DS-5 Installation Path
▪Select the path of where ARM DS-5 is installed
10
ARM Compiler Installation Path
▪Select the path of where the ARM compiler is installed
11
Eclipse Configuration
▪Select LDRA
12
Upload Method
▪Select SCRIPT
Additional Tools
▪Run the LDRA Launcher and then select “Additional Tools”
13
Additional Tools
▪Select the LDRA Eclipse Plug-in
14
Eclipse Plugins
▪Select the DS-5 plugins directory
15
16
ARM_DS5_Workspace
▪Once installed, locate the following workspace
Load the ARM_DS5_Workspace
▪Start ARM DS-5 and select the provided workspace
17
Import
▪ Import the provided Projects
18
Import Project: DS5_Safe_Utilities
▪ Import the following project
19
20
Cortex-M3
▪ This simple project targets the Cortex-M3
21
LDRA Configuration
▪Select the LDRA Configuration
22
LDRA Configuration
▪An LDRA configuration is added to the project to be tested to allow any necessary options such as increased stack size, or suppression of warnings to be set
23
Build
▪Build the project
24
DS-5 Debugger Configuration
▪Create a DS-5 Debugger Configuration
25
DS-5 Debugger
▪Create a new Debug Configuration
26
DS-5 Debugger Configuration
▪Configure the DS-5 Debugger Configuration
27
Debug
▪Now the program can be debugged
28
target.cli
▪ Finally export the DS-5 Debugger Configuration to a file called target.cli
▪ This is a file that will be used by LDRA in order to run on the target
Requirements
29
30
Requirements
▪Open the Requirements folder
31
System Level Requirements
▪ This is a very simple document that contains just one requirement that starts with SYS_
32
High Level Requirements
▪ This document contains High Level Requirements that all start with HLR_ and which have a link to a System Level Requirement
33
Styles
▪Note how in this document, styles have been used to help identify requirements ex:
34
Low Level Requirements
▪ The Low Level Requirements are in an excel document and contain links to High Level Requirements
35
Restore_Initial_State.bat
▪Run the Restore_Initial_State.bat file, this will create a new TBmanager project and in case the tutorial has already been started or completed, will delete any existing results
36
TBmanager
▪Open the newly created TBmanager project, by double-clicking on the .tbp file
37
UniView
▪By default the UniView should be shown
38
Groups
▪ The following groups have been added▪ SLR: System Level Requirements
▪ HLR: High Level Requirements
▪ LLR: Low Level Requirements
▪ HLT: High Level Tests
▪ LLT: Low Level Tests
39
Import System Level Requirements
▪We want to import the System Level Requirements from the Word document into the Group SLR
▪ First switch to the documents view, where we can see that the Requirements documents have already been added
40
Import from Word Document…
▪Select the Safe_Utilities_SLR.docx file, right-click and select “Import from Word Document…”
41
Regular Expressions
▪Note how regular expressions have previously been created, to extract the System Level Requirements
42
Preview
▪Click on Refresh Preview to see that the regular expressions have correctly identified the requirement: number, name and body
▪Note also that the requirements will be imported into the group SLR
43
Import
▪Click on OK to first view the requirements, then on OK again to import them
44
Imported Requirement
▪We should now be able to see the imported requirement in the Project Tree
45
Import High Level Requirements
▪Next we want to import the High Level Requirements from the Word document into the Group HLR
▪Select the Safe_Utilities_HLR.docx file and “Import from Word Document…”
46
Rule 1: Requirement_ID Style
▪ This time since styles have been used, it is much easier to identify the requirements. Different rules have been created for each style ex:
47
Rules 2 & 3
▪Similarly rules have been created for the styles Requirement_Text and Traceability_Data
48
Preview
▪As before, refresh the preview to check the rules
▪Check also that the requirements will be imported into the group HLR
49
Import
▪Click on OK to first view the requirements, then on OK again to import them
50
Imported Requirements
▪We should now be able to see the imported requirements in the Project Tree, nested below SYS_100, since they all have a link to that requirement
51
Import Low Level Requirements
▪Next we want to import the Low Level Requirements from the Excel document into the Group LLR
▪Select the Safe_Utilities_LLR.xlsx file, right-click and select “Configure .xlsx Format…”
52
Columns
▪Each column is identified as a specific attribute ex: requirement: number, name, body, …
53
Group LLR
▪ The first row is ignored and the requirements will be imported into the group LLR
54
Get Requirements from File
▪Click OK to close this menu, then to import the requirements, select “Get Requirements from File”
55
Imported Requirements
▪We should now be able to see the imported requirements in the Project Tree
56
Relationships View
▪Switch to the relationships view
57
SLR Requirements
▪Right-click in the 1st column and add all the requirements from the group: SLR
58
Traceability
▪Now we can observe the upstream and downstream traceability by clicking on any requirement
59
UniView
▪Switch to the UniView view
60
Traceability Matrix
▪Right-Click on the HLR group and select “Traceability Matrix Report to Requirement”, then select Group: LLR
61
High level Requirement <Not Covered>
▪We can see clearly that one High Level Requirement is not covered!
Traceability to Code
62
63
Source Code
▪We now need to look at the traceability between the source code and the Low Level Requirements
▪ First we need to analyse the source code, the simplest way to do this is via the LDRA Eclipse plug-in in ARM DS-5
▪Start ARM DS-5
▪Select the LDRA Perspective
64
Set As Active Project
▪Ensure that the Active Configuration is LDRA
▪Right-click on the project and set as LDRA Active Project
65
Analysis
▪Now perform the analysis
66
Source Code
▪We now need to look at the traceability between the source code and the Low Level Requirements
▪ First we need to analyse the source code
▪ In the Project Tree, right-click on Source Code and select “Add Compiler Project…”
67
Code Violations
▪As can be seen, there are a number of detected violations, we will look at these later
▪But for now, close ARM DS-5
68
TBmanager
▪Back in TBmanager, right-click on Source Code and select “Add Multiple Files as Set…”
69
DS5_Safe_Utiltiies
▪Select the set that was created by the LDRA plug-in called DS5_Safe_Utilties
70
Map Source View
▪Switch to the Map Source View and expand each file
71
Map Source Code
▪Map each function (except main) to the appropriate Low Level Requirement by dragging and dropping
72
Relationships View
▪Switch to the Relationships view and note that every function should now trace back to a requirement
Tests
73
74
Tests
▪Before testing against the requirements, we also want to perform the following tests:▪ Check that the code is compliant to MISRA C:2012/AMD1
▪ Check that the code is clear, maintainable and testable
▪ Then after testing against the requirements, we will want to perform the following test:▪ Check that we have 100% Structural Coverage
75
TCI Grid View
▪Switch to the TCI Grid View
76
Map TCIs to Set
▪Map each TCI (Test Case Identifier) to the System Set by dragging and dropping ex:
77
Map Source View
▪Switch to the Map Source View and the TCIs should be shown
78
Code Review
▪Right-click on the TCI_CodeReview and select “Verify with LDRA tool suite…”
79
Code is Not Compliant
▪ The Green dots indicate that there are no Mandatory or Advisory violations, but there are some Required violations
80
Callgraph
▪Drag the System Set onto the Output Callgraph placeholder
81
Callgraph – Programming Standards
▪Double-click to open the callgraph
82
Violations
▪Clicking on a function, highlights the coding standard violations
83
Code Review Report
▪Alternatively double-click on the Code Review Report
84
View Results with LDRA tool suite
▪Or view the results with the LDRA tool suite
85
Quality Review
▪Next verify the quality of the code
▪Right-click on the TCI_QualityReview and select “Verify with LDRA tool suite…”
86
Code is Clear, Maintainable and Testable
▪ The Green dots indicate that the quality of the code is good, it is clear, maintainable and testable. All the measured metrics are within the specified thresholds
87
Callgraph
▪Drag the System Set onto the Output Callgraph placeholder, then double-click to open the Callgraph
88
Maintainability View
▪Select the Maintainability View
89
Sort Metrics
▪Clicking on the column title, sorts the metrics by value, making it easy to locate the most complex function
▪ The flowgraph for each function can be viewed ex:
90
Flowgraph
High Level Tests
91
92
High Level Tests
▪Next we want to verify the High Level Requirements
▪ The Safe_Utilities_HLT.xslx file contains just a single High Level Test
93
High Level Tests
▪ For this test, we are simply going to execute the main which exercises every function and afterwards check the structural coverage
94
Import High Level Tests
▪Switch to the Documents View, select the Safe_Utilities_HLT.xslx file and “Get Test Cases from File”
95
Set External Task
▪Set the External Task to be the following
96
Build Instrumented
▪As the code executes, we need to measure the structural coverage. In order to do this, we first need to instrument the source code and then to perform a build
▪ This can be done by executing the batch file Build_Instrumented.bat
▪Click on the following menu to do this
97
Verify with External Task
▪Now we can right-click on the HLT and verify with External Task
98
Execution History
▪At the end of the execution, the compressed execution history is uploaded to the host
▪ This can now be processed by running the batch file Get_Coverage.bat
▪Click on the following menu to do this
99
Code Coverage
▪Now switch to the Map Source View and verify the code coverage
100
Callgraph
▪Drag the System Set onto the Output Callgraph placeholder
▪Double-click to view the pass/fail coverage Callgraph
101
Callgraph
▪As expected, every function has been invoked, but of course we don’t have 100% coverage
102
Flowgraph
▪View the Coverage Pass/Fail Flowgraph for the function safe_uncompress
103
Defensive Programming
▪Since the code checks that the parameters aStringand anArray are not NULL, we don’t have 100% structural coverage
▪ In order to test this “defensive programming”, we will need to perform unit testing
Low Level Tests
104
105
Low Level Tests
▪Next we want to verify the Low Level Requirements
▪ The Safe_Utilities_LLT.xslx file contains the Low Level Tests
106
Import Low Level Tests
▪Switch to the Documents View, select the Safe_Utilities_LLT.xslx file and “Get Test Cases from File”
107
Associated Test Case File (.tcf)
▪With each Low Level Test, there is an associated Test Case File which contains a sequence of test cases
▪ TCF files can be regressed using TBrun
▪Select the following Low Level Test and “Verify Test Interactively in TBrun...”
108
TBrun
▪ For each Test Case, we can see the list of inputs and expected outputs
109
Run
▪ The Test Cases can now be compiled, linked and executed on the target
▪ The Test Cases all pass with 100% coverage
110
Test Passes
▪Exiting TBrun will update the status of the Low Level Tests in TBmanager
▪Now verify the next Low Level Test
111
Failed Test Case
▪ This time the Test Case fails
112
Failed Test Case
▪ This is exactly why we unit test, the function safe_uncompress does not work correctly and must be modified
113
Regression Report
▪Viewing the Regression Report shows why the test case failed
114
Fail
▪Since the Test failed, it has a red dot
115
TCI Grid
▪Switch to the TCI Grid
116
Filter
▪Press “Control + L” to create the following filter
117
2nd Filter
▪Press “+” to create a second filter as shown
118
Regress All Unverified Tests
▪Press “Control + A” to select all the unverified Test Cases, then regress them
119
One Unverified Test Case Identifier
▪Once the filter is refreshed, there should now be just the single unverified Test Case Identifier
120
Code Coverage
▪Now that all the unit tests have been run, check that we have 100% structural coverage
121
Objectives
▪Next import the objectives from a standard
122
ISO 26262
▪Any standard can be imported, but for the moment, select ISO 26262 and click OK
123
Standards
▪ The existing standards can be customised or additional standards added ex:
124
Objectives View
▪Switch to the Objectives View
125
Placeholders
▪ For each objective, it is possible to add placeholders for all the various artifacts / assets that need to be created or produced in order to satisfy it
126
Placeholders
▪Expand the objective “Part 6: Section 5: Table 1: 1a”
▪Note the placeholders for Artifacts that need to be created/produced in order to satisfy this objective
127
Realising Objectives
▪As the Artifacts/Assets get produced, they can be used to satisfy the placeholders
▪Right-click on each placeholder and associate the appropriate document ex:
128
Satisfied Placeholders
129
Fulfilled Objective
▪Once the associated documents, have been reviewed, the status of the objective can be changed to Fulfilled
130
Objectives Report
▪Generate an Objective Summary Report and navigate from it to view the documents
131
Reports
▪ Finally, reports such as the Project Coverage Detailed Report can be generated
132
Command Line
▪Everything that was performed manually in this tutorial can be automated from a batch file
▪Close TBmanager and try running the following batch file
Summary
133
134
Summary
▪ In this simple example, we have seen:▪ How the traceability from requirements to code can be
verified
▪ How the code in a ARM DS-5 project can be analysed and checked for MISRA C:2012/AMD1 compliance as well as ensuring that it is clear, maintainable and testable
▪ How the High Level Requirements and Low Level Requirements can be verified by executing tests on the target
▪ How the Structural Coverage can be measured
▪ How everything can be automated from a batch file
135
For More Information
▪ For more information, please view the following tutorials