Vision for Systematic V&V - Makers of MATLAB and Simulink · PDF fileMathWorks Vision for...

1© 2016 The MathWorks, Inc.

MathWorks Vision for Systematic

Verification and Validation

Bill Aldrich

Senior Development Manager

Simulink Verification and Validation, Simulink Design Verifier

2

Growing Complexity of Automotive Controls

Engine Management

Transmission Control

Forward Camera

Electric Power Steering

Smart Junction Box

Smart Junction Box

Battery Management

Propulsion Motor Control

DC/DC Converter

Stability Control

Infotainment

HVAC Control

Navigation

Instrument Panel

Vehicle-to-Vehicle

Vehicle-to-

Infrastructure

Short-Range Radar

Ultrasonic Sensor

Long-Range Radar

Stability Control

AirbagEmergency Braking

Automatic Parking

Adaptive Cruise Control

All-Wheel Drive

Active Damping

4-Wheel Steer

Back-up Camera

Body Control Module

Tire Pressure Monitor

Voice Recognition

Adaptive Front

Lighting

Power Window

Power Seat

Keyless Entry

Power Liftgate

E-Call

Source of graphic: http://360.here.com/2013/11/28/putting-firmly-drivers-seat/

3

Growing Complexity of Automotive Controls

Engine Management

Transmission Control

Forward Camera

Electric Power Steering

Smart Junction Box

Smart Junction Box

Battery Management

Propulsion Motor Control

DC/DC Converter

Stability Control

Infotainment

HVAC Control

Navigation

Instrument Panel

Vehicle-to-Vehicle

Vehicle-to-

Infrastructure

Short-Range Radar

Ultrasonic Sensor

Long-Range Radar

Stability Control

AirbagEmergency Braking

Automatic Parking

Adaptive Cruise Control

All-Wheel Drive

Active Damping

4-Wheel Steer

Back-up Camera

Body Control Module

Tire Pressure Monitor

Voice Recognition

Adaptive Front

Lighting

Power Window

Power Seat

Keyless Entry

Power Liftgate

E-Call

2000 2015Lines of Code

16 M

2-3M

6 M

Siemens, “Ford Motor Company Case Study,” Siemens PLM Software, 2014

McKendrick, J. “Cars become ‘datacenters on wheels’, carmakers become software companies,” ZDJNet, 2013

4

Growing Complexity of Automotive Controls

Engine Management

Transmission Control

Forward Camera

Electric Power Steering

Smart Junction Box

Smart Junction Box

Battery Management

Propulsion Motor Control

DC/DC Converter

Stability Control

Infotainment

HVAC Control

Navigation

Instrument Panel

Vehicle-to-Vehicle

Vehicle-to-

Infrastructure

Short-Range Radar

Ultrasonic Sensor

Long-Range Radar

Stability Control

AirbagEmergency Braking

Automatic Parking

Adaptive Cruise Control

All-Wheel Drive

Active Damping

4-Wheel Steer

Back-up Camera

Body Control Module

Tire Pressure Monitor

Voice Recognition

Adaptive Front

Lighting

Power Window

Power Seat

Keyless Entry

Power Liftgate

E-Call

2000 2015Lines of Code

16 M

2-3M

6 M500K

50M+

3-4M

100MToday cars

Ford Taurus

2012

Boeing 777

Space

Shuttle

Source:

https://interact.gsa.gov/sites/default/files/J3061%20JP%20presentation.pdf

5

Growing Complexity of Automotive Controls

Engine Management

Transmission Control

Forward Camera

Electric Power Steering

Smart Junction Box

Smart Junction Box

Battery Management

Propulsion Motor Control

DC/DC Converter

Stability Control

Infotainment

HVAC Control

Navigation

Instrument Panel

Vehicle-to-Vehicle

Vehicle-to-

Infrastructure

Short-Range Radar

Ultrasonic Sensor

Long-Range Radar

Stability Control

AirbagEmergency Braking

Automatic Parking

Adaptive Cruise Control

All-Wheel Drive

Active Damping

4-Wheel Steer

Back-up Camera

Body Control Module

Tire Pressure Monitor

Voice Recognition

Adaptive Front

Lighting

Power Window

Power Seat

Keyless Entry

Power Liftgate

E-Call

Source of graphic: http://360.here.com/2013/11/28/putting-firmly-drivers-seat/

6

Development Challenges

7

Development Challenges

Representing complex systems

8

Development Challenges

Representing complex systems

Coordinating work across teams

9

Development Challenges

Representing complex systems

Coordinating work across teams

Working efficiently

10

Development Challenges

Representing complex systems

Coordinating work across teams

Working efficiently

Ensuring quality

11

Traditional Development Process

Textual

Requirements

Design

Specification

C/C++

Hand code

Object

code

Compilation

and Linking

Manual

Coding

12

Models for Specification

Textual

Requirements

C/C++

Hand code

Object

code

Compilation

and Linking

Manual

Coding

Executable

Specification

13

Model Abstraction – Work at an appropriate level of detail

Simulink

Simulink

Stateflow MATLAB

Simscape

Driveline

Simscape Multibody

Simscape Fluids

14

Complete Model Based Design Workflow, Concept to Code

Textual

Requirements

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Executable

Specification

Model used for

production code

generation

15

Complete Model Based Design Workflow, Concept to Code

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

How do you ensure correctness?

16

Model-Based Design Maturity, Automotive Industry

0

20

40

60

80

100

Modeling

Simulation and Analysis

Implementation

Verification andValidation

Process, Toolsand Infrastructure

Enterprise Management

17

Model-Based Design Maturity, Automotive and Aerospace

0

20

40

60

80

100

Modeling

Simulation and Analysis

Implementation

Verification andValidation

Process, Toolsand Infrastructure

Enterprise Management

Automotive

Aerospace

18

Model Based Design Verification Workflow

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Component and system

testingEquivalence

testing

Equivalence

checking

Review and

static analysis

Simulink Models

19

Model Based Design Verification Workflow

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Component and system

testing

• Perform simulation

• Link and review requirements

• Isolate and test components

• Measure model coverage

• Address missing coverage

• Property proving

Simulink Models

20

Ad-Hoc Simulation: Explore Behavior Virtually

21

Model Based Design Verification Workflow

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Component and system

testing

• Perform simulation

• Link and review requirements

• Isolate and test components

• Measure model coverage

• Generate tests for missing coverage

• Manage and organize tests

• Property proving

Simulink Models

22

Test Harnesses

From any subsystem …

23

Test Harnesses

Isolate it with content it to

drive inputs and analyze

outputs

Can be embedded in design

model file.

From any subsystem …

Simulate

independently

24

Test Sequence Block

A test sequence block can drive

inputs

25

Test Sequence Block

A test sequence block can drive

inputs and asses outputs

26

Test Sequence Block Syntax

27

Test Sequence Block Syntax

Define

Inputs

28

Defining Pass/Fail Criteria

x

29

Model Coverage

Identify testing gaps:

• Untested switch positions

• Subsystems not executed

• Transitions not taken

• Many more …

30

Addressing Missing Coverage

Test Cases

Partial Coverage

31

Test

Generator

Simulink Design Verifier

Addressing Missing Coverage

Test Cases

Partial Coverage

32

Addressing Missing Coverage

Test Cases

Partial Coverage

Test

Generator

Simulink Design Verifier

New Test Cases

33

Addressing Missing Coverage

Test Cases

Full Coverage

New Test Cases

34

Model Based Design Verification Workflow

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Component and system

testing

• Perform simulation

• Link and review requirements

• Isolate and test components

• Measure model coverage

• Generate tests for missing coverage

• Manage and organize tests

• Property proving

Simulink Models

35

Model Based Design Verification Workflow

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Review and

static analysis

• Manual review

• Standards compliance checking

• Design error detection

• Complexity analysis

Simulink Models

36

Detecting Hidden Run-Time Design Errors

Design Model

Design error detection

Highlighted Model Integer overflow

Division by zero

Array out-of-bounds

Range violations

Dead Logic

37

Detecting Hidden Run-Time Design Errors

38

Detecting Hidden Run-Time Design Errors

39

Model Based Design Verification Workflow

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Equivalence

checking

• Perform SIL Testing

• Measure code coverage

• Verify code with Polyspace

• Verify consistency with Simulink

Code Inspector

40

Coverage for Generated Code (R2016a)

cruise_control

(SIL)

41

Coverage for Generated Code (R2016a)

Press Play

cruise_control

(SIL)

42

Coverage for Generated Code (R2016a)

Press Play

Generated Code Coverage

cruise_control

(SIL)

43

Coverage for Generated Code (R2016a)

Press Play

Can also be highlighted on model

cruise_control

(SIL)

44

Model Based Design Verification Workflow

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Equivalence

testing

• Perform PIL Testing

• Perform HIL Testing

45

Model Based Design Verification Workflow

Textual

Requirements

Executable

Specification

Model used for

production code

generation

Generated

C/C++ code

Object

code

ModellingCompilation

and Linking

Code

Generation

Component and system

testingEquivalence

testing

Equivalence

checking

Review and

static analysis

Simulink Models

46

Model Based Design Verification Workflow

47

Systematic Verification

Simulink Design Verifier

Simulink Verification & Validation

Simulink Test

Ensure that verification

is systematically

performed across:

– All requirements

– Complete model structure

– Complete code structure

– All design behaviors

48

• Essential activity

Test and Verification

• Expensive to repeat

• Complex to execute

Pain Points

49

→ Faster

Test and Verification

• Essential

• Expensive

• Complex

→ More Complete

→ Simpler

Frequent,

Incremental

T&V

50

Thank You!