View
492
Download
1
Category
Preview:
DESCRIPTION
Slides of a paper at ERTSS'2014 co-authored by Nicolas NAVET (University of Luxembourg), Shehnaz LOUVART (Renault), Jose VILLANUEVA (Renault), Sergio CAMPOY-MARTINEZ (Renault) and Jörn MIGGE (RealTime-at-Work). Early stage timing verification on CAN traditionally relies on simulation and schedulability analysis, also known as worst-case response time (WCRT) analysis. Despite recent progresses, the latter technique remains pessimistic especially in complex networking architectures with gateways and heterogeneous communication stacks. Indeed, there are practical cases where no exact WCRT analysis is available, and merely upper bounds on the response times can be derived, on the basis of which unnecessary conservative design choices may be made. Simulation, on the other hand, does not provide anyguarantees per se and, in the context of critical networks, should only be used along with an adequate methodology. In this paper, we argue for the use of quantiles of the response time distribution as performance metrics providing an adjustable trade-off between safety and resource usage optimization. We discuss how the exact value of the quantile to consider should be chosen with regard to the criticality of the frames, and illustrate the approach on two typical automotive use-cases.
Citation preview
Timing verification of automotive communication architecture using quantile estimation
Nicolas NAVET (Uni Lu), Shehnaz LOUVART (Renault), Jose VILLANUEVA (Renault), Sergio CAMPOY-MARTINEZ (Renault) and Jörn MIGGE (RealTime-at-Work).
ERTSS’2014 - Toulouse, February 5-7, 2014. February 07, 2014
1 Outline
- 207/02/2014ERTSS'2014
Early-stage timing verification of wired automotive
buses – CAN-based communication architectures
Schedulabilityanalysis versus
simulation
Performance metrics : the
case for quantiles
derived by simulation
2 typical automotive use-cases
2 Automotive communication architectures
- 307/02/2014ERTSS'2014
Increased bandwidth requirements & timing constraints
More complex & heterogeneous architectures with
black-box ECUs
Optimized CAN networks for higher bus loads:
priorities, frame offsets, gateways, communication
stacks, etc
Verification activity of higher importance today, higher
load levels calls for more accurate verification models
no margin for errors
Main performance metrics: frame response time =
communication latency
Upper bounds on the perf.
metrics Safe if model is correct
and assumptions met
Often pessimistic over-
dimensioning
Might be a gap between
models and real systems!
unpredictably unsafe then
07/02/2014 - 4
Schedulability analysis “mathematic model of the worst-case possible situation”
Schedulability analysis : “mathematic model of the worst-case possible situation”
Simulation “program that reproduces the
behavior of a system”
max number of
instances that can
accumulate at critical
instants
max number of
instances arriving after
critical instants
VS
Models close to real systems
Fine grained information
Worst-case response times are
out of reach! Occasional deadline
misses must be acceptable
ERTSS'2014
07/02/2014 - 5
RTaW : “enable designers to build provably
safe and optimized critical systems”
– Simulation and schedulability analysis for networks and ECU
CAN, CAN FD, Arinc825, Ethernet, FlexRay, AFDX, etc…
– OEM customers: Renault, PSA, Eurocopter, Astrium, ABB
− RTaW/Sim Starter edition can be downloaded from www.realtimeatwork.com
− No black box software: all schedulability analysis that are implemented are published
Used in this study
RTaW-Sim CAN simulator
with schedulability analysis
and configuration algorithms
ERTSS'2014
Metrics for the evaluation of frame latencies: the case for quantiles
2
Frame response time distribution
07/02/2014 - 10
Pro
bab
ility
Response time
(actual) worst-case
response time (WCRT)
Upper-bound with
schedulability analysis
Easily observable events Rare eventsInfrequent events
Testbed /
Simulation
Long
Simulation
Schedulability
analysis
Simulation max.
Q1
Q2
Q1: pessimism of schedulability analysis ?!
Q2: distance between simulation max. and WCRT ?!
ERTSS'2014
07/02/2014 - 11
Pro
babili
ty
Response time
Simulation max.
Upper-bound with
schedulability analysis
Quantile Qn: P[ response time > Qn ] < 10-n
Q5Q4Probability
< 10-5
Using quantiles means accepting a controlled risk
one frame
every 100 000
No extrapolation here, won’t help to say anything about what is
too rare to be in simulation traces
ERTSS'2014
07/02/2014 - 12
Pro
babili
ty
Response time
Simulation max.
Q5Q4
Identifying both deadline and tolerable risks
1. Identify frame deadline
2. Decide the tolerable risk target quantile
3. Simulate “sufficiently” long
4. If target quantile value is below deadline,
performance objective is met
ERTSS'2014
deadline
07/02/2014 - 13
1) Quantiles vs average time between
deadline misses
Quantile One frame
every …
Mean time to failure
Frame period = 10ms
Mean time to failure
Frame period = 500ms
Q3 1 000 10 s 8mn 20s
Q4 10 000 1mn 40s ≈ 1h 23mn
Q5 100 000 ≈ 17mn ≈ 13h 53mn
Q6 1000 000 ≈ 2h 46mn ≈ 5d 19h
… … …
Warning : successive failures in some cases might be
temporally correlated, this must be assessed!Use of distributions of successive quantile overshoots, linear and
non-linear dependency analysis
ERTSS'2014
07/02/2014 - 14
2) Determine the minimum simulation length
time needed for quantile convergence
reasonable # of values: a few tens …
Tool support can help here:
e.g. numbers in gray
should not be trusted
[RTaW
-sim
scre
enshot]
Reasonable values for Q5 and Q6
(with periods <500ms) are obtained in
a few hours of simulation (with a high-
speed simulation engine) – e.g. 2 hours
for a typical automotive setup
ERTSS'2014
Typical use-cases of quantile-based
performance evaluation
3
Use-case 1: OBD2 request through a gateway
07/02/2014 - 16
50% load – 500kbit/s
40% load – 500kbit/s
Time between the OBD2 request frame
and reception of the first answer frame
must not be greater than 50ms once every
1000 requests
ERTSS'2014
[RTaW-sim screenshot]
OBD2
request
response
Simulated
production
delay
Conservative assumptions:
FIFO, transmission errors
Use-case 1: OBD2 request through a gateway
07/02/2014 - 17
Time between the OBD2 request frame
and reception of the first answer frame
must not be greater than 50ms once every
1000 requests
ERTSS'2014
Metrics
OBD
response
times
Min 31.94
Average 34.29
Q3 46.55
Q4 49.31
Q5 53.45
Q6 55.32
Max 56.57Response time distribution
Q4Q3
07/02/2014 - 18
Functional level impact: less than 1 frame every 106
above deadline=10ms is acceptable
ERTSS'2014
10ms
T13 frame
Use-case 2: end-to-end response time of a 10ms
control frame
Q6 = 8.9
max= 12.1
07/02/2014 - 19
Concluding remarks
Simulation is well suited to systems that requires timing guarantees but
Are not well amenable to schedulability analysis Or can tolerate deadline misses with a controlled level of risk
2
3 Some methodological aspects
Determine quantile wrt criticality, and simulation length wrt to quantile Simulator and models validation High-performance simulation engine needed for higher quantiles
ERTSS'2014
Timing verification techniques & tools should not be trusted blindly1
Recommended