Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
Quantitative Trace Analysis using
Extended Timing Diagrams
Andreas Richter and Klaus Kabitzsch
Dresden University of Technology, Institute of Applied Computer Science,Chair of Technical Information Systems, D-01062 Dresden, Germany
RV12 - September 26, 2012
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
MotivationRequirements
MotivationEmbedded Systems in the Automotive Domain
• Application of networked embedded controllers haspermanently increased over the last decades
• Cars are complex reactive systems
• Up to 100 electronic control units (ECU)• Over 1000 (distributed) functions• More than 100 MB control application code
• ECU strongly interconnected
• CAN, FlexRay, LIN, MOST• Ethernet, Powerline, WLAN
• Ever increasing complexity
• Hybrid technologies• Driver assistance, Car-IT• Car2Car Communication
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 2/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
MotivationRequirements
Quality AssuranceTesting vs. Trace Analysis
• Veri�cation and quality assurance are key topics in these domains
• Development and testing processes with strong tool support
• Many problems not detectable or avoidable by traditional testingand diagnosis
• Complex driver and environment behaviour• Reactivity of systems• Real-time aspects• ↪→ Sporadic and transient errors• ↪→ Non-reproducible faults
B Increase degree of validation through runtime monitoring andsubsequent trace analysis
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 3/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
MotivationRequirements
Quality AssuranceTesting vs. Trace Analysis
• Veri�cation and quality assurance are key topics in these domains
• Development and testing processes with strong tool support
• Many problems not detectable or avoidable by traditional testingand diagnosis
• Complex driver and environment behaviour• Reactivity of systems• Real-time aspects• ↪→ Sporadic and transient errors• ↪→ Non-reproducible faults
B Increase degree of validation through runtime monitoring andsubsequent trace analysis
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 3/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
MotivationRequirements
Runtime Veri�cation for Industrial ApplicationRequirements
Veri�cation Methods
• Typical questions
• How often did a pattern occur?• When and to which extent was a timing constraint violated?• Is an event drifting towards one endpoint of a given timing interval?
• Quantitative trace analysis can answer those!
• Operates on concrete system executions• Access to concrete system state values and timings
Speci�cation Languages
• Su�cient expression power (↗ quantitative properties)
• Understandable and usable for practitioners (visual formalisms)
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 4/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
MotivationRequirements
Runtime Veri�cation for Industrial ApplicationRequirements
Veri�cation Methods
• Typical questions
• How often did a pattern occur?• When and to which extent was a timing constraint violated?• Is an event drifting towards one endpoint of a given timing interval?
• Quantitative trace analysis can answer those!
• Operates on concrete system executions• Access to concrete system state values and timings
Speci�cation Languages
• Su�cient expression power (↗ quantitative properties)
• Understandable and usable for practitioners (visual formalisms)
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 4/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
OverviewQuantitative ExtensionsEvaluation
Visual Speci�cationTiming Diagrams (TD)
• Established speci�cation language in the engineering domains
• Focus on the states of signals, state changes and correspondingtiming relations
• Speci�cation of I/O behaviour of networked systems
• TD resemble the signal display of oscilloscopes
• Former applications
• Interaction modelling during software design (UML2)• Speci�cation language for model checking [Fis99]• Veri�cation of hardware designs [DJS95]
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 5/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
OverviewQuantitative ExtensionsEvaluation
Visual Speci�cationTiming Diagrams (TD)
• Established speci�cation language in the engineering domains
• Focus on the states of signals, state changes and correspondingtiming relations
• Speci�cation of I/O behaviour of networked systems
• TD resemble the signal display of oscilloscopes
• Former applications
• Interaction modelling during software design (UML2)• Speci�cation language for model checking [Fis99]• Veri�cation of hardware designs [DJS95]
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 5/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
OverviewQuantitative ExtensionsEvaluation
Timing DiagramsSyntax Overview
• One or more signals along the vertical axis
• Timeline running in positive horizontal direction
• Signals hold waveforms, formed by a sequences of edges
• State expressions constrain expected signal values
• Only partial order of state value changes
• Relationships (arrows) with interval annotations [min,max] specifytiming constraints between edges
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 6/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
OverviewQuantitative ExtensionsEvaluation
Quantitative Timing DiagramsEdges and State Expressions
• Edges types
• Event edges: exact number of samples that satisfy the expression• Conditional edges: multiple consecutive sample values
• State expressions
• All evaluable relational expressions for the diagram and trace• State expression grammar in EBNF• Keywords for enhanced convenience and expressiveness
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 7/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
OverviewQuantitative ExtensionsEvaluation
Quantitative Timing DiagramsAssignments
• Value assignments, usable on all diagram elements
• Waveforms, edges, relationships, diagram
• Denoted as Element : Assignment similar to [FSS02]
• Evaluated when the de�ning element is evaluated to TRUE
• Assignment expression grammar for complex value calculations
• Evaluated assignments have
• Unique name• Assigned value• Trace time stamp of evaluation
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 8/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
OverviewQuantitative ExtensionsEvaluation
Quantitative Timing DiagramsEvaluation
• Evaluation - Advancing cutlines
1 Create new TD live copy on every activation2 Set cutline to initial edges3 Advance cutline with every incoming signal value
• Success• All waveforms are traversed until their �nal states• No timing constraints were violated
• Fail• Incoming signal value can't advance current cutline• Advancing the cutline violates a timing constraint
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 9/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
OverviewQuantitative ExtensionsEvaluation
Quantitative Timing DiagramsEvaluation
• Evaluation - Advancing cutlines
1 Create new TD live copy on every activation2 Set cutline to initial edges3 Advance cutline with every incoming signal value
• Success• All waveforms are traversed until their �nal states• No timing constraints were violated
• Fail• Incoming signal value can't advance current cutline• Advancing the cutline violates a timing constraint
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 9/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
OverviewQuantitative ExtensionsEvaluation
Quantitative Timing DiagramsEvaluation
• Evaluation - Advancing cutlines
1 Create new TD live copy on every activation2 Set cutline to initial edges3 Advance cutline with every incoming signal value
• Success• All waveforms are traversed until their �nal states• No timing constraints were violated
• Fail• Incoming signal value can't advance current cutline• Advancing the cutline violates a timing constraint
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 9/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
EditorTool IntegrationVisualization
Timing Diagram Editor
• Python-implementation of diagram editor and evaluation engine
• Detailed XML result �les for all calculated values and timings
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 10/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
EditorTool IntegrationVisualization
Integration with TRACE-CHECK
• TRACE-CHECK1 also supports veri�cation of properties formulatedin temporal logic (MTL) and via Python-implemented scripts
• Evaluation operates on traces with non-equidistant time stamps
1http://www.trace-check.de
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 11/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
EditorTool IntegrationVisualization
Integration with TRACE-CHECKReport View
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 12/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
EditorTool IntegrationVisualization
Lessons learnedUse Cases from the Automotive Domain
• Test engineers highly appreciate timing diagram speci�cation
• They often struggle with translating requirements into TL• In logic complex timing relationships between signal curvesunavoidably lead to complicated, strongly nested expressions
• TD nicely complement temporal logic
• Focus on the 'good-cases' or 'positive patterns' of speci�cation• TL: easier to formulate that something must not happen• QTD: testify whether a �nite trace segment behaves like expectedand give detailed information about the execution conditions
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 13/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
EditorTool IntegrationVisualization
Lessons learnedUse Cases from the Automotive Domain
• Test engineers highly appreciate timing diagram speci�cation
• They often struggle with translating requirements into TL• In logic complex timing relationships between signal curvesunavoidably lead to complicated, strongly nested expressions
• TD nicely complement temporal logic
• Focus on the 'good-cases' or 'positive patterns' of speci�cation• TL: easier to formulate that something must not happen• QTD: testify whether a �nite trace segment behaves like expectedand give detailed information about the execution conditions
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 13/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
EditorTool IntegrationVisualization
Visualization Prototype
• Aggregate and overview analysis results by processing result �les
• Diagram activations are plotted along a horizontal timeline
• Zoom, �lter and access concrete timings and values
• Stack and overlay multiple analyses for comparison
• B Visualize system executions on a functional level
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 14/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
ConclusionReferences
Conclusion
Summary
• Extended timing diagrams as speci�cation language for quantitativetrace analysis
• Implementation of TD editor and evaluation engine
• Integration into industrial veri�cation tools
• First application results and prototype for result visualisation
Future Work
1 Give formal syntax and semantics for TD evaluation over �nite traces
2 Use continuous signal descriptions inside state expressions
3 Logically and hierarchically combine multiple diagrams
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 15/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
ConclusionReferences
Conclusion
Summary
• Extended timing diagrams as speci�cation language for quantitativetrace analysis
• Implementation of TD editor and evaluation engine
• Integration into industrial veri�cation tools
• First application results and prototype for result visualisation
Future Work
1 Give formal syntax and semantics for TD evaluation over �nite traces
2 Use continuous signal descriptions inside state expressions
3 Logically and hierarchically combine multiple diagrams
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 15/ 16
IntroductionTiming Diagrams
ApplicationConclusion and Future Work
ConclusionReferences
References I
Werner Damm, Bernhard Josko, and Rainer Schlör.Speci�cation and validation methods.chapter Speci�cation and veri�cation of VHDL-based system-levelhardware designs, pages 331�409. Oxford University Press, Inc., NewYork, NY, USA, 1995.
Kathi Fisler.Timing diagrams: Formalization and algorithmic veri�cation.Journal of Logic, Language and Information, 8:323�361, 1999.10.1023/A:1008345113376.
Bernd Finkbeiner, Sriram Sankaranarayanan, and Henny B. Sipma.Collecting statistics over runtime executions.In Proc. of Runtime Veri�cation (RV02), pages 36�55. Elsevier,2002.
Andreas Richter Quantitative Trace Analysis using Timing Diagrams 16/ 16