Embed Size (px)
Citation preview
Validation and
verification of
specification models
Test4Rail, Braunschweig
Dr. Oliver Lemke
V2.0
Agenda
− Introduction
− Needs
− Process
− Conclusion
18.10.2017 2
18.10.2017 3
SIGNON business activities
Planning Technical Consulting Engineering
Signalling systems
Telecommunications
Power supply
Systems
Software
Safety
Studies
Methodology
Processes
Introduction
− Using models and (semi-)formal languages like SysML have become more common
over the last years for specifying railway systems:
▪ NeuPro – DB’s standardisation of interlocking architecture in Germany
▪ EULYNX – European counterpart
− EULYNX (https://www.eulynx.eu) is an initiative of 12 European infrastructure
managers (IMs) to harmonize interlocking architectures
− This is accomplished by creating unified operator’s specification documents for the
supply industry
18.10.2017 4
NO_OPERATING_VOLTAGE
BOOTING FALLBACK_MODE
OPERATIONAL
INITIALISING
OPERATING_VOLTAGE_SUPPLIED
F_EST_SubS_TDS - Behaviour [SubSTDS STD1]
when( T1_P ower_On_detec ted )/when( T2_P ower_Off_detec ted )/
when( T5_S IL_not_fulfilled )/
when( T3_Res et )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T4_Booted )[D13_S tart_Tim e_s ync hronisation_is_initiated]/
when( T9_Proces s_Data_Interface_connec tion_established )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T7_Invalid_or_m iss ing_Configurat ion_Data_carrier )/
when( T10_Safe_Communication_Protocol_Connection_disconnected
)/
SysML state machine diagram Simulation interface
Specification document
18.10.2017 5
Agenda
− Introduction
− Needs
− Process
− Conclusion
18.10.2017 6
Needs
− The claim is, that model-based, (semi-)formal specifications improve the
specification quality by:
▪ Being correct
▪ Being consistent
▪ Being unambiguous
− But this is only true if, the underlying model is unambiguous, consistent and correct
− As models can be reused for system acceptance tests, integration tests and various
simulations, these requirements for model quality are even aggravated.
Hence assuring a high level of quality for the models becomes essential.
18.10.2017 7
Agenda
− Introduction
− Needs
− Process
− Conclusion
18.10.2017 8
Process - simplified CENELEC V-model
Implementation
phase
Validation
Verification
P1
P5
18.10.2017 9
P9
Process - small V-model for specification phase
State machine (STM)
implementation
STM
acceptance
Validation
Verification
User
requirements
Formalised
requirements
18.10.2017 10
User
Reqs. Formalised reqs.
State machine
implementation
State machine
acceptance
System Env.
Stimulus A
Response a
Response d
Stimulus B
Stimulus C
Scenarios as SysML
sequence diagrams (ca. 20 –
50 scenarios per subsystem)
NO_OPERATING_VOLTAGE
BOOTING FALLBACK_MODE
OPERATIONAL
INITIALISING
OPERATING_VOLTAGE_SUPPLIED
F_EST_SubS_TDS - Behaviour [SubSTDS STD1]
when( T1_P ower_On_detec ted )/when( T2_P ower_Off_detec ted )/
when( T5_S IL_not_fulfilled )/
when( T3_Res et )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T4_Booted )[D13_S tart_Tim e_s ync hronisation_is_initiated]/
when( T9_Proces s_Data_Interface_connec tion_established )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T7_Invalid_or_m iss ing_Configurat ion_Data_carrier )/
when( T10_Safe_Communication_Protocol_Connection_disconnected
)/
SysML state machines in
modelling tool Executable simulator
Know-
ledge
Informal
documents
Creation - Modeller
Verification - Tester
Validation - Stakeholder
18.10.2017 11
Process - verification
Verification step 1 (Black-Box-Verification):
− Verify that state machines (STM) react as specified in sequence diagrams (SD)
− Implemented by stimulating the executable simulator according to the SDs and
reading back its responses, comparing them to the responses defined in the SDs
− Test execution is highly automated through GUI testing tools (e.g. Ranorex)
Verification step 2 (White-Box-Verification):
- Verify that STM does not add implicit behaviour not specified in sequence diagrams
- Checked by verifying that all SDs fully cover the STM according to defined coverage
criteria, e.g. full state and transition coverage
- Unmarked states and transitions are not covered by sequences and therefore describe
additional behaviour
- Generate SDs covering the missing elements in STM and discuss with stakeholder
18.10.2017 12
NO_OPERATING_VOLTAGE
BOOTING FALLBACK_MODE
OPERATIONAL
INITIALISING
OPERATING_VOLTAGE_SUPPLIED
F_EST_SubS_TDS - Behaviour [SubSTDS STD1]
when( T1_P ower_On_detec ted )/when( T2_P ower_Off_detec ted )/
when( T5_S IL_not_fulfilled )/
when( T3_Res et )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T4_Booted )[D13_S tart_Tim e_s ync hronisation_is_initiated]/
when( T9_Proces s_Data_Interface_connec tion_established )/
when( T5_SIL_not_fulfilled )/
T12_Interrupt_Safe_Communication_Protocol_Connection := true;
when( T3_Res et )/
when( T7_Invalid_or_m iss ing_Configurat ion_Data_carrier )/
when( T10_Safe_Communication_Protocol_Connection_disconnected
)/
18.10.2017 13
Process - validation
Know-
ledge
Informal
documents
Operational stakeholders validate the STM against the input documents by using their own
test cases. This assures diversity and coverage of domain knowledge.
18.10.2017 14
Agenda
− Introduction
− Needs
− Process
− Conclusion
18.10.2017 15
Experiences
Statistics for a SysML model of the interface between interlocking and axel counting
system.
Model
− Number of SDs as formalised requirements: 45
− Number of states in the STMs: 15
− Number of transitions in the STMs: 38
Detected errors (after multiple manual reviews and quality checks)
− Verification - functional errors (STM does not match SDs): 5
− Verification - implicit behaviour (STM contains behaviour not specified in SDs): 3
− Validation - functional errors: 3
18.10.2017 16
Conclusion
− The process presented is able to improve the specification quality
− The quality of model-based specifications is typically higher than of text-based
specifications
− The additional benefit of using formal verification techniques must be evaluated, as
the efforts for applying them in the real world are still very high
Thank you!
18.10.2017 17
