Upload
iram
View
66
Download
0
Embed Size (px)
DESCRIPTION
From Validating Models to Validating Systems. Peter Denno 2013-02-25 University of Maryland ISR Colloquium. Outline. Introduction / Scoping Requirements for MBSE Exchange Form Validation NIST Work. Goals. Describe a “design philosophy” for systems that assist in systems engineering - PowerPoint PPT Presentation
Citation preview
1
From Validating Models to Validating Systems
Peter Denno2013-02-25
University of Maryland ISR Colloquium
2
Outline
• Introduction / Scoping• Requirements for MBSE• Exchange Form Validation• NIST Work
3
Goals
• Describe a “design philosophy” for systems that assist in systems engineering– Framework for linking multiple viewpoints– Framework for research
• Link the design philosophy to NIST workin exchange form validation, requirements engineering, supply chain logistics simulation
4
What is special about V&V ? (1)
• IBM Watson- New techniques – exceed human capability in
knowledge-intensive tasks
- “Machine understanding is not human understanding.”
- “Knowledge is not the destination.”
5
What is special about V&V ? (2)
• Validation & Verification- Knowledge is the destination. – knowledge, or at least
credible rationale.
• Requirement:- Minimally: be able to explain how the design space was
characterized and demonstrate that requirements are being met.
- Ideally: Provide deductive arguments where appropriate- Show how certain alternatives are indeed incompatible- Reference principles of operation, functions
6
Outline
• Introduction / Scoping• Requirements for MBSE• Exchange Form Validation• NIST Work
7
Basis for SE decision making
• SE decision making macro level:– Trade studies, simulations, risk assessment, etc.
• SE decision making micro level:– A web (conceptual schema) of information• Uncertain • Conflicting• Isolated
• Uncertainty is quantified• Conflicts resolved• Inter-relation revealed
8
Strategy for the micro-level information
• Characterize elements of rationale for SE decision making.
• Each research project touches on only a few of these elements– No single overarching system design intended
9
9 Elements of Rationale (1)• Measurement Conditions
– Confidence in the process or environment under which it was measured– “Capacitance was measured using the AC impedance technique.”
• Logical Consistency– Confidence due to consistency with theory. Type consistent.– “P = .05, as we’d expect from the law on conservation of energy.”
• Associativity Across Views– Individuals: knowledge that two references made from different viewpoints refer to the
same thing– “The region P on the CAD model corresponds to these elements
in the FEA mesh.” (Individuals)– Concepts: knowledge that two conceptualizations can be used for the same purpose.– “What the supplier is calling ‘rated maximum pressure’ is what we call ‘rated pressure.”
(Concepts)
10
9 Elements of Rationale (2)• Change process
– Knowledge of precursors and the history of properties that distinguish them.
– “The value of P that we calculated for this design is close to what we found in earlier models.”
• Authority– The power that information has due to an approval that is granted or
an estimate of its maturity– “Supplier-provided data also suggest P=.05 is obtainable.”
• Origin in Requirements*– Belief that a requirement is sensitive to it– “Our ability to achieve requirement x diminishes as P exceeds 0.07.”
11
9 Elements of Rationale (3)• Origin in organization infrastructure
– Belief because you obtained it in ways consistent with the organization’s best practices.
– “P was obtained from the aero model in the preliminary design library.”
• Consistency with other belief– Belief due to consistency with prevailing contingent facts– “P=0.5 is reasonable in products using component y.”
• V&V Process– Belief that the system in place to manage the other 8 elements is sound
and comprehensive.– “The value of P is confirmed through simulation that is routinely
performed in validation of this product line.”
12
9 Elements : Observations
• Coupling and overlap– Authority / Origin in Organizational Infrastructure– Associativity across views / measurement conditions– etc.
• Though these are found in models, they can be expressed from a more comprehensive viewpoint where – Contradictions can be exposed– Cohesion across views can be noted– Trace to requirements is more evident– (These are all parts of V&V)
13
MBSE Concepts / Logical View
14
Sentence detail / rationale
15
Example Usage Patterns
• V & V– Origin in requirements – Automated generation of test cases
• Requirements Engineering– Origin in other belief, emphasis on tracking
contingent facts and engineering change– Refinement
16
Outline
• Introduction / Scoping• Requirements for MBSE• Exchange Form Validation• NIST Work
17
Exchange Form Validation : Two Methods
1. Axiomatic: How: Map the exchanged content to sentences Identify errors: ex falso quodlibet with a reasoner Advantage: Ontology explains intent Disadvantage: Proofs hard to interpret
2. Metamodel: How: Map the exchanged content to objects Identify errors: Direct structural, with OCL, etc. Advantage: Constraints relate to exchange form Disadvantage: Constraints look like code
18
Example use of metamodel
View / Viewpoint: Can be both consistent with a form (a view), and the form by which otherconceptualization are stated (a viewpoint.)
19
Example from the UML Metamodel
20
Example Specification Constraints
21
In MBSE, metamodels play a key role
• Metamodel =(1) a specification of the form a model can take. (well-formedness conditions)
(2) a formalization of the viewpoint that models will express
22
Metamodels also play a key role in model exchange
• Metamodel =(1) a specification of the form a model can take. (well-formedness conditions)
Definition of structure serialization
(2) a formalization of the viewpoint that models will express
Illuminate what program structures the elementsof exchange content map to/from.
23
Communication with Exchange Standards
24
Outline
• Introduction / Scoping• Requirements for MBSE• Exchange Form Validation• NIST Work– Model Interchange Working Group– Supply Chain Logistics Simulation– Collaborative Requirements Engineering
25
Outline
• Introduction / Scoping• Requirements for MBSE• Exchange Form Validation• NIST Work– Model Interchange Working Group– Supply Chain Logistics Simulation– Collaborative Requirements Engineering
26
OMG Model Interchange Working Group
• Goal: Improve the ability of OMG MOF-based tools (UML, SysML) to exchange information – XMI Serialization – common to MOF-based tools.
• Process– Group: Produce Test Case diagram and reference file– Tool developers: create diagram in their tool, serialize as XMI– Use NIST tool to identify errors (in files and metamodels)– Correct tools and specifications
27
NIST UML / SysML Validator
Enter below a file to upload:
28
NIST UML / SysML Validator
29
NIST UML / SysML Validator
30
NIST UML / SysML Validator
31
MIWG Results• Stakeholders witness significant improvement
in interoperability
Elaasar & Labiche, 2012
32
Outline
• Introduction / Scoping• Requirements for MBSE• Exchange Form Validation• NIST Work– Model Interchange Working Group– Supply Chain Logistics Simulation– Collaborative Requirements Engineering
33
Supply Chain Logistics Simulation
• Goal: Demonstrate integrated use of models toward enterprise goals
• Design: Map models, guided by metamodels into sentences that guide compilation of a discrete event simulation.
• Models– UML of ordering / logistics objects– QVT-r mapping of messages to orders– BPMN “stereotyped” + OCL of business decisions– Discrete Event Simulation
34
Round Trip Engineering of Supply Chain Logistics
35
Logistics Processes (1)
36
Logistics Processes (2)
37
Business Rule
38
Simulation Results
39
Outline
• Introduction / Scoping• Requirements for MBSE• Exchange Form Validation• NIST Work– Model Interchange Working Group– Supply Chain Logistics Simulation– Collaborative Requirements Engineering
40
Collaborative Requirements Engineering
• Goal: Demonstrate Engineering from Product Data Sheets
• Design: Map product data sheets in to sentences about requirements. Use these to guide engineering simulation and reasoning about alternative designs
41
Conclusions
• Continuing roles for deductive reasoning in the automation of SE processes – The nature of V&V, requirements engineering, the
way we think when we engineer, require it.
• Preparing and interpreting macro-level SE decision processes is aided by the integration of multi-viewpoint, micro-level information.
• Metamodels facilitate this integration.
42
References• Welty, C; Inside the mind of Watson, 2nd ESWC Summer School, Kalamaki,
2012, http://videolectures.net/eswc2012_welty_watson
• Denno, P; Thurman, T, Mettenburg, J; Hardy, D; On enabling a model-based systems engineering discipline – 18th INCOSE International Symposium (2008)
• Denno, P; Harrison, T; Using Legacy Modeling Artifacts in Supply Chain Logistics Simulation (in draft, 2013)
• ISO 15288 (2008) – Systems and software engineering – System life cycle processes. (2008)