Independent Verification and Validation (IV&V) Techniques for Object Oriented Software Systems
SAS meeting July 2003
COMPANY PROPRIETARY2
APC0342
Outline
IV&V of OO artifacts
Identifying Risks
Proposed Techniques
GQM (Goal Question Metric) Application
BBN (Bayesian Belief Network) Application
COMPANY PROPRIETARY3
APC0342
Problem Statement
NASA has considerable experience applying IV&V to traditional function-based software systems
However the emerging use of OO brings unique challenges and risks
There is a need to develop IV&V techniques for addressing these risks
Verify these techniques
COMPANY PROPRIETARY4
APC0342
Objective
Understand risks unique to large OO systems
Establish a framework for identifying, evaluating, and implementing IV&V techniques for mitigating these risks
COMPANY PROPRIETARY5
APC0342
Multi -Year Goals
Identify Risks
Develop base set of IV&V Techniques
Apply & evaluate techniques
Focus additional research and match emerging techniques to prioritized needs
COMPANY PROPRIETARY6
APC0342
Approach
Phased Approach– Identify Risks, Develop Base set of Techniques, Evaluate Base set– Focused research to adapt techniques for a broader spectrum of OO
projects
Reasoning– Level of maturity of IV&V’s understanding of the problem, developers
processes, and V&V techniques specific to OO system development– Limited availability of defect data specific to OO development
COMPANY PROPRIETARY7
APC0342
Focused research to map, adapt, or develop techniques to prioritized IV&V activities based on experienced and perceived needs for improvement in efficiency, accuracy, and capacity.
1) Identify OO Development Risks
2) Develop Base IV&V Techniques
– Requirements– Analysis– Design
3) Evaluate Base IV&V Techniques
Continuous research &
evaluation of em
erging techniques
Current Phase Next Phase
COMPANY PROPRIETARY8
APC0342
Activities to-date
Evaluation of current & emerging OO IV&V techniques
Identification of Risks with OO development
Investigating framework for OO IV&V
Developing base set of IV&V techniques
COMPANY PROPRIETARY9
APC0342
State of IV&V Practice
Rudimentary manual techniques for Use cases or OO analysis
Techniques limited to static analysis
No automated tools or metrics being used for requirements, analysis & design
Manual techniques ensure • Specifications use UML compliant notation • Design is adequately documented• Design specifications will meet requirements
Techniques do not address the quality of OO design objectives• Encapsulation, coupling, cohesion, polymorphism etc.
COMPANY PROPRIETARY10
APC0342
Emerging Techniques
Some examples of emerging techniques– Dynamic Analysis Techniques e.g. Yacoub et. al. "Dynamic Metrics for
Object Oriented Designs"– Semantic Metrics e.g. Etzkorn et. al. “Towards a Semantic Metrics Suite
for OOD”– Application of BBN e.g. Fenton et.el. “Bayesian Belief Network Model for
the Safety Assessment of Nuclear Computer-Based Systems”
State of practice– These techniques offer exciting promise for improving IV&V products– Need further enhancement for practical IV&V application
COMPANY PROPRIETARY11
APC0342
Risks to OO Development
Risk to OO development fall into three major categories– Programmatic Risks
• Project termination, software component integration, utilization of tools, and project failure
– Technology Risks• Issues with technology used to build the product, such as: Operating
system, Compilers, Libraries, and middleware – Methodology Risks
• Type of OO methods being used, incorrect application of a method, and issues related to lifecycle phases
COMPANY PROPRIETARY12
APC0342
Programmatic Risks
Project termination – Sub par schedule and performance of product
Methods abandonment– Lack of progress or OO related benefits
High learning curves – Insufficient budget and schedule for training
Not meeting OO objectives– Autonomy between organizations, absence of short iterative development cycle
Dependencies on Tools– Cost and tool deployment requirements
COMPANY PROPRIETARY13
APC0342
Technology Risks
Object Model – Technology does not fully support the object model e.g. asynchronous
communication between objects in relational databases
Persistence– Poor support for object persistence e.g. OODB, relational DB, ad-hoc flat
files
COMPANY PROPRIETARY14
APC0342
Methodology Risks
Incorrect application of methodology, insufficient methodology, or no methodology– Design mixed with requirements/use cases , insufficient information in
design models
Software performance – Language/hardware combination fails to supports performance
High defect level – System usability, reliability, and safety
COMPANY PROPRIETARY15
APC0342
GQM Approach
Identify software product metrics that can support analysis goals
For Use Case diagrams– GOALS- completeness, traceability, correctness– Questions-How to achieve the goals– Metrics-use case metrics such as by Marchesi
Later plans for application to Class diagrams
COMPANY PROPRIETARY16
APC0342
BBN Approach
For Use Case diagrams– Use of BBN to determine readiness for project transition into the next
lifecycle phase– Actors, actions, communication association, as part of network topology– Work is on going for determining characteristics of nodes, e.g. weight,
child and parent levels– CPT tables
Assumption: Use Cases completely expanded
COMPANY PROPRIETARY17
APC0342
Near term Goals
Develop base set of IV&V techniques for– Requirements phase products assuming Use Case Artifacts.– Analysis phase products assuming class and sequence diagrams.– Design phase products assuming class, sequence, and deployment diagrams
and state charts.
Solidify feasibility of applying – GQM for Requirements and Analysis Phases– BBN to gauge readiness for lifecycle phase transition