Engineering Process Improvement in Heterogeneous …qse.ifs.tuwien.ac.at/wp-content/uploads/2014_EUROSPI_RS.pdf · Defects and root causes are hard to ... Context Definition and Study

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Institute of Software Technology and Interactive Systems

Engineering Process Improvement in Heterogeneous Multi-Disciplinary Environments

with Defect Causal Analysis

Olga Kovalenko1 Dietmar Winkler1 Marcos Kalinowski2Estefania Serral3 Stefan Biffl1

1TU Vienna, Institute of Software Technology, CDL-Flex, Austria2Federal University of Juiz de Fora, Brazil

3KU Leuven, Belgium

http://cdl.ifs.tuwien.ac.at


Motivation & Goals

Motivation: Heterogeneous and Multi-Disciplinary Engineering (ME)

Environments. Defects and root causes are hard to find

(even across disciplines)

Key research questions focus on: How to support stakeholders in efficiently

find root causes of defects for future defect prevention? How to implement an improvement strategy with

the defect causal analysis (DCA) approach?

Goals of the paper: Adapted DCA Approach & Evaluation Improvement strategy with DCA

Software Engineer

Process Engineer

Electrical Engineer

Mechanical & process properties

Electrical properties, e.g., Transmission lines.

Software (SW) properties, e.g., Logical Behavior

Risks & Quality Issuesin overlapping areas

1c

1a

1b

2

Pipe &

Instru

mentat

ion

Circuit

Diagram

Functi

on

Block

mech.models

elect. models

SWmodels

2


Engineering Process Data in ME Projects

Goals in ME Projects:– Consistent and stable engineering data in related disciplines.– Early defect detection and repair.– Defect prevention for future projects based on defect causes.

Traditional and Sequential Engineering Process (derived from our industry partner)

(Manual) Synchronization in Multi-Disciplinary Engineering Environments

Electrical Engineering

Software Engineering

Mechanical Engineering

Dis

cipl

ine-

Spec

ific

Engi

neer

ing

Synchronized Project Data

Defects Repair Sy

nchr

oniz

atio

n

Heterogeneous Data

Identified Defects

!

Consistency CheckingChange Propagation

Defects Detection

3


Defect Causal Analysis (DCA)

DCA has been successfully applied in Software Engineering*.

Select problem sample

Classify Defects

Identify systematic

errors

Determine main

causes

Develop action

proposals

Document meeting results

1 2 3 4 5 6

*Kalinowski M., Card D.N., Travassos G.H.: “Evidence-based guidelines to defect causal analysis”, IEEE Software 29(4), pp16-18, 2012.

Systematic Process Improvement based on DCA– Expert Workshop involving different stakeholder.– Starting Point: Identified (critical) defects.– Goal: Identifying root causes for defect repair and

prevention.

Defect Classification Schemes Set of attributes that describe a defect, e.g., defined by IEEE, IBM, or HP. Focus on Software Engineering need for extension for multi-disciplinary

and heterogeneous engineering projects.

4


Research Questions & Solution Approach

Research questions include How to adapt the DCA process to the context of ME projects? How to adapt a defect classification (DC) scheme to the context of ME projects?

Solution Approach Adaptation of the DCA process approach and … … extension of the defect classification scheme … … to address multi-disciplinary engineering projects in heterogeneous environments.

Feasibility Study Initial feasibility study at our industry partner to address most critical root causes based

on DCA findings. Implementation of improvement actions to address root causes. Second study for evaluation of implemented measures.

5


Adapted DCA Process

Basic Process Steps are similar!

Select problem sample

Classify Defects

Identify systematic

errors

Determine main

causes

Develop action

proposals

Document meeting results

1 2 3 4 5 6

A C B C A B C A C

Communication



Systematic Error


Tools

Method/ProcessInputOrganization

People

Adaptation focus on characteristics of multi-disciplinary engineering projects: Different (involved) engineering disciplines (A). Heterogeneous artifacts and data (B). Inter-disciplinary dependencies in project data (C).

Adapted Ishikawa diagram*:

*Ishikawa K.: “Guide to Quality Control”, Asian Productivity Organization Press, 1986.6


Defect Classification Scheme

Defect Classification (DC) scheme for DCA must cover*: Defect insertion to identify the cause of the defect (1). Defect detection to identify a strategy for defect detection method improvement (2). Defect type (i.e., nature of defect) supporting information for both (1) and (2).

Insertion Context• Discipline• Artifact Type• Artifact• Activity• Project Phase

* Kalinowski M., Card D.N., Travassos G.H.: “Evidence-based guidelines to defect causal analysis”, IEEE Software 29(4), 2012.

Detection Context• Discipline• Artifact Type• Artifact• Activity• Project Phase

Impact

Rating• Priority to fix• Severity (risk)

Current Status within Defect Life Cycle

Defect Type

Defect Mode

Adapted DC scheme based on IEEE consists of 7 attributes:

Different (involved) engineering disciplines (A)Heterogeneous artifacts and data (B)Inter-disciplinary dependencies in project data (C)

7


Feasibility Study

Study Process Summary Feasibility study in 4 steps to evaluate the adapted DCA process.

Context Definition and Study Planning (1)

DCA Workshops (2) and (4) Involving key stakeholders from our industry partner Focus on the most critical defects.

Lessons Learned: Identification of an improvement strategy (3)

Context Definition

Initial DCA Application

Improvement Implementation

DCA Application(Verification)

1 2 3 4

8


Context DefinitionFeasibility Study

Context Automation Systems Development Projects,

e.g., Hydro Power Plants. Involvement of various disciplines, e.g., mechanical, electrical,

and software engineering. Isolated tools and data models are not or loosely connected.

9


Core Defects and Errors Inconsistent Project and Engineering Data. Occur in the End-To-End Test.

Wiring

S2S3

S1

Sensors I/O

Physical Topology PLC Code

VariablesD1D2

Communication



variable type does not correspond to sensor type


Tools

Input

Method

Poor testing due to tight time frames

PeopleProgrammer

mistake

Physical topology is out of datePeople

Engineer forgot to notify about changes

Problem in data export

Method

No automated change notification

No automated change propagation

Root Cause Analysis Lack in interoperability of data models and data. No automation supported change propagation/notification

Initial DCA ApplicationFeasibility Study

Proposed Solution (Lessons Learned) Tool-Supported synchronization based on semantic technologies with the ASB*.

*Automation Service Bus (http://cdl.ifs.tuwien.ac.at)10


Improvement and Second DCA ApplicationFeasibility Study

However … (the results of the second DCA application) Data Model Transformation must be stable and correct. Incorrect transformation rules might lead to (systematic) mapping errors but

they are easier to handle.

Implemented Improvements Synchronization of heterogeneous

data models based on common concepts.

Automated Change propagation / notification consistent project and engineering data.

Systematic Error (identified in a second DCA cycle) Inconsistent engineering data due to incorrect transformations (model transformation

errors)

Improvement Options Additional Quality Assurance Step to verify/validate model transformation.

11


Summary & Future Work

Summary Multi-Disciplinary Engineering Projects include additional risks due to distributed and

heterogeneous data models that have to be synchronized manually DCA enables the identification of root causes of a certain set of defects systematically. However DCA and Defect Classification approaches, applied in Software Engineering

must be adapted to meet ME projects. A sequence of DCA applications can lead to an improvement strategy applicable in ME

domains.

Future Work Towards tool supported DCA. Large-Scale application and

Case Study in industry environment.

Manual Synchronization

Initial DCA

Inconsistent Project Data

Manual SyncHuman errors(unsystematic)

ASB application

Second DCA

Inconsistent Project Data

Invalid Mapping (Systematic)

Quality Assured ASB

Error

DifferentRoot Causes

QA

Consistent Project Data

Pro

duct

Qua

lity

12


Thank you ...

Engineering Process Improvement in Heterogeneous Multi-Disciplinary Environments with Defect Causal Analysis

Olga Kovalenko1, Dietmar Winkler1, Marcos Kalinowski2, Estefania Serral3, Stefan Biffl1

1TU Vienna, Institute of Software Technology, CDL-Flex, Austria2Federal University of Juiz de Fora, Brazil

3KU Leuven, Belgium

[email protected]

Documents

Engineering Process Improvement in Heterogeneous …qse.ifs.tuwien.ac.at/wp-content/uploads/2014_EUROSPI_RS.pdf · Defects and root causes are hard to ... Context Definition and Study