Requirements quality analysis - A successful case study in the Railway Industry

Requirements Quality Analysis:

A successful case study in the Railway industry

CSD&M’16

Paris. December 13-14, 2016

José Fuentes (The REUSE Company)

Hugo-Guillermo Chalé-Góngora (Alstom)

2

Contents

Introduction

The impact of poor quality in our projects

Requirements Quality Analysis

Practical case study

Goals, inputs and expected outputs

Tools benchmark

The PoC Process

PoC results

Requirements Quality Analysis - A successful case study

3

The impact of poor quality

4

The impact of poor quality projects


5

Why Requirements Quality Analysis?


Doing the right thing right (verification)

http://www.theguardian.com/world/2014/may/21/french-railway-operator-sncf-orders-trains-too-big

http://elpais.com/elpais/2015/02/04/inenglish/1423052376_326956.html

6

Requirements Quality

Analysis

FOCUS

7

Why focusing on requirements quality: source of defects

Project Success Factors % of Responses1. User Involvement 15.9%

2. Executive Management Support 13.9%

3. Clear Statement of Requirements 13.0%

4. Proper Planning 9.6%

5. Realistic Expectations 8.2%

6. Smaller Project Milestones 7.7%

7. Competent Staff 7.2%

8. Ownership 5.3%

9. Clear Vision & Objectives 2.9%

10. Hard-Working, Focused Staff 2.4%

Other 13.9%

+40% directly related with

requirements definition and

management

15.9%

13.0%

8.2 %

2.9 %

Based on requirements

(Source: CHAOS Report, 2004)


8

Why Requirements Quality Analysis?


Source : INCOSE SE Handbook V4.0. 2015

95%

85%

70%

Time

Cu

mu

lati

ve p

erce

nta

ge

Life

cylc

e C

ost

Operations through Disposal

100%Production

and test

50%

8%Design

15% 20%Concept

Commited Costs

3-6x

500-1000x

20-100x

Development

9

Systems and Requirements Engineering life-cycles


CONOPS

Stakeholders

Requirements

System

Requirements

System

Design

Equipment

Requirements

Equipment

Design Equipment

Verification

System

Equipment

System Verification

Product

Product Verification

Requirements

Verification

Requirements

Verification

Requirements

Verification

Design

ValidationDesign

Verification

Requirements

Validation

http://www.google.es/url?sa=i&source=images&cd=&ved=0CAgQjRw&url=http://ebgconsulting.com/blog/category/ellen-gottesdiener/&ei=Hik1VcG_EIbJPOSJgaAG&psig=AFQjCNEdp91ciOQP6sbSpD1wMk-n6OcdhQ&ust=1429633694334726

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://www.diyalarms.net/system_design.html&ei=ZSs1VYPAFcnbPI63gNAP&psig=AFQjCNFLB94RGp-VWAc8TdxFV1KcmIhbag&ust=1429634277421348

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://globe-views.com/dreams/aircraft.html&ei=XzI1Ve2DNIH7PNjEgZAO&psig=AFQjCNFgZ_GnAffKepduZEvVNoO4MPTNOQ&ust=1429636063927303

http://www.google.es/url?sa=i&source=images&cd=&cad=rja&uact=8&ved=0CAgQjRw&url=http://en.wikipedia.org/wiki/Aircraft&ei=nzI1VYexFYeqOvapgPgB&psig=AFQjCNHphBW9uKFpY7zGF1lb4lZURvPDjw&ust=1429636127467650

10

Systems and Requirements Engineering life-cycles


Elicitation Analysis Specification Validation

close gapsclarify

rewrite

re-evaluate

confirm and correct

Source: Karl Wiegers

11

Practical Application at

12

Quick Proof of Concept on Requirements Quality Improvement

OBJECTIVES

Reduce the impact of poor quality in our projects

Perform correctness, completeness and consistency analyses of requirements (individually and

collectively) to improve the quality of requirements specifications

Assess the computer-aided requirements authoring feature to accelerate the learning curve of new

practitioners (or improve the capability of current practitioners) in requirements development

GOAL

Exonerate engineers from format concerns (structure) and allow them to concentrate on content

(essence of requirements): technical data useful for design

Relieve engineers from repetitive and error-prone tasks

Allow time for collaboration and creativity

Improve the common understanding of the problem


13

Also, a mean to improve current practices

External audits results

“… Requirements Characterization is not complete: Derived/uncovered requirements justification,

Contribution, Categories (technical vs non-technical), V&V Methods…

…V&V Plan is not complete: Verification activities, or agreed alternate practices (waivers) and

associated deliverables…”

CMMI for Development

Requirement Development process area – SG 3 Analyze and Validate Requirements

“… Analyze requirements to determine whether they satisfy the of higher level requirements.

Analyze requirements to ensure that they are complete, feasible, realizable, and verifiable…”

Verification process area – SG 2 Perform Peer Reviews

“… Establish and maintain checklists to ensure that the work products are reviewed

consistently...

Rules of construction , Completeness, Correctness…”


14

Requirements Quality Analysis tool - Benchmark

Tool Acronym Vendor

DESIRe DES HOOD

Lexior LEX Cortim

Qualicen QCEN Qualicen

QVscribe QVS QRA

Requirements Assistant RA Sirius Requirements

Requirements Quality Suite RQS The REUSE Company

Semios SEM Prometil


15

RQS – Requirements Quality Suite

The Requirements Quality Suite (RQS) intends to tackle requirements quality management by offering a set of tools and processes

Automatic measurement of requirements quality metric

Support to Requirements Authoring

RQS models requirements quality metrics using the CCC approach (Correctness, Consistency and Completeness)

Requirements Quality Analyzer (RQA): to setup, check and manage the quality of a requirements specification

Requirement Authoring Tool (RAT): to assist authors while they are creating or editing requirements

Knowledge Manager (KM): to manage knowledge around a requirements specification: the ontology it is based on, the structure of the requirements to be used in the project, the communication between authors and domain architects


https://www.google.es/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&docid=QsDpGlkai9R4FM&tbnid=ra3RwYKyt1hIhM:&ved=0CAUQjRw&url=https://help.rallydev.com/ibm-doors&ei=OMPhU8L-Jais0QX-iIG4Dg&bvm=bv.72197243,d.d2k&psig=AFQjCNHbwoatil1oOCZbKzYlsM5nrV1LDA&ust=1407390896885195

16

Requirements quality metrics: CCC Approach

CCC – Correctness, Consistency and Completeness


Correctness

Consistency

Requirements quality

Completeness

Individual requirements

Sets of requirementsSets of requirements

../../../../../Projects - Repository/Requirements Quality Management V2014/Book/Chapter 12 - Requirements Quality Metrics/INCOSE Writing Requirements Guide-ver4-r2ce.pdf

17

One-Week Workshop

Requirements Quality

Organizational Knowledge

Analysis

RQS Tools Training

One-Week Use Case definition

Documents Analysis: Standards, Glossaries and

System Documents

Quality Rules Definition

Two-Week Requirements Quality

Analysis

Requirements Quality Analysis

Requirements Improvement:

Rephrasing

Two-Week Knowledge Base

Elaboration

Reuse Organizational

Know-How

Writing Good Requirements for Alstom: Patterns

and Rules

Week 1 Week 2 Week 3 Week 4 Week 5 Week 6

2 Alstom Engineers

2 TRC Consultant

2 Alstom Engineers

2 TRC Consultant

2 Alstom Engineers

2 TRC Consultant

2 Alstom Engineers

2 TRC Consultant


Proof of Concepts: Work Methodology

18

Final QualityResults


Quality MetricsAnd

Knowledge

AlstomKnowledge Base

V1

Alstom OriginalReqs. Specification


INCOSE + TRCQuality Results

Default Quality AssessmentINCOSE

TRCISO29148

Quality Assessment withAlstom White Belt

OrganizationKB V1

AlstomQuality Results


SPECIFICATIONUPDATE

AlstomQuality Results

OrganizationKB V1

AlstomKnowledge Base

V2


Alstom Improved Reqs.

Specification

Proof of Concepts: Work Process

19

Conclusion and main achievements


http://www.reusecompany.com

@ReuseCompany

[email protected]

Margarita Salas, 16 2nd Floor

Innovation Center

LEGATEC Technology Park

28919 Leganés – Madrid

SPAIN – EU

Tel: (+34) 912 17 25 96

Fax: (+34) 916 80 98 26

http://www.reusecompany.com/

https://twitter.com/ReuseCompany

mailto:[email protected]