Activity 6Trustworthy Process

Author: Etiel Petrinja (Free University of Bolzano, Italy)Date: 27th October 2010Location: Tokyo

Allow companies to use FLOSS to build their mainstream

products and services

Activity Overview - Vision

Definition of a CMMI-like model for FLOSS to improve

the trust in FLOSS software

OMM

QualiPSo Activity 6 1/16

A6 Workplan plan

• WP6.1: (M1-M24) Analysis of the clusters of Open Source processes in OS Communities and in companies

• WP6.2: (M7-M24) Identification of the trustworthiness elements in the processes

• WP6.3: (M12-M36) Definition of a CMM-like model for OSS systems development

• WP6.4: (M19-M42) Identification, specification, and development of tools to support the CMM-like model for OSS system development

• WP6.5: (M25-M48) Experimentation support• WP6.6: (M30-M48) Training material

WP6.3 - Model

WP6.3:– Definition of the latest version of OMM 1.1

– Refining the model's structure

– Improvement of the assessment process and the rating mechanism

– Conducting OMM assessments

– Establishing the maintenance process and discussing about the future evolution of the OMM model

QualiPSo Activity 6 3/16

WP6.3 - Refinement of OMM elements

• FLOSS specific elements (TWE); input from

WP1 and WP2

• CMMI specific elements

(TWE)

• Decisions of which

element belongs to

which maturity level

QualiPSo Activity 6 4/16

WP6.3 – Creation of the questionnaires

• Simplification of the structure of tables

• Improvement of the questions

• Integration of the GQM and the CMMI tables

QualiPSo Activity 6 5/16

WP6.3 – Creating Assessment Questionnaires

• Definition of 25 tables containing all elements necessary for the complete assessment of the FLOSS development process

8/16

WP6.3 – Definition of the Structure

• Renaming Questions

to Practices for

compatibility with CMMI

• Definition of the

implementation priority (n. 4)

• Distribution of metrics on two

levels (n. 3)

• Creation of LookFors (n. 1)

• Practices are the key building

blocks of the Model

QualiPSo Activity 6

Level

Implementation Priority

TWE 1

TWE n

Goal 1

Practice n.1

Practice n.m

Metric 1

LookFor X

Metric 2

- Companies- Communities - mandatory - recommended - not important

44

2

3

1

Constructing Element of the OMM

Legend:

Contains

Structure of the OMM-model

WP6.3 – Identification of use cases

• The OMM process model is expected to be– useful for the following users:

– Companies that adopt/develop FLOSS projects

– Companies that integrate FLOSS into their products

– FLOSS community practitioners, such as developers (analysts, programmers, testers) and FLOSS project managers

• The main goals, the OMM users can reach are:– Improving their development processes following OMM practices

– Development of a project by following OMM

– Evaluation of a project hosted externally to the QualiPSo Factory

– Evaluation of a project hosted internally to the QualiPSo Factory

QualiPSo Activity 6 7/16

WP6.4 - Tools

WP6.4 progress:– Further improvement of the Spago4Q tool

– Improvement of the OMMQuest tool

– Finalizing the connection of the OMMQuest tool with Spago4Q

– Further development of a set of extracting tools (Bicho, CVSAnaly, MailingListStats)

– Connecting external tools (OSCL, Jupiter, Junit, Xplaner)

– Update of the OMM web site, with detailed information about the last version of the model

QualiPSo Activity 6 10/16

WP6.4 – Architeture overview

11/16

Source Forge QualiPSo Forge

Aggregation&

Visualization

Spago4QOMMQuest

Tool

Potential OMM users

Competence Centres

FLOSS Integrators

FLOSS Communities

FLOSS Users

OMM On-line Documentation

OSLC

Bicho

CVSAnalySemi Automatic measurement tools

Assessments repository

FLOSS Projects

JUnit

Bicho

CVSAnaly

MailStats

RE

ST

interface

Spago4Q

WS

WS

WS

PDF File

Simple textSimple numeric valuesGraphic elements

WS

XML file

XML file

XML file

Automatic tools connection - WP6.4

WP6.4 – OMM Web Documentation

• Development of web content describing the whole OMM

model, its potential users, and use cases.

• Available here: http://qualipso.icmc.usp.br/OMM/

QualiPSo Activity 6

WP6.5 Validation

WP6.5 progress during the last year:– Definition of experiments/pilots of the model and the tools

– Identification of potential subjects, tasks, and environments

– Definition of surveys validating the model

– Conducting initial experiments/pilots

– Testing the model by individuals involved in Competence Centres

QualiPSo Activity 6 13/16

WP6.5 – First Experimentation results

• Comparing OpenBRR, QSOS, and OMM models

OMM

Firefox Chrome

Mean Mean

Product Documentation (PDOC) ,3 9 ,0 0 ,3 5 ,0 1

Popularity of the SW Product (REP) ,2 7 ,0 1 ,2 4 ,0 4

,3 8 ,0 3 ,3 1 ,0 7

,3 1 ,0 5 ,2 3 ,0 4

Quality of the Test Plan (QTP) ,4 2 ,0 6 ,3 3 ,0 8

Relationship between Stakeholders (STK) ,4 1 ,0 1 ,2 9 ,0 1

Licenses (LCS) ,3 4 ,0 2 ,2 7 ,0 5

Technical Environment (ENV) ,3 9 ,0 7 ,2 8 ,0 8

,3 5 ,0 1 ,3 5 ,0 0

Maintainability and Stability (MST) ,3 8 ,0 4 ,2 7 ,0 4

,3 2 ,0 1 ,2 7 ,0 3

,1 8 ,0 3 ,2 1 ,0 1

Standard deviation

Standard deviation

Use of established and Widespread Standards (STD)

Availability and Use of a Roadmap (RDMP)

Number of Commits and Bug Reports (DFCT)

Contributions to the FLOSS project from SW Companies (CONT)

Results of Assessment of the Product by 3rd Party Companies (RASM)

Strongly disagree

Disagree

Agree

Strongly agree

% 25% 50% 75% 100%

The process areas in the assessment method cover all significant software development process requirements

OpenBRRQSOSOMM

QualiPSo Activity 6 15/16

Project reports

Detailed report

RDMP: Roadmap

,3 33Purpose: Create and maintain product roadmap

Goal/Practice Goal/Practice description

Plan product roadmap ,3 33

Define responsibility for the roadmap 4

Define responsibility of developers involved in the definition and management of the roadmap 4

Ensure that the roadmap includes plans for at least the next two major versions / releases 3

Details for different aspects of development (bugs, issues, features, releases etc) 2

The roadmap is prepared for a period longer than one year 2

The roadmap contains a detailed description of new features 3

The roadmap contains a release date for new features 3

4

The roadmap contains a version plan for the product 3

Ensure different types of releases (minor, major, etc) 3

Check the frequency of major releases 3

Check the frequency of minor releases (bug fixes) 3

Goal RDMP 1Practice RDMP-1.1

Practice RDMP-1.2

The roadmap elements are well described indicating also their importance for the future version of the product

Practice RDMP-1.3

Overview report

Dettailed report

Agregated results

PDOC STD QTP LCS ENV DFCT MST CM PP1 REQM RDMP

1

2

3

4

2,11

3,38

2,6

3,78

2,83

2

3,17

3,433,22

3,53,33

OMM Basic levelTrustworthy elements assessment

Trustworthy elements

Ass

ess

ed

va

lue

PDOC STD QTP LCS ENV DFCT MST CM PP1 REQM RDMP

0

2

4

6

8

10

12

14

33%

13%

20%83%

50%

57%33%

50%

67%

44%

25% 60%

11%

17%

100%

17%

22%22%

63%

20%

89%

33%43%

44%

50%33%

OMM Basic levelPractices assessment value

4 3 2 1

Trustworthy elements

Ass

ess

me

nt v

alu

e o

f pra

ctic

es

Agregated results

RDMP2 STK PPQA PMC TST 1 DSN 1 PP2

1

2

3

4

1,33

3

2,67

3,63

3

3,25

3,6

OMM Intermediate levelTrustworthy elements assessment

Trustworthy elements

Ass

ess

ed

va

lue

RDMP2 STK PPQA PMC TST 1 DSN 1 PP2

0

2

4

6

8

10

12

14

33%

20%

33%

13%

25%

67%

13%

20%

75%

50%

25%

60%

60%

67%

13%

13%

75%

40%

OMM Intermediate levelPractices assessment value

4 3 2 1

Trustworthy elements

Ass

ess

me

nt v

elu

e o

f pra

ctic

es

CONT RASM REP PI DSN 2 RSKM TST2

1

2

3

4

1,67

2,22

3,2

3,56

2,86

3,6

OMM Advanced levelTrustworthy elements assessment

Trustworthy elements

Ass

ess

ed

va

lue

CONT RASM REP PI DSN 2 RSKM TST2

0

2

4

6

8

10

12

14

67%

11%

29%

33%

33%

100%

56%80%

44%

57%

40%

20%

56%

14%60%

OMM Advanced levelPractices assessment value

4 3 2 1

Trustworthy elements

Ass

ess

me

nt v

alu

e o

f pra

ctic

es

OMM Training material

Training availability– Collaboration with A11 for the creation of the first and

second version of the training material presenting the OMM model

QualiPSo Activity 6 16/16

QualiPSo Activity 6