5 Feb 2008

Roberta Allsman

LSST Corporation

LSST Data Management

DC2 Post-mortem

2

Software Quality ControlTransitioning from DC2 to DC3

Software Quality Control Introduction

DC2: The Way It Was

DC2: The Regrets and Promise of Change

DC2 Transition to DC3

3

Software Quality Control (SQC)

SQC

– Verifies thru reviews: deliverables satisfy standards

– Validates thru testing: deliverables satisfy requirements

Standards and Requirements include

• Product requirements specification (use case driven)

• Design document (UML)

• Coding, Documentation and Testing standards

4

DC2 SQC: The Way It Was

Requirements, Standards, Policies

– Project requirements and science use cases

– Design standards (DM UML model and DC2 specialization)

– Coding standards (language, best practices, layout)

– Documentation standards (in-line)

– Toolset policy (sanctioned support tools & nominations of)

– Product release policy

5

DC2 SQC: The Way It Was

Quality Control Procedures

– Change Control Board

• reviewed need and feasibility of feature additions

– Code Review Procedure

• Defined responsibilities of developer and code reviewer

• Specified task workflow

– Tasks were not forgotten and progressed to completion

• Specified first validation testing: unittest

6

DC2 SQC: The Way It Was

Support

– Software Developers Start-up Guide

• Training for new developers in setting up LSST DM installation

– Component Design Overviews

• Background implementation overview

– Trac

• web-based software project management and issue tracking system

7

DC2 SQC: The Way It Was

TRAC

WIKI Web-based Document Manager

– Easy to use

– Enabled issues to be recorded and debated

– Made project decisions/mandates readily accessible

Ticket Management

– Current state of each issue (aka ticket) readily available

– At-a-glance summary reports based on tickets attributes

Source Browser

– On-line browsing of source and its change history

8

DC2: The Regrets and Promise of Change

Standards

• Code Review Procedure defined a workflow which created a

bottleneck at point of design resynchronization with

implementation. A proposed DC3 workflow alleviates this.

• Unit test requirements have not been adequately documented.

• Uniform naming syntax for DB columns is missing.

Code Review

• As a group, code reviewers were inconsistent in application of

language and documentation requirements. Using an automated

standards checker would simplify review process.

9

DC2: The Regrets and Promise of Change

Trac WIKI

• Add new V0.11 blog feature to foster copy-cat of Becker's

popular 'how-did' blog

• WIKI's intrinsic flat hierachy leads to orphaned pages

– Concept: Modify 'New Page' function to require user to

enter page locator into Table-of-Contents page

SQC

• A testbed needs to be setup on each supported hardware

configuration for integration tests

10

DC2: The Regrets and Promise of Change

Source Management

• Remove superfluous named branch: BRANCHES

• UML/source sync is too manpower intensive

– Change source layout to be 'better' for EA

– Define uniform naming of swig generated file

– Setup EA user environment for consistent source location

Testing

• More formalized testing was needed

– Will be addressed in DC3 SQC Standards upgrade

11

DC2: The Regrets and Promise of Change

Trac Ticket Management

• More numerous DC2 milestones were needed

– Define finer grained milestones

– Accurately assign tickets to milestones

– Generate reports showing ticket/milestone progress

• Ticket workflow wasn't detailed enough

– New Trac V0.11 has flexible workflow definition

12

DC3 transition: Upgrade Standards Documents

Select/develop new use cases to define requirements

Complete/revise list of configurations to support

Develop Test Standards document

• Specify categories of testing required

• Specify which testing categories are required for

– No-guarantees release (TICKETS branch)

– Beta release (TRUNK branch)

– Production release (TAGS branch)

• Refine development lifecycle (aka workflow) to include new testing

13

DC3 transition: Upgrade SQC Procedures

Acquire configurable language and coding standards checker

• Examples: ProQA, CodeCheck, Apex, QStudio

Develop new workflow procedure document

• Develop procedures for each new testing category

Use Trac project management features more effectively

• Track ticket time to resolution

• Track milestone progress

• Assign priorities to tickets based on milestones & dependencies

• Expand ticket tracking workflow

14

DC3 transition: Preliminary DC3 Workflow

Change Review Board

Requirements review for new functionality

Task assignment

Implementation in TICKETS branch

Software development

Code review

Language/documentation/best practices review

Testing

Unit test, Component test

15

DC3 transition: Preliminary DC3 Workflow

Merge from TICKETS into TRUNK branch

Conflict resolution due to source merge

Testing

Unit test, Component test, Integration test

Copy from TRUNK into TAGS branch

Testing

End-to-End test, Platform test, Distribution test, Stress test

Synchronize UML design with implementation

Release of verified and validated product