5 Feb 2008
Roberta Allsman
LSST Corporation
LSST Data Management
DC2 Post-mortem
2
Software Quality ControlTransitioning from DC2 to DC3
Software Quality Control Introduction
DC2: The Way It Was
DC2: The Regrets and Promise of Change
DC2 Transition to DC3
3
Software Quality Control (SQC)
SQC
– Verifies thru reviews: deliverables satisfy standards
– Validates thru testing: deliverables satisfy requirements
Standards and Requirements include
• Product requirements specification (use case driven)
• Design document (UML)
• Coding, Documentation and Testing standards
4
DC2 SQC: The Way It Was
Requirements, Standards, Policies
– Project requirements and science use cases
– Design standards (DM UML model and DC2 specialization)
– Coding standards (language, best practices, layout)
– Documentation standards (in-line)
– Toolset policy (sanctioned support tools & nominations of)
– Product release policy
5
DC2 SQC: The Way It Was
Quality Control Procedures
– Change Control Board
• reviewed need and feasibility of feature additions
– Code Review Procedure
• Defined responsibilities of developer and code reviewer
• Specified task workflow
– Tasks were not forgotten and progressed to completion
• Specified first validation testing: unittest
6
DC2 SQC: The Way It Was
Support
– Software Developers Start-up Guide
• Training for new developers in setting up LSST DM installation
– Component Design Overviews
• Background implementation overview
– Trac
• web-based software project management and issue tracking system
7
DC2 SQC: The Way It Was
TRAC
WIKI Web-based Document Manager
– Easy to use
– Enabled issues to be recorded and debated
– Made project decisions/mandates readily accessible
Ticket Management
– Current state of each issue (aka ticket) readily available
– At-a-glance summary reports based on tickets attributes
Source Browser
– On-line browsing of source and its change history
8
DC2: The Regrets and Promise of Change
Standards
• Code Review Procedure defined a workflow which created a
bottleneck at point of design resynchronization with
implementation. A proposed DC3 workflow alleviates this.
• Unit test requirements have not been adequately documented.
• Uniform naming syntax for DB columns is missing.
Code Review
• As a group, code reviewers were inconsistent in application of
language and documentation requirements. Using an automated
standards checker would simplify review process.
9
DC2: The Regrets and Promise of Change
Trac WIKI
• Add new V0.11 blog feature to foster copy-cat of Becker's
popular 'how-did' blog
• WIKI's intrinsic flat hierachy leads to orphaned pages
– Concept: Modify 'New Page' function to require user to
enter page locator into Table-of-Contents page
SQC
• A testbed needs to be setup on each supported hardware
configuration for integration tests
10
DC2: The Regrets and Promise of Change
Source Management
• Remove superfluous named branch: BRANCHES
• UML/source sync is too manpower intensive
– Change source layout to be 'better' for EA
– Define uniform naming of swig generated file
– Setup EA user environment for consistent source location
Testing
• More formalized testing was needed
– Will be addressed in DC3 SQC Standards upgrade
11
DC2: The Regrets and Promise of Change
Trac Ticket Management
• More numerous DC2 milestones were needed
– Define finer grained milestones
– Accurately assign tickets to milestones
– Generate reports showing ticket/milestone progress
• Ticket workflow wasn't detailed enough
– New Trac V0.11 has flexible workflow definition
12
DC3 transition: Upgrade Standards Documents
Select/develop new use cases to define requirements
Complete/revise list of configurations to support
Develop Test Standards document
• Specify categories of testing required
• Specify which testing categories are required for
– No-guarantees release (TICKETS branch)
– Beta release (TRUNK branch)
– Production release (TAGS branch)
• Refine development lifecycle (aka workflow) to include new testing
13
DC3 transition: Upgrade SQC Procedures
Acquire configurable language and coding standards checker
• Examples: ProQA, CodeCheck, Apex, QStudio
Develop new workflow procedure document
• Develop procedures for each new testing category
Use Trac project management features more effectively
• Track ticket time to resolution
• Track milestone progress
• Assign priorities to tickets based on milestones & dependencies
• Expand ticket tracking workflow
14
DC3 transition: Preliminary DC3 Workflow
Change Review Board
Requirements review for new functionality
Task assignment
Implementation in TICKETS branch
Software development
Code review
Language/documentation/best practices review
Testing
Unit test, Component test
15
DC3 transition: Preliminary DC3 Workflow
Merge from TICKETS into TRUNK branch
Conflict resolution due to source merge
Testing
Unit test, Component test, Integration test
Copy from TRUNK into TAGS branch
Testing
End-to-End test, Platform test, Distribution test, Stress test
Synchronize UML design with implementation
Release of verified and validated product