CCSM Software Engineering Coordination Plan

Tony Craig

SEWG Meeting

Feb 14-15, 2002

NCAR

Outline

• Background• New (and improved) policies and procedures

– Change Review Boards– Repository Access– Testing– Status Accounting– Documents– Planning and Prioritization– Management– Other Issues

• Roles• Summary

Acronyms

• CCSM - Community Climate System Model• CGD - An NCAR division• SSC - CCSM Scientific Steering Committee• SECP - Software Engineering Coordination Plan• CSEG - CCSM Software Engineering Group• ESMF - Earth System Model Framework, a NASA

funded project• SCIDAC - Scientific Discovery through Advanced

Computing, a DOE funded project• CVS - A source code control tool• CRB - Change Review Board

Background

• CCSM started as an NCAR centric project with a handful of developers and scientists. Little “management” was required.

• There has been rapid growth in the number of developers, number of collaborators, and importance of the work.

• CCSM coordination procedures are behind considering the growth and size of the project. Now CCSM needs to catch up.

Overall Goals

• Improve quality.• Improve coordination, communication, and control.• Improve productivity.• Continue to allow “individual” development.• Recognize that CCSM is a community project.• Minimize overhead and burden or “process”.

New Policies and Procedures

• Change Review Boards• Repository Access• Testing• Status Accounting• Documents• Planning and Prioritization• Management• Other Issues

New Policies and Procedures, the picture

Developer

REPO

Testing

ChangeReviewBoard

Releases

REPO Tools

Status Accounting

CVS

TestingScripts

BUG ReportingAnd Tracking

ExperimentDatabase

GUI

Documents, WEB

Change Review Boards (CRB)

• Goals– Improve communication and coordination of both

scientific and software tasks.– Better control of production versions.– Better prioritization of development tasks.– Improve quality by implementing reviews.– Formally distinguish tasks that are under

development and tasks that are ready for production.

Change Review Boards

• How will they operate– CRB for each component and full CCSM.– Developer would submit change request.– CRB (approx 4 people) is responsible for

prioritizing model changes.– CRB asks community for input on change request.– Reviewed for “improvement”, performance, testing

status, coding quality, and documentation.– CRB accepts, rejects, defers, or returns request.

Change Review Boards

• Implementation status– Developing procedures for atmosphere CRB,

meetings to start soon. – Developing procedures for CCSM CRB, meetings

to start soon.– Evaluating need for land, ocean, and ice

component CRBs.– Need to choose/develop status accounting tool.

Developing requirements.– Have implemented changes in repository policy to

create consistency between repository and CRB.

Repository Access

• Goals– Open CCSM repository to more developers.– Formalize use policy and procedures.– Improve coordination and control of development

efforts.– Clearly separate development tasks in the

repository from production versions of the models.– Have all development occur on branches. – Place some limitations on individual access, and

have an ability to enforce limitations.

Repository Access

• How will this work– Continue to use CVS.– Developers formally request access to repository.

Access granted on a limited basis, read/write permissions specified.

– All development will take place on CVS branches. Gatekeepers manage main trunk.

– Developers provide regular progress updates to CRBs or working groups.

– CCSM will have tools to monitor usage. Managers will retract access privileges if necessary.

Repository Access

• Implementation Status– Repository access policy exists.– Repository access request web page exists.– Need to review current developers’ status.– Need to clarify the review process for granting

access.– Need to improve tools to monitor repository usage.– Need to develop procedure for coordination of

development, including testing requirements.

Testing

• Goals– Improve testing capabilities, ease of testing.– Provide developers with model requirements, test

requirements, test scripts.– Develop an overall CCSM testing strategy to

include nightly build and smoke tests, unit tests, regression testing, port validation capabilities, and error growth validation.

– Discuss where scientific validation fits into overall testing strategy.

Testing

• Implementation Status– Atm model has some test scripts, being used by

the development community.– CCSM coupled model has some test scripts.– Simple GUI under development, used in-house for

configuring and testing the coupled model.– Need to develop model requirements.– Hiring a new person to focus full-time on testing

issues (SCIDAC funds).– Need to develop test strategy and hierarchy of

scripts for components and fully coupled models.

Status Accounting

• Goals– Track change requests and project priorities.– Improve bug reporting and tracking capability.– Create an experiment/dataset database.– Improve communication of status of model

releases and development versions.– Link many aspects of development; CRB,

repository, bugs, testing, web, and experiment database.

Status Accounting

• Implementation Status– ccsm-bugs request/tracking tool exists.– Implementing ccsm-logs experiment tracking log.– Need to develop “tool” for tracking change

requests for components and CCSM.– Need tools to monitor repository status.– Hiring a new software engineer to help us develop

our tools and infrastructure (SCIDAC funds).

Documents

• Goals– Improve communication and coordination.– Reduce the amount of confusion and rework.

Documents

• Implementation Status– Many documents exists, not kept up to date.– Need to develop strategy for sharing, using, and

updating documents; version control.– Need to implement/improve documents for

• production (users guides)• development (charters, requirements, design, and

reference documents; testing procedures, CVS help)• management (policies, procedures, plans)

Planning and Prioritization

• Goals– Improve communication.– Clarify priorities.

Planning and Prioritization

• Implementation Status– Limited number of planning documents exist.– Developing requirements documents and task

lists.– Change review boards and associated accounting

tools will be used to communicate some short/medium term priorities.

– Need to develop long term planning process to regularly review issues like base code rewrites, hardware issues, and the status of infrastructure support.

Management

• Goals– Clarify organizational structure.– Put in place procedures that improve coordination

and communication without burdening the project.– Need to strengthen decision capabilities.

Management

• Implementation Status– Some management in place; CCSM manager

(Kiehl), CCSM liason (Merilees), and CSEG manager (Craig). Role of CSEG becoming better defined.

– Writing charters for components and CCSM.– Need to clarify relationship between working

groups, CRBs, CSEG, SSC, and CGD sections.

Other Issues

• Data Management.• Code Review.• Library/Utility implementation plan.• Coordination with SCIDAC and ESMF.

– Providing funds for three (3) new Software Engineers in CSEG.

• Computer scientist in CCSM/CSEG.• Training.

Roles

• CSEG to take a leading role in developing the infrastructure tools and coordinating the day-to-day process.

• Component working groups to help manage repository access, change review, development, planning, and documentation of component models.

• SEWG will continue to oversee the software engineering process and make recommendations.

Summary

• New policies and procedures– Change Review Boards– Repository Access– Testing– Status Accounting– Documents– Planning and Prioritization– Management– Data, Utilities, Training

New Policies and Procedures, the picture

Developer

REPO

Testing

ChangeReviewBoard

Releases

REPO Tools

Status Accounting

CVS

TestingScripts

BUG ReportingAnd Tracking

ExperimentDatabase

GUI

Documents, WEB

50%

10%

40%

10%80%

80%

0%

On The WEB

• www.ccsm.ucar.edu - CCSM web page• www.cgd.ucar.edu/~csm/ - CCSM Developers web

page (csm/rio)– SECP page– Repository access policy– Repository access request page– Bug reporting and tracking– More!

THE END

CSEG Responsibilities

• Component development, performance..• Component model gatekeepers; may have to help

coordinate testing, development, change requests, repository status, etc.

• Testing; one (1) full time test engineer for CCSM plus everyone helps improve test scripts and automation.

• Infrastructure support and tools; one (1) full time tools support person for CCSM plus others in CSEG as needed.