sG
Configuration Configuration ManagementManagement
www.scmGalaxy.com
scmGalaxy
Author: Rajesh [email protected]
www.scmGalaxy.com
scmGalaxy
Agenda
Introduction to software configuration management (SCM), build and release (BnR)
Initiating CM activities Case study - 1.5 hours case study to help
participants get a complete view of a CM process in a project.
SCM tools comparison CVS – deep dive ANT – basics and advanced tasks Automated builds using cruise control
www.scmGalaxy.com
scmGalaxy
Agenda for session 1
Introduction to software configuration management (SCM), build and release (BnR) The need for SCM, BnR in a project lifecycle Objectives of a SCM BnR system Role of a configuration manager Typical activities of a configuration manager
Initiating CM activities Identifying the inventory of configurable items Defining a CM plan Defining a release traceability with tagging and CR numbers Build and build automation Introduction to environments, Tools used in CM
www.scmGalaxy.com
scmGalaxy
Objectives
Understand the role of a CM Understand the type of activities a CM is involved in How to initiate and administer CM activities in a
project Strategies for build, environments and automated
builds Appreciate what is the list of tasks of a CM and be
able to begin working as a CM
sG
Need for Need for Configuration Configuration ManagementManagement
www.scmGalaxy.com
scmGalaxy
The Rambo developer
John Rambo is a single developer of a mid sized project
A one man team taking care of requirements, design , development, testing and delivery
He has the whole source code on his machine and he overwrites his code each time a bug is fixed or a new change is brought in
Application is simple easily testable. Overwriting the same piece of code is a rare occurrence
Only his hard disk is backed up as a disaster recovery plan
www.scmGalaxy.com
scmGalaxy
Rambo developer and his woes
With increasing complexity the same parts of the code were getting overwritten
More often than not, business users would request a change in a feature and soon expect to go back to the old feature or expect a mix of old and new features
John soon finds himself with many requests to revert back to the old code
Often john finds himself re-writing parts of the same code because of his overwrites
www.scmGalaxy.com
scmGalaxy
1Guys, I heard you
have finally cracked the cluster problem!!
Congratulations !! What was it?
2It’s a silly mistake
boss!! An old version of the code was
over-written which did not serialize the
property files.
3How did the old
version get in there?4
Prakash was trying to debug some
problem last week from on-site and he fixed the problem in the old source and
pushed them!!
A disorganized team
www.scmGalaxy.com
scmGalaxy
The problems
What is missing in these projects?How would you solve it?
www.scmGalaxy.com
scmGalaxy
The problems and solutions
Currently there is no code changes being tracked.
Once a change is made the old code is over written, there is no way to get back to the old code
There is no release policy that defines what features need to be released
A code release is unable to show the user the specific set of features that have been released
A way to track changes and maintain versions of his code. A Version management tool is necessary
A release policy should be defined
A release should be done as a unit and not as a set of individual units
A release document that explains the changes released in each release should be generated
There should be a way to reproduce all parts of a release when ever required
sG
Role of a Role of a Configuration Configuration
managermanager
www.scmGalaxy.com
scmGalaxy
Definitions - wikipedia
Configuration Management - The control of changes, including the recording thereof, that are made to the hardware, software, firmware, and documentation throughout the system lifecycle
Software Configuration Management - set of activities designed to control change by identifying the work products that are likely to change, establishing relationships among them, defining mechanisms for managing different versions of these work products, controlling the changes imposed, and auditing and reporting on the changes made - pressman
SCM concerns itself with answering the question: somebody did something, how can one reproduce it?
www.scmGalaxy.com
scmGalaxy
Software Configuration Management
“A discipline applying technical and administrative direction and surveillance to: identify and document the functional and physical characteristics of a configuration item, and control changes to those characteristics, report change processing and implementation status, and verify compliance with specified requirements”
Source: IEEE
www.scmGalaxy.com
scmGalaxy
Activities involved
Management
& Planning
(SCMP)
Software Configuration Management
Software
Configuration
Identification
Software
Configuration
Control
Software
Configuration
Status
Accounting
Software
Configuration
Auditing
Software
Release Mgmt
& Delivery
www.scmGalaxy.com
scmGalaxy
Need for a Configuration Manager
With increasing complexity of a system one needs a dedicated person who can manage the changes to a large software unit
What is complexity of a software system Long running projects Large teams Distributed teams (distributed development) Multiple software units that form one large chunk Assets related to the code – scripts, test cases Assets not related to the code – documents, release notes …
www.scmGalaxy.com
scmGalaxy
What is a CM’s role
A configuration manager works with the project leadership to provide an environment that controls, tracks and reports changes made to any unit of information
Broadly speaking the various buckets of activities that a CM needs to own are Identification of configurable items Control the changes of these items Provide status and reports related to the changes to an asset Manage the build, packaging and release of the software units Provide a mechanism of traceability from the software units to
the root cause of a change or the primary requirements Audit the project and ensure that the CM process is being
followed
www.scmGalaxy.com
scmGalaxy
Role of CM at various stages
www.scmGalaxy.com
scmGalaxy
CM – stage 1 - Identify and establish
The first phase of the project
Along with the project initiation phase, the CM plan needs to be defined
Most of the tasks in this phase is with respect to definition of a CM plan and establishing the infrastructure for the project
Processes, templates, naming conventions etc. will need to be defined for the whole team to adopt in the time to come
www.scmGalaxy.com
scmGalaxy
CM – stage 1 - Identify and establish
Objective List of CI identified Directory structure
defined for all Tool for version
management Change control
process defined CMP defined,
reviewed and accepted
www.scmGalaxy.com
scmGalaxy
CM – stage 2 - Build and release definition
Design and development phases defines the code structure, the tools, the frameworks to be used in the project
CM defines a build strategy for the project
First cut build should ensure that it integrates well with the version management tool
Provide automation to obtain a single integrated unit of software that can be used for development and testing
Iterative process and prioritization is the key
www.scmGalaxy.com
scmGalaxy
CM – stage 2 - Build and release definition
Objectives Base line Build script Strategy to enhance
build script Integration of build
with version management
Strategy to release code
Ability to support various environments
www.scmGalaxy.com
scmGalaxy
CM – stage 3 - enhance and strategize
The development and testing phases bring with it new tools and changes to source code structure
The build script needs to evolve and automate as many tasks as possible
Towards the end of the development phase the need to begin a release process would have arrived
www.scmGalaxy.com
scmGalaxy
CM – stage 3 - enhance and strategize
The ability to tag a release and the ability to roll back a release should be available
In the near future code will move into a maintenance mode.
The strategy to support multiple branches of code and simultaneous support and enhancements should be thought through
www.scmGalaxy.com
scmGalaxy
CM – stage 3 - enhance and strategize
Objectives Enhanced build script Integrated build and
release process Tagging of specific
releases One shot deployment
and roll-back capability Reports and data
analysis Branching strategy
www.scmGalaxy.com
scmGalaxy
CM – stage 4 - Continuous Improvement and auto-pilot
Mostly in administration mode The build script and all
process have lived their time and proven their capabilities
Often a full time CM may not be required
Often this phase is the point to let go of a full time CM
As with every software asset, continuous improvement is a must
www.scmGalaxy.com
scmGalaxy
CM – stage 4 - Continuous Improvement and auto-pilot
Objectives Reports of releases and
release notes with each release
Minor changes to build script Ability to regression test and
manage releases Manage multiple branches and
multiple releases Focus on traceability from
released features to bugs. Control changes to the repository
sG
CM – important CM – important taskstasks
www.scmGalaxy.com
scmGalaxy
Important tasks
Among all the phases a few tasks will be the most important part of a CM’s job
The next few slides will delve deep into some of these tasks The aim would be to understand each of these tasks in detail The intent is to provide you with a view of the over all objective of
the task and there on to arrive at a few strategies for each of them
www.scmGalaxy.com
scmGalaxy
Important tasks - list
Identify CI Version management and tool selection Prepare CMP Release and Branching strategy (Multiple) Environment management Build script Integrating tools with build Audit of the CM process Automating builds
sG
Configuration Configuration itemsitems
www.scmGalaxy.com
scmGalaxy
Identify Configurable Items
What is a Configurable Item (CI) A CI, is a unit of configuration that can be individually
managed and versioned. Term used to describe any entity / asset
(code and non-code) that may change across the project phases
The type of asset and its users will help in defining a versioning strategy and its availability
A CI list should be a comprehensive list of assets (or asset types) and should feed into the tool identification step
www.scmGalaxy.com
scmGalaxy
CI list
Asset type Audience
Documents / PDFs/ excel / ppt
Source code
Access control
Frequency of updates
Need for CM
Client supplied documents
Charts, reports, existing design documents etc.
Client / whole team
Yes Probably Usually available to the whole team
Never to rare
Helps in base lining the initial artifacts given by the client. Could be crucial during UAT
PM assets
•Status Reports•MOM•Plans•Process documents (including CI)•Bug analysis reports•Audit documents
Team. Selectively to the client
Yes No Selectively available to the whole team
Rare Single point of lookupMaintains versionsMOM and versions of MOM will be crucial during acceptance
www.scmGalaxy.com
scmGalaxy
CI list
Asset type Audience
Documents / PDFs/ excel / ppt
Source code
Access control Frequency of updates
Need for CM
Project documents
•requirements•Design and strategy document
Whole team and client
Yes No All people involved in a project should be given easy access
High Ability to track the assets as they evolve.Need to revert back to versionsOne point lookup
Application Source
•Source code relevant to all modules•Configuration files•DB scripts
Whole teamClient may not be granted access
No Yes All people internal to the organization and in the project should be given easy access
Very high
Version managementNeeds to be used for build and releaseOne point lookup for all source code
www.scmGalaxy.com
scmGalaxy
Where to version the CI
Choice of network share, internal CVS and an external facing CVS
Classified reports and sensitive documents may not be shared with the whole team or even the client.
Access control and level of access should be considered while choosing the location of store
Use a network share or a private CVS repository Different repository for client and internal teams
with scheduled synch-up between the two A network share is not version sensitive but can be
used for static items like 3rd party software downloads, documentation etc.
www.scmGalaxy.com
scmGalaxy
Directory structure
Over all a directory structure is mostly common sense
Ability to logically separate out asset types and put them into separate folders
Based on the version management tool, one can setup directory level permissions or permissions at a repository level
It may also be prudent to understand the need for private work areas and private folders for each individual in the project
Leverage the existing structures at an organization / group / account level for a start point
www.scmGalaxy.com
scmGalaxy
Directory - first level
<repository name> Requirements Design App Templates QA PM UI reports
www.scmGalaxy.com
scmGalaxy
Naming conventions
Naming conventions (even if a file is organized into directories), should describe the asset type to the fullest using some meta data of the asset. intent of the document (or asset) –
GlobalOne_reference_exisitng_report_template.doc date (specially with MOM) –
MOM_admin_role_clarififcaton_010207.doc module it is relevant to – design_admin_logging.doc
All asset types identified in the CI list should be given a naming convention
Source code will follow the naming convention given by the coding standards document
sG
Version Version management management
toolstools
www.scmGalaxy.com
scmGalaxy
Version management tools
There are various tools that allow version management
The primary features of these tools are Version management Concurrent development (lock or merge) Client server model Branching Tagging Multiplatform support Integration with IDE Multiple authentication modes
www.scmGalaxy.com
scmGalaxy
Version management tools
Some tools provide the following features Web interface (integrated with the product) Distributed development using multiple
repositories Auto-replication
www.scmGalaxy.com
scmGalaxy
Selection criteria
Criteria Locking and merging Authentication and authorization mechanism Need for distributed repositories / distributed clients Web interface Licenses (cost and ease of setup)
Mostly the client / organization standard will be used as the default for a project
However, in case we are presented with a ground up scenario, then the criteria listed above can be used for selection of the tool
www.scmGalaxy.com
scmGalaxy
Various types of repository access
•Understand the impact of •Network speed•Stale data•Cost of setup•Security•Authentication
www.scmGalaxy.com
scmGalaxy
Version management – version number
The basic versioning of an asset will be 1.0 onwards in most version management tools.
Incremental changes are recorded as 1.1, 1.2 and so on
From an asset perspective, a version history will be maintained in the asset itself to define the changes made across each released version
The version numbering at a tool level can be used as is
Thought should be applied more to tagging of assets and base lining assets
www.scmGalaxy.com
scmGalaxy
Version management - tagging
A tag helps in creating a logical name for all assets within a repository at a particular instance
A tag can be given at a branch level and based on the tool even at a directory level
Tagging is the most important feature of a version management tool that helps in recreating a release
The name of a tag should be intuitive enough to show the user information about the date, version number, major and minor release etc
www.scmGalaxy.com
scmGalaxy
Files in the repository
www.scmGalaxy.com
scmGalaxy
Tagging the repository
The number of versions that each file take is dependent on each owner
However, when a delivery needs to be done, all files across multiple versions need to be taken.
One needs to take these files and have the ability to recreate the version to solve a bug if need be
Tagging allows us to set a string as the “tag” to various files each with different versions
A tag is like a snapshot that can be recreated
sG
CMP – CMP – configuration configuration management management
planplan
www.scmGalaxy.com
scmGalaxy
Intent of a CMP
A CMP describes the configuration management plan for a project
Describes the repositories, naming strategy, versioning strategy and more
Reference point for the team to begin working in a project
Helps in documenting all the CM activities and delegating responsibilities of a CM activity to team members
A CM initiates the CM processes and rolls out a process
It is for the project there on to adhere to it there on. The CMP defines the various roles and their responsibilities
www.scmGalaxy.com
scmGalaxy
Intent of a CMP
To be used by the whole team, the auditors of the project and the new entrants into the team to understand get hold of various assets.
The tools used and their setup, the process of getting access can be a separate handbook
The CMP sets the direction and helps in future audits of the project from a CM perspective
www.scmGalaxy.com
scmGalaxy
Intent of a CMP
A CMP should document enough data to help answer the following questions What kind of items are CI? Where should I put what CI? What are the naming conventions for various items? What kind of documents should I maintain and what is my
responsibility from a CM perspective? What is the versioning strategy of any artifact? How will project level changes be tracked? What is base lining and how will releases be managed
www.scmGalaxy.com
scmGalaxy
Change management
A key task of the CM is to control the changes to an asset
A change control process defines the monitoring / approval process that allows incorporating changes into a CI
Requirements, tools, criteria of acceptance etc. are CI that are stored in the repository
A change in one of them could affect many other assets and the project schedule
The intent here is not to resist changes, but to allow a controlled way of managing the changes
www.scmGalaxy.com
scmGalaxy
Change management
The CMP should record the ways to track / control the changes to each CI
if not all at least the critical assets such as requirements, design, acceptance criteria, plan, strategies specific to the project and more should be considered
Document the ways to accept changes to a particular asset bug tracker / email can be a source of change requests.
The CMP should also call out the person responsible to manage these changes
This level of clarity in change management ensures that changes to appropriate documents are controlled
www.scmGalaxy.com
scmGalaxy
Change management
Another aspect of change management is the process that one should follow to get a change incorporated
More relevant to a PM However the CM needs to ensure that a CCB has
been instituted for the project The change management for appropriate assets
would need a CCB approval (schedule change / feature list increase)
www.scmGalaxy.com
scmGalaxy
Read a sample CMP
Demonstration of a CMP
www.scmGalaxy.com
scmGalaxy
Key points in a CMP
Ensure that all CI have a repository to go to Ensure that all CI have an appropriate location in
the repository (directory structure) Ensure that each repository identified is backed up
and an appropriate person / group is given the responsibility
Ensure that all CI have an owner (person, group, roles). These owners will maintain the versions of these items
Most importantly, ensure that the CMP is well circulated and call out the responsibilities in a team meeting or in a short training program
Do not assume that people will know about their responsibilities.
www.scmGalaxy.com
scmGalaxy
Rolling out a CM process
Rolling out the process, often is taken too lightly and the team is not always in synch with the process
Identify who needs to be told what and tell them the process using training programs / meetings
Do not shy away from setting up a meeting each time the team size swells up.
For large teams, work with each module rather than a large team
www.scmGalaxy.com
scmGalaxy
Access to the repository
A well defined way to setup the client software and integrating it with the IDE should be documented
Re-use the standard within the organization for a particular type of repository
The steps could be simple and easy, but do not assume that a team of 10-50 people will know it
Ensure that a setup document, is available within the repository / network share
The setup document should speak about Where to get the installable (ensure that nobody uses their own
versions) How to install it (ensure that the installation document has
screenshots) How to integrate my IDE with the repository (ensure that people
to not have copies of the code in their own machines)
www.scmGalaxy.com
scmGalaxy
Check point
At this point, the CM process has been initiated and established
The team can setup the clients and begin versioning their assets
The next set of steps is related to building the code and creating a single composite unit
Keeping the build process aside it is important to understand the concept of release, branching and traceability
A build process could be a script or a manual one. The intent of the script is to create a comprehensive unit of code that can be released to an environment
Lets work with the assumption that a build script exists and understand the concept of release
sG
Release Release ManagementManagement
www.scmGalaxy.com
scmGalaxy
Need for a release
A single unit of software has many dependencies on other entities Database Environment parameters Application parameters
Parameters could differ based on the environment the application is being deployed into
The intent of a release is to build that comprehensive unit and deploy to an environment
A release could be a manual one, automated one or a blend
the key point to note is that it should modify all related entities with a particular version of code and allow users to view new features
www.scmGalaxy.com
scmGalaxy
Notes about a release
Important considerations for a release The whole software unit should be one single unit (e.g. ear file) Ensure that all external dependencies are modified along with a
release• Database• Properties file
These modifications can be manual (not preferred), automated script (better), integrated within the application deployment (like an installation exe - ideal)
Ensure that a set of happy flows are tested before the release is used by everybody
Release notes should be used to document the “new features” and the “known issues” in a release
www.scmGalaxy.com
scmGalaxy
Traceability
Traceability helps to relate a component to its requirement
Once the software is released to an environment there is the additional complexity of bugs
Each change to the software to resolve a bug, should also be traceable to the bug and the requirements / updated requirement (in the form of a bug)
A change could even be initiated by a review comment.
All changes to the unit should be traceable to its original cause for change
www.scmGalaxy.com
scmGalaxy
Traceability
This concept of being able to trace back a need or a cause (requirement) from the effect (code base) is traceability
A cause of change could be a requirement, a review or even a change to design
A configuration manager needs to enable traceability with the CM processes
It is up to the team to make adequate comments, put into version history etc to manage the traceability but the infrastructure to bring in traceability is the task of a CM
www.scmGalaxy.com
scmGalaxy
Traceability in Documents
Documents such as requirements or design are generally described in a word document
These assets are also CI and therefore will be versioned
The comments while checking them into the versioning tool is important to bring about some level of traceability against time. The comments will be seen in the future to understand why a check in happened
The version history table in each of these documents provide a better point for tracking the change
Descriptive version history and more importantly relating these changes to other artifacts in the repository (MOM, status reports) is most important
www.scmGalaxy.com
scmGalaxy
Traceability with code assets
Traceability is most relevant to code assets The traceability of a code asset to a requirement or a
design is often done using external matrices and code structure Package structure
Documentation of a method can trace back to a design asset
Traceability to a change / bug resolution is also more important
This level of traceability is also relevant in a continuous maintenance mode
www.scmGalaxy.com
scmGalaxy
Traceability and tagging
The ideal way to work with release is to first tag the code and then retrieve the code based on that tag
Between 2 tags (or releases) one can determine the number of changes done to a repository and track the comments against each of these assets
Ensuring that a comment exists and the comment follows a particular structure is one way of brining in traceability E.g. – comment should begin with BUG #NNNNN - <comment>
ensures that the user writes the bug number before he checks in the code
These comments can be used as a way to define the release notes
www.scmGalaxy.com
scmGalaxy
Strategy for release numbers
A tag that is applied to the repository should be self explanatory of the release and can be used as the release number too
If a different release number and tag number is being used, then a separate document should track the relation between the tag and the release
A release number should have the following parts to it Code base name / app name Major release number (sequence) Minor / patch release number (sequence) Time (day, month, year) Type of release (qa, prod, dev, RC) Branch name
www.scmGalaxy.com
scmGalaxy
Demo of release and tagging
Demonstrate tagging and obtaining a release Demonstrate the ability to trace file changes between 2
releases Show the release readiness template
www.scmGalaxy.com
scmGalaxy
How to support multiple projects / features
Once a project is delivered, and tagged, often the next release is decided at a pre-defined interval
In the interim the developers continue to edit the files to prepare for the next release
In the interim, if there are bugs on the production release, the files latest files cannot be used to fix them
At each point where a release is made, once needs to create a branch
A branch is like a copy of the whole code base except that it starts of from a particular tag / version
The main branch is called the ‘HEAD’ branch. All other branches can be named as required
Branches can be merged once it is no longer relevant
View CodeBranch_Merge.vsd
www.scmGalaxy.com
scmGalaxy
Note on branching
A new branch is more responsibility on the CM and the dev team
Each bug and change there on should be done on the right piece of code
Update the setup documents to reflect the steps needed to retrieve sources from a particular branch and not just the main trunk
Branching as a concept should be taught to novice developers
Builds and environments to deploy the build is also impacted with each new branch
Often the number of CM in a project may need to be increased
www.scmGalaxy.com
scmGalaxy
Branching in CVS
www.scmGalaxy.com
scmGalaxy
Impact of branching
branching of a repository impacts Tags release process frequency of builds developer environments build scripts Setup documents Authentication permissions Build machine hardware requirements
Understand the impact of branching for your project A new branch is a simultaneous project on the same
code base but on different threads. It is simple only if pain is taken early on to strategize
www.scmGalaxy.com
scmGalaxy
Demo of branching
www.scmGalaxy.com
scmGalaxy
Supporting multiple environments
www.scmGalaxy.com
scmGalaxy
Common issues with CM and a team
Issues a team often faces with a bad CM I do not know how to configure my client environment? What is the process to follow when I check in my code? What is traceability? What is my role in it? The DB does not reflect the latest status, what should I do to get
the DB changes into the code base? Where is the documentation of the 3rd party library? May I should
download and use the latest version? The build takes too long, all I wanted to do was to deploy a JSP
page? Which environment is the tester getting these bugs? How many
environments are there?
www.scmGalaxy.com
scmGalaxy
Audit of a process
Need for audit Understand whether the course being taken is
as per the original plan Deviations are not unnatural, but the impact of
deviations should be well understood De-risk the process / project to ensure that
there are no serious lapses against the plan
www.scmGalaxy.com
scmGalaxy
SCM audit
The purpose of the Configuration audits is to: Assure that correct versions of Configuration items
are placed under the appropriate directory / folders in the central repository.
Changes to be included in the version of the product to be delivered are included.
All required items of hardware, software, system, data, procedures, and documentation are available with correct identification and labeling and storage.
Configuration Status is tracked Show a sample audit template
www.scmGalaxy.com
scmGalaxy
Frequency of audit
Quarterly audits or auditing between phases will keep the process compliance high
Audit will usually done by PM / Leads from other projects
Audits within the project is also a good practice The CM is the owner of the CMP and has to ensure its
adherence Regular checks by the CM will help in the long run
comments when doing a check-in maintaining the version history in documents adherence to the naming conventions
www.scmGalaxy.com
scmGalaxy
CM best practices
Integrate all users with the repository. Build, development IDE and other touch points should use the repository as the source
Dissuade users from making copies of the repository items
Focus on automating as many tasks in the build script
Automate the build kick off using tools like cruise control
Manage only the source and not the derived binaries Arrive at an appropriate branching strategy for the
project Roll-out the CM process and educate the team
sG
Mindtree CM’s Mindtree CM’s rolerole
www.scmGalaxy.com
scmGalaxy
SCMP Process at MindTree
www.scmGalaxy.com
scmGalaxy
What are PM’s Responsibilities?
PM’s Responsibilities CM’s Responsibilities
Identify Configuration Manager
Identify SCM tool Identify SCM tool Ensure creation of SCM Plan
Create SCM Plan
Ensure CCB is established
Establish Change Control Process
Ensure setup of SCM Processes
Establish SCM Processes (checkin,checkout,build,release) & ensure compliance
Plan SCM related trainings for the team
Train the team
Plan SCM Audit & ensure compliance
Conduct internal SCM audits
www.scmGalaxy.com
scmGalaxy
What are PM’s Responsibilities?
PM’s Responsibilities CM’s Responsibilities
Communicate the importance of SCM to the team
Baselining
Naming Conventions
Directory Structure
Tagging
Branching
Setup / Automation of builds
Releases (QA, UAT, Production)
sG
Case studyCase study
sGwww.scmGalaxy.com
Author: Rajesh [email protected]