Object-Oriented Analysis and Design

Lecture 9

The Capability Maturity Model

Background Above all, this is about management,

not technology. Technology promises productivity

gains, but they will likely be unfulfilled in a “maelstrom of an undisciplined, chaotic project.”

1986, the SEI and Mitre decide to do something: a process maturity framework to help organizations improve their software process.

Background (cont.)

The methods are “software process assessment'' and “software capability evaluation.''

Note the ideas, methods and the language are DoD-centric.

Once again, “guidance.”

Background (cont.)

The result of the SEI effort is the Capability Maturity Model (CMM), which is

“...sets of recommended practices in a number of key process areas that have been shown to enhance software process capability.”

A better mousetrap will come from a better mousetrap process.

The Immature Organization Software processes are generally

improvised during the course of a project. Managers are reactionary, solving

immediate crises. Functionality and quality are compromised

when hard deadlines approach. “The death march to Egghead.”

No objective basis for judging product quality; thus quality is difficult to predict.

The Mature Organization The software process is accurately

communicated to existing and new staff.

Work activities are actually carried out according to the planned process.

The mandated processes are usable and consistent with the way work actually gets done.

Roles and responsibilities are clear, schedules and budgets are based on historical performance.

Fundamental Concepts

Software process: methods, activities, practices used to develop and maintain software

Software process capability: the range of expected results achieved by following a software process.

Software process performance: the actual results achieved by following a software process.

More Concepts

Software process maturity: the extent to which a specific process is explicitly defined, managed, measured, controlled, and effective.

As an organization gains in process maturity, it institutionalizes its process by policies, standards and organizational structures.

The Five Levels

The key idea is to identify an evolutionary path from immature to mature.

There are a number of well-defined plateaus along the way.

Each plateau (level) is a set of process goals which, when satisfied, stabilize some component of the software process.

The process goals are prioritized, so organizations understand what to do next.

The CMM Levels

Level 1: The Initial Level

Unstable environment Makes commitments that can't be

meet with an orderly process. Plans are scrapped during crises,

jumping to coding and testing. Can success happen? Yes, but only

through heroic efforts.

Level 1 (cont.)

Such efforts will not likely be repeatable.

Capability is a characteristic of individuals, not the organization.

Would you buy from such an organization?

Level 2: The Repeatable Level Management policies are in place. Planning and managing are based

on experience. Policies are enhanced on a project

by project basis. Commitments are realistic.

Level 2 (cont.)

Managers track software costs, schedules, functionality.

Project standards are defined and followed.

The process is disciplined because planning and tracking are stable.

Earlier successes can be repeated.

Level 3: The Defined Level

A standard process for software engineering and management across the organization is documented.

There is a group responsible for this standard software process.

There are organization-wide training programs.

Level 3 (cont.)

Projects tailor the standard process to account for their unique features.

Such a tailored process contains readiness criteria, inputs, verification mechanisms, outputs and completion criteria.

This process capability is based on a common, organization-wide understanding of activities, roles and responsibilities.

Level 4: The Managed Level The organization sets quantitative quality

goals for software products and processes.

A process database is used to collect and analyze data from projects' processes.

Variation in process performance is narrowed to acceptable levels.

Software processes are instrumented.

Level 4 (cont.)

Risks in new product development are well understood.

The process capability is quantifiable and predictable.

Software products are of predictably high quality.

Level 5: The Optimizing Level

The entire organization is focused on continuous process improvement.

Weaknesses can be identified and remedies found.

Data on software process allows cost-benefit analyses on new technologies.

Management View of “Visibility” Into a Project Moving through the levels,

managers are better informed, and have greater control to do good…

Management View of “Visibility” Into a Project Level 1:

The project is a black box Requirements flow into the project

haphazardly The staging of activities is hidden Managers can’t establish project

status

Management View of “Visibility” Into a Project Level 2:

Requirements are controlled Visibility at defined occasions The project is a sequence of black

boxes Management reacts to problems as

they occur

Management View of “Visibility” Into a Project Level 3:

Tasks inside each black box are known

Management and engineers understand their “roles and responsibilities” within each box

Management is proactive in dealing with problems, because of rapid status updates

Management View of “Visibility” Into a Project Level 4:

Processes are “instrumented” (details of estimates and outcomes are recorded)

Decisions are based on hard facts Outcomes can be predicted more

accurately Variability in outcomes gets smaller

Management View of “Visibility” Into a Project Level 5:

Improved methods of software development are regularly tried

Defect-prone activities are replaced Managers can predict the impact of

process changes

Process Capability and Prediction of Performance Three improvements are expected

as the software process matures: The difference between targeted

results and actual results decreases. The variability of actual results

around targeted results decreases. Targeted results improve as maturity

increases.

Operational Definitions

These are supposed to be specific recommendations for an organization to increase its maturity.

There are at least four uses for them:

Operational Definitions (cont.) Assessment teams use CMM to identify

strengths and weaknesses. Evaluation teams use CMM to identify

risks in selecting contractors. Upper management will use CMM to

figure out how to launch a process improvement program.

Staff and process improvement groups will be guided by CMM.

Key Process Areas: Level 2

Requirements management Software project planning Software project tracking and

oversight Software subcontract management Software quality assurance Software configuration management

Key Process Areas: Level 3

Organization process focus Organization process definition Training program Integrated software management Software product engineering Intergroup coordination Peer reviews

Key Process Areas: Level 4

Quantitative process management Software quality management

Key Process Areas: Level 5

Process change management Technology change management Defect prevention

Evaluation

Process maturity is evaluated by SEI teams (and those trained by them).

For each Key Process Area, an organization is rated according to: Commitment to perform Ability to perform Activities performed Measurement and analysis Verifying implementation

Comments From Ed Yourdon This is not the only model (cf.

Software Productivity Research, the Capers Jones outfit)

What about small, innovative firms? According to the CMM, “Apple Computer should not exist.”

Small organizations probably cannot afford the overhead required by CMM.

Other Implications

You can't skip levels (these are cultural changes).

It takes time to go from one level to the next (10 years total? What about regression?).

New technology should be avoided at lower levels (these are management issues).

New software organizations won't start at level 3 (the all-star team).

Evaluations important for winning contracts.

A Serious Rebuttal

From James Bach: “The CMM is a broad and increasingly

deep set of assertions about good software development practice.”

Where do these assertions come from? Are they complete and correct?

Bach (cont.)

“My thesis...is that the CMM is a particular mythology about software process evolution that cannot legitimately claim to be a natural or essential representation of software processes.”

More Bach

The CMM is “at best a consensus among a particular

group of software theorists and practitioners”

“at worst a whitewash that obscures the true dynamics of software engineering [and] suppresses alternative models”

“If an organization follows it for its own sake, it may lead to the collapse of that company's competitive potential.”

Other Comments

The CMM was originally designed to evaluate the ability of government contractors to develop a software project. Will/does it work?

Is it useful outside of this context? The biggest (popularly-known) names

in software (Borland, Symantec, Apple, Microsoft) are likely Level 1. (Note in Jim McCarthy's book, CMM isn't even mentioned.)

Shrink-Wrap Perspective

No theoretical basis; fashioned by “very knowledgeable people,” so the de facto principle is that experts know what they're doing.

Any other model from “very knowledgeable people” has equal veracity.

Vague empirical support; this support doesn't exclude other models; there is no comparison of methods.

Shrink-Wrap (cont.)

It doesn't account for the success of shrink-wrap companies.

CMM reveres process but ignores people; process makes up for mediocrity; where is the justification for this?

Shrink-Wrap (cont.)

Imagine a process definition for playing chess.

Institutionalization of process guarantees nothing; such efforts lead to a simplified public process and an undercover private process.

Process dynamics are lost; why isn't training at level 1, for example? Level 1 is nothing; just get to level 2!

Shrink-Wrap (cont.)

Most of the Key Practices could be introduced at level 1, depending on the organization.

The CMM replaces the true mission of better process to the artificial mission of higher maturity level; “level envy.”

Other Arguments

The most powerful argument against CMM is the many successful companies that should not exist. Microsoft is level 1, IBM is mature.

Process should support innovation; systematic problem-solving leadership to enable innovation rather than cookie-cutter solutions.

Still More

CMM is profoundly ignorant of the dynamics of innovation. CMM pushes authority up in the organization.

For CMM, heroism is an unsustainable sacrifice on the part of individuals. Personal mastery is left out of the equation.

An alternative: well, Bach obviously has his own story...