Improving Productivity on an SEI Level IV Project presented by Kelly Ohlhausen

Improving Productivity on an SEI Level IV Project

presented by Kelly Ohlhausen

Page 2 © 2002, Raytheon Company. All Rights Reserved.8 February 2002

Agenda

• Project Description - Where we were…

• Why is it important?

• How did we find the answers?

• What did we find?

• Where we are now…

• Details of what we did

• Summary


Where we were...

• One of four projects evaluated during an SEI CBA-IPI Level IV assessment. (June 2001)

• Productivity identified as a risk from the beginning (productivity = ELOC/hour)

• Software bid was aggressive

• 10% management challenge

• 12% customer challenge

• Real-time embedded system to be completed in two deliveries to the customer


Where we were...

154.2%

109.6%

131.4%

83.1%

64.5%

74.0%

82.9%

0.0%

82.9%77.9%

0.0%

77.9%

87.6%

78.3%

85.5%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

140.0%

160.0%

SD OD IM SIL I&T CTD for Release

Release 1 Release 2 Phase CTD

Planned Productivity


Why aren’t we meeting the goal?

• Was the productivity goal set unrealistically high?

• Process - Are these the right processes? Are there too many? Too few?

• Was the mix of engineering skills appropriate?

• Was there enough training?

• Software projects can’t ever reach

goals so why bother setting them?


Why is this important?

• Productivity estimates impact customer and management satisfaction, profits, and potentially future business.

• Our customers need to be able to trust the numbers we use in a bid.

• Profit margins disappear when projects are bid with unrealistically high productivity levels.


Finding the Answers

• Initiated a Raytheon Six Sigma project to determine what, if any, actions could be taken to improve productivity during the implementation stage

• Raytheon Six Sigma is an iterative knowledge-based process for improvement


Analysis Factors

• Attrition

• Learning curve (training new people)

• Experience level

• Hardware availability

• Overlapping stages

• System engineering

• Customer directed slowdown


Information Sources

• Metrics: size, cost, schedule, delivery, staffing, defect containment, requirements stability

• Earned value progress metrics, audits, action items

• Lessons Learned from prior stages

• Two other projects using the same processes

• R6Sigma Defect Containment Metric Analysis project results


Information Sources

• Meeting minutes (e.g. weekly progress, metrics analysis)

• Inputs from engineers and managers


Information Sources

The search for information took place during R2 object design stage.

Software Development R1 R2- Requirements Analysis - System Design - Object Design - Implementation - Software Integration & Test - System Integration & Test- Acceptance Test


Requirements for Application

• Changes were to be put into place immediately.

• The R2 implementation stage schedule had to be shortened by four weeks without removing scope to compensate for a schedule slip. The software integration and test stage had to start on time.

• Changes could not have a negative impact on quality.


What we found...

Moving toward SEI Level V (Optimizing) with a number of

smaller actions involving defect containment and

process change management could significantly impact

productivity.


Where we are now...

* as of 2/1/02

160.0%

108.5%

133.7%

86.2%

63.8%

75.3%

86.0%

120.3%

97.4%

80.9%

0.0%

80.9%

90.9%95.5%92.4%

0.0%

20.0%

40.0%

60.0%

80.0%

100.0%

120.0%

140.0%

160.0%

180.0%

SD OD IM SIL I&T CTD for Release

Release 1 Release 2 Phase CTD

Planned Productivity34.3% increase


Where we are now...

Earned Value Metrics

0

1000

2000

3000

4000

5000

6000

7000

9/21 9/28 10/5 10/12 10/19 10/26 11/2 11/9 11/16 11/23 11/30 12/7 12/14 12/21 12/28 1/4 1/11 1/18 1/25 2/1 2/8 2/15 2/22

Ear

ned

Val

ue

Total Earned Value Plan Forecast 1 Forecast 2 Labor Actuals

* as of 2/1/02


What was changed...

• Escaping defects– What should be covered by a unit test was

addressed in detail during the implementation stage training (JIT)

– Stress was placed on pushing boundaries and not relying just on nominal values. More special case unit tests were used.


What was changed...

• Escaping defects (cont.)– Run the unit tests on the target processors during

implementation. Don’t wait until integration to discover a platform issue.

– A step was added to the stage process for extra testing to identify rogue pointers before unit testing

The full impact of these changes will not be known until integration and testing stage.


What was changed...

• Developed the code within the configuration management tool from the beginning. Gave people a chance to learn the tool in small steps instead of all at once.

• All tools and hardware identified and checked out BEFORE the stage began (matching versions across test lab, compiler updates in place, instructions available)


What was changed...

• We did better defining the earned value progress metrics– Made sure there was enough detail (stage process

steps were defined at the correct granularity)– Used realistic weighting

Because…An incorrect plan yields misleading information.

Time is wasted investigating non-issues or not identifying a problem in time.


What was changed...

• Did better at following the standards and processes. – Invited the right people for the review.– Assigned roles to focus the review, esp. facilitator– The implementation stage Just-In-Time (JIT) training

was used to remind the team of these issues.

Because…– Products that do not meet standards have an effect on

time during reviews as well as rework.


Summary

• There was no conclusive evidence that the productivity goal was unrealistic.

• Relatively small changes made an impact.

• The research uncovered improvements for every stage.

• If we had applied these improvements from the beginning of the project, we can only imagine what the overall productivity would have been.


Conclusion

• Take time to look at what you are doing.

• Learn, not just from your own experience but from the experience of others.

• Continuously try to identify optimization opportunities.

• Improvement can be made,

even on an SEI Level IV program.


The greatest obstacle to discovering the shape of the earth, the

continents and the ocean was not ignorance but the illusion of

knowledge.

Daniel J. BoostinThe Discoverers

Documents

Improving Productivity on an SEI Level IV Project presented by Kelly Ohlhausen