Validation Missteps Making Us Full Time Firefighters

Presented to Silicon Valley DVClub January 26th, 2010 Allison Goodman, allison.goodman@intel.com

My Background

Products that survived the fires to market (some minus a few features)

Products were consumed by fires late in development

4

SSD

3

What missteps along the way turn us into firefighters?

 Insufficient Planning  Not Designing for Test  Not creating and integrating

Feedback Loops  Lack of Transparency

Insufficient Planning- Don’t have what you need to test Original Test Coverage Not Enough

Undocumented Assumptions

Missed Dependencies

SCOPE

Scope

Lack of Coverage

If you don’t plan for it…it will surprise you.

Every surprise will end up as a fire …

As you get closer to trying to ship a product you become a full time firefighter.

Plan! (in writing!)

Take the time to write it all down -who does what (owners!) -how does a feature really work -what “done” is -checkpoints along the way -success criteria

Keep track of the assumptions

And what everyone really needs

test

Not Designing for Test Very difficult to repeat tests for debug Waste time

My Design is Perfect No plan tests- late implementation

Pass? Fail? Who knows?

Getting lucky vs knowing it works

Getting test to fail again

If you don’t design for test, you won’t find or replicate issues … they will surprise you.

Every surprise will end up as a fire …

As you get closer to trying to ship a product you become a full time firefighter.

Design for Testing!

Hooks & Bread Crumbs • Where you are (state) • How you got there • How to get back

Ability to Artificially inject fails

Figure out coverage

Define Pass & Fail with Team Partner with Architects & Design, you are helpful!

tests?

Not creating & integrating Feedback Loops-

Planned Testing

New Market / Customer Requirements

Planned Testing Stale

Testing

Hardware Software Firmware Marketing

Working in Silos Implementation != Documentation

If you don’t create and implement feedback loops, you’ll miss issues…they will surprise you.

Every surprise will end up as a fire …

As you get closer to trying to ship a product you become a full time firefighter.

Create & integrate Feedback Loops

Validation Team

Tight & Ongoing Feedback -Problems found on current product -new info about system interactions -actual implementation changes

Hardware Software Firmware Marketing

Planned Testing

New Market Requirements

Additional Testing

Adaptable to new requirements/usages

Is it Effective? Efficient?

Change it!

Run Test

Lack of Transparency

Team telling mgmt…or worse each other, it’s all fine

What you feel is really happening

DONE!

Schedules run late, validation is squeezed

Fewer iterations, narrower

coverage, less thorough

investigations

If your team is not transparent, you’ll ignore risks and coverage gaps…escapes will surprise you.

Every surprise will end up as a fire …

As you get closer to trying to ship a product you become a full time firefighter.

Foster a Transparent Team

Everyone plays a role…

Don’t declare DONE until it is really there…. Don’t punish for calling it out

Tell the real story, Encourage others to do the same

Stop the Firefighting!

 Plan  Design for Test  Create & Integrate Feedback Loops  Foster a Transparent Team

Questions??

 Allison Goodman  Allison.Goodman@intel.com  Linked In  www.engineeringgal.com