InfoStretch - Million Dollar Bug: Discover it before it discovers you

phone: (408) 727-1100 | website: www.infostretch.com | email: [email protected]

©2004-2011 InfoStretch Corporation. All rights reserved.All trademarks are the property of their respective owners.©2004-2010 InfoStretch Corporation. All rights reserved.

August 30, 2011


©2004-2011 InfoStretch Corporation. All rights reserved.

1. Introduction

2. What is a Million Dollar Bug (MDB)?

1. Characteristics

2. Impact

3. Examples

3. How to discover MDB?

4. How to minimize the impact of MDB?

5. Q & A

Agenda

INFOSTRETCH CORPORATE(p) (408) 727-1100 | (w) ww.infostretch.com | (e) [email protected]©2004-2010 InfoStretch Corporation. All rights reserved.



(p) (408) 727-1100 | (w) ww.infostretch.com | (e) [email protected] | Presenter: Rutesh Shah


InfoStretch OverviewElevating Confidence

mailto:[email protected]



Partners



InfoStretch Overview

Organization

HQ in Silicon Valley Offices in New York, Seattle 100+ Clients and 160+ Projects Well Established Recruitment, retention and training

programs

People

Average 5 years experience 350+ Engineers/Consultants Industry Expertise in Finance,

Retail, Mobile and Healthcare Strong Project Mgmt. team

Infrastructure

Extensive Platform Compatibility and Performance Test Labs

2000+ Mobile Handset Test Lab ISO 9000-2001 Certified High Availability and 24 hours

secure offshore centers ISO 27001 Certified




Improved Time to Market

Flexibility & Scalability

Professional services for

implementation and

integration

Operational Cost Optimization

Comprehensive Quality Strategy

Improved Time to the Market

Extensive Skills & Focus on

Efficiency

Custom Process – Agile/Waterfall

Qmetry- Visibility and metrics

Leverage automated regression

End-to-End Mobile Solutions

Native App and Web App support

On Device Live Network Testing

Expertise - iOS, Android, WebOS,

Windows Mobile, J2ME

Centers of Excellence

(p) (408) 727-1100 | (w) www.infostretch.com | (e) [email protected]©2004-2010 InfoStretch Corporation. All rights reserved.

Serving Finance, Retail, HealthCare, Hospitality Industries

SaaS SolutionsFocus on The Core

QA SolutionsPathway to

Performance

Mobile SolutionsAccelerate

Success



Serving 5 out of large 7 Mobile Appstores in America

Media Buzz

INFOSTRETCH CORPORATE(p) (408) 727-1100 | (w) ww.infostretch.com | (e) [email protected] | Presenter: Rutesh Shah©2004-2011 InfoStretch Corporation. All rights reserved.

19th Fastest growing privately owned

companies in Bay Area

Awarded Inc 500/5000

company of the year 2010

Among top 1000 in 2011

Among10 Most Promising

Testing Services Company

GESIA Rising Star Award 2010 – Best

Software Product

157th Fastest growing Technology company in USA

Gartner Top 20 QA Service

ProviderExceptional QA Services and Solutions for Enterprise,

Mobile and SaaS.

Lead411HOT Companies in Silicon Valley

Qmetry – Market Mover Software Testing Platforms


http://www.inc.com/inc5000/2010/index.html

http://www.thesmarttechie.com/index.php

http://www.thesmarttechie.com/index.php

http://www.gesia.org/index.php

http://www.gesia.org/index.php

http://www.vokeinc.com/





A Million Dollar Bug (MDB)What is it?






Audience Poll

What is a Million Dollar Bug?- A bug that takes a Million Dollar to

discover?- A bug that costs business more

than a Million Dollar in damage?- A bug that is reported by a

customer?- All of the above





Defining the Million Dollar Bug (MDB)

Undiscoverable System Flaw

• Cannot usually be detected even by a well defined QA process

• Cannot be captured in the requirements that can be verified by QA team

• Even increased testing or increased QA budgets cannot help

• Might not be a bug itself, but a flaw that can expose the system to security risks like fatal viruses

Causes irreversible damages

• Millions of dollars in financial losses (in some cases, billions)

• Damaged brand reputation

• Loss of customer loyalty

• Loss of key employees• Unprecedented

disruptions to business functioning

Is a competitors’ rejoice

• An opportunity for industry underdogs to topple over the industry leader overnight

• A great lesson for the other players in the industry, at your expense





Examples of MDB – Toyota Prius Recall

Huge financial losses $2B+ in revenue/cost

Damaged brand reputation No longer a company that delivers

quality products Competitors’ rejoice

Loss of market share Flawed follow-up strategy

Initial denial followed by a recall Legal action by government Took too long to act





Examples of MDB – BART Software Bug

3 times computer shutdowns due to software upgrade in 3 days

Created unprecedented disruption for commuter services in Bay Area

Loss of more than $1.5 Million in revenue, additional cost of temporary transit





Examples of MDB – Y2K Bug

Needs an introduction?Cobol programmers reinvented -Created Billion+ Dollar industry for 3 years





Characteristics of MDB Very difficult to find as part of standard operating

procedure (test cycle, QA effort…) Once effects reach the customers - the impact is

significant (quick escalation in the organization) Competitors Rejoice– they are not sure if this can

happen with them or not but they are happy If you are industry leader – you are more vulnerable Creates new opportunities in the industry It hits you suddenly – there are no warnings – results

of “a perfect storm” conditions Impact is system wide and affects large

audience/customer base





Impact of MDBThe impact cannot be nullified by spending

Million Dollars in response an MDBIndustry Leaders may lose top position –

Toyota exampleMay result in a permanent scar on your

brandLoss of customer base - respective loyalty

and faithOrganizational shake-up and/or loss of

employee morale





Is a Million Dollar BugAn End of World for the product?

NOT REALLY – If addressed properly – it can create opportunities





So What do you Do?

Tactical - Address Customer Impact/Service Disruption in a timely manner

Strategic - Find the Real Cause to address the solution

Preventive - the Problem Does not Reoccur





Tactical Solution – Step1

Understand the Business Impact and Create/Communicate a Quick Remedy





Tactical Solution – Step2

Regain Customer Confidence Quickly





Strategic Solution – Step3

Don’t test ‘more’ , Test SmartOut of The Box Thinkers





Strategic Solution – Step4

Short-Term Workaround and Long-Term SolutionsFind the Real Cause to Derive the Solution





Preventive Solution – Step5

Let QA and testing group Do what they do best – Test against the requirements






Focus on intrinsic motivation, not on financial incentives






Have a simple definition of escalation and disaster recovery process





Can you Really Prevent MDB?





Prevent – Not Completely Today business solutions are delivered with

many dependencies and hence it is not entirely possible to prevent the MDB

BUT

You can certainly prepare the organization, Busienss to minimize the impact of MDB with various processesBusiness ProcessesTechnology ProcessesCustomer Information System





An Example Approach- Business Process

Scenario – Online brokerage firm. An unknown bug resulted in halting trading activity.

Impact - Resulted in loss of 55000+ trades in 30 mins

Approach - Usual War room approach to find the cause. Back-up system to initiate trades on phone at no additional cost. Executed all market trades from lost trades at customer discretion.

Advantages – Customers were impacted but were not disgruntled due to proactive actions from brokerage firm. The problem was discovered and the cause was a air-condition condensation leak of water in server room resulting in failure of SAN





An Example Approach – QA Process

Let QA and testing group focus on testing against the requirements – White Box and Black Box

Create an Out-of-the box SQUAD – Their job is to identify MDB causing scenarios Don’t give them requirements and/or testcases Focus is on end-to-end testing – not only subset

of systems Focus is on customer scenarios with real data Focus is on breaking the rules Focus is on learning from others’ mistakes in

the industry Deploy this team across many projects/products Test failover capabilities of the system

Mobile App DesignIt’s an art, not a science

Presenter: Morgan Russell, InfoStretch Corporation

All trademarks are the property of their respective owners.©2004-2011 InfoStretch Corporation. All rights reserved.

InfoStretch Next Webinar

Mark your CalendarApril 21, 2011

Mark your CalendarApril 21, 2011

Santa ClaraHeadquarters2880 Lakeside Drive, #200Santa Clara, CA 95054(408) 727-1100

Partners

New York17 Tanglewood Rd,Scarsdale, NY 10583(703) 395-9791

Ahmedabad5th Floor, Regency Plaza, Anand Nagar, Ahmedabad-15, Gujarat, India91-79-40266266

MumbaiVashi InfoTech Park, B103, Tower # 1, Vashi Rail Station, New Mumbai, Maharashtra, India91-22-6947900

Website: www.infostretch.com

