Shay Ganon - Manager, QA System Test Group Jul …legacy17.sela.co.il/s/sigist12/_AllPresentations/18July/...testing framework Thorough and deep white-box level testing of product

© 2009 IBM Corporation

The Million $ Question: How to retain value with global low-cost test options competing for your budget?

Shay Ganon - Manager, QA System Test Group

Jul 18th, 2012


Session takeaways

• Selecting focus areas that will keep your test group highly relevant

• When globalization could be effective

• Tips for collaborating with global groups

© 2012 IBM Corporation 2

© 2009 IBM Corporation © 2012 IBM Corporation


What is IBM XIV Storage?

IBM XIV is a high-end storage system, with a design that provides:

Consistent high performance

High reliability

Exceptional ease of use

What is Storage? • Storage is a key IT infrastructure layer, retaining and

protecting the data created by an organization’s systems

• Today’s complex IT environments require storage to

be: easy to use, highly reliable, high performing,

and easy to scale



XIV Overview

Founded in 2002 by 5 Talpiot graduates and

Moshe Yanai, “Father of Modern Storage”

Acquired by IBM In December 2007; today,

fully integrated within the IBM Storage

portfolio

+6000 systems deployed worldwide

R&D Headquarters at Azrieli Center, Tel Aviv

Manufacturing in Israel, California & Mexico



“IBM XIV Storage System is allowing us to meet our recovery time objectives while reducing our storage total cost of ownership” Greg Johnson, Director & CTO, IT Technology & Engineering Services, VCU Health Systems

“We are exceeding our SLAs and driving cost down". Maher Atwah, Ph.D. Vice President and CTO Health Data Management Solutions (HDMS) a Aetna Subsidiary

Among our customers…


© 2009 IBM Corporation 7 © 2012 IBM Corporation

Our testing organization: An overview

• Distributed over several countries: USA, China and Israel

• Majority of developers in Israel but ~70% of the test work force is outside Israel

• Tests coverage consists of: Software & Hardware combined

• Testing is mission critical due to high impact of field defects:

– Risk of data loss

– Risk of data integrity

– Inability to access data (e.g. credit card transactions blocked)

© 2009 IBM Corporation 8 © 2012 IBM Corporation

Shanghai, China

- Feature level testing

- Service packs release

Tel Aviv, Israel

- Feature level testing

- Integration testing

- Service packs release

Poughkeepsie, NY USA

- E2E scenarios

- Customer environments

Tucson, Arizona USA

- System level testing

- Performance testing


Why do top global companies founding R&D centers in Israel?

Because Israeli centers bring VALUE.

Passion

What are the core advantages of the Israeli engineers?



But we have competition…!

The overall cost of an

engineer in China is roughly

a third than in Israel

The technical skills and core competencies of a Chinese engineer are top rated

The number of Computer Science graduates per year is significantly higher than Israel and USA

Example: China’s rising advantages



Bottom line:

The traditional strategic assets of the Israeli high

tech industry and education system are delivering

less competitive advantage than before!



Advancing Authentication

1) Leverage your natural “Israeli Assets”

2) Pursue a greater impact on the core business

3) Capitalize on your proximity to R&D and business groups

How can the Testing Discipline deliver unique

value to maintain competitive advantage?



1) Leverage your “natural Israeli” assets ;-)

Agile mentality: – Versatile & flexible state of mind

– Creative, “out of the box” approach, capable to improvise

– Proactive initiatives

– Fast context switching

"System-wide” perception matched with the ability to

“dive deep” under the surface

Technical depth and analytical orientation

Willing to dare and embrace challenge

Nurture your flexibility + take reasonable risks



2) Pursue greater impact on the core business

Increase the value proposition of your group by: –Extend your coverage scope to important non functional areas (e.g.

security, privacy)

–Contribute to the product design during early planning stages

–Emphasize user’s point of view

–Exercise “dynamic engineering” - abstract thinking and problem-

solving

Optimize your testing methodology: –Design test plans for high risk features and focus more on exploratory

– Implement more automation

–Focus on impacting on the product itself rather than testing

procedures and “artifacts”

Hire test engineers capable of: –Acting as feature owners

–Writing code

Focus on testing that matters © 2012 IBM Corporation 14


3) Leverage your proximity to R&D

Coupling R&D and Testing is a key factor in establishing

unique added value, enabling you to: – Offer in-depth knowledge about the low level architecture and

act as technical experts (SME)

– Provide feedback regarding testability requirements

– Support R&D efforts with short-term test cycles (e.g. integration,

exploratory)

– Respond rapidly to needs as they surface (e.g. urgent service

packages)

– Be immediately available for ad-hoc meetings, early-level

discussions such as on product requirements, etc

Leverage information from business development and

product groups

Deepen your knowledge + drive more services



Leveraged

advantage

Description Added value

Technical depth Developed innovative in-house tool.

Our automation group designed and

implemented a state-of-the-art automated

testing framework

Thorough and deep white-box level

testing of product functions

(following system logic and states)

Proximity to

R&D

Training of testing teams worldwide.

Our local team has taken on an added role

of subject matter experts, taking

responsibility for transferring technical

knowledge to worldwide testing teams

Effective and efficient knowledge

transfer by adequate "translators,"

who understand the language of the

developers as well as ww testers

Proximity to

R&D

Direct support to R&D.

Our team provides R&D with fast feedback

regarding specific investigations

Improved development speed and

quality

Flexible mindset Fast turnaround and ability to adapt to

changing situations.

Our team easily switches between tasks

and handles change well.

Rapid delivery of urgent service

packs, with minimal impact on

scheduled milestones

Technical depth

Set up an E2E lab in Tel Aviv.

Our team has the knowledge to support

testing of customers’ real life environments.

Deliver valuable testing coverage

due to early execution of customer

scenarios in live environments

What we are doing at XIV: a use case

The group is the hub for technical discussions © 2012 IBM Corporation 16


When is globalization right?

Globalization is inevitable and beneficial!

Your organization may require globalization based on policy. If within

your control, consider globalization for: – Long-term, standalone tasks

– Manual tasks

– Regression of stable components

– Low risk tasks

How to maintain motivation & commitment of the remote team? – Provide them with adequate independence and direct exposure to R&D

– Create a balance between progression and regression tasks

– Plan together (“buy-in”) and maintain ongoing communication

– Provide them with increasingly challenging and complex tasks

“Globalization” - doing the work at the right place with the right skills at the right costs


(Based on my personal observation)


Globalization – the hidden costs…

The accumulated TCO of globalization is higher than just

comparing raw salary costs!

Some of the hidden costs: –Remote training / support

–Remote management

–Knowledge transfer

–Labs construction and maintenance

–Traveling overhead (flights / time)

–Slower progress / response

–Regulations

The real cost is more than meets the eye…


(Based on my personal observation)


Challenges collaborating with global teams



Challenges working with remote sites:

Effective communication barriers: –Language (even if the “functional language” is clear)

–Time difference / distance

– Identifying the communication channels

Diverse planning & management approaches: –How to prioritize tasks and manage risks?

–How to change an existing plan?

–How to estimate work duration?

Mentality / culture gaps: –How the balance the rhythm of business?

–What are the roles and responsibilities of the stakeholders?

–How to communicate “bad news” or feedback?

–How to keep the motivation high?

How to overcome technical knowledge gaps?



Common sense for working with overseas teams Differences I have observed and how to address them

Challenge How to address it?

Communication •Soften you communication and pay attention to proper words selection

•Align the terminology (e.g. severity level definition?)

•Manage disagreements tactfully

•Let the other side repeat and summarize what was understood and what are

the action items

•Publish written meeting minutes or mutual decisions

•There is no substitute for personal acquaintanceship (although the cost…)

Planning •Invest in explaining the overall context and motivation (don’t just explain the

raw task)

•Learn the local management style, share yours, and mutually find the

balance (e.g. how to manage buffets)

•Plan together and get early commitment

Knowledge •On ram-up stages of the remote team, review their deliverables before they

are publicly shared

•Invest in helping the remote team to build a good professional reputation

•Knowledge sharing should be part of your formal workplan

•Encourage local initiatives

Sometimes the value is in the difference…



Summary

Some tasks will be outsourced - we need to determine which are

best to keep here

Overseas teams are talented, yet your proximity to R&D and

flexibility enable you to offer unique added value

Stay agile, stay creative, leverage your strengths and do not fight to

retain ownership of the rest

Drive Testing (QC) as an integrated part of engineering.

Use the Globalization reality to scale the scope of what the

local Testing team brings to the organization.



Related resources:

Information about XIV: http://www.ibm.com/systems/storage/disk/xiv

IAIA: http://www.iaiaconsulting.com

“All That Testing is Getting in the Way of Quality” - James Whittaker:

http://www.youtube.com/watch?v=yOrPuMCdVXA

Please feel free to drop a note to me… [email protected]


http://www.iaiaconsulting.com/

© 2009 IBM Corporation © 2012 IBM Corporation 24


Thank You!



It’s all about perception…

How Israelis preserve

themselves

Direct outspoken & informal

Assertive Rude & impolite

Flexible about plans and casual

about rules

Creative, capable to improvise

Proactive, taking initiative

High self confident

Wary, alert realistic

Assertive Constantly arguing and lack of tact

Informal and direct

Flexible planning approach

Rude and impolite

Spontaneous Not respecting other people time

Not respecting rules

How Israelis perceive themselves?

How others may preserve us?


Documents

Shay Ganon - Manager, QA System Test Group Jul …legacy17.sela.co.il/s/sigist12/_AllPresentations/18July/...testing framework Thorough and deep white-box level testing of product