TEST Magazine - August-September 2009

Inside: Risk-based testing | Crowd testing | Testing as a service

James Christie takes the agile approach

I N T O U C H W I T H T E C H N O L O G Y

T H E E U R O P E A N S O F T W A R E T E S T E RVolume 1: Issue 3: September 2009

BRIDGING THE GAP

Supported by

T.E.S.T in now online at www.testmagazineonline.com

T.E.S.T

TH

E E

UR

OP

EA

N S

OF

TW

AR

E T

ES

TE

RV

OL

UM

E 1

: IS

SU

E 3

: SE

PT

EM

BE

R 2

00

9

www.testmagazineonline.com

Full-Time Quality Assurance Manager—Immediate Opening

Tes

tTra

ck® P

ro

Test

Trac

k® T

CM

Test

Trac

k® S

tudi

o Su

rrou

nd S

CM®

Seap

ine

CM®

QA

Wiz

ard®

Pro

Te

stTr

ack®

TCM

Iss

ue M

anag

emen

t Te

st C

ase

Man

agem

ent

Test

Pla

nnin

g &

Trac

king

Co

nfigu

ratio

n M

anag

emen

t Ch

ange

Man

agem

ent

Auto

mat

ed Te

stin

g Te

st C

ase

Man

agem

ent

www.seapine.com/testmag Satisfy your quality obsession.[ [

© 2009 Seapine Software, Inc. All rights reserved.

Don’t work yourself to death. Use TestTrack® TCM to manage your testing effort.

• Knowinstantlywhichtestcaseshavebeenexecuted,what yourcoverageis,andhowmuchtestingremains.

• Managesuitesofplatform-specificcompliancetests,functional tests,andperformancetestsinonecentrallocation.

• AssignteststoyourQAteam,trackresults,andreporton performanceandworkload.

TestTrack TCMputsyouincontroloftestcaseplanningandtracking,providingbettervisibilityoverthetestingeffortandgivingyoumoretimetomanageyourteam.WithTestTrackTCMyourteamcanwriteandmanagethousandsoftestcases,selectsetsofteststorunagainstbuilds,andprocessthepass/failresultsusingyourdevelopmentworkflow.

• Usetestvariantstotargetmultipleplatformswiththesame testcaseformoreefficienttestcasemanagement.

• StreamlinetheQA>Fix>Re-testcyclebypushingtestfailures immediatelyintothedefectmanagementworkflow.

• Achievecompletetraceabilitybetweentestcasesanddefects withseamlessTestTrackProintegration.

• Ensureallstepsareexecuted,andinthesameorder,formore consistenttesting.

September 09 | T.E.S.T www.testmagazineonline.com

Inside: Risk-based testing | Crowd testing | Testing as a service

James Christie takes the agile approach


T H E E U R O P E A N S O F T W A R E T E S T E RVolume 1: Issue 3: September 2009

BRIDGING THE GAP

Supported by

T.E.S.T in now online at www.testmagazineonline.com

T.E.S.T

TH

E E

UR

OP

EA

N S

OF

TW

AR

E T

ES

TE

RV

OL

UM

E 1

: IS

SU

E 2

: JU

NE

20

09

Twelve months ago we launched The European Software Tester (T.E.S.T) to an expectant industry. The

magazine was born into what it’s fair to say were very ‘interesting’ times. The fi nancial sector was still coming to terms with the credit crunch and the global economy was fi rmly in the grip of the worst recession since World War Two.

It will come as no surprise then that we were a little anxious that our newest venture might founder and the magazine would go the way of so many other brave but ultimately doomed periodicals. However, after only a relatively short period of time it became clear that T.E.S.T was not only desired by a largely voiceless industry, but was in extremely high demand from all quarters. Subscriptions came in thick and fast from Europe of course, but also from the Americas, India, and Australasia. Editorial content was submitted at such a rate it made my head spin – clearly the industry needed an outlet and cometh the hour, cometh the magazine!

This is now the fourth issue of T.E.S.T and we have covered an awful lot of ground over the last year. From agile to outsourcing, from risk-based testing to the black art of masking, obfuscating and scrambling, throughout we have strived to maintain the business-focus that we believe sets T.E.S.T apart from other titles and services that are focussed on the technicalities of testing. While this is undoubtedly very important, it is our business-led approach that we believe the industry not only wants but requires at the present time.

The next step is to take the project a stage further. Our aim has always been to publish T.E.S.T in as many formats as possible so with that in mind we are delighted to announce that as well as the quarterly printed magazines, T.E.S.T Online has now well and truly gone live. So for the latest news; in-depth thought leadership features; case studies; cutting edge opinion pieces; best practice and strategy articles; industry events news; vendor listings; digital archive; PDF versions of past issues; and a whole lot more, much of it exclusive to the website, you can now visit: www.testmagazineonline.com

We hope you enjoy the site and very much look forward to any comments or feedback you may have.

Happy surfing...

Matt Bailey, Editor

Leader | 1

Introducing T.E.S.T Online

Our aim has always been to publish T.E.S.T in as many formats

as possible so with that in mind we are delighted to announce that

as well as the quarterly printed magazines, T.E.S.T Online has now

well and truly gone live at: www.testmagazineonline.com

Matt Bailey, Editor

Editor Matthew [email protected] Tel: +44 (0)1293 934464

To advertise contact:Grant [email protected]: +44(0)1293 934461

Production & DesignDean Cook [email protected] Barrington [email protected]

Editorial & Advertising Enquiries 31 Media, Crawley Business Centre, Stephenson Way, Crawley, West Sussex, RH10 1TNTel: +44 (0) 870 863 6930Fax: +44 (0) 870 085 8837Email: [email protected] Web: www.testmagazineonline.com

Printed by Pensord, Tram Road, Pontllanfraith, Blackwood. NP12 2YA

© 2009 31 Media Limited. All rights reserved.

T.E.S.T Magazine is edited, designed, and published by 31 Media Limited. No part of T.E.S.T Magazine may be reproduced, transmitted, stored electronically, distributed, or copied, in whole or part without the prior written consent of the publisher. A reprint service is available.

Opinions expressed in this journal do not necessarily reflect those of the editor or T.E.S.T Magazine or its publisher, 31 Media Limited.

ISSN 2040-0160

Published by:

T H E E U R O P E A N S O F T W A R E T E S T E R

T.E.S.T | September 09 www.testmagazineonline.com

Simply visit www.testmagazine.co.uk/subscribe Or email [email protected]

The European Software Tester

SUBSCRIBE TO T.E.S.T.

Published by 31 Media Ltd

www.31media.co.uk

In Touch With Technology

Telephone: +44 (0) 870 863 6930

Facsimile: +44 (0) 870 085 8837

Email: [email protected]

Website: www.31media.co.uk

*Please note that subscription rates vary depending on geographical location

Inside: Delivering with agile; The future of software testing; Anarchy in the QAGetting it right with Risk-based testing Handling the risk


T H E E U R O P E A N S O F T W A R E T E S T E R

Sponsored by

Inside: Delivering with agile; The future of software testing; Anarchy in the QA

Inside: Delivering with agile; The future of software testing; Anarchy in the QAGetting it right with Risk-based testing

Getting it right with Risk-based testingHandling the riskHandling the risk


E S T E R


Contents | 3

1 Leader column Editor Matt Bailey announces the arrival of the all new T.E.S.T Online website.

4 Cover story – Bridging the gap with agile Independent consultant James Christie looks at the sometimes troubled relationship

between usability designers and software engineers and sees agile as a possible bridge

between the two.

8 Use the best to test John Lunt discusses the importance of software testing, the possible pitfalls and

challenges of outsourcing, and how to conduct it in-house.

12 A risky business Why does testing SAP need business process expertise and why has risk-based process

testing become a necessity? Matt Pickin explains.

16 Seven books that shaped my career Test consultant Phil Kirkham champions the cause of literacy on a journey through the

seven books that have shaped his career.

20 The sound of the crowd Testing bloggers Andréas Prins and Ewald Roodenrijs discus the emerging phenomenon

of crowd testing and its implications for the testing industry.

26 When requirements go bad In the first of a two-part series, Kurt Bittner shows how understanding the causes for

requirements errors can help prevent them.

30 A name to bank on Santander Consumer Bank AS in Norway is weathering the financial storm remarkably

well. Its services live and die by the functionality of their IT. T.E.S.T editor Matt Bailey

spoke to the bank’s Nordic region CTO Geir Berglind about its software testing activities.

34 STaaS in their eyes Many companies will tackle today’s economic challenges by deploying new soft-

ware, but how can they be sure it won’t let them down? Richard Terry explains how

software testing as a service (STaaS) could help.

38 Welcome to the new BCS David Clarke, chief executive of the BCS, explains the changes that have taken place

in the organisation, the direction in which it is now heading and the challenges that

lie ahead.

42 T.E.S.T Directory

48 The Last Word – Joel Montvelisky Joel Montvelisky searches his homeland of Costa Rica for some signs of a global

testing community and decides we’re not quite there yet.

CONTENTSSEP 09

20

30

4

36

SUBSCRIBE TO T.E.S.T.

4 | Test cover story

Independent software testing consultant James Christie looks at the sometimes troubled relationship between usability designers and software engineers and sees agile as a possible bridge between the two.

Bridging the gap with agile

For most of its history software engineering has had great difficulty building applications that users found enjoyable. Far

too many applications were frustrating and wasted users' time. That has slowly changed with the arrival of web developments, and I expect the spread of agile development to improve matters further.

I'm going to explain why I think developers have had difficulty building usable applications and why user interaction designers have struggled to get their ideas across. For simplicity I'll just refer to these designers and psychologists as UX. There are a host of labels and acronyms I could have used, and it really wouldn't have helped in a short article.

Why software engineering didn't get UXSoftware engineering's problems with usability go back to its roots, when geeks produced applications for fellow geeks. Gradually applications spread from the labs to commerce and government, but crucially users were wage slaves who had no say in whether they would use the application. If they hated it they could always find another job!

Gradually applications spread into the general population until the arrival of the Internet meant that anyone might be a user. Now it really mattered if users hated your application. They would be gone for good, into the arms of the competition.

Software engineering had great difficulty coming to terms with this. The methods it had traditionally used were poison to usability. The Waterfall lifecycle was particularly damaging.

The Waterfall had two massive flaws. At its root was the implicit assumption

www.testmagazineonline.comT.E.S.T | September 09


that you can tackle the requirements and design up front, before the build starts. This led to the second problem; iteration was effectively discouraged.

Users cannot know what they want till they've seen what is possible and what can work. In particular, UX needs iteration to let analysts and users build on their understanding of what is required.

The Waterfall meant that users could not see and feel what an application was like until acceptance testing at the end when it was too late to correct defects that could be dismissed as cosmetic.

The Waterfall was a project management approach to development, a means of keeping control, not building good products. This made perfect sense to organisations who wanted tight contracts and low costs.

The desire to keep control and make development a more predictable process explained the damaging attempt to turn software engineering into a profession akin to civil engineering.

So developers were sentenced to 20 years hard labour with structured methodologies, painstakingly creating an avalanche of documentation; moving from the current physical system, through the current logical system to a future logical system and finally a future physical system.

However, the guilty secret of software engineering was that translating requirements into a design wasn't just a difficult task that required a methodical approach; it's a fundamental problem for developers. It's not a problem specific to usability requirements, and it was never resolved in traditional techniques.

The mass of documentation

obscured the fact that crucial design decisions weren't flowing predictably and objectively from the requirements, but were made intuitively by the developers – people who by temperament and training were polar opposites of typical users.

Why UX didn't get software developmentNot surprisingly, given the massive documentation overhead of traditional techniques, and developers' propensity to pragmatically tailor and trim formal methods, the full process was seldom followed. What actually happened was more informal and opaque to outsiders.

Test cover story | 5

Software engineering's problems with usability go back to its roots, when geeks produced applications for fellow geeks. Gradually applications spread from the labs to commerce and government, but crucially users were wage slaves who had no say in whether they would use the application. If they hated it they could always find another job!

www.testmagazineonline.comT.E.S.T | September 09

6 | Test cover story

The UX profession understandably had great difficulty working out what was happening. Sadly they didn't even realise that they didn't get it. They were hampered by their naivety, their misplaced sense of the importance of UX and their counter-productive instinct to retain their independence from developers.

If developers had traditionally viewed functionality as a matter of what the organisation required, UX went to the other extreme and saw functionality as being about the individual user. Developers ignored the human aspect, but UX ignored commercial realities – always a fast track to irrelevance.

UX took software engineering at face value, tailoring techniques to fit what they thought should be happening rather than the reality. They blithely accepted the damaging concept that usability was all about the interface; that the interface was separate from the application.

This separability concept was flawed on three grounds. Conceptually it was wrong. It ignored the fact that the user experience depends on how the whole application works, not just the interface.

Architecturally it was wrong. Detailed technical design decisions can have a huge impact on the users. Finally separability was wrong organisationally. It left the UX profession stranded on the margins, in a ghetto, available for consultation at the end of the project, and then ignored when their findings were inconvenient.

An astonishing amount of research and effort went into justifying this fallacy, but the real reason UX bought the idea was that it seemed to liberate

them from software engineering. Developers could work on the boring guts of the application while UX designed a nice interface that could be bolted on at the end, ready for testing. However, this illusory freedom actually meant isolation and impotence.

The fallacy of separation encouraged reliance on usability testing at the end of the project, on summative testing to reveal defects that wouldn't be fixed, rather than formative testing to stop these defects being built in the first place.

There's an argument that there's no such thing as effective usability testing. If it takes place at the end it's too late to be effective. If it's effective then it's done iteratively during design, and it's just good design rather than testing.

So UX must be hooked into the development process. It must take place early enough to allow alternative designs to be evaluated. Users must therefore be involved early and often. Many people in UX accept this completely, though the separability fallacy is still alive and well. However, its days are surely numbered. I believe, and hope, that the agile movement will finally kill it.

Agile and UX – the perfect marriage?The mutual attraction of agile and UX isn't simply a case of ‘my enemy's enemy is my friend’. Certainly they do have a common enemy in the Waterfall, but each discipline really does need the other.

Agile gives UX the chance to hook into development, at the points where it needs to be involved to be effective. Sure, with the Waterfall it is possible to smuggle effective UX

If developers had traditionally viewed functionality as a matter of what the organisation required, UX went to the other extreme and saw functionality as being about the individual user. Developers ignored the human aspect, but UX ignored commercial realities – always a fast track to irrelevance.


Test cover story | 7

James Christie

Independent software

testing consultant

www.clarotesting.com

techniques into a project, but they go against the grain. It takes strong, clear-sighted project managers and test managers to make them work. The schedule and political pressures on these managers to stop wasting time iterating and to sign off each stage is huge.

If UX needs agile, then equally agile needs UX if it is to deliver high quality applications. The opening principle of the Agile Manifesto states that ‘Our highest priority is to satisfy the customer through early and continuous delivery of valuable software’.

There is nothing in agile that necessarily guarantees better usability, but if practitioners believe in that principle then they have to take UX seriously and use UX professionals to interpret users' real needs and desires. This is particularly important with web applications when developers don't have direct access to the end users.

There was considerable mutual suspicion between the two communities when agile first appeared. Agile was wary of UX's detailed analyses of the users, and suspected that this was a Waterfall-style big, up-front requirements phase.

UX meanwhile saw agile as a technical solution to a social, organisational problem and was rightly sceptical of claims that manual testing would be a thing of the past. Automated testing of the human interaction with the application is a contradiction.

Both sides have taken note of these criticisms. Many in UX now see the value in speeding up the user analysis and focussing on the most important user groups, and agile has recognised

the value of up-front analysis to help them understand the users.

The agile community is also taking a more rounded view of testing, and how UX can fit into the development process. In particular check out Crispin and Gregory's four agile testing quadrants. UX straddles quadrants two and three. Q2 contains the up-front analysis to shape the product, and Q3 contains the evaluation of the results. Both require manual testing, and use such classic UX tools as personas (fictional representative users) and wireframes (sketches of webpages).

Other people who're working on the boundary of agile and UX are Jeff Patton, Jared Spool and Anders Ramsay. They've come up with some great ideas, and it's well worth checking them out. Microsoft has also developed an interesting technique called the RITE method.

This is an exciting field. Agile changes the role of testers and requires them to learn new skills. The integration of UX into agile will make testing even more varied and interesting.

There will still be tension between UX and software professionals who've been used to working remotely from the users. However, agile should mean that this is a creative tension, with each group supporting and restraining the others.

This is a great opportunity. Testers will get the chance to help create great applications, rather than great documentation!

James Christie's comments about the relationship between developers and testers will appear at: www.testmagazineonline.com in October.

There will still be tension between UX and software professionals who've been used to working remotely from the users. However, agile should mean that this is a creative tension, with each group supporting and restraining the others.

This is a great opportunity. Testers will get the chance to help create great applications, rather than great documentation!


8 | Outsourcing testing

There is no point in companies investing large amounts of money in developing software when it can

potentially cause costly problems in the future. Testing is a vital part of the software development lifecycle and ensures that once installed it is error-free. Over-looking this important stage from the outset can seriously affect sales, customer satisfaction, the brand name, and potentially increase the workload to fix unscheduled problems. Although testing does take time and effort, it

reduces any unnecessary risk and cost in the future. In fact, fixing a problem is often more expensive if left until the product has gone to market. Prevention is better that cure!

Software Quality Assurance (SQA) is incredibly important to ensure products are designed and developed to a high quality that meets regulatory standards. A company that ignores this, runs the risk of non-tested software failing. As customers look at ways to modernise their back-office operations and IT infrastructure to support their growth, they look for

Use the best to testManaging director and co-founder of IT asset management specialist Certero, John Lunt, discusses the importance of software testing, the possible pitfalls and challenges of outsourcing, and how to conduct it in-house.

T.E.S.T | September 09


Outsourcing testing | 9

software that is reliable and fully compliant without risks attached. Products below quality assurance levels can put customers off and force them to look elsewhere for a solution. It is easy for an organisation to gain a negative reputation for providing unreliable software products and the hazards of launching a product that is not fully prepared can be catastrophic and cause irreversible damage.

Outside helpCompanies increasingly hire third parties to carry out testing on their behalf, particularly in tough financial times. Outsourcing is an attractive option as it means companies do not have the burden or costs associated with employing an in-house testing team or investing in internal testing technology. Third parties offer access to a skilled team of specialist testers who have the ability and knowledge to use specially designed software-testing tools.

Improved turnaround times and reduced software delays ensure

deadlines are met and product releases take place on time. When deciding to outsource testing, businesses need to determine how project delays affect their bottom line profits compared to the costs involved.

The pitfallsAlthough an enticing option, outsourcing does have its problems. Outsourcing needs constant management, which can be extremely time consuming and, in general, is the most common reason why outsourced projects fail. If a company does decide to outsource testing, it needs to dedicate a manager to be responsible for the relationship. This manager needs to have the capacity to be in constant communication with the supplier in order to build a close working relationship and effectively manage the project. In some instances, outsourcing can compromise the quality of testing, where a company can be left feeling dissatisfied. To avoid this, and ensure quality testing is delivered and continuous product improvements are met, the manager needs to check that the outsourced company understands the testing requirements at every level.

There are offshore software testing companies all around the world, including the UK, US and technology-rich countries such as Japan, India and Singapore. Geographical locations can create further frustrations when managing a relationship that spans from one country to another. This can increase workloads and can be more time consuming than working with someone in the same country. Time differences can put additional strain on the relationship and cause delays, especially when a problem needs resolving quickly. Constant communication can combat this and by arranging scheduled telephone meetings every day or week, issues can be identified and resolved in a timely manner.

Although testing does take time and effort, it reduces any unnecessary risk and cost in the future. In fact, fixing a problem is often more expensive if left until the product has gone to market. Prevention is better that cure!


10 | Outsourcing testing

In-house: the belt and braces approachConducted comprehensively, in-house testing should ensure quality and standards are met and should provide value for money. Not having an outside relationship to manage can also lead to a happier workforce.

If companies decide to carry out their software testing in-house, they need to embrace a ‘belt and braces’ approach to guarantee quality assurance. This involves a strict and uncompromising method - once a product, or an enhancement to an existing product, is made, then it needs to be tested by a separate testing function, not by the development function.

After passing all set criteria, the product should be driven through an external beta testing programme where companies can use a selected number of existing customers to identify any performance issues or bugs prior to an official product release. Software should never be released without beta testing, as it is impossible for developers to test their own products in all environments or predict how a new product will react with particular software and hardware. Only when this process is complete, should the software be made available for general release.

There is the perception that it is hard for companies to be critical of their own developments. However,

having two separate teams in place – one to write the software and one to conduct testing – means that products are examined exhaustively. Testing technicians that employ scripts and plans to check every software function, guarantee systematic analysis and precise examination.

Essential testingSoftware testing is an integral part of the development system and is not an afterthought. Whether a company decides to conduct software testing in-house or outsource to an external tester depends on its needs and resources. Software development companies should assess what they aim to get out of testing to help them evaluate the costs involved and help them determine what approach to take. Regardless of which route an organisation decides upon, software testing is essential. It enhances the quality of the product offering and ultimately, guarantees customer satisfaction.

What is the point of developing something that is unreliable and may cause future headaches? Software testing reduces unnecessary risks for the long term and ensures that companies are offering solutions they can be proud of. Producing a product that fails and is unreliable not only leaves you red faced, but it can damage your brand immensely.

Some considerations

Outsourcing: Do not outsource just because it is a financially attractive option. Think about what you want to get out of it before entering into a partnership with anyone.

Sourcing oversees help: How would you manage a long distance relationship from one country to another? Do you have someone that can dedicate time to managing this relationship?

Keeping it in-house: Remember to use a method that ensures you are critical of your own development and find those all important beta testers!John Lunt

Managing director

Certero

www.certero.co.uk


For exclusive news, features, opinion, comment, directory, digital archive and much more visit


The European Software Tester

The Whole Story

www.31media.co.uk

Print Digital Online


It’s becoming more and more apparent that IT and the business need to be better aligned if companies want to deliver

successful business services and achieve competitive advantage. With the right approach and methodology, testing will help a business achieve its objectives - going live ‘on time’ and ‘to budget’. In addition, if aligned correctly with the business, testing should drive improvements in business activities by ensuring the system delivered by the IT department is truly supporting the heart of the business. Companies that

adopt a risk-based process approach to their testing can see their total project timelines shorted by up to 25 percent leading to cost savings, internal resource liberation – driving both business agility and competitive advantage.

With this being the case it’s surprising that so few companies have recognised the role testing plays in achieving these business outcomes and still focus efforts on testing down at the most granular of levels ie, unit testing and basic integration tests. The reality within an SAP landscape, as within the majority of

12 | Risk-based testing

A risky businessMatt Pickin principle consultant at Experior explains why testing SAP requires business process expertise and why risk-based process testing has become a necessity.


Risk-based testing | 13

The reality within an SAP landscape, as within the majority of other application landscapes, is that if you are not testing your business processes then you are wasting time, effort and resource, while not properly protecting against failure and performance issues. In today’s economic climate, no organisation can afford to do this.

other application landscapes, is that if you are not testing your business processes then you are wasting time, effort and resource, while not properly protecting against failure and performance issues. In today’s economic climate, no organisation can afford to do this.

Testing is still seen by most people as costly and time consuming. This is hardly surprising when you consider the number of companies that have delayed go-live dates on multiple occasions due to failures identified in testing. Others have had to increase the spend associated with testing by up to three times original estimates due to Gantt Chart driven test plans or suffer stabilisation periods as long as two years because of poor application quality. Testing is supposed to remove these challenges for organisations, not add to them.

Companies frequently ask, “What should we test?” Surely, this basic question should be easily answered by every tester; you need to test what presents the business with the most risk, what truly matters to the business. Knowing how to determine this true understanding of business risk can only come from an understanding of the business processes. This will enable alignment of testing activities to critical business processes and drive efficiencies into the test cycle. This seems like common sense, but still you can witness approaches such as, 1) attempting to test everything 2) testing only what is allowed by project timelines, or 3) testing only the system, technology & code and not taking into account reality - the business processes.

Why testing has failed to meet the challenge of Business Process TestingBorn as a discipline of IT, testing is a victim of its own heritage and, from the start became overly focused on tools. The roles and skills required by testers along with the importance of intelligent testing methodologies have been neglected. Experior has a testing methodology that combines business process and analysis knowledge, with SAP Systems knowledge and testing expertise – ensuring that testing not only aids the business in reaching its objectives, but also achieves ROI faster.

In the past, businesses deploying ERP solutions demanded heavily bespoke systems leaving IT (and subsequently testing) focused on resolving technical integration issues. Removed from the business and skilled in test tool functionality, testers were compromised by their limited knowledge of processes, applications or the dependencies behind the data they were testing. The approach was inevitably orientated around technology. In testing SAP, business processes were over-looked in preference to individual transactions. The focus was largely by SAP module, with a limited appreciation of how each business process could span multiple modules or how processes had the potential to affect seemingly disparate ends of an organisation. Testers, focused on the merits of their test tools, lacked a knowledge or appreciation of business goals and could not deliver the predictable outcomes sought by the business.

Today we have to recognise that the key to unlocking the value in testing is through early involvement, understanding business processes and the potential knock-on effects that exist within companies’ unique business frameworks. In addition to possessing business understanding and acumen, testing experts need to understand how SAP works; the dependencies behind the data, its enterprise-wide integration and the technical challenges presented by NetWeaver and SOA. Only then will an individual’s technical understanding of test tools be of maximum value to testing SAP and the business. This blend of skill is not common but the rewards are well worth the effort.

What is risk-based process testing?One of the common mistakes is to take a blanket approach and trying to test everything. However, this is never an option; to replicate every variation of every activity that a business conducts in a test phase would always be too costly. A risk-based approach is not just desirable or the latest big idea, but a necessity. When you can’t test everything deciding what to test is crucial.

Once you have a thorough understanding of business and business process, only then can you


14 | Risk-based testing

have a true understanding of risk. Risk comes in the form of both a business impact, the impact of the system no longer supporting a particular route through the business process and the technical risk of any part of the system no longer working as specified. Combining these two elements together gives you the true picture of what presents the most risk to your business.

Risk-based process testing (RBPT) gives you the systems and processes to understand and manage your business risk combining the best of breed tools - IDS Scheer’s market leading ARIS Business Architect for SAP and ARIS Test Scenario Composer. Underpinned by, SAP Solution Manager and SAP Quality Center by HP with Experior’s SAP testing methodology.

How does risk-based process testing work?Business process is at the core of RBPT starting with a definition of your

business activities in SAP Solution Manager. This information is passed through to ARIS Business Architect for SAP where your business processes are mapped and enriched. Your business processes are then passed back into Solution Manager where your system is built to support your business processes. This should be related to standard solution design and system development project activity.

ARIS Test Scenario Composer can then be used to examine your business process maps and generate both your test requirements and your end-to-end process tests to be loaded as the foundation stones for the risk-based test approach in SAP Quality Center by HP. Test collateral is then built in Quality Center and linked to the business requirements.

In Quality Center you then complete your risk assessment - updating test requirements with both business impact and technical risks.

When changes are made to your solution, for example by installing a

Today we have to recognise that the key to unlocking the value in testing is through early involvement, understanding business processes and the potential knock-on effects that exist within companies’ unique business frameworks.

support or enhancement pack, SAP’s business process change analyser tool in Solution Manager will highlight the test requirements that are impacted. In Quality Center you then use the release functionality to group these impacted test requirements. The risk analysis in Quality Center is then used to determine the scope of your testing.

The industry-leading test management functionality in Quality Center can not only be used to execute your testing but to raise and manage defects too. This information can then be passed through to Solution Manager to give your project, development and test Leads the same view of the test information, creating a single version of the truth.

Not just another IT initiativeThe requirement to test should no longer be seen as just another IT initiative, it should be seen for its true value in the context of delivering continual business process


Risk-based testing | 15

Matt Pickin Principle Consultant Experiorwww.experiorgroup.com

STOCKHOLM,

30 NOV- 03 DEC

EuroSTAR 2009 is Europe’s Largest Software Testing Conference & Expo! It is currently in

it’s 17th year and includes an incredible line up of inspirational testing experts & the very latest

insights in world-class testing. Visit the website for full programme details.

EuroSTAR 2009Are you ‘Testing for Real?’Can you afford noT to be there?

Book before September 25th to receive a 10% discount

on conference fees. Register your test team now to avail of great Group Discounts!

EuroSTAR Online Community coming soon – Join the debate! Forum, Blog, Facebook, Archives, Resources and much more…

w w w. e u r o s t a r c o n f e r e n c e s . c o m

The theme for EuroSTAR 2009 is... ‘Testing for Real, Testing for Now‘

Send us your suggestion for the EuroSTAR 2010 theme to be in with a chance to win one of 2 FREE Conference places to Stockholm later this year!

Enter online before September 25th at...www.eurostarconferences.com/competitions/enter-testmag.aspx

improvement. Indeed, testing should be structured and viewed as accelerating project lifecycles and reducing costs – rather than as a beleaguered must-have that only adds to the expenditure and delay of projects. An effective testing approach validates the value of testing, aligning it with other corporate initiatives that continually enhance an organisation’s competitive advantage.

Whether it is technically-driven tests to optimise performance, ‘keep the lights on’ release packs or qualitative-driven exercises such as UAT, all testing must be designed with specific reference to validating and improving business process and mitigating risk. Businesses should start using methods that allow business analysts to reuse and automate test assets expediting test cycles while maintaining high levels of accuracy and consistency. When you adopt best practice methods such as these in

automation you can free-up resources formerly distracted with the mundane requisites of testing, allowing dedicated attention to the less certain, more critical areas of change within the business.

Risk based process testing solves common issues while mitigating risk, lowering costs, speeding up timelines, achieving faster ROI and reducing TCO.


16 | Testing literature

Seven books that shaped my career “The man who doesn't read good books has no advantage over the man who can't read them.” Test consultant Phil Kirkham ponders the wisdom of Mark Twain and wishes he’d bought shares in Amazon as he lists the seven books that have shaped his career.

I have always loved books, the 'Amazon recommends' feature always sucks me in and my wish list is now four pages long. Lee

Copeland in his Nine Forgettings talk bemoans the fact that the majority of testers haven't read a book on testing and with the recent increase in blog and online communities (hello Software Testing Club members!) and the Kindle from Amazon then maybe books are dying. Or maybe not, there is something about the feel of a book in your hands and the look of them on a bookshelf that is still very appealling.

A recent blog post by Michael Lopp (aka Rands in Repose, the author of Managing Humans), The Book Staker1, outlined his love of books and how he didn't really trust someone until he had seen their bookshelf. He also admitted to having seven precious books. As a great bibliophile, I loved reading it and it gave me the idea to list the seven books that have influenced me most.

1. Zen and the Art of Motorcycle Maintenance by Robert M PirsigI first read it when I was 14, just getting interested in motorbikes and found this copy on my brother’s bookshelf. I was disappointed to find it wasn't really about motorcycles and I didn't really understand it but there was just something about it. I still have

the now 30-year old dog-eared copy and keep finding it cropping up when reading testing sites and books. In the book, Pirsig explores the meaning and concepts of ‘quality’ – impossible to measure but its presence or absence is what makes something good or bad.

The quest for quality (hopefully not taken the Pirsig’s obsessive lengths) and how to measure it is an obvious one for all of us in this field - and so I think there should be the equivalent of Godwin’s Law – call it Pirsig’s Law – that states that a discussion about quality is over when someone drags out a quote from this book.

2. In Search of Excellence - Lessons from America's Best-Run Companies by Tom Peters and Robert WatermanFull of stories about how the top companies became top and what they do to stay there, how their management works and how they treat their employees. Being reasonably good at maths I knew the possibility of a lottery win and retirement to a desert island was very low so that meant I would be working for most of my life so why not try to get something from it. Plenty of tales in the book of wanting to produce an excellent product, an appreciation for the customer and attention to the small details – isn't that a job spec for a tester?

The quest for quality (hopefully not taken the Pirsig’s obsessive lengths)and how to measure it is an obvious one for all of us in this field - and so I think there should be the equivalent of Godwin’s Law – call it Pirsig’s Law – that states that a discussion about quality is over when someone drags out a quote from this book.


Test literature | 17

The book can be an interesting read now to see if the companies mentioned in it are still there and still being excellent.

3. Testing Computer Software, 2nd Edition by Cem Kaner, Jack Falk, and Hung Q. NguyenThe first testing book I read when I was thinking of moving from writing bugs to finding them. It gave me a huge shock as I realised that what we had called testing was nothing more than keyboard bashing. It gave me an idea of the complexity and potential that the testing field offered and was the push that led to me changing from being a developer who wrote defects to a being a tester who found them. My copy is now looking a bit sorry for itself as this is the book I usually hand out to newbie testers.

4. Everyday Scripting with Ruby: for Teams, Testers, and You by Brian MarickI was investigating Watir to try out some automation testing and this led me to looking at Ruby and hence this book. It showed me how automation could be more than just automating the program under test and could be used for other mundane monotonous tasks.

Ruby is a fun language to play around with and writing and running programs – even small quick ones – is a good reminder about what developers do and how easily mistakes can be introduced into a program.

5. Lean Software Development: An Agile Toolkit by Mary and Tom Poppendieck This got me thinking about agile and how wasteful some projects I had worked on had been. For example, two and a half years working with a team of top-notch developers to produce a program that sold less than 100 copies – which would be OK if each copy sold for a million pounds but this was supposed to be a shrink-wrapped mass production program.

This led me into finding out more about agile and Scrum which have grown tremendously since I first read this book. After reading this book I went on to learn about the Toyota


18 | Testing literature

Phil Kirkham Test consultant www.acutest.co.uk

No list of test/QA books could be complete without a Weinberg book. All seven of my selections herecould have been his books and the man himself is a fine example – he is in his 70s and still writing and still enthusiastic and keen to share his knowledge and experience.

Production System, the Theory of Constraints and I am currently reading up on one of the latest buzzes, Kanban. The arguments still continue on mailing lists and blog posts about whether producing a computer program is like engineering or art, can we apply lessons from building bridges or learn from how drama companies produce a play?

6. The Secrets of Consulting by Jerry WeinbergNo list of test/QA books could be complete without a Weinberg book. All seven of my selections here could have been his books and the man himself is a fine example – he is in his 70s and still writing and still enthusiastic and keen to share his knowledge and experience.

I settled for this one as I was moving into the consulting field and so needed to know about the Law of Raspberry Jam, the Orange Juice Test, Prescott's Pickle Principle and The Jiggler Role. As you might be able to tell from that list, the book is not a checklist of actions to be done – it makes you think about yourself, interactions with people and the role of a consultant.

7. Bridging the Communication Gap by Gojko AzdicSkillsmatter offers free courses and I attended one about agile testing presented by Antony Marcano. There is usually a small audience at these talks so it’s easy to get to know other people and I found myself outside a pub chatting with Antony and a

guy called Gojko Azdic. I'd heard of him through his work on Fitnesse, we exchanged email addresses and a while later I was one of several people helping review his latest work – Bridging the Communication Gap. An excellent read and I was flattered to find my name on the credit and a quote from me at the start. This has given me some inspiration for thinking about possibly writing a book myself – and this article could be the first small step. The book itself has been getting (quite rightly) excellent reviews and explains how specification by example and agile acceptance testing work and the problems they attempt to solve.

As you can see from the photo, my bookcase has far more than seven books in but the books listed above are the ones that have really made me think, been an influence on me – and now potentially others as I lend out my copies (I have become known as 'The Librarian' at work).

This article might now influence others. I'll have to keep an eye on the Amazon sales ranking to see if there is a spike of sales in any of the books listed!

Phil Kirkham's comment, War! What is it good for? Absolutely nothin’! appears on the new T.E.S.T Online website at www.testmagazineonline.com.

1www.randsinrepose.com/archives/2009/08/08/the_book_stalker.html



20 | Crowd testing

Nowadays we see huge changes in the world around us, borders are fading and the economy is

working around the clock. Much of this has been driven by the Internet and its inherent advantages.

The changes also influence our private lives – just think about the online communities where you participate and the social networks and the blogs you use for personal broadcasting - and have an impact on how we organise our work and the work around us.

And all this will change the way we test! Now we test based on risks, test coverage and/or budget, but if we deliver our testing activities to a crowd, they will test it for us and if they accept it and give their agreement the job is done!

The blurring of boundaries in our private life will have an impact on the boundaries we nowadays have between companies. A lot of people already use search engines and blogs to gain information, but formally we still do our work at our companies. Have you ever thought about how you

The sound of the crowd Testing bloggers Andréas Prins and Ewald Roodenrijs discuss the new phenomenon of crowd testing and share their vision of the way traditional testing is changing and how testing companies will fit into this developing picture.


Crowd testing | 21

A crowd consists of people all over the world from different background, cultures and companies. This potential group of billions can test for you and your client! So everybody in the world could potentially be part of the crowd.

will work in the future? What will it be like? And what will you do when the boundaries finally disappear?

Crowd testingWhat exactly is crowd testing and what does it do? Crowd testing is a form of crowd sourcing where you use both known and unknown people – the ‘crowd’ – to test software. In other words crowd testing is the use of a crowd of people and knowledge when testing software applications or systems. An important difference compared to traditional testing is that the test is focused on accepting the software instead of finding defects and has a different reward compared to the current models.

A crowd consists of people all over the world from different background, cultures and companies. This potential group of billions can test for you and your client! So everybody in the world could potentially be part of the crowd.

So others will test the software we testers normally test? What happens to my work?

If done properly, crowd testing shouldn’t be undertaken on software that hasn’t already been tested. It should be ready for acceptance. The professional testers need to have executed their system tests and can in fact go on to be participants in the crowd if they wish.

The crowdAn important part of crowd testing is the crowd itself and its makeup. Indeed the crowd is the success formula of crowd testing. So it is important to have the crowd you or the client wants for testing the application. So who is in the crowd? It consists of many different people who want to test. These people can be testers (beginners, experts and specialists), end users, or just interested people or potential users. It consists of your audience and if necessary paid testers.

When determining the use of the crowd it is important to know what they can or cannot do, why they want to test, what is their knowledge or background and origin. If this information is needed it needs to be available. This can be done by adding this information, and more, to a ‘crowd atlas’ which details who your crowd is and what it can do; so your crowd is a known quantity.

How do you build up this crowd? This is perhaps the most difficult task. The test crowd can be built up using existing networks like communities around crowd sourcing, around specific topics like OWASP or around networks like LinkedIn and can include all who are interested in a particular kind of software. These people can be friends, family, co-workers, competitors, or anybody else you may know. And


22 | Crowd testing

they – the crowd – are organised in a crowd atlas.

The reasons for crowd testingAfter answering the question ‘who is your crowd’, another question comes up: ‘Why do companies want to use the crowd for testing?’ The quick answer is to reduce costs. A crowd tester can be everybody and you don’t have to pay them as an expert (further down this article we’ll explain what the reward for the tester can be).

When you use the crowd to test you don’t have to pay expensive experts or buy expensive tooling and you can leave your in-company end users to do the work they’re paid to. But most of the time reducing costs isn’t the most important argument; with crowd testing the focus is on accepting the application, so it allows you to interact with the market and your audience.

The reason for a company to go for crowd testing is made up of three parts: company, people and product. And the interaction between these three parts is crucial. It is the interaction between company, people and product that makes crowd testing so desirable. Figure 3 shows the major reasons for crowd testing, but there are probably many more.

Activities for the crowdNow we know why we should use crowd testing, the next step is to find out what tests can be done. As previously stated, tests are not focused on the maximum coverage or finding defects but on accepting the software. From this perspective there are many tests that can be done.

First, where do you position crowd testing in the V-model or other development lifecycle models? Crowd testing is focused on accepting the product and therefore it is easy to position it as an acceptance test. When executing system or development tests it is much more difficult to use a crowd. It is, however, possible to work with a crowd for development or system tests, but

because of the accepting of the product you shouldn’t release a product that is full of bugs when crowd testing. When a product is still full of bugs, acceptation by the crowd will be less than anticipated. Of course there can still be defects, but the application does needs to work. When this isn’t the case, word can get round that the application is bad and full of bugs. That isn’t good marketing for the software and acceptance will be a long way off.

Activities that can be executed by the crowd include:• Testingwherealotoffeedbackis

required;• Testingwithnosensitivedataof

functionality;• Testingmulti-platformsoftware;• Testinglargeend-to-endsystems

over multiple companies;• Testinggames;• Testingbetasoftware.

Testing on multi-platforms is an activity that will become more and more important over the coming years. People are getting more flexible and have more software running. As a result there is a greater requirement for applications that can work on more than one platform, especially platforms like mobile phones.

End-to-end testing over multiple companies is a relatively new idea. Testing across companies brings many challenges, including the use of end users. The crowd can be build around the existing end users and they can reduce the challenge to get everybody involved on time at the same moment. When you have a dedicated crowd - with the right reward - the challenges are reduced and everybody is working towards the same goal.

A rewarding jobThere has to be a specific reward available for your dedicated testers out in the crowd, it is an important motivator to get them testing. As a company you have to answer the following question: Why will this crowd test for me?

There are people who want to try out a new application or new game.

FIG.3 REASONS FOR CROWD TESTING

FIG.4 MULTI-PLATFORM TESTING WITH MOBILE PHONES


Crowd testing | 23

In this case the games industry has an advantage over other software producers. People are more likely to try out games compared to administrative software etc. But with the right reward people can also be motivated to test business and admin applications. When people want, or need, to use specific software they will tend to help you with it. A good example is the beta testing of Windows or Google Apps. One of the motivations for people testing this software is that they want a good application in the end.

Everybody ideally wants to work for money. But if you compare crowd testing with crowd sourcing for example you see that the rewards can be, but are not always, non-financial. Rewards include:• Being part of a social network.

Because the (social) network needs to use an application, people in the network are pleased to test the software for the whole group. • Enjoyment of the tests. Testers

don’t always do those tests that are fun; they do the tests that are needed. But it can be fun to execute tests that you think are most

rewarding to you. So in the crowd you can do your own personal tests. • Income per test. A tester in the

crowd can be paid. They personally execute the tests and get money for it. • Interaction with different cultures.

Some people like to interact with different cultures. When the crowd expands globally, multiple cultures will be present. It can be rewarding to learn from them. • Get a fee if a product is sold.

Potential users test a product and when the product goes on the market they receive a fee for each product sold. • Competition to be the best.

When people want to be the best, working in a crowd is an excellent place to show their skills. With the friendly competition in the crowd it is possible to prove your talent. There is a star rating for the best testers in the competition and by executing more difficult tests their rating increases. Companies can ask the best testers with the highest rating to execute their tests for the best results.

Maybe the most valuable role for a test provider is to manage the crowd atlas which is linked to the rating system as described earlier. The test provider can use the atlas to see who has the capabilities and is available for a test. If you have a clear overview of the participants in the crowd then you have added value for the client.


24 | Crowd testing

• Sharing knowledge. People love to share their knowledge. They are glad to share what they know and receive knowledge back from others in return.• Charity. Charity is another strong

motivation for people to test for free. This can be with charitable organisations, but also as a gesture to programmers. Think about open source applications or freeware.• Discount on software’s release.

When a commercial product is finished, its testers receive a discount on that product.• Vouchers. Testing a product and

receiving vouchers to buy other products cheaper or for free from the same company can be a reward. This can also be linked with receiving the product for free after a certain test effort.

As a result it can be said that the reward for crowd testing can be a financial, but also non-financial, focused on being a member of a community or personal honours or developing own skills and doing something you enjoy. These rewards can be set in a diagram where the honour and financial rewards can be set.

Control of the testAfter all these things are taken into account, a new question comes up. For which sorts of software is crowd testing suitable? The answer is almost all, but there can be a problem with the sensitivity of the tested software. It can be difficult to convince a manager to let an unknown person test sensitive software. But this problem can be addressed by how you set out your crowd.

As mentioned earlier, the crowd can consist of a range of different often unknown people, but also known people. For example the crowd can comprise only the end users of the program. But, clearly, for classified software crowd testing is not a sensible option. Classified software or content needs a high level of control, for example, it is not a good idea to use a crowd to test a national security system; this should be done with screened testers using traditional testing methods.

But perhaps the most important part of crowd testing is the control of what is tested. With traditional testing

an analysis of what are the most important parts that need to be tested most thoroughly is done first. These parts are equally important in the crowd test so there should be a log of what is and isn’t tested available.

There is clearly a need to control what is tested. Control is implemented by tracking the tests or the presence of individual testers in the software. By doing this a client and/or test manager can exert a high degree of control on the test. Where certain important parts of the software are receiving less attention and other parts which are less important are tested thoroughly there should be ways to divert the crowd to the important parts.

The best way to do this is by removing the attractive, but unimportant part of the software altogether and attracting more attention to the important bits – there should be an option to ‘remove’ parts of the software without disrupting the application itself. To achieve all this there needs to be a way of tracking and analysing the behaviour of the crowd within the application.

The conditions for crowd testingNow you know who the crowd is; why a company may want to use a crowd; why the crowd wants to test; what to do to control the test; and how IT companies can use the crowd, the next step is to do it!

To use crowd testing there are conditions that need to be met to get the full potential. First, the crowd needs to be established. If there is no crowd available this should be addressed first. When there is a crowd and crowd atlas available, it needs to be determined which testers should be contacted to help with the testing. This is when a specific specialty or group is needed; for example, for security testing the client needs the best security testers in the crowd. Of course, other testers can also participate. Contacting them will be done later, when the software is almost ready for testing.

When the software is ready for testing a few prerequisites need to be met. One of these conditions has already been mentioned, the application needs to be divided into separate components which can work independently of one other. The reason for this is that parts of the software

FIG.5 REWARDS DIAGRAM

FIG.6 CONTROL CROWD TESTING

www.testmagazineonline.com September 09 | T.E.S.T

Crowd testing | 25

Ewald RoodenrijsTesting bloggerhttp://testingthefuture.net

will need to be removed from and inserted into certain parts of the application in order to control what needs to be tested and how the flow of the test proceeds.

An additional prerequisite is accessibility to the testing environment. Testers can come from around the world, working with every available system (if needed), so the testing environment should be accessible through the Internet which is the best option for all devices and applications to connect.

The system under test should also work well enough as it stands so that a crowd test can proceed. Crowd testing is about accepting the application and when the system works with a lot of (technical) faults the base for acceptation will be low. The crowd will also be less motivated to test the application if it is full of bugs. Thus there should be a pre-test on the test environment to prove that the application will run well enough for acceptance.

The future for testing companiesThe question at the end of this article is: how can existing testing companies earn their money with crowd testing? It is obviously unacceptable for them to post their personnel for free in a crowd just for honour or charity. Before we can answer the question we need to explain what makes testing companies special:• Theyhavetheirowntest

methodologies;• Theyknowtheirclientsandthe

requirements they have for testing;• Testexpertsknoweverythingabout

the right test strategy and test coverage;• Testcommunitieshaveyearsof

experience with defect management, test infrastructure, end-to-end testing, etc.

In a nutshell, testing companies have to provide a crowd testing service to their clients when they ask to use a crowd. With this role they can earn their money!

The test provider can be a link between the current hierarchic organisations and the crowd. Currently, organisations aren’t aware of how the network model of the crowd works and don t have the contacts from the hierarchic model in the network.

Therefore a third party can arrange the contacts and divide the work in the network model. The testing provider must know the crowd and the entities who participate in the crowd.

Other profitable jobs for the test provider could be the management of the test infrastructure, offering tooling for the registration of the crowd, and defect management, all of which can be managed from a central position. The advantage of managing this all from a central position is that it allows you to make reports and status updates for the client.

Managing the star rating system – giving the best tester the highest rating – however, is not a task for the test provider. The rating needs to be done by the crowd itself or by the client. Ratings are awarded when the client is satisfied with the work of the test crowd. Making this ratings system available to the client can eventually be a task for the test provider though.

Maybe the most valuable role for a test provider is to manage the crowd atlas which is linked to the rating system as described earlier. The test provider can use the atlas to see who has the capabilities and is available for a test. If you have a clear overview of the participants in the crowd then you have added value for the client.

Andréas and Ewald’s blog can be found at: http://testingthefuture.net

Andréas Prins Testing bloggerhttp://testingthefuture.net

FIG.7 TASKS FOR THE TEST SERVICE PROVIDER


26 | Testing requirements

In the first of a two-part series, Kurt Bittner, chief technology officer at Ivar Jacobsen International shows how understanding the causes for requirements errors can help prevent them.

When requirements go bad

That error in requirements is a contributing cause to the failure of many projects is nothing new. It is easy to

attribute the problem to lack of clarity in the requirements - that if only the ambiguities could be eliminated all would be well.

Ambiguous, unclear or incomplete requirements are serious problems, to be sure, but they are not the sole source of requirements errors. Requirements errors have many sources, however, and ambiguous or unclear requirements are only part of the problem. In order to reduce requirements errors, it's worth exploring a number of common types of errors to find their source and strategies for dealing with them. For the purposes of discussion, I find it useful to group requirements errors into three major categories:

- errors of conception - errors of specification - errors of implementation

Each one of these types of errors has separate symptoms and causes, and each must be solved in a different way.

What is a requirement, really? Before talking about requirements errors, it's worth pausing for a moment to consider what requirements are. A requirement is a statement that describes a desirable, or sometimes a mandatory, aspect of a solution. The degree to which the requirement is mandatory is often the subject of lengthy scoping debates.

The observation that a requirement describes a solution is important, but at first this statement probably seems innocuous and perhaps even, well, obvious. The importance of recognising a requirement as

In the extreme case they may actually be contradictory, much like the old logic teaser: The following statement is true. The previous statement is false. Taken individually, each of these statements is clear and unambiguous, but taken together they are nonsensical. It is easily possible for requirements to exhibit the same qualities.



Testing requirements | 27

describing a solution comes from the possibility that the solution that the person writing the requirement has in mind may not actually be the only solution, let alone the best solution, to the problem at hand. If more than one person is writing requirements it is possible that each person actually has a slightly different mental picture of the solution, which means that even if individual requirements are clear and unambiguous, taken as a whole the requirements may not be consistent with one another.

In the extreme case they may actually be contradictory, much like the old logic teaser: The following statement is true. The previous statement is false. Taken individually, each of these statements is clear and unambiguous, but taken together they are nonsensical. It is easily possible for requirements to exhibit the same qualities.

Some teams try to look at requirements as a complete and unambiguous definition of the solution to be delivered. In today's world where a premium is typically placed on speedy delivery of a "good enough" solution, this perspective is often impractical. It is virtually impossible to specify every interesting aspect of a solution; just as it is impossible to completely and precisely describe even the simplest of chemical compounds, it is also impossible to completely and precisely describe every interesting aspect of something as complex as a software application. Even if it were possible to do, it would be prohibitively expensive to do so.

This leads to something more pragmatic: the requirements of a solution are the means by which the stakeholders and the development team arrive at a shared understanding of what it is the project is trying to

do. The requirements themselves can be useful in facilitating interesting discussions and documenting understanding, but the important thing is the communication that occurs, not the requirements themselves.

The view that requirements are a complete and unambiguous definition of the solution derives from the use of requirements as a kind of contract. This may be appropriate in cases where an external firm is being hired to build the system, but there are typically contractual mechanisms for accomplishing this. Contracts in legal settings are most useful as establishing the terms for lawsuits.

In the context of software development, if you have to resort to ‘contractual commitments’ the project has typically failed and the participants are simply trying to lay blame. It is better to work more proactively to achieve success, using the requirements as a communication vehicle rather than as a contractual document. In this respect not all requirements have to be stated in an absolutely context-free and unambiguous manner. As long as there is a consensus about what is needed, some requirements can be less formal and open to creative alternatives, whereas some other requirements must be absolute.

As an example, the exact look and feel of the user interface often presents a great opportunity for creative innovation, but the financial calculations performed by a billing system are absolute and not open to creative interpretation.

Treating all requirements with the same rigor is a waste of time and may squelch creativity. The key is to understand what must be precise and what can be less formal, and make

In the context of software development, if you have to resort to ‘contractual commitments’ the project has typically failed and the participants are simply trying to lay blame. It is better to work more proactively to achieve success, using the requirements as a communication vehicle rather than as a contractual document.


28 | Testing requirements

sure that everyone understands the decision criteria. Armed with this perspective, we can now turn our attention to different ways in which this communication can break down.

Errors of conception Errors of Conception occur when the requirement is poorly conceived, when it solves the wrong problem or is based on flawed assumptions about the solution. These types of errors usually occur because the objectives of the solution are poorly understood. When these errors are present, the resulting requirements many be unambiguous and clear, but nevertheless wrong. As a result of these errors, even when the implementation is sound and of high quality, the resulting solution is of little or no value. It is in cases such as this where resorting to the defence that “the requirements were satisfied” is of little consolation. Symptoms of errors of conceptions include: - lack of clarity in the solution goals or the problem being solved;- unused or unneeded features; - unneeded complexity.

There have been some interesting studies that suggest that this is actually a fairly large problem. These studies have shown that up to 50 percent of consumer electronics product returns are due to complexity - users could not figure out how to use the products (but presumably wanted the functionality that the product provided). The important thing about this is that significant effort was expended to create the capabilities that people were unable to use, or perhaps did not even want.

Other data from the Standish Group's Chaos Report suggests that even for successful projects, up to 30 percent of the function they delivered were unneeded or unused. Clearly someone is not clear about what is really needed in these solutions.

Root causes and avoidance strategies It would be easy, but wrong, to write these errors off as simply due to poor communication. Poor communication is always present, but in most cases of errors of conception, someone is

actively communicating the wrong information.

The usual source of the problem is talking to the wrong people, not finding the right subject matter expert with the right perspective to define the right solution. A story about a failed project illustrates the point: A customer service system was developed and deployed at great expense. The leader of the requirements effort was very keen to use the latest windowing technologies, with lots of sophisticated capabilities to provide expert advice on how to handle different kinds of inquiries. The system was released to great fanfare and was, surprisingly, a complete failure.

It turned out that the customer service representatives who used the system were accustomed to using an old CRT system that had keystroke-accelerator commands for frequently performed actions. In addition, the staff were extensively trained on customer scenarios and had years of industry experience and did not need or welcome the intrusive guidance on how to do their jobs. As a result of all the extra mouse-clicks and distractions of the new system, call handling throughput actually declined and the old system was eventually turned back on.

What went wrong here? Basically, the ‘expert’ was solving problems that did not need to be solved and had ignored the real needs of the intended users. Because the requirements dictated the ‘wrong’ solution and were not questioned, the development team built the wrong solution. A couple of things could have been done to avoid this problem: - First, find the right subject matter expert. Since the development team rarely has the necessary business subject matter expertise, finding the right person is essential. In a sense, you should do enough investigation of the business problem to have a sense when purported experts really know what they are talking about. - Second, get more than one opinion. There should be a consensus among the stakeholders about the nature of the problem being solved, and as



Testing requirements | 29

Kurt BittnerChief Technology Officer Ivar Jacobsen International

www.ivarjacobson.com

the definition of the solution evolves you should continuously validate the "concept" of the solution with the stakeholders. - Third, this continuous validation can't occur if you are heads-down writing requirements documents. You need to capture the ideas simply and succinctly and communicate them frequently.

An old adage to remember is that users don't know what they want, but they know what they don't want when they see it. This means that you will probably have to develop a number of different prototypes to walk-through various aspects of the system in order to make sure that you are building the right solution.

Other sources for errors of conception relate to poor understanding of the needs of the business (or sponsoring organisation if the solution is being developed for a not-for-profit organisation or government agency), which really means the needs of the stakeholders for the project. By this I mean going beyond what they say they need and uncovering the real needs that often lie hidden.

An example from another domain illustrates the difference: A patient walks into a doctor's office saying that he needs to get a prescription for painkillers. When the doctor asks why, the patient complains of severe headaches, but insists that it's no big deal, he just needs to get something to dull the pain and he doesn't need more tests or treatments. Against these protestations the doctor insists on more tests and eventually finds that the patient has an operable but benign tumour. After the operation, the patient's headaches disappear completely and he makes a speedy recovery.

In this example, giving the patient what he is asking for is exactly the wrong course of treatment. Similarly, if you give your users exactly what they ask for you may simply be treating the symptoms of a problem and fail to find a better solution to the real problem at hand. Users are not very good at coming up with solutions to their problems – that's where you can add real value – and letting them ‘dictate’

a solution by dictating requirements usually ends up with everyone less than satisfied.

There are a variety of techniques to breaking through the initial requirements posed by the users, but they all centre on focusing on uncovering the needs of the stakeholders and not, at least initially, soliciting requirements from them.

Eventually you will uncover requirements, and as you do you need to relate them directly back to needs; if the connection to needs is weak then eliminate the requirement - it's either not needed (or there is a need that you are not clear about).

A final problem can occur that is a kind of ‘error of conception’: sometimes the problem to be solved or the needs are clear, but it's just not a problem that is worth solving. This situation occurs when the cost of developing a solution outweighs the benefits, or the solution is simply technically infeasible.

It's best to figure this out early, and so focussing early on identifying needs and then exploring the cost and technical feasibility of solutions through one or more prototypes is the best way to figure this out quickly.

Disambiguation Errors of conception are the most significant kind of requirement errors, and they are the hardest to detect, but avoiding them has the greatest benefit. By preventing them you ensure that the solution will better meet the real needs of stakeholders; failing to prevent them, no amount of ‘disambiguation’ of requirements language will yield a good result.

No amount of precision can help a requirement that specifies the wrong behaviour. In this first article in a two-part series, I have covered the ‘errors of conception’, which relate to requirements errors that arise from problems with the basic conception of the solution. In the next issue I will discuss ‘errors of specification’, which relate to errors in how the requirement is expressed, as well as ‘errors of implementation’, which arise from problems in how the requirements are implemented.

Studies have shown that up to 50 percent of consumer electronics product returns are due to complexity - users could not figure out how to use the products (but presumably wanted the functionality that the product provided). The important thing about this is that significant effort was expended to create the capabilities that people were unable to use, or perhaps did not even want.


Few would deny that the banks have been having a rocky time of it recently. There can’t be many of the larger

global financial institutions that have been left untouched by the travails of the credit crunch and the fallout from the global economic downturn. And despite the proverbial ‘green shoots’ of recovery that may or may not be starting to sprout as I write, many in the industry are still wary of announcing a slowdown in these tough times, let alone any more permanent good news.

One institution that has weathered the storm remarkably well however is Santander. Indeed, in 2008 when many a once rock-solid name in the banking world was going under, it became the third largest international bank in the world based on profits. With over 14,000 branches (more than any other international bank) in more than 40 countries and over 90 million

customers, Santander grew at an astonishing rate.

Banking on technologyIn a world where money is more often than not ‘virtual’, existing only as code inside the electronic brains of increasingly complex financial systems, it’s no exaggeration to say that the role of the software is critical. To emphasise just how critical, take a moment to recall the panic that surrounded the ‘millennium bug’ and its implications a decade ago. Now, factor in how far things have come since then and you will get an idea of the scale of the challenges faced by the testing professionals inside large financial organisations.

Geir Berglind, CTO of Santander Consumer Bank AS (SCB), took time out of his busy globetrotting schedule to tell us a little about the company and its software testing activities. “We are a part of Grupo Santander

Take a moment to recall the panic that surrounded the ‘millennium bug’ and its implications a decade ago. Now, factor in how far things have come since then and you will get an idea of the scale of the challenges faced by the testing professionals inside large financial organisations.

Santander Consumer Bank AS in Norway is weathering the financial storm remarkably well. Like all financial institutions in the modern world, its services fundamentally live and die by the functionality of their IT. T.E.S.T editor Matt Bailey spoke to the bank’s Nordic region CTO Geir Berglind about its software testing activities.

A name to bank on

30 | Customer profile



Customer profile | 31

Over the past 14 years Seapine has grown from its initial two employees to over 100, and has added offices in London, Munich, and Melbourne. With all of this growth, the company says it has remained faithful to its internal culture and a philosophy of exceeding customer expectations.

which is one of the largest banks in the world,” he said. “SCB Nordic, which covers Norway, Sweden, Finland and Denmark, has the aim of becoming the leading Nordic car, leisure and consumer finance company. In fact, our goal is to be one of the leading consumer finance companies anywhere in the world. As a step towards achieving this strategy, we have recently concentrated on broadening our product and service range as well as expanding our operations in all of the Nordic countries. In 2007, we established companies in Finland and Denmark providing a full range of Santander Consumer Bank’s products and services.”

Berglind’s role as CTO gives him responsibility for much of the group’s IT infrastructure. The role of Seapine Software’s testing solutions in the organisation has been crucial since before he joined the company. “I

am the Nordic CTO and this gives me responsibility for architectural guidance, infrastructure, tools and so on,” he explained. “Actually Seapine products were already present when I started working in SCB two years ago. We are currently using TestTrack Pro, Surround SCM, TestTrack TCM, and QA Wizard Pro from Seapine.”

As an application lifecycle management (ALM) solution provider, Seapine offers many tools to support the application development lifecycle. Specifically, it offers QA Wizard Pro for automated functional and regression testing, Surround SCM for software change management, TestTrack RM for requirements management, TestTrack Pro for issue and development process management, and TestTrack TCM for test case planning and management. It also offers multiple levels of training for all products, custom software development, data conversion, and process consulting services.


32 | Customer profile

“Integration is a key feature of our products,” says Rick Riccetti. “TestTrack Pro and Surround SCM support linking issues and change requests with source code changes for improved traceability of software changes. QA Wizard Pro integrates with TestTrack Pro to automate pushing test failures into users’ defect tracking workflow. QA Wizard Pro also integrates with TestTrack TCM so users can link test scripts with TestTrack TCM tests cases and automate running tests. TestTrack Pro and TestTrack TCM are integrated in a single application so defects can be traced to the test that generated them. Defects can also be promoted to new test cases to help improve test coverage.”

“TestTrack Pro has been used in SCB for quite a while,” comments Berglind. “It is currently used for bug tracking on production systems and in ongoing development projects. It is used as a tool both internally in the IT/Project departments as well as a tool for communicating with external developers. Most of our IT development is outsourced.”

Seapine’s solutions are designed to help quality assurance organisations create, enforce, automate, track, and measure development and QA processes. Beginning with TestTrack TCM, they give companies a specialised tool which helps them manage the thousands of test cases needed to test today’s complex applications. And, as already stressed, applications in the financial sector can be massive and complex.

With TestTrack Pro, companies like Santander can have a solution for managing issues found during software testing to ensure they are resolved or at least considered in future builds. QA Wizard Pro is a platform for automating functional and regression testing and Seapine brings expertise to companies to help them implement a formal automated testing programme.

“Surround SCM is of course used for source control, both in our Java and .Net environments; it is also connected

to CruiseControl for daily builds,” confirms Berglind. “TestTrack TCM and QA Wizard Pro are new products for us, and we are currently exploring ways to use them in our organisation.”

A partner in qualityAfter fourteen years in the business of supplying solutions and training to software testers, Cincinnati-based Seapine Software can rightfully claim to be one of the pioneers in the sector. The company specialises in helping customers like Santander Consumer Bank AS identify the problems associated with developing quality products and implementing cost-effective solutions to those challenges. These solutions can come in many forms, such as consulting or tools, or a combination of both.

Seapine is a privately-owned corporation headquartered in Cincinnati, Ohio in the USA. It was founded by husband and wife team Rick and Kelly Riccetti in 1995. The company’s founding mission was to reduce the complexity of software development tools, allowing developers and quality assurance professionals more time to focus on their tasks in hand—producing quality software. And that remains its mission to this day.

Over the past 14 years Seapine has grown from its initial two employees



Customer profile | 33

Geir BerglindNordic region CTO Santander Consumer Bank AS Norway

to over 100, and has added offices in London, Munich, and Melbourne. With all of this growth, the company says it has remained faithful to its internal culture and a philosophy of exceeding customer expectations.

Integration“Integration is a key feature of our products,” says Rick Riccetti. “TestTrack Pro and Surround SCM support linking issues and change requests with source code changes for improved traceability of software changes. QA Wizard Pro integrates with TestTrack Pro to automate pushing test failures into users’ defect tracking workflow. QA Wizard Pro also integrates with TestTrack TCM so users can link test scripts with TestTrack TCM tests cases and automate running tests. TestTrack Pro and TestTrack TCM are integrated in a single application so defects can be traced to the test that generated them. Defects can also be promoted to new test cases to help improve test coverage.”

TestTrack RM, Seapine’s new requirements management solution, rounds out their ALM offering by providing end-to-end traceability of requirements, issues, and tests. With TestTrack RM, customers can easily manage the complete requirement lifecycle, including planning,

workflow, traceability, review, change management, and reporting.

The company also offers the Quality-Ready Assessment (QRA). According to Seapine, this online assessment helps companies measure their product development capabilities in the four key areas of testing, change control, tracking, and automation. After completing the assessment, the survey results and recommendations for improving any noted deficiencies are emailed to the respondent within a few minutes.

On the subject of the current economic uncertainty, Rick Riccetti is upbeat, seeing it as an ideal opportunity to automate. “Economically turbulent times are a wakeup call for companies to improve efficiencies across the organisation,” says Riccetti. “For software development companies and IT teams, test automation can have the biggest impact in helping them accomplish more with their QA resources. So there is a real opportunity to introduce automated testing to companies. However, these tools have a higher cost-per-seat than many other tools and cash flow is being watched closely. The challenge (and opportunity) is to demonstrate to companies how quickly an ROI can be realised on the automated testing investment.”

Santander Consumer Bank AS Norway history

1963 ELCON Leasing established as one of the first finance companies in Norway

1982 ELCON Leasing changes name to ELCON Finans through a merge with Inter Finans

2000 ELCON establishes business in Sweden

2001 ELCON establishes business in Denmark

2001 Bankia Bank established

2004 ELCON is sold to Santander Consumer Finance, a company within Grupo Santander, Spain.

2005 ELCON demerges through separation and sale of factoring- and equipment leasing part of the business to Societe Generale, France

2005 Santander Consumer Finance acquires Bankia Bank which merges with ELCON

2005 Through this merger, the company changes name to Santander Consumer Bank

2006 Santander Consumer Bank establishes business in Finland

2007 Santander Consumer Bank establishes business in Denmark

34 | Feature


34 | Testing as a service

Many companies will tackle today’s economic challenges by deploying new software, but how can they be sure it won’t let them down? Richard Terry, Deputy CEO for Sogeti, explains how software testing as a service (STaaS) could help.

STaaS in their eyes

Software can transform a business: it empowers customers to serve themselves, helps marketers to fi nd the

right prospects, and makes sure everybody gets paid at the end of the month. When it fails, though, it can transform the business in an altogether different way. Software applications are woven into the fabric of every business. If that fabric tears, the business is no longer covered for even its most basic activities.

As companies face up to the challenging economy, many will look at how they can use software to cut their costs and improve their effectiveness. At the same time, they must minimise the risk of software failure.

The big pictureApplications, be they custom or standard packages, cannot just be dropped into an organisation’s IT infrastructure. They need to be tested so that performance can be measured, security holes and bugs identified – in short, to make sure it’s

actually doing what it’s meant to as efficiently as possible. A business that bypasses the testing stage is risking an underperforming application, full of glitches and security threats that can have a ripple effect on the whole IT infrastructure. And that can lead to something far more damaging – a loss in shareholder confidence.

A lot of organisations already use in-house teams to test their IT, but in a difficult economy this is maybe not an ideal way of doing things. In the majority of enterprises, the testing team will experience periods where

they have nothing to do. Sure, the people can be repurposed for other jobs, or they can just sit around playing Tetris. But hiring people that will not do what they’re there for, for much of the time, is not a good strategy when budgets are tight. Unless a lot of applications need testing consistently throughout the year, an organisation is unlikely to have enough work to keep an in-house testing team busy.

Testing as a serviceA new model has emerged for outsourcing software testing, called

A business that bypasses the testing stage is risking an underperforming application, full of glitches and security threats that can have a ripple effect on the whole IT infrastructure. And that can lead to something far more damaging – a loss in shareholder confidence.



Testing as a service | 35

Businesses need to react swiftly to changes in an unpredictable market. They need to be able to counter competitor moves. They need to able to bring new products or services to market as soon as possible. In short, with the world as it is, businesses need results, and they need them yesterday.

‘Software testing as a service’ (STaaS). It’s inspired by software as a service, a model where software is provided over the internet, requiring minimal infrastructure and enabling rapid start-up and low maintenance operation. STaaS makes software testing similarly accessible, enabling projects to begin quickly, and making it viable for smaller software testing projects to be undertaken.

By outsourcing testing, a business can minimise software risks and control costs, ensuring that it only spends money on application testing when it needs to. Studies have shown that outsourced software testing can cut the cost of testing by 25 percent or more.

It’s not just for companies without an in-house team either. STaaS can help to smooth the cost of an in-house team by creating flexible capacity. In some larger enterprises, the in-house testing team could find itself in a position where it is unable to efficiently manage the volume of testing required at times. The business could hire more staff,

but when the workload decreases to manageable levels again, it’s going to need to keep paying them. STaaS means this long-term investment is no longer necessary to address the short term bottleneck. An outsourced testing professional can work with the in-house testing team to help them clear the workload much faster. Fast responseOne of the reasons that traditional software testing is often expensive is that it can take a long time to get underway. Setting up can be a process that can sometimes take months. And when things do finally get underway, the results can be a long time coming. In many cases too long – businesses need to react swiftly to changes in an unpredictable market. They need to be able to counter competitor moves. They need to able to bring new products or services to market as soon as possible. In short, with the world as it is, businesses need results, and they need them yesterday.

Outsourced testing specialists have dedicated, industry-approved testing environments, which are ready to go


36 | Testing as a service

STaaS will deliver cost and time savings to many businesses, at a time when they need them most. But it does have its limitations. If you’re looking to deploy an expansive enterprise-wide application, such as Siebel CRM or SAP ERP, STaaS would certainly not be the best way to go.

Richard Terry Deputy CEO Sogetiwww.uk.sogeti.com

at any time. As a result, for the vast majority of applications, tests can be started in as little as two days. This, of course, means that the results can be delivered in a much shorter time-frame than with the traditional testing model. STaaS ensures that testing doesn’t cause a costly bottleneck during application deployment, which is one less headache for the organisation, and allows the business much greater operational flexibility.

The specialised infrastructure, test environment, and methodology that testing outsourcers offer also mean that applications can be tested, even if the business has little or no documentation. In that case, the outsourcer could even help the organisation to start producing the critical documentation that will support future tests.

STaaS will deliver cost and time savings to many businesses, at a time when they need them most. But it does have its limitations. If you’re looking to deploy an expansive enterprise-wide application, such as

Siebel CRM or SAP ERP, STaaS would certainly not be the best way to go.

Losing controlSome organisations might be uncomfortable with the idea of handing over control of their testing. What these companies may not realise is that, although the testing processes are handled by a third party, the relationship is still collaborative. Before starting tests, a reliable STaaS provider will visit the company and ensure it has a complete understanding of the organisation’s systems. The provider will make certain it knows what the customer wants and expects, so it can ensure this is what it delivers. Through collaboration, the testing project can benefit from a combination of testing experience and a deep understanding of the customer’s corporate culture and systems.

The recession is a challenging time for many companies, but smart businesses can save themselves a lot time and money by letting someone else do the software testing for them.


38 | Feature


38 | Testing qualifi cations

BCS was created in 1957 to bring together a number of different groups to try and deliver on the grand dream of

making all aspects of computing into a compelling public benefi t. We have made great strides since then, and BCS has evolved continually to meet the ever changing world we serve, never more so than in the last few years.

TransformationAs we evolve, we see the world changing quicker than ever, and we know this pace of change will only increase. We must, therefore, respond and adapt quickly and more broadly to be relevant to the world we now live in so that we are ready for the future. We need not just to change BCS but to completely

transform it, and this month we are launching ‘BCS, The Chartered Institute for IT’, with the aim not just of serving the ever changing world of the IT/computing professional in all its many guises, but more broadly ‘Enabling the Information Society’ as an international force.

More than a million people work directly in information technology in the UK alone and computer science lies behind almost everything that happens in society today. To really support not only these knowledge workers but an increasingly IT-dependent society as a whole, our activities and capabilities have to become even more relevant to our members, the academic community, industry, government, volunteers and the wider public.

David Clarke, chief executive, explains the changes that have taken place in the BCS, the direction in which the organisation is now heading and the challenges that lie ahead.

Welcome to the new BCS

Today’s economy is built on knowledge and it’s on our ability to process, share and manage information that our society thrives or dies. How well the UK does these activities will in the end determine our financial standing. This information is the most vital commodity we possess.

Feature sponsored by:


Testing qualifi cations | 39

KnowledgeToday’s economy is built on knowledge and it’s on our ability to process, share and manage information that our society thrives or dies. How well the UK does these activities will in the end determine our financial standing. This information is the most vital commodity we possess.

We increasingly rely upon it to protect us from crime and terror; diagnose and treat us when we are ill; monitor and mitigate our impact on a wounded planet; provide open access to goods and services to fulfil our human needs; help us to save and spend our money wisely and profitably; collect, allocate and redistribute our taxes; ensure efficient and transparent markets; and work more effectively and efficiently on a daily basis. Ultimately, we rely upon information-based technologies to support the most human need of all, to communicate: sharing our needs, desires and aspirations with others.

Information societyIT could reasonably be called the first truly global profession, and the world needs an organisation that leads an IT profession fit to drive the information society, not just to follow it. BCS, The Chartered Institute for IT, is intent on becoming that organisation, and this re-launch is our next major step in a long-term plan to achieve this position.

To put IT to work, we must actively address a range of social integration challenges:The digital divide: Contrary to popular belief, the digital divide has not closed. We too easily mistake broadband access for information literacy. A real

and persistent gulf exists between the information haves and have-nots.Information vulnerability: From identity theft to benefit fraud, money laundering and deception – information is still not secure.Lax information management: Data is still not highly valued by the public, by modern institutions or even by our profession.Project failure: Many public and private IT projects still fail to deliver: too often, and too often on a grand scale.Skills shortages: Nationally, and even globally, too few people are training in IT and too few are trained in the underlying fundamentals of computer science to solve our pressing problems. Lack of a clear career path: Finally, too few of those who do train actually make it up the chain of command into management whether inside or outside the IT function. We have developed a clear strategy based around five specific pillars, which you will see much of in the coming years, in order to address these challenges. They will support:1. Bridging the gap between education

practice and research;2. Giving practitioners the professional

development and career support they deserve;

3. Informing public policy on how IT can contribute to society;

4. Ensuring everyone benefits from IT;5. Championing the global IT

profession.This new BCS will take shape

between now and the end of 2010, with a wide range of new qualifications, products and services all supported by a brand new website

(check out www.bcs.org) offering simplified sign-up, subscription management, better industry-academia contact, state of the art knowledge sharing and career development tools.

The challenge and the opportunity for BCS is clear. The UK and the world need an IT profession that understands its social impact and has a long-term strategy to address it.

Software testingCertification for software testers, which began with the launch of the ISEB Foundation Certificate in 1998, has grown steadily in importance in the UK and around the world ever since. Employers have embraced the qualifications as evidence of effort and attainment, through improved communications between software development and testing professionals, and in greater consistency of testing practice, and many have incorporated a level of certification into the criteria for software testing jobs.

Training providers have enjoyed a steady stream of attendees for their accredited certificate courses. Individual testers have benefited from the enhanced status of software testing as a discipline and from the codification of bodies of knowledge in the form of syllabi and glossaries to support them.

ISEB Qualifi cationsISEB has a comprehensive portfolio of qualifications, of which software testing is one of the most popular in terms of certificates awarded. In recent times ISEB has recognized the value of treating its qualifications as a

IT could reasonably be called the first truly global profession, and the world needs an organisation that leads an IT profession fit to drive the information society, not just to follow it. BCS, The Chartered Institute for IT, is intent on becoming that organisation, and this re-launch is our next major step in a long-term plan to achieve this position.



40 | Testing qualifi cations

coherent portfolio with the potential for multiple routes that can both broaden and enrich qualifications; at the same time it has concentrated effort on the quality and consistency of its operations. The ISEB qualification program supports the Continuous Professional Development that is so important to all IT professionals.

The software testing qualifications have been leaders in this process, with well documented and effectively managed examination processes that ensure complete consistency wherever the qualifications are offered in the world. Some ISEB Foundation and Intermediate exams are also available online via Prometric and Pearson VUE.

InternationalThe international IT certification market is very much influenced by the IT industry. Many certificates are offered by the well known IT vendors. Some domains are more influenced by vendor neutral certification, such as the ITIL certification which dominates the market of service management.

Since the beginning of the

certification of software testers by ISEB in 1998, the software testing market has developed into an international market. The International Software Testing Qualifications Board (ISTQB) was established as a result of this growth.

ISTQB has adopted a strategy of encouraging as much international participation in its scheme as possible, each member country having its own national board. These national boards work together in a loosely federated structure. ISEB can be positioned as an international certification body, working closely together with national boards, especially the UK test board and the Dutch and Belgian test boards. The original foundation exam has grown into a global exam, translated into many languages and delivered through a variety of channels. ISEB is market leader in this area.

ISEB examinations are set centrally and translated as necessary and appropriate for an international audience. The same processes and quality standards apply to all examinations. The ISEB model of quality has been steadily improved

over the years, and is now well documented and subject to regular audit, both internally and by external bodies.

The quality and consistency of ISEB processes, and the clear benefits of acquiring ISEB certificates for both individuals and organisations, especially where organisations operate internationally and need consistency for all their areas of operation, has helped to maintain and enhanced the reputation of the software testing qualifications.

Software Testing FoundationThe Foundation Certificate continues to be a popular qualification. It provides not only a broad-based introduction to the software testing specialism, but also a sound introduction to the terminology and basic approach of software development in its various forms. The average pass rate is 80 percent and most candidates pass the exam at the end of a three-day accredited course.

For many the Foundation Certificate meets all their needs, but there is a significant number who wish or



Testing qualifi cations | 41

need to progress to a more in-depth understanding of how testing works. The syllabus is maintained on a regular basis and the programme continues to attract many participants. The Foundation certificate is by far the most popular, also in comparison with some other software testing qualifications available in the market. ISEB has accredited over 30 training providers to offer the foundation level training for software testing.

The higher levels of the software testing programmeThe training on the Foundation level is very much an introduction. Those interested in more in-depth knowledge and a practical perspective will find the ISEB Intermediate Certificate a great next step. The Intermediate Certificate explains how the principles of testing are actually applied in the context of real software projects. The Intermediate Certificate prepares its holders with a sound practical knowledge of the testing discipline from which it is possible to develop the skills and knowledge of a software testing professional.

The Intermediate level was launched in Autumn 2007 and since then 2,700 candidates have taken the examination.

The Practitioner level currently provides the knowledge and skills that effectively define the more senior roles in software testing.

ISTQB has introduced the advanced certificate for software testing which includes three specialist disciplines within software testing. The Advanced level covers all three of these disciplines.

In time, the various streams of software testing certification for the higher professional levels are likely to be harmonised so that valuable experience can be harnessed for the benefit of testers around the world.

The examsIn setting the objectives for its certificates, ISEB has made extensive use of Bloom’s taxonomy, a scale of six levels of knowledge labelled K1 to K6, which provides criteria to characterise each level and offer guidance for examiners setting

questions at each level. ISEB has adopted an assessment strategy that begins with a relatively broad examination at the Foundation level, the examination, lasts one hour and contains 40 multiple choice questions with a defined mixture of questions at the lower (K1 – K3) levels and a balance of syllabus topics.

The Intermediate level is a one hour examination with 25 multiple choice scenario-based questions. There is a defined structure of questions at the K2 – K4 levels and a balance of syllabus topics. The scenarios provide context for practical questions requiring some analysis of the scenario to select correct answers. At the current highest level, the two Practitioner qualifications are each examined in a two hour written examination at the K2 – K6 levels.

At the ISTQB Advanced level the required knowledge is in the K2 – K4 band and is examined during a three hour multiple choice examination with 65 weighted questions, some of which are scenario-based.

Unmatched experienceISEB’s long and unmatched experience across a spectrum of IT disciplines, coupled with its commitment to software testing certification and a strong desire to enhance and extend its certification scheme in cooperation with the corporate market, the training providers and organisations such as ISTQB, make ISEB a powerful partner for any organisation.

Leading the IT professionThese are exciting and important times for us. As the natural leader of the IT profession, responsible for awarding the much sought-after Chartered IT Professional (CITP) status, we must stand up and act, as a hugely respected and independent professional body, for the benefit of the entire industry.

This now embraces all professionals and practitioners from software engineers to business analysts, and the critical world of academia with its computer science foundations for all technology. Across the globe. Join us in enabling the information society.

These are exciting and important times for us. As the natural leader of the IT profession, responsible for awarding the much sought-after Chartered IT Professional (CITP) status, we must stand up and act, as a hugely respected and independent professional body, for the benefit of the entire industry.

David Clarke Chief Executive BCSwww.bcs.org



The Information Systems Examinations Board (ISEB) is part of the BCS, the Charted Institute for IT, and is an international examination body created to raise the standard of competence and performance of people working in IT. We’re leading the way in qualifications for IT professionals – delivering more than 380,000 exams in over 200 countries.

Our qualifications are internationally recognised and cover eight major subject areas in: Software Testing, ITIL /IT Service Management, IT Assets and Infrastructure, Systems Development, Business Analysis, Project Management, IT Governance, Information and Security and our new qualification Sustainable IT.

These are available at Foundation, Practitioner and Higher Level to suit each individual candidate. ISEB Professional Level is also available.

For more information visit www.iseb-exams.com.

These qualifications are delivered via a network of high quality accredited training and examination providers. The breadth and depth of our portfolio is one of its key strengths as it encourages knowledge, understanding and application in specific business and IT areas.

Candidates develop their competence, ability and aptitude – and therefore their professional potential – giving employers the edge they’re looking for

BCS

BCS, The Chartered Institute for IT, promotes wider social and economic progress through the advancement of information technology science and practice.

We bring together industry, academics, practitioners and government to share knowledge, promote new thinking, inform the design of new curricula, shape public policy and inform the public. As the professional membership and accreditation body for IT, we serve over 70,000 members including practitioners, academics and students, in the UK and internationally. A leading IT qualification body, we also offer a range of widely recognised professional and end-user qualifications.

BCS membership for software testers

BCS membership gives you an important edge; it shows you are serious about your career in IT and are committed to your own professional development, confirming your status as an IT practitioner of the highest integrity.

Our growing range of services and benefits are designed to be directly relevant at every stage of your career.

Industry recognition

Post-nominals – AMBCS, MBCS, FBCS & CITP – are recognised worldwide, giving you industry status and setting you apart from your peers.

BCS received its Royal Charter in 1984 and is currently the only awarding body for Chartered IT Professional (CITP) status, also offering a route to related Chartered registrations, CEng and CSci.

Membership grades

Professional membership (MBCS) is our main professional entry grade and the route to Chartered(CITP) status.

Professional membership is for competent IT practitioners who typically have five or more years of IT work experience. Relevant qualifications, eg a computing-related degree, reduce this requirement to two or three years of experience. Associate membership (AMBCS) is available for those just beginning their career in IT, requiring just one year’s experience.

Joining is straightforward – for more information visit:www.bcs.org/membership where you can apply online or download an application form.

Best practice

By signing up to our Code of Conduct and Code of Good Practice, you declare your concern for public interest and your commitment to keeping pace with the increasing expectations and requirements of your profession.

Networking opportunities

Our 44 branches, 16 international sections and over 40 specialist groups including Software Testing (SIGIST) and Methods & Tools, provide access to a wealth of experience and expertise. These unrivalled networking opportunities help you to keep abreast of current developments, discuss topical issues and make useful contacts.

Specialist Group in Software Testing (SIGIST)

With over 2,500 members SIGIST is the largest specialist group in the BCS. Objectives of the group include promoting the importance of software testing, developing the awareness of the industry’s best practice and promoting and developing high standards and professionalism in software testing. For more information please visit: www.sigist.org.uk.

Information services

The BCS online library is another invaluable resource for IT professionals, comprising over 200 e-books plus Forrester reports and EBSCO databases. BCS members also receive a 20 percent discount on all BCS book publications. This includes Software Testing, an ISEB Foundation and Intermediate. As well as explaining the basic steps of the testing process and how to perform effective tests, this book provides an overview of different techniques, both dynamic and static, and how to apply them.

Career development

A host of career development tools are available through BCS including full access to SFIA (the Skills Framework for the Information Age) which details the necessary skills and training required to progress your career.

ISEB

BCS, First Floor, Block D, North Star House, North Star Avenue, Swindon, SN2 1FA, United KingdomTel: +44 (0) 1793 417655 Fax: +44 (0) 1793 417559 Email: [email protected] Web: www.iseb-exams.com

42 | T.E.S.T company profile


T.E.S.T company profile | 43

www.seapine.com

United Kingdom, Ireland, and Benelux: Seapine Software Ltd. Building 3, Chiswick Park, 566 Chiswick High Road, Chiswick, London, W4 5YA UKPhone:+44 (0) 208-899-6775 Email: [email protected]

Americas (Corporate Headquarters): Seapine Software, Inc. 5412 Courseview Drive, Suite 200, Mason, Ohio 45040 USA Phone: 513-754-1655

With over 8,500 customers worldwide, Seapine Software Inc is a recognised, award-winning, leading provider of quality-centric application lifecycle management (ALM) solutions. With headquarters in Cincinnati, Ohio and offices in London, Melbourne, and Munich, Seapine is uniquely positioned to directly provide sales, support, and services around the world.

Built on flexible architectures using open standards, Seapine Software’s cross-platform ALM tools support industry best practices, integrate into all popular development environments, and run on Microsoft Windows, Linux, Sun Solaris, and Apple Macintosh platforms.

Seapine Software's integrated software development and testing tools streamline your development and QA processes—improving quality, and saving you significant time and money.

TestTrack RM

TestTrack RM centralises requirements management, enabling all stakeholders to stay informed of new requirements, participate in the review process, and understand the impact of changes on their deliverables. Easy to install, use, and maintain, TestTrack RM features comprehensive workflow and process automation, easy customisability, advanced filters and reports, and role-based security. Whether as a standalone tool or part of Seapine’s integrated ALM solution, TestTrack RM helps teams keep development projects on track by facilitating collaboration, automating traceability, and satisfying compliance needs.

TestTrack Pro

TestTrack Pro is a powerful, configurable, and easy to use issue management solution that tracks and manages defects, feature requests, change requests, and other work items. Its timesaving communication and reporting features keep team members informed and on schedule. TestTrack Pro supports MS SQL Server, Oracle, and other ODBC databases, and its open interface is easy to integrate into your development and customer support processes.

TestTrack TCM

TestTrack TCM, a highly scalable, cross-platform test case management solution, manages all areas of the software testing process including test case creation, scheduling, execution, measurement, and reporting. Easy to install, use, and maintain, TestTrack TCM features comprehensive workflow and process automation, easy customisability, advanced filters and reports, and role-based security. Reporting and graphing tools, along with user-definable data filters, allow you to easily measure the progress and quality of your testing effort.

QA Wizard Pro

QA Wizard Pro completely automates the functional and regression testing of Web, Windows, and Java applications, helping quality assurance teams increase test coverage. Featuring a next-generation scripting language, QA Wizard Pro includes advanced object searching, smart matching a global application repository, data-driven testing support, validation checkpoints, and built-in debugging. QA Wizard Pro can be used to test popular languages and technologies like C#, VB.NET, C++, Win32, Qt, AJAX, ActiveX, JavaScript, HTML, Delphi, Java, and Infragistics Windows Forms controls.

Surround SCM

Surround SCM, Seapine’s cross-platform software configuration management solution, controls access to source files and other development assets, and tracks changes over time. All data is stored in industry-standard relational database management systems for greater security, scalability, data management, and reporting. Surround SCM’s change automation, caching proxy server, labels, and virtual branching tools streamline parallel development and provide complete control over the software change process.

Seapine SoftwareTM



For over 20 years Parasoft has been studying how to

efficiently create quality computer code. Our solutions

leverage this research to deliver automated quality

assurance as a continuous process throughout the SDLC.

This promotes strong code foundations, solid functional

components, and robust business processes. Whether you

are delivering Service-Orientated Architectures (SOA),

evolving legacy systems, or improving quality processes

– draw on our expertise and award winning products to

increase productivity and the quality of your business

applications.

Parasoft's full-lifecycle quality platform ensures secure,

reliable, compliant business processes. It was built from

the ground up to prevent errors involving the integrated

components—as well as reduce the complexity of testing

in today's distributed, heterogeneous environments.

What we do

Parasoft's SOA solution allows you to discover and

augment expectations around design/development

policy and test case creation. These defined policies are

automatically enforced, allowing your development team

to prevent errors instead of finding and fixing them later

in the cycle. This significantly increases team productivity

and consistency.

End-to-end testing:

Continuously validate all critical aspects of complex

transactions which may extend through web interfaces,

backend services, ESBs, databases, and everything in

between.

Advanced web app testing:

Guide the team in developing robust, noiseless regression

tests for rich and highly-dynamic browser-based

applications.

Application behavior virtualisation:

Automatically emulate the behavior of services, then

deploys them across multiple environments – streamlining

collaborative development and testing activities. Services

can be emulated from functional tests or actual runtime

environment data.

Load/performance testing:

Verify application performance and functionality under

heavy load. Existing end-to-end functional tests are

leveraged for load testing, removing the barrier to

comprehensive and continuous performance monitoring.

Specialised platform support:

Access and execute tests against a variety of platforms

(AmberPoint, HP, IBM, Microsoft, Oracle/BEA, Progress

Sonic, Software AG/webMethods, TIBCO).

Security testing:

Prevent security vulnerabilities through penetration

testing and execution of complex authentication,

encryption,

and access control test scenarios.

Trace code execution:

Provide seamless integration between SOA layers by

identifying, isolating, and replaying actions in a

multi-layered system.

Continuous regression testing:

Validate that business processes continuously meet

expectations across multiple layers of heterogeneous

systems. This reduces the risk of change and enables rapid

and agile responses to business demands.

Multi-layer verification:

Ensure that all aspects of the application meet uniform

expectations around security, reliability, performance, and

maintainability.

Policy enforcement:

Provide governance and policy-validation for composite

applications in BPM, SOA, and cloud environments to

ensure interoperability and consistency across all SOA

layers.

Please contact us to arrange either a one to one briefing

session or a free evaluation.

Web: www.parasoft.com Email: [email protected] Tel: +44 (0) 208 263 6005

ParasoftImproving productivity by delivering quality as a continuous process.


T.E.S.T company profi le | 45

TechExcel is the leader in unifi ed Application Lifecycle

Management as well as Support and Service solutions

that bridge the divide between product development and

service/support.

This unifi cation enables enterprises to focus on the

strategic goals of product design, project planning,

development and testing, while enabling transparent

visibility with all customer-facing initiatives.

TechExcel has over 1,500 customers in 45 countries and

maintains offi ces in UK, US, China and Japan.

Application Lifecycle Management DevSuite

is built around the best-practices insight that knowledge

is central to any product development initiative. By

eliminating the silos of knowledge that exist between

different teams and in different locales, DevSuite helps

enterprises transform their development processes,

increasing effi ciency and overall quality.

DevSpec DevSpec is an integrated requirements

management solution that is specifi cally designed to

provide visibility, traceability and validation of your

product or project requirements. DevSpec provides a

framework to create new requirements, specifi cations and

features that can be linked to development and testing

implementation projects.

DevPlan DevPlan is a project, resource, and task

management tool. It allows users to plan high level areas

of work, assign team members to work in these areas, and

then track the tasks needed to complete the activities.

DevTrack DevTrack is the leading project issue and

defect tracking tool that is used by development teams

of all sizes around the globe. Its confi gurable workfl ows

allow DevTrack to meet the needs of any organisation's

development processes.

DevTest From test case creation, planning and

execution through defect submission and resolution,

DevTest tracks and manages the complete quality lifecycle.

DevTest combines the test management features of

DevTest, DevTrack and TestLink for test automation into

one integrated solution.

KnowledgeWise KnowledgeWise is the knowledge

management solution at the core of the entire suite.

It is the centralised knowledge base for all company

documents including: contracts, processes, planning

information and other important records as well as

customer-facing articles, FAQs, technical manuals and

installation guides.

More information at: www.techexcel.com/products/

devsuite.

Service and Support Management Service and

Support Management solutions provide enterprises with

total visibility and actionable intelligence for all service

desk, asset management and CRM business processes.

ServiceWise ServiceWise is a customisable and

comprehensive internal Helpdesk, ITSM- and ITIL-

compliant solution. Automate and streamline services

and helpdesk activities with confi gurable workfl ows,

process management, email notifi cations and a searchable

knowledge base. The self-service portal includes online

incident submission, status updates, online conversations

and a knowledgebase. ServiceWise includes modules such

as incident management, problem escalation and analysis,

change management and asset management.

CustomerWise CustomerWise is an integrated CRM

solution focused on customer service throughout the

entire customer lifecycle. CustomerWise allows you to

refi ne sales, customer service and support processes to

increase cross-team communication and effi ciency while

reducing your overall costs. Combine sophisticated process

automation, knowledgebase management, workfl ow, and

customer self-service to improve business processes that

translate into better customer relationships.

AssetWise AssetWise aids the process of monitoring,

controlling and accounting for assets throughout their

lifecycle. A single and centralised location enables

businesses to monitor all assets including company IT

assets, managing asset inventories, and tracking customer-

owned assets.

FormWise FormWise is a web-based form

management solution for ServiceWise and CustomerWise.

Create fully customised online forms and integrate them

directly with your workfl ow processes. Forms can even

be routed automatically to the appropriate individuals

for completion, approval, and processing, improving your

team's effi ciency. Web-based forms may be integrated

into existing websites to improve customer interactions

including customer profi ling, surveys, product registration,

feedback, and more.

DownloadPlus DownloadPlus is an easy-to-use

website management application for monitoring fi le

downloads and analysing website download activities.

DownloadPlus does not require any programming or

HTML. DownloadPlus provides controlled download

management for all downloadable fi les, from software

products and documentation, to marketing materials and

multimedia fi les.

More information at: www.techexcel.com/products/itsm/

Training

Further your investment with TechExcel, effective training

is essential to getting the most from an organisation's

investment in products and people. We deliver professional

instructor-led training courses on every aspect of

implementation and use of all TechExcel’s software

solutions as well as both service management and industry

training. We are also a Service Desk institute accredited

training partner and deliver their certifi cation courses.

More information at: www.techexcel.com/support/

techexceluniversity/servicetraining.html

For more information, visit www.techexcel.com or call 0207 470 5650.

TechExcel



31 Media31 Media is a business to business media company that publishes high quality magazines and organises dynamic

events across various market sectors. As a young, vibrant, and forward thinking company we are flexible, proactive, and

responsive to our customers' needs.

www.31media.co.uk

T.E.S.T OnlineSince its launch in 2008 T.E.S.T has rapidly established itself as the leading European magazine in the software testing

market. T.E.S.T is a publication that aims to give a true reflection of the issues affecting the software testing market.

What this means is that the content is challenging but informative, pragmatic yet inspirational and includes, but is not

limited to: In-depth thought leadership; Customer case studies; News; Cutting edge opinion pieces; Best practice and

strategy articles

The good news is that the T.E.S.T website, T.E.S.T Online has had a root and branch overhaul and now contains a

complete archive of previous issues as well as exclusive web-only content and testing and IT news.

At T.E.S.T our mission is to show the importance of software testing in modern business and capture the current state

of the market for the reader.

www.testmagazine.co.uk

VitAL MagazineVitAL is a journal for directors and senior managers who are concerned about the business issues surrounding the

implementation of IT and the impact it has on their customers. Today senior management are starting to realise

that implementing IT effectively has a positive impact on the internal and external customer and it also influences

profitability. VitAL magazine was launched to help ease the process.

www.vital-mag.net

Customer MagazineCustomer Magazine was launched to address and assist with the various challenges senior professionals face when

establishing a customer-centric business. Our editorial takes a pragmatic approach to what has become a series of

complex issues and delivers dynamic, provocative, and insightful articles, case studies, opinion pieces, and news stories

that not only challenge our readers but also bring clarity and vision to the many challenges they face.

www.customermagazine.co.uk

31 Media www.31media.co.uk [email protected] Crawley Business Centre, Stephenson Way, Crawley, West Sussex, RH10 1TN, United Kingdom

Phone: +44 (0) 870 863 6930 Fax: +44 (0) 870 085 8837


T.E.S.T company profile | 47

iTrinegyNetwork emulation & application testing toolsiTrinegy is Europe’s leading producer of network emulator technology which enables testers and QA specialists to conduct realistic pre-deployment testing in order to confirm that an application is going to behave satisfactorily when placed in the final production network.

Delivering more realistic testingIncreasingly, applications are being delivered over wide area networks (WANs), wireless LANs (WLAN), GPRS, 3G, satellite networks etc, where network characteristics such as bandwidth, latency, jitter and packet error or loss can have a big impact on their performance. So, there is a growing need to test software in these environments.

iTrinegy Network Emulators enable you to quickly and easily recreate a wide range of network environments for testing applications, including VoIP, in the test lab or even at your desktop.

Ease of useOur network emulators have been developed for ease of use:

• Noneedtobeanetworkexpertinordertousethem• Pre-suppliedwithanextensiverangeofpredefinedtestnetworkscenariostogetyoustarted• Easytocreateyourowncustomtestscenarios• Alltestscenarioscanbesavedforsubsequentreuse• Automatedchangesinnetworkconditionscanbeappliedtoreflecttherealworld• Workseamlesslywithloadgenerationandperformancetoolstofurtherenhancesoftwaretesting.

A comprehensive range to suit your needsiTrinegy’s comprehensive range of network emulators is designed to suit your needs and budget. It includes:

• Softwareforinstallationonyourowndesktoporlaptop(trialcopiesavailable)• Small,portableinlineemulatorsthatsitsilentlyonthedesktopandcanbesharedamongstthetestteam• Largerportableunitscapableofeasilyrecreatingcomplexmulti-path,multi-site,multi-usernetworksforfull

enterprise testing• Highperformancerack-mountunitsdesignedtobeinstalledindedicatedtestlabs• Veryhighperformanceunitscapableofreplicatinghighspeed,highvolumenetworksmakingthemidealfortesting

applications in converged environments.

If you would like more information on how our technology can help you ensure the software you are testing is ‘WAN-

ready’ and going to work in the field, please contact iTrinegy using the details below:

Email: [email protected] Tel: +44 (0)1799 543 345 Web: www.itrinegy.com/testmagazine

The Software Testing Club is a relaxed yet professional place for software testers to hang out, find likeminded software testers and get involved in thoughtful and often fun conversations.

Interesting things happen at The Software Testing Club. It started out as an experiment. Now two years on it has turned into vibrant online community of software testing professionals.

You'll find members are dedicated to their profession and you can find them in deep conversation within the forums. However, it's more than just forums and your standard niche social network. As the club grows we are finding things happening. This includes things like a Job Board, a Mentoring Group, a collaborative Software Testing Book and a crowd sourced testing initiative called Flash Mob Testing.

The Software Testing Club is a grassroots effort. It's for the members and grows according to what we believe they want.

Come join and let us know what you think.

Rosie Sherry – Founder & Community Manager Email: [email protected] Tel: +44 (0)7730952537 Web: www.softwaretestingclub.com

The Software Testing Club


48 | The Last Word

Joel Montvelisky of Practitest searches his homeland of Costa Rica for some signs of a global testing community.

We’re on our way to a global testing community, but not there yet…

I was back in my homeland of Costa Rica during August this year. Even though I visit there every couple of years this was the fi rst time – after

15 years living in Israel and working in software testing and QA – that I decided to learn more about the status of testing in the local professional environment.

I got in touch with the National Chamber of IT and Communications and organised a seminar on software testing and testing Intelligence. The seminar sold-out within a couple of days, so there clearly was an audience for the subject of testing, but what caught my attention was that over half this audience were not testers but project managers and business analysts.

A different evolutionary pathDuring the seminar and in the meetings I had after the event I learned a couple of interesting things:1. The reality for Costa Rica is very similar to that of the whole Central American region (and still relevant for a vast number of South American countries);2. Many of the working assumptions I had taken for trivial during my work as a tester were less trivial in these parts of the World.

I understood that because of a different evolutionary path and maybe even a cultural gap, the subject of software testing had developed less and in a different way than in most of the places I had visited and worked in so far.

For example, in many local companies testing was something coordinated by the project managers that utilised end users or subject matter experts to validate that the software did what was expected from it. This is what I would call UATs or user acceptance tests in my process, but the rest of the testing tasks and operations appeared to be missing from the process.

I don’t want to be misunderstood; these countries are not falling short on their technology or engineering skills. They have very competent developers working with the latest and greatest technologies creating good-looking software products. There are also some companies in these countries where they use a very robust testing process, but these are isolated firms that mostly belong to or are closely linked to international corporations that bring with them their end-to-end processes.

As I already stated, I think this is the result of a separate evolution taken by the software market in these countries. Places where ten or 15 years ago they had little or no local software development and operations had created a local development culture from scratch in a very short time, managing to close the par in the technical levels but leaving the methodological aspects behind, and with it most of the areas related to the testing process.

Also in these places there is a cultural aspect where the average end users appears to be less demanding and thus allow local firms to ‘get away’ without doing much testing, at least for now.

The most interesting and frustrating part of this phenomenon is the fact that the people leading the development processes understand they are missing something, but they lack the tools and the know-how to find and fill the missing pieces.

Is there change on the horizon?I think that in the same way that nature flows to close existing gaps and reach a state of equilibrium so do geographical markets. This means that two things should start happening (if they are not already) in these places:1. Software development companies that want to export their products to other markets will start running into

commercial barriers due their lack of testing processes;2. Local buyers will stop being so ‘forgiving’, since they get access to international firms that provide the same products at competitive prices but with higher quality. The effects of both of these processes will be to generate the need for change within the local development markets and fi rms, triggering an improvement process that should push the quality of their products and processes up to international standards. This will be the time when we will start seeing a more active representation of these countries within the international testing community, when places like Central America or the Middle East start taking an active role in the development and enrichment of our testing methodologies and techniques.

This change may take place within the next couple of months or years, depending on each separate country and market, but I have no doubt it will take place, closing the existing gaps and generating a truly global testing community.

Joel Montvelisky XxxxxxxxPractitestwww.practitest.com

The most interesting and frustrating part of this phenomenon is the fact that the people leading the development processes understand they are missing something, but they lack the tools and the know-how to find and fill the missing pieces.

the last word...

September 09 | T.E.S.T www.testmagazineonline.com31 Media will keep you up to date with our own products and offers including VitAL Magazine. If you do not wish to receive this information please write to the Circulation Manager at the address given.

Please tick here ■ if you do not wish to receive relevant business information from other carefully selected companies.

If you have not already subscribed then visit www.vital-mag.net to download

a subscription form which you can fax to +44 (0) 870 085 8837 or post to:

31 Media, Crawley Business Centre, Stephenson Way, Crawley, West Sussex, RH10 1TN.

News, Views, Strategy, Management, Case Studies and Opinion Pieces

Subscribe to the most VitAL source of information

www.vital-mag.netInspiration for the modern business

vital

News, Views, Strategy, Management,


• InternationalbenchmarkofskillsandexperienceforSoftwareTesters• 80%ofallSoftwareTestingexamsworldwidetakenthroughISEB• Hierarchicalstructureprovidingabasisforskillsandcareerprogression• Over70,000SoftwareTestingexamsdeliveredworldwide• SupportedbyISEBFoundationandIntermediatelevelpublications www.bcs.org/books

ISEB is part of the BCS, The Chartered Institute for IT, and is an International examination body. We have delivered over 380,000 exams worldwide and our exams are available in over 200 countries. For more information visit www.iseb-exams.com or call 01793 417655

Book a training course at www.iseb-exams.com/st Book an exam at www.iseb-exams.com/computerbasedexams

Promoteyourself...TestyourselfISEBSoftwareTestingcertification

MTG/AD/689/0909

The British Computer Society (Registered charity no. 292786)

14124_ST_test_adV2.indd 1 10/09/2009 16:33

Documents

TEST Magazine - August-September 2009