Joint Application Testing (JAT)conference.eurostarsoftwaretesting.com/wp-content/uploads/f7cove… · IT department. It was in that capacity that I came in contact with the concepts

P R E S E N T A T I O N

International Conference On

Software Testing, Analysis & ReviewNOV 8-12, 1999 • BARCELONA, SPAIN

Presentation

Bio

Return to Main Menu

Presentation

Bio

Return to Main Menu F7

Friday, Nov 12, 1999

Joint ApplicationTesting (JAT)

Wim Blommaert

Paper

Joint Joint Application TestingApplication Testing

Cop

yrig

ht ©

199

9 ps

_tes

twar

e -

Join

t App

licat

ion

Tes

ting

- 1

1 9 9 4 - 1 9 9 9

Software Testing Services

5th Anniversary

Wim BlommaertWim BlommaertEurostarEurostar 1999 1999

1 9 9 4 - 1

9 9 9

Software Testing Service

s

5th Anniversary

Copyright © 1999 ps_testware - Joint Application Testing - 2

toctoc

??T

oolsT

oolsFactsFacts

toctoc

Joint Joint Application TestingApplication Testing

BetterBetter, , CheaperCheaper, , FasterFaster


??T

oolsT

oolsFactsFacts

toctoc

IntroductionIntroduction

Wim BlommaertWim Blommaert

psps__testwaretestware


??T

oolsT

oolsFactsFacts

toctoc

AgendaAgenda•• Some factsSome facts•• Joint Application TestingJoint Application Testing•• A classical gap in testingA classical gap in testing•• The engineering modelThe engineering model•• About communicationAbout communication•• User involvementUser involvement•• Interviews & workshopsInterviews & workshops•• Kernel teamsKernel teams•• Convincing IT and UsersConvincing IT and Users•• Portfolio analysisPortfolio analysis•• ExamplesExamples

1 9 9 4 - 1

9 9 9


s

5th Anniversary


toctoc

??T

oolsT

oolsFactsFacts

toctoc

Some FactsSome Facts


??T

oolsT

oolsFactsFacts

toctoc

Y2K Functional testingY2K Functional testing

1/8/981/8/98 1/4/991/4/99 1/6/991/6/991/2/991/2/991/12/981/12/98

362

187

416

IT Business Testers

No knowledgeNo knowledge

No documentationNo documentation

No reusable testsNo reusable tests

CriticalCritical??


??T

oolsT

oolsFactsFacts

toctoc

100

20 30

0

50

100

Cost (Mandays)

Experience with test planningExperience with test planning

12

3 4

0

5

10

15

Lap Time (Weeks)

0

100

Quality (Business Link)

classical finished Y2K Finished

classical planned

??MoneyMoneyTime

Time

Qu

alit

yQ

ual

ity


??T

oolsT

oolsFactsFacts

toctoc

How did we do this?How did we do this?

IT and USERS joining together in testingIT and USERS joining together in testing

To provide high quality software, To provide high quality software, on time, within budgeton time, within budget

By essentially filling the testing “Gap”By essentially filling the testing “Gap”

Using super efficient communication Using super efficient communication techniquestechniques

??JADJADJATJAT


??T

oolsT

oolsFactsFacts

toctoc

A classical gap in testing A classical gap in testing (1/3)(1/3)

Program designProgram design

CodingCodingCoding

V-modelV-model

CodingCoding

Audit testAudit testAudit test

Acceptance testAcceptance test

System testSystem test

Integration testIntegration test

Module testModule test

StrategicStrategicStrategic choiceschoiceschoices

RequirementsRequirements

Logical designLogical design

Technical designTechnical design

TestingTestingTesting

Technical CorrectnessTechnical Correctness

Functional CorrectnessFunctional Correctness

Right application / trustRight application / trust


??T

oolsT

oolsFactsFacts

toctoc


IT :IT :“Technically correct?”“Technically correct?”

User :User :“Is this what I need?”“Is this what I need?”

“Should I use it?”“Should I use it?”

Module & Integration TestingModule & Integration Testing Acceptance TestingAcceptance TestingSystem TestingSystem Testing

ITIT UserUser

??


??T

oolsT

oolsFactsFacts

toctoc


System tests are sometimes done twice (overlap)System tests are sometimes done twice (overlap)

More often system tests are not done properlyMore often system tests are not done properly

System tests are often technical or System tests are often technical or not detailed enoughnot detailed enough

ConsequencesConsequences


??T

oolsT

oolsFactsFacts

toctoc

This car is This car is approvedapproved

by factory QA by factory QA

This SW is This SW is approvedapproved by IT QA by IT QA

The engineering model The engineering model (1/2)(1/2)

How nice. How nice. Let’s try it.Let’s try it.

UserUserITIT

Module & Integration TestingModule & Integration TestingAcceptance TestingAcceptance Testing

System TestingSystem Testing


??T

oolsT

oolsFactsFacts

toctoc

The engineering model The engineering model (2/2)(2/2)

Acceptance testing becomes a test driveAcceptance testing becomes a test driveserving two purposesserving two purposes

Assuring that the right software is builtAssuring that the right software is built

Build confidenceBuild confidence


??T

oolsT

oolsFactsFacts

toctoc

About communication About communication (1/2)(1/2)

What are the priorities?What are the priorities?What shall we test?What shall we test?

What is it supposed to do?What is it supposed to do?......

I’ll tell youI’ll tell you

UserUserITIT Communication is essentialCommunication is essential


??T

oolsT

oolsFactsFacts

toctoc

About communication About communication (2/2)(2/2)

What are the priorities?What are the priorities?What shall we test?What shall we test?

What is it supposed to do?What is it supposed to do?......

ITIT Building consensus quickly is essentialBuilding consensus quickly is essential


??T

oolsT

oolsFactsFacts

toctoc

User involvementUser involvement


CodingCodingCoding

V-modelV-model

CodingCoding

Audit testAudit test





Strategic choicesStrategic choices




TestingTestingTesting

JADJAD JATJAT

1 9 9 4 - 1

9 9 9


s

5th Anniversary


toctoc

??T

oolsT

oolsFactsFacts

toctoc

ToolsTools

Using super efficient communication Using super efficient communication techniquestechniques


??T

oolsT

oolsFactsFacts

toctoc

What do you need to know?

InterviewsInterviews

10 Commandments10 Commandments

No consensus building neededNo consensus building needed

EasyEasy

BonusBonus


??T

oolsT

oolsFactsFacts

toctoc

WorkshopsWorkshops(1/4)(1/4)

We need to build consensus

Moderated and StructuredModerated and Structured

Consensus building neededConsensus building needed

DifficultDifficult


??T

oolsT

oolsFactsFacts

toctoc

Workshops Workshops (2/4)(2/4)

A meeting to quickly build consensusA meeting to quickly build consensus

•• based on groupsbased on groups

•• moderatedmoderated

•• interactiveinteractive

•• clear objectivesclear objectives

•• defined scopedefined scope

•• limited in timelimited in time

•• agendaagenda

•• agreed resultsagreed results

��

��

��

��

��

��

��

��


??T

oolsT

oolsFactsFacts

toctoc

WorkshopsWorkshops(3/4)(3/4)

FacilitationFacilitation OrganisationOrganisation

RulesRules

Establish and maintain Establish and maintain an environment that an environment that

enables super efficient communicationenables super efficient communication and consensus building and consensus building


??T

oolsT

oolsFactsFacts

toctoc

Workshops Workshops (4/4(4/4))

FacilitatorFacilitator

Does not participate in Does not participate in discussiondiscussion

ResultResult FormatFormat


??T

oolsT

oolsFactsFacts

toctoc

Kernel teams Kernel teams (1/3)(1/3)

People participate in more than People participate in more than one phase of the V-modelone phase of the V-model

The purpose is to improve the The purpose is to improve the transfer of knowledgetransfer of knowledge

Sometimes a trade off has to be madeSometimes a trade off has to be made between independence and between independence and

optimal communicationoptimal communication


??T

oolsT

oolsFactsFacts

toctoc



V-modelV-model

CodingCoding





Strategic choicesStrategic choicesStrategic choices




UserUser


JADJADAnalystAnalyst


??T

oolsT

oolsFactsFacts

toctoc



V-modelV-model

CodingCoding





Strategic choicesStrategic choicesStrategic choices



RequirementsRequirements Acceptance testAcceptance test

User

AnalystAnalyst

JATJAT


??T

oolsT

oolsFactsFacts

toctoc

Convincing IT and Users Convincing IT and Users (1/6)(1/6)

UserUser

Acceptance testing becomes a test drive, Quality is provided by the factory.

Testing will be done by professionals.

•• I don’t have time to do a lot of testing I don’t have time to do a lot of testing•• (System) Testing is not my job (System) Testing is not my job•• We have to do a lot of tests ourselves We have to do a lot of tests ourselves


??T

oolsT

oolsFactsFacts

toctoc


UserUser

You will fully participate in the System Tests providing knowledge

•• I do not trust IT to do my tests I do not trust IT to do my tests•• IT does not know the business IT does not know the business

You will still do an acceptance test

Your acceptance test will be used as input for the system test


??T

oolsT

oolsFactsFacts

toctoc


UserUser

Your test set will be fully re-used when Your test set will be fully re-used when possible and tests will be added to possible and tests will be added to

provide more depth if neededprovide more depth if needed

•• I already have a test set. I already have a test set. •• We will just do the work all over again We will just do the work all over again

Your test set is still needed for the Your test set is still needed for the acceptance testacceptance test

System tests and Acceptance tests System tests and Acceptance tests have different goals, are different testshave different goals, are different tests


??T

oolsT

oolsFactsFacts

toctoc


ITIT

Your test set will be fully re-used when possible and tests will be added to

provide more depth if needed

•• I already have a good test set I already have a good test set

•• How can the users help us? How can the users help us?

They know the business. They will provide resources.


??T

oolsT

oolsFactsFacts

toctoc


ITIT

Do you expect your car to turn the right degree when you turn the wheel?

•• Shouldn't the users test the functionality? Shouldn't the users test the functionality?

•• What’s in it for me? What’s in it for me?

The users will gain confidence in the SW The relationship with the users will improve

You will get extra resources


??T

oolsT

oolsFactsFacts

toctoc


Yes, you are but now you can Yes, you are but now you can do a better job do a better job

• So I am no longer responsible for the functionality?

ITIT


??T

oolsT

oolsFactsFacts

toctoc

Portfolio AnalysisPortfolio Analysis

L

L H

H

Need for consensus

Res

ourc

es ,

com

mit

men

t

P1P4

P3

P2

•• Workshops Workshops•• Straw man model Straw man model•• Interviews Interviews

•• Interviews Interviews•• Confirmation workshop Confirmation workshop

•• Interviews Interviews

•• Workshops Workshops

FollowFollow--upup

Test Test executionexecutionTestTest Development DevelopmentTest PlanningTest Planning

TestTest RepairRepair Ret estRet estSco peSco pe PlanPlan Des ignDes ign BuildBuild

™

•• Pragmatic ApproachPragmatic Approach

WHAT?


??T

oolsT

oolsFactsFacts

toctoc

Example Example (1/3)(1/3)

Y2K Projects


V-modelV-model

CodingCoding






Acceptance testAcceptance testAcceptance testRequirementsRequirementsRequirements

JATJAT

Test coachTest coach(test knowledge(test knowledge))

UserUser(business knowledge(business knowledge))

DeveloperDeveloper(application knowledge)(application knowledge)


??T

oolsT

oolsFactsFacts

toctoc


New Project


V-modelV-model

CodingCoding






Acceptance testRequirements

WHATWHATTSTTSTUserUser


??T

oolsT

oolsFactsFacts

toctoc


New Project


V-modelV-model

CodingCoding






Acceptance testRequirements

TSTTST

UserUser

1 9 9 4 - 1

9 9 9


s

5th Anniversary


toctoc

??T

oolsT

oolsFactsFacts

toctoc

QuestionsQuestions

1

Wim Blommaert

Wim Blommaert has eleven years of IT experience mainly in a banking environment. During six years heworked at a large data centre as a capacity planner. Subsequently, he spent two years leading adevelopment methodology group supporting +/- 800 developers. Wim Blommaert was made responsiblefor setting up a large test organisation for a big European bank. As a Y2K test manager, he finally managedtheir Y2K test program. Wim Blommaert joined ps_testware in June 1998 as a management consultant. Inthis capacity, he was involved in setting up and managing Y2K programs and test organisations for differentcompanies (including large banking organisations and production environments). In 1999, Wim becameService Delivery Co-ordinator for ps_testware managing a large number of projects in the BeneluxCountries.

2

JJooiinntt AApppplliiccaattiioonn TTeessttiinngg

WWiimm BBlloommmmaaeerrtt

ps_testware nv ps_testware BVTiensesteenweg 329B-3010 LeuvenTel. +32-16-359380Fax +32-16-359388E-mail: [email protected]

Stephensonweg 14NL-4207 HB GorinchemTel. +31-183-699773Fax +31-183-699774

Abstract

The JAT concept was not something I developed overnight. The concepts had been lingering in my mindfor at least five years. They then started to surface out of pure necessity. Only after I had seen most ofthem at work in practice, I had the time to bring them all together in the concept of JAT.

In my previous job, I supported developers being responsible for the development methodology in a largeIT department. It was in that capacity that I came in contact with the concepts of Joint Applicationdevelopment (JAD). I followed courses, wrote guidelines and finally became a JAD tutor. I was also thefacilitator in numerous workshops.

The concept of JAD focuses on the co-operation of developers and users when developing software.Most of the time, it is associated with Rapid Application Development (RAD) but this is mainly due to lackof understanding in this area. You don’t need to apply RAD to use the JAD concept. The key technique iswhat is called a workshop or a JAD session.

A curious fact (which I could only appreciate later when I got involved in testing) was that in the entirestory of JAD not a single word was spent on testing.

When I got involved in testing four years ago, I was so busy that thoughts about JAD again dozed off. Until1998 when I got heavily involved in setting up and running Y2K test projects.

These Y2K test projects were facing the following problems:

• Extreme time pressure

mailto:[email protected]

3

• Low user involvement• Lack of test resources• Lack of knowledge regarding the applications that had to be tested

Confronted with these issues, I started to implement a number of ideas that, in retrospect were things we hadused when applying JAD but now applied to testing. So it was obvious (at least after a while) to call theconcept JAT referring to the fact that it is JAD applied not to the left hand side of the V-model but to theright hand side.

It’s this concept of JAT that is explained in this document. If applied properly you can benefit from it justas you can from JAD. Since I borrowed a lot of the ideas of JAD it should not surprise you thatoccasionally I will refer to JAD to clarify certain elements.

Enjoy your reading.

Key words:Joint Application Testing (JAT), Joint Application Development (JAD), System Testing, User involvement,testing gap, Engineering model, Kernel teams, Portfolio analysis, Super efficient communication,Interviews, Workshops.

4

1. Objectives

This paper supports the following objectives:

• The reader will understand the concept of Joint Application Testing (JAT).• The reader will be able to start using the JAT concepts to his or her advantage.• The reader will be provided with a number of practical tips.• The reader will know how to convince IT and Users to participate in JAT.

This paper does NOT support the following objectives:

• The reader will not be able to execute workshops.

2. Intended Audience.

People that might benefit from this presentation include:

• System Testers who want to co-operate intensively with the user community.• Users involved in testing.• Managers who need to convince both IT and Users to co-operate in testing.

5

3. Joint Application Testing (JAT)

3.1 Some facts

In this chapter, I hope to illustrate the benefits of JAT if applied properly. I will do this by providing someobjective information on projects to which it was applied.

Y2K

In one company, we managed to successfully test (in a structured way) 20 applications for Y2K in fewmonths time using JAT.

Users were always involved in these projects providing at least 20 % of the resources. The applications thatwere tested were selected based on following criteria:

• They were business critical• No documentation was available• No or very limited knowledge was available• No reusable tests existed

It was perhaps a doubtful honour to test only these applications but the results were nevertheless very good.The number of Y2K errors found in these applications was 10 times higher compared to (less difficult)applications that were tested in a non-structured way without using the concept of JAT.

After the users had been involved in these tests (following the concepts of JAT) they had no more need todo any tests of their own. This was not the case for the other applications on which both IT and usersexecuted tests.

Test planning

In another company, we had been defining what to test without using any of the JAT techniques in theplanning phase of a test project. Everyone realised that this took too much time and consumed a lot ofresources. Alternatives, however, were not presented

When we started a Y2K project with a size comparable to the new projects in the same company we hadbeen doing, we started using JAT techniques since we were confronted with the problems that were listedabove (time pressure). The same task (defining what to test) took only 25 % of the time and 20 % of theresources. Now even when we take into consideration that these projects were not exactly the same size andthat they already existed (legacy systems), these numbers were very significant.

When the planning for the next new project had to be made, we felt sure enough to offer the customer afixed price contract in which, compared to the previous project, both lap time and cost were reduced by afactor 3. These reductions were made possible by the introduction of JAT.

Remarkable was that not only the lap time and the cost were reduced but that the quality of the work alsoincreased. Whereas in the first project the decision of what to test had been made mainly from a technicalpoint of view, in the Y2K project these decisions were taken from a business point of view. This was madepossible by the fact that the users (following the concept of JAT) were heavily involved in this phase of theproject.

6

This proves that by the introduction of JAT was it not only possible to reduce cost and time but also toincrease quality. It should not have come as a big surprise since this is exactly what is promised if youapply Joint Application Development (JAD) properly.

3.2 How it was done

When I finally thought about the concept of JAT, I came up with three main elements that had helped us torealise the improvements described above. We were able to reduce time and cost and to increase qualityby:

• IT and Users joining together in testing.• Essentially filling what I call the testing “gap”.• Using super efficient communication techniques.

I will explain each of these topics in the next chapters. Now to announce that one is going to improve in thethree classical dimensions of software development (time, cost, quality) seems odd or even unbelievablesince most people assume that a gain in one (or two) of the dimensions has to be at the expense of the otherone(s). If you are familiar with the concepts of JAD, this should be less surprising because when JAD isapplied properly, it is supposed to produce a benefit in all three dimensions. Further lecture will hopefullyclarify why this is possible.

Keep in mind that the benefits of JAD and / or JAT can only be high when they are applied properly. I haveseen a lot of projects in which the application of these concepts backfired on the inexperienced team. Inshort, applying them is not easy but can be very rewarding.

7

3.3 A classical Gap in testing

If one looks at the V-model (The Gartner Version also used at ps_testware) simply put the test phases havethe following purpose:

Build phase test phase Purpose Who tests?Define Requirements Acceptance testing Build the right product

Build ConfidenceUsers

Functional Analysis System testing Functional correctness ?Technical Analysis Integration testingProgram Design Module testing

Technical correctness IT

Coding

Now more often then not IT people are mainly interested in technical correctness focussing heavily onmodule & integration testing. IT clearly sees these tests as being part of their responsibility.

Users on the other hand are more focussing on acceptance testing. They would like to assume that bothtechnical and functional correctness was already achieved at a lower level. Users will normally (although Ihave seen cases where they to my surprise did not) assume a natural responsibility for acceptance testing.

The problem occurs in the middle where it is not so clear who should take the lead. Who is responsible forsystem testing?

For some the answer is trivial but reality shows it is not. As a result of this confusion sometimes systemtests are done double once by IT and then repeated by the users. Some people even suggest that usersshould exactly repeat the systems tests made by IT and call them acceptance tests. This redundancy costs alot of money.

Often systems tests are not done properly because both IT and Users see it as the responsibility of the otherparty to do them resulting in none being done. Or IT executes Module & Integration tests and calls itsystems tests. The same often goes for the users whom all to easily promote their acceptance tests to systemtests.

Even if one party clearly takes ownership of the system tests problems often occur. If IT takes ownershipthe system tests they execute are often too technical in nature (instead of functional). If the users assumethe role of systems testers these tests are often not done detailed enough because the users are alreadyassuming a certain level of functional correctness or are more thinking about executing an acceptance test.

In reality the end result is very often a gap in the right hand side of the V-model or in testing.

8

3.4 The engineering modelWhen we want to plug this gap in the V-model, the first answer of course is who should take responsibilityfor the system tests.

My answer to this question is, what I call, the engineering model. The engineering model is something weall use frequently but as a reader it might sound very unfamiliar. The engineering model is related to theway commodity goods like cars, VCR’s, PC’s… are made.

When you buy a car, VCR, PC, … the only type of tests you perform as a user are acceptance tests. Youwill certainly not do any technical tests1. Will you do system tests? These tests are related to functionalcorrectness. When I buy a car I assume that it is fully technical and functional operational. I will not riskstepping in this car when this is not the case.

The only thing I normally do is an acceptance test, a test drive in fact. There are only two things I should beworried about:

• Is this the car I want, need, asked for (size, colour,)?• Do I have enough confidence in this car to buy it?

All the other things should be the concern of the factory. Why should we not want to act the same way inthe case of software? We all know that the word software engineering does not yet provide the sameguarantees we are used to as far as the production of cars is concerned. But decent testing should help tomake progress in this area.

If one thinks about software in terms of the engineering model, it is clear that system testing is theresponsibility of IT and that acceptance testing can be reduced to a test drive on condition that systemtesting is done properly

The purpose of acceptance testing becomes:

• To assure that the right software is built (based on the user requirements)• To give the users confidence in the fact that they will be able to do their business with this piece of

software.

3.5 About communication

What is the key factor to success in software engineering and in testing? Having the smartest programmers?Having the best testers? In fact, when you take some time to consider it, the most important issue iscommunication. Being able to communicate effectively and efficiently is of the utmost importance.In fact what is produced is pure information (unless you consider the disc on which the software can beput). In a lot of cases, communication is in a lot of cases the cause of failure in projects. Getting prioritiesmixed up, implementing the wrong functions, testing parts of the software that hardly support the business,not testing important business functions. That is why both JAD and JAT focus heavily (perhaps evenexclusively) on the improvement of communication in IT projects.

1 I know that some of you in fact do (more as a hobby) but I’m quite happy to use my car without having totake it apart. I would never be able to get it assembled again anyway.

9

One aspect of communication is getting consensus. When more than one person is involved in a project (bydefinition the case?) getting consensus becomes important, consensus on priorities, test strategies, …Getting consensus can be very time and resource consuming (or even impossible) without the righttechniques. This is why both JAD and JAT also concentrate on techniques for rapid consensus building.In example 2 (test planning) of chapter 3.3, it took the company extremely long to define what to test. Thiswas mainly a problem of getting consensus. People who do not know techniques that allow for rapidconsensus building get used to endless discussions. In the end, it is the manager who has to decide resultingin no consensus at all.

3.6 User involvement in JAD and JAT

In classical development it is often assumed that user involvement is minimal once the user requirementshave been defined since IT can take over from there. In a lot of cases that I have seen users involvementwas not even clear in this first phase (defining user requirements) IT took over right from the start.

The concept of JAD is based on a strong co-operation between IT and users. The effect is that users getinvolved very early in the development process and stay involved at least until functional analysis (possiblyeven later). Unless the user interface is defined in the more technical phases, user involvement in thesephases is often not needed.

The key element in this co-operation are workshops (or JAD sessions) during which both IT and usersparticipate at least until the processes, data and user interface have been defined.

If you look at the right hand side of the V-model (testing), the same ideas are still valid. In classical testingthe users would receive the software fairly late in the process. They are then expected to accept thesoftware using an acceptance test. Almost never, have I seen users who got involved in the tests IT did, noteven when JAD was used. So even when people use JAD effectively, involving the users more on the lefthand side of the V-model it is not so obvious to do the same thing on the right hand side.

If you use the concept of JAT the users do get involved in the tests executed by IT. How this can realisedwill be explained in the next chapters. This user involvement will mainly focus on the system tests.Although IT will drive these system tests, users will get involved heavily. The benefits of doing this havealready been mentioned. The resulting system tests will be of a higher quality reducing the need foracceptance testing. Next to this, the users will gain additional confidence in the product because they getinvolved in these system tests. This will again reduce the need for acceptance testing.

User involvement in the more technical tests (module and integration tests) is normally not needed.

10

4. Tools: using super efficient communicationtechniques

4.1 Interviews

Interviews are very useful when you do not have to build consensus in a group of people. E.g. When youneed information from a specialist, interviews are relatively easy to execute (at least when compared toworkshops). Nevertheless there are some rules that can help when interviewing people. I call them the tencommandments of a good interview. Make sure you know the answer to the following questions.

1. SETTING: why do you want to interview this person?2. SCOPE: what do you want to know?3. DETAIL: what do you specifically want to know from this person?4. RELATIONSHIP: what type of relationship do you want with this person in the future?5. RESULT & WRAP UP: what are you going to do after the interview?6. INTRODUCTION: do you have an introduction to the interview?7. STRUCTURE: are you going to structure the interview?8. SCENARIO: what would be a good scenario for the interview?9. TIME: how much time do you allocate for the interview?10. LOGISTICS: Are there any practical points of attention (room, drinks, …)

4.2 Workshops

Workshops are a completely different ball game. Within the scope of this text, it is not possible to discussthe issues related to workshops in any level of detail. I will therefore limit myself to the main issues.

Workshops are extremely useful when consensus has to be reached within a group of people. Since we aretalking about the concept of JAT, these groups will typically include both developers and users. Withoutthe proper use of workshops, getting consensus can take forever or will possibly never be reached.

A workshop is lead by a moderator or facilitator. This is one of the key elements of workshops. It’s veryhard to be a good moderator in a workshop. People often underestimate this. Especially when the groupsare big (12 people has to be considered the maximum) and the participants have very different opinions, aworkshop can easily become a facilitator’s nightmare.

Workshops are very structured. There is a fixed set of rules that the participants will have to follow (trygetting everyone to be exactly on time and to turn of their mobiles). At first some people might argue thatthe workshop is too formal but once the participants get used to these rules and start appreciating theefficiency of communicating in this way they will easily accept them.

Often I ask people if they use workshops. All too often the answer is yes. When I then make a furtherinquire into how this is done it turns out that what they call workshops does not fit the following definition.

A workshop is a meeting used to build consensus quickly and:

• is based on groups• is moderated• is very interactive• has clear objectives• has a defined scope

11

• is limited in time• is built around an agenda• produces agreed results• is formal

Some of these so-called workshops turn out to be nothing more than inefficient informal chats. Somepeople even tell me they have two moderators in their workshops. Or they tell me that after the workshopno written minutes of the results are made.

If workshops want to be successful they have to be very formal. This is where the real power of theworkshops lies.

The three pillars of a good workshop are:

• A good facilitator• Good organisation (including logistics)• Appropriate use of the rules.

Getting a skilled moderator will solve all of this because he or she will know what to do. The whole idea isto create and maintain an environment in which communication can be super efficient. Difficult but veryrewarding if you succeed.

One last word about the facilitator. Not everyone is fit to be a facilitator in a workshop but it is a skill thatcan be learnt given the person has some initial (mainly communication) skills in this area.

One of the hardest parts of being a facilitator is not being allowed to participate in the discussions. Thefacilitator really has to keep two things in mind. On the one hand, to make sure a result is reached withinthe given time limit and on the other hand to make sure this result is reached in a civilised way payingattention to how it is reached.

The facilitator has to focus on the content as well as on the format of the workshop. This may seem easybut it is not. Often a facilitator will under pressure of time drop any consideration for the format of theworkshop and start concentrating on reaching a result one way or the other. I have witnessed (more thanonce) facilitators being very rude to the participants in a workshop in their urge to get a result. Onefacilitator even ordered one of the participants (literally) to shut up.

The other extreme is also possible. Some facilitators are so eager to have a nice and civilised workshopfollowing all the rules that they forget that a result has to be reached. The end result is that they manage torun a model workshop but end up without any result whatsoever.

5. Kernel teamsOne of the concepts that I used in a previous job was something we called the use of kernel teams.

The concept is simple. Instead of people specialise in the execution of one phase of the V-model you havesome people participating in more than one phase.

I remember a project in which we had a small team (the kernel team hence the name) executing all thephases. Per phase additional people were then brought in just for that phase. This meant that analysts wrotelines of code whereas programmers defined entity relation diagrams.

The biggest advantage is an improved communication. Instead of just information (documents), people willpass from one phase to the other. This will improve communication a lot.

12

An example on the left hand side of the V-model could be a user who has been involved in setting up theuser requirements becoming a full team member in the development team participating in the functionalanalysis. This will seriously reduce the possibility of the requirements being misinterpreted.

A few months ago it occurred to me that the same concept could be of value on the right hand side of the V-model. The motivation would be the same: to improve communication.

An example could be a user who has been involved in defining the user acceptance test joining the systemtest team. This will help to make sure that the right priorities are set while defining these system tests andto make sure systems test are done based on the reality of the system taking into account what it willactually be used for.

One can even imagine an analyst who has been involved in writing the functional specs joining the systemtest team.

This example clearly shows that one has to be careful in defining kernel teams in testing. Taking part inmore than one phase on the right hand side of the V-model might not always be a good idea.

The problem is that in testing a certain level of independence between the person that wrote the specs andthe one that tests them is assumed. From this point of view a lot of the interesting kernel team scenarios arenot possible because the independence will be lost.

Does this mean that these scenarios are to be avoided? I personally do not think so. The issue here is thebalance between the need for independence and the need for optimal communication. Both advantageshave to be balanced the one against each other. The choice really depends on the actual situation. Ifcommunication is the most essential issue, independence might be sacrificed in order to gain it.

13

6. Convincing IT and Users

Supposing you want to try JAT you will have to convince both the users and IT to participate in this. Inpractice, I have often seen that both parties have their reasons to resist in case the organisation wants to startusing JAT. I will give a list of the reasons that I have come across and the answers I think are appropriatein dealing with this.

Since the concept of JAT is based on the co-operation between IT and users being able to get theircommitment is crucial.

Point of view of the users:

Q:

• “System testing is not my job”.• “I do not have a lot of time to do all these tests”.• “I already have enough work doing my acceptance tests”.

A:

• It is the job of IT to do the system tests but good system testing will reduce the need for acceptancetesting.

• Professionals will do system testing.

Q:

• “I do not trust IT to do the systems tests”.• “IT does not know the business”.

A:

• You will still do an acceptance test.• You will fully participate in the system tests providing business knowledge.• Your acceptance test can be used as input for the system test.

Q:

• “We already have tests”.• “We will just do the work all over again”.

A:

• The existing tests will be reused where possible and additional tests will only be added if needed.• Your tests are still needed to execute the acceptance test.• System tests and acceptance tests are different type of tests and have different goals. Your acceptance

test might not be suitable to be the basis for the system test.

Point of view of IT:

14

Q:

• “We already have a test set”.• “What do we need the users for? How can they help?”

A:

• The existing tests will be reused where possible and additional tests will only be added if needed.• The users know the business and will provide additional resources.

Q:

• “Is it not the job of the users to test the functionality?”• “What do I have to gain from this co-operation?”

A:

• Do you expect your car to turn to the right the correct amount when you turn the wheel?• You get additional resources.• The users will gain additional confidence in the software.• The relationship with the users will improve.

Q:

• “Users get involved. So IT is no longer responsible for testing the functionality then?”

A:

• Yes you are but now you can do a better job.

15

7. Portfolio Analysis

I wanted to include some lines on Portfolio analysis because it is about being practical. Portfolio analysis isbased on the fact that a standard way of doing things (like JAT) may be nice but it will never exactly fitevery project in your organisation. Projects can be very different in nature needing a different approach.Now on the other hand we can not afford to come up with a new approach for every project.

You do not have to. What ps_testware always suggests is to make a portfolio analysis of your projects tohelp you decide how a general concept (like JAT) can be modified to fit certain type of projects.

The first step is to group similar projects together. The next step is to define the approach per group butstill starting from the general concept of JAT.

I will explain this by using an example. Suppose we grouped the projects according to two parameters: theneed for consensus building and the ability to do workshops (based on availability of resources andcommitment from users and IT).

There could be 4 groups:

1. The need for consensus is high, and so is the ability to do workshops.2. The need for consensus is high but the ability to do workshops is low.3. The need for consensus is low, and so is the ability to do workshops.4. The need for consensus is low but the ability to do workshops is High.

The approach for each group could be:

1. Use workshops.2. The trick here is to reduce the work that has to be done in the workshops. Use interviews to prepare

the workshops. You can also use a straw man model to further reduce the work to be done in theworkshops.2

3. Use interviews.4. Use interviews to build the result. You can use a workshop to confirm this in-group.

Portfolio analysis will help you to avoid trying to apply a theoretical concept in a situation where it will notwork.

2 When using a straw man model the facilitator has prepared what he or she thinks to be the solution. Thefacilitator will then try steering the group in that direction. Very dangerous. I personally never use it.

Wim Blommaert

Wim Blommaert has eleven years of IT experience mainly in a bankingenvironment. During six years he worked at a large data centre as acapacity planner. Subsequently, he spent two years leading adevelopment methodology group supporting +/- 800 developers. WimBlommaert was made responsible for setting up a large test organisationfor a big European bank. As a Y2K test manager, he finally managedtheir Y2K test program. Wim Blommaert joined ps_testware in June1998 as a management consultant. In this capacity, he was involved insetting up and managing Y2K programs and test organisations fordifferent companies (including large banking organisations andproduction environments). In 1999, Wim became Service Delivery Co-ordinator for ps_testware managing a large number of projects in theBenelux Countries.

Documents

Joint Application Testing (JAT)conference.eurostarsoftwaretesting.com/wp-content/uploads/f7cove… · IT department. It was in that capacity that I came in contact with the concepts