11

Software testing with Software testing with Rational TeamtestRational Teamtest

Ashish SoniAshish Soni

Lecturer, ITPLecturer, ITP

asoni@usc.eduasoni@usc.edu

22

OutlineOutline

Miscellaneous testing factsMiscellaneous testing facts What is software testing?What is software testing? Why do we test?Why do we test? What methods and tools are available to What methods and tools are available to

test applications test applications DemoDemo FutureFuture Conclusion and questionsConclusion and questions

Source: Cigital CorporationSource: Cigital Corporation 33

Interesting factsInteresting facts

Recent studies show that : Recent studies show that : C++ : 25% more defects than CC++ : 25% more defects than C A OO C++ defect takes 2-3 times as long to A OO C++ defect takes 2-3 times as long to

debug regardless of defect complexitydebug regardless of defect complexity Inheritance engenders 6 times more defectsInheritance engenders 6 times more defects Java: multi-threading and inheritance: nearly Java: multi-threading and inheritance: nearly

impossible to write correctly and worse to impossible to write correctly and worse to test!!test!!

No of lines of code in applications growing No of lines of code in applications growing rapidly, e.g. Windows 3.1 – Windows XPrapidly, e.g. Windows 3.1 – Windows XP

44

What is software testing?What is software testing?

Finding out If something works or notFinding out If something works or not Determine if overall objectives are Determine if overall objectives are

being metbeing met Testing involves operation of a Testing involves operation of a

system or application under system or application under controlled conditions and evaluating controlled conditions and evaluating the results the results

55

Software TestingSoftware Testing

Testing is the feedback mechanism Testing is the feedback mechanism in the software development process.in the software development process.

It tells you where corrections need to It tells you where corrections need to be made to stay on course at any be made to stay on course at any given iteration of a development given iteration of a development effort. It also tells you the current effort. It also tells you the current quality of the system being quality of the system being developed developed

66

Why do we need to test?Why do we need to test?

Real life examples:Real life examples: First US mission to Venus failedFirst US mission to Venus failed

(Reason: missing comma in a FORTRAN do loop)(Reason: missing comma in a FORTRAN do loop) Australia: Man jailed because of computer Australia: Man jailed because of computer

glitch. He was jailed for traffic fine although he glitch. He was jailed for traffic fine although he had paid it 5 years agohad paid it 5 years ago

June 1996: Ariane-5 space rocket, self-June 1996: Ariane-5 space rocket, self-destruction, $500 million.destruction, $500 million.

(reason: reuse of software from Ariane-4 (reason: reuse of software from Ariane-4 without recommended testing)without recommended testing)

77

Why testing?Why testing?

Software is never correct no matter Software is never correct no matter what development technique is usedwhat development technique is used

Any software must be verifiedAny software must be verified Important to control the quality of Important to control the quality of

the productthe product Very expensive:Very expensive:

In a lot of cases 50% of development In a lot of cases 50% of development costscosts

88

Software qualitySoftware quality

Software quality can be measured Software quality can be measured across 4 dimensionsacross 4 dimensions

ReliabilityReliability FunctionalityFunctionality Application performanceApplication performance System performanceSystem performance

99

Software development eraSoftware development era

Over the past decade software Over the past decade software development has changed radicallydevelopment has changed radically

Client server era – annual releases of new Client server era – annual releases of new versions of productsversions of products

We also knew how many users we had We also knew how many users we had since all users were employeessince all users were employees

If system went down it could be restored If system went down it could be restored from backup or orders were taken from backup or orders were taken manuallymanually

1010

E-businessE-business

The big change right now is E-businessThe big change right now is E-businessWhich includes e-commerce and ERPWhich includes e-commerce and ERP

Business process automated using the web.Business process automated using the web. Users are customers and not Employees!!Users are customers and not Employees!! We cant send a broadcast to users saying We cant send a broadcast to users saying

“don’t login for one hour until we reboot the “don’t login for one hour until we reboot the system” system”

Lost customers = Lost Revenue ~ bankruptcyLost customers = Lost Revenue ~ bankruptcy This makes testing very important.This makes testing very important.

Source: Rational corporationSource: Rational corporation 1111

Release checklistRelease checklist

Does the application operate without Does the application operate without crashing, hanging or causing exceptions?crashing, hanging or causing exceptions?

Does it meet your business requirements?Does it meet your business requirements? Has it been tested thoroughly to meet the Has it been tested thoroughly to meet the

demands of your customers?demands of your customers? Can you track your progress against Can you track your progress against

requirements, defects and other metrics?requirements, defects and other metrics? Does everyone on the team have the Does everyone on the team have the

same critical testing information?same critical testing information?

1212

Methodologies of testingMethodologies of testing

Glass box TestingGlass box Testing White box TestingWhite box Testing Structural TestingStructural Testing All three methodologies involve All three methodologies involve

testing internal code structures, testing internal code structures, loops etc.loops etc.

1313

MethodologiesMethodologies

Black box and functional testingBlack box and functional testing Involves testing functionality of the Involves testing functionality of the

application, interfaces etc.application, interfaces etc.

1414

Web application testing Web application testing methodologiesmethodologies

Performance Performance FunctionalFunctional reliabilityreliability

1515

Rational Team testRational Team test

Suite of tools to automate the testing Suite of tools to automate the testing processprocess

Provides tools for Provides tools for Functional testing Functional testing Performance testing Performance testing Test case management Test case management Defect Tracking and change requests Defect Tracking and change requests

1616

Rational Team testRational Team test

Major tools in test suite:Major tools in test suite: Rational Test Manager: Used to Manage all test Rational Test Manager: Used to Manage all test

activities:activities: PlanningPlanning DesignDesign DevelopmentDevelopment ExecutionExecution analysisanalysis

Rational Robot: used to develop and execute Rational Robot: used to develop and execute functional tests for your windows and web functional tests for your windows and web applicationsapplications

1717

Rational Team TestRational Team Test

Rational Administrator:Rational Administrator: Used to create and manage rational Used to create and manage rational

projectsprojects Rational Clearquest:Rational Clearquest:

Used to track any kind of change activity Used to track any kind of change activity associated with software development, associated with software development, these include enhancement requests, these include enhancement requests, defect report etc.defect report etc.

1818

Rational Team TestRational Team Test

Rational Quality Architect:Rational Quality Architect: Used to test middleware components Used to test middleware components

built using EJB and com.built using EJB and com. Virtual testing : scalability and Virtual testing : scalability and

workload , server response timesworkload , server response times Functional and performance testingFunctional and performance testing

1919

Rational Team TestRational Team Test

Site CheckSite Check Used to manage internet and intranet Used to manage internet and intranet

websiteswebsites

2020

Installation and Setup Installation and Setup guidelinesguidelines

Standard InstallStandard Install Build and setup examples provided Build and setup examples provided

with team testwith team test Create a Project in Rational Create a Project in Rational

Administrator and associate various Administrator and associate various artifacts with the project artifacts with the project

2121

Create ProjectCreate Project

2222

2323

Project ArtifactsProject Artifacts

2424

Create Test Data StoreCreate Test Data Store

2525

Create Clearquest DBCreate Clearquest DB

You can use any of the Database You can use any of the Database belowbelow Microsoft accessMicrosoft access Microsoft SQL serverMicrosoft SQL server OracleOracle

Choice of DB depends on number of Choice of DB depends on number of users of applicationusers of application

Pre-requisite: the Database has to be Pre-requisite: the Database has to be created before it can be associatedcreated before it can be associated

2626

Contd..Contd..

2727

Project status and detailsProject status and details

2828

Rational Test managerRational Test manager

Once project has been created we Once project has been created we are now ready to create test plans are now ready to create test plans and test casesand test cases

You can associate test plans and test You can associate test plans and test cases with your requirementscases with your requirements

2929

Testing phases in RUPTesting phases in RUP

5 testing steps in RUP:5 testing steps in RUP: Planning – involves Test PlansPlanning – involves Test Plans Designing – involves Test casesDesigning – involves Test cases Implementing – involves scriptstImplementing – involves scriptst executingexecuting evaluating evaluating

Source: Rational corporationSource: Rational corporation 3030

Testing lifecycleTesting lifecycle

3131

Test ManagerTest Manager

3232

Create a Test PlanCreate a Test Plan

3333

Contd..Contd..

3434

Create test case folderCreate test case folder

3535

Contd..Contd..

3636

Configure Test case folderConfigure Test case folder

3737

Contd..Contd..

3838

Create Test CaseCreate Test Case

3939

Contd.Contd.

4040

Contd..Contd..

4141

Test casesTest cases

For each Test case you can:For each Test case you can: Associate iterationAssociate iteration Associate configurationAssociate configuration Select manual or automated Select manual or automated

implementationimplementation If automated then you can select If automated then you can select

appropriate test scriptappropriate test script Specify pre and post conditionsSpecify pre and post conditions Specify acceptance criteriaSpecify acceptance criteria

4242

Test Case propertiesTest Case properties

4343

Execute testsExecute tests

You can execute test cases from Test You can execute test cases from Test Manager by selecting the “Run” Manager by selecting the “Run” option.option.

4444

Contd.Contd.

4545

Test managerTest manager

Lots more you can do with Test Lots more you can do with Test Manager!!Manager!!

Execute other Rational applicationsExecute other Rational applications Please read documentation on Please read documentation on

rational CD.rational CD.

4646

Rational RobotRational Robot

Set of components for automating the Set of components for automating the testing of windows client /server and testing of windows client /server and internet applicationsinternet applications

Can record and playback:Can record and playback: GUI scripts: functional testingGUI scripts: functional testing VU scripts: performance testingVU scripts: performance testing

Proprietary language used to write scriptsProprietary language used to write scripts Can use java or visual basic to write Can use java or visual basic to write

custom scriptscustom scripts

4747

Rational RobotRational Robot

4848

Advanced FeaturesAdvanced Features

Performance testing using “virtual Performance testing using “virtual test” agents deployed on several client test” agents deployed on several client machinesmachines

Accepting test inputs from datapools : Accepting test inputs from datapools : this requires knowledge of sqabasic this requires knowledge of sqabasic languagelanguage

Please refer to Rational documentation Please refer to Rational documentation on CD for language detailson CD for language details

4949

Rational Site CheckRational Site Check

Internet and intranet website testingInternet and intranet website testing Checks for broken links, structure etcChecks for broken links, structure etc Can deploy website to target Can deploy website to target

destination using Site checkdestination using Site check

5050

DemoDemo

5151

Contd..Contd..

5252

Other Rational ToolsOther Rational Tools

Rational test Factory – used to Rational test Factory – used to automatically detect bugs automatically detect bugs

Quality architect – EJB and COM Quality architect – EJB and COM testingtesting

5353

Current issues Current issues

In addition to reliability and functionality In addition to reliability and functionality software has to be secure!software has to be secure!

Various hacks are made possible because Various hacks are made possible because of poor programming practices and of poor programming practices and inadequate testing (e.g buffer overflow)inadequate testing (e.g buffer overflow)

Most hacked software : Microsoft IIS Most hacked software : Microsoft IIS Shorter deadlinesShorter deadlines Bug fixes open up new weaknesses in Bug fixes open up new weaknesses in

softwaresoftware

5454

Secure Programming Secure Programming PracticesPractices

Building Secure software – Addison Building Secure software – Addison wesleywesley

www.shmoo.comwww.shmoo.com – very good secure – very good secure programming guidelinesprogramming guidelines

5555

Additional resourcesAdditional resources

www.rational.comwww.rational.com Getting the most from your automateGetting the most from your automate

d testing tool ( article rational edge d testing tool ( article rational edge magazine)magazine)

Automated Testing – silver bullet? ( aAutomated Testing – silver bullet? ( article rational edge magazine)rticle rational edge magazine)

5656

Did you know that:Did you know that:

Software testing consultants cost!Software testing consultants cost! Junior level test engineer: $80 - $100 (1-2 Junior level test engineer: $80 - $100 (1-2

yrs of experience)yrs of experience) Medium level test engineer - $100 - $150/ Medium level test engineer - $100 - $150/

hrhr Senior level test engineer : $150- Senior level test engineer : $150-

$300/hour$300/hour If testing for If testing for security security these figures go these figures go

even highereven higher

5757

Questions??Questions??