Software TestingSAJID SIDI & PANKAJ KHARE

What is Testing?According to ANSI/IEEE 1059 standard, Testing can be defined as - A process of analysing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item.

Why Testing Required?

Software testing is really required to point out the defects and errors that were made during the development phases. It's essential since it makes sure of the Customer's reliability and their satisfaction in the application.

Software Testing

SDLC and STLC

Software Testing

QA, QC & TestingQuality Assurance Quality Control Testing

QA includes activities that ensure the

implementation of processes,

procedures and standards in context

to verification of developed software

and intended requirements.

It includes activities that ensure the

verification of a developed software

with respect to documented (or not in

some cases) requirements.

It includes activities that ensure the

identification of bugs/error/defects in

a software.

Focuses on processes and procedures

rather than conducting actual testing

on the system.

Focuses on actual testing by executing

the software with an aim to identify

bug/defect through implementation

of procedures and process.

Focuses on actual testing.

Process-oriented activities. Product-oriented activities. Product-oriented activities.

Preventive activities. It is a corrective process. It is a preventive process.

It is a subset of Software Test Life

Cycle (STLC).

QC can be considered as the subset of

Quality Assurance.

Testing is the subset of Quality

Control.

STLC

Software Testing Life Cycle

Software Testing

Test Plan

Software Testing

Testing Models

Waterfall model

V model

Incremental model

Iterative model

Spiral model

Agile model

Software Testing

Agile Model

Software Testing

Types of Testing

Software Testing

Traditional Testing Flow

Definitions◦ Fault/Defect/Bug – The cause of a failure, e.g., a missing or incorrect piece of code

◦ Error – An intermediate unstable state

◦ Failure – The manifested inability of the program to perform the function required, i.e., a system malfunction evidenced by incorrect output, abnormal termination or unmet time and space constraints.

Fault Error Failure ◦ This chain can recursively iterate: a fault in turn can be caused by the failure of some other interacting

system.

◦ Bug Tracking Tool -A bug tracking system or defect tracking system is a software application that keeps track of reported software bugs in software development projects. It may be regarded as a type of issue tracking system. (ex. JIRA, RedMine, Bugnet, Quality Center, Mantis, Bugzilla etc.)

Software Testing

Bug Life Cycle

Software Testing

Emerging TrendsIN SOFTWARE TESTING

Emerging Trends

oMobile application testing (Rapid growth in Mobile testing)

oCloud-based Testing

oTesting in the Agile Development Environment

oContext-driven test

oSecurity Testing

oCrowd sourced testing

Software Testing

Mobile application testing

Challenges◦ Smooth Downloading

◦ Device Availability

◦ Network Provider

◦ Automation

◦ Devices Vs. Emulator/Simulator

Cloud-based TestingCloud testing is a form of software testing in which web applications use cloud computing environments (a "cloud") to simulate real-world user traffic.

oOn-Demand test execution by third parties

oOnline test execution in a public could

oVirtual/Online user access simulation

oTest Environment are readily available

oLimited Storage Space

oBased on SLA

oTesting Cost

Testing in Agile

Context-driven test

oWhy? What? Where? When? How?

oMonitor major test planning challenges

oClarify your mission

oAnalyze the product

oAnalyze product risk

oDesign the test strategy

oPlan

oShare the plan

Security Testing

oVulnerability Scanning

oPenetration testing

oRisk Assessment

oSecurity Auditing

oEthical hacking

Test Case DesignIN STLC

Test Case Format

Design a Test Scenario

Case StudyREAL IMPACT OF BUG(S) MISSED WHILE TESTING

Heathrow Terminal 5 Opening

Just before the opening of Heathrow’s Terminal 5 in the UK, staff tested the brand new baggagehandling system built to carry the vast amounts of luggage checked in each day. Engineers testedthe system thoroughly before opening the Terminal to the public with over 12,000 test pieces ofluggage. It worked flawlessly on all test runs only to find on the Terminal’s opening day thesystem simply could not cope. It is thought that ‘real life’ scenarios such as removing a bag fromthe system manually when a passenger had left an important item in their luggage, had causedthe entire system to become confused and shut down. Over the following 10 days some 42,000bags failed to travel with their owners, and over 500 flights were cancelled.

NASA’s Mars Climate Orbiter

On its mission to Mars in 1998 the Climate Orbiter spacecraft was ultimately lost in space.Although the failure bemused engineers for some time it was revealed that a sub contractor onthe engineering team failed to make a simple conversion from English units to metric. Anembarrassing lapse that sent the $125 million craft fatally close to Mars’ surface afterattempting to stabilize its orbit too low. Flight controllers believe the spacecraft ploughed intoMars’ atmosphere where the associated stresses crippled its communications, leaving it hurtlingon through space in an orbit around the sun.

Patriot Missile Error

Sometimes, the cost of a software glitch can’t be measured in dollars. In February of 1991, a U.S.Patriot missile defence system in Saudi Arabia, failed to detect an attack on an Army barracks. Agovernment report found that a software problem led to an “inaccurate tracking calculationthat became worse the longer the system operated.” On the day of the incident, the system hadbeen operating for more than 100 hours, and the inaccuracy was serious enough to cause thesystem to look in the wrong place for the incoming missile. The attack killed 28 Americansoldiers. Prior to the incident, Army officials had fixed the software to improve the Patriotsystem’s accuracy. That modified software reached the base the day after the attack.

Knight’s $440 Million Error

Knight Capital, a firm that specializes in executing trades for retail brokers, took $440m in cashlosses Wednesday due to a faulty test of new trading software.

On the day of 8/1/2012, Knight Capital's software went out and bought at the "market",meaning it paid ask price and then sold at the bid price--instantly. Over and over and over again.One of the stocks the program was trading, electric utility Exelon, had a bid/ask spread of 15cents. Knight Capital was trading blocks of Exelon common stock at a rate as high as 40 tradesper second--and taking a 15 cent per share loss on each round-trip and at the end of the debacle45 minutes later, Knight Capital had lost $440m and was teetering on the brink of insolvency.

Q & A

AutomationOF SOFTWARE TEST

What is automation testing ?

Test automation is the use of special software (separate from the software being tested) tocontrol the execution of tests and the comparison of actual outcomes with predicted outcomes.

Test automation can automate some repetitive but necessary tasks in a formalized testingprocess already in place, or perform additional testing that would be difficult to do manually.Test automation is critical for continuous delivery and continuous testing.

Why Automation?

oManual Testing of all work flows, all fields , all negative scenarios is time and cost consuming

oIt is difficult to test for multi lingual sites manually

oAutomation does not require Human intervention. You can run automated test unattended(overnight)

oAutomation increases speed of test execution

oAutomation helps increase Test Coverage

oManual Testing can become boring and hence error prone.

What to Automate?

oHigh Risk – Business Critical Test cases

oTest cases that are executed repeatedly

oTest cases that are very tedious or difficult to perform manually

oTest cases which are time consuming

What not to Automate?

oTest Cases that are newly designed and not executed manually

oTest Cases for which the requirements are changing frequently

oTest Cases which are executed on ad-hoc basis

Automation Process

Feasibility

oFeature that are important for the business

oScenarios which have large amount of data

oCommon functionalities across applications

oTechnical feasibility

Tool Selection

Test Tool selection largely depends on the technology the Application Under Test is built on. Forexample Selenium does not support Desktop application hence it is not suitable for desktopproject . In this case we can use UFT or any other tool which support Desktop application

Test Planning, Design & Development

During this phase, Generally Automation strategy & plan has created, which contains..

o Framework design and its features

o In-Scope and Out-of-scope items of automation

o Automation Script creation

o Schedule and Timeline of scripting and execution

o Deliverables of automation testing

Script Execution

Automation Scripts are executed during this phase. The scripts needs input test data beforethere are set to run. Once executed they provide detailed test reports.

Execution can be performed using the automation tool directly or through the Test Managementtool which will invoke the automation tool.

Example: Quality center is the Test Management tool which in turn it will invoke QTP forexecution of automation scripts. Scripts can be executed in a single machine or a groupof machines. The execution can be done during night , to save time.

Maintenance

As new functionalities are added to the System Under Test with successive cycles, AutomationScripts need to be added, reviewed and maintained for each release cycle. Maintenancebecomes necessary to improve effectiveness of Automation Scripts.

Architecture/Framework

Automation DemoPRAC T I CAL DEMONST RAT ION ON AUTOMAT ION T EST US I NG SELENI UM WEBDRI VER

Q & A

For any further queries and solution, please feel free to contact us.

Sajid Sidi@: sajid.sidi@gmail.comM: 9558203909

Pankaj Khare@: Pankaj.0405@gmail.comM: 8141287206