Fundamentals of testing 1

Tester Training Course

Hoang V. Nguyen

startnewday85@gmail.com

startnewday85.blogspot.com

Dept. of Computer Science – FIT - HUA

Tester Training Course

FUNDAMENTALS OF TESTING

Tester Training Course 2011

Explorapedia, World of Nature, version 1.0

Pepsi 349 Scandal

European Space Agency Ariane 5

After 37 ½ seconds, was destroyed $370 millions

London Ambulance Service

What happened?

What Failures’ effects?

What Failures’ effects? Can harm:

• people

• companies

• the environment

Can lead to:

• Loss of money

• Loss of time

• Loss of business reputation

• Injury

• Death

DEFECTFAULT

Why do defects occur in products?

Made by human

• Who know some thing, but not everything

• Who have skills, but aren’t perfect

==> Who do make mistakes(errors)

Under pressure

• deadline

• complexity ( system , organization)

• individual reasons

==> no time to check==> systems may be incomplete==> increased frequency of errors

Reliability and Defect

Reliability: the probability that software will not cause the failure of the system for a specified time under specified conditions

Can a system be defect-free?

Can a system be reliable but still contains defects?

Is a ”defect-free” system always reliable?

Error/Mistake

a human action which produces anincorrect result

Defect/Fault/Bug

a manifestation of an error insoftware

Failure

deviation of the software from itsexpected delivery or service

Error – Defect – Failure

Failure is an event; defect is a state of the software, caused by an error

When do defects arise in SD context?

Correct requirement

Designed to meet

requirement

Built to meet design

Product works as expected

Correct requirement

Designed to meet

requirement

Mistakes make in built

Product has bugs in it

Correct requirement

Mistakes make in design

Product has design flaws

Mistakes made in

requirement

Designed to meet

requirement

Wrong product

delivered

What does It cost to fix a defect?

A mistake made by a person;

A run-time problem experienced by a user;

The result of an error or mistake;

The result of a failure, which may lead to an error?

A bug or defect is:

Question

When what is visible to end-users is a deviation from the specific or expected behavior, this is called:

Question

An Error

A fault

A Failure

A defect

A mistake

Testing/checking whether the software performs correctly;

Checking that a previously reported defect has been corrected;

Identifying the cause of a defect, repairing the code and checking the fix is correct;

Checking that no unintended consequences have occurred as a result of a fix

Debugging is:

Question

Why is testing necessary?

Because software is likely to have faults

To learn about the reliability of the software

To fill the time between delivery of the software and the release date

To prove that the software has no faults

Because testing is included in the project plan

Because failures can be very expensive

To avoid being used by customers

To stay in business

Why is testing necessary?

Testing and Quality

Quality: get the satisfaction of all stakeholders

Quality

Budget

Satisfy customers

Technically excellent

Testing help us to measure and improve the quality of software

• give confidence in the quality of software

What is testing?The process consisting of all lifecycle activities, both static anddynamic, concerned with planning, preparation and evaluationof software products and related work products

to determine that they satisfy specified requirements, todemonstrate that they are fit for purpose and to detect defects

This is a suitable definition of testing for any level of testing.

How much testing is enough?

system has20 screens

Average: 10 fields / screen2 types input / field(date as Jan 3 or 3/1)(number as integer or decimal)Around 100 possible values

Total for 'exhaustive' testing:

20 x 4 x 3 x 10 x 2 x 100 = 480,000 tests

If 1 second per test, 8000 mins, 133 hrs, 17.7 days

(not counting finger trouble, faults or retest)

Avr. 4 menus3 options / menu

“Test everything”

When all the testers are exhausted;

When all the planned tests have been executed;

Exercising all combinations of inputs and preconditions

What is exhausive testing?

Question

Infinite time;

No much time;

Impractical amount of time.

How much time will exhaustive testing take?

Question

Testing principle

Exhaustive testing is impossible

Testing everything (all combinations of inputs andpreconditions) is not feasible except for trivialcases. Instead of exhaustive testing, we use risksand priorities to focus testing efforts.

It’s never engough;

When you have done what you planned;

When your customer/user is happy;

When you have proved that the system works correctly;

When you are confident that the system work correctly;

It depends on the risk for your system.

RISK and TESTING The number of test depends on RISK

• RISK of missing important faults

• RISK of incurring failure costs

• RISK of releasing untested or under-tested software

• RISK of losing credibility and market share

• RISK of missing a market window

• RISK of over-testing, ineffective testing

use RISK to determine:

• what to test first

• what to test most

• how thoroughly to test each item

• What not to test (this time)

use RISK to:

• Allocate the time available for testing by prioritising

testing ...

When can we meet our test objectives?

Testing principle

Early testing

Testing activities should start as early as possiblein the software or system development life cycleand should be focused on defined objectives.

Is the software defect free?

Testing principle

Testing shows presence of defects

Testing can show that defects are present, butcannot prove that there are no defects. Testingreduces the probability of undiscovered defectsremaining in the software but, even if no defectsare found, it is not a proof of correctness.

How are defects distributed?

Testing principle

Defect clustering

A small number of modules contain most of thedefects discovered during pre-release testing orshow the most operational failures.

The defect clusters change over time

Testing principle

Pesticide paradox

If the same tests are repeated over and over again,eventually the same set of test cases will no longer find anynew bugs. To overcome this 'pesticide paradox', the testcases need to be regularly reviewed and revised, and newand different tests need to be written to exercise differentparts of the software or system to potentially find moredefects.

Testing principle

Absence-of-errors fallacy

Finding and fixing defects does not help if thesystem built is unusable and does not fulfill theusers' needs and expectations.

Testing principle

Testing is context dependent

Testing is done differently in different contexts.For example, safety-critical software is testeddifferently from an e-commerce site.

Different testing is needed depending upon the application.

All software is tested in the same way.

A technique that finds defects will always find defects.

Which statement is most true?

Question

A technique that has found no defects is not useful.

When time and budget are exhaused.

When there is enough information for sponsors to make an informed decision about release.

When there are no remaining high priority defects outstanding.

When is testing complete?

Question

When every data combination has been exercised successfully.

• Why testing is necessary?

• Failure, Fault, Bug, Defect, Error, Mistake

• What is testing?

• 7 general testing principles

• Testing and Reliability

• Testing and Quality

• Testing and Debugging

Fundamentals of testing 1

Technology

Software Testing Fundamentals - WordPress.com

Chapter 1- Fundamentals of Testing

Chap1. Fundamentals.1 Fundamentals on Testing and Design for Testability

01 Testing Fundamentals

Chapter 1: Performance Testing Fundamentals - Packt · PDF fileChapter 1: Performance Testing Fundamentals. Chapter 2: Recording Your First Test. Chapter 3: ... er.apache.org/usermanual/jmeter

Unit testing solid fundamentals

100 Fundamentals of Hypothesis Testing 1

Unit Testing Fundamentals

Charge Roller Testing: Fundamentals and Practice › wp-content › uploads › 2015 › 04 › ... · 1 Charge Roller Testing: Fundamentals and Practice Quality Engineering Associates

Wp-t-304-Fiber Testing Fundamentals Whitepaper Brown (1)

Fundamentals Formation Testing Overview

Python Testing Fundamentals

DIFFUSION TESTING FUNDAMENTALS INTRODUCTION

Hypothesis Testing Fundamentals

Testing- Fundamentals of Testing-Mazenet solution

The Fundamentals of Testing

BGA Reliability Testing Fundamentals

Software testing fundamentals

Well Testing Fundamentals - Markit Testing Fundamentals Well Testing Fundamentals)) + )60"0!"0 /& "! + "-"/#,/*"!20&+$ /(&1 0 ")) "01&+$0, 4 /" ")) "01

HVAC Fundamentals & Testing