3. introduction to software testing

1

Testing, QA, and QC

Discovering the unexpected is more important than confirming the known.

Introduction to Testing, QA, and QC

2

What is Testing?


Testing is the process of identifying defects, where a defect is any

variance between actual and expected results.

Testing should intentionally attempt to make things go wrong to

determine if things happen when they shouldn't or things don't happen

when they should. It is oriented to 'detection'.

3

What is Software Testing?


Software testing is a process used to identify the correctness,

completeness, and quality of developed computer software. It includes

a set of activities conducted with the intent of finding errors in software

so that it could be corrected before the product is released to the end

users.

In simple words, software testing is an activity to check whether the

actual results match the expected results and to ensure that the software

system is defect free.

4

Why Software Testing?


To get adequate trust and confidence on the product.

To meet the Organizational Goals like meeting requirements, satisfied

customers, improved market share, zero defects, etc

Since the software can perform 100000 correct operations per second, it

has the same ability to perform 100000 wrong operations per second, if

not tested properly.

Testing is important because software bugs could be expensive or even

dangerous (See examples in the next slides.).

5

What is Quality Assurance (QA)?


Quality means consistently meeting user requirements in terms of cost,

delivery, schedule and service.

Quality Assurance involves the entire software development process.

Process is monitoring and improving the process, making sure that any

agreed-upon standards and procedures are followed, and ensuring that

problems are found and dealt with.

QA ensures that all parties concerned with the project adhere to the

process and procedures, standards and templates, and test readiness

reviews.

QA focus early in life cycle.

It is oriented to ‘prevention’.

6

What is Quality Control (QC)?


QC is a process by which entities review the quality of all factors

involved in production. This process places an emphases on 3 aspects

Elements such as job management, defined and well managed

processes, performance and integrity criteria etc.

Competence such as knowledge, skills, experience, and qualifications

etc.

Soft elements such as confidence, organization culture, motivation,

team spirit, and quality relationships etc.

QC ensures that the products or services that are provided to meet

specific requirements and characteristics, such as being dependable,

satisfactory, and safe.

7

QA versus QC


QA QC

A planned and systematic set of activities

necessary to provide adequate confidence that

requirements are properly established and

products or services conform to specified

requirements.

The process by which product quality is

compared with applicable standards; and

the action taken when nonconformance is

detected.

An activity that establishes and evaluates the

processes to produce the products.

An activity which verifies if the product

meets pre-defined standards.

Helps establish process. Helps implement the process.

Identifies weaknesses in processes and improves

them. Identifies defects for the primary purpose of

correcting defects.

Identifies defects for the primary purpose

of correcting defects.

QA is the responsibility of the entire team. QC is the responsibility of the tester.

Prevents the introduction of issues or defects. Detects, reports and corrects defects.

QA improves the process that is applied to multiple

products that will ever be produced by a process.

QC improves the development of a

specific product or service.

QA personnel should not perform quality control

unless doing it to validate quality control is

working.

QC personnel may perform quality

assurance tasks if and when required.

8

Some Software Failures


In UK in June of 2012 RBS/Natwest banking payment system crashed for

several days. This failure left millions of people unable to perform

transactions like Money has not been flowing into accounts after being

deposited, payrolls have been delayed, credit ratings downgraded, utility bills

not being met.

It’s the worst computer system outage the UK financial system has ever seen.

In Asia in July of 2011 reported that software bugs in a national

computerized testing and grading system resulted in incorrect test results for

tens of thousands of high school students. The national education ministry had

to reissue grade reports to nearly 2 million students nationwide.

A smart phone online banking application was reported in July 2010 to have

a security bug affecting more than 100,000 customers. Users were able to

upgrade to a newer software version that fixed the problem.

9



It was reported in August 2009 that a large suburban school district

introduced a new computer system that was 'plagued with bugs' and

resulted in many students starting the school year without schedules or

with incorrect schedules, and many problems with grades. Upset students

and parents started a social networking site for sharing complaints.

Software system problems at a large health insurance company in

August 2008 were the cause of a privacy breach of personal health

information for several hundred thousand customers, according to news

reports. It was claimed that the problem was due to software that 'was not

comprehensively tested'.

10



Tens of thousands of medical devices were recalled in March of 2007

to correct a software bug. According to news reports, the software

would not reliably indicate when available power to the device was too

low.

Etc.

All these problems and failures due to lack or ineffectiveness of

testing effort.

11

Successful Software Testing


It depends on how efficiently the Testing Process is integrated into

the Software Development Process.

12

Why does software have bugs?


Miscommunication or no communication - as to specifics of what an

application should or shouldn't do (the application's requirements).

Software complexity - the complexity of current software

applications can be difficult to comprehend for anyone without

experience in modern-day software development. Multi-tier

distributed systems, applications utilizing multiple local and remote

web services applications, data communications, enormous relational

databases, security complexities, and sheer size of applications have

all contributed to the exponential growth in software/system

complexity.

Programming Errors

13



Changing requirements (whether documented or undocumented) - the end-

user may not understand the effects of changes, or may understand and

request them anyway - redesign, rescheduling of engineers, effects on other

projects, work already completed that may have to be redone or thrown out,

hardware requirements that may be affected, etc. If there are many minor

changes or any major changes, known and unknown dependencies among

parts of the project are likely to interact and cause problems, and the

complexity of coordinating changes may result in errors. Enthusiasm of

engineering staff may be affected. In some fast-changing business

environments, continuously modified requirements may be a fact of life. In

this case, management must understand the resulting risks, and QA and test

engineers must adapt and plan for continuous extensive testing to keep the

inevitable bugs from running out of control

Time pressures - scheduling of software projects is difficult at best, often

requiring a lot of guesswork. When deadlines loom and the crunch comes,

mistakes will be made.

14



Egos - people prefer to say things like:

• No problem

• It’s a piece of cake

• I can whip that out in a few hours

• It should be easy to update that old code

instead of:

• That adds a lot of complexity and we could end up making a lot of mistakes

• We have no idea if we can do that; we'll wing it'

• I can't estimate how long it will take, until I take a close look at it

• We can't figure out what that old spaghetti code did in the first

place' If there are too many unrealistic 'no problem's', the result

is bugs.

15



Poorly documented code - it's tough to maintain and modify code that is

badly written or poorly documented; the result is bugs. In many

organizations management provides no incentive for programmers to

document their code or write clear, understandable, maintainable code. In

fact, it's usually the opposite: they get points mostly for quickly turning out

code, and there's job security if nobody else can understand it ('if it was

hard to write, it should be hard to read').

Software development tools - visual tools, class libraries, compilers,

scripting tools, etc. often introduce their own bugs or are poorly

documented, resulting in added bugs.

Etc.

16

Approaches to Software Testing


There are different approaches to software testing.

• White Box and Black Box Testing

• Dynamic and Static testing

• Manual and Automated Testing

• Verification and validation Techniques

White Box Testing: This testing that takes into account the internal mechanism

of a system or component.

Example: Unit Testing

Black Box Testing: This testing that tests the functionality of an application as

opposed to its internal structures or workings.

Example: Functional Testing

Static Testing: In this testing software code will be examined without being

executed

Example: Code Inspections, Reviews, Walk-through etc.

17

Reviews and Walkthrough


Walk-through: It will be conducted informal, usually conducted by the

members of the group which will include the author. It’s main purpose is

understanding. Usually, little or no preparation is required.

Example: Improving Programming Logic, Design approach, Test case steps,

Test scenarios etc.

Reviews: It is similar to Walk-through, conducted formal by a group of

persons (author need not be one among them) look for errors, mistaken

assumptions, lack of clarity and deviation from standard practice. It’s main

purpose is decision making.

Example: Review of Business Requirements, Test Plans, Process Improve

documents, Project Plans etc. and make go or no go decisions.

Inspection: A formal individual and group checking using standards as per

defined rules and procedures. It’s main purpose is find defects.

Example: Inspect Organization policies, User manuals & procedures etc.

18



Dynamic Testing: In this testing software code will be executed and tested without

necessarily being examined.

Example: Black Box testing

Manual Testing:

Manual testing is the process of manually testing software for defects. It requires

Testers manually execute test cases without using any automation tools.

Automation Testing:

Automation Testing means using an automation tool to execute your test case

suite. The automation software can also enter test data into the System Under

Test , compare expected and actual results and generate detailed test reports.

19



Verification: It is a process of determining whether output of one phase of

development conforms to its previous phase.

It is a set of activities carried out to check whether we are ‘building the system

right’.

Verification is concerned with phase containment of errors

Example: Set of white box testing activities.

Validation: It is a process of determining whether a fully developed system

conforms to its Software Requirement Specification (SRS) document.

It is a set activities carried out to check whether we are ‘building the right system’.

Validation is concerned about the final product to be error free.

Example: Set of black box testing activities.

Education

3. introduction to software testing