Software Testing Workshop 2014

May 3rd 2014. FAST, Islamabad.

Introduction

About Us • Majd Uddin 15+ years of experience with a decade in software testing

• “Promoting” software quality:

• M.Sc. (CS) from PUCIT, Lahore. B.E. (Mech) from UET, Taxila.

• Other interests include hiking, reading and blogging.

About Us

• Sohail Sarwar 5+ years of experience in Software Industry

– NUST-SEECS and Kraysis-Soft Lahore

– NUST-SEECS and DTS Japan

– NetsolAce Inc.

• Associate Software Quality Analysis

– Bentley Systems Pakistan (www.bentley.com)

• Software Quality Analyst (Platform Technologies)

– National Engineering and Scientific Commission (NESCOM)

• Assistant Manager (Software Engineering)

• M.S.(IT)- Specialization in Software Engineering from SEECS ,NUST

• PhD (CS) – Iqra University Islamabad [In Progress]

ABOUT YOU

Today’s mission

Breaking down the Mission

Knowledge Tester

Asking Questions

Designing tests

Coding tests

Exploring systems

Is this a Quality phone?

Photo from HubSpot free image library

(Agile) Testing Quadrants

From ‘Agile Testing’ book by Lisa Crispin and Janet Gregory

Functional Tests Examples

Story Tests Prototypes Simulations

Exploratory Testing Scenarios

Usability Testing UAT (User

Acceptance Testing) Alpha/Beta

Unit Tests Component Tests

Performance Tests Load Testing

Security Testing “ility” Testing

Business facing

Technology facing

Critiq

ue P

rod

uct

Sup

po

rt t

he

team

Q1

Q3

Q4

Q2

Automated & Manual

Automated Tools

Manual

Imaginary Software

Client Programmer Tester

EXERCISE 1

Asking Questions

END OF MODULE 1

Software Testing Workshop 2014

May 3rd 2014. FAST, Islamabad.

Designing Tests

Test Cases

A Test Case or a Test is an Action

performed on System Under Test (SUT)

where we expect a Behavior and

We can state if a Test Passed, Failed

or needs discussion

Ideas for Tests (Heuristics)

• S is for Structure

• F is for Function

• D is for Data

• P is for Platform

• O is for Operations

• T is for Time

• H is for History

• I is for Image

• C is for Comparable Products

• C is for Claims

• U is for User Needs

• P is for Product

• P is for Purpose

• S is for Statutes

SFDPOT by James Back, HICCUPS by Michael Bolton

Documenting Test Cases

• Test Cases Repository

• Feature Test Plan in form of sheets/docs

• Mind Maps

• Check Lists

• …

Just enough Documentation

ےہ دایتسب یلھچم اتزہ اہیں

EXERCISE 2

Designing Tests

END OF MODULE 2

Software Testing Workshop 2014

May 3rd 2014. FAST, Islamabad.

Coding Tests

The objectives

1

Software Test

Automation?

2

Automation Tools

3

Automation Implications

Software Test Automation

• Software Test Automation is to control the execution of tests by comparing actual and expected outcomes.

– Tester writes scripts and uses another software to test the

software

• Automated tests once developed, can be run repeatedly

http://www.tutorialspoint.com/software_testing/

Rationale for Test Automation

• Manual execution of test plans for regression testing is laborious and time consuming.

– Test automation is handy for products with long maintenance life

• It can test the application from performance, load and stress point of view.

• Saves time and money compared to error-prone manual testing.

• Improvises test accuracy, coverage and productivity.

How we can automate?

What to Automate???

Choosing an automation tool

Creating a testing infrastructure for automation

Designing of test cases

Development of scripts

Execution of Scripts

Analysis of Results

Sharing the results with team Maintenance of Scripts

Automation Tools

• Different tools with respective features are there: – API Testing

• Nunit, Junit

• GoogleTest, Visual Studio Test Suite

– GUI Testing • SilkTest, TestComplete

• WinRunner

– Performance Testing • LoadRunner, WebLoad

NUnit

• NUnit is a unit testing framework for the .NET Language

• NUnit promotes the idea of "first testing then coding“

• NUnit is an open source framework which is used for writing & running tests

• Provides annotations, assertions and test runners

Environment for NUnit

• Microsoft Visual Studio 2010

• .NET 4.0

• NUnit 2.6.0

xUnit Design

Code to Test

Test Code

Test Runner

Unit Tests in NUnit

• Unit Test is code that ensures another part of code (method) works as expected.

• Unit tests can be written using a test framework such as Nunit.

• Unit test can be characterized with a known input and an expected output.

• Input corresponds to pre-condition and expected output corresponds to post-condition.

• Each requirement may have two test cases: positive and negative ones.

Writing Tests using NUnit

• Let’s write our first test……..

Automating Adder Application

• Application: Adder application which takes two 2-digit no.s and displays the sum.

? 2

? 3

5

Adder Application Test Scenarios -I

Adder Application Test Scenarios -II

NUnit Features

• Assertions – Equal, Null, True [with respective vice versa]

• Annotations of Nunit – [Test] – [Test Fixture]

• [SetUp] - before each test method is called • [Teardown] - after each test method is run • [TestFixtureSetUp] - performed once prior to executing all tests in

fixture • [TestFixtureTearDown] - performed once after all tests are completed

– [ExpectedException] – [Ignore()]

Parameterized Tests • Help in running the same tests over and over

again using different values

• Nunit offers the same feature via [TestCase(12,3,4)]

[TestCase(12,2,6)]

[TestCase(12,4,3)]

public void DivideTest(int n, int d, int q)

{

Assert.AreEqual( q, n / d );

}

Generating Test Documentation

[TestFixture, Description("Fixture description")] public class SomeTests { [Test, Description("Test description here")] public void DocTest() { /* ... */ } }

A Step Ahead----

• DAO design Pattern

– Business Logic Layer

– Data Transfer Objects

– Data Access Objects

• DB Utility

Do and Don’t of Test Automation

• Smoke Testing – Run with every build to make go/no-go decision on deeper testing.

• Regression Testing – Assures that a change, bug fixes or new functionality , did not introduce

new bugs

• Data Driven Testing – Testing the operations with their related data sets in a framework

• Exception or Negative Testing – Forcing error conditions in the system.

• Functional Testing – Testing that operations perform as expected for relatively stable

application

• Testing on different Operating Systems/Environments • Repetitive tasks

Do and Don’t of Test Automation

• Exploratory testing

• Usability testing

• Documentation Testing

• Graphics Testing

• For Features which are under changes/unstable

Software Testing, A research Perspective

• Automated Tools for Reliability calculation of Critical system

• Test Case Generation for Regression testing

• Test Case prioritization technique

• Software testing in offshore culture with agile models

• Testing of semantic Web applications.

Questions

Questions

• Automated Testing Best Practices – http://downloads.seapine.com/pub/papers/AutomatedTestingBestPractices.pdf

• Open Source Testing Tools in C# – http://csharp-source.net/open-source/testing-tools

• Automation Concepts – http://www.exforsys.com/tutorials/testing/automated-testing-best-practices.html

• Google C++ Testing Framework – http://code.google.com/p/googletest/wiki/GoogleTestPrimer

• 'Testing Computer Software' book by Cem Kaner, Jack Falk and Hung Q. Nguyen.

References..

END OF MODULE 3