Upload
hollie-hopkins
View
224
Download
1
Embed Size (px)
Citation preview
Chapter 1:Fundamental of
TestingSystems Testing & Evaluation (MNN1063)
Fundamental of Testing
0 • Why is testing necessary?0 • What is testing? 0 • General testing principles 0 • Fundamental test process0 • The psychology of testing0 • Code of ethics
Why is testing necessary?
0Human make mistakes0Mistakes – unimportant, expensive or dangerous0Must check – some things can always go wrong0Mistakes may comes from:
0 Bad assumption0 Blind spot
Software Systems Context
0Software that did not work as expected?0 Error on a bill, delay on credit card processing, website
that did not load correctly etc.0Different software carry different level of risk0Problems – trivial, costly and damaging (loss of
money, time or business reputation, or results in injury or death
RISK: A factor that could result in future negative consequences; usually expressed as impact and likelihood
Causes of Software Defects
0People who design or build the software can make mistakes
0Mistakes means flaws in the software => defects/bugs/faults
0Defects may caused system to fail to do what it should do (or do something it shouldn’t) => causing failure
Error(Mistakes)
Defects(Bugs/Faults)
Failure
Do our mistakes matter?
0Think about the consequences of mistakes…0Human being, programmer, tester => can make error0Error => defects => defect code executed => systems
failure0Systems and project are complicated
Defects & Failures
0Errors in specifications, design and implementation0Errors in use of the systems0Environmental conditions0 Intentional damage0Potential consequences of earlier errors, intentional
damage, defects and failures
When do defects arise?Requirement 1
Correct requirement
Designed to meet requirement
Build to meet design
Product works as expected
Requirement 2
Correct requirement
Designed to meet requirement
Mistakes made in build
Product has bugs in it
When do defects arise?Requirement 3
Correct requirement
Mistake made in design
Built to meet design
Product has design flaws
Requirement 4
Mistakes made in requirement
Design to meet requirement
Built to meet design
Wrong product delivered
What is cost of defects?0The cost of finding and fixing the defects rises
considerably across the life cycle
Cost
Time
Cost of fi
nding and fixing defects in
creases
over time
Requirements Design Build Test Live use
Role of Testing in Software Development, Maintenance & Operations
0We have seen human errors can caused defect or fault at any stage of software development life-cycle
0Rigorous testing is necessary during the maintenance & development to identify defects => to reduce failures in operational environment
0To increase quality of operational systems0Executing tests helps us move towards improved
quality of product and services
Testing & Quality
0Testing helps us to measure quality of software in terms of:0 The number of defects found0 The number of tests run0 The system covered by the test
0Both functional and non-functional software requirements
0Testing gives us confidence in quality0Poor test may uncover few defects
What is Quality?
0Project aim to deliver software that meet customer needs & correct specification
0Validation – “ is this the right specification”0Verification – “is the system correct to specification”0Need to understand what customer understand by
quality and what their expectation are
What is root cause analysis?
0Detect failures => track them back to their root cause0Several ways & involving a group brainstorming ideas
and discussion0Assignment #1:
0 Find 3 ways to do Root Cause Analysis.0 Discuss how it works.
How much testing is enough?
0We have a choice:0 Test everything?0 Test nothing?0 Test some part?
0Example: One-digit field – how many test is enough?0Real example: one screen with 15 input fields0Pressures on project: time & budget0Need test approach which provides the right amount of
testing0Risk assessment – helps to decide how much testing
to do
What is testing?
0Recall the common objectives of testing0What is testing? – checking if the software is OK?0Lets discuss an analogy of: driving test0The format of the driving test and conduct of
examiner:0 The test is planned & prepared0 The test has known goals0 The test is therefore carried out to show that the driver
satisfied the requirement for driving & demonstrate that they are fit to drive
Defining software testing
0 ISTQB definition of software testing0Process – Testing is a process rather a single activity –
there are series of activities 0All life cycle activities – software development life
cycle (discuss this in Chapter 2)0 Requirements 0 Design (to prevent defects introduced into code) – test
via the test design
Defining software testing
0 Both static and dynamic 0 Dynamic testing – execute codes & demonstrate the results0 Static – reviewing documents including the source code
0 Planning – to manage the testing: plan, control, report testing progress, close testing
0 Preparation – selecting test conditions & design test cases0 Evaluation – check results and evaluate software under
test & completion criteria0 Software products and related work product – not only test
code, requirements & design specification
Defining software testing
0Objectives for testing0 Determine the software product satisfy specified
requirements0 Demonstrate that the software products are fit for
purpose0 Detect defects
0Objectives may be different in each stage
General testing principles
0Principle 1: Testing shows the presence of defects0Principle 2: Exhaustive testing is impossible0Principle 3: Early testing0Principle 4: Defect clustering0Principle 5: Pesticide paradox0Principle 6: Testing is context dependent0Principle 7: Absence-of-errors fallacy
Fundamental test process
0Fundamental test process:0 Planning and control0 Analysis and design0 Implementation and execution0 Evaluation exit criteria and reporting0 Test closure activities
Test Planning and control
0We make sure we understand the goal and objectives of the customer, stakeholders, the project and the risks
0Based on the understanding, we set the goal & objectives for the testing itself
0To help us, we may have:0 Test policies – rules for testing0 Test strategy – Overall high-level approach
Test planning and control
0 Test planning has the following major task:0 Determine the scope and risks; Identify the objectives of
testing0 Determine the test approach (techniques, test items,
coverage, identifying and interfacing the teams involve in testing, testware)
0 Implement the test policy and/or the test strategy0 Determine the required test resources (people, test
environment, PCs)0 Schedule test analysis and design tasks, test implementation,
execution and evaluation0 Determine the exit criteria
Test planning and control
0Test control is an on-going activity0We compare the actual progress with planned
progress0Test control has the following major tasks:
0 Measure & analyze the results of reviews and testing0 Monitor and document progress0 Provide information on testing0 Initiate corrective actions0 Make decisions
Test Analysis and Design
0Test analysis and design has the following major tasks:0 Review the test basis (product risk analysis,
requirements, architecture, design specification and interface)
0 Identify test conditions (behavior and structure)0 Design the test (Chapter 4)0 Evaluate testability of the requirements and systems0 Design the test environment set-up and identify any
required infrastructure and tools
Test implementation and Execution
0 Implementation:0 We take the test conditions and make them into test
cases and set up the test environment0 Having the high-level design – we now build them0 We set up the environment0 Build our test data
0Execution:0 Execute the test cases0 Log the outcome of test execution0 Compare actual result with expected result
Evaluating exit criteria and reporting
0 Is an activity where test execution is assessed against the defined objectives
0Evaluating exit criteria involve following major tasks:0 Check the test logs against the exit criteria0 Assess if more test needed0 Write a test summary report