Non Functional Testing

Avoiding unpleasant surprises

Agenda

• Definition• Types of Non Functional Testing• Performance Testing– When to Performance Test– Performance Testing Process– Trust IV Performance Testing

• Other Non Functional Testing

Non Functional Requirements

• Requirements of the system not related to business function for example:– Scale– Response time– Service hours– Availability– Recoverability– Security– Operability– Servicability

Non Functional Requirements

• Often– Not captured– Not considered– Assumed– Not tested

• But when not met can make lead to:– total business failure of the application– brand damage– the nastiest of nasty surprises

Non Functional Testing

• Involves the use of tools to simulate user activity, with the benefit that once captured the test script is re-usable

• Needs a near production infrastructure available for it

• Requires much more understanding of the infrastructure

• Is not easy• Is often abandoned

Performance Testing

• Tests for Scale and Response Time in a statistical framework

• Is the most common non functional testing carried out

• Is almost never an exact copy of the full production system

• Is not required for every application

When to Performance Test

• As a result of a risk analysis when one or more of the following apply: – First use of new infrastructure– Exacting requirements for response time or

scale– Business criticality – Regulatory change– Public visibility

• As soon as you can

Performance Testing

• A consultancy lead process:– Identify elements that require test– Identify test characteristics for those elements– Design test environment and tests

• Build test environment , simulating elements not to be tested

• Capture test scripts • Run test• Analyse results

Trust IV Performance Testing

• Making Performance Testing more affordable :– Experienced test consultants who ensure that the most

appropriate choices are made to protect test integrity but minimise the effort

– Skilled technicians that can operate a range of tools effectively– Proven test project managers who ensure that they are carried

out to time and budget

• Making it possible to test more widely and regularly

• Getting rid of unpleasant surprises

Other Non Functional Testing

• Uses simulated load to test other aspects of the system:– Failover, DNoS, batch overrun, etc.

• Many similarities to Performance Testing:– Consultancy lead– Extensive use of specialist tools– Complex test environments

• Done less frequently and hence an even greater value in experience

The Changing Balance Between Risk and Cost• The Risk is combination of probability and

impact– Probability is increasing as complexity rises– Impact is increasing as more systems are

exposed to the public • With Trust IV’s experience the Cost of Non

Functional Testing is now lower and can be confidently predicted

• The balance has changed significantly

Trust IVwww.trustiv.co.uk

0844 870 0301