deeper IBM Global Business Services – Application Services

deeper

IBM Global Business Services – Application Services

© Copyright IBM Corporation 2005

April 11, 2023

Version 1.0

Application Testing Services

University of Stellenbosch – System Testing

DisclaimerThis document does not constitute a formal offer, but serves to outline the discussion elements for contractual consideration based on IBM’s standard terms and conditions.

University of Stellenbosch – System Testing2



The current situation




Ad Hoc Statements

John Roodt : Senior Architect

“…you guys are brutal. Brutally honest…”

Jolene Saayman : Lead Java Developer

“…if it weren't for you I would have left this company a long time ago…”

Adam Granger : Lead Java Developer

“…you are our safety net…”

Michael Kiergard : Customer Project Manager

“…you must be involved. I trust you guys…. I don’t trust the developers…”

Karen Greeves : Project Delivery Executive

“…I can rest assured that after it has been through your hands, then the application is pretty solid…”

Meike Voight : SAS Developer

“…but what are they going to test if I have already tested it?...”Cherryl Lundt : Project Manager

“…I now want to involve you on all my projects...”

“As a software tester, the fate of the world rests on your shoulders. This statement is not an exaggeration if you accept the dual premises that computer software runs the modern world and that all software has bugs – then you will quickly reach the inescapable conclusion that unless the most disruptive bugs are removed, the world as we know it will grind to a halt.”

Scott Loveland, Geoffrey Miller, Richard Prewitt, Jr., and Michael Shannon have collectively spent more than 70 years in the software testing trenches. They currently hunt bugs in the IBM mainframe development laboratory in Poughkeepsie, New York. Collectively they have spent more than 70 years in the software testing trenches.




Agenda

Being a tester Test Strategy : Example case study of mature client Test Methodology : Verification and Validation

________________________________________________________________

What every developer needs to know about testing Characteristics of a good tester




Being a Tester

What people think we are : Testers are negative thinkers, they complain, they like to break things, they take a special thrill in delivering bad news.

BUT WE PROTEST !!

Who we really are : Tester’s don’t complain, we offer evidence. Tester’s don’t like to break things, we dispel the illusion that things work. Testers don’t enjoy giving bad news, we enjoy freeing our clients from the thrall of false belief.




Why Test?

To define the role of a testing group is not easy. The following metaphor might best explain this role:

"A project is like a road trip. Some projects are simple and routine, like driving to the store in broad daylight. But most projects are more like driving a truck off-road, in the mountains at night. Those projects need headlights. The tester is the headlight. The tester illuminates the road ahead so the developers and managers, however they struggle with the map, can at least see where they are, what they're about to run over, and how close they are to the cliff."

The detailed mission of testing groups might differ from company to company, but behind those details is a common factor. Testing is done to find information. Critical decisions about the project or the product are made on the basis of the information found by the test group.




Test Strategy – Example Case study

Business Case

Feasibility

Analysis

Specification Design

Architectural Design

Module Design

Construction

Module Test

Acceptance Test

System Test

Implementation

Integration Test

Business Realization

Business

Process

Owners

Business

IT

Vendor

or

Int IT

Business Governance




Test Strategy – Example Case Study

Business Case

Feasibility

Analysis

Specification Design

Architectural Design

Module Design

Construction

Module Test

Acceptance Test

System Test

Implementation

Integration Test

Business Realization

Business

Process

Owners

Business

IT

Vendor

or

Int IT

• Business case

• Requirement Catalogue

• Change / Decision register• Non Functional Requirement Specification

• Interface Specification• Software System Requirement

Specification• Application Architecture• Technical Architecture Definition• Test strategy• Implementation Strategy• Deliverables Catalogue• Project Contract/Schedule

• AsIs and ToBe model• Logical Data Model• System Context Diagram• Project approach• Project Management Plan

• System Design Document• Physical Data Model• QA Plan• Detailed Test Plan• Implementation Plan

• Application Impact• Implementation Requirement

Specification• User Manual• Test Data• Operational Acceptance Test plan

• Unit Test report• Integration Test report• Data Migration Test report• Factory Acceptance report• Application support documentation• Global HelpDesk documentation• Operational Management support• Operations Handbook• User Acceptance Test report• Stress and Performance report• Application Impact Assessment• Operations Acceptance test report

• Maintenance and Support Contracts• Handover Report to Business Owner• Close Down report




Test plansAUT, AIT, FAT

Test casesAUT, AIT, FAT Execute

AUT

ExecuteAIT

ExecuteFAT

Test plansSIT, UAT, SPT

Test casesSIT, UAT, SPT

ExecuteSIT

ExecuteSPT

ExecuteUAT

PromotePROD

WarrantyPeriod

DesignAnalysis Construct and Vendor test Cust Test Impl Prod

ExecuteOAT

ExecuteAIA

Test Strategy – Example Case Study




Testing Methodology - ValidationARE WE BUILDING THE RIGHT SYSTEM?

Ideal Reviewers:

Review Granularity

Business Owners

Business Representative

Business Analyst

Conceptual Application

Business Owner

Architects

Test Engineers

Prototypes

Simulations

High Level Designs

Developers

Deployers

Architects

Detailed Design

ExecutionActivities

For reviewphases

Requirements/Specification reviews

Architectural Design reviews

Detailed Design reviews

Test Planning




Ideal Reviewers:

Developers

DB / Net / System Administrators

Deployers

Structural

(“White Box”)

Test Engineers

Test Technicians

(Some) Developers

Behavioral

(“Black Box”)

Tech Support / Help Desk

Sales / Marketing

Business Analyst / User

Customer

Live

(“Alpha/Beta”)

ExecutionActivities

For reviewphases

Unit

Component

Integration

System

Acceptance

Pilot

Testing Methodology - VerificationARE WE BUILDING THE SYSTEM RIGHT?

Review Granularity




What every developer needs to know :1. The product is more than Software

Application

-Code

-Framework

Documentation

-Manuals

-Operator instructions

Kit OS Infrastructure




What every developer needs to know :2. Quality is more than the lack of bugs (FURPS)

Functionality

Suitability

Correctness

Interoperability

Compatibility

Security

Installability

Usability

Understandability

Learnability

Operability

Performance

Reliability

Maturity

Fault tolerance

Integrity

Recoverability

Safety

Maintainability

Analyzability

Changeability

Stability

Testability

Portability

Does it solve the problem when it works

Is it practical to use

Does it keep on working Is it economical to build and maintain




What every developer needs to know :3. Quality Assurance is more than testing

Quality Assurance is everything you do to minimizethe risk of failure and to promote excellence

Risk Management

Customer Involvement

Skillfull developers

Process definition and improvements

Inspections and active testing

Experience based improvements




What every developer needs to know :4. Testing is hard to do

Anticipate your users:

Data

Skills

Actions

Expectations

Environment

…to examine a product that is..

Invisible

Volatile

Sensitive

Complex

Unfamiliar

...using a process that is often…

Endless

Ambiguous

Negative

Boring

Laborious

Think of the permutations!!

Functions vs Input data vs state of data

Products vs platforms

External factors

Expectations of customers

Timeframes

Versions of the product




What every developer needs to know :5. You can make testing easier

Document your design User internal error checking Test units before Integration Tell the tester what is new, concerns you or is shaky Test the build yourself…….First Evolve product in functional layers.. Ie Iterative development with testability in

mind Build in testability




Characteristics of a good tester Know at least some programming. There’s a popular myth that testing can be staffed with people who have little or no programming knowledge.

Since they’re testing software, without know some programming, they can’t have any real insights into the kinds of bugs that come into software and the likeliest place to find them.

Know the application. The ideal tester has deep insights into how the users will exploit the program’s features and the kinds of cockpit errors that users are likely to make.

Intelligence. The single most important quality for testers (just as for programmers) is raw intelligence, good testers, just as programmers, are smart people.

Hyper-sensitivity to little things. Good testers notice little things that others miss or ignore. Testers see symptoms, not bugs. Tolerance for chaos. People react to chaos and uncertainty in different ways. If the tester waits for all issues to be fully resolved before starting test

design or testing, he won't get started until after the software has been shipped. Testers have to be flexible and be able to drop things when blocked and move on to another thing that is not blocked. Testers always have many irons in the fire.

People skills. You can be an effective programmer even if you are hostile and anti-social; that won’t work for a tester. Testers can take a lot of abuse from outraged programmers. A sense of humor and a thick skin will help the tester survive. Testers may have to be diplomatic when confronting a programmer with a fundamental goof. Diplomacy, tact, a ready smile- all work to the independent tester’s advantage.

Tenacity (Merriam-Webster’s dictionary explains it as: persistent in maintaining or adhering to something valued or habitual). An ability to reach compromises and consensus can be at the expense of tenacity. The best testers are both socially adept and tenacious where it matters. The best testers are so skilful at it that the programmer never realizes they’ve been had.

Organized. There’s just too much to keep track of to trust to memory. Good testers use files, Databases and all the other accoutrements of an organized mind.

Sceptical. That doesn’t mean hostile. I mean skepticism in the sense that nothing is taken for granted and that all is fit to be questioned. Only tangible evidence in documents, specifications, code and test results matter. While they may patiently listen to the reassuring words from the programmers ("Trust me. I know where the bugs are.") - and do it with a smile - they ignore such in-substantive assurances.

Self-sufficient and tough. If they need love, they don’t expect to get it on the job. They can’t be looking for interaction between them and programmers as a source of ego-gratification and/ or nurturing. Their ego is gratified by finding bugs, with few misgivings about the pain (in the programmers) that such finding might engender.

Cunning. Systematic test techniques such as syntax testing and automatic test generators have reduced the need for much cunning, but the need is still with us and undoubtedly always will be because it will never be possible to systematize all aspects of testing. There will always be room for that off-beat thinking that will lead to a test case that exposes a really bad bug.

Technology hungry. They hate dull, repetitive work; they’ll do it for a while if they have to, but not for long. The silliest thing for a human to do, in their mind, is to pound on a keyboard when they’re surrounded by computers.

Honest. Testers are fundamentally honest and incorruptible. They’ll compromise if they have to, but they’ll righteously agonize over it. This fundamental honesty extends to a brutally realistic understanding of their own limitations as a human being.




Thank you

Documents

deeper IBM Global Business Services – Application Services