Software Quality Engineering & Testing Basics - Belal Raslan

Software Quality Engineering & Testing Basics

Belal RaslanDate: 25-11-2007


Contents

• Part 1: Classic SDLC Models ..

• Part 2: Software Testing ..

• Part 3: Software Quality Engineering ..

Part 1

Classic SDLC Models

The classic SDLC Models:

1. Big Bang2. Code and fix3. Waterfall4. Spiral


Part 1 : Classic SDLC Models


Big-Bang Model :

• The Approach

People+Money+Energy=Perfect Software or Nothing!

• Advantages

Simple, needs less planning, less formal development process

• Disadvantages

No formal testing activities, can’t monitor, high risk



Code and Fix Model :

• The Approach

Informal specification loop of code and fix till enough-is-enough decisionRelease

• Advantages

Little planning and documentation

work.

• Disadvantages

Hard to manage the testing



Waterfall Model :

• The Approach

Discrete phases, a review after each phase will decide to move next or stay at this phase till it’s ready.

•AdvantagesEasy to test (clear requirements)

• Disadvantages

High bugs fixing cost (bugs creep), customer is far from this till the end



Spiral Model :

• The Approach

Round1 (Waterfall Prototype)Evaluate Round1Round2 (Waterfall Prototype)Loop

•AdvantagesEasy to test, the customer is tightlyinvolved

• Disadvantages

Need risk-assessments expertise


Part 2

Software Testing

Part 2 : Software Testing

What is software testing?

A quality control activity aimed at evaluating a software item against the given system requirements.



What is software testing?

A quality control activity aimed at evaluating a software item against the given system requirements.


Testing is

a QC

activity


What is the V&V?

Verification: Confirming that something (software) meets its specification.

Validation: Confirming that it meets the user’s requirements.

The two major V&V activities are reviews, and testing. *



What is the V&V?

Verification: Confirming that something (software) meets its specification

Validation: Confirming that it meets the user’s requirements.

The two major V&V activities are reviews, and testing.


Testing is

a part of V

&V

activitie

s


What should the tester do?

1. Find the bugs

2. Find them early

3. Make sure that they have been fixed



What should not the tester do?

1. Fix the bugs! *

2. Trust the others!

3. Test with no requirements!

4. Not to report the obvious bugs!

5. Test with good faith!



What are the testing types?

1. Static Testing

Testing a software that is not running.

2. Dynamic Testing

Test running software.

Unit (white box), Functional, Integration, System, Regression, Volume, Load, Stress Integration, Usability, Localization, Live (Production), Compatibility, Smoke, Ad-hoc, User Acceptance



2. Dynamic Testing (Cont’d)

Unit Test: A test of one component of the system.

White Box Test: Testing software with complete knowledge of its internal code and logic.

Black Box Test: Testing software based on output only without any knowledge of its internal code or logic.

Gray Box Test: Testing software with some knowledge of its internal code or logic.

Functional Test: Testing functional requirements of software, such as menus and key commands.

System Test: Overall testing in the user environment.




Acceptance Test: The test performed by end users to approve the system.

Alpha Test: The first testing of a product in the production environment before going live.

Beta Test: Testing done by end users.

Volume testing: testing a software application for a certain data volume.*

Load testing: creating demand on a system or device and measuring its response.

Stress testing: determine the stability of a given system or entity.

Compatibility testing: measures how well the system operates on different operating environment. *




Recovery Test: Testing a system's ability to recover from a hardware or software failure.

Accessibility testing: measure how well the system interface serve users with disabilities.*

Negative Test: Using invalid input to test a program's error handling.

Regression testing: Testing to make sure that the fix is working as expected.

Ad Hoc Test: Informal testing without a test case.

Automated Test: Using software to test software.*



What is the ‘bug’ ?

Things the software does that while it is not supposed to do, OR something the software doesn't while it is supposed to.



What are the difference between the bug severity and priority?

Priority means how urgently bug is needed to fix.

Severity means how badly it harms the system.



How to report a bug?


Report the pattern not an example.

Don’t describe what's wrong, only!

Be direct to the point (don’t tell a story!).

Report the bug, but not how to solve it.

Don’t mix between priority and severity.


How to report a bug? (Cont’d)


Don’t use: CAPS, red letters, red circles, ‘!’ , ‘?’.

Don’t use your personal judgment.

Write with the fixer language, not with yours.

Minimize the options.

Don’t use the word ‘Sometimes’.


How to report a bug? (Cont’d)


Don’t group the bugs.

Don’t update the bug description.

Don’t update the bug severity.


When not to fix a bug?


It’s too risky to fix it.

It’s not worth fixing.

There is no enough time.

Its not really a bug.


What are the testing activities?


1. Test Plan.

2. Test Analysis.3. Test Design. 4. Test Cases Development. 5. Test Cases Execution. 6. Test Reporting. 7. Bugs Reporting. 8. Releasing reporting.

9. User Acceptance Testing.


When to stop testing?


1. Deadline has be been reached.

2. Test cases completed with passed percentage.

3. End of test budget.

Part 3

Software Quality

Part 3 : Software Quality

What is 'Software Quality Assurance’?


Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to 'prevention'.

What is the ‘Quality Software’?


Quality software is fairly bug-free, delivered on time and within budget, meets requirements and/or expectations, and is maintainable. However, quality is obviously a subjective term.


What are the main quality attributes?


Availability: Is it available when and where I need to use it?

Efficiency: How few system resources does it consume?

Flexibility: How easy is it to add new features?

Installability: How easy is it to correctly install the product.

Interoperability: How easily does it interconnect with other systems?

Maintainability: How easy it is to correct defects or make changes?

Portability: Can it be made to work on other platforms?




Reliability: How long does it run before causing a failure?

Reusability: How easily can we use components in other systems?

Testability: Can I verify that it was implemented correctly?

Usability: How easy is it for people to learn or to use?

Performance: the response time, utilization, and throughput behavior of the system.




Security: system’s ability to resist unauthorized attempts at usage or behavior modification, while still providing service to authorized users.


What are the types of software ‘reviews’?


1. Peer Review

2. Walkthrough

3. Inspection

More

form

al


What is the characteristics of ‘Peer reviews’?


Informal.

Audience: Owner(author) and peer(s) of similar expertise (no managers).

It require guidelines and checklist.


What is the characteristics of the ‘walkthrough’?


More formal.

Audience: 3-5 , at least 1 senior, the owner act as a reader and recorder.

The owner should send a copy of things that need review prior to the meeting.

Reader should write a report with the feedback and how to address it.

2nd meeting can be conducted to follow up.


What is the characteristics of the ‘Inspection’?


The most formal.

Audience: typically 3-8 people including a moderator, reader, and a recorder to take notes.

The reader should not be the owner.

The owner should send a copy of things that need review prior to the meeting.

Each Inspectors should review the code for a different perspective.


What is the characteristics of the ‘Inspection’? (Cont’d)


After the inspection meeting the moderator call for another meeting to discuss the feedback and how the owner will address them.

Require training.


Quality standards and auditing organizations


ISO (International Organization for Standardization).

CMMI (Capability Maturity Model Integration).

IEEE (Institute of Electrical and Electronics Engineers).


ISO 9001:2000


ISO 9001 is a series of documents that define requirements for the Quality Management System Standard. ISO 9001 is one of the documents in this set; it contains the actual requirements an organization must be in compliance with to become ISO 9001 Registered.


Why do companies want ISO 9001:2000 ?


Many organizations decide to Implement ISO 9001 and obtain registration because it assures customers that the company has a good Quality Management System (QMS) in place. An organization with an effective QMS will typically meet customer expectations better than an organization that does not have an effective QMS. Many organizations require their suppliers to have ISO 9001 Registration.


What is CMMI ?


Capability Maturity Model® Integration (CMMI) is a process improvement approach that provides organizations with the essential elements of effective processes. It can be used to guide process improvement across a project, a division, or an entire organization. CMMI provide guidance for quality processes, and provide a point of reference for appraising current processes.


What are the 5 levels of CMMI ? (Cont’d)


Level One – Initial

Company has no standard process for software development. Nor does it have a project-tracking system that enables developers to predict costs or finish dates with any accuracy.

Level Two - Managed

Company has installed basic software management processes and controls. But there is no consistency or coordination among different groups.




Level Three – Defined

Company has pulled together a standard set of processes and controls for the entire organization so that developers can move between projects more easily and customers can begin to get consistency from different groups.

Level Four – Quantitively Managed In addition to implementing standard processes, company has installed systems to measure the quality of those processes across all projects.




Level Five - OptimizedCompany has accomplished all of the above and can now begin to see patterns in performance over time, so it can tweak its processes in order to improve productivity and reduce defects in software development across the entire organization.


What is the IEEE ?


The Institute of Electrical and Electronics Engineers" (IEEE) is a non-profit organization that develops, defines, and reviews electronics and computer science standards.


Documents

Software Quality Engineering & Testing Basics - Belal Raslan