KUVEMPU UNIVERSITY

B.Sc.(I.T.)5thSEMESTER

SOFTWARE QUALITY AND TESTING

CHITRANGADA CHAKRABORTY

Roll Number: 131B1020

Software testing should play a major role in the definition and improvement of software development processes, because testing is the most precise, most easily measured and controlled part of the software lifecycle. However, unless testing goals are clearly related to true measurements of software quality, the testing may appear to improve, but the software will not. Much of current testing theory and practice is built on wishful thinking. In this paper, testate of the testing art, and the theory describing it, is critically examined. It is suggested that only a probabilistic theory, similar to reliability theory, but without its de ciencies, can describe the relationship between test measurements and product quality. The beginnings of a new theory of \dependability “are sketched

B.Sc.(I.T.)5thSEMESTER 2013

K u v e m p u U n i v e r s i t y

Assignments for B.Sc.(IT) & M.Sc.(IT) Courses

Subject: Software Quality and Testing

Subject Code: BSIT - 54

Assignment: TA

1. What is software testing? Software testing is tougher than

hardware testing, justify your answer.

Software testing means check the correctness and completeness of a product or a

project with respect to client requirement is known as software testing. Software testing

is a critical element of software quality assurance and represents the ultimate process to

ensure the correctness of the product. The quality of product always enhances the

customer confidence in using the product thereby increasing the business economics

Software testing is tougher than hardware testing as because you need to make sure it

works with all the hardware, operating systems, system versions, interfaces, and

different countries, languages etc.

2.Explain the test information flow in a typical software test life cycle.

In every organization testing is an important phase in the development of a software product. However, the way it is carried out differs from one organization to another. It is advisable to carry out the testing process from the initial stages, with regard to the Software Development Life Cycle to avoid any complications. Testing is a complex process and requires efforts similar to software development.

Software Configuration includes a Software Requirements Specification, a Design Specification, and source code.

A test configuration includes a Test Plan and Procedures, test cases, and testing tools.

It is difficult to predict the time to debug the code, hence it is difficult to schedule.

SOFTWARE QUALITY AND TESTING Page 2

B.Sc.(I.T.)5thSEMESTER 2013

Once the right software is available for testing, proper test plan and test cases are

developed. Then the software is subjected to test with simulated test data. After the test

execution, the test results are examined. It may have defects or the software is passed

without any defect. The software with defect is subjected to debugging and again tested

for its correctness. This process will continue till the testing reports zero defects or run

out of time for testing.

3.What is risk in software testing? How risk management improves

the quality of the software?

Risk is future uncertain events with a probability of occurrence and a potential for loss” Schedule Risk:

Budget Risk: Operational Risks: Technical risks: Programmatic Risks:

Risk Management Software is used to balance risk with potential reward. It is used by

insurance companies to determine insurance rates for clients without posing too much risk

to the company.

4.Explain the black and white box testing? Explain with an

example. Which method better? List out drawbacks of each one.

Black-box testing - This testing looks at the available inputs for an application and the expected outputs that should result from each input. It does not have any relation with the inner workings of the application, the process undertaken or any other internal aspect of the application.

SOFTWARE QUALITY AND TESTING Page 3

B.Sc.(I.T.)5thSEMESTER 2013

Disadvantages of Black Box Testing:

Test cases are challenging to design without having clear functional specifications. It is difficult to identify tricky inputs if the test cases are not developed based on specifications. It is difficult to identify all possible inputs in limited testing time. As a result, writing test cases

may be slow and difficult. There are chances of having unidentified paths during the testing process. There is a high probability of repeating tests already performed by the programmer.

White-box testing - This testing looks into the complex inner working of the application; it

tests the processes undertaken and other internal aspects of the application. While black

box testing is mainly concerned with the inputs and outputs of the application

Disadvantages of white box testing: Expensive Miss cases omitted in the code As knowledge of code and internal structure is a prerequisite, a skilled tester is needed

to carry out this type of testing, which increases the cost. And it is nearly impossible to look into every bit of code to find out hidden errors, which

may create problems, resulting in failure of the application. Not looking at the code in a runtime environment. That's important for a number of

reasons. Exploitation of vulnerability is dependent upon all aspects of the platform being targeted and source code is just of those components. The underlying operating system, the backend database being used, third party security tools, dependent libraries, etc. must all be taken into account when determining exploitability. A source code review is not able to take these factors into account.

Very few white-box tests can be done without modifying the program, changing values to force different execution paths, or to generate a full range of inputs to test a particular function.

Neither is necessarily better than the other. A black box approach tends to be a user focused approach. So this is a good way to ensure the usability and correctness of an application from a user perspective. The drawback to testing from just a black box perspective is many of the code paths may remain unexercised. This is where white box testing comes into play. Using both together is frequently referred to as grey box testing and it allows you to build user focused scenarios as well as verify you are getting good code coverage as well as efficient use of your test cycles.

5.What is cyclomatic complexity? Explain with an illustration.

Discuss its role in software testing and generating test cases.

Cyclomatic complexity (or conditional complexity) is a software metric (measurement). It was developed by Thomas J. McCabe, Sr. in 1976 and is used to indicate the complexity of a program. It directly measures the number of linearly independent paths through a program's source code. The concept, although not the method, is somewhat similar to that of general text complexity measured by the Flesch-Kincaid Readability Test.

Cyclomatic complexity is computed using the control flow graph of the program: the nodes of the graph correspond to indivisible groups of commands of a program, and

SOFTWARE QUALITY AND TESTING Page 4

B.Sc.(I.T.)5thSEMESTER 2013

a directed edge connects two nodes if the second command might be executed immediately after the first command. Cyclomatic complexity may also be applied to individual functions, modules, methods or classes within a program.

The cyclomatic complexity of a section of source code is the count of the number of linearly independent paths through the source code. For instance, if the source code contained no decision points such as IF statements or FOR loops, the complexity would be 1, since there is only a single path through the code. If the code had a single IF statement containing a single condition there would be two paths through the code, one path where the IF statement is evaluated as TRUE and one path where the IF statement is evaluated as FALSE.

Mathematically, the cyclomatic complexity of a structured program is defined with reference to the control flow graph of the program, a directed graph containing the basic blocks of the program, with an edge between two basic blocks if control may pass from the first to the second. The complexity M is then defined as

M = E − N + 2P

where

E = the number of edges of the graph

N = the number of nodes of the graph

P = the number of connected components (exit nodes).

M = E − N + P

This may be seen as calculating the number of linearly independent cycles that exist in the graph, i.e. those cycles that do not contain other cycles within themselves. Note that because each exit point loops back to the entry point, there is at least one such cycle for each exit point.

SOFTWARE QUALITY AND TESTING Page 5

B.Sc.(I.T.)5thSEMESTER 2013

For a single program (or subroutine or method), P is always equal to 1. Cyclomatic complexity may, however, be applied to several such programs or subprograms at the same time (e.g., to all of the methods in a class), and in these cases P will be equal to the number of programs in question, as each subprogram will appear as a disconnected subset of the graph.

It can be shown that the cyclomatic complexity of any structured program with only one entrance point and one exit point is equal to the number of decision points (i.e., 'if' statements or conditional loops) contained in that program plus one.

Cyclomatic complexity may be extended to a program with multiple exit points; in this case it is equal to:

π - s + 2

where π is the number of decision points in the program, and s is the number of exit points.

Implications for Software Testing]

Another application of cyclomatic complexity is in determining the number of test cases that are necessary to achieve thorough test coverage of a particular module.

It is useful because of two properties of the cyclomatic complexity, M, for a specific module:

M is an upper bound for the number of test cases that are necessary to achieve a complete branch coverage.

M is a lower bound for the number of paths through the control flow graph (CFG). Assuming each test case takes one path, the number of cases needed to achieve path coverage is equal to the number of paths that can actually be taken. But some paths may be impossible, so although the number of paths through the CFG is clearly an upper bound on the number of test cases needed for path coverage, this latter number (of possible paths) is sometimes less than M.

All three of the above numbers may be equal: branch coverage   cyclomatic complexity   number of paths.

For example, consider a program that consists of two sequential if-then-else statements.

if( c1() ) f1();else f2(); if( c2() ) f3();else f4();

SOFTWARE QUALITY AND TESTING Page 6

B.Sc.(I.T.)5thSEMESTER 2013

The control flow graph of the source code above; the red circle is the entry point of the function, and the blue circle is the exit point. The exit has been connected to the entry to make the graph strongly connected.

In this example, two test cases are sufficient to achieve a complete branch coverage, while four are necessary for complete path coverage. The cyclomatic complexity of the program is 3 (as the strongly connected graph for the program contains 9 edges, 7 nodes and 1 connected component) (9-7+1).

In general, in order to fully test a module all execution paths through the module should be exercised. This implies a module with a high complexity number requires more testing effort than a module with a lower value since the higher complexity number indicates more pathways through the code. This also implies that a module with higher complexity is more difficult for a programmer to understand since the programmer must understand the different pathways and the results of those pathways.

Unfortunately, it is not always practical to test all possible paths through a program. Considering the example above, each time an additional if-then-else statement is added, the number of possible paths doubles. As the program grew in this fashion, it would quickly reach the point where testing all of the paths was impractical.

One common testing strategy, espoused for example by the NIST Structured Testing methodology, is to use the cyclomatic complexity of a module to determine the number of white-box tests that are required to obtain sufficient coverage of the module. In almost all cases, according to such a methodology, a module should have at least as many tests as its cyclomatic complexity; in most cases, this number of tests is adequate to exercise all the relevant paths of the function.

As an example of a function that requires more than simply branch coverage to test accurately, consider again the above function, but assume that to avoid a bug occurring, any code that calls either f1() or f3() must also call the other. Assuming that the results of c1() and c2() are independent, that means that the function as presented above contains a bug. Branch coverage

SOFTWARE QUALITY AND TESTING Page 7

B.Sc.(I.T.)5thSEMESTER 2013

would allow us to test the method with just two tests, and one possible set of tests would be to test the following cases:

c1() returns true and c2() returns true c1() returns false and c2() returns false

Neither of these cases exposes the bug. If, however, we use cyclomatic complexity to indicate the number of tests we require, the number increases to 3. We must therefore test one of the following paths:

c1() returns true and c2() returns false c1() returns false and c2() returns true

Either of these tests will expose the bug.

6.What is coupling and cohesion? Mention different types of

coupling and cohesion. Explain their role in testing.

COUPLINGAn indication of the strength of interconnections between program units.

Highly coupled have program units dependent on each other. Loosely coupled are made up of units that are independent or almost independent.

Modules are independent if they can function completely without the presence of the other. Obviously, can't have modules completely independent of each other. Must interact so that can produce desired outputs. The more connections between modules, the more dependent they are in the sense that more info about one modules is required to understand the other module.

Three factors: number of interfaces, complexity of interfaces, type of info flow along interfaces.

Want to minimize number of interfaces between modules, minimize the complexity of each interface, and control the type of info flow. An interface of a module is used to pass information to and from other modules.

In general, modules tightly coupled if they use shared variables or if they exchange control info.

Loose coupling if info held within a unit and interface with other units via parameter lists. Tight coupling if shared global data.

If need only one field of a record, don't pass entire record. Keep interface as simple and small as possible.

Two types of info flow: data or control.

Passing or receiving back control info means that the action of the module will depend on this control info, which makes it difficult to understand the module.

SOFTWARE QUALITY AND TESTING Page 8

B.Sc.(I.T.)5thSEMESTER 2013

Interfaces with only data communication result in lowest degree of coupling, followed by interfaces that only transfer control data. Highest if data is hybrid.

Coupling can be "low" (also "loose" and "weak") or "high" (also "tight" and "strong"). Some types of coupling, in order of highest to lowest coupling, are as follows:

Procedural programming

A module here refers to a subroutine of any kind, i.e. a set of one or more statements having a name and preferably its own set of variable names.

Content coupling (high)

Content coupling (also known as Pathological coupling) is when one module modifies

or relies on the internal workings of another module (e.g., accessing local data of

another module).

Therefore changing the way the second module produces data (location, type, timing)

will lead to changing the dependent module. Common coupling

Common coupling (also known as Global coupling) is when two modules share the

same global data (e.g., a global variable).

Changing the shared resource implies changing all the modules using it. External coupling

External coupling occurs when two modules share an externally imposed data format,

communication protocol, or device interface. This is basically related to the

communication to external tools and devices. Control coupling

Control coupling is one module controlling the flow of another, by passing it information

on what to do (e.g., passing a what-to-do flag). Stamp coupling (Data-structured coupling)

Stamp coupling is when modules share a composite data structure and use only a part

of it, possibly a different part (e.g., passing a whole record to a function that only needs

one field of it).

This may lead to changing the way a module reads a record because a field that the

module doesn't need has been modified. Data coupling

Data coupling is when modules share data through, for example, parameters. Each

datum is an elementary piece, and these are the only data shared (e.g., passing an

integer to a function that computes a square root). Message coupling (low)

SOFTWARE QUALITY AND TESTING Page 9

B.Sc.(I.T.)5thSEMESTER 2013

This is the loosest type of coupling. It can be achieved by state decentralization (as in

objects) and component communication is done via parameters or message passing

(see Message passing). No coupling

Modules do not communicate at all with one another.

COHESIONMeasure of how well module fits together.

A component should implement a single logical function or single logical entity. All the parts should contribute to the implementation.

Many levels of cohesion:

1. Coincidental cohesion: the parts of a component are not related but simply bundled into a single component.

harder to understand and not reusable.

2. Logical association: similar functions such as input, error handling, etc. put together. Functions fall in same logical class. May pass a flag to determine which ones executed.

interface difficult to understand. Code for more than one function may be intertwined, leading to severe maintenance problems. Difficult to reuse

3. Temporal cohesion: all of statements activated at a single time, such as start up or shut down, are brought together. Initialization, clean up.

Functions weakly related to one another, but more strongly related to functions in other modules so may need to change lots of modules when do maintenance.

4. Procedural cohesion: a single control sequence, e.g., a loop or sequence of decision statements. Often cuts across functional lines. May contain only part of a complete function or parts of several functions.

Functions still weakly connected, and again unlikely to be reusable in another product.

5. Communicational cohesion: operate on same input data or produce same output data. May be performing more than one function. Generally acceptable if alternate structures with higher cohesion cannot be easily identified.

still problems with reusability.

6. Sequential cohesion: output from one part serves as input for another part. May contain several functions or parts of different functions.

SOFTWARE QUALITY AND TESTING Page 10

B.Sc.(I.T.)5thSEMESTER 2013

7. Informational cohesion: performs a number of functions, each with its own entry point, with independent code for each function, all performed on same data structure. Different than logical cohesion because functions not intertwined.

8. Functional cohesion: each part necessary for execution of a single function. e.g., compute square root or sort the array.

Usually reusable in other contexts. Maintenance easier.

7.Compare and contrast between Verification and Validation with

examples.

             Verification              Validation

1. Verification is a static practice of verifying documents, design, code and program.

1. Validation is a dynamic mechanism of validating and testing the actual product.

2. It does not involve executing the code. 2. It always involves executing the code.

3. It is human based checking of documents and files.

3. It is computer based execution of program.

4. Verification uses methods like inspections, reviews, walkthroughs, and Desk-checking etc.

4. Validation uses methods like black box (functional)  testing, gray box testing, and white box (structural) testing etc.

5. Verification is to check whether the software conforms to specifications.

5. Validation is to check whether software meets the customer expectations and requirements.

6. It can catch errors that validation cannot catch. It is low level exercise.

6. It can catch errors that verification cannot catch. It is High Level Exercise.

7. Target is requirements specification, application and software architecture, high level, complete design, and database design etc.

7. Target is actual product-a unit, a module, a bent of integrated modules, and effective final product.

8. Verification is done by QA team to ensure that the software is as per the specifications in the SRS document.

8. Validation is carried out with the involvement of testing team.

9. It generally comes first-done before validation.

9. It generally follows after verification.

Example of verification and validation are explained below:-

Suppose we have the specifications related to the project than by checking that specifications without executing to see whether the specifications are up to the mark or not is what we have done in verification.

SOFTWARE QUALITY AND TESTING Page 11

B.Sc.(I.T.)5thSEMESTER 2013

Similarly Validation of the software is done to make sure that the software always meets the requirements of the customer by executing the specifications of the project and product. 

Note that the customer and end users are concerned in validation of the software. 

It is also crucial to differentiate between end users, and customers. Considering example, if you are developing a library monitoring system, the librarian is the client and the person who issue the books, collect fines etc. are comes under the category of the end users.

Techniques or Methods of Verification and Validation

Methods of Verification

1. Walkthrough

2. Inspection3. ReviewMethods of Validation1. Testing

2. End Users

Conclusion:

1) Verification and Validation both are necessary and complementary.2) Both of them provides its own sets of Error Filters.3) Each of them has its own way of detect out the errors left in the software.

Lots of people use verification and validation interchangeably but both have different meanings. 

Verification process describes whether the outputs are according to inputs or not, and 

Validation  process describes whether the software is accepted by the user or not.

8. What is stress testing? Where do you need this testing? Explain.

Stress testing (sometimes called torture testing) is a form of deliberately intense or thorough testing used to determine the stability of a given system or entity. It involves testing beyond normal operational capacity, often to a breaking point, in order to observe the results. Reasons can include:

to determine breaking points or safe usage limits to confirm intended specifications are being met to determine modes of failure (how exactly a system fails) to test stable operation of a part or system outside standard usage

Stress testing, in general, should put computer hardware under exaggerated levels of stress in order to ensure stability when used in a normal environment. These can include extremes

SOFTWARE QUALITY AND TESTING Page 12

B.Sc.(I.T.)5thSEMESTER 2013

of workload, type of task, memory use, thermal load (heat), clock speed, or voltages. Memory and CPU are two components that are commonly stress tested in this way.

There is considerable overlap between stress testing software and benchmarking software, since both seek to assess and measure maximum performance. Of the two, stress testing software aims to test stability by trying to force a system to fail; benchmarking aims to measure and assess the maximum performance possible at a given task or function.

When modifying the operating parameters of a CPU, such as temperature, humidity, overclocking, under clocking, overvolting, and undervolting, it may be necessary to verify if the new parameters (usually CPU core voltage and frequency) are suitable for heavy CPU loads. This is done by running a CPU-intensive program for extended periods of time, to test whether the computer hangs or crashes. CPU stress testing is also referred to as torture testing. Software that is suitable for torture testing should typically run instructions that utilize the entire chip rather than only a few of its units. Stress testing a CPU over the course of 24 hours at 100% load is, in most cases, sufficient to determine that the CPU will function correctly in normal usage scenarios such as in a desktop computer, where CPU usage typically fluctuates at low levels (50% and under).

Hardware stress testing and stability are subjective and may vary according to how the system will be used. A stress test for a system running 24/7 or that will perform error sensitive tasks such as distributed computing or "folding" projects may differ from one that needs to be able to run a single game with reasonably reliability.

9. Software testing is difficult than implementation. Yes or no, justify

your answer.

Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not limited to the process of executing a program or application with the intent of finding software bugs (errors or other defects).

Software testing can be stated as the process of validating and verifying that a computer program/application/product:

meets the requirements that guided its design and development, works as expected, can be implemented with the same characteristics, and satisfies the needs of stakeholders.

Software testing, depending on the testing method employed, can be implemented at any time in the software development process. Traditionally most of the test effort occurs after the requirements have been defined and the coding process has been completed, but in the Agile approaches most of the test effort is on-going. As such, the methodology of the test is governed by the chosen software development methodology

SOFTWARE QUALITY AND TESTING Page 13

B.Sc.(I.T.)5thSEMESTER 2013

10. What are test cases? Explain the importance of Domain

knowledge in test case generation. Mention the difficulties in

preparing test cases.

A test case, in software engineering, is a set of conditions or variables under which a tester

will determine whether an application, software system or one of its features is working as it

was originally established for it to do. The mechanism for determining whether a software

program or system has passed or failed such a test is known as a test oracle. In some

settings, an oracle could be a requirement or use case, while in others it could be

a heuristic. It may take many test cases to determine that a software program or system is

considered sufficiently scrutinized to be released. Test cases are often referred to as test

scripts, particularly when written - when they are usually collected into test suite

The difficulties in preparing test cases

a. Test Cases are prone to regular revision and update

b.  Test Cases are prone to distribution among the testers who will execute thesec.  Test Cases are prone to clustering and batchingd. Test Cases have tendency of inter-dependencee. Test Cases are prone to distribution among developers (especially in TC driven

development environment)

Assignment :TB

PART - A 1. What is Software Testing.

Software testing means check the correctness and completeness of a product or a project with respect to client requirement is known as software testing. Software testing is a critical element of software quality assurance and represents the ultimate process to ensure the correctness of the product

2. Define the quality of Software. In the context of software engineering, software quality refers to two related but distinct notions that exist wherever quality is defined in a business context:

Software functional quality reflects how well it complies with or conforms to a given design, based on functional requirements or specifications. That attribute can also be described as

SOFTWARE QUALITY AND TESTING Page 14

B.Sc.(I.T.)5thSEMESTER 2013

the fitness for purpose of a piece of software or how it compares to competitors in the marketplace as a worthwhile product;

Software structural quality refers to how it meets non-functional requirements that support the delivery of the functional requirements, such as robustness or maintainability, the degree to which the software was produced correctly.

3. What are the concepts of quality ? Quality Quality control Quality assurance Cost of qualityThe American heritage dictionary defines quality as “a characteristic or attribute of something”. As anattribute of an item quality refers to measurable characteristic-things, we are able to compare to knownstandards such as length, color, electrical properties, and malleability, and so on. However, software,largely an intellectual entity, is more challenging to characterize than physical object.Nevertheless, measures of a programs characteristics do exist. These properties include1. Cyclomatic complexity2. Cohesion3. Number of function points4. Lines of codeWhen we examine an item based on its measurable characteristics, two kinds of quality may beencountered: Quality of design Quality of conformance

4. What are static testing ? Mention its importance in software testing.

Static testing: Static testing is done to verify the conformance of a software system to itsspecification without executing the code. This testing involves analysis of the source text byindividuals. Any document created as a part of the software development process can be tested usingthis approach. Static testing is usually more cost-effective than other types of testing and allowsdefect-detection to be combined with other quality checks. Static testing techniques include reviewsand walkthroughs

5 .What is static program analysis ?

Static program analysis is the analysis of computer software that is performed without actually executing programs (analysis performed on executing programs is known as dynamic analysis). In most cases the analysis is performed on some version of the source code and in the other cases some form of the object code. The term is usually

SOFTWARE QUALITY AND TESTING Page 15

B.Sc.(I.T.)5thSEMESTER 2013

applied to the analysis performed by an automated tool, with human analysis being called program understanding, program comprehension or code review.

6. What is Dynamic testing of software application ?

Dynamic testing involves executing the source code to check if it works asexpected. Dynamic testing may be conducted using the following approaches:· Functional approaches· Structural approaches

7. What is basis path testing ?

Basis path testing, or structured testing, is a method for designing test cases intended to examine each mathematically possible path of execution at least once. By creating a set of basis paths, creating and executing tests for these paths, 100% statement and 100% branch coverage can be guaranteed.

8. What do you mean by automation of testing program ?

Automation of testing is the state of the art technique where in number of tools will help in testingprogram automatically. Programmers can use any tool to test his/her program and ensure the quality.There are number of tools are available in the market. Some of the tools which helps the programmer are:1. Static analyzer2. Code Auditors3. Assertion processors4. Test file generators5. Test Data Generators6. Test Verifiers7. Output comparators.

Programmer can select any tool depending on the complexity of the program.

9. What is white box testing?

White-box testing - This testing looks into the complex inner working of the application; it tests the processes undertaken and other internal aspects of the application

10. Briefly describe the basic concepts of verification and validation. Validation checks that the product design satisfies or fits the intended use (high-level checking), i.e., the software meets the user requirements. This is done through dynamic testing and other forms of review.

SOFTWARE QUALITY AND TESTING Page 16

B.Sc.(I.T.)5thSEMESTER 2013

Verification and validation are not the same thing, although they are often confused. Boehm succinctly expressed the difference between them:

Validation: Are we building the right product? Verification: Are we building the product right?

According to the Capability Maturity Model (CMMI-SW v1.1),

Validation: The process of evaluating software during or at the end of the development process to determine whether it satisfies specified requirements. [IEEE-STD-610]

Verification: The process of evaluating software to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase. [IEEE-STD-610]

In other words, validation ensures that the product actually meets the user's needs, and that the specifications were correct in the first place, while verification is ensuring that the product has been built according to the requirements and design specifications. Validation ensures that "you built the right thing". Verification ensures that "you built it right". Validation confirms that the product, as provided, will fulfill its intended use.

From testing perspective:

Fault – wrong or missing function in the code. Failure – the manifestation of a fault during execution. Malfunction – according to its specification the system does not meet its specified

functionality.

Within the modeling and simulation community, the definitions of validation, verification and accreditation are similar:

Validation is the process of determining the degree to which a model, simulation, or federation of models and simulations, and their associated data are accurate representations of the real world from the perspective of the intended use(s).

Accreditation is the formal certification that a model or simulation is acceptable to be used for a specific purpose.

Verification is the process of determining that a computer model, simulation, or federation of models and simulations implementations and their associated data accurately represent the developer's conceptual description and specifications.

11. What criteria’s are used to complete the testing procedure?

A test plans outlines the classes of tests to be conducted and a test procedure define specific test cases that will be used in an attempt to uncover errors in conformity with requirements. Both the plan and procedure are designed to ensure that:

All functional requirements are satisfied All performance requirements are achieved Documentation is correct and human-engineered and Other requirements like portability, error recovery, and maintainability are met.

SOFTWARE QUALITY AND TESTING Page 17

B.Sc.(I.T.)5thSEMESTER 2013

12. What is unit testing?

In computer programming, unit testing is a method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures, are tested to determine if they are fit for use Intuitively, one can view a unit as the smallest testable part of an application. In procedural programming a unit could be an entire module but is more commonly an individual function or procedure. In object-oriented programming a unit is often an entire interface, such as a class, but could be an individual method.  Unit tests are created by programmers or occasionally by white box testers during the development process.

Ideally, each test case is independent from the others: substitutes like method stubs, mock objects, fakes and test harnesses can be used to assist testing a module in isolation. Unit tests are typically written and run by software developers to ensure that code meets its design and behaves as intended

13. What is Integration testing?

Integration testing (sometimes called Integration and Testing, abbreviated "I&T") is the phase in software testing in which individual software modules are combined and tested as a group. It occurs after unit testing and before validation testing. Integration testing takes as its input modules that have been unit tested, groups them in larger aggregates, applies tests defined in an integration test plan to those aggregates, and delivers as its output the integrated system ready for system testing.

14. What is system testing? System testing of software or hardware is testing conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. System testing falls within the scope of black box testing, and as such, should require no knowledge of the inner design of the code or logic.

As a rule, system testing takes, as its input, all of the "integrated" software components that have passed integration testing and also the software system itself integrated with any applicable hardware system(s). The purpose of integration testing is to detect any inconsistencies between the software units that are integrated together (called assemblages) or between any of the assemblages and the hardware. System testing is a more limited type of testing; it seeks to detect defects both within the "inter-assemblages" and also within the system as a whole.

15. What is Debugging process? Debugging is a methodical process of finding and reducing the number of bugs, or defects, in a computer program or a piece of electronic hardware, thus making it behave as expected. Debugging tends to be harder when various subsystems are tightly coupled, as changes in one may cause bugs to emerge in another. Many books have been written about debugging as it involves numerous aspects, including interactive debugging, control flow, integration testing, log

SOFTWARE QUALITY AND TESTING Page 18

B.Sc.(I.T.)5thSEMESTER 2013

files, monitoring (application, system), memory dumps, profiling, Statistical Process Control, and special design tactics to improve detection while simplifying changes

PART - BAnswer any five: 1. a) Explain the testing approaches to software testing.

Many approaches have been defined in literature. The importance of any approached depends on thetype of the system in which you are testing. Some of the approaches are given below:Debugging-oriented:This approach identifies the errors during debugging the program. There is no difference betweentesting and debugging.Demonstration-oriented:The purpose of testing is to show that the software works. Here most of the time, the software isdemonstrated in a normal sequence/flow. All the branches may not be tested. This approach is mainly tosatisfy the customer and no value added to the program.Destruction-oriented:The purpose of testing is to show the software doesn’t work.It is a sadistic process, which explains why most people find it difficult. It is difficult to design testcases to test the program.Evaluation-oriented:The purpose of testing is to reduce the perceived risk of not working up to an acceptable value.Prevention-oriented:It can be viewed as testing is a mental discipline that results in low risk software. It is always better toforecast the possible errors and rectify it earlier.In general, program testing is more properly viewed as the destructive process of trying to find theerrors (whose presence is assumed) in a program. A successful test case is one that furthers progress inthis direction by causing the program to fail. However, one wants to use program testing to establish somedegree of confidence that a program does what it is supposed to do and does not do what it is notsupposed to do, but this purpose is best achieved by a diligent exploration for errors.

b) Explain the testing fundamentals of software. Before understanding the process of testing software, it is necessary to learn the basic principles oftesting.

Testing Objectives“Testing is a process of executing a program with the intent of finding an error.A good test is one that has a high probability of finding an as yet undiscovered error.A successful test is one that uncovers an as yet undiscovered error.”The objective is to design tests that systematically uncover different classes of errors and do so witha minimum amount of time and effort.Secondary benefits include:Demonstrate that software functions appear to be working according to specification.

SOFTWARE QUALITY AND TESTING Page 19

B.Sc.(I.T.)5thSEMESTER 2013

Those performance requirements appear to have been met.Data collected during testing provides a good indication of software reliability and some indicationof software qualityTesting cannot show the absence of defects, it can only show that software defects are present.

Test Information FlowSoftware configuration includes a Software Requirements Specification, a Design Specification,and source code.A test configuration includes a test plan and procedures, test cases, and testing tools.It is difficult to predict the time to debug the code, hence it is difficult to schedule.

Test Case DesignSome of the points to be noted during the test case design are:Can be as difficult as the initial design.Can test if a component conforms to specification - Black Box Testing.Can test if a component conforms to design - White box testing.Testing cannot prove correctness as not all execution paths can be tested.

2. a) Explain different types of software testing. Static testing: Static testing is done to verify the conformance of a software system to itsspecification without executing the code. This testing involves analysis of the source text byindividuals. Any document created as a part of the software development process can be tested usingthis approach. Static testing is usually more cost-effective than other types of testing and allowsdefect-detection to be combined with other quality checks. Static testing techniques include reviewsand walkthroughs

Dynamic testing: Dynamic testing involves executing the source code to check if it works asexpected. Dynamic testing may be conducted using the following approaches:Functional approachesStructural approaches

Stress testing: This involves testing the system in a manner that demands resources in abnormalquantity, frequency, or volume. This type of testing is usually done by simulating a productionenvironment and is done for the following reasons:To check that the system works efficiently when subjected to normal transaction volumesTo check that the system works efficiently when subjected to above-normal transaction volumesTo check that the system is capable of meeting expected turnaround timesStress testing attempts to break a system by overloading it with a large volume of transactions.

SOFTWARE QUALITY AND TESTING Page 20

B.Sc.(I.T.)5thSEMESTER 2013

Execution testing: This is conducted to determine if the system achieves the desired level ofperformance. It can also be used to determine response times and turnaround times. Executiontesting can be performed using the actual system or a simulated model of the system. The objectivesof execution testing include the following:To determine response timeTo determine turnaround timeTo check whether hardware and software are being optimally utilizedRecovery testing: This is conducted to verify the ability of the system to recover from varyingdegrees of failure. This testing technique ensures that operations can be continued after a disaster.Recovery testing has the following objectives:To check if adequate backups have been madeTo ensure that the backup data is secureTo ensure that recovery procedures have been documentedTo ensure that recovery tools are availableRecovery testing can be done using any of the two methods:By assessing the procedures, methods, tools, and techniquesBy introducing a simulated disaster Operations testing: When an application is developed, it is tested and then integrated into theoperating environment. The application is then executed by the operation staff using the normaloperating procedures and documentation. Operation testing ensures that the operation staff canproperly execute the application using the operating procedures and documentation.

Compliance testing: This verifies whether the application is developed in accordance withinformation technology standards, procedures, and guidelines. Compliance testing may be done by apeer group of programmers who test a computer program line-by-line to ensure that it complies withprogramming standards. Security testing: This is done to identify security defects in the software. Security defects are not aseasily identifiable as other types of defects. The following are some objectives of security testing:To ensure that security risks are identifiedTo ensure that security measures are implementedTo ensure that implemented security measures function properly

Requirements testing: This is conducted to verify that a system performs correctly over acontinuous period of time. The objectives of requirements testing are given below:To ensure that the specified requirements are implementedTo ensure that the system works correctly for extended time periodsTo check if processing is done using defined proceduresTo check if processing is done in accordance with government rules

SOFTWARE QUALITY AND TESTING Page 21

B.Sc.(I.T.)5thSEMESTER 2013

Regression testing: When a change is made to one segment of a system, it may have a disastrouseffect on another segment that had been thoroughly tested earlier. This cascading effect may becaused because of any of the following reasons:The changes were implemented incorrectlyThe changes introduce new data or parameters that cause problems in a previously testedsegmentRegression testing retests previously tested segments to ensure that they function properly when a changeis made to another part of the application. Error-handling testing: Error-handling testing is done by a group of individuals who thinknegatively to anticipate what can go wrong with the system. This type of testing is used throughoutthe development life cycle. The impact of errors is identified at all stages in the development processand appropriate action is taken to reduce those errors to an acceptable level. Manual-support testing: This involves testing the interface between users and the applicationsystem. The users must be able to enter transactions and obtain the results of processing. In addition,the users must be able to derive useful information out of the results and use it to take necessaryaction. Intersystem testing: Applications are often interconnected with other applications. Intersystemtesting is done to ensure that the interconnections between applications function correctly. This typeof testing involves operating multiple systems in the test and ensuring that proper parameters anddata are correctly passed between the applications

Control testing: This is conducted to ensure that processing is performed in accordance with theintent of the management. Control testing involves testing the mechanisms that oversee the properfunctioning of an application system by methods such as audit trails, documentation, backup andrecovery. Parallel testing: When a new system is developed, it may be run in parallel with the old system sothat performance and outcomes can be compared and corrected before deployment. This type oftesting is called parallel testing

3. a) Explain the background issues and activities of SQA. Background IssuesQA is an essential activity for any business that produces products to be used by others.

SOFTWARE QUALITY AND TESTING Page 22

B.Sc.(I.T.)5thSEMESTER 2013

The SQA group serves as the customer in-house representative. That is the people who perform SQAmust look at the software from customer’s point of views.The SQA group attempts to answer the questions asked below and hence ensure the quality of software.The questions are1. Has software development been conducted according to pre-established standards?2. Have technical disciplines properly performed their role as part of the SQA activity?SQA ActivitiesSQA is comprised of a variety of tasks associated with two different constituencies1. The software engineers who do technical work likePerforming Quality assurance by applying technical methodsConduct Formal Technical ReviewsPerform well-planed software testing.2. SQA group that has responsibility forQuality assurance planning oversightRecord keepingAnalysis and reporting.QA activities performed by SE team and SQA are governed by the following plan.Evaluation to be performed.Audits and reviews to be performed.Standards that is applicable to the project.Procedures for error reporting and trackingDocuments to be produced by the SQA groupAmount of feedback provided to software project team.What are the activities performed by SQA and SE team?Prepare SQA Plan for a projectParticipate in the development of the project’s software descriptionReview software-engineering activities to verify compliance with defined software process.Audits designated software work products to verify compliance with those defined as part ofthe software process.Ensures that deviations in software work and work products are documented and handledaccording to a documented procedure.Records any noncompliance and reports to senior management.

b) Explain the defect amplification model of software design.

A defect amplification model can be used to illustrate the generation and detection of errors duringpreliminary design, detail design, and coding steps of the software engineering process. The model isillustrated schematically in Figure A box represents a software development step. During the step, errors may be inadvertently generated.Review may fail to uncover newly generated errors from previous steps, resulting in some number oferrors that are passed through. In some cases, errors passed through from previous steps, resulting insome number of errors that are passed through. In some cases errors passed through from previous steps are amplified (amplification factor, x) by current work. The box subdivisions represent each of these characteristics and the percent efficiency for detecting errors, a function of the thoroughness of

SOFTWARE QUALITY AND TESTING Page 23

B.Sc.(I.T.)5thSEMESTER 2013

review each test step is assumed to uncover and correct fifty percent of all incoming errors without introducing new errors (an optimistic assumption). Ten preliminary design errors are amplified to 94 errors before testing commences. Twelve latent defects are released to the field considers the same conditions except that design and code reviews are conducted as part of each development step. In this case, ten initial preliminary design errors are amplified to 24 errors before testing commences.Only three latent defects exist. By recalling the relative cost associated with the discovery andcorrection of errors, overall costs (with and without review for our hypothetical example) can be established.To conduct reviews a developer must expend time and effort and the development organization mustspend money. However, the results of the preceding or previous, example leave little doubt that we have encountered a “Pay now or pay much more lately” syndromeFormal technical reviews (for design and other technical activities) provide a demonstrable cost benefit and they should be conducted.

4. a) Explain an error checklist for inspection in detail. An important part of the inspection process is the use of a checklist to examine the program forcommon errors. The checklist is largely language independent as most of the errors can occur with anyprogramming language.Data-Reference Errors1. Is a variable referenced whose value is unset or uninitialized? This is probably the most frequentprogramming error; it occurs in a wide variety of circumstances.2. For all array references, is each subscript value within the defined bounds of the correspondingdimension?3. For all array references, does each subscript have an integer value? This is not necessarily anerror in all languages, but it is a dangerous practice.4. For all references through pointer or reference variables, is the referenced storage currentlyallocated? This is known as the “dangling reference” problem. It occurs in situations where thelifetime of a pointer is greater than the lifetime of the referenced storage.5. Are there any explicit or implicit addressing problems if, on the machine being used, the units ofstorage allocation are smaller than the units of storage addressability?6. If a data structure is referenced in multiple procedures or subroutines, is the structure definedidentically in each procedure?7. When indexing into a string, are the limits of the string exceeded?

Data-Declaration Error1. Have all variables been explicitly declared? A failure to do so is not necessarily an error, but itis a common source of trouble.2. If all attributes of a variable are not explicitly stated in the declaration, are the defaults wellunderstood?3. Where a variable is initialized in a declarative statement, is it properly initialized?4. Is each variable assigned the correct length, type, and storage class?5. Is the initialization of a variable consistent with its storage type?Computation Errors1. Are there any computations using variables having inconsistent (e.g. No arithmetic) data types?2. Are there any mixed mode computations?

SOFTWARE QUALITY AND TESTING Page 24

B.Sc.(I.T.)5thSEMESTER 2013

3. Are there any computations using variables having the same data type but different lengths?4. Is the target variable of an assignment smaller than the right-hand expression?5. Is an overflow or underflow exception possible during the computation of an expression? Thatis, the end result may appear to have a valid value, but an intermediate result might be too big ortoo small for the machine’s data representations.6. Is it possible for the divisor in a division operation to be zero?7. Where applicable, can the value of a variable go outside its meaningful range?8. Are there any invalid uses of integer arithmetic, particularly division? For example, if I is aninteger variable, whether the expression 2*I/2 is equal to I depends on whether I has an odd oran even value and whether the multiplication or division is performed first.Comparison Errors1. Are there any comparisons between variables having inconsistent data types (e.g. comparing acharacter string to an address)?2. Are there any mixed-mode comparisons or comparisons between variables of different lengths?If so, ensure that the conversion rules are well understood.3. Does each Boolean expression state what it is supposed to state? Programmers often makemistakes when writing logical expressions involving “and”, “or”, and “not”.4. Are the operands of a Boolean operator Boolean? Have comparison and Boolean operatorsbeen erroneously mixed together?Control-Flow Errors1. If the program contains a multi way branch (e.g. a computed GO TO in Fortran), can the indexvariable ever exceed the number of branch possibilities? For example, in the Fortran statement,GOTO(200,300,400), IWill I always have the value 1,2, or 3?2. Will every loop eventually terminate? Devise an informal proof or argument showing that eachloop will terminate3. Will the program, module, or subroutine eventually terminate?4. Is it possible that, because of the conditions upon entry, a loop will never execute? If so, doesthis represent an oversight? For instance, for loops headed by the following statements:DO WHILE (NOTFOUND)DO I=X TO ZWhat happens if NOTFOUND is initially false or if X is greater than Z?5. Are there any non-exhaustive decisions? For instance, if an input parameter’s expected valuesare 1, 2, or 3; does the logic assume that it must be 3 if it is not 1 or 2? If so, is the assumptionvalid?

Interface Errors1. Does the number of parameters received by this module equal the number of arguments sent byeach of the calling modules? Also, is the order correct?2. Do the attributes (e.g. type and size) of each parameter match the attributes of each correspondingargument?3. Does the number of arguments transmitted by this module to another module equal the numberof parameters expected by that module?4. Do the attributes of each argument transmitted to another module match the attributes of thecorresponding parameter in that module?5. If built-in functions are invoked, are the number, attributes, and order of the arguments correct?

SOFTWARE QUALITY AND TESTING Page 25

B.Sc.(I.T.)5thSEMESTER 2013

6. Does the subroutine alter a parameter that is intended to be only an input value?Input/output Errors1. If files are explicitly declared, are their attributes correct?2. Are the attributes on the OPEN statement correct?3. Is the size of the I/O area in storage equal to the record size?4. Have all files been opened before use?5. Are end-of-file conditions detected and handled correctly?6. Are there spelling or grammatical errors in any text that is printed or displayed by the program?

b) Explain walkthroughs briefly. The code walkthrough, like the inspection, is a set of procedures and error-detection techniques forgroup code reading. It shares much in common with the inspection process, but the procedures are slightlydifferent, and a different error-detection technique is employed.The walkthrough is an uninterrupted meeting of one to two hours in duration. The walkthrough teamconsists of three to five people to play the role of moderator, secretary (a person who records all errorsfound), tester and programmer. It is suggested to have other participants like:

A highly experienced programmer, A programming-language expert, A new programmer (to give a fresh, unbiased outlook) The person who will eventually maintain the program, Someone from different project and Someone from the same programming team as the programmer.

The initial procedure is identical to that of the inspection process: the participants are given the materials several days in advance to allow them to study the program. However, the procedure in the meeting is different. Rather than simply reading the program or using error checklists, the participants “play computer”.The person designated as the tester comes to the meeting armed with a small set of paper test cases representative sets of inputs (and expected outputs) for the program or module. During the meeting, each test case is mentally executed. That is, the test data are walked through the logic of the program. The state of the program (i.e. the values of the variables) is monitored on paper or a blackboard.The test case must be simple and few in number, because people execute programs at a rate that is very slow compared to machines. In most walkthroughs, more errors are found during the process of questioning the programmer than are found directly by the test cases themselves.

5. a) Explain the notations used for control structures.

In programs, conditions are very important and testing such conditions is more complex than other statements like assignment and declarative statements. Basic path testing is one example of control structure testing.

There are many ways in which control structure can be tested.

Conditions TestingCondition testing aims to exercise all logical conditions in a program module. Logical conditions may be

SOFTWARE QUALITY AND TESTING Page 26

B.Sc.(I.T.)5thSEMESTER 2013

complex or simple. Logical conditions may be nested with many relational operations.Can define:Relational expression: (E1 op E2), where E1 and E2 are arithmetic expressions. For example,(x+y) – (s/t), where x, y, s and t are variables.Simple condition: Boolean variable or relational expression, possibly proceeded by a NOT operator.Compound condition: composed of two or more simple conditions, Boolean operators andparentheses along with relational operators.Boolean expression: Condition without relational expressions.Normally errors in expressions can be due to one or all or the following:Boolean operator errorBoolean variable errorBoolean parenthesis errorRelational operator errorArithmetic expression errorMismatch of typesCondition testing methods focus on testing each condition in the program of any type of conditions.There are many strategies to identify errors.Some of the strategies proposed include:Branch testing: Every branch is executed at least once.Domain Testing: Uses three or four tests for every relational operator depending on the complexityof the statement.Branch and relational operator testing: Uses condition constraints. Based on the complexity ofthe relational operators, many branches will be executed

Example 1: C1 = B1 & B2where B1, B2 are boolean conditionsCondition constraint of form (D1, D2) where D1 and D2 can be true (t) or false (f).The branch and relational operator test requires the constraint set {(t,t),(f,t),(t,f)} to be coveredby the execution of C1.Coverage of the constraint set guarantees detection of relational operator errors

Data Flow TestingFirst, a proper data flow diagram like control flow(see basis path flow) is drawn. Then selects testpaths according to the location of definitions and use of variables. Any variables that have been defined in any program behave in the following way:D: define the variable, normally defined in declarative section,U: use the variables which are defined earlier, in the program.K: kill the variable, which is another state of the variable at any time of the execution of the program.Any variable that is part of the program will undergo any of the above states. However, the sequenceof states is important. We can avoid following anomalies during the program execution:DU: Normal,UK, UU: Normal,DD: SuspiciousDK: Probable bugKD: NormalKK: Probable bugKU: bugUD: NormalFor example,

SOFTWARE QUALITY AND TESTING Page 27

B.Sc.(I.T.)5thSEMESTER 2013

DU: Normal means a variable is defined first and then used in the program which is normal behaviorof the data flow in the program.DK: Probable bug means a variable is defined and then killed before using in the program. This may bebug as why the variable is defined and killed without using in the program.

Loop TestingLoops are fundamental to many algorithms. Loops can be categorized as, define loops as simple,concatenated, nested, and unstructured. Loops can be defined in many ways.Examples:To test the loops, following guidelines may be followed:Simple Loops of size n:Skip loop entirelyOnly one pass through the loopTwo passes through the loopm passes through loop where m<n.(n-1), n, and (n+1) passes through the loop. This helps in testing the boundary of the loops.Nested LoopsStart with inner loop. Set all other loops to minimum values.Conduct simple loop testing on inner loop.Work outwards and take the next nested loop.Continue until all loops are tested.Concatenated LoopsIf independent loops, use simple loop testing.If dependent, treat as nested loops.Unstructured loopsDon’t test - redesign. This is known as poor design

b) Explain the concept of cyclomatic complexity in detail.

Cyclomatic complexity (or conditional complexity) is a software metric (measurement). It was developed by Thomas J. McCabe, Sr. in 1976 and is used to indicate the complexity of a program. It directly measures the number of linearly independent paths through a program's source code. The concept, although not the method, is somewhat similar to that of general text complexity measured by the Flesch-Kincaid Readability Test.

Cyclomatic complexity is computed using the control flow graph of the program: the nodes of the graph correspond to indivisible groups of commands of a program, and a directed edge connects two nodes if the second command might be executed immediately after the first command. Cyclomatic complexity may also be applied to individual functions, modules, methods or classes within a program.

The cyclomatic complexity of a section of source code is the count of the number of linearly independent paths through the source code. For instance, if the source code contained no decision points such as IF statements or FOR loops, the complexity would be 1, since there is only a single path through the code. If the code had a single IF statement containing a single

SOFTWARE QUALITY AND TESTING Page 28

B.Sc.(I.T.)5thSEMESTER 2013

condition there would be two paths through the code, one path where the IF statement is evaluated as TRUE and one path where the IF statement is evaluated as FALSE.

Mathematically, the cyclomatic complexity of a structured program is defined with reference to the control flow graph of the program, a directed graph containing the basic blocks of the program, with an edge between two basic blocks if control may pass from the first to the second. The complexity M is then defined as

M = E − N + 2P

where

E = the number of edges of the graph

N = the number of nodes of the graph

P = the number of connected components (exit nodes).

M = E − N + P

This may be seen as calculating the number of linearly independent cycles that exist in the graph, i.e. those cycles that do not contain other cycles within themselves. Note that because each exit point loops back to the entry point, there is at least one such cycle for each exit point.

For a single program (or subroutine or method), P is always equal to 1. Cyclomatic complexity may, however, be applied to several such programs or subprograms at the same time (e.g., to all of the methods in a class), and in these cases P will be equal to the number of programs in question, as each subprogram will appear as a disconnected subset of the graph.

It can be shown that the cyclomatic complexity of any structured program with only one entrance point and one exit point is equal to the number of

SOFTWARE QUALITY AND TESTING Page 29

B.Sc.(I.T.)5thSEMESTER 2013

decision points (i.e., 'if' statements or conditional loops) contained in that program plus one.

Cyclomatic complexity may be extended to a program with multiple exit points; in this case it is equal to:

π - s + 2

where π is the number of decision points in the program, and s is the number of exit points.

Implications for Software Testing]

Another application of cyclomatic complexity is in determining the number of test cases that are necessary to achieve thorough test coverage of a particular module.

It is useful because of two properties of the cyclomatic complexity, M, for a specific module:

M is an upper bound for the number of test cases that are necessary to achieve a complete branch coverage.

M is a lower bound for the number of paths through the control flow graph (CFG). Assuming each test case takes one path, the number of cases needed to achieve path coverage is equal to the number of paths that can actually be taken. But some paths may be impossible, so although the number of paths through the CFG is clearly an upper bound on the number of test cases needed for path coverage, this latter number (of possible paths) is sometimes less than M.

All three of the above numbers may be equal: branch coverage   cyclomatic complexity   number of paths.

For example, consider a program that consists of two sequential if-then-else statements.

if( c1() ) f1();else f2(); if( c2() ) f3();else f4();

SOFTWARE QUALITY AND TESTING Page 30

B.Sc.(I.T.)5thSEMESTER 2013

The control flow graph of the source code above; the red circle is the entry point of the function, and the blue circle is the exit point. The exit has been connected to the entry to make the graph strongly connected.

In this example, two test cases are sufficient to achieve a complete branch coverage, while four are necessary for complete path coverage. The cyclomatic complexity of the program is 3 (as the strongly connected graph for the program contains 9 edges, 7 nodes and 1 connected component) (9-7+1).

In general, in order to fully test a module all execution paths through the module should be exercised. This implies a module with a high complexity number requires more testing effort than a module with a lower value since the higher complexity number indicates more pathways through the code. This also implies that a module with higher complexity is more difficult for a programmer to understand since the programmer must understand the different pathways and the results of those pathways.

Unfortunately, it is not always practical to test all possible paths through a program. Considering the example above, each time an additional if-then-else statement is added, the number of possible paths doubles. As the program grew in this fashion, it would quickly reach the point where testing all of the paths was impractical.

One common testing strategy, espoused for example by the NIST Structured Testing methodology, is to use the cyclomatic complexity of a module to determine the number of white-box tests that are required to obtain sufficient coverage of the module. In almost all cases, according to such a methodology, a module should have at least as many tests as its cyclomatic complexity; in most cases, this number of tests is adequate to exercise all the relevant paths of the function.

As an example of a function that requires more than simply branch coverage to test accurately, consider again the above function, but assume that to avoid a bug occurring, any code that calls either f1() or f3() must also call the other. Assuming that the results of c1() and c2() are independent, that means that the function as presented above contains a bug. Branch coverage

SOFTWARE QUALITY AND TESTING Page 31

B.Sc.(I.T.)5thSEMESTER 2013

would allow us to test the method with just two tests, and one possible set of tests would be to test the following cases:

c1() returns true and c2() returns true c1() returns false and c2() returns false

Neither of these cases exposes the bug. If, however, we use cyclomatic complexity to indicate the number of tests we require, the number increases to 3. We must therefore test one of the following paths:

c1() returns true and c2() returns false c1() returns false and c2() returns true

Either of these tests will expose the bug.

SOFTWARE QUALITY AND TESTING Page 32