Upload
softwarecentral
View
1.416
Download
1
Embed Size (px)
Citation preview
Regression Testing of COTS
Component-Based Software System
A Thesis Submitted in partial fulfillment of the requirement for the Award
of the Degree of
Master of Technology IN
COMPUTER SCIENCE & ENGINEERING ((22000066--22000088))
Under the Supervision of Ms. Gagandeep
SSuubbmmiitttteedd BByy YYooggeesshh KKuummaarr
DDeeppaarrttmmeenntt ooff CCoommppuutteerr SScciieennccee PPuunnjjaabbii UUnniivveerrssiittyy,, PPaattiiaallaa..
1
Acknowledgment
While this report may bear only one name on the cover, it is the result of inputs from
others who have helped me during the thesis work. In an effort to recognize the
contributions of those who helped bring thesis to life, I would like to acknowledge and
thank them.
First and foremost, I sincerely thank my project supervisor, Ms. Gagandeep, for her
continuous guidance with the thesis work. Without her encouragement, this thesis would
not have been possible. I would also like to thank Dr. Gurpreet Singh Lehel, Head,
Department of Computer Science, for his help throughout.
I am also thankful to the Department of Computer Science, because I have utilized the
resources available here for research purposes, and this has enormously helped me
throughout the duration of this thesis.
And last but not least, I wish to express my eternal gratitude towards my family for their
everlasting support. I am greatly indebted to them.
Yogesh Kumar
2
Abstract
The primary role of component-based software engineering is to address the development
of systems as an assembly of parts (components), the development of parts as reusable
entities, and the maintenance and upgrading of systems by customizing and replacing
such parts. This requires established methodologies and tool support covering the entire
component and system lifecycle including technological, organizational, marketing, legal,
and other aspects. The traditional disciplines from software engineering need new
methodologies to support component -based development.
Component-based Software Engineering (CBSE) has become recognized as such a new
sub-discipline of Software Engineering. Component based software engineering (CBSE)
also offers convenience as well as fast design/ implementation. Testing UML design
models is challenging because a design model consists of many models that describe
different design views. An approach to generating inputs for testing UML designs
requires a mechanism that combines information from both the structural and behavioral
views of a design.
Regression testing is a testing process which is applied after a program is modified. It
involves testing the modified program with the old test cases in order to re-establish
confidence in the program. As a part of our research work, we have proposed a
methodology to generate regression test suite based on UML sequence diagrams.
We have constructed and implemented control flow graph (CFG) in C++ for the storage
of UML sequence diagrams. The sequence diagrams have been drawn with the help of
NET BEANS tool. We have practically experimented on different component from
different application domain. We have stored the sequence diagrams for each component
into the control flow graph (CFG) and traversed it for the generation of the regression test
suite.
The results have been evaluated by calculating the total running time for each component.
Our technique of automating the regression test suite generation in effective in terms of
reduction in number of test cases and time efficient as compared to manual technique.
3
.
TABLE OF CONTENTS
1. Introduction 1 1.1 Problem …………………………………………………………1 1.2 Overview of the solution and research challenges………………2 Scope of the research…………………..………………………...3 1.3 Thesis organization………………………………………………3-4 2. Regression Testing …………………………………………………5 2.1 Need for regression testing……………………………………….5 2.2 Types of regression testing……………………………………….6 2.3 Difference between testing and regression testing………………..7 2.4 Similarities between testing and regression testing……………....8 2.5 Problem of regression testing…………………………………….9 2.6 Retest approach…………………………………………………..9-13 3. Literature Survey………………………………………………….14 3.1 Techniques of regression testing………………………………..14-16 4. Case studies of the existing techniques …………………………. 18 4.1 Generating regression tests via model checking………………...18-26 4.1 Regression testing UML design…………………………………27-36 5. Related work in component-based Regression testing and UML.37-39 6. Control flow graph based criteria to generate regression test suite…………………………………………………………….40-46 7. Case Study …………………………………………………………..47 7.1 Case study with ATM using Net Beans Tools for UML sequence diagram…………………………………………………………....47-51
4
7.2 Case study with E-Ticket of Cinema for UML sequence diagram..52-56 7.3 Case study with Library Record System for UML sequence diagram……………………………………………………………57-61 8. Conclusion and Future work ………………………………………62 8.1 Evaluation of the case studies……………………………………62-64 8.2 Summary of the contribution………………………………………65 8.3 Future work…………………………………………………….......65 Reference……………………………………………………………….66-69
5
LIST OF TABLE
Table 4.1.1 Test suite of the window 98 notepad………………………19 Table 4.1.2 Test suite of window XP notepad………………………….20 Table 4.1.3 Test suite of comparator…………………………………...23 Table 4.1.4 Test suite of coverage selector ……………………………24-25 Table 4.1.5 Test suite of test generator………………………………...25-26 Table 4.2.1 Test suit T of the Program P (simple program of factorial)..30 Table 4.2.2 Test suit T’ of the Program P’ (modified program of factorial)..………………………………………………….32-33
Table 4.2.3 UML Test suits of combined of T and T’…………………33-34 Table 4.2.4 Final Test cases …………………………………………35-36 Table 7.1.1 Case scenarios based on the ATM sequence diagram of first version……………………………………………………...48
Table 7.1.2 Case scenarios based on the ATM sequence diagram of modified version …………………………………………...50
Table 7.1.3 Final Regression Test case generation from the ATM Component ………………………………………………….51
Table 7.2.1 Case scenarios of first version of the E-Ticket system of Cinema.. …………………………………………………..52-53
6
Table 7.2.2 Case scenarios of modified version of the E-Ticket system of Cinema ……………………………………………………54-55
Table 7.2.3 Final Regression Test case generation from the E-Ticket System of Cinema Component……………………………………..55-56
Table 7.3.1 Case scenarios of the first version of the Library Record System component……………………………………………58
Table 7.3.2 Case scenarios of the modified version of the Library Record System component …………………….………….59-60 Table 7.3.3 Final regression test suite for Library record system……...60-61
7
LIST OF FIGURES
Figure 2.1 Regression testing ……………………………………………6 Figure 2.2 The retest regression testing process for corrective regression testing………............................................................................11 Figure 2.3 An operational overview of Retest…………………................12 Figure 4.1.1 Functionalities of comparator…………….….........................22 Figure 4.1.2 Coverage Selector……………………………........................24 Figure 4.2.1 Object Method Directed Acyclic Graph (OMDAG) for simple program of factorial ...............................................29 Figure 4.2.2 Object Method Directed Acyclic Graph (OMDAG) for modified version of factorial………………………….....31 Figure 6.1 Working of the control flow graph (CFG) based criteria for generation of regression test suite…………………………......40 Figure 6.2 Sequence diagram of student enrollment system……………...41 Figure 7.1.1 First version of the ATM Sequence Diagram………………..47 Figure 7.1.2 Extending the functionality of first version of ATM sequence
Diagram…………………………………………………….49
Figure 7.2.1 Sequence diagram of first version of E-Ticket of Cinema component…………………………………………52 Figure 7.2.2 shows the sequence diagram of modified version of E-Ticket of Cinema……………………………………………………….54
8
Figure 7.3.1 Sequence diagram of the first version for the Library Record System………………………………………………………..57 Figure 7.3.2 Sequence diagram of the modified version for the Library Record system………………………………………………..59
Figure 8.1.1 Bar chart of the total number of regression test suite for ATM
Component…………………………………………………….62
Figure 8.1.2 Bar chart of the total number of regression test suite for E-
Ticket System for Cinema Component………………………63
Figure 8.1.3 Bar chart of the total number of regression test suite for Library
Record System Component…………………………………..64
9
Chapter 1
Introduction
1.1 Problem In recent years, Component-based software has been widely used in various application
domains and becomes a fairly popular software form. However, Due to the lack of
information about the externally developed components, system testers (i.e., component
users) generally can't perform effective testing (especially regression testing) on their
component based systems. Component users don't know the details about change in
component, so they aren't able to select the proper test cases to retest the modified system
[3].
Meanwhile, Component-Based Software Engineering (CBSE) is a continuously growing
field. However, there is a lack for good environment support for the CBSE activities,
mainly because most of the environments do not offer comprehensive, integrated support
for the full range of CBSE activities. The construction of this kind of environments
involves general environment issues, such as tool integration, and specific CBSE issues,
such as domain engineering and application composition. The goal of component-based
software engineering is to increase the productivity, quality [3].
In component-based software systems, regression test case selection is a challenge since
vendor of commercial-off-the-shelf (COTS) component does not release source code.
Regression testing is applied to modified software to provide confidence that modified
code behaves as intended, and does not adversely affect the behavior of unmodified code.
Regression testing plays an integral role in software maintenance; without proper
regression testing we are reluctant to release modified software [7].
One characteristic distinguishing regression testing from developmental testing is the
availability, at regression test time, of existing test suites. If we reuse such test suites to
retest a modified program, we can reduce the effort required to perform that testing.
Unfortunately, test suites can be large, and we may not have time to rerun all tests in such
suites. Thus, we must often restrict our efforts to a subset of the previously existing tests.
We call the problem of choosing an appropriate subset of an existing test suite the
10
selective retest problem; we call a method for solving this problem a selective retest
method [18].
We use the strategy for regression test selection in component based software
engineering. This is achieved by the component developer providing information about
the changes in the first one version to next and the component user utilizing this
information in regression test case selection. We use Unified Modeling Language (UML)
and its sequence diagram to formally specify the test case selection process. UML is the
accepted standard for component design, and by specifying the regression test case
selection process [19].
1.2 Overview of the Solution and the research challenges Improvements in software testing techniques are needed to address the increasingly
complex applications that are implemented by today’s software systems. The UML and
its diagrams are widely used to visually depict the static structure and more importantly
for us, the dynamic behavior of such applications. This trend provides us with an
excellent opportunity to meld our proven test generation technology (TDE) with the
UML and give developers and testers the capability of automatically generating black-
box conformance tests early on.
For regression testing, we can derive tests from UML Sequence diagrams, which embody
the behavioral description of each component of interest and more importantly, the
interactions or events between them. The Unified Modeling Language (UML) [14]
reduces the efforts in constructing the models in software development and increases
maintainability and reusability during the life cycle. Another well accepted method is
provided through design patterns. Likewise, frameworks provide users with an efficient
means to combine component reuse with design reuse.
Component based software engineering (CBSE) also offers convenience as well as fast
design/ implementation. Testing UML design models is challenging because a design
model consists of many models that describe different design views. An approach to
generating inputs for testing UML designs requires a mechanism that combines
information from both the structural and behavioral views of a design. We conducted the
case studies to investigate the effectiveness of our testing approach[3].
11
The studies include the effective way of generating the UML regression test case
selection based on UML sequence diagram. In my thesis, I have generated the storage
graph for sequence diagram using C++ program of data structure. The studies
demonstrate the cost of testing in terms of number of test inputs and effectiveness of test
case generation in terms of execution time.
Scope of the Research
The work described in my thesis focuses on effective way of generating the UML
regression test suite with the help of control flow graph (CFG) for the storage of the
sequence diagram. And the traversal of the control flow graph will automatically produce
the final regression test suite. The practical approach for generating the regression test
suite will provide more accuracy as compared to the manually generating the test suite.
The control flow graph for regression test selection techniques attempt to reduce the
time required to retest a modified program by selecting some subset of the existing test
suite. The purpose of RTS techniques is to reduce the high cost of retest-all regression
testing by selecting a subset of possible test cases.
In our approach, a control flow graph (CFG) will store the each events of the sequence
diagram.
They store the complete methods with parameters and information about the sender and
the receiver and with their pre-conditions and post-conditions. The specifications consist
of pre- and post-conditions expressed in the OCL.
1.3 Project Organization The rest of the thesis is organized as follows:
• Chapter 2 describe the Regression testing and its types.
• Chapter 3 presents Literature Survey of regression testing and defines the various
techniques of the regression testing.
• Chapter 4 describes the case studies of the existing Techniques.
• Chapter 5 discusses related work in component based regression testing and
UML.
• Chapter 6 Control flow graph (CFG) based criteria to generate regression test
suite.
12
• Chapter 7 describes our technique for generating the final regression test suite
from UML sequence diagram from different software component.
• Chapter 8 Conclusion and Future work.
13
Chapter 2
Regression Testing
Regression testing is a testing process, which is applied after a program, is modified. It
involves testing the modified program with some test cases in order to re-establish our
confidence that the program will perform according to the (possibly modified)
specification. In the development phase, regression testing may begin after the detection
and correction of errors in a tested program. A tested program is a program which has
been tested with a high quality test plan.
Regression testing is a major component in the maintenance phase where the software
system may be corrected, adapted to new environment, or enhanced to improve its
performance. Modifying a program involves creating new logic to correct an error or to
implement a change and incorporating that logic into an existing program. The new logic
may involve minor modifications such as adding, deleting, rewriting a few lines of code,
or major modifications such as adding, deleting [9].
2.1 The Need for Regression Testing Software inevitably changes however well conceived and well written it initially
may be. Operational failures expose faults to be repaired. Mistaken and changed
requirements cause the software to be reworked. New uses of old software yield new
functionality not originally conceived in the requirements. The management of this
change is critical to the continuing usefulness of the software. The new functionality
added to a system may be accommodated by the standard software development pro-
cases.
Regression testing [9] attempts to revalidate the old functionality inherited from the
old version. The new version should behave exactly as the old except where new
behavior is intended. Therefore, regression tests for a system may be viewed as partial
operational requirements for new versions of the system. Figure1 shows a typical
example of a sequence of time intervals during the life of a software system.
14
Figure 2.1 Regression Testing
2.2 Types of Regression Testing Two types of regression testing can be identified based on the possible modification of
the specification. Progressive regression testing involves a modified specification.
Whenever new enhancements or new data requirements are incorporated in a system, the
specification will be modified to reflect these additions. In most cases, new modules will
be added to the software system with the consequence that the regression testing process
involves testing a modified program against a modified specification [7].
In corrective regression testing, the specification does not change. Only some
instructions of the program and possibly some design decisions are modified. This has
important implications because most test cases in the previous test plan are likely to be
valid in the sense that they correctly specify the input-output relation. However, because
of possible modifications to the control and data flow structures of the software, some
existing test cases are no longer testing the previously targeted program constructs. The
corrective regression testing is often done after some corrective action is performed on
the software.
15
2.3 Differences between Testing and Regression Testing Most people assume that regression testing is a simple extension of testing. However, it
is not always the case. There are several major differences between these two processes.
Availability of test plan Testing begins with a specification, an implementation of the
specification and a test plan with test cases added during the specification, design and
coding phases. All these test cases are new in the sense that they have not been used to
exercise the program previously. Regression testing starts with a possibly modified
specification, a modified program and an old test plan which requires updating. All test
cases in the test plan were previously run and were useful in testing the program.
Scope of test The testing process aims to check the correctness of a program, including its
individual components (e.g., functions and procedures) and the interworking of these
components. Regression testing is concerned checking the correctness of parts of a
program. The portion of a program which is not affected by modifications need not be
retested. An interesting problem is to determine the affected portion of a program in an
efficient manner. A solution to this problem may be the use of retestable units.
Time allocation Testing time is normally budgeted before the development of a product.
This time can be as high as half the total product completion time. However, regression
testing time, especially time for corrective regression testing, is not normally included in
the total product cost and schedule. Consequently, when regression testing is done, it is
nearly always performed in a crisis situation. The tester is normally urged to complete
retesting as soon as possible and most often is given limited time to retest.
Development information In testing, knowledge about the software development
process is readily available. In fact, the testing group and the development group may be
the same. Even if an organization has a separate testing group, the testers can usually
query the developers about any uncertainty in the software. But in regression testing, the
testers most likely will not be the developers of the product. Since regression testing may
be done at a different time and place, the original developers may no longer be available.
This situation suggests that any relevant development information should be retained if
regression testing is to be successful.
16
Completion time The completion time for regression testing should normally be less than
that for testing since only parts of a program are being tested.
2.4 Similarities between Testing and Regression Testing Several aspects of regression testing are similar to that of testing. In particular, the
purposes and testing techniques used are almost the same.
Purposes: The purposes of testing and regression testing are quite similar. They both aim
to:
1. Increase one’s confidence in the correctness of a program.
2. Locate errors in a program.
Some additional goals of regression testing are to: preserve the quality of the
software; the modified software should be at least as reliable as its previous version; this
may be achieved in many ways; one possible method is to insist that the same structural
coverage is achieved by both versions of the software; ensure the continued operation of
the software; this is an important goal because some users may become dependent on the
software product, and software developers have a responsibility to continue to provide
the same service to users.
Testing techniques Since test cases in a test plan depend on the chosen testing
technique, the testing technique used by both testing and regression testing should be the
same if the regression testing process involves the reuse of test cases. If regression testing
were to involve a different testing technique, then it would be difficult to reuse the
existing test plan. Another reason for using the same testing technique is that it is easier
to evaluate the quality of two software products if they are tested by the same technique.
At the current state of the art, it is difficult to compare the relative test effectiveness of
two different testing techniques [7].
17
2.5 The Problem of Regression Testing The problem of regression testing [8] may be broken down into two subproblems: the
test selection problem and the test plan update problem. The test selection problem is
concerned with the design and selection of test cases to fully test a modified program.
Some test cases may be selected from those in the existing test plan, while others have to
be created based on modifications made to the program. The test plan update problem
deals with the management of a test plan as a program is undergoing successive
modifications. Certain test cases will become obsolete and new test cases must be added
to test the modified and new features of the software. The test plan update problem may
be defined as follows:
Test Plan Update Problem:
Given a program P, and its specification S, a test plan T for testing P, a program P’ which
is a modified version of P, and its specification S’, generate a test plan T’ for P’ from T,
P, S, F” and S’.
From this definition, we can break down the test plan update problem into two
subproblems:
1. Test case classification problem given a program P, and its specification S, a test
plan T for testing P, a program P’ which is a modified version of P, and its
specification S’, group the test cases in T into three mutually exclusive sets:
reusable, retestable, and obsolete.
2. Update the test plan by deleting the obsolete test cases, and adding in the new-
structural and new-specification test cases. Observe that before we can solve
subproblem (2), we (2) Update problem need a solution to the test selection
problem [15].
2.6 The Retest Approach In this section, we describe an approach for corrective regression testing. The Retest
approach [9] tries to reuse the test plan in a way that will reduce the amount of testing.
The previous test plan is used to assist in test selection, and is analyzed in order to
classify the various test cases; only a subset of the previous test plan is rerun. The Retest
regression testing process consists of two major phases: the test case classification phase
18
and the test plan update phase (see following Figure2.2). We next describe the major
components of this regression testing process.
Static Analyzer This component is static in the sense that no test case is executed. All
analysis is done using the change information and the information stored in the test plan.
The static analyzer uses this information to classify the existing test cases into two
classes: reusable and unclassified. The reusable test cases are those tests which do not
exercise any modified code, while the unclassified test cases are those which do. Another
function of the static analyzer is to compute the structural coverage measure achievable
from the reusable test cases based on the stored dynamic behavior of these test cases.
Both the coverage measure and the unclassified test cases are passed to the Dynamic
Analyzer.
Dynamic Analyzer
This component is dynamic because it actually executes some test cases in order to:
1) Test the program,
2) Satisfy the structural coverage criterion, and
3) Classify the unclassified tests into obsolete and retestable
test classes.
Figure 2.2 the Retest regression testing process for Corrective Regression Testing
19
The Dynamic Analyzer executes each test which is unclassified in an order implied
by the test-case-order. These test cases will be executed according to their ordering
within the test-case-order. Note that all specification-based tests will be run first and put
in the retestable class. They will not become obsolete because the specification is
assumed to be unchanged. Any structural test which does not increase the structural
coverage measure will be put into the obsolete test class. All others are grouped into the
retest able test class.
The Dynamic Analyzer will stop test case execution whenever the structural coverage
criterion is satisfied. Any remaining unclassified tests may be placed into the obsolete
class, or handled according to the testing guideline specified in the test an output of the
Dynamic Analyzer is the coverage measure achieved by both the reusable and the
retestable test cases. If this coverage measure is not satisfactory, then the Dynamic Tester
is invoked; otherwise the testing process enters the test plan update phase.
Dynamic Tester: This component is similar to the Dynamic Analyzer and is invoked only
when the structural coverage criterion is not satisfied by both the reusable and retestable
test cases. A major function of this component is to execute the new structural test cases
which are designed to exercise the modified code or to satisfy the structural coverage
criterion. Only the test cases which will increase the structural coverage measure will be
stored. Once the structural coverage criterion is satisfied, the Test Plan Updater is
invoked [9]
20
Figure 2.3 an operational overview of Retest
A major difference between the Dynamic Analyzer and the Dynamic Tester is that the
former involves the changing and deleting of existing information in the test plan, while
the latter involves the execution of new tests and the addition of new information to the
test plan. There are two advantages in running the unclassified tests before running the
new-structural test cases:
• Some unclassified tests will test the modified code and therefore they can reduce
the effort in designing new tests.
• Some unclassified tests may be used to assist in test generation.
21
Test Plan Updater: This component creates a new test plan from the reusable, retestable
and new-structural test cases. The objective is to generate an up-to-date test plan for the
next cycle of modifications and regression testing. Most of the information stored in the
new test plan have actually been collected by the Dynamic Analyzer and the Dynamic
Tester. An operational overview of the Retest process is presented in above Figure 3.2.
22
Chapter 3
Literature Survey
Regression testing aims at providing confidence that the modifications are correct
and have not adversely affected other parts of the program. Regression testing can be
progressive or corrective the former involves retesting major changes to the program’s
specification. The latter is performed on a specification that essentially remains
unchanged, so that only minor modifications, which do not affect the overall program
structure, require retesting. For regression testing, it would be costly to repeat the whole
set of test cases used in the initial development of the program and unreliable to choose a
random subset of these test cases [4].
Therefore, it is important to select a suitable subset of test cases that accomplish
the objectives of regression testing. The selection of suitable test cases can be made in
different ways, and a number of regression testing approaches and algorithms have been
proposed. A strategy based on input partitioning and cause-effect graphing of the
program specification has been described. Leung and White have suggested classifying
the initial test cases as reusable, retestable, obsolete, and adding new-structural and new-
specification test cases and, then, selecting test cases from some of or all five classes.
They have also proposed methods to limit integration regression testing to a small set of
modules.
4.1 Techniques of Regression Testing:
Gupta, Harrold, and Soffa’s proposed slicing algorithm which is data flow based. It
uses slicing to explicitly detect definition-use pairs that are directly or indirectly affected
by a program modification. A program slice consists of all statement segments, including
conditionals that might affect the values of variables in the modified statement/segment.
The advantage of this slicing approach is that no data flow history is needed, nor is the
recomputation of data flow for the entire program required after its modification [8].
Leung and White have proposed the concept of “firewall” to assist the tester in
integration regression testing. For our study’s purpose, we have implemented the
“firewall” concept at the segment (vs. module) level and hence the name adapted firewall
algorithm. Leung and White suggest building “firewalls” to confine integration
23
regression testing to a small set of modules rather than allowing it to spread to many
other modules. A “firewall” involves the modules that are modified and their direct
ascendants and direct descendants [8].
An optimal retesting approach based on a 0-1 integer programming problem
formulation has been proposed by Fischer and extended in and solved by natural
optimization in Some software tools, which are based on some of the afore-mentioned
techniques, have been constructed.
Agrawal, Horgan, and Krauser’s incremental algorithm which selects test cases from T
whose outputs may be affected by the modifications made to the program [8]. The
incremental approach exploits the following observations:
• Not all statements in the program are executed under all test cases.
• If a statement is not executed under a test case, it can not affect the program
output for that test case.
Even if a statement is executed under a test case, it does not necessarily affect the
program output for that test case. Every statement does not necessarily affect every part
of he program output. These observations can be used to determine execution, dynamic,
and then relevant slices for the test cases.
Rothermel and Harrold divide regression testing into two phases for the purpose of cost
analysis. During the preliminary phase, changes are made to the software, and the new
version of the software is built. During the critical phase, the new version of the software
is tested prior to its release to customers. They described regression testing techniques as
follows: “Given a program P, a modified version P’, and a set T of test cases used
previously to test P, regression analysis and testing techniques attempt to make use of a
subset of T to gain sufficient confidence in the correctness of P’ with respect to behaviors
from P retained in P’.” [14]
Regression testing is an active area of research. Most of the work involves white
box testing where code is available to determine the test cases. Agrawal et al. addressed
the problem of determining test cases in the regression test suite on which new and old
programs may produce different outputs [ 8] .
They refer the problem of determining regression test suite on which the modified
program may differ from the original program as the incremental regression test problem.
24
They proposed good approximate solution to this problem with execution slice technique
to save regression testing effort with the focus on finding the test cases on which the
original and the modified program differ.
Wu et al. proposed a regression testing technique that selects test cases by utilizing
static information from the analysis of the program structure and dynamic information by
tracing the function-calling sequence. Their approach is based on data-flow method with
optimization to handle OO features, such as inheritance, dynamic binding polymorphism
and message passing. They need to create Functional Calling Graph (FCG) of each test
case to trace the functional execution sequence.
Harrold et al. Proposed the use of component meta-content or metadata to support
regression testing of component-based software. Metadata stores additional information
along with the components. In this sense, it is similar to the version information we
require for components[8].
25
Chapter 4
Case studies of the existing techniques
4.1 Generating Regression Tests via Model Checking
Regression Testing is a maintenance activity that attempts to validate modified
software and ensure that modifications are correct and have not inadvertently affected the
software. Selective regression testing attempts to identify test cases for reuse in testing
modified portions of the system. Specification-based regression testing techniques select
reusable test cases from the original test suite with respect to formal specifications of the
system. One should make sure that the cost of selecting the regression tests is lower than
retesting with the entire original test suite.
In this implementation, our work is to check the ability of model checkers to
generate counterexamples, where each counterexample can be considered as a complete
test Sequence. The general idea of our work is to use the difference between two versions
of software specifications– the original one and the modified version. Our approach,
therefore, supports different test coverage criteria. Currently, our work allows users to
choose from a limited set of FSMbased test coverage criteria to generate regression test
suites, including state transition level (STL), syntactic trigger events level (SynL) and
semantic trigger events level (SemL).
Component
We are taking the example of GUI component. In this, we are comparing the
functionalities of two version of GUI; first one is window 98 notepad and second is
window XP notepad and possibly generates the test suit from these techniques.
We follow the approach, RTMC (Regression Testing via Model Checking), which relies
on the existence of two different specification models – Spec#2(XP notepad) is the
specification of a modified version of the same system described by Spec#1(98 notepad).
The idea is to use the difference between the two models to guide regression test case
generation.
26
98 notepad Spec1#
Notepad of window 98 Specifications File Menu Edit Menu Select 1 New to open new file Cut cut the
selected text
Find
Finds the specified text in a file and stop at first occurrence of text
2 Open to open existing file size should be less then 64kb
Copy
copy the selected text Find Next
Finds the next occurrence of text entered in find box
3 Save Save current file Paste paste the text at cursor position
4 Save as save a copy of current opened file
Undo revert the previous action
5 Page Setup Set the margins of page for printing including header & footer and layout of page
Time/Date
insert current date and time
6 Print print the current file or selected text
Word Wrap Wrap text to opened notepad windows margin, Horizontal scrollbar is off
7 Exit exit from notepad Delete delete the selected text
8 Set Font format the font
9 Select All select all text in a file
Table 4.1.1 Test suite of the Window 98 notepad
27
XP notepad Spec2#
File Menu
Edit Menu Format Menu
View Menu
1 New to open new file
Cut cut the selected text
Word Wrap
Wrap text to opened notepad windows margin, Horizontal scrollbar is off
Status Bar
Will Show status bar for this option word wrap option should be off
2 Open to open existing file size should be less then 64kb
Copy
copy the selected text
Font format the font
3 Save Save current file
Paste paste the text at cursor position
4 Save as
save a copy of current opened file
Undo
revert the previous action
5 Page Setup
Set the margins of page for printing including header & footer and layout of page
Time/Date
insert current date and time
File Menu
Edit Menu
6 Exit exit from notepad
Delete delete the selected text
28
Set Font format the font
Select All select all text in a file
Table 4.1.2 Test suite of window XP notepad
Comparator Comparator is basically used to compare the previous version i.e. window 98 notepad
with the modified version i.e. window XP notepad.
• Spec#1 represent the window 98 notepad
• Spec#2 represent window XP notepad
Following is basics steps of comparator:
Spec#1 and Spec#2 are fed into a comparator;
• The differences between the specifications based on different test coverage
criteria (Spec Diff) will be generated and used to derive properties
• Each specification have the two types of the properties:
Hold properties (Not changed properties): Each property that holds is considered not to
have changed.
Not hold properties (changed properties): Each property that does not hold (a Non-Hold
Prop) is considered to have changed
29
Figure 4.1.1 show the comparator.
30
1 98 Notepad XP Notepad 2 File Menu File Menu 3 New Hold Prop New 4 open Hold Prop open 5 Save Hold Prop Save 6 Save As Hold Prop Save As 7 Page Setup Hold Prop Page Setup 8 Print Hold Prop Print 9 Exit Hold Prop Exit 10 Edit Menu Edit Menu 11 Cut Hold Prop Cut 12 Copy Hold Prop Copy 13 Paste Hold Prop Paste 14 undo Hold Prop undo 15 Time/Date Hold Prop Time/Date 16 Word Wrap non hold Replace 17 delete Hold Prop delete 18 set Font non hold Goto 19 Select All Hold Prop Select All 20 Find 21 Find Next 22 Search Format Menu 23 Find non hold Word Wrap 24 Find Next non hold Font
Table 4.1.3 test suite of comparator
Based on diagram 4.1.1 we have the above test suits that show the Hold and Not hold
properties of 98 notepad and XP notepad.
Coverage Selector
Comparator discovers the differences between two versions of the specification. It
works together with the coverage selector which allows user to choose test criteria that
produce the Regression Test Specification (RT Spec),
Coverage selector has the following specification level:
a) State Transition Level
b) Syntactic Trigger
c) Semantic Trigger level
Following test suits display the working of above three level:
31
Figure 4.1.2 Coverage Selector
State
Transition Level
Syntactic
Trigger
Level
Event
Semantic
Trigger
Level Event
diff in
trigger
events
Events has been totally modified
1 Edit.replace Will
replace
the
selected
text with
Edit.set Font Format.font
Edit.wodwrap
search.Find
search.Find
Next
Format.wordWrap
Edit.Find
Edit.Find Next
32
the text in
clipboard 2 New Events
Added
View.status
Edit.Goto
Table 4.1.4 Test suite of Coverage selector based on above diagram
Test Generator Test Generator produces a set of counterexamples for each non-hold property.
Currently, Test Generator chooses the shortest counterexample from the list,
translating it into corresponding test case.
Test Case ID
Module/Interface/ Event
Test Case Title Test Case Objective Test steps Expected Result
1 Edit.replace replace text
Replaces the selected text with text given in find box
copy the text then select the text which is going to be replaced goto Edit->Replace
Selected text will be replaced by the copied text
2 Format.. Font font format the font Format menu->font
open a font dialog box with font,font style,size,script option
3 Format.wordWrap wrap
Wrap text to opened notepad windows margin, Horizontal scrollbar is off
Format menu->wordwrap
horizontal scroll bar will disappear
4 Edit. Find find Finds the specified text in a file Edit->find open a find dialog box
33
and stop at first occurrence of text
asking find what text box with ok cancel button,up/down and match case options. After typing the text and selecting ok button first instance of that text will be searched
5 Edit.Find Next find next Finds the next occurrence of text entred in find box Edit->Find Next
same find dialogbox will be opened to find next occurrence of that word if it reaches at the end of document and no matching word is found then alert bx will display no match found
6 View. status vstatus Will Show status bar view-> status bar
Will Show status bar for this option wordwrap option should be off
7 Edit.Goto egoto Goto to specific line edit->goto
goto dialog box will be opened and will ask for enter line number.For this wordwrap option should be off
Table 4.1.5 Test suite of Test Generator
Result RTMC are the concept of using a model checker as part of the regression test generation
tool. They support a specification description, and different test coverage criteria. Most
importantly, it offers a significant opportunity to reduce testing costs and automatically
produce regression tests from formal specifications with greater confidence
34
Chapter 5
Related Work in component-based Regression testing and UML In the current scenario, focus has shifted on using UML design models to perform testing
activity. We summarize existing work on generating test inputs from Regression Testing
based on UML models. Work on testing using the UML has likely to be focus on
generating requirements and criteria for code testing from UML diagrams.
In Regression Testing UML design of software components we identify changes to UML
designs to assist in classifying UML test cases. The UML designs we consider consist of
UML Class Diagrams, Sequence Diagrams, and OCL expressions. The one formal
implicit in our approach is that we must rely on the integrated model. In UML integrated
model, The Class Diagram consists of the following primary elements: classes, attributes,
methods, and associations. The Sequence Diagram contains: objects, lifelines, conditions
and messages. Each of these elements can be changed by addition, deletion, or
modification.
Scheetz et al. [20] Towards automated support for deriving test data from UML
sequence diagrams paper describes an approach to generate system test inputs from
UML class diagrams. The class diagrams are restricted to contain only classes,
associations and specifications and specification structures. In this approach we first
identify test objectives for every single class. A test objective describes a set of objects in
terms of the states they can take on. Test objectives are derived from defining desired
states of the class instances after the test is executed. A state of an object is defined based
on its attribute values and links to other objects. Test objectives for a complete class
diagram can be aggregated from test objectives for each individual class specified in the
class diagram.
In this approach, we used the UML and its associated object constraint language (OCL)
to describe the information about the changes from one version to next which was
provided by component developer and comprised modified method and
directly/indirectly affected methods, then utilized this information to direct the test case
selection. This method is quite simple and can be easily implemented in practice, but this
method is so rough that the selected subset of test cases generally contains many
irrelevant test cases
35
Pilskalns et al. [15] Regression testing design paper describes the purpose of a Graph-
based approach to combine the information from structural and behavioral diagrams
(Class Diagram and Sequence Diagrams). In this approach, each sequence diagram is
transformed into an Object-Method Directed Acyclic
Graph (OMDAG). Each node in an OMDAG represents a method call or a return action,
as well as the class of the object that initiates the call. The directed arcs represent control
flow. OMDAGs can be used to generate the execution paths in the sequence diagram that
satisfy different test adequacy criteria. UML testing technique proposed to define
regression testing for UML designs.
UML testing relies upon building an integrated model from Class Diagrams, Sequence
Diagrams and OCL. Building the integrated model consists of three steps. The first step
maps Class Diagrams into tuples, called Class Tuples (CT). The CT consists of a class
name, attributes (represented as tuples if non-primitive) from the class and super classes
(if applicable), and methods (represented as tuples) for the class and super classes (if
applicable). A class tipple is a mathematical representation of a class and is similar to the
idea of representing a class using the XMI specification. OCL statements are mapped into
the CT based upon their context.
The second step consists of mapping Sequence Diagrams into an Object Method Directed
Acyclic Graph (OMDAG). The OMDAG maps the dynamic information in a Sequence
Diagram to a directed acyclic graph. The OMDAG is created by mapping object and
sequence method calls from a Sequence Diagram to vertices and arcs in a directed acyclic
graph.
They propose a regression testing technique based on UML sequence and class diagrams.
Their approach does not take into account the pre and post conditions of the operations
which affect behavior of a class. Also, their approach does not handle concurrency.
So finally we have concluded that there is some lack of points in the existing approach.
These points are as follow:
1. This method is quite simple and can be easily implemented in practice, but this
method is so rough that the selected subset of test cases generally contains many
irrelevant test cases.
36
2. It’s difficult to compute the efficiency from manually UML based test case
generation.
3. Above discuss approach does not take into account the pre and post conditions of
the operations which affect behavior of a class. Also, their approach does not
handle concurrency.
In our work, we would require to extend the work of manual UML based Regression
testing and will generates efficient UML regression test case generation for software
component by using programmatically Graph-Based approach.
37
Chapter 6 Control Flow Graph (CFG) based criteria to generate Regression Test Suite. Our approach is to generate Regression Test suite for software components is based on
the UML sequence diagram storage graph.
Figure 6.1 shows the overall working of the control flow graph based criteria for the
generation of the regression test suite.
We construct the storage graph using C++ program of data structure. We have used two
flow graphs; control flow graph CFG and extended current flow graph ECFG to store the
UML sequence Diagram of the software components. The basic information of sequence
diagram is sequence of events that are extracted from the diagram. The UML sequence
diagram designed with the help of Net Beans Tool. Net Beans Unified Modeling
Language (UML) tools enables you to generate UML Diagrams such as sequence
diagrams. A net beans tool is the java enabled tool, there is no need to drawn the
38
sequence diagram manually using the net beans tool, it works as reverse engineering tool.
It takes the java program source code and generates the sequence diagram automatically.
The two flow graphs are compared to identify the changes and hence to generate test
cases. The different steps of our approach are as follows:
1. Built Sequence diagrams for software components.
2. To construct the CFG to store sequence diagram.
3. Traversal of the Storage graph to generate regression test suites.
1. Built Sequence diagram for software components: Sequence Diagrams: A Sequence Diagrams models system behavior by specifying how
object interact to complete the task. An interaction is expressed by the messages between
lifelines. A lifeline is participant in an interaction. A lifeline represents a class instance.
In this thesis, a message can represent a message invocation, a reply message, a creation
and deletion of a class instance.
Figure 6.2 shows the Basic Sequence Diagrams of Students who are enrolling in
seminar in University.
2. To construct the Control Flow Graph (CFG) to store sequence
diagram. In this part of our approach we construct the control flow graph( CFG) using the C++
program of data structure. The minimum requirement for the storage graph is window 98
39
or higher version and minimum 32 MB RAM. In the control flow graph the States are
represented by vertices of graph and methods by edges through which the object changes
from one state to another. The precondition and postcondition must be satisfied all the
methods and their parameter are stored as edges. The construction Storage Graph is built
using Graph of Data Structure, Where each Edges and Vertices are represented as State.
Struct EdgeState; Struct VertexState; Struct VertexState { Char StateId [10]; int IsInitial, IsFinal; VertexState *NextVertex; EdgeState *EdgeRoot; VertexState () { IsInitial=IsFinal=0; EdgeRoot=NULL; } The composing of vertex node is as follows:
State Name Next State Method
Pointer
Precondition Postcondtion
The Edge node is as follows:
Method
Id
State
From
State To Sender Receiver Attributes Next
method
Each attribute is abstract type and consists of:
Date Type Attribute Name
Algorithm for the generation of storage graph for the sequence
diagram: A state is to be added in the sequence diagram. A graph in the sequence diagram and
states represent the vertices of the graph. INIT is the initial state; if it is NULL then we
can say that the sequence diagram is missing.
40
if INIT = NULL then
Display “No diagram found”
Endif
To add a state S into graph G1 representing sequence diagram of first version, we will
use it as:
if INIT = NULL then INIT = S
Display “first assume intail state is empty then add in it one state.”
// Last state is extracted with the help of a loop traversing each state of a graph
Else t = G1.lastState then t.NextState= S
Endif
Adding methods/ transitions into States of the control flow graph. A method represents the transition from state s1 to state s2 in graph. It has many
attributes like methodid, parameters, sender, receiver, and return type of the method etc. Let s1 = source state of the transition
s2 = destination state of the transition
M = method of the of the each transition
x = G1.findsourceState (s1)
If X = NULL then
Display “invalid source state”
Endif
End
t1 = x.lasttransition ( ) // list transition in the list of transition as from state x
will be extracted
t1-> NextTransition = M // new method will added in the transition list.
3. Traversal of the storage graph to generate regression test suites.
After written of the control flow graph we compare both version of the sequence
diagram. And traverse the storage graph for both the version of the sequence diagram.
We used depth first search technique for the traversal of the regression test case
41
algorithm. And based on the comparison of the storage graph, we generate the
regression test suites.
G1= Graph for first Sequence Diagram
G2= Graph for Modified Version of G1
T1= Traverse Stats of G1
T2= Traverse States of G2
E1= Edges of the Graph G1
E2= Edges of the Graph G2
1. T2= G2[INIT] // initial states of Graph G2
2. while T2 != NULL // repeat for all states of graph while their exist a transition
E2= T2 [METHOD POINETR]
While E2 ! = NULL // repeat for all transition of current states i.e. for all
methods that is applicable at current state
CALL Check Match (G1, E2) // search the method E2 in G1
E2=E2 [NEXTMETHOD] // jump to next transition/method
End while
T2=T2 [NEXTSTATE] // jump to next state
End while
3. End
// ChekMatch method
CheckMatch (G1, E2) // this function match the method E2 with in
sequence diagram G1
1. T1= G1[INIT]
Match=false // assume match is not present
While (T1 != NULL) // start search match for each states and each method
within state.
E1= T1 [METHOD POINETR]
While (E1 != NULL)
42
// all attributes of E1 and E2 matches
If E1=E2 then
Display “Reusable”
Return
// if method name match but attributes do not mark as true
Else if E1 [MethodId] = E2 [MethodId]
Match= True
Endif E1= E1 [NEXT METHOD]
End while
// if match is true that is method was found but attributes did not matched that method is
invalid
If Match= True then Display “Obsolete”
Else if Match= false
// if nothing matches not even method name then it newly introduced method
Display “test case are newly added or ReTestable”
43
Chapter 7 Case Studies
In the previous chapter we have constructed an algorithm of control flow graph for the storage of sequence diagrams. In which, we created the states and transition for the graph. And after that, we have written an algorithm for the traversal of the storage graph for the generation of the regression test suites. Now these algorithms will be practically experimented on actual components and the component is drawn with the help of Net Beans Tool. In this chapter, we have generated the regression test cases using the sequence diagrams of the three different components. 7.1 Case Study with ATM using the Net Beans Tools for UML Sequence
Diagram a) The algorithm for constructing the storage graph will store all possible information of
ATM into the States and Edges, before that:
First we generated the Sequence Diagram of both the version using Net Beans tools.
Figure 7.1.1 shows the first version of the ATM Sequence Diagram..
And with help of the algorithm of storage graph for the sequence diagram. We store
each states and transitions of the ATM sequence diagram.
Source Destination Method &Parameter Sender Receiver
44
ST1
ST2
ST3
ST3
ST4
ST2
ST1
ST2
ST4
ST3
Request(int amount, int AccNo)
getcash(int amount, string recpt)
Invalid(int AccNo, String Msg)
verfybal(int amount, int AccNo)
balinfo(int bal, int AccNo)
USER
ATM
CONSORT
CONSORT
BRANCH
ATM
CONSORT
ATM
BRANCH
CONSORT
Table 7.1.1 shows the case scenarios based on the ATM sequence diagram of first
version.
The scenarios show all the transition in which we have a different source and
destinations, methods, sender and receiver. We traverse the graph and store all the
information from UML Sequence diagram.
45
Figure 7.1.2 show extending the functionality of first version of ATM sequence
Diagram.
The modified version of the ATM shows some extra functionality from the previous
version of the ATM. The modified component has some new events such as states
and transition which was not in the previous version of the sequence diagram.
Source Destination Method &Parameter Sender Receiver
46
ST1
ST2
ST3
ST3
ST4
ST4
ST5
ST5
ST5
ST2
ST1
ST2
ST4
ST3
ST3
ST4
ST4
ST2
Request(int amount, int AccNo)
getcash(int amount, string recpt)
Invalid(int AccNo, String Msg)
verfybal(int amount, int AccNo)
balinfo(int bal, int AccNo)
clear(int AccNo, int amount)
failure(int AccNo, string Msg)
Success(int balance, int Recptno)
Recpt(int prvbal, int currbal)
USER
ATM
CONSORT
CONSORT
BRANCH
BRANCH
CASH_ST
CASH_ST
CASH_ST
ATM
CONSORT
ATM
BRANCH
CONSORT
CASH_ST
BRANCH
BRANCH
ATM
Table 7.1.2 shows the nine case scenarios based on the ATM sequence diagram of
modified version.
b) Based on the case scenarios of the both the diagrams, the algorithm of traversal
will compares the events of the both the version. And generates the regression test
suites. The algorithm of Regression Testing will compare both the version of the
sequence diagrams or interaction diagrams. So that we can find out how many test
cases are same in both the version and how many tests are new in modified
version, and how many tests are not valid.
TEST ID STATE METHOD (PARAMETER) SENDER RECEIVER TEST ID : 01 ST2 Request(int amount, int AccNo) USER ATM Reusable ------------------------------------------------------------------------------------------------------------------------------------
47
TEST ID : 02 ST1 getcash(int amount,string recpt) ATM USER Reusable -------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 03 ST3 vldacc(int AccNo,int cardno) CONSORT BRANCH Obsolete ---------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 04 ST2 invalid(int AccNo,string msg) CONSORT ATM Reusable --------------------------------------------------------------------------------------------------------------------------------------- TEST ID : 05 ST4 verfybal(int 0.0000,int AccNo) CONSROT BRANCH Obsolete ----------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 06 ST3 balinfo (int cbal,int AccNo) BRANCH CONSORT Reusable ------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 07 ST5 clear(int AccNo,int amount) BRANCH CASH_ST NewAdded -------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 08 ST4 failure(int AccNo,string msg) CASH_ST BRANCH NewAdded --------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 09 ST4 success(int balance,int recptno) CASH_ST BRANCH NewAdded --------------------------------------------------------------------------------------------------------------------------------------------- TEST ID: 10 ST2 recpt (int prvbal, int currbal) CASH_ST ATM NewAdded ---------------------------------------------------------------------------------------------------------------------------------------------- Table 7.1.3 shows the final Regression Test case generation from the ATM component.
In regression test suites, we find the changes occur in the modified version of the ATM
component. In the regression test suites half of the test cases are from the modified
component. So they all are new added.
6.2 Case Study with E-Ticket of Cinema for UML Sequence Diagram Our second case study with the E-Ticket of Cinema for movie, we have generated the
sequence diagram first for both the version of the component. In the E-Ticket of Cinema,
the sequence diagram describes how the customer will login for ticket, and after login
they can able to see the detail of the movies. They can choose the movie from the
available list.
48
In our first step, we have generated the sequence diagram for both the version of Cinema
component,
Figure 7.2.1 shows the sequence diagram of first version of E-Ticket of Cinema
component. Source Destination Method(parameter) Sender receiver
S1 S2 Login(string Un) Customer Boooking
S1 S2 TickDet(int TickNo,int seat)
Customer Boooking
S2 S1 LoginFail() Booking Customer
S2 S3 ChFilm(string name) Booking Movie
S2 S4 ChCinema() Booking Custom
S2 S5 ChSeat(int seatNo) Movie SeatPlan
S3 S2 AvailFilm(list film) Movie Booking
S3 S4 GetMovie() Cinema Cinema
49
S4 S3 Details(list films) Movie Movie
S5 S2 SeatAlot(list SeatNo) Seatplan Booking
Table 7.2.1 shows the Case scenarios of first version of the E-Ticket system of Cinema. This case scenario is generated with the help of the storage graph algorithm; we traverse
each states and transition of the graph. And in the transition we have various sender and
receiver, method and their parameters.
Figure 7.2.2 shows the sequence diagram of modified version of E-Ticket of Cinema. Source Destination Method(parameter) Sender receiver
S1 S2 Login(string Un) Customer Boooking
S1 S2 TickDet(int TickNo,int seat)
Customer Boooking
S2 S1 LoginFail() Booking Customer
S2 S3 ChFilm(string name) Booking Movie
50
S2 S4 ChCinema() Booking Custom
S2 S5 ChSeat(int seatNo) Movie SeatPlan
S3 S2 AvailFilm(list film) Movie Booking
S3 S4 GetMovie() Cinema Cinema
Source Destination Method(parameter) Sender Receiver
S4 S3 Details(list films) Movie Movie
S5 S2 SeatAlot(list SeatNo) Seatplan Booking
S5 S6 Showstat() Seatplan Seats
S6 S5 Status() Seats Seatplan
S1 S6 Fill_form() Customer InpForm
S6 S1 Validform() InpForm Customer
Table 7.2.2 shows the Case scenarios of modified version of the E-Ticket system of
Cinema.
Based on the comparison of the both the control flow graph of first version of the E-Ticket of Cinema and Extended flow graph of the modified version of the E-Ticket of Cinema. The traversal of the regression test algorithm will produce the final regression test suite. Test_ID State Method(Parameter) Sender Receiver Comments
01 S2 Login(string Un) Customer Booking ReUsable
02 S2 TickDet(int TickNo,int seat)
Customer Booking ReUsable
03 S7 Fillform() Cinema InpForm New Added
04 S1 LoginFail() Booking Customer ReUsable
05 S3 Choose(string name) Booking Movie New Added
06 S4 ChCinema() Booking Cinema Obsolete
51
07 S5 ChSeat(int seatNo) Booking Seatplan ReUsable
08 S2 AvailFilm(list film) Movie Booking ReUsable
Test_ID State Method(Parameter) Sender Receiver Comments
09 S4 GetMovie Movie Cinema ReUsable
10 S3 Details (list films) Cinema Movie ReUsable
11 S2 SeatAlot(int SeatNo) Seatplan Booking Obsolete
12 S6 ShowStat() Seatplan Seats New Added
13 S5 Status() Seats Seatplan New Added
14 S1 ValidForm() InpForm Customer New Added
Table 7.2.3 shows the final Regression Test case generation from the E-ticket of cinema
component.
7.3 Case Study with Library Record System for UML Sequence
Diagram Now practically experiment our approach with third component i.e. Library Record System,
52
a) Based upon the steps, first generates the sequence diagram for both of the library record system. Then possible stores those into the storage graph. And then traversal of the storage graph for regression test case selection.
Figure 7.3.1 shows the Sequence diagram of the first version for the Library Record system
Source Destination Method(parameter) Sender receiver
S1 S2 FillForm(int form_no,int AmtFee)
Person membship
S2 S3 ReqIssue(int BookNo,int BName)
Membship Librarian
S2 S1 Verify(string PerName,string Address)
Person Membship
S2 S1 Register(int PerId,string msg)
Membship Person
S2 S3 RetBook(int BookNo,int Date)
Membship Librarian
S3 S2 Issued(int MembId,int BookNo)
Librarian membship
53
S3 S2 Fine(int delay, float amount)
Librarian membship
S3 S4 Request(list order,float cost)
Librarian PurDept
Table 7.3.1 shows the case scenarios of the first version of the Library Record System
Figure 7.3.2 shows the Sequence diagram of the modified version for the Library Record system. Source Destination Method(parameter) Sender receiver
S1 S2 FillForm(int form_no,int AmtFee)
Person membship
S2 S3 ReqIssue(int BookNo,int BName)
Membship Librarian
S2 S1 Verify(string PerName,string Address)
Person Membship
S2 S1 Register(int PerId,string msg) Membship Person
S2 S3 RetBook(int BookNo,int Membship Librarian
54
Date)
S3 S2 Issued(int MembId,int BookNo)
Librarian membship
S3 S2 Fine(int delay, float amount) Librarian membship
Source Destination Method(parameter) Sender receiver
S3 S4 Request(list order,float cost) Librarian PurDept
S4 S5 Order(list olderls) PurDept BookDepo
S4 S5 IsAvail(list books) PurDept BookDepo
S5 S4 Status() BookDepo PurDept
S5 S4 Bill(float amount) BookDepo PurDept
Table 7.3.2 shows the case scenarios of the modified version of the Library Record
System.
b) Based on the comparison of the control flow graph of first version of the library record
system and extended flow graph of the modified version of the library record system. The
traversal of the regression test algorithm will produce the final regression test suite.
Test_ID State Method(parameter) Sender Receiver Comments
01 S2 FillForm(int formno ,int
AmtFee)
Person Memship ReUsable
02 S3 ReqIssue(int BookNo, int
BName)
Memship Librarian ReUsable
03 S1 Verify(string
PerName,string Address)
Memship Person Obsolete
55
04 S1 Register(int PerId, string
msg)
Memship Person ReUsable
Test_ID State Method(parameter) Sender Receiver Comments
05 S3 RetBook(int
BookNo,string Date)
Memship Librarian Obsolete
06 S2 Issued(int MembId, int
BookNo)
Librarian MembShip ReUsable
07 S2 Fine(int delay, float
amount)
Librarian MembShip ReUsable
08 S4 Request(list order, float
cost)
Librarian Purdept ReUsable
09 S5 Order(list olderls) Purdept Bookdepo New Added
10 S5 IsAvail(list books) Purdept Bookdepo New Added
11 S4 status () Bookdepo Purdept New Added
12 S4 Bill(float amount) Bookdepo Purdept New Added
Table 7.3.3 shows the final regression test suite for Library record system
56
Chapter 8
Conclusions and Future work
8.1 Evaluation of case studies: a) Evaluation of the ATM component
We can check the running time of the ATM component for producing the final regression
test suite. This can be checked with the help of the bar chart.
New ATM test case File
Final Regression Test suite
Old ATM test case File
Component: ATM
Total regression test cases Reusable Obsolete Retestable 10 4 2 4
10
42
4
0
5
10
15
Series1 10 4 2 4
Total ReUsable Obsolete Retestable
Running Time: 0.011
Figure 8.1.1 shows the Bar chart of the total number of regression suite for ATM
component
The bar chart shows the total number of the test cases and the shows how many test cases are Retestable, Reusable, and Obsolete in regression testing along with their running time.
57
b) Evaluation of the E-Ticket of Cinema component
We can check the running time of the E-ticket of Cinema component for producing the
final regression test suite. This can be check with the help of the bar chart.
.
Component: E-Ticket of Cinema
Total regression test cases Reusable Obsolete Retestable
14 7 2 5
New E-ticket of Cinema test file
Final Regression test suite
Old E-ticket of Cinema test file
14
7
2
5
0
5
10
15
Series1 14 7 2 5
Total ReUsable Obsolete Retestable
Running time: 0.0264
Figure 8.1.2 shows the total number of regression suite for E-ticket of Cinema component
The bar chart shows the total number of the test cases and the shows how many test cases
are Retestable, Reusable, and Obsolete in regression testing along with their running
time.
58
c) Evaluation of the Library Record System component
We can check the running time of the Library Record System component for producing
the final regression test suite. This can be check with the help of the bar chart.
Final Regression test suite
Component: Library Record System
Total regression test cases Reusable Obsolete Retestable 12 6 2 4
New Library Record System test file
Final Regression test suite
Old Library Record System test file
12
6
24
0
5
10
15
Series1 12 6 2 4
Total ReUsable Obsolete Retestable
Running Time: 0.0247
Figure 8.1.3 shows the total number of regression test suite for Library record system
component.
The bar chart shows the total number of the test cases and the shows how many test cases
are Retestable, Reusable, and Obsolete in regression testing along with their running
time.
59
8.2 Summary of the contribution.
Regression testing is defined as the execution of a set of application tests to verify that
the application’s quality has not degraded. Regression testing is necessary after a change
to the application program, a part of the application to a different vendor’s machine, or
change in the underlying operating system. Regression testing techniques generally apply
to test modified code, not designs.
We have proposed a systematic technique of UML based regression testing applied on
sequence diagrams. The primary goals in the design of the UML are to provide users with
a ready-to-use, expressive visual modeling language so they can develop and exchange
meaningful models. The UML Sequence diagrams are the most popular UML artifact for
dynamic modeling, which focuses on identifying the behavior within your system.
We started with the manual Regression testing for UML design. We have performed
black-box as well as white box testing for the generation of the UML based regression
test suite. As a part of our research work, programs have been written in C++ to generate
and traverse control flow graph for the storage and traversal of UML sequence diagram.
The sequence diagrams of different versions of software components are initially stored
and then compared automatically to generate regression test suite.
We have practically demonstrated the approach by considering three different case
studies from different application domains. The results have been verified by the running
time of the regression test suite generation technique.
8.3 Future Work Currently we have implemented our programs in C++ to construct control flow graph for
the storage of the UML sequence diagrams and they have been traversed automatically
for generation of the regression test suite. So we are only using the UML sequence
diagrams for the generations of the final regression test suite. In future, to generate the
regression test suite; other UML diagrams like activity and use case models can also be
considered. It will widen the domain of analysis of the software component. Thus,
resulting in effective test suite.
60
References
[1] Atifah Ali1, Aamer Nadeem1, Muhammad Zohaib Z. Iqbal2, Mohammad Usman,
“Regression Testing based on UML Design Models”, in the proceeding of the 10th
International Conference on Information Technology, IEEE press 2007.
[2] Bruce W. Weide, "Modular Regression Testing", Connections to Component-Based
Software, Dept. of Computer and Information Science, The Ohio State University , 2015
eil Ave. Columbus, OH 43210,USA +1 614 292 1517 [email protected] , IEEE press
2005 .
[3] Chengying Mao, Yansheng Lu,” Regression Testing for Component-based Software
Systems by Enhancing Change Information”, Proceedings of the Asia-pacific Software
Engineering Conference (APSEC’05), Document number 0-7695-2465-6/05, IEEE press
2005.
[4] David Binkleyf “Reducing the Cost of Regression Testing by Semantics Guided Test
Case Selection”, Proceedings of the Department of Computer Science Department
Loyola College in Maryland 4501 North Charles Street Baltimore, Maryland 21210
binkley @cs.loyola.edu, IEEE press 1995.
[5] Demissie B. Aredo “Sematics of UML Sequence Diagrams in PVS”, in the
proceeding of the Department of Informatics, University of Oslo Institute for Energy
Technology.
[6] Fevzi Belli, Christof J. Budnik, “Towards Self-Testing of Component-Based Software
“, Proceeding of the 29th Annual International Software and Applications Conference
(COMPSAC ’05), IEEE, 2005.
61
[7] Gregg Rothermel and Mary Jean Harrold “A Safe, Efficient Algorithm for
Regression Test Selection”, Proceedings of the Department of Department of Computer
Science Clemson University Clemson, SC, 29634-1906, {grot her,
harrold}@cs.clemson.edu, IEEE press 1993.
[8] Ghinwa Baradhi and Nashat Mansour “A Comparative Study of Five Regression
Testing Algorithms”, in the procededing of the Department Ghinwa Baradhi and Nashat
Mansour Computer Science Lebanese American University, [email protected]. Lb,
IEEE press 1997.
[9] Hareton K. N. Leung “Insights into Regression Testing”, Proceedings of the
Department of Computing Science University of Alberta Edmonton, Alberta, Canada
T6G 2H1, IEEE press 2000.
[10] Jiang Zheng1, Laurie Williams1, Brian Robinson2, Karen Smiley “Regression Test
Selection for Black-box Dynamic Link Library Components”, Proceedings of the
Department of Computer Science, North Carolina State University, Raleigh, NC 27695
{jzheng4, lawilli3}@ncsu.edu 2 ABB Inc., US Corporate Research {brian.p.robinson,
karen.smiley}@us.abb.com, IEEE press 2007.
[11] L. C. Briand Y. Labiche G. Soccar “Automating Impact Analysis and Regression
Test Selection Based on UML Designs” , Software Quality Engineering Laboratory
Carleton University Department of Systems and Computer Engineering 1125 Colonel By
Drive Ottawa, ON, K1S5B6, Canada {briand} labiche}@sce.carleton.ca, IEEE press
2002.
[12] Li Bao-Lin, Li Zhi-shu, Li Qing, Chen Yan Hong “Test Case automate Generation
from UML Sequence diagram and OCL expression”, 15th International Conference on
Advanced Computing and CommunicationsChina, IEEE 2007.
62
[13] Lihua Xu, Marcio Dias, and Debra Richardson “Generating Regression Tests via
Model Checking” Proceedings of the Department of Informatics, School of Information
and Computer Science, University of California at Irvine, CA, USA {lihuax, mdias, djr}
@ ics.uci.edu, IEEE press 2004.
[14] Monalisa Sarma and Rajib Mall “Automatic Test Case Generation from UML
Models”, in the proceeding of the 15th International Conference on Advanced
Computing and Communications , IEEE press 2007.
[15] Orest Pilskalns and Gunay Uyan “Regression Testing UML Designs”, Proceedings
of the Washington State University Vancouver, WA orest,[email protected],
IEEE press 2006.
[16] Orest Pilskalns, Daniel Williams, Damir Aracic “Security Consistency in UML
Designs”, in the proceeding of the School of Engineering and Computer Science
Washington State University, Vancouver, WA 99686, IEEE press 2006.
[17] Rami Bahsoon and Nashat Mansour “Methods and Metrics for Selective Regression
Testing”, Proceedings of the Department of Computer Science, University College
London, Gwer Street, London, WClE 6BT, UK. E-mail: r.bahsoon @cs.ucl.ac.uk, IEEE
presses 2001.
[18] Sujit Kumar Chakrabarti, and Y. N. Srikant “Specification Based Regression Testing
Using Explicit State Space Enumeration”, Proceedings of the International Conference on
Software Engineering Advances (ICSEA'06), IEEE press 2006
[19] Siros Supavita and Taratip Suwannasart “Testing Polymorphic Interactions in UML
Sequence Diagrams”, in the proceeding of the Department of Computer Engineering,
Faculty of Engineering Chulalongkorn University, Bangkok,IEEE press 2007.
63
[20] Scheetz et al. and j.cui “Towards Automated Support For Deriving Test Data From
UML Sequence Diagrams”, in the proceeding of the 6th international conference on the
UML. San Franscisco, CA, USA, October 2003.
[21] Weiqun Zheng and Gary Bundell “Model-Based Software Component Testing: A
UML-Based Approach”, in the procededing of the Department Weiqun Zheng and Gary
Bundell Centre for Intelligent Information Processing Systems, School of Electrical,
Electronic and Computer Engineering, , IEEE press 2007.
64