A MODEL DRIVEN APPROACH TO REGRESSION TESTING OF … · 2018-09-29 · component [2]. The regression testing improves the con dence in some other aspects like software patches, traditional

A MODEL DRIVEN APPROACH TOREGRESSION TESTING OFREUSABLE SOFTWARE

Prabhakar K.1, A. Ananda Rao2,K. Venugopala Rao3, K. K. Swamy4,

1,4Department of Computer Science & Engineering,JNT University Ananthapur,

Vardhaman College of Engineering, India2Department of Computer Science & Engineering,

JNT University Ananthapur, India3Department of Computer Science & Engineering,

GNITS, Hyderabad, India [email protected];[email protected];

[email protected];[email protected]

June 13, 2018

Abstract

Regression testing became a tedious task, due to theunpredictable integration of reusable software componentswith other components. And the cost of this regressiontesting increased in the form of effort and computingresources. The existing approaches like dependencyanalysis, mutual collaboration, test case- minimization andprioritization are not sufficient to resolve this problem.Because there is no guarantee that the reusable componentthat behaves correctly in one environment may show thesame behavior in the other or new environment. And thereis no approach to say, how a component has to be tested

1

International Journal of Pure and Applied MathematicsVolume 120 No. 6 2018, 3571-3591ISSN: 1314-3395 (on-line version)url: http://www.acadpubl.eu/hub/Special Issue http://www.acadpubl.eu/hub/

3571

at what extent in new the environment [1]. Therefore thispaper proposes a new technique to address theabove-mentioned problem. The proposed approach ismodel driven regression testing where use cases areconsidered primarily for deriving test cases. At thisjuncture, test cases are automatically generated from themodel for functionalities and the model itself willre-generate the new test cases for future enhancements.The generated test cases are 100% reusable. This approachis evaluated by considering a database component andobtained satisfactory results. Regression testing iscontinues process at every usage of reusable componentthroughout its life. Hence, using this approach, it ispossible to maximize test coverage, to reduce regressiontesting cost, time and effort. This approach is also usefulto detect errors early in the development.

Keywords:Model driven approach, Quality assurance,Regression testing, Software reuse, Test suite generation.

1 INTRODUCTION

REGRESSION testing is mandate for every reusable softwarecomponent at maintenance or modernization or at componentbased construction to validate the system, whether it isimplemented as it was intended or not [2]. Because 80% of errorsare caused by 20% usage of Reusable components or componentbased construction only [1]. Hence, the regression testing confirmsthat the newly adopted functionalities or modifications are notaffecting the other parts of an existing system or systemcomponent [2]. The regression testing improves the confidence insome other aspects like software patches, traditional and agilesoftware development process where more number of revisions willtake place.

The model based/driven testing is an evolving technique toaddress the regression testing complexities. The use case modelsare used/taken from the behavioral models of Unified ModelingLanguage (UML) for generating test cases. From the functionalrequirements of the system the use case model is designed toidentify all system functionalities such as ¡Main flows¿,

2

International Journal of Pure and Applied Mathematics Special Issue

3572

¡alternative flows¿, ¡extends¿, ¡includes¿, ¡generalization¿ and¡other relation¿ of software system, The test cases are derivedfrom the use cases and used in regression testing to validate thesoftware intensive system.

As a part of this research, the database component has beentaken into consideration as a reusable software component. Thereason for taking a database component as a running example isthat one day a technology may fade away from our eyes, but there isa less chance to fade out a database. For every reusable component,the developer will provide its specifications and appropriate testcases. From the given specifications all the functional requirementsare identified and listed. On the chosen database component, threeapproaches are applied to generate test cases. These approachesare General regression testing criteria, Test suite minimization andModel driven approach.

Through the first approach, functional test cases andBoundary Value Analysis (BVA) test cases are generated from therequirements. In this approach the time, effort and cost of there-testing are estimated. Through the second approach, the testcases are minimized by combining functional and BVA test case.So that, the second approach will reduce the test effort and costconsiderably than the first approach. The third approach is theprecise solution for the regression testing of a component basedsoftware system. Because, through this approach, we gotminimized test suite, additional test coverage, low test executiontime, reduced effort and cost.

The rest of the paper is organized as the background andrelated work is discussed in section 2. The model driven approachto regression testing is discussed in section 3. The cost effectivequality assurance is discussed in section 4 which includes threedifferent approaches for test case generation. This section is alsotalking about effort and cost estimation. In section 5 results andits comparisons are discussed. In section 6 conclusions and futuredirections are presented.

3


3573

2 RELATED WORK

There is a great need of model based regression testing in softwaremaintenance, because, according to the L. Erlikh, 85-90% of thesoftware projects are under maintenance [3]. Hence, the regressiontesting plays a critical role in maintenance. Peter A. Lindsay, usedbehavioral trees as a graphical notation for the requirements, Peteridentified relations among functional requirements and investigatedtest cases from the relations. Peter used symbolic model checker forensuring the correctness and completeness of all test cases related tothe system model [4]. But, the behavioral trees and symbolic modelcheckers are not sufficient to generate all test cases. Because theseare used for describing and analyzing the behavior of a softwareintensive system and may not focus on data dependencies.

Sebastian Ulewicz, presented an approach to identification andclassification of software changes. The changes among programversions are based on the models of program structure [5]. Thesechanges may have the impact on the test cases. Hence, thisclassification may predict the new test suites need to bedeveloped. This approach is well suitable for unit and integrationtests but not sufficient for all other tests. Further, moreclassification and syntax analysis of change may lead to additionalchange. Moreover, the Object Constraint Language (OCL) maynot support complete change impact model which includes a set offormal rules.

Philipp Caliebe presented an approach to test caseprioritization and selection. This approach is based ondependency analysis among the components. The model for thisapproach is derived from the requirements and systemarchitecture [6]. Test case prioritization and selection is done byComponent Dependency Model and deriving System Graph.Here, the author himself specified few issues related to presentedapproach like exhaustiveness of path search, the correctness ofpath selection and effectiveness of test suite.

Erik Rogstand applied Expectation-Maximization(EM-clustering) algorithm for grouping the regression testdeviations which are caused by the versions and faults [7]. Itseems to be good but, in independent testers point of view theremay be a chance of getting confusion between system model

4


3574

properties and clusters. More over this methodology can beapplied to databases only. Marry Jaen Harrold, specifiedanalogous challenges in component based software engineering/software reuse. One of the challenges is how regression test suiteis interacting with other suites and how to execute the test suiteat system level [8]. This challenge is addressed as part of theproposed research.

Ana M. Fernndez-Sez, did a survey on UML models used forsoftware maintenance and its challenges [9]. Susanne Rsch, did asurvey and explained the importance of models in softwaretesting, challenges in software maintenance and emergency ofmodels in regression testing [10]. Jim Heumann, outlined thegeneration of test cases from behavioral models so that thedevelopers can simplify their testing process [11]. Jim, proposed atest case matrix with data values. This is very basic and it doesnot address the problems with maintenance and regressiontesting. Adtha Lawanna, talking about regression suite sizereduction, so that they can speed up the execution time [12], thisis all about suite reduction, prioritization but not talking aboutearly detection of requirement errors. The proposed approach hasgood functional coverage and boundary value analysis capabilities.

3 A MODEL DRIVEN APPROACH

TO REGRESSION TESTING

Most of the disasters happening in the cyber world are due to lack ofproper testing. The live example is Ariane 5, a European spaceshipwas crashed within 37 seconds of its take-off. This is what exactlythe software testing failure where no proper validation is done tothe reusable software [13].

The reusable component based software developmentenvironment is an unpredictable environment, where source codeis unavailable and components may be developed in differentprogramming languages. This Unpredictable componentintegration and the dependencies among components are makingregression testing as a tough task. Moreover, execution of a singletest case is also becoming a tedious task.

The proposed approach presented in Figure 1 explains the

5


3575

model based regression testing for component based systems.Whenever a system is developed with reusable softwarecomponents (COTS: Commercial-Off-The-Self) or a system comesfor maintenance or modernization, it is mandated to do re-testingknown as regression testing. To perform regression testing, thetest suite has to be prepared and executed by Quality Assurance(QA) team. The regression test suite is prepared by combiningexisting test cases with enhanced test cases. The existing testcases are already available with components because, whiledeveloping the individual components, it is the responsibility of adeveloper to provide component specifications and necessary testcases.

Figure 1: The model based regression testing for component basedsystems.

Now, the enhanced test cases are generated from the system’sbehavioral model. The best behavioral model is use case model ofUML. Because it is very much close to the behavior of the systemand it is an exact blue print. The use cases are drawn from therequirements and test cases are derived from use cases byconsidering all flows of the system like <Main flows>,

6


3576

<alternative flows¿, <extends>, <includes>, <generalization>and <other relation> etc. It is replicating the DU- path testing(Defined and Used). This regression testing will cover <allcontext dependencies>, <all events>, <all interfaces> and<other artifacts> of a newly developed system. The generatedtest cases are executed automatically by the tool or manually.

3.1 Component Development

A component is a physical, sovereign, replaceable and reusablepart of a software system. It can work for a large application byintegrating with other components. Figure 2 shows the data basecomponent having five different actions on data.

Figure 2: Database Component

Table 1: The Database component implementation.

Table 1 shows the implementation of the database component.The implementation table consists of 5 columns and columns names

7


3577

are given from column 1 to column 5, column 1 is created for integervalues, column 2 is for a string value, column 3 is for a singlecharacter value, column 4 is for double value and column 5 is fordate value.

Figure 3 Show the User Interface (UI) implementation ofdatabase component having five fields which match with thedatabase. Here, ID field is a number data type, the Name field isa string type, Gender is a character type, Salary is a double typeand DOB is a date type.

Figure 3: the database component user interface implementation

3.2 Deriving Test Cases for ReusableComponent

In regression testing process, the regression test suite generallycontains Functional test cases for validating system functionality,Boundary value analysis test cases for validating of each boundaryof a functionality. System performance and Stress test cases forperformance and stress checking. Among these categories of testcases, the functional and boundary value analysis test cases aremajor and important. Here, three approaches are presented forgenerating test cases and estimating the regression testing effortand cost. They are, General approach, Test suite minimizationand Model driven approach.

8


3578

3.3 General Approach

In this approach, the test cases are derived from the functionalrequirements of a software system. The functional requirementsare gathered based on different layered approaches [14] [15] [16].This general approach focuses on deriving functional test casesand boundary value analysis test cases independently byconsidering the database component from Figure 2 and it is notfocusing on stress and performance test cases which are the minortest cases of the given component. This approach also estimateseffort and cost.

Table 2: List of Functional Test Cases

3.4 Functional Test Cases

Table 2 shows the functional test cases which are derived fromthe functional requirements of the database component. The tableconsists of nine fields from test case identity (TC Id) to TC Priority.Each test case will have an Id, Name, Description, Pre or/and Post-condition, Input field, Expected result, Actual result, Test caseresult and Test case priority

Test cases result and priority depends on the constraints givenin Figure 4, it may depend on severity of the Bug. The priorities

9


3579

are given as Low, High and Medium that are explained in Figure4.

3.5 Boundary Value Analysis (BVA) Test Cases

In regression testing, the second major category of test cases areboundary value analysis test cases. The boundary value analysistest cases are used to test the boundaries i.e. Upper bound (U-Limit) and Lower bound (L-Limit) of each and every functionalityof a software system. All boundary analysis test cases are listen inTable 3

Figure 4: Algorithm for test case result and priority

3.6 Regression Testing Effort and CostEstimation

The regression testing effort can be estimated by using the existingequation Eq1, and the related information (activities and estimatedtime) are presented in Table 4. The total cost of the regressiontesting is given in equation Eq2. Here the terms Ctotal is the totalcost, Se is the amount paid to the software engineer per hour andEffort is taken from equation Eq1. The total amount paid to asoftware engineer is 100$ per hour [2] [17] [18].

Effort = Tenv + ((Nt ∗ Te)/60) + Trg + Tra + Tvb + Tbr (1)

10


3580

Table 4: Regressing Testing activities and estimated TimeTesting Activity Abbreviation Average Estimated Effort

Environmental setup for testing Tenv 3 hrs.Verification of fixed bugs Tvb 20 min/bug.

Test suite execution Te 1.2 min/ test caseTest report generation Trg 9 min.

Test report analysis Tra 20 minBugs reporting Tbr 18 min

No. of Test Cases per test cycle Nt Product/system dependent

Ctotal = Se ∗ Effort (2)

For the given system

Effort = 3 + ((5 ∗ 1.2) + 29 + 80 + 72)/60)

= 6.21Man−Hours

Ctotal = 100 ∗ Effort = 100 ∗ 6.21

= $621.P er −Hour.

3.7 Test Suite Minimization (CombiningFunctional and BVA Test Cases)

The second approach in this paper is test case minimizationtechnique [1] where functional test cases are combined withboundary value analysis test cases. The Figure 5 illustrates theprocess of combining test cases and reduction of a test suite.

Figure 5: Algorithm for combining test cases.

The regression test suite is reduced for the given databasecomponent from given algorithm in Figure.5 and minimized testcases are listed in Table 5. By this method, test cases arecombined and obtained satisfactory results.

11


3581

Table 5: List of Minimized Test Cases for regression testing.

4 MODEL DRIVEN APPROACH TO

TEST SUITE GENERATION

In this proposed method the use cases are used primarily forgenerating test cases. The use case model is one of the behavioralmodels of UML provided by OMG. The use cases are the best wayto develop the blue prints of a software intensive system.

12


3582

All alternative flows of a database component are tabulated inTable 6. There are three main flows which are insertion, updationand deletion. Along with these three, there are seven alternativeflows. The alternate flows are value type, U-Limit, L-Limit, insertor update, update or insert, existing value and replace value. Theten use cases which are presented in Figure 6 are sufficient torepresent the system.

Use cases are very close to the behavior of a software systemneeded by the stakeholder. So, the best way to verify the systemis from its behavior only. This approach will address all kinds ofrequirements [10] [11] which includes <Direct Flows>, <AlternateFlows>, <Includes>, <Extends>, <Generalization> and other<Relations> of a system. Figure 6. Shows the use case diagramfor given database component having all given flows.

Figure 6: Use case diagram for database component

The test cases are derived from all possible flows of the given

13


3583

database component form Table 6. By considering ten use cases,five test cases are derived and tabulated in Table 7. Here, there isno chance of missing test cases because, from use cases all functionaland BVA test cases can be generated. For example in previous twoapproaches a test case called “Replace” was not identified. Here, itis identified and combined with delete test case (Utc5).

4.1 Regression Test Effort and Cost Estimationfor Model Driven Approach

The regression testing can be estimated with the help of Eq3 foruse case method [18]. For the given database component, all testcases are very simple test cases. So, there is no need to classify usecases and actors. More over this system is not addressing activitiesrelated to actors. It is mainly concentrating on use cases. Hence,Equation Eq5 can be used in equation Eq4. In this equation, PWEis considered as Planning, Writing and Executing test cases. Inthis case planning and writing will depends upon the experience ofa tester but execution depends upon the tool used.Table 6: Different flows of Use Case model for a database component

As a part of this experimental work Junit and Selenium areconsidered. Figure 7 shows the running time of the test cases ofdatabase component. On the top left corner of the Figure 7, we canobserve that the running time of test suite is 59.999 seconds, which isless than 0.5 hours. If PWE is not provided, QA team can take thisfactor as 0.5. In Eq5 the term TEF is the total environment factorwhich is assumed as 0.5.

The total Regression testing effort = verification of fixed bugs

+Effort through use case method(3)

The effort through use case method = adjusted use cases ∗ PWE (4)

14


3584

Table 7: List of Minimized Test Cases for regression testing through usecases.

The adjusted use cases = total no. of simple use cases

∗[0.65 + (0.01 ∗ TEF)](5)

From the equations Eq3 to Eq5 the following are computed:The adjusted use cases = 10 *[0.65+ (0.01*0.50)] = 6.55The effort through use case method = 6.55*0.5 = 3.28The total Regression testing effort = [20 munits+5.17 hours]

= 3.48 Man-Hours.The total regression testing cost = $100*Effort

= $348 per-Hour

15


3585

Finally, the regression test effort is estimated for use case methodas 3.48 Man-Hours and testing cost is $348 per-Hour. This approachis providing an effective regression testing than previous approaches.Because, it takes less effort, time and cost.

Figure 7: Evaluation of test cases with selenium IDE testing tool

5 RESULTS AND DISCUSSIONS

The regression test cases are generated through different approaches.The first approach is the general way of deriving test cases from therequirements of a software system. The effort and cost of the regressiontesting is estimated by executing the test suite. The total effort requiredis 6.21 man-hours and cost is $ 621. In the second approach test casesare generated and minimized by combining functional test cases andBVA test cases. Here, the minimized test suite is executed. The time,effort & cost of re-testing are estimated. Through this approach, thetotal effort required is 6.11 man-hours and cost is $ 611. The Figure 8shows the complete validation of a database component, which is basedon Selenium Integrated Development Environment (IDE) testing tooland this tool not showing execution time, thats why Junit also used.

The third approach is the proposed approach, in which the use casemodel is used to derive the test cases, and satisfactory results are

16


3586

Figure 8: Regression testing effort estimation

Figure 9: Regression Testing Costt estimation

17


3587

obtained. Through this approach, the total effort required is 3.48man-hours and cost of the regression testing is $ 348. Figure 9 showsthe effort required for performing regression testing through threedifferent approaches and Figure 10 shows the regressiont testing costtestimation. The three test suites are General test suite, Minimized testsuite and Model driven test suite. From Figure 9 and Figure 10, it canbe observed that the model driven test suite approach provides betterresults than other approaches, that is cost and effort are reduced.

6 CONCLUSIONS AND FUTURE

DIRECTIONS

This paper presented a holistic approach to derive test cases frombehavioral models of the system. The derived test cases can beexecuted manually or automatically. Through this approach it ispossible detect defects in early stage, it generates the minimized testsuit with maximum test coverage. It gives less effort & cost. Hence, itleads to low project maintenance cost

Further research includes usage of MBT techniques for legacysoftware maintenance where no design documents are available. We areplanning to work with an experimental setup through COTScomponents construction.

References

[1] Barry Boehm, “Software Management” 7th Edition, August 2006,2006, Wiley-IEEE Computer Society Press, ISBN: 978-0-471-77562-1.

[2] Prof. A. Ananda Rao et al,“An Approach to Cost EffectiveRegression Testing in Black-Box Testing Environment”, IJCSIInternational Journal of Computer Science Issues, Vol. 8, Issue 3,No. 1, ISSN (Online): 1694-0814,May 2011.

[3] L. Erlikh, “Leveraging legacy system dollars for e-business”, IEEE,IT Professional, Volume: 2, Issue: 3, May/Jun 2000.

[4] Peter A. Lindsay et al, “Automation of Test Case Generation fromBehavior Tree Requirements Models”, IEEE-24th AustralasianSoftware Engineering Conference, pp: 118-127, ISBN: 1530-0803/15, DOI: 10.1109/ASWEC.2015.23, IEEE 2015.

18


3588

[5] Sebastian Ulewicz et al, “Software Change in Factory Automation:Towards Automatic Change Based Regression Testing” IEEE pp:2617-2623, ISBN: 978-1-4799-5/14, IEEE 2014.

[6] Philipp Caliebe et al, “Dependency-Based Test Case Selectionand Prioritization in Embedded Systems”, IEEE-FifthInternational Conference on Software Testing, Verificationand Validation, pp: 731-735, ISBN: 978-0-7695-4670-4/12, DOI:10.1109/ICST.2012.164. IEEE 2012.

[7] Erik Rogstad et al, “Clustering Deviations for BlackBox Regression Testing of Database Applications”, IEEETransactions on Reliability, pp:1-15, ISBN:0018-9529, DOI:10.1109/TR.2015.2437840, May 2015.

[8] Mary Jean, A. Oeso, “Retesting Software During Development andMaintenance”, Workshop on Testing 2008, IEEE, ISBN: 978-1-4244-2655-3, pp: 99-108, Feb 2008.

[9] Ana M. Fernndez-Sez, “On the use of UML documentationin software maintenance: Results from a survey in industry”,MODELS-2015 ACM/IEEE 18th International Conference onElectronic, ISBN: 978-1-4673-7231-2, 2015.

[10] Susanne Rosch et al, “Review of Model-Based Testing Approachesin Production Automation and Adjacent DomainsCurrentChallenges and Research Gaps” Journal of Software Engineeringand Applications, 2015, 8, 499-519

[11] Jim Heumann, “Generating Test Cases from Use Cases”Rational edge, Copyright Rational Software 2001, Privacy/LegalInformation, 2001.

[12] Adtha Lawanna, “test case design based technique for theimprovement of test case selection in software maintenance”, IEEEXplore: Electronic ISBN: 978-4-907764-50-0, 21 November 2016

[13] Ian Sommerville, “Software Engineering”, Eighth Edition, Pearsonpublication. ISBN: 978-81-317-2461- 3, LPE-India. PP: 477-477.

[14] A.AnandaRao “Five Layered model for identification of softwareperformance requirements”, International Journal of SoftwareEngineering and Applications(IJSEA),Volume 3,No.5, pp 47-61,September 2012.

19


3589

[15] A.AnandaRao, “Four Layered Approach to Non-FunctionalRequirements Analysis”, International Journal of ComputerScience and Issues (IJCSI Volume 8, Issue 6, pp 371-379, November2011.

[16] A.AnandaRao, “Layered Approach for performance requirementselicitation” International Journal of Electrical Electronics andComputer Systems(IJEECS),Volume 9,Issue1, pp 568-575, July2012.

[17] Prabhakar K., “Cost Effective Model Based Regression Testing”,“IAENG- World Congress on Engineering”, Vol. I-WCE 2017,pp241-246, July 5-7, 2017, London, U.K., 2017.

[18] Suresh Nageswaran, “Test Effort Estimation Using Use CasePoints”, Copyright(c) 2001, Cognizant Technology Solutions,Quality Week 2001, San Francisco, California, USA, June 2001.

20


3590

3591

3592

Documents

A MODEL DRIVEN APPROACH TO REGRESSION TESTING OF … · 2018-09-29 · component [2]. The regression testing improves the con dence in some other aspects like software patches, traditional