Abstract - Exhaustive product evolution and testing is required to ensure the quality of product. Regression testing is crucial to ensure software excellence. Regression test cases are applied to assure that new or adapted features do not relapse the existing features. As innovative features are included, new test cases are generated to assess the new functionality, and then included in the existing pool of test cases, thus escalating the cost and the time required in performing regression test and this unswervingly impacts the release, laid plan and the quality of the product. Hence there is a need to select minimal test cases that will test all the functionalities of the engineered product and it must rigorously test the functionalities that have high risk exposure. Test Suite- Test Case Refinement Technique will reduce regression test case pool size, reduce regression testing time, cost & effort and also ensure the quality of the engineered product. This technique is a regression test case optimization technique that is a hybrid of Test Case Minimization based on specifications and Test Case Prioritization based on risk exposure. This approach will facilitate achievement of quality product with decreased regression testing time and cost yet uncover same amount of errors as the original test cases.

Keywords - Test Case Optimization, Regression Test, Test Suite Prioritization, Test Suite Minimization, Risk Based Prioritization, Specification Based Selection.

I. INTRODUCTION Products are tested to investigate its quality standards. Thus testing is fundamental in improving and ensuring the quality of product. When system, component, or a group of related units are added to the existing functionalities, they need to be tested to check that the quality of product is not hampered. Hence regression testing plays a vital role to ensure that the quality of the product is retained after versioning or modification. Regression test [1] is a black box testing technique [2] used to test product (system, component, or a group of related units) to guarantee that the modification is working precisely as intended. Test Suite Minimization is a technique used in regression test to select test case in a test suite. It strives to achieve a representative set of the test suites which will have minimum no of test cases that cover all the functionalities and ensures the quality of the product [3].

Specifications are most important aspect of a program. Hence the test performed must be such that it covers all the specifications of a given program.

Specification based Minimization technique selects only minimal set of test cases in test suite that will fulfil all the specification related to that scenario.

Test Suite Prioritization [4] is a technique used in regression test to prioritize test case in a test suite and prioritize the test suites as well. It ranks/grades the test suites so as to increase the test suites fault detection rate [5]. Thorough testing is required for the parts of software that has high risk. Hence test cases that cover such high risk areas are required to be selected and given the high priority. Regression test assures the stability and reliability of the software, so risk based prioritization [6] will help achieve adequate confidence in software quality. Risk based Prioritization is a specification based regression test selection [7].

II. TEST SUITE REFINEMENT TECHNIQUE

A. Proposed Technique

The below shown fig. 1. describes the proposed technique in block diagram. Test Suite Refinement Technique [8] accepts all the test suites- test cases available for the product to be tested. The output of this technique is the reduced set of test suites- test cases i.e. representative set that is made available for regression test.

Fig. 1. Block Diagram of Test Suite Refinement Technique

Test Suite Refinement Technique is a hybrid of Test Suite Minimization based on Specification and Test Suite Prioritization based on Risk Exposure. Hence in fig 2. the intermediate refinement process is indicated, where in the refinement process accepts only risk exposure based

Optimization of Test Suite- Test Case in Regression Test

Ahlam S.A.Ansari1, Prof. Kailas K. Devadkar2, Dr. Prachi Gharpure3 1,3 Department of Computer Engineering, Sardar Patel Institute of Technology, University Of Mumbai, Mumbai, India 2 Department of Information Technology, Sardar Patel Institute of Technology, University Of Mumbai, Mumbai, India

1ansariahlam@ymail.com, 2kailas_devadkar@spit.ac.in, 3prachi_gharpure@spit.ac.in

978-1-4799-1597-2/13/$31.00 ©2013 IEEE

prioritized and selected test cases; and specification based minimized and selected test cases.

Fig. 2. Block Diagram of Intermediate state in Test Suite Refinement

The detailed process of the refinement and test suite - test case selection is explained with help of algorithm in the next section. B. Algorithm Major goals of regression testing are to assure system stability and reliability [6]. Our technique focuses on test cases which test the risky areas as well as is able to test all the functionalities of the engineered product. Hence, it can help us achieve adequate confidence in software quality. Our test selection method consists of two parts: test case selection based on risk exposure and test case minimization based on specifications. Part I: Test Case Prioritization and selection based on risk exposure Step 1: Calculate the risk exposure for individual test case [6] [9]. RE =P(f) X C(f) (1) Where RE is the risk exposure, P(f) is the probability of failure and C(f) is the cost of failure Both P(f) and C(f) are scaled 1 to 5; where 1 is trivial and 5 is intolerable. Risk exposure is also converted to 1 to 5 scale. Step 2: Prioritize the test cases in descending order of it risk exposure. Step 3: Select only those test cases that have the risk exposure as intolerable, substantial and moderate. Step 4: Add the selected test cases into the regression test pool. Part II: Test Case Minimization based on specification Step 1: Check if the existing test case in the regression test pool test all the specifications of the product [10]. If it does then no need to perform this step. Step 2: If test cases in regression pool do not cover all the functionalities then select those test cases that test the uncovered functionalities. Step 3: Add the selected test cases into the regression test pool.

IV. RESULTS

A. Output The result is divided into three parts. The fig. 3. shows the result of test suite minimization based on specification.

Fig. 3. Result of Test Suite Minimization Technique

The output shown in fig. 4. is the output of test suite prioritization based on risk exposure. Here for a specific project all the test cases that have risk exposure intolerable, substantial and moderate are only selected.

Fig. 4. Result of Test Suite Prioritization Technique

As shown in fig. 5. the test suite refinement technique results. The test cases that have high risk exposure and covers all the functionalities are selected as for regression test.

2013 IEEE International Conference on Computational Intelligence and Computing Research

Fig. 5. Result of Test Suite Refinement Technique

B. Comparison Results: The result based on comparison between Minimization Technique, Prioritization Technique and Proposed Technique based on different criteria are presented below.

Fig. 6. Comparison based on size of test cases reduction.

As Proposed Technique takes in account all functionalities and the risk exposure criteria the number of test cases are more in it and hence the effort , cost and time required in testing is more then test suite prioritization and test suite minimization. But the percentage of fault detection of proposed technique is same as the orignal set of test cases. Even significant amount of reduction in test case pool size is achieved using the proposed technique.

Fig. 7. Comparison based on time taken by test cases.

The fig. 6. shows the comparison between test suite prioritization and test suite minimization and proposed technique based on size of representative set generated. The fig. 7. shows the comparison between test suite prioritization and test suite minimization and proposed technique based on time taken by test cases.

Fig. 8. Comparison based on effort taken by test cases.

The fig. 8. shows the comparison between test suite prioritization and test suite minimization and proposed technique based on effort taken by test cases.

Fig. 9. Comparison based on cost of test cases.

The fig. 9. shows the comparison between test suite prioritization and test suite minimization and proposed technique based on cost of the test cases.

2013 IEEE International Conference on Computational Intelligence and Computing Research

Fig. 10. Comparison based on fault detection by test cases.

The fig. 10. shows the comparison between test suite prioritization and test suite minimization and proposed technique based fault detection.

V. CONCLUSION The product versioning leads to addition of more features and will have more number of test suites and test cases generated to test the quality of the versioned product. Thus by selecting only those test cases that will exercise all the functionalities of the product as well as risk exposure of test cases in account will ensure the quality of the product as well as will reduce the size of test case - test suite pool required to perform regression test. And as the test cases will be reduced, the cost of testing the product as well as time required to test the product is decreased. Through results and comparisons we can conclude that test suite refinement technique is simple and effective and achieves significant test suite- test case size reduction and ensures the quality of the product. It also reduces the cost and time of regression testing and thereby reducing the cost of maintenance activity and maintenance effort. Also provides ample number of test cases to perform regression testing to ensure the reliability of the modified software.

ACKNOWLEDGMENT Our thanks to Sardar Patel Institute of Technology, Department of Computer Engineering and Department of Information Technology, for giving us the initiative to do constructive work. We also thank anonymous reviewers for their constructive suggestions.

REFERENCES [1] Rothermel, G., Untch, R.H., Chengyun Chu; Harrold, M.J.,

“Prioritizing test cases for regression testing,” Software Engineering, IEEE Transactions on, vol.27, no.10, pp.929,948, Oct 2001.

[2] Ruchika Malhotra, Arvinder Kaur, Yogesh Singh, “A Regression Test Selection and Prioritization Technique,” 2010 KIPS (ISSN 1976-913X) .

[3] Srinivas Yeluripaty PMP, Prabhu Meruga, “Managing Large Regression Test Suites,” at ISQT Conference, Hyderabad, India 25/Jul/2008.

[4] Carlson, R. , Hyunsook Do, Denton, A., “A clustering approach to improving test case prioritization: An industrial case study,” Software Maintenance (ICSM) , 2011 27th IEEE International Conference on , vol., no., pp.382,391, 25-30 Sept. 2011. [5] Md. Imrul Kayes, “Test Case Prioritization for Regression Testing Based on Fault Dependency,” at IEEE international World Congress, 2011. [6] Yanping Chen, Robert L. Probert, “A Risk-based Regression

Test Selection Strategy, ” in Chillarege Press, 2003. [7] Mark Broadie, Yiping Du, “Risk Estimation Via Weighted

Regression,” at Simulation Conference, 2011.

[8] Ahlam Shakeel Ahmed Ansari, Prof. K. K. Devadkar, Dr. Prachi Gharpure, “An Optimized Technique For Test Suite Refinement In Regression Test,” in International Journal of Advancements in Research & Technology, Volume 2, Issue 7, July-2013. [9] Souza E., Gusmao C., Alves K., Venancio, J., Melo, R., “Measurement and control for risk-based test cases and activities,” Test Workshop, 2009. LATW '09. 10th Latin American , vol. no. pp.1,6, 2-5 March 2009. [10] Gorthi R.P., Pasala A., Chanduka K.K.P., Leong B., “Specification-Based Approach to Select Regression Test Suite to Validate Changed Software,” Software Engineering Conference, 2008. APSEC '08. 15th Asia-Pacific , vol. no., pp.153,160, 3-5 Dec. 2008.

2013 IEEE International Conference on Computational Intelligence and Computing Research