ClicQA helped an E-commerce Application to Improve its Performance

Case Study

About ClientClient is an E-commerce company, providing user friendly online shopping experience for their customers across the globe. Client has developed web and mobile applications to increase their customer base and market reach, and the developed applications go-live in the impending Holiday Season.

Requirement Scalability: Application must be able to handle the load of 10,000 concurrent users from web tra�c and 5,000 concurrent users from mobile tra�c.

E�ciency: Customer expectation includes the highly e�cient application with minimal page navigation response time.

Fail Over: Application should be able to behave as expected under speci�ed load and should have the Fail Over control.

Problem Statement Client has observed a couple of performance outages due to load at peak time and wanted to evaluate their web and mobile E-commerce application performance level ASAP, so that client can optimize Web Service/ APIs or upgrade their infrastructure. Alongside load testing the web application with 10,000 concurrent users and mobile app with 5,000 concurrent users, client wanted to �nd the performance bottlenecks as quick as possible.

Solution O�eredWith an understanding of the client pro�le, problem statement and requirement, our team has taken the �rst step – choosing the right tool with their extensive exposure to several commercial and open source performance testing tools. Our team has shortlisted Jmeter, an open source Load Testing tool that o�ers a powerful, realistic load tests for thousands of users running business scenarios across a broad range of enterprise application environments.

Clic

QA

Con�

dent

ial

Continuous Quality Assurance

Case Study

We have used Jmeter to achieve following aspects of load testing:

¤ Breaking point of application

¤ Stress/Scalability/Capacity Testing

¤ Spike and Synchronization Testing

¤ Soak, Endurance Test and Reliability Testing

¤ Performance Tuning and Diagnostics

Our Approach towards load testing using Jmeter:

¤ Setting up a test environment that matches with the expected production environment

¤ Creating Test plan using Jmeter GUI using all possible controls to make the identi�ed scenarios more realistic

¤ Scheduling the test execution at distinct times, including peak hours

¤ Analysing the metrics obtained from the test execution to prepare a load test report

¤ Re-executing the test and verifying the improvement in performance after re-work on the bottlenecks identi�ed

Clic

QA

Con�

dent

ial

.apk .apk

ClicQA helped an E-commerce Application to Improve its Performance

Continuous Quality Assurance

Case Study

Clic

QA

Con�

dent

ial

Step-1

Step-2

Step-3

Step-4

Step-5

Step-6

Plan the Tests

Create Virtual Users Script

Create the Scenarios

Run the Scenarios

Monitor the Scenarios

Analyze the Test Results and prepare the Test Reprot

ClicQA helped an E-commerce Application to Improve its Performance

Continuous Quality Assurance

Outcome

¤ Early detection of bottlenecks in the performance and reduced risk of page load delays caused by performance issues.

¤ Client E-commerce application was able to handle 10,000 concurrent users from web tra�c and 5,000 concurrent users from mobile tra�c.

¤ As a part of bottleneck analysis, we have used “new relic” - an APM tool to monitor JVM stats. We have found out that for few customers, there were many loops on the sql calls as each customer had multiple purchases and this increased the response times and we have suggested DB admins to correct the queries or use cache.

Performance Testing Process Flow:

ClicQA helped an E-commerce Application to Improve its Performance

Case Study

¤ We have also found that memory was leaking out for longer duration tests. We have maintained di�erent JVM memory con�gurations after thorough testing. After client had implemented suggested changes on speci�c pages, we have observed overall 150% improvement on the application.

¤ Client was able to comprehend the performance of the server and accordingly client had identi�ed the production server/ infrastructure requirements and was able to determine the monitoring thresholds.

¤ Client has enhanced the overall system and was able to meet the performance benchmark as client was expecting more users in the coming days.

ConclusionOur team’s expertise and our diligence towards client’s business need & requirement has given client con�dence to go-live in the Holiday Season. Even before the season started, client’s application was ready to perform as it should be at the expected load and stress levels.

Clic

QA

Con�

dent

ial

Continuous Quality Assurance

Outcome

¤ Early detection of bottlenecks in the performance and reduced risk of page load delays caused by performance issues.

¤ Client E-commerce application was able to handle 10,000 concurrent users from web tra�c and 5,000 concurrent users from mobile tra�c.

¤ As a part of bottleneck analysis, we have used “new relic” - a APM tool to monitor JVM stats. We have found out that for few customers, there were many loops on the sql calls as each customer had multiple purchases and this increased the response times and we have suggested DB admins to correct the queries or use cache.

¤ We have also found that memory was leaking out for longer duration tests. We have maintained di�erent JVM memory con�gurations after thorough testing. After client had implemented suggested changes on speci�c pages, we have observed overall 150% improvement on the application.

¤ Client was able to comprehend the performance of the server and accordingly client had identi�ed the production server/ infrastructure requirements and was able to determine the monitoring thresholds.

¤ Client has enhanced the overall system and was able to meet the performance benchmark as client was expecting more users in the coming days.

ConclusionOur team’s expertise and our diligence towards client’s business need & requirement has given client con�dence to go-live in the Holiday Season. Even before the season started, client’s application was ready to perform as it should be at the expected load and stress levels.