Upload
cigniti-technologies-ltd
View
40
Download
0
Embed Size (px)
Citation preview
Are you Set for Black Friday?
Rajesh Sarangpani Asst. Vice President ,Cigniti Technologies Cigniti
Dan Boutin Senior Product Evangelist, SOASTA
June, 2015 2 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Agenda
Housekeeping & Introductions
Best Practices & Lessons Learned with SOASTA CloudTest
Customer Case Study with Cigniti
Questions
Wrap-Up
June, 2015 3 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Retail E-Commerce Requirements
E-Commerce Testing Strategy in place Understand how the site/application is used
Rich Content (Rich Relevance) Third-party content Promotions! Social Media Coupons CDN
Require a good understanding of their users & customers
June, 2015 4 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Retail E-Commerce Best Practices
Test in Production
Know Your Users
5 March 3, 2015 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
SOASTA Global Test Cloud Worldwide Locations, SOASTA Centers of Excellence (COE)
17 Providers
54 Locations
800K Servers
500M Virtual Consumers * Additional Locations on demand
June, 2015 6 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Automated Grid Provisioning
Your environment must be flexible & scalable
June, 2015 7 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
You need a Kill Switch – No Fear Factor!
June, 2015 8 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Eliminate the Fear Factor
“We don’t test in production.”
June, 2015 9 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
You need drill down by Load Generation Location
June, 2015 10 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Detailed Error Analysis
You need detailed error analysis - LIVE
June, 2015 11 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Multi-Test-Run Comparison
Compare results of a LIVE test with previous test executions
You need to know: Are we better than last time?
June, 2015 12 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Detailed Transaction, Page, and URL Analysis
• Detailed Transaction and Page Analysis of Web and Mobile Load Tests
• Detailed URL Analysis of Web and Mobile Load Tests
You need web & mobile analysis
June, 2015 13 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Detailed Page Component Breakdown
Your analytics should have visibility into your CDN
June, 2015 14 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Reflects growth in cloud hours – Amazon only! (17 other providers!)
7 Year Growth of cloud testing: SOASTA
….goodbye Fear Factor…
CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015
Fix Your process! No more outdated test creation
CONFIDENTIAL – Not for Distribution © 2015 SOASTA. All rights reserved. January 13, 2015
Analyze the most common session paths of real users
June, 2015 17 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Possess a deep Real User Analytics capability Then take your mobile data analysis even further & deeper
Performance Cluster Analysis Performance Conversion Path Analysis
Dimension Co-Occurrence Analysis Conversion Rate by Load time for View Groups Analysis
June, 2015 19 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Cigniti helping its retail clients achieve business objectives
June, 2015 20 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Cigniti improves customer experience of one of America’s largest apparel retailer
Client is one of America’s largest Apparel and Home Furnishing retailers with over 1000+ stores across all the states and has a huge online presence with the ability to ship goods across the world
• Current Business model challenges (trying to evaluate its core)
• Attracting new customers v win over existing customers to buy more
• Significant erosion of online revenues
• Transformation from single channel to multi channel
• Coping up with outdated technology
• Poor Operational efficiencies
June, 2015 21 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Client’s Testing Challenges
Business – IT collaboration Lock-in
Lack of visibility and transparency across the teams
Lack of comprehensive strategy to address holiday period peak demands
Availability and Reliability of current infrastructure was way behind industry peers
Testing as an afterthought
Specifically non functional testing was planned very late in the life cycle
Old testing methods, Insufficient test coverage and non availability of context specific tools
Testing across Channels, devices, understanding real user experience were some of the key gaps
Struggling to stand up test bed due to high capex involved
Testing was also conducted under lab conditions
June, 2015 22 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Solution Summary – How did we address the challenges!
Moved from reactive to proactive approach– Implemented proven best practices
Realistic test coverage -Ensured 100% coverage by implementing log analytics
Improved operational efficiencies by moving away from on-premise to cloud based performance test beds and tools
Implemented focused testing cycles that carried out performance testing on production
Continuous Improvement : Proactive performance monitoring and feedback control
Created a framework to involve business teams throughout the lifecycle from planning to reporting
June, 2015 23 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Plan: Visual Workload Model
Main business flows
% User mix across the flows
Average session time
Average/Peak concurrent
sessions/ hour
# of transactions/min (e.g.
Orders, bookings etc)
Geo distribution
Channel distribution
CDN versus non CDN traffic
June, 2015 24 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Analyse & Design: Business KPI
Business KPI
Number of Orders
Per minute
Gross Margin
Total orders
per day,
week , month
New visitors
New customers
Abandonment
Metrics
(Cart, Checkout,
Home page)
Average
order value
June, 2015 25 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Execute & Report: Actionable & Intuitive Business Reports
16.89%
25.03%
21.41%
13.97%
6.88%
4.28% 2.84% 2.26% 1.73% 1.29% 1.26%
0.61% 0.56% 0.24% 0.22% 0.12% 0.07% 0.02% 0.05% 0.05% 0.07% 0.05% 0.05% 0.05%
Time in Seconds
How long most of your users wait for Home Page?
When 1000 users are on your application, only 170 users are going to like it. 250 users might want to search for other options. 214 users are not going to come back. 366 users don't even see what is in your website.
June, 2015 26 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
IT reports: Detailed Drill down to fix the root cause
June, 2015 27 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
Improved Availability and Reliability of the architecture
5 X Improvement in Scalability
Reduced the response time across geo by 50%
Proactive performance monitoring solutions to aid timely action
Outcomes
June, 2015 28 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
So what value we can add ?
Smart Tools™ Test
Accelerators
Tool Partnerships
and Alliances Innovation
and IP
Enterprise
Frameworks Process Asset
Library
Test
Automation
Compatibili
ty Testing Globalization
Testing
Security
Testing
Regression
Testing
Performance
Testing
Functional
Testing
Test Advisory
Services
Test Center of
Excellence
Usability
Testing
Testing in
Agile
DevOps QA
Continuous
Testing Big Data &
Analytics
SOA & Web
Services
Core Services
Quality Engineering Services
Next Gen Services
Digital QA
Social Media
Testing
Mobile
Testing
Cloud
Migration
Enablers
June, 2015 29 © 2015 SOASTA. All rights reserved. © 2015 Cigniti . All rights reserved.
CIO
SRA
AAT
TO
Independent Testing
IP Led Testing
Test Cloud Test Accelerators
Certified for Services
Agility
» Niche player
» Unbiased
» Industry Leader IV & V
» Certified & Passionate Career Testers
» IP-led testing services
» Deployed at no charge
» Accelerates events across
Test Life Cycle
» Pro Agile
» First organization to offer testing
services on cloud
» Flexible pay as you go model
» No upfront CAPEX
» Test accelerators at no charge
» ERP test accelerator for SAP and Oracle
» Test accelerator to support web 2.0 and
POS
» Reusable process asset
library
» World’s first CMMI-SVC v 1.3
» ISO 27001 Compliant
Delivery Centre
» 100% focus and specialization in
testing
» Flexible engagement models
» Agile in functioning
» Executive team involvement
Collaborate
Measure
Govern
Leverage
Innovate
Ensure
Global
Engageme
nt Model
What makes us different ?