Speakers...• Impact Analyzer – Code, component, functional and business processes • MBT platform • Script refactoring techniques • Process accelerators • Script migration

Speakers

Diego Lo Giudice (Guest Speaker)Vice President, Principal Analyst Serving Application

Development & Delivery Professionals, Forrester

Kalyana Rao KondaPresident ,

Cigniti Technologies

© 2019 FORRESTER. REPRODUCTION PROHIBITED. Cign i t i W eb inar - Acce le ra te D ig i ta l Trans fo rmat i on wi th Ag i l e & DevOps Cont i nuous Tes t i ng

Continuous Testing Trends and Practices

Diego Lo Giudice, VP & Principal Analyst serving AD&D

July 2019

2© 2019 Forrester Research, Inc. Reproduction Prohibited Cign i t i W eb inar - Acce le ra te D ig i ta l Trans fo rmat i on wi th Ag i l e & DevOps Cont i nuous Tes t i ng

WhyContinuous

Testing ?

3© 2019 FORRESTER. REPRODUCTION PROHIBITED. Cign i t i W eb inar - Acce le ra te D ig i ta l Trans fo rmat i on wi th Ag i l e & DevOps Cont i nuous Tes t i ng

Digital Transformation Is The Goal For Every Industry

Base: 354 business and technology leadersSource: Forrester’s Q1 2018 Digital Process Automation Survey


But, most digital transformation initiatives are struggling

Source: “The Sorry State Of Digital Transformation In 2018” Forrester report.

Base: 1,559 business and technology decision makers


8%9%

20%22%

25%26%26%

28%30%31%31%31%

33%44%

51%

None of the above

Other (please specify)

Lack of automation tools and/or infrastructure (e.g., Agile PM tools, automated builds, test…

Scaling Agile at the enterprise by replicating success throughout the organization to more teams

Connecting existing product portfolio management (PPM) with Agile projects

Lack of Agile testing skills (e.g., unit testing, test automation, testing in sprints, exploratory…

Lack of dedicated cross functional teams

Lack of Agile skills in downstream or DevOps (XP, development, testing, continuous integration,…

Lack of Agile executive leadership

Incompatibility of our existing funding process to support Agile projects

Lack of Agile skills in project management practices, backlog grooming, planning, requirements,…

Lack of Product Owners commitment from the business

Incompatibility of our existing budgeting process to support Agile projects

Lack of skilled Product Owners from the business

People behavioral change (true buy-in into lean and agile values and principles)

Which of the following are barriers to your organization’s Agile adoption? (Check all that apply) Culture

Base: 232 professionals with knowledge of their firm’s Agile practicesSource: Forrester’s Q3 2017 Global Agile Software Online Survey

Top challenges in scaling Agile and DevOps: Culture, Buzz PO, Budgeting/Funding

SkillsSkills

Automation

Buzz POBudgeting


Development Velocity Is Stuck – App Releases

Base (2018): 3,228 DevelopersBase: 546-719 Developers who work for a software company, for internal IT, or in technology servicesSource: Forrester Business Technographics Developer Survey, 2014-2018

0%

5%

10%

15%

20%

25%

30%

We average onerelease a year (orless frequently)

We average tworeleases a year

We average onerelease per quarter

We average onerelease a month

We average onerelease a week

We average a releasea day

We release manytimes per day

"How often does your team or teams release applications?"

2018 2017 2016 2015 2014

One release a month


Testing Is Definitively Part Of The Problem

Base: 232 tech professionals at firms that have implemented or are implementing Agile. Source: Forrester’s Q3 2017 Global Agile Software Application Development Online Survey

20%

22%

25%

26%

26%

28%

30%

31%

Lack of automation tools and/or infrastructure

Scaling Agile at the enterprise level

Connecting Agile with existing product portfolio management(PPM)

Lack of dedicated cross functional teams

Lack of Agile testing skills

Lack of Agile skills in downstream or DevOps

Lack of Agile executive leadership

Lack of Agile skills in project management practices

Which of the following are barriers to your organization’s Agile adoption? (Check all that apply) (not all answers shown)

a


Developers Still Don’t Spend Enough Time On Testing


Test automation plateaus too, and manual testingresists to go away


“…if our technology products do not operate as intended, our business could be materially and adversely affected.”

“Our business success and results of operations depend in part on effective information technology systems and on continuing to develop and implement improvements in technology.”

Source: The 2018 “Risk Factors” section of 10K SEC filings from the respective companies.

“If the technology-based systems that give our customers the ability to shop with us online do not function effectively, our operating results, as well as our ability to grow our eCommerce business globally, could be materially adversely affected.”

Poor Quality Software Puts The Business At Risk

11© 2019 Forrester Research, Inc. Reproduction Prohibited Cign i t i W eb inar - Acce le ra te D ig i ta l Trans fo rmat i on wi th Ag i l e & DevOps Cont i nuous Tes t i ng

How can we address these challenges?


From Digital Business Excellence To SW Delivery Execution Excellence….Your new mantra…!Business Must Master Digital To Stay In Business

Software Powers The Digital Experience

Software Execution Excellence Equals Business Execution Excellence


Software execution excellence needs enhanced CI/CD practices…

Source: Master Your SDLC For Modern Application Delivery

..and BTW, testing is part of it…


Continuous Testing done right


Sprint definition of "done" includes your sprint testing goals (Done, test data strategy, automation, cntrl. Gates…)Define Test Scenarios using a DSL (e.g: Gherkin)Include Performance Testing in Acceptance CriteriaFeature toggles, AB Multivariate…

Prioritize bugs on backlog

PO, Testers and Developers


Design automation of functionaltests (and test data).Build automation of functionaltests (UI, API).Build automation for non functional (SVT, SL PT, Security)Scriptless, scripting, coding

Exploratory testing (manual)

Technical Testers, Dev-testers, Business testers


Version Control test assetswith source codeIntegrate in CI/CD to automate all test execution(unit, functional, non-functional)


Automated integrationtesting (Service Virtualization)Automated governance for release automation or feature deployment


Production monitoring feeds your testingMachine Learning predictswhat to test nextIntegrated AB testing in SDLC helps improve CX


Source: See The Forrester report “Shift Performance Testing Left To Streamline App Delivery”.

And it all has to happen in sprints…


Next Trends: We Put More Intelligence In CT !

Visual Testing (ubiquitous )

Test Automation & Generation

Test LifecycleOptimization

Unit test code generation

API Test generation

Defect management (clustering, MTTR

reduction..)

Production

User Experience

Monitoring

Role of IP in Enabling Continuous Testing

www.cigniti.com | Unsolicited Distribution is Restricted. Copyright © 2019 -20, Cigniti Technologies 2Cign i t i W eb inar - Acce le ra te D ig i ta l Trans fo rmat i on wi th Ag i l e & DevOps Cont i nuous Tes t i ng

Key Benefits

Test asset migration platforms - (QTP/UFT to LeanFT), (QTP/UFT to Selenium), (QTP/UFT to Test Complete) etc.

Quality Engineering Platform and Dashboard with predict capabilitiesAnalytics from descriptive, diagnostic, predictive and prescriptive viewpoints

Test Accelerators for Retail, Hospitality & Automation Accelerators for Kiosk, POS, COTS, ERP and Digital Applications

Process accelerator platform (CLASS, ADePT, THRIVE, VERIFY and BEST)to transform existing end-to-end SDLC practices

Captures end users feedback from app stores and social channels to provide analysis on end user Engagement, Emotion, Experience

Automatically generates software test artifacts using system requirements leveraging patented algorithm.

BlueSwan - Cigniti's Next Gen Software Testing Platform

Predictable business outcomes &

Collaboration

Accelerates time to value Reduce tool cost Process

Transformation

Increase Test coverage & reduced

cost

Improve end user experience


BlueSwan-IP Value tree

BLUESWAN

Process

Standardized

Efficiency

Improved Productivity

Effectiveness

Reduce wastage and bottlenecks

Product Quality

Higher Quality

Cost Impact

Reduced R & D Spend

Business Impact

Competitiveness1 – CESA2 – Cesta3 – Praxia4 – Prudentia5 – Velocita6 - Verita

Mitigate risks – 1,3,6

Higher test case yield – 4

Early Identification of Risks – 1,3,6

Testable requirements – 3,4

Improved Predictability – 1,6Improved Customer Satisfaction - 1

Increased Test coverage – 3,4,5

Reduced Outages – 1,3

Lower Operating Costs – 1,2,3,4,5,6

Increased Revenue – 1,3,6

Enhances Brand reputation - 1

Accelerates time to value – 1,2,,3,4,5,6

Higher Maintainability – 4,5

Reduced TCO – 2,3,4,5

Higher Reusability – 4,5

Higher ROI – 1,2,3,4,5,6

Reduction in framework setup time – 2,5

Higher test execution throughput – 3

Lower Test design time – 3,4

Shorten cycle times – 2,3,4,5

Less Production Issues – 1,3,4


Business case for Verita

Test Script Authoring Productivity

Number of Test Scripts Passed/Failed per Cycle

Test Automation Technical Debt

Real time insights –Multiple stakeholders spending less time on pulling together data from disparate sources and generating reports

Accuracy – Make informed decisions on accurate information made available real time through automated integration to life cycle tools

Business advantage through new insights:Reduce costs:Defect prevention, Defect detection, Defect resolution, Regression testing costs

Improve time to Market: Test cycle duration, Defect aging, Development velocity, Early defect detection

Increase revenues and client satisfaction:Reduction in post production defect density, Ability to release new features quickly

Employee satisfaction –Stakeholders focus on finding answers to relevant questions and not on wrangling data and time consuming report creation

Evolving enterprise tool ecosystem – Open architecture for easy integration


Quality Engineering Platform and Dashboard

Verita is role-based system, catering to the needs of diverse stakeholders inside the enterprise with a holistic view of

Lead

Tim

e

TCO

Q

Qua

lity

Rel

ease

R

eadi

ness

Verita comes with the ability to analyze data and provide data from the following viewpoints:

Descriptive Diagnostic Predictive Prescriptive

Improves Predictability

Drives Business Outcomes

AidesTransformation

Promotes Collaboration

Quality Metrics LivePredictive Analysis Environmental

Details

Features

Lines of code : Quality Test Case Effectiveness

Devops Dashboard

CIO

Release Manager

Scrum Master

Performance Engineer

Security Engineer

Integration with current tools

Data Sources

Role Based


Velocita : Enterprise-wide continuous testing factoryTechnology Hubs

Infra Automation Lab

Containers

Infra as Code best practices

Automated Environments validation techniques

Checklists for Environment readiness

Cloud Digital Studio

Quality Engineering Lab

BlueSwanTM -VeritaDescriptive Diagnostic Predictive

Financial profitability metrics

Efficiency & effectiveness metrics

Real Metrics Monitoring (integrated with SLA & KPIs)

Automation progression score cards

100% 60%

APIs

75% 80%

Regression Unit

NFT

BPT

UIStory Level

80%

40%

Cor

e Se

rvic

e As

sets

Automation Performance Security• Impact Analyzer – Code,

component, functional and business processes

• MBT platform

• Script refactoring techniques

• Process accelerators

• Script migration tool

• CI Framework

• Code optimization guidelines

• Capacity Planning framework

• Load Generation enabled by cloud platforms

• Root cause analysis framework

• Predictive analytics

• User experience evaluation

• Orchestrated Dev-Sec-Ops platform

• Predictive risk analysis

• Automated enterprise security audit framework

• Secure coding guidelines and best practices

• Integrated Dashboard

Automation Process Framework Global Object Repositories

Test Suite Orchestrat

Global APIs

Log Manager

Risk Scoring Mechanism

In-built Reporting

Pre-built process assets for each phase of STLC

Execution Grid Test Data Analysis

Platform

NFR Template

Workload Modeling solution

Test Engineering Platform (TEP)Hub

Assets

Digital Core Data and Data Science

Voice Recognition & IVR

BDD Automation Framework UX NFR Pattern Digital Channel Preference

Metrix

Thread Scripting Techniques Automated document

comparison testing (OCR Engine)

Data quality governance framework ETL Automation Framework Analytical models testing

techniques

IVR Automation Engine Load simulators for Voice

Traffic Speech to Text Engine

ALM

Test Data Management

TEM, TDM Service Management FrameworkProposed Tools


Velocita : Enterprise test automation framework

Generic Utility Libraries

Log Generator

Dialog Handlers

Web Accelerators

MobileAccelerator

s

Data DriverAPI

AcceleratorSOA

P

Automated Scripts

Build Tool

Launch Script

Test Controller

function myFunction(a, b) {return a * b;

}


}


}


}


}


}


}


}


}


}


}


}


}


}


}

var x = function (a, b) {return a * b};var z = x(4, 3);















Test Input Data Object Management Reporting Content Management and Database Validation

Service Virtualization

Application Specific Library

Parallel Execution Application Under Test Parallel Execution

Application Under Test

Local Execution

Cloud ExecutionAmazon

Cigniti CI Framework

Automation Tools

Test Management Tools

Technology

Tools


Wha

t & W

hy ?

Maximize your Customer Experience & Service

Gaining Competitive Advantage

Drive Business Decisions Effectively

Revitalizing a brand

Detect patterns in customer behaviours, and track early product feedback to improve the product / app quality

Busin

ess V

alue

Inability to meet growing customer’s expectations, thus impacting the revenue

Vital to monitor the differences in perception of app / product from consumer and competitors’ view point

Finding Overall Sentiment of a conversation ( Positive, Negative, Neutral ) to enhance the customer experience

Detecting Emotions (Advocate, Satisfied, Neutral, Dissatisfied) for better customer service

Better decision making by sentiment categorization and segregation Peer comparison & Biz Impact Dashboards helps in finding the

business & financial intelligence

Sample Dashboard

HO

W?

Sentiment Analyzer


56%

17%

15%

12% 12%

62%

15%

11%

38%

38%

15%

9%23%

54%

17%

6% 12%

69%

13%6%

User Experience-74.6% Functional-14.2% Service Quality-5.8% Performance-4.4% Compatability-0.7%

Advocate

Dissatisfied

Neutral

Satisfied

WOW them by superior Ensure Application's

Focus areas to "delight" your customers and make them your advocates

1. User Experience2. Service Quality

1. Functionality2. Performance

Key TakeawaysSentiment Distribution by categories

Focus areas to avoid "disappointment"


Conversations Trend Word Cloud

Sentiment Analytics – A Leading QSR

Conversations Trend Word Cloud

Last Updated Last UpdatedApril 13, 2017 April 17, 2017

Analysis Summary


Cesta – Integrated Test Asset Migration Platform

Cesta ensures

15 times faster migration

10 times cost efficient

90% reduced effort

Ability to Migrate

Cost to Migrate

Time to Migrate

Challenges for

Migration

World’s first integrated test asset Migration platform consisting of QuickLean, Migrate2Selenium and M2TC tools to migrate HP QTP/UFT to LeanFT, Selenium and

TestComplete respectively.

A lot of companies today are in the process of migrating hundreds of old, as well as existing test scripts. Companies find it challenging in terms of ability to migrate, cost to migrate and time to migrate. It requires experts and expertise. Migrate HP QTP/UFT to HP Lean FT

Migrate HP QTP/UFT to Selenium

Migrate HP QTP/UFT to SmartBear TestComplete

Cesta is world’s 1st integrated test asset migration platform consisting of:

100% migrated and executable scripts


Q&A

Documents

Speakers...• Impact Analyzer – Code, component, functional and business processes • MBT platform • Script refactoring techniques • Process accelerators • Script migration