Managed Testing Services - Systems Plus …...Managed Testing Services Systems Plus Transformations Confidential Page 2 Introduction In an increasingly complex business environment,

Managed Testing Services “Helps you Run the Business, Change the Business Optimally”

Author: C.V. Narayana

Managed Testing Services

Systems Plus Transformations Confidential Page 1

Contents

Introduction ..................................................................................................................................... 2

MTS is Critical for Enterprises .......................................................................................................... 4

Key Challenges ................................................................................................................................. 4

MTS Overview .................................................................................................................................. 6

Governance ..................................................................................................................................... 8

MTS Roadmap ................................................................................................................................. 9

Value Proposition .......................................................................................................................... 11

Strategic Benefits ................................................................................................. 11

Operational Benefits ............................................................................................ 11



Introduction

In an increasingly complex business environment, superior customer satisfaction, efficient service delivery

and operational excellence are imperative for enterprises to stay competitive.

IT management teams are focusing their attention on scalable performance and ensuring minimal

application downtime to “Run the Business” as usual. From an application quality perspective, this would

mean the need to ensure rigorous quality checks to identify and eliminate performance bottlenecks as well

as minimise the impact of any new patches or application updates on the production environment. Lack of

consistent QA processes and highly skilled teams lead to deterioration of application quality resulting in

lower customer satisfaction levels, unhappy customers and eroding market share. Keeping the Lights On

(KTLO) though may be a repetitive process, is essential for the survival of a business.

The proliferation of business events like mergers & acquisitions, new technology paradigms (such as social

platforms, mobility, and data proliferation) new growth markets etc. are driving enterprises to “Change the

Business” continuously to stay competitive in the new competitive map. Enterprises are stretched to adopt

newer sophisticated IT systems within short time cycles to get to market faster with differentiated products

and services, and expand to newer high-growth geographies. This puts the spotlight firmly on the QA

organization, adding tremendous pressure to accelerate the testing cycles for new applications even while

ensuring high quality at an optimal cost. All of these also lead to an exponential increase in the complexity

of the IT landscape, which becomes unmanageable in the absence of consistent processes and tools.

Managed Testing Services (MTS) powered by business event-based testing model helps enterprises ‘Run

the Business’ and ‘Change the Business’ at will, without compromising on the quality of service delivery.



MTS behaves as an extension of the customer organisation by infusing the best of breed of centralised

teams and localised test team. MTS team is an independent verification and validation arm for all

application streams, providing end-to-end testing services - right from Test Strategy to Test Planning, Test

Execution, Reporting and Optimisation. This translates to an ownership for ensuring the completeness,

availability and change management of your business application landscape, managed by SLAs.



MTS is Critical for Enterprises

Typical enterprise applications fall under the 3 large buckets as given above. There are specific teams that

address web, bespoke (home grown applications), legacy, and packaged applications. Each of these teams

have scope specific to their needs. In addition, each of these teams will have specific process, tools and

standards. These structures lead to following challenges:

Key Challenges:

Spiraling testing costs

o License proliferation

o Underutilised teams

o Duplication of efforts

o Low absorption of best practices and maturity models

o Costs that add because of variable teams

o Knowledge management

o End to end testing visibility and hence issues in production and loss of business



o Insufficient automation to address the repeated tasks in testing

Product quality

Transparency in testing

Lesser no of cycles of testing

Overloading specialist departments with testing duties

No standardisation of testing activities to align with maturity models

Higher consequential costs caused by errors during live operation such as loss of business down

time, productivity and response times to customers

To address the above challenges, the IT management in an enterprise, have multiple options such as add

teams onsite at customer place or work with a near site team or an offshore team or a combination of

these. This is where Managed Testing Services (MTS) by a specialised vendor can offer onshore/ near shore

and offshore can provide a quality, value and cost effective solution.



MTS Overview

Managed Testing Services from Systems Plus helps customers, in terms of taking the responsibility of

testing of applications across the business processes in a defined and predictable manner built around SLAs

(Service Level Agreements.)

The pillars of MTS as defined in the diagram are:

1. Test execution process addressing

Alignment with Test Maturity Model

Metrics / KPI to measure the performance of the MTS and provide a quantitative approach to

improvements

Test management to address the scope of testing and plan based on SLAs.

Defect management to ensure the stated DDE (defect detection efficiency) is achieved from inception

to transition of IT applications.

Continuous improvements are carried out both at the operational level based on KPIs (Key

Performance Indicators) and engagement level based on SLAs.



2. Bouquet of services addressing functional and NFR testing based on ETVX paradigm, templates and

standard operating procedure which can be aligned with the customer processes. We offer unique

testing experience which is business oriented (Business process testing) built on branded or open-

source tools. The testing services provided are:

Test Process Consulting

To define the approach to MTS based on TMMi processes, analysis of existing test assets,

structures and challenges in enterprise

Functional Testing

Based on test cases, test data and test design which lend itself for reusability. Extension to

the service is also available to address Business process Testing (BPT).

Performance Testing

Services address the enterprises investment dilemma by allowing for Opex (operation

expenditure) instead of CAPEX (capital expenditure). This service also reduces the cost of

performance testing by judicious mix of open source and branded licenses to reduce the

cost of performance testing.

Security Testing

Services address the application exposure to threats on the web. There are reusable test

cases, methodology and tools to address the OWASP compliance based on STRIDE

(Spoofing, Tampering data, Repudiation, Information disclosure, Denial of Service,

Elevation of service) classification of threats and DREAD (Damage potential,

Reproducibility, Exploitability, Affected users, Discoverability) rating.

Automation Testing

Built on keyword driven assets which can clearly define ROI for the manual testing with

specific period for the ROI. This service is usually preceded by Automation consulting,

which clearly spells out the business benefits of automation and the expected period for

ROI.

Accessibility Testing (I18N)

Addresses the need for differently abled individuals to access the website and in

compliance to WCAG 2.0 standards

ERP Testing

Which can cover specific business process oriented testing reducing the manual effort and

specialists’ involvement, every time the ERP product is modified.



Governance

The governance of a Global delivery model is essential to creating a performing engagement on a

structured basis. This is addressed at 3 levels

Operational review addresses the Test plan and actual and the reasons for any deviations and

corrections necessary to address the deviations such as schedule, quality of deliverables. This is carried

out on a weekly basis

Strategic review addresses the engagement issues such as CSAT (customer satisfaction), overall

performance and change management. In steady state this is carried out once a quarter. SLA review is

one of components of the strategic review which looks at the committed SLAs such as DDE, ROI etc and

the compliance and corrective actions

Project teams exchange project data on a daily basis to make sure the project goals are met

Tools and reusable components

Test operation tools. We have extensive experience in setting up Test center of excellence (TCOE) using

open source tools and branded tools depending on the customer needs



Automation (keyword driven automation). This can reduce the number of licenses for automation, time

to automate and can support open source tools like Selenium and HP QTP. We also have libraries to

migrate the scripts from one platform to another

Business process testing components helps business users to define the expected outcome of a

business process such as order to cash or procure to pay etc., The components built on the keyword

driven library can build the test assets on top of the functional screens by abstracting the manual

process of entering the data for each of the functional screen.

OWASP compliant security test assets

Performance testing framework which converts the capex to opex

WCAG compliant accessibility test assets

MTS Roadmap

MTS roadmap is broadly classified into 3 stages with each stage clearly establishing a stage gate and

deliverables.



MTS setup: During this phase a small team travels onsite to the customer location to understand the

business processes, IT applications supporting the same, the test team structure, tools and technologies

and the challenges faced by the testing team. Based on the assessment, the team presents the

understanding of the challenges and a roadmap for a managed testing process. As a part of the knowledge

transition, the team will demonstrate the understanding of the testing process and application

understanding by participating in onsite mock office. Offshore facility setup, testing process, transition

roadmap for each of the applications, communication, parallel testing will be the key deliverables of this

setup phase.

Service optimisation comprises of 2 sub stages namely optimised delivery model and resource optimisation.

As part of the optimised delivery, the applications are grouped under web, legacy applications, packaged

products and custom software applications. This enables the human capital to be pooled or optimised

based on the type of technology and hence re-use of technology assets and the team. This increases the

test coverage and reduction in defect detection efficiency leading to reduction in Cost of Quality. Optimal

blended delivery model with an onsite local partner and an offshore delivery partner will provide the most

optimal delivery in terms of customer communication and execution efficiency and cost.

As a part of Resource pooling/ Resource optimisation sub-stage, the NFR reusable assets such as

Automation test assets, Security, performance and accessibility test assets will be used to reduce the cost

of quality and increase the test coverage. As a part of this stage all the stable metrics such as schedule

adherence, defect detection efficiency, y-o-y business benefits, productivity, and demand management will

be baselined

Steady State (SLA based) stage addresses the end to end testing of the applications. During this stage based

on the baselined metrics of the service optimization phase, commercial and operational SLAs are signed.

The team structure will have a fixed and variable team depending on the nature of work such as “Run the

business” or “Change the business”. Sample SLAs include

Y-o-Y reduction in cost of testing

Defect deduction efficiency

Test coverage

Schedule adherence



Value Proposition

Strategic Benefits

Business event based testing; focus on meeting business needs

Faster turnaround times for go to market initiatives

Drastically reduced management overheads

Overall improvement in end-user satisfaction

Cost reduction of 30-50 per cent by global delivery model

o Introduction of maturity models

o Reusable assets for NFR testing such as automation, security testing, performance testing

and accessibility testing

o Increased level of test automation

o Consolidation of licenses and capex to opex model

Quality of deliverables with increase in defect detection efficiency (DDE) and test coverage

Removal of the burden on specialist departments

Ability to manage fixed and variable teams effectively

With SLA management, the outcomes are more predictable leading to better quality

Operational Benefits

Lower resource- & team-cost, achieved through effective utilization & sharing of resources

Unbiased and objective testing from the business users’ perspective

Streamlined testing processes across multiple application streams as well as reusage of test

artifacts

Defect analysis and feedback to the development team for error reduction and early error

detection

Reusable assets such as automation assets, security and accessibility test assets

Standardized processes

Flexibility in processes and team ramp-up

Documents

Managed Testing Services - Systems Plus …...Managed Testing Services Systems Plus Transformations Confidential Page 2 Introduction In an increasingly complex business environment,