Bringing forth business value via proper test management process

AVACORP Technologies Bringing forth business value via properTest Management

Process

Page 1 of 10

Bringing forth business value via proper Test Management process

==================================================================================

Author: D.B.V.Phanindra Kishore.

E-Mail: [email protected]

Company: AVACorp Technologies Pvt Ltd, Chennai - 600034

==================================================================================


Process

Page 2 of 10

Table of Contents

1.0 Abstract ....................................................................................................................................... 3

2.0 Tactical Perspective: ......................................................................................................................... 4

2.1 Early involvement of testing teams in SDLC process .................................................................... 4

3.0 Strategic Perspective: ....................................................................................................................... 5

3.1.1 Fine tune team selection process .............................................................................................. 5

3.1.2 Dynamic “Centre of Excellence” groups to leverage internal expertise .................................... 6

3.1.3 Ecosystem for knowledge management .................................................................................... 7

3.1.4 Focus on reusability ................................................................................................................... 8

3.1.5 Define and measure metrics based on business goals .............................................................. 9

4.0 Conclusion: ...................................................................................................................................... 10


Process

Page 3 of 10

1.0 Abstract

Organizations across the world and across product lines depend on their ability to deliver products

that are of high quality without having to invest significantly in their quality improvement programs.

This is equally applicable in the software industry where the key mantra is to deliver products on

time, within budget and with minimum defects that get detected by the end customer. However, it

is only rarely that organizations realize the importance of testing the testing team’s ability to deliver

value beyond just testing the application. By making minor changes to the way testing team is

structured and refactoring the team’s involvement in the overall product development life cycle,

organization can significantly enhance their ROI while reducing the time to market along with the

overall cost of ownership.

Forrester define “Quality software” as: “Software that meets business requirements, provides a

satisfying user experience, and has fewer defects.” Lacking proper Test management or well thought

out roadmap for quality, projects/products are generally not able to meet the expected quality

standards. It might also be possible that organizational inefficiency, lack of proper knowledge on

continuous process improvements and a host of similar factors result in budget and schedule

overrun. This intern led to poor quality deliverables to the customer. To meet the quality standards

organizations can bench mark their test process against standards like TPI & TMM. Assessing with

standard processes will help in identifying the gaps and then focusing on these improvements will

result in overall improvement in the testing organization.

While there are a host of factors that can be considered as part of an organization’s test capability

improvement program, there are few that occupy an important position. Together the six focus

areas (listed below) help test organization reduce redundancy, improve reusability, enhance team’s

capabilities, manage organizational knowledge (Knowledge management) and build the team’s skill.

1. Early involvement in SDLC

2. Fine tune team selection process

3. Implement CoE’s to develop expertise

4. Develop an eco-system for knowledge management

5. Implement and enhance organizational reusability of tools and processes

6. Design, develop and measure success through meaningful metrics

The six factors listed above and elaborated below can be divided into two categories (a) The tactical

approach and (b) The strategic approach. Tactical approach talks about what can be improved in the

short term with the minimum investment of time and resources, whereas the strategic approach


Process

Page 4 of 10

talks about areas of improvement that can have an extended period of time. Depending on context

to context, this can be as low as six months and can go up to one year. However, it needs to be kept

in mind that the time frame is indicative and should not be considered as a standard.

2.0 Tactical Perspective:

2.1 Early involvement of testing teams in SDLC process

The traditional approach in many organizations consists of involving the testing team after the

completion of coding (or towards end of coding cycle) or as part of the SIT test cycle. This

approach suffers from drawbacks that include not knowing the quality of the code till the end of

project phase, reduced testing time in case of delays in the development schedule. However,

latest changes in the SDLC landscape like “Agile”, “Test Driven Development” and similar models


Process

Page 5 of 10

have changed the rules. These models are very clear about the need to involve testing team

from day one of the project.

After deciding the appropriate model (Agile, TDD or iterative), the first and foremost step is to

involve the testing team in requirement gathering / analysis (from a test execution perspective

and not as a substitute for business analyst). This helps from different perspectives like:

Provides clarity on the different functional flows early in the life cycle

Conflicting requirements or non testable requirements can be eliminated

Based on their experience, testing team can identify potential defects (application

failures). This leads to defect prevention

UAT / CAT test cases or scenarios can be defined upfront

Helps develop an appropriate test strategy (in a very short period of time), identify risks,

issues, dependencies and assumptions

Helps establish and measure requirement clarity, define the scope and help identify

resourcing needs early in the life cycle

In short, early involvement of the testing team will help the project reduce redundancy, reduce

the test life cycle, improve the product quality and finally result in improved ROI.

3.0 Strategic Perspective:

3.1.1 Fine tune team selection process

It is common knowledge that most of the times projects face difficulties with respect to

resources. This could be in the form of lack of expertise, not adhering to deadlines, inability to

pick up skills despite training and a host of similar factors. Unfortunately, it would be difficult to

bridge these gaps in a short span of time or when specifically associated with a project with

clearly deadlines. This problem gets compounded in case of organizations that do not or cannot

maintain a huge headcount that consists of people with a variety of skills.

The good news, however is that with a little bit of planning and execution excellence,

organization can build a pool of highly talented individuals that will not only meet your project

objective but also help build test assets that can be used at an organizational level.

Listed below are some of the activities that can be implemented from an organization

perspective which will help build such robust and scalable teams:


Process

Page 6 of 10

Proper skill gap analysis of resources based on the project needs

Planning for trainings relevant to project based on skill gap analysis

Proper capacity planning in line with project deliverables will avoid unnecessary head

counts in projects.

Proper evaluation mechanism for improvements of resources to build highly effective

teams by having transparent rewarding process in place.

Having proper communication mechanism maintaining the open culture and high work

ethics between teams will lead to quality in productivity.

3.1.2 Dynamic “Centre of Excellence” groups to leverage internal expertise

Creation of centre of excellence groups (CoE’s) will help in streamlining the test process in

testing organization and at the same time they will help in on boarding the right resources to the

projects. Having proper CoE’s will help organizations to build and manage domain and

technologies capability. CoE’s which is now a known term essentially relates to grouping a set of

individuals that have a common skill set (or to build a specific skill set). Example of CoE’s include

‘Test Automation’, ‘Performance Testing’, ‘Mobile Testing’, ‘Domain based CoE’, ‘Web services

and SOA’, ‘Security Testing’, ‘Virtualization Testing’ and ‘Testing the Cloud based applications’.

This capability building activities depends on a host of activities that include the services being

provided by the organization, client needs and the strategic direction the organization intends to

take.

As part of centre of excellence groups, testing organizations need to define objectives and set

deadline driven goals that can be achieved in short term, medium term and long term. One of

the key objectives of such CoE’s could be to build reusable test assets, frameworks and solution

accelerators or to develop technical capabilities. This will help projects to get instant access to

readymade information on related technologies; develop tools that will save time, precast test

strategy and performing POC’s when needed. As part of defining goals or when setting

objectives, organizations need to consider usage of open source tools that can help reduce the

cost of licenses.

As part of team building activities, CoE’s should focus on building innovation in the way the

teams work. This should lead to smarter solutions considering the fundamentals of reusability;

this reduces time and improves efficiency.

Listed below are some of the sample goals and objectives for CoE’s. These are just indicative of

the dimension but do not cover the entire spectrum of what can be done.


Process

Page 7 of 10

Careful observation of the goals indicates that some of them cross the testing organization

barrier and move on to provide support to other departments. As an example: developing of a

reusable framework will not only help reduce testing time (while this is a benefit for the test

organization) but also help improve the organization’s brand (which is a goal from the

organization perspective). This in turn can help sales / marketing team especially when they

need to showcase organization capability.

3.1.3 Ecosystem for knowledge management

Knowledge management is very important aspect for testing organization to gather the reusable

assets, test artifacts, estimation standards, risk register and test strategies of past projects.

These will become assets and reference documents for future similar project engagements,

hence time can be saved and risks involved in handling similar type of projects can be mitigated.

Some of additional knowledge management activities are categorized as knowledge related to

domain, metrics trends to define the standards based on the organizations business goals.


Process

Page 8 of 10

The above mentioned KM activities can be carried inline with the CoE’s goals and objectives.

Thus testing teams no need to spend extra time to manage KM activities separately. These

assets should be made available as common repository to the testing organizations to access

and use them in projects.

Having proper knowledge management system will help in gaining the following business

benefits

Having cross functional knowledge sharing process in organizations will save project

teams life cycle time

Whenever unknown information with respect to domain, tools and techniques are

required for project teams, KM will help in getting the required information from the KM

repository

Having proper KM system in testing organization will help indirectly in building robust

reusable assets, frameworks, referable sales collaterals information

This will help in cross leveraging the experience between testing teams

3.1.4 Focus on reusability

Test Process improvements can be achieved in testing organizations by having and usage of

proper tools and techniques throughout the life cycle. These tools usage can be categorized into

different types for testing organization. Ex: Test Management tools, Automation tools,

Performance testing tools, web service and SOA tools, security etc. The main objective of these

tools usage is achieving reusability by reducing the cycle time and improving the quality. At the

same time selecting a right tool also very important based on the skills of the resources have and

tool support.

The main objective of tools selection is to build quality by reducing the defects. Automation

frees the resources from mundane testing to focus on highest priority tests and increases test

cycle’s repeatability. In automation strategy adoption process concentrate on open source based

framework development considering the reusability as main mantra to achieve optimization and

achieving ROI. But this is not a standard for everything to use open source tools; it is based on

the technology and strategy the organization adopts.

Reusability can be achieved by developing the frameworks in relevant test types and substituting

them with the time consuming activities of manual test process. Hence test organization will

save time in testing the application on combinations of OS, DB, different application servers by

means of proper automation process.


Process

Page 9 of 10

Following are some of the business benefits can be achieved by usage of tools and techniques in

test organization by focusing on reusability.

Reduced test execution time

Reduced cost of test execution

Increase in ROI

Increased reusability of test assets across organization

Ensures scalability and minimization of rework

Consistent and repeatable test execution

Improved utilization of resources (people and machines)

Frees up test engineers time and helps them explore corner test conditions

3.1.5 Define and measure metrics based on business goals

The resultant outcomes tracked from testing organizations, individual projects deliverables need

to be mapped or these need to be derived from organizations business goals and metrics to the

individual projects. Metrics will help testing organizations in knowing the health of projects

deliverables, help in identifying the ways to mitigate the risks, corrective actions and also gives

early warnings on the level of quality the current system has. Proper analysis of these metrics

will give insight to deliver teams in terms of quality, time and milestones achievement, HR and

RM teams about team’s productivity and utilization, Finance teams about the cost of project.

Some of the key testing organizations activities need to be considered to attain business goals

are

Implement bets practices, standards in expertise

Establish better controls, metrics and SLA’s

Establish and create reusability, frameworks as part of tools and technologies

improvements

Maintain quality dash boards through proper metrics

Testing organization should have periodic reviews based on the project deliverables and conduct

proper analysis to know the project health. Based on the action items testing team’s or project

teams should work accordingly based on the correction item suggested. This will help in reaching the

quality goals and will ensure smooth delivery within budget. At the same time this assessment

process will help in mitigating the risks indirectly.


Process

Page 10 of 10

Some of most important metrics that need to be captured as part of the quality dash board analysis

which is applicable to testing organizations are listed below

Defect density (Code, test case, test script), Defect removal efficiency, Defect per test case, DSI

(Code, Test case, Test script), DRR, Unit Test code coverage, Build stability, Unit test pass rate, QA

test case code coverage, Fix quality, Maintainability Index are some of them which will play vital role

in knowing the project health.

4.0 Conclusion:

"Quality must move beyond the purview of just QA professionals to become an integrated part of

the entire software development life cycle to reduce schedule delays, improve user satisfaction, and

reduce the product risk”.

This eco system also builds a vibrant culture where knowledge sharing thrives and Test organization

specific assets and productivity enhancers are built automatically.

While Quality improvement programs are a great way, organization should focus on not just

improving the Quality but should also focus on building the organizational structure in such a way

that it addresses the organizational needs like having robust frameworks, having skilled resources,

elimination of redundancy and ability to reuse test assets across the board with meaning metrics

analysis in place. CoE’s definitely one of the options that can help achieve these goals. Ultimately

organizations can deliver products/applications within budget, as per timeline and with expected

quality leading to high customer satisfaction.

Technology

Bringing forth business value via proper test management process