Embed Size (px)
Citation preview
Using Metrics and Customer Satisfaction
Survey For Customer Management
(with a case study of Lionbridge Tech. Pvt. Ltd)
Dissertation Submitted to the
Padmashree Dr. D.Y. Patil University
in partial fulfillment of the requirements
for the award of the Degree of
MASTERS IN PHILOSOPHY (Business Management)
Submitted by: SATISH RAO
(Roll No. DYPMPHIL-0701004)
Research Guide:
Dr. R. Gopal
Director, Dean & Head of Dept.
Padmashree Dr. D.Y. Patil University,
Department of Business Management
Sector 4, Plot No. 10, CBD Belapur,
Navi Mumbai. 400 614
June 2009
Using Metrics and Customer Satisfaction
Survey For Customer Management
(with a case study of Lionbridge Tech. Pvt. Ltd.)
3
DECLARATION
I hereby declare that the dissertation “Using Metrics and Customer
Satisfaction Survey For Customer Management (with a case study of
Lionbridge Tech. Pvt. Ltd.)” is a bonafide research submitted for the
Degree of Masters in Philosophy (Business Management) at
Padmashree Dr. D.Y. Patil University’s Department of Business
Management is my original work and the dissertation has not formed the
basis for the award of any degree, associate ship, fellowship or any other
similar titles.
Place: Mumbai (Satish Rao)
Date: Signature of the Student
4
CERTIFICATE
This is to certify that the dissertation entitled “Using Metrics and Customer
Satisfaction Survey For Customer Management (with a case study of
Lionbridge Tech. Pvt. Ltd.)” is the bonafide research work carried out by
Mr. SATISH RAO, student of Master in Philosophy (Business
Management), at Padmashree Dr. D. Y. Patil University’s Department of
Business Management during the period 2007 - 2008, in partial fulfillment of
the requirements for the award of the Degree of Master in Philosophy
(Business Management) and that the dissertation has not formed the basis
for the award previously of any degree, diploma, associate ship, fellowship or
any other similar title.
Place: Mumbai (Dr. R. Gopal)
Date: Signature of the Guide
5
ACKNOWLEDGEMENTS
In the first place, I am indebted to the Padmashree Dr. D.Y.Patil University’s
Department of Business Management, which has accepted me for M. Phil
program and provided an excellent opportunity to carry out this research
project. I sincerely thank Prof. Dr. R. Gopal, Director, Dean and Head of the
Department at Department of Business Management of Padmashree Dr.
D.Y.Patil University, for having given me his valuable guidance for the
project. Without his help it would not have been possible for me to complete
this project.
I would also like to thank the various people from the IT industry who have
provided me with a lot of information and in fact even sharing some of the
data – many of which I have used in this report and without which this project
could not have been completed.
I would be failing in my duty if I do not acknowledge with a deep sense of
gratitude, the sacrifices made by my wife Aruna, daughter Aishwarya and son
Anirudh for allowing and supporting me to spend my free time on this project
work and thus have helped me in completing the project work successfully.
Place: Mumbai (Satish Rao)
Date: Signature of the Student
6
PREFACE
While searching for a suitable topic for the M.Phil. Dissertation, I came across
a problem that I faced at work on every customer project. On discussing with
several people from the IT industry, it transpired that the ‘Customer
Management’ is one of the main problems that every Software Vendor faces.
The topics having aroused my curiosity, discussions were held with several
people in the IT sector to understand the veracity of the above thought
process and also understand the real issues plaguing the industry.
All these aspects then resulted in the development of the project report titled
‘Using Metrics and Customer Satisfaction Survey For Customer Management
(with a case study of Lionbridge Tech. Pvt. Ltd.)’
It is strongly hoped that this project covers not only the various requirements
of the Project Study but also of the Industry.
(Satish Rao)
Signature of the Student
7
CONTENTS
8
CONTENTS
1.0 INTRODUCTION .................................................. 22
2.0 CONCEPT AND THEORY.................................... 25
2.1 WHAT IS A CUSTOMER?...................................................... 25
2.2 WHAT IS CUSTOMER SATISFACTION?................................... 26
2.3 WHAT DETERMINES CUSTOMER SATISFACTION? .................. 28
2.4 WHY CUSTOMER SATISFACTION? ........................................ 30
2.5 MEASURING CUSTOMER SATISFACTION................................ 34
2.6 THE PROFIT CHAIN ............................................................. 45
2.7 IMPORTANCE OF CUSTOMER SATISFACTION AND LOYALTY..... 47
2.8 WHAT IS OUTSOURCING? .................................................... 49
2.9 WHY OUTSOURCING? ......................................................... 50
2.10 CUSTOMER RELATIONSHIP MANAGEMENT ......................... 54
2.11 CUSTOMER COMPLAINT MANAGEMENT.............................. 54
2.12 WHAT ARE METRICS ..................................................... 56
3.0 LITERATURE REVIEW ........................................ 58
4.0 OBJECTIVES OF THE STUDY ............................ 72
5.0 RESEARCH METHODOLOGY............................. 74
9
5.1 RESEARCH OVERVIEW ........................................................ 74
5.2 RESEARCH DESIGN............................................................. 76
5.3 SAMPLE DESIGN ................................................................. 85
5.4 DATA COLLECTION.............................................................. 85
6.0 CASE STUDY OF LIONBRIDGE.......................... 87
6.1 ABOUT LIONBRIDGE ........................................................ 87
6.2 SAMPLE SELECTION ............................................................ 92
7.0 DATA ANALYSIS AND FINDINGS ...................... 94
7.1 DATA ANALYSIS .................................................................. 94
7.2 FINDINGS FROM THE STUDY.............................................. 101
8.0 CONCLUSIONS AND RECOMMENDATIONS... 103
8.1 SUMMARY OF OBJECTIVES AND OUTCOMES....................... 104
8.2 LIMITATIONS OF CURRENT RESEARCH ............................... 106
8.3 RECOMMENDATIONS ......................................................... 106
10
LIST OF ANNEXURES
ANNEXURE 1 - CSAT QUESTIONNAIRE TEMPLATE ........... 108
ANNEXURE 2 - CSAT QUESTIONNAIRE RESPONSES ........ 114
ANNEXURE 3 – PROJECT METRICS DATA........................... 146
ANNEXURE 4 – BIBLIOGRAPHY............................................. 156
ANNEXURE 5 – COPIES OF FEW ARTICLES REVIEWED.... 160
11
LIST OF TABLES
Table 1 - Timeline for the execution of this research ............ 84
Table 2 – Services provided by Lionbridge ............................. 90
Table 3 – Representative client list of Lionbridge .................. 90
Table 4 – Industry focus of Lionbridge .................................... 91
Table 5 - CSAT Survey responses for Sep 2007 ..................... 96
Table 6 - CSAT Survey responses for Nov 2007 ..................... 98
Table 7 - Summary of Objectives And Outcomes................. 104
Table 8 - Legends ..................................................................... 147
Table 9 - Metrics data for August 2007................................... 148
Table 10 - Metrics data for September 2007 .......................... 149
Table 11 - Metrics data for October 2007 ............................... 151
Table 12 - Metrics data for November 2007 ........................... 152
Table 13 - Metrics data for December 2007 ........................... 154
Table 14 - Trend Data for the Projects in Amber................... 155
12
LIST OF FIGURES
Figure 1 - ACSI Model of Customer Satisfaction .................... 28
Figure 2 - AT&T Customer-Supplier Model.............................. 33
Figure 3 - Performance Analysis Model ................................... 44
Figure 4 - The Profit Chain......................................................... 45
Figure 5 - Relationship between CSAT and Loyalty ............... 48
Figure 6 - Research Methodology............................................. 75
Figure 7 - Trends of the projects in Amber............................ 100
13
LIST OF ABBREVIATIONS
ASCI American Customer Satisfaction Index
ASP Application Service Provider
CSAT Customer Satisfaction
CRM Customer Relationship Management
CSM Customer Satisfaction Measurement
DRE Defect Removal Efficiency
ED External Defects
EV Effort Variance
ID Internal Defects
IT Information Technology
LTPL Lionbridge Technologies Pvt. Ltd.
MDC Mesa Data Centre
PDC Pearson Data Centre
PSR Project Status Reports
QA Quality Assurance
SLAs Service Level Agreements
SV Schedule Variance
UK United Kingdom
USA United States Of America
VOC Voice Of Customer
14
EXECUTIVE
SUMMARY
15
Executive Summary
Business environment is becoming very competitive day by day.
Corporations want to have very highly productive employees and want all
employees focus on generating profit. Every measure is taken to reduce the
cost and maximize profit. But while doing this they don’t want to compromise
on quality. With globalization CEOs want the best of class for their companies
in all the areas of their operations. More and more corporate have adapted
outsourcing to avail the best in industry.
Knowledge and expertise no longer have to be homegrown. They can be
contracted for. Outsourcing provides access to world-class capabilities.
Outsourcing providers and specialty consulting firms can bring worldwide,
world-class capabilities and expertise to a firm. Outsourcing is an effective
tool for redefining and re-energizing an organization. It helps firms focus on
their core business by assuming that some or all of their noncore functions
can be outsourced.
Access to an outside provider's lower-cost structure may result in greater
economy of scale and is one of the most compelling tactical reasons for using
outsourcing to professional services consultants.
Outsourcing cuts across geographic boundaries and today companies
outsource work to outsourcing providers who work from more than 5,000 to
10,000 miles away from their base. One of the challenges faced by the
outsourcing provider is to measure their Customer’s CSAT.
The most recent experience on a particular project, especially if it is not so
good, becomes the overall perception and such anecdotal references then
become the performance indicator for the outsourcing provider. This is more
subjective in nature and may not be reflective of the correct status. The
perceptions are always relative to expectations. Because expectations are
dynamic, evaluations may also shift overtime, from person to person.
16
What is considered good quality services that satisfy the customers today
may be different tomorrow. The entire discussion of customer satisfaction is
based on customers’ perception of the service and not some predetermined
objective criteria of what service is or should be.
More and more companies recognize that they can compete more effectively
by distinguishing themselves with respect to improved customer satisfaction.
Many companies are spending more time and money understanding the
underpinnings of customer satisfaction and ways that they can improve.
These companies try different ways to measure CSAT. An effective customer
satisfaction measurement system results in reliable information about
customer ratings of specific product and service features and about the
relationship between these ratings and the customer’s likely future market
behavior.
Customer Survey is a very important tool to measure CSAT. If one wants real
value from Customer Satisfaction Measurement [CSM] there are some key
things one just have to get right, but many organizations don’t. It is not a
matter of asking a few questions to some customers and looking at what they
say. This kind of attitude often results in responsibility for CSM being
delegated to untrained staff with little knowledge of research techniques.
Subsequent results will often be unreliable and may mislead decision
makers. If the company really wants to know how satisfied its customers feel,
the questions asked in the survey have to cover customers’ main
requirements.
Interpretation and Analysis of the CSAT responses is very important. In
reality, there is a huge difference between gathering some customer
feedback and obtaining an accurate measure of customer satisfaction.
Failure to get these things right will, at best, cast doubt over the accuracy of
the satisfaction measures obtained and at worst seriously weaken the ability
17
of the organization to meet its fundamental objective of improving customer
satisfaction and business performance.
Leading companies have developed Profit Chain models enabling them to
forecast the effect on financial performance of improving customer and / or
employee satisfaction. Since customer satisfaction measures usually occupy
a pivotal place in such models, the reliability of the measures becomes
critical.
There is a strong relationship between CSAT and Loyalty in all competitive
industries. Satisfied customers tend to be more loyal and these loyal
customers spend more, are willing to pay higher prices, refer new clients, and
are less costly to do business with. It costs five times more to find a new
customer than to keep an existing one happy. A firm cannot create loyal
customers without first creating satisfied customers. To create satisfied
customers, the organization needs to identify customers’ needs, design the
production and service systems to meet those needs, and measure the
results as the basis for improvement.
This current research study is an attempt to create a framework for
evaluating the health of a customer engagement for an IT outsourcing
provider. Hence manage the customer better. This tool will encompass
quantitative and qualitative indicators.
The objectives of the research are:-
• To understand Customer Satisfaction.
• To understand the relationship between CSAT and loyalty.
• To create a CSAT survey tool for Qualitative measurement of Customer
Satisfaction (CSAT score).
• To define key metrics for quantitative measurement to support the above
qualitative CSAT score.
18
• To study the relationship between project performance and CSAT.
Several articles were reviewed to understand certain concepts. These
concepts were used during the study.
Two sources of data used for the study were:
1) Project performance data i.e. Metrics data from ongoing projects were
captured to understand the project performance.
2) CSAT survey questionnaire was prepared to capture the qualitative
inputs from the customers. The CSAT questionnaire used for the study
was discussed with LTPL managers and their feedback was
incorporated before it was administered.
The combination of the project metrics data trends and the CSAT was used
for arriving at the health of the customer engagement and hence manage the
customer better.
A case study of Lionbridge Technologies Pvt. Ltd. was used for this study
purpose and for data collection, one of the customers of LTPL was selected.
This customer had twelve managers who interacted with LTPL team on a day
to day basis. The CSAT questionnaire was administered to these selected
customer managers on a quarterly basis. As a part of this study the
responses for two quarterly CSAT surveys were taken up for analysis.
The questionnaire consisted of following three sections:
1. Customer Information & Purpose
2. LTPL Performance
3. Comments
Data analysis was done using Microsoft Excel tool. The major findings of the
study were:
19
• Through the article reviews, it was understood that satisfaction is
customers’ evaluation of a product or service in terms of whether that
product or service has met their needs and expectations. Failure to
meet the needs and expectations is assumed to result in
dissatisfaction with the product or service.
• A framework for measuring CSAT was created to arrive at the Overall
Customer Satisfactions score which is a finite number between 1 and
5, is quantifiable and will be taken as the CSAT Index. As this is a
finite number, it gives an indication of the overall customer satisfaction.
Hence the CSAT will not depend on the perception of the person
concerned.
• Minimum set of metrics that would be required for any project
monitoring was identified. The trend of these metrics indicates the
performance of the delivery teams. Hence this information will be the
backup information for the CSAT score.
• Through the literature reviews and by validating using the project data,
it can be easily established that there is a very strong linkage between
CSAT and loyalty.
The limitation of this study was regarding the population size for conducting
CSAT for a company. Mid size IT outsourcing companies will have approx.
12-15 large customers who would be executing multiple projects
simultaneously. Hence getting a large sample as per statistical techniques
may not be possible.
The population for the projects for the customer for whom the CSAT is being
conducted should be big. This may not be possible most of the time. Hence
setting trends and control limits may be difficult.
Key Findings:
For the CSAT survey conducted in Sep 2007, the CSAT score was 3.29 and
this score for Nov 2007 CSAT survey was 4.00. The project performance in
20
the above time as discussed in section 7.1.2 shows an improvement in the
schedule variance metric. Section 7.2 has listed the findings indicating that
whenever the project deliveries are made on time, the CSAT scores for that
period are better. This means that there is a positive relationship between
these CSAT score and project metrics.
Recommendations:
The framework created in this study can be used for assessing the health of
the customer engagement. In the current study the Schedule variance is
used for analysis. The other four metrics can analyzed for assessing the
project performance. These project metrics need not be limited to five. These
could be further extended based on the project needs and the nature of
engagement. More metrics can be added and one such example is ‘Turn
around Time’ or TAT. This metrics is very important for Support type of
projects.
21
CHAPTER 1
INTRODUCTION
22
1.0 INTRODUCTION
Investing in Information Technology is the need for every mid and large size
corporations irrespective of what their space of operation is. Corporations
want to reduce their operating costs and increase their profits. They want to
leverage IT to make them competitive in the market. These corporations want
world-class capabilities in their operations / services to meet their growing
business. At the same time there is cost pressure and these corporations
want to keep their costs under check.
Outsourcing allows the firm to focus on its core activities, while support
services are assumed by an outside provider, such as a specialty consultant.
It can enable an organization to accelerate its growth and success through
expanded investment in areas offering the greatest competitive advantage.
IT outsourcing has been recognized as a strategy for increasing efficiency
and cutting costs of the information systems implementations. A properly
implemented outsourcing strategy brings together industry knowledge and IT,
to create systems that help organizations to acquire and maintain a
competitive advantage and provide better service at a lower cost.
In IT outsourcing environments, customers' requirements and feedback are
essential to the development of information system applications and the
improvement of the service quality of IT service vendors or companies. From
the vendor's perspective, it is important to minimize the reasons for
complaints and dissatisfaction as well as the cost of a service recovery plan.
It is also important for vendors to establish a track of feedback from
customers on their reactions to complaints and dissatisfaction. Therefore, it is
useful to develop a customer satisfaction evaluation system IT outsourcing
providers.
IT Oursourcing vendors have customers who were mainly from USA and
Europe. These customers outsourced the development & maintenance of
23
their software applications & products to this company. On several occasions
the senior management asked the delivery teams about what the CSAT for
their customers was. The customer managers when asked the same
question give different responses at different times. The reason being, in the
absence of following any structured tool for measuring CSAT, the response is
subjective and depends of the most recent events encountered by the
customer in their respective engagements.
In order to monitor and maintain a high service quality of IT outsourcing
vendors, it is necessary to develop a system to evaluate CSAT for an IT
outsourcing provider. This research study is an attempt to develop a
framework for evaluating the health of the customer engagement of IT
outsourcing vendors using CSAT and metrics data.
Using such evaluation system as a tool for measuring IT outsourcing
customer satisfaction, IT outsourcing providers can monitor their service level
and precisely understand customers' requirements. The observed values of
customer satisfaction can provide important guidelines in the improvement of
IT outsourcing services and improve their competitive position in the market.
24
CHAPTER 2
CONCEPTS AND THEORY
.
25
2.0 CONCEPT AND THEORY
This section discusses the concept of terminologies came across during the
study.
2.1 What Is A Customer?
Following are some of the interesting quotes about the ‘Customer’ which
should be noted by everyone in every industry:-
A Customer is the most important person ever in our organization … in
person or mail.
A Customer is not dependent on us … we are dependent on him.
A Customer is not an interruption of our work … he is the purpose of it. We
are not doing a favor by serving him … he is doing us a favor by giving us the
opportunity to do so.
A Customer is not someone to argue or match wits with. Nobody ever won an
argument with a Customer.
Customer is the one who brings us his wants. It is our job to handle them
profitably to him and to ourselves.
Today, more companies are recognizing the importance of satisfying and
retaining customers.
There are enough data showing that new customer acquisition costs much
more that retaining the existing one.
Satisfied customers constitute the company’s relationship capital. If a
company was to be sold, the acquiring company would have to pay not only
26
for the fixed assets and brand name but also for the delivered customer base
i.e. number and value of the customers who do business with this company.
Following are some interesting facts bearing on customer retention:-
Acquiring new customers can cost five times more than the cost involved in
satisfying and retaining current customers.
The average company loses 10% of its customers each year.
A 5% reduction in the customer defection rate can increase profits by 25% to
85%, depending on the industry.
The customer profit rate tends to increase over the life of the retained
customer.
… source ‘Marketing Management’ by Philip Kotler.
2.2 What Is Customer Satisfaction?
Everyone knows what satisfaction is, but it seems difficult to give a definition.
Richard L Oliver respected expert and longtime writer and researcher on the
topic of customer satisfaction expresses the challenge of defining this most
basic of customer concepts.
Oliver offered his own formal definition as follows:-
Satisfaction is the consumer’s fulfillment response. It is a judgment that a
product or service feature, or the product or service itself, provides a
pleasurable level of consumption related fulfillment.
27
In simple words, Satisfaction is customers’ evaluation of a product or service
in terms of whether that product or service has met their needs and
expectations. Failure to meet the needs and expectations is assumed to
result in dissatisfaction with the product or service.
2.2.1 American Customer Satisfaction Index
The econometric model used to produce ACSI, links customer satisfaction to
its determinants:
• Customer expectations
• Perceived quality and
• Perceived value.
Customer satisfaction, in turn, is linked to customer loyalty, which has an
impact on profitability.
American Customer Satisfaction Index (ACSI):-
• Measures customer satisfaction at national level.
• Introduced in 1994 by University of Michigan and American Society for
Quality.
• Continual decline in index from 1994 through 1998 with a small
improvement into 2000 suggests that quality improvements have not
kept pace with consumer expectations.
28
Figure 1 - ACSI Model of Customer Satisfaction
Source: The Management And Control Of Quality by James R Evans and
William M Lindsay
2.3 What Determines Customer Satisfaction?
2.3.1 Product And Services Features
Customer satisfaction with a product or service is influenced significantly by
the customer’s evaluation of product or service features. Research has
shown that customers of services will make tradeoffs among service features
depending on the type of services being evaluated and criticality of the
service. Hence one needs to understand what is important to the customer.
2.3.2 Customer Emotions
Customer emotions can also affect their perception of satisfaction with the
products and services e.g. the customer being in a very happy frame of mind,
will influence how he feels about the services they may experience.
29
Alternatively, when they are in a bad mood, the negative feeling may carry
over into how they respond to the services causing him to overreact or
respond negatively to any little problem.
2.3.3 Other Consumers And Coworkers
A service will be individuals’ personal experience with the product and
service but also by what others say about it, how others use it and feel about
it and how widely it is adopted in the organization.
2.3.4 Responsiveness To Customer Needs And Requests
The satisfaction in service also depends on how adaptable is the service
delivery system is when the customer has special needs or requests that
place demands on process. In this case the customers judge service
encounter quality in terms of the flexibility of the employees and the system.
2.3.5 Technology Failure
Many dissatisfying stories relate to the technology not working as promised
e.g. features not implemented properly, too many defects, and response
times not as expected.
2.3.6 Poor Design
The dissatisfaction due to the way the solution was designed. This resulting
in poor maintainability, non-scalable and cannot be interfaced with other
systems.
30
2.3.7 Process Failure
The features as published in the feature/ requirements literature is not
functioning as expected.
2.3.8 Service Quality Dimension
The following are some of the key dimensions for Service Quality:-
Reliability: ability to perform the promised service dependably and
accurately.
Responsiveness: willingness to help customers and provide prompt service.
Assurance: employees’ knowledge and courtesy and their ability to inspire
trust and confidence.
Empathy: caring individualized attention given to customers.
Tangibles: appearance of physical facilities, equipment, personnel and
written materials.
2.4 Why Customer Satisfaction?
Outsourcers need to focus more on the customer as demands and
expectations rise, writes Neil Anderson, managing director of outsourcing
company QCOM, in his article titled “There can be no substitute for meeting
customers' expectations”.
We all know the world is changing at an ever faster pace and on many fronts.
While this rapidly shifting economic landscape is creating new business
31
opportunities, it is also forcing companies to respond to new client
expectations.
Whether this is fixing mobile technology on the move rather than returning it
to the workshop, or providing a same-day response to servicing a printer,
customer demands are growing.
All this is overlain by trends in globalization and overseas innovation, as well
as the emergence of Generation Y in the workplace and as consumers. The
reality is that India’s business strategies need to be both alert to sector
trends, and adaptable to enable companies to survive.
What this requires is a company-wide focus on customer service; a vital part
of any and all transactions.
2.4.1 Essential Service And After Sales-support
For service providers, it is understood that service and after-sales support
add value, build customer loyalty, and directly affect the bottom line.
The challenge of delivering good service, however, has been made more
complex with the advent of mobile technology, the need to reduce down-time
and demand for bespoke service packages.
Yet, it is really only possible to achieve good customer if it has a seat in the
boardroom and if corporate thinking is aligned (or realigned) around
customer-centric activities. There needs to be far greater awareness, for
example, that service engineers are important brand ambassadors. They can
act as the eyes and ears of a company, capable by turns of sporting trouble
brewing and (just as importantly) identifying new business opportunities.
Service providers also need to consider mat around half of the modern
service call is fixing the customer rather than the technology, educating the
32
user on the best use of the systems they are operating. In other words,
product servicing is about people and every service call should be treated as
a moment of truth which can either add value or seriously damage a product
or company in an instant.
2.4.2 Supplier Relationship
Where all or part of a customer’s operation is transferred to an outsourced
supplier, it's usually done either to allow the client to concentrate on their
core business, or to allow them to complement the service they already offer
customers in these areas.
Technical outsourcing expertise can help the outsourcer enhance their
profitability and it can support business development and expansion.
But the relationship needs to be managed and where possible, companies
should be looking for borderless collaboration with their outsourcing partner.
Relationships with outsourcers should go way beyond placing them in a
supplier box, or indeed just white labeling their services
In short the:-
• Customer expectations of service quality are rising and must be
addressed
• Service engineers are in perfect position to identify new business
opportunities
• After-sales packages can add value and profit, and grow customer
loyalty
33
Figure 2, illustrates the Customer-Supplier Model AT&T has implemented
and indicates the feedback mechanism from the internal & external
customers which is very important for implementing any improvement
process. The natural customer-supplier linkages among individuals,
departments, and functions build up the “chain of customers” throughout an
organization that connect every individual and function to the external
customers and consumers, thus characterizing the organization’s value
chain.
Figure 2 - AT&T Customer-Supplier Model
Source: The Management And Control Of Quality by James R Evans and
William M Lindsay
The experience faced by the customers may or may not up to their
expectations and satisfaction. There is no single tool that can be used to give
the stakeholders the correct picture of the progress of their projects which are
being executed 5,000 to 10,000 miles away from their shores. This is mainly
because every project is unique and there is no single solution which works
for all.
It is very common that the most recent experience on a particular project,
especially if it is not so good, becomes the overall perception and such
anecdotal references then become the performance indicator of the vendor.
34
This is more subjective in nature and may not be reflective of the correct
status. The perceptions are always relative to expectations. Because
expectations are dynamic, evaluations may also shift overtime, from person
to person.
What is considered good quality services that satisfy the customers today
may be different tomorrow. The entire discussion of customer satisfaction is
based on customers’ perception of the service and not some predetermined
objective criteria of what service is or should be.
More and more companies recognize that they can compete more effectively
by distinguishing themselves with respect to improved customer satisfaction.
This research project is an attempt to find out the relationship between the
identified key delivery metrics and customer satisfaction. Then evaluate this
as a tool to measure the health of customer engagement which is more
quantitative in nature and also include qualitative indicators.
Many companies are spending more time and money understanding the
underpinnings of customer satisfaction and ways that they can improve.
2.5 Measuring Customer Satisfaction
An effective customer satisfaction measurement system results in reliable
information about customer ratings of specific product and service features
and about the relationship between these ratings and the customer’s likely
future market behavior.
Customer Survey is a very important tool to measure CSAT. If one wants real
value from Customer Satisfaction Measurement [CSM] there are some key
things one just have to get right, but many organizations don’t.
35
Customer surveys are sometimes seen as a ‘common sense’ activity that
anyone can organize. After all, some say, it’s only a matter of asking a few
questions to some customers and looking at what they say. This kind of
attitude often results in responsibility for CSM being delegated to untrained
staff with little knowledge of research techniques. Subsequent results will
often be unreliable and may mislead decision makers. An example of this is a
much publicized example of misleading CSM results of the surveys
conducted by the privatized rail companies in the US. According to Which?
Magazine, the rail companies’ “current surveys were close to useless”
because:-
i) The questions avoid customers’ main requirements
ii) They distort definitions of satisfaction
iii) They use misleading trend data.
2.5.1 Who Sets The Agenda
If the company really wants to know how satisfied its customers feel, the
questions asked in the survey have to cover customers’ main requirements.
Companies are tempted to include questions on areas where they’ve
invested heavily or made improvements, but if these are of marginal
importance to customers they will make little impact on how satisfied their
customers feel.
2.5.2 Meaningless Definitions Of Satisfaction
Another old trick is to claim that survey results demonstrate satisfaction when
they do nothing of the kind. e.g. Person may claim that an average score
above 5 on a 10 point scale shows customer satisfaction. One should think if
he/ she were the respondent, would they give a score of 5 if they were
satisfied?
36
There are evidences from a huge database of CSM results showing that only
average scores of 8 or above mean satisfaction. Average scores in the 7s
should be interpreted as ‘quite satisfied’, in the 6s as ‘marginally satisfied’
and in the 5s as ‘marginally dissatisfied’. Many companies are now saying
that only ‘top box’ scores (i.e. average scores above 9 on a 10 point scale)
are acceptable. Another cunning way of distorting the results is to ask
customers to rate performance against expectation. If one is meeting or
exceeding customers’ expectations surely they must be doing well.
2.5.3 Misleading Trend Data
The period for this trend plays a large role and hence trends are often
misleading. If one achieves a satisfaction score of 4.9 on a 10 point scale its
trend data can show improvement. But if the period of data is different it may
show average.
In reality, there is a huge difference between gathering some customer
feedback and obtaining an accurate measure of customer satisfaction.
Rather than attempting to cover all aspects of the CSM process, one should
focus on the key areas that are often overlooked or under emphasized.
Failure to get these things right will, at best, cast doubt over the accuracy of
the satisfaction measures obtained and at worst seriously weaken the ability
of the organization to meet its fundamental objective of improving customer
satisfaction and business performance.
There are seven fundamental principles that organizations should adhere to
for an accurate and effective CSM process.
1. Allow Customers To Define The Criteria For Measurement
Many organizations determine the criteria for measurement internally, yet
research has often shown that suppliers rarely have an accurate
37
understanding of customers’ priorities. Since satisfaction is about ‘doing
best what matters most to customers’, it is essential that one measures
‘what matters most’. The only way of ensuring that ones survey measures
the issues of most importance to customers is to allow the customers
themselves to determine the criteria to be measured. To do this one
needs to carry out exploratory research to identify customers’ most
important requirements. This ‘qualitative’ research involves detailed
discussions to fully understand customers’ priorities. In consumer markets
or with internal customers, focus groups will often be used at this stage
whilst in business markets, one to one depth interviews are more
common. Either way, wide ranging discussions must first draw out the
issues of most importance to customers before quantifying their relative
importance. After several focus groups in consumer markets or 10-20
depth interviews in business markets, one would have identified
customers’ most important priorities. One should include the 15-20 things
that matter most to customers on the questionnaire for the main survey.
2. Measure Importance As Well As Satisfaction
Think about what makes one a satisfied or a dissatisfied customer. If ones
requirements are met then he/ she is satisfied but if they are not met
he/she will be dissatisfied. This tells that satisfaction is a relative, not an
absolute concept. It is relative to what was important to the each person in
the first place. So to produce an accurate measure of customer
satisfaction one has to survey both sides of that equation. By comparing
importance scores with satisfaction scores everyone in the organization
can see at a glance where one is meeting, exceeding or failing to meet
customers’ requirements.
38
3. Robust samples
Exploratory research will ensure that one asks the right questions but will
not guarantee an accurate measure of customer satisfaction if one asks
them to the wrong people. The third key element of a reliable result is
therefore a robust sampling process.
To be reliable, a sample must be:
• Large enough i.e. Sample Size
• Representative
• Unbiased
(a) Sample size
The accuracy of a sample is based on its absolute size, not its
proportion of the total population. A larger sample will always be more
reliable than a smaller sample whatever the total size of the
population. As a general rule, 200 is regarded a minimum reliable
sample size. Whatever the size of the customer base a sample of 500
would provide a very reliable result at the overall level. It is not a large
customer base that creates a requirement for a large sample but the
number of segments that the results need to be broken down into. As
a rule of thumb, a sample of 50 is regarded as the minimum per
segment, and preferably 100.
(b) Representative samples
Obtaining a representative sample is often straight forward. A retailer,
for example, might include customers in the correct proportions by
age, gender, spend, frequency of visit etc. However, business to
business companies typically have a Pareto effect, a small number of
39
large customers accounting for a large proportion of the company’s
business. If the result of ones CSM study is to adequately reflect
customers’ disproportionate values, the sample must be stratified. To
achieve this the customer list must first be sorted by account value
then divided into strata – typically large, medium and small accounts. If
ones top 20 customers account for 30% of sales they should be 30%
of the sample. Perhaps a further 200 customers in the medium
segment account for another 50% of sales, so should also account for
50% of the sample. 2000 small customers might account for the
remaining 20% of sales and of the sample. Assuming a sample size of
200 that would mean 40 small customers who would be randomly
selected from the 2000. It would mean 60 large customers, but in this
example there are only 20 large customers, so 3 respondents would
be required from each of the 20 companies concerned. Including more
than one respondent from the largest customers is common practice in
business markets, but which individuals should be included? If one
simply include his/ her ‘main contact’ at each company he/ she will end
up with a sample biased in favor of the individuals with whom he/ she
have most contact, who may be more favorably disposed towards him/
her. To be accurate, the individuals surveyed should be representative
of the people with whom one deals and are influential in determining
customers’ level of satisfaction. The exploratory research will identify
the range of individuals involved and the extent to which they influence
ones customers’ satisfaction. So if Purchasing Managers, for example,
are most influential and account for, say, 40% of the decision process
across customers generally, they should make up 40% of the sample
and if Quality Managers influence 10% of the decision they should
make up 10% of the sample.
40
(c) Unbiased samples
To be unbiased, a sample must be randomly selected. It is easy to
sample at random the required proportion of each customer value
group. The problem comes at the response stage. The theoretical
model of sampling reliability assumes a 100% response rate from the
customers randomly sampled. A lower response rate reduces the
reliability of the sample due to the problem of ‘non-response bias.’ The
lower the response rate, the more the sample is likely to reflect
extreme views since the most satisfied and the least satisfied
customers have the strongest motivation to respond. It is generally
considered that a response rate of at least 50% is required to reduce
the problem of non-response bias to an acceptable level. It is no use
sending out more questionnaires to get more back since it is the low
response rate which causes the problem – so the only way to solve it
is to exceed 50% response.
This is why many organizations are abandoning self completion
surveys for CSM in favor of telephone surveys where the response
rate will be higher and the result more reliable.
4. Explain The Survey To Customers
Whether the survey is self completion or telephone, it is still important to
maximize the response rate. Explaining the survey properly to customers
is the most effective way of increasing response rates and provides an
opportunity to enhance ones image as a customer focused organization.
Quite simply one needs to inform customers about what he/ she will be
doing and why this is being done. If one has a very small number of
customers in a business market it is most productive to explain the
process personally to each one through well briefed customer contact
staff. Larger samples require a personal letter to each respondent.
41
There are three things one needs to tell them:-
• Why you are doing it?
• How you are going to do it?
• The feedback you will provide afterwards
(a) The Purpose Of The Survey
Don’t assume that customers will correctly interpret the purpose of a
CSM survey. Many people confuse surveys with selling approaches.
Others correctly distinguish research from selling but may see all
research as a brain picking exercise which benefits the supplier rather
than the customer. So although it seems obvious, point out that the
purpose of the survey is to identify whether customers’ requirements
are being met so that action can be taken to improve customer
satisfaction where necessary.
(b) The Survey Details
If the introductory letter accompanies a postal questionnaire it will
include the instructions for completing and returning it. If one
undertakes a telephone survey the introductory letter should give brief
details of the topics that will be covered and should stress that an
appointment will be made to interview customers at a time convenient
to them. It is also useful to reiterate how valuable the customer
feedback is in order to encourage the highest possible participation
rates.
(c) Feedback
Research evidence demonstrates that promising feedback is the
single most effective element in increasing response rates. The
introductory letter should therefore inform customers that they will
receive feedback on the results, on the key issues that have been
42
identified by the survey and on the actions that one plans to take to
address any issues.
5. An Accurate Satisfaction Index
Senior managers like to have an overall outcome from a customer
satisfaction survey – something to monitor over time. There are several
ways of obtaining this ‘satisfaction index’ but some are much less reliable
than others. One method is to include a catch-all question at the end of
the questionnaire such as: “And overall, how satisfied are you with the
products and services of…….?”
The more variables one asks people to consider when responding to a
question, the less reliable the answer is, and there are a lot of variables in
that overall satisfaction question. In reality, one do not need to ask that
catch-all question if he/ she has already asked each respondent about all
the main things that make them satisfied or dissatisfied. A second
approach would therefore be to calculate the overall average of all the
satisfaction scores. That would be better, but not ideal, because some
things are more important to customers than others, and the most
important requirements influence their satisfaction judgment more than
things that are less important. An accurate satisfaction index therefore
has to be more strongly influenced by the attributes with the highest
importance scores.
6. Provide Extensive Internal Feedback
Feeding back CSM results to employees is essential if one expects them
to take any action. The amount of feedback provided sends messages
about the importance of the customer survey to everybody in the
organization. Rather than providing superficial feedback through
newsletters, notice boards or e-mail, the results should be personally
43
presented via feedback workshops, preferably to all employees but at
least to those who have a role in delivering satisfaction. For large
organizations, feedback workshops may be costly, but will be less
expensive than the cost of failing to improve customer satisfaction.
7. Provide Accurate And Ongoing Customer Feedback
Many organizations fail to realize the potential value of feeding back the
CSM results to their entire customer base. To achieve gains in customer
satisfaction, customers must notice improvements made by ones
organization and modify their attitudes accordingly. Many organizations
take it for granted that customers will notice improvements but typically
customers have other things on their mind and often fail to notice the
changes. Once customers have noticed the changes, they still have to
modify their attitudes before they will feel more satisfied, and certainly
before they will communicate that increased satisfaction to anyone else.
The more one can do to accelerate that process, the more effective the
CSM program will be. By working on communications, one can, to an
extent, ‘talk up’ customer satisfaction. Providing feedback on the survey
results and on the actions ones organization plans to take in response to
the customers’ views is the first step in that direction.
2.5.4 Performance-Importance Analysis
Appropriate customer satisfaction measurement identifies processes that
have high impact on satisfaction and distinguishes between low performing
processes, low performance and those that are performing well.
44
Figure 3 - Performance Analysis Model
Source: Author’s representation from the study of article on Importance-
Performance Analysis by Nick Johns
2.5.5 Difficulties with Customer Satisfaction Measurement
Some of the difficulties faced while measuring CSM are listed below:-
• Poor measurement schemes
• Failure to identify appropriate quality dimensions
• Failure to weight dimensions appropriately
• Lack of comparison with leading competitors
• Failure to measure potential and former customers
• Confusing loyalty with satisfaction
45
2.6 The Profit Chain
Many organizations fail to apply adequate rigor to their customer satisfaction
research process and consequently produce misleading results. This is
detrimental even if the results are used only as a guide for service
improvement strategies but could be very damaging if they are to contribute
to strategic decisions.
In the USA, some leading companies have developed Profit Chain models
enabling them to forecast the effect on financial performance of improving
customer and / or employee satisfaction. Since customer satisfaction
measures usually occupy a pivotal place in such models, the reliability of the
measures becomes critical.
Figure 4 - The Profit Chain
Source: Customer Satisfaction Measurement: how not to do it, how to do it
and why it should be done by Nigel Hill
46
Number of leading organizations are developing the Profit Chain Model to
improve the understanding of how customer satisfaction and loyalty inter-
relate to drive business performance.
Most have also recognized that satisfied and motivated employees are more
effective in meeting customers’ requirements so models usually quantify
linkages between employee commitment, customer satisfaction, customer
loyalty and business results (typically sales and profit).
Some companies exactly know how much they need to improve customer
satisfaction to achieve specific revenue goals. e.g. Sears knows that a 5%
increase in employee satisfaction drives a 1.3% improvement in customer
satisfaction which in turn results in a 0.5% incremental revenue gain.
(Source: Customer Satisfaction Measurement: how not to do it, how to do it
and why it should be done by Nigel Hill)
2.6.1 Beyond Satisfaction
Although satisfying customers is an important goal, it is not an end in itself. It
is a means to an end. The main goal is loyal (retained) customers because
there is almost invariably a strong correlation between customer retention
and profit.
For most businesses, there is a strong correlation between satisfaction and
retention although the linkage can be distorted by factors such as lack of
product / service differentiation, extreme price competition and lack of
customer interest and involvement in the product / service category.
Satisfaction can therefore be seen as a necessary, but not always a sufficient
pre-condition for customer loyalty.
Put simply, achieving business goals depends on keeping loyal customers
whose satisfaction is assured by attracting and keeping motivated and
committed employees. Of course, the most effective models will
47
accommodate other variables that can influence the Profit Chain such as
location, intensity of competition and economic climate. However, in essence,
the main drivers of profit are usually employee commitment, customer
satisfaction and customer loyalty and customer satisfaction occupies a pivotal
position in most models. So unless customer satisfaction is measured
accurately the models will not work and decisions will be misconceived.
To ensure that ones CSM system is accurate and effective he/ she must let
the customers define the criteria to be measured, measure satisfaction and
importance across reliable samples and use importance scores to weight
satisfaction scores to produce meaningful Satisfaction Index. It is also very
helpful to communicate extensively with employees and customers before
and after the survey.
Customer wants and needs drive competitive advantage, and statistics show
that growth in market share is strongly correlated with customer satisfaction.
2.7 Importance of Customer Satisfaction and Loyalty
This section discusses the importance of customer satisfaction, loyalty and
the linkage between the two.
• “Satisfaction is an attitude; loyalty is a behavior”
• Loyal customers spend more, are willing to pay higher prices, refer
new clients, and are less costly to do business with.
• It costs five times more to find a new customer than to keep an
existing one happy.
• A firm cannot create loyal customers without first creating satisfied
customers.
Figure 5 illustrates the relationship between Customer Satisfaction and
Loyalty (customer retention) in competitive Industries.
48
Figure 5 - Relationship between CSAT and Loyalty
Relationship between CSAT and Loyalty in competitive
industries
Very satisfied
Satisfied
Very dissatisfied
Dissatisfied
Neither satisfied nor dissatisfied
0
20
40
60
80
100
Satisfaction Measure
Lo
yalt
y (
rete
nti
on
%)
Source: J.L. Heskett, W.E. Sasser Jr. and L.A. Schlesinger; The Service
Profit Chain: How Leading Companies Link Profit and Growth to Loyality,
Satisfaction and Value.
49
2.8 What is Outsourcing?
IT outsourcing is defined as the act of subcontracting part or all of a
company's IT function to one or more external vendors (Cheon, 1995 1;
Gelbstein, 20022; Grover et al., 19963).
Corporations introduced IT outsourcing until the mid-1990s chiefly to achieve
cost-effectiveness and, thus, mostly pushed ahead with computing-related
services or system integration in the form of strategic alliances (Grover et al.,
1996; McFarlan & Nolan, 19954).
However, recently, with new forms of IT outsourcing models such as ASP
having been developed, and with existing computing services or system
integration or system management outsourcing having been segmented and
specialized, appearing are new characteristics totally different from those of
traditional IT outsourcing.
1 Cheon, M.J., Grover, V., & Teng, J.T.C. (1995). Theoretical perspective on the outsourcing
of information systems. Journal of Technology Information, 10, 209-219.
2 Gelbstein, E. (2002). Outsourcing. In H. Bidgoli, et al. (Eds.), Encyclopedia of information
systems (pp. 428-430).
3 Grover, V., Cheon, M.J., & Teng, J.T.C (1996). The effect of service quality and partnership
on the outsourcing of information systems functions. Journal of Management Information
System, 72(4), 89-116.
4 McFarlan, F. W. & Nolan, R.L. (1995). How to manage an IT outsourcing alliance. Sloan
Management Review, 36(2), 9-23.
50
2.9 Why Outsourcing?
Outsourcing provides access to world-class capabilities. Outsourcing means
specialization. Outsourcing providers and specialty consulting firms can bring
worldwide, world-class capabilities and expertise to a firm. Using world-class
advertising and public relations (PR) firms is one example of how to leverage
their expertise as ones own. PR consultants also have easier access to
editors of national publications and can place articles in client-read
magazines easier.
Knowledge and expertise no longer have to be homegrown. They can be
contracted for. Outsourcing is an effective tool for redefining and re-
energizing an organization. It helps firms focus on their core business by
assuming that some or all of their noncore functions can be outsourced for
the following good reasons:-
• Outsourcing leverages relationships instead of amassing resources;
• Outsourcing delivers value instead of directing activities; and
• Outsourcing ensures results, since it is under contract.
• Outsourcing delivers quality and timeliness as one has access to
world-class and the SLAs are in place.
The effective and efficient use of a firm's marketing and business
development personnel is a major element for achieving firm-wide success.
Firm after firm has come to the realization that investing in and maintaining
inhouse specialty departments may not make dollars and sense in a
recession.
Relying upon outsourcing organizations and specialty consultants to supply
critical support services provides access to a wealth of intellectual capital
without investing in the personnel, equipment, or infrastructure. From payroll
51
processing to accounting, IT services to recruiting, market research to
corporate video presentations, outsourcing is a cost-effective method of
acquiring these services.
In weighing the benefits and costs associated with outsourcing, one may
want to consider the reasons firms commonly cite for outsourcing noncore
functions and the benefits they expect to receive, as reported by the
Outsourcing Institute (source: www.outsourcing.com).
2.9.1 Improves Firm's Focus
Outsourcing allows the firm to focus on its core activities, while support
services are assumed by an outside provider, such as a specialty consultant.
It can enable an organization to accelerate its growth and success through
expanded investment in areas offering the greatest competitive advantage. A
Web site consultant, who designs and maintains a company’s Web site, is a
good example.
2.9.2 Gains Access To World-class Capabilities
Outsourcing means specialization. Outsourcing providers and specialty
consulting firms can bring worldwide, world-class capabilities and expertise to
a firm. Using world-class advertising and public relations (PR) firms is one
example of how to leverage their expertise as ones own. PR consultants also
have easier access to editors of national publications and can place articles
in client-read magazines easier.
2.9.3 Shares The Risks
Outsourcing providers and specialty consultants makes investments not on
behalf of just one firm but on behalf of all its main clients, which eliminate the
52
necessity of investment in equipment or other inventory to provide these
special services by the firm itself.
2.9.4 Using Consultants As An Outsourcing Opportunity
Outsourcing can involve the transfer of assets from the firm to the outside
provider. Using consultants provides a valuable resource, as each is a
specialist in their particular discipline-expertise that the firm does not have to
maintain in-house. Working with specialty consultant is essential in gaining
government contracts as they are large contracts.
2.9.5 Reduces And Controls Operating Costs
Firms outsource because they do not have access to the required resources
from within, and they don't want the expense to build these resources from
the ground up. E.g. advertising firms need access to services such as
photography, printing, direct mail fulfillment, video and computer-based
presentations. While some large firms have incorporated such facilities in-
house in the past, many have since abandoned them in favor of outsourcing
them to specialty photographers, audio/video providers, printers, and mail
houses.
2.9.6 Offers Resources Not Available Internally
Access to an outside provider's lower-cost structure may result in greater
economy of scale and is one of the most compelling tactical reasons for using
outsourcing to professional services consultants. This includes services for
market research, client image surveys, client satisfaction surveys, and other
strategic research for the firm. It extends to graphic design, corporate identity,
53
annual reports, brochures, and other promotional material for an advertising
firm.
2.9.7 Indian Context Of IT Outsourcing
Due to the reasons discussed above, outsourcing has become one of the
topmost agenda for the CIOs, CEOs and business leaders of companies who
invest a substantial amount in Information processing.
India is one of main destinations for outsourcing software development and
maintenance. More and more projects are being outsourced to Indian
Software vendors.
Some of the key players in the Indian IT outsourcing space are Tata
Consultancy Services [TCS], Wipro, Infosys and HCL. These Indian software
vendors execute the software projects using a mix of onsite-offshore delivery
model. This model encompasses a very lean onsite presence and majority of
the delivery team is based at offshore. But the key is to successfully deliver
these projects and to maintain a good CSAT.
2.9.8 Challenges Faced By Outsourcing Vendors
Software vendors find it very challenging to complete projects from the
following three critical parameters standpoint:
• On time
• On budget and
• As per their customer’s requirements
54
2.10 Customer Relationship Management
To improve products and processes effectively, companies must do more
than simply fix the immediate problem. They need a systematic process for
collecting and analyzing complaint data and then using that information for
improvements. Some of the important aspects to it is listed below:-
• Accessibility and commitments
• Selecting and developing customer contact employees
• Relevant customer contact requirements
• Effective complaint management
• Strategic partnerships and alliances
• Exploiting CRM technology
It is important to assign one person as single point of contact for every
customer. This person will be called as the Customer Relationship Manager.
This person’s primary responsibility is to interface between the customer and
his company. He will be the voice of the customer [VOC] and has to manage
all his customer’s requirements, complaints and call for periodic engagement/
relationship reviews.
2.11 Customer Complaint Management
There are several off the shelf tools available for this. In the IT context, what
is required is a mechanism to take the customer complaints and based on the
severity and priority, these complaints should be attended.
There are multiple channels to log these complaints e.g. Phone, Emails,
Web-site, Call centers etc. Once the complaint is logged a token or complaint
number is generated for further reference. The Complaint logging system
typically captures the exact complaint, severity, date and time etc. These
55
complaints are categorized into Defects and New requirements/
enhancements. These different categories as discussed below.
2.11.1 Complaints
The customer complaints are categorized into following subcategories.
Level 1
If the complaint are of high severity and high priority e.g. the entire
application is down, such complaints have to be attended immediately and
may require resolution immediately. There could be penalties attached based
on the amount of downtime of the system.
Level 2
If the complaints are of medium severity, e.g. issues are seen but they don’t
impact the total functioning of the application, then such complaints needs to
be attended within 24 to 48 hours.
Level 3
If the complaints are of low severity, e.g. issues are seen in reports or on the
screen and these don’t impact the functioning of the application, then such
complaints needs to be attended within a week or so.
2.11.2 Change Requests
There could be complaints regarding additional requirements or changes to
the existing functionality. Such complaints are taken separately as they may
need estimation and appropriate approvals.
56
On a periodic basis analysis of the complaints logged is done to evaluate the
performance of the complaint handling team. The performance of this will
have a large bearing on the CSAT.
2.12 What are METRICS
Business is all about the numbers, and information technology must be part
of the equation. But the metrics used to measure the value of IT resources
and investments are changing as CIOs go beyond classic return-on-
investment and total-cost-of-ownership formulas to prioritize their
investments. Indeed, good numbers are no substitute for hard-earned
experience. Metrics can get in the way of getting customer satisfaction and
getting business done. In some cases, IT organizations are applying well-
established metrics in new scenarios. CIOs must think hard about whether
they have got the right tools to track IT's contribution to business objectives.
In CIO Effectiveness survey conducted by Optimize magazine, of more than
700 CXOs, CIOs, IT staffers, and line-of-business managers, 7 of 10 CXOs
say the most important criterion in considering the effectiveness of a CIO is
the CIO's ability to support companywide business strategies. Metrics
demonstrate that kind of support in hard-to-argue terms.
57
CHAPTER 3
LITERATURE REVIEW
58
3.0 LITERATURE REVIEW
Several articles and books pertaining to CSAT, Customer Management and
Outsourcing were reviewed during this study. Summary of some of these is
given in this section and copies of few are attached in the Annexure.
Article 1
Title /
Publisher
The Customer Satisfaction Program
Case No. 591-055 (Boston-Harvard Business School 1991).
Author H A J Menezes, J Serbin, Xerox Corporation
The case study illustrates the model used by Xerox corporation to win their
customers by providing superior proactive customer service which included
after sales training and support. They also provided a very good network of
their service centers and connectivity so that they are easily approachable
and contactable.
Article 2
Title /
Publisher
Outsource Your Noncore Functions.
(2009, February). Principal's Report, 09(2), 5-7. Retrieved
October 11, 2009, from ABI/INFORM Trade & Industry.
(Document ID: 1639890091).
Author Anonymous. Principal's Report. New York: Feb 2009. Vol. 09, Iss.
2; pg. 5, 3 pgs.
59
This article highlights why, what and when companies should outsource. It
discusses how to distinguish between core and non core function within a
company and how to identify outsourcing opportunities. The benefits of
outsourcing are listed and how to determine if a company should outsource is
discussed.
Article 3
Title /
Publisher
There can be no substitute for meeting customers' expectations.
MicroScope,18. Retrieved October 11, 2009, from ABI/ INFORM
Trade & Industry. (Document ID: 1867847791).
Author Neil Anderson. MicroScope. Sutton: Sep 7-Sep 13, 2009. pg. 18,
1 pgs.
This article recognizes the fact that the world is changing at an ever faster
pace and on many fronts. While this rapidly shifting economic landscape is
creating new business opportunities, it is also forcing companies to respond
to new client expectations. Whether this is fixing mobile technology on the
move rather than returning it to the workshop, or providing a same-day
response to servicing a printer, customer demands are growing. What this
requires is a companywide focus on customer service; a vital part of any and
all transactions. It is not just the service engineer who should be focusing on
customer service. This should be a boardroom discussion point and
corporate thinking has to be aligned (or realigned) around customer-centric
activities. There needs to be far greater awareness, for example, that service
engineers are important brand ambassadors. They can act as the eyes and
ears of a company, capable by turns of sporting trouble brewing and (just as
importantly) identifying new business opportunities
60
Article 4
Title /
Publisher
An Evaluation System for IT Outsourcing Customer Satisfaction
Using the Analytic Hierarchy Process.
Journal of Global Information Management, 13(4), 55-78.
Retrieved October 11, 2009, from ABI/INFORM Global.
(Document ID: 905476031).
Author YongKi Yoon, Kun Shin Im. Journal of Global Information
Management. Hershey: Oct-Dec 2005. Vol. 13, Iss. 4; pg. 55, 24
pgs.
This article discusses how many companies have choosing IT outsourcing in
response to complicated information systems and various internal
requirements. In order to monitor and maintain a high quality of IT
outsourcing vendors' services, it is necessary to develop a system to
evaluate IT outsourcing customer satisfaction. The system can be used as a
tool for choosing IT outsourcing providers. Through the literature reviews and
expert interviews, it has proposed the evaluation system of the IT outsourcing
customer satisfaction. Using AHP (analytic hierarchy process) technique,
attributes associated with customer satisfaction in IT outsourcing
environments are rated in terms of their importance. The customer
satisfaction evaluation system is applied to IT outsourcing service receivers
in Korea to demonstrate its practical implications.
61
Article 5
Title /
Publisher
THE MOMENT OF TRUTH.
Training Journal,75. Retrieved October 12, 2009, from
ABI/INFORM Global. (Document ID: 1857935301).
Author Margo Manning. Training Journal. Ely: Sep 2009. pg. 75, 1 pgs.
This article discusses between ‘Quality and Quantity’ which one is important.
Are they mutually exclusive?
The author conducts workshops for managers & customer service staff. At
one such workshop he once asked the participants "What are your KPIs?"
one replied that "my team, for the most part, answered all calls within the ten-
second target". The author asked: "What were the first time fix rate
statistics?" and he responded: "That was not so great; however they [the
service desk] did answer 98 per cent of their calls within ten seconds." This
could be a clear case of quantity over quality? What is the cost of choosing
quantity? To name but a few: a bad reputation, an increased volume of calls
due to unhappy return callers and, I would imagine, very frustrated service
desk engineers.
While delivering a customer care workshop to a team of managers, including
a training manager, the author posed the simple question: "How do you know
if your customer is delighted?" The responses were varied: turnover, repeat
business and the usual key performance indicators.
The article describes how the customer service staff were not aware of how
to measure customer satisfaction.
62
It concludes stating that we need to stop seeing customers as a service level
agreement and KPI. Let's start seeing them as individuals who have options,
one of which is to go elsewhere. Internal customers have options as well and
one of those is to bring in externally-sourced providers.
Article 6
Title /
Publisher
Importance-performance analysis using the profile accumulation
technique.
The Service Industries Journal, 21(3), 49-63. Retrieved October
11, 2009, from ABI/INFORM Global. (Document ID: 79168593).
Author Nick Johns. The Service Industries Journal. London: Jul 2001.
Vol. 21, Iss. 3; pg. 49, 15 pgs.
This article takes a new methodological approach to importance-performance
analysis, using the profile accumulation technique to identify service
attributes and measure importance scores. Performance scores were
measured independently, using a Likert-style questionnaire based upon the
service attributes identified previously.
The study used the profile accumulation technique (PAT) of Johns and Lee-
Ross to produce an importance-performance analysis for a professional
association. Ninety PAT forms obtained from a random sample of association
members were used to draw up a 22 item closed questionnaire, which was
administered to a different, randomly selected sample. Means from the items
on 388 returned questionnaires were used as the performance data for the
analysis. PAT scores obtained from the previous survey were employed as
the importance values, so that the resulting matrix used comparable data
from two different sources, reducing methodological bias. The technique
63
made it possible to investigate importance-performance relationships both
overall among the 22 items and for different membership segments, within an
item which showed significant differentiation between segments. The
reliability and validity of the questionnaire and of the resulting data and
implications of the study for further research are also discussed.
Article 7
Title /
Publisher
Outsourcing Trends, Metrics & Insights
Author Anonymous. PR Newswire. New York: Jun 2, 2008.
The article analyzes the insights, performance benchmarks and outsourcing
trends.
• Manage their IT Sales & Marketing support service systems and find
an adequate sourcing mix for IT Sales & Marketing support services,
including: help desk operations, hardware support, data management
and production support.
• Make the business case for their insourcing and outsourcing strategies
by providing key reference points from leading pharmaceutical
companies.
• Determine internal staffing levels required to manage vendor
relationships as well as in-house functions.
• Capture service level agreements for outsourced processes that
simultaneously manage and incent IT vendors to achieve superior
customer satisfaction.
64
Article 8
Title /
Publisher
Quality Over Quantity
Author Mary K Pratt. Computerworld. Framingham: Jul 14, 2008. Vol. 42,
Iss. 29; pg. 35, 1 pgs.
Rick Franckowiak, director of the technology office at Johnson & Johnson
Pharmaceutical Research & Development LLC (J&JPRD), led an application
support project that brought a strategic shift in how services are delivered,
trimming the number of vendors while also increasing the quality of services
and cutting costs. Much of the work needed for a successful outcome was
done internally. Franckowiak says his team also used the project to gradually
introduce offshore services and to develop and implement more detailed
metrics to measure success and customer satisfaction. The team also had to
push the vendor for continual process improvement, which the contract
specified, says application support manager Frank Drust.
Article 9
Title /
Publisher
Linking Customer Loyalty to Growth.
MIT Sloan Management Review, 49(4), 51-57. Retrieved October
11, 2009, from ABI/INFORM Global. (Document ID: 1520405501).
Author Timothy L Keiningham, Lerzan Aksoy, Bruce Cooil, Tor Wallin
Andreassen. MIT Sloan Management Review. Cambridge:
Summer 2008. Vol. 49, Iss. 4; pg. 51.
65
This article relates Customer Retention to Growth. To most senior managers,
growth is the engine of prosperity and success. Growing companies flourish;
shrinking companies die. Good managers understand that the road to growth
runs through customers not just attracting new customers, but holding on to
the ones you have, motivating them to spend more and getting them to
recommend the company’s products and services to the people they know.
However, it is one thing to believe that customers are the driving force for
profitable growth. Its altogether different to know how to measure and
manage the customer relationship effectively. Using the right customer
metrics is essential to assessing and monitoring how companies deliver for
customers and determining customers new and unmet needs. In a world
where managers are looking to analytics to help clarify their most critical
decisions, this presents a challenge: How do managers measure how
customers really feel and what they are likely to do? More importantly, what
impact can this information have on the business? The article research into
the link between Net Promoter and growth was conducted in two parts. For
the first investigation, data from a two-year study of more than 8,000
customers of companies in three industries (retail banking, mass merchant
retail and Internet services) was examined to explore the relationship
between customers survey responses and their subsequent loyalty
behaviors. Individual customer ratings on common satisfaction and loyalty
metrics were monitored over two years. In the second year of the study,
customers purchasing (retention and share of wallet) and recommendation
behaviors were also tracked.
66
Article 10
Title /
Publisher
Empirical evidence for the relationship between customer
satisfaction and business performance. Managing Service
Quality, 12(3), 184-193. Retrieved October 11, 2009, from
ABI/INFORM Global. (Document ID: 266829621).
Author Ton van der Wiele, Paul Boselie, & Martijn Hesselink. (2002).
This article focuses on the analysis of empirical data on customer satisfaction
and the relationship with hard organizational performance data. The
organization is Start Flexcompany with its headquarters in The Netherlands,
but also operating in other countries in Europe. The empirical data on
customer satisfaction and business performance stem from 1998 and 1999,
from which it can be concluded that it is possible to find evidence for the
hypothesis that there is a positive relationship between customer satisfaction
and organizational performance indicators, although the relationship is not
very strong. Various factors might influence the time-lag between a change in
customer satisfaction and an expected effect on sales margin, or other output
indicators. However, the analyses provide answers to questions related to the
quality dimensions as underlying factors behind the items in the customer
satisfaction questionnaire. Also, there are some indications for the relation
between customer satisfaction and changing behaviour of customers.
67
Article 11
Title /
Publisher
HOW TO KNOW YOUR CUSTOMER BETTER.
Inc, 31(7), 65-68. Retrieved October 12, 2009, from ABI/INFORM
Global. (Document ID: 1849312041).
Author Anonymous. Inc. Boston: Sep 2009. Vol. 31, Iss. 7; pg. 65, 4 pg.
This article emphasizes on the importance of knowing ones customer. If you
don't know your customer, then you don't know your business. You won't
know how to respond if you see changes in your sales patterns. And because
it's so hard to hang on to customers you don't know intimately, you will
forever be chasing new ones. Unfortunately, though most business owners
like to think they know their customers, many are really only guessing. And
when it comes to forecasting sales -- in fact, when it comes to virtually every
aspect of business planning -- an empirical understanding beats gut instinct
almost every time. Now is the time to get the facts. Here are five tips on how
to know your customers: 1. Identify the customers to survey. 2. Decide on a
format. 3. Probe customer satisfaction. 4. Dig for demographics. 5. Test the
survey first. You could try to resurvey, using the existing results to write more
probing and targeted questions, or you could convene a focus group.
Article 12
Title /
Publisher
HOW TO: KNOW YOUR CUSTOMER BETTER.
Inc, 31(7), 65-68. Retrieved October 12, 2009, from ABI/INFORM
Global. (Document ID: 1849312041).
Author Anonymous. Inc. Boston: Sep 2009. Vol. 31, Iss. 7; pg. 65, 4 pgs.
68
This article emphasizes on the importance of knowing ones customer. If you
don't know your customer, then you don't know your business. You won't
know how to respond if you see changes in your sales patterns. And because
it's so hard to hang on to customers you don't know intimately, you will
forever be chasing new ones. Unfortunately, though most business owners
like to think they know their customers, many are really only guessing. And
when it comes to forecasting sales -- in fact, when it comes to virtually every
aspect of business planning -- an empirical understanding beats gut instinct
almost every time. Now is the time to get the facts. Here are five tips on how
to know your customers: 1. Identify the customers to survey. 2. Decide on a
format. 3. Probe customer satisfaction. 4. Dig for demographics. 5. Test the
survey first. You could try to resurvey, using the existing results to write more
probing and targeted questions, or you could convene a focus group.
Article 13
Title /
Publisher
Relationships and impacts of service quality, perceived value,
customer satisfaction, and image: an empirical study. The Service
Industries Journal, 29(2), 111. Retrieved October 11, 2009, from
ABI/INFORM Global. (Document ID: 1648537101)
Author Hu, H., Kandampully, J., & Juwaheer, T.. (2009).
This article discusses how in today's world of intense competition, satisfying
customers is only the base line and may not be sufficient for survival.
Management should focus on gaining customer loyalty by enhancing
customer perceptions of service quality and increasing as perceived by the
consumer value. Although previous studies have addressed the importance
of service quality, satisfaction, perceived value, and image, the precise
69
nature of the relationships that exist between these constructs and the
understanding of their effect on customer behaviour still remains a key issue.
This empirical study seeks to understand the relationships that exist between
service quality and perceived value and how they impact customer
satisfaction, corporate image, and behavioural intentions. The proposed
model indicates that delivering high quality service and creating superior
customer value can result in achieve high customer satisfaction, thus
effecting the firm's corporate image, and ultimately leading to consumer
retention.
Article 14
Title /
Publisher
Methods for determining areas for improvement based on the
design of customer surveys. The Service Industries
Journal, 29(2), 143. Retrieved October 11, 2009, from
ABI/INFORM Global. (Document ID: 1648537121).
Author Yang, C., Yang, K., Yeh, T., & Pai, F.. (2009).
This article emphasizes on the importance of designing of customer surveys.
Many firms conduct satisfaction surveys of their customers, with a view to
using the analyzed results to identify areas of potential improvement. The
weakness in this approach is that attributes with a lower satisfaction level
might not be in need of improvement. They might merely be of little concern
to customers. The present research develops several methods to identify
areas that do require improvement. These methods include the importance-
satisfaction (I-S) model, the improvement index, the quality-improvement
index, the SERVQUAL survey, Kano's model, the extent of satisfaction, and
the extent of dissatisfaction. An integrated model of service-quality
measurement is used to cover all the customer surveys required for an
international certification service company. In the case study, valuable
70
information of various kinds is obtained from an analysis of the customer
surveys. On the basis of this information, the firm can make appropriate
decisions on areas for improvement, which are significantly different from
those identified on the basis of a simple satisfaction survey.
71
CHAPTER 4
OBJECTIVES OF THE STUDY
72
4.0 OBJECTIVES OF THE STUDY
This current research study is an attempt to create a framework for
evaluating the health of a customer engagement for an IT outsourcing
provider. This tool will encompass quantitative and qualitative indicators.
The objectives of the research are:-
• To understand Customer Satisfaction.
• To understand the relationship between CSAT and loyalty.
• To create a CSAT survey tool for Qualitative measurement of
Customer Satisfaction (CSAT score).
• To define key metrics for quantitative measurement to support the
above qualitative CSAT score.
• To study the relationship between project performance and CSAT.
The combination of the two would be used for arriving at the health of the
customer engagement and hence manager the customer better.
73
CHAPTER 5
RESEARCH METHODLOGY
74
5.0 RESEARCH METHODOLOGY
Research is a process involving a number of activities which could be carried
out either in a sequential or in parallel. The Research depends on various
factors like the person under taking the research, his ability, research topic
time available and budget allotted.
5.1 Research Overview
The research encompassed designing an in-depth CSAT survey
questionnaire. The CSAT survey was piloted on a couple of managers to
validate and refine the questions of it. This survey was administered to the
selected sample after careful explanation of the purpose of the survey.
These managers were given two weeks time to revert back and their
completed responses are tabulated as in Table 5. This CSAT scores in this
table was studied and analyzed. The weak areas as perceived and as per the
scores given by the customer were identified so that the action plan for their
improvements could be worked out.
The current research followed a very structured methodology. The process
followed is as shown in Figure 6.
75
Figure 6 - Research Methodology
The figure below illustrates the general steps involved in this research.
The stages involved in this research are as follows:-
, , , , QuestionnaireQuestionnaireQuestionnaireQuestionnaire
76
5.2 Research Design
During the research design stage it was spelt out how the research work will
be carried out. This was at a conceptual structure within which the research
was to be conducted. The following were identified: method of data collection,
the time, the frequency, other resources available, the financial implications
etc. These were reviewed at each stage to validate and verify if it needs to
under any changes as the study progresses.
The following subsections discuss the various parameters handled during the
research design phase.
5.2.1 What is the study about?
One of the challenges faced by the project teams working of IT outsourcing
companies on projects is what the CSAT for their project is. This is one of the
criteria for measuring the performance of the project teams. In the absence of
any formal technique subjective assessments are made which was based on
the perception and anecdotal information, and more often the latest incident
would determine the customer satisfaction.
The current study is an attempt to create a framework using a mix of
objective and subjective assessment, to measure CSAT for customers who
outsourced their application maintenance and development to IT outsourcing
companies, using the project Metrics data and CSAT Survey tool.
5.2.2 Why is the study being made?
This study will help in measuring and managing customer satisfaction using
operations data and not depend of the customer’s perception and anecdotal
references. It will have qualitative and quantitative data.
77
5.2.3 What type of data will be collected and where would this
data be collected?
There were two types of data identified for data collections:
• CSAT survey responses
• Project metrics data
As the part of the study, Customer survey questionnaire which was created
with relevant questions will be used to compute the CSAT score to indicate
the satisfaction level of various project owners at the customer’s side. The
survey also provides a section for the customers to give a descriptive opinion
about the services they received and improvement areas expected by them.
The CSAT was planned to be administered once every quarter to the each of
the customer managers. The sample CSAT (blank) form is shown in the
Annexure.
The consolidated CSAT responses from the selected sample of customer
managers to whom the survey was administered are attached under the
section ‘Data Collection’. The overall CSAT score was computed and this is
shown under the ‘Data Analysis’ section.
In addition to the CSAT survey, the minimum metrics that will be required to
indicate the health of each project in terms of the Schedule, Budget and
Quality was identified to be captured on monthly basis. These metrics are the
three main parameters any customer would be very keen to monitor and
would provide quantitative indicator about the project health.
The metrics data from the projects was planned to be collected on a monthly
basis so that the monthly metrics can be computed and shared with the
customer managers during the monthly project status reviews. The monthly
data captured is attached under the section ‘Data Collection’.
78
The first one month of the study comprised of:
• Identifying the minimum set of metrics that would be required
• Creating the questionnaire for CSAT.
The first task was Identifying the minimum set of metrics that would be
required and more important it should be possible to collect from various
projects. Several metrics were listed out, but some of these had process
overhead to capture Hence the metrics that were taken up for the first pass
were as follows.
• Schedule Variance = Planned End date – Actual End date
• Effort Variance = Planned effort – Actual effort
• Internal Defects = Defects logged by internal QA team
• External Defects = Defects logged after delivery made to customer
• Defect Removal Efficiency % = (No. of Defects found in Internal
testing / Total No. of Defects found in Testing) * 100
Schedule Variance is the difference between the planned end date and
actual end date of the project.
Effort Variance is the difference between the planned effort and actual effort
of the project.
Internal Defects are the defects caught during internal testing of the
application.
External Defects are the defects reported after delivery of the application.
79
Defects Removal Efficiency indicates the effectiveness of the internal QA
process and process capability to capture as much defects as possible
before the release of the application.
5.2.4 Why was the above metrics taken up?
The first most important requirement for any project was to deliver the
customer projects on the agreed date. i.e. as per the agreed schedule. As
there may be dependent tasks planned by the customer post the delivery of
the project by the IT outsourcing provider, this may have a huge impact on
the customer’s business. Hence Schedule Variance was selected for
tracking.
The next most important requirement was to deliver the project with near
Zero defects (as it is very difficult to have a Zero defect product). Hence it
was decided to track the defects reported. To measure the effectiveness of
the internal QA and to track the defects leaked to the customer it was
decided to distinguish between the defects caught during internal testing of
the application i.e. Internal Defects and the defects reported after delivery of
the application i.e. External Defects. The aim is to deliver the application to
the customer with very less defects as possible. The metric Defects Removal
Efficiency was used to measure the effectiveness of the internal QA process
and process capability to test / capture as much defects as possible and have
minimal leakage of defects to the customer.
The third most important requirement for any project to make business sense
was to complete within the agreed budgets. Projects start with a mutually
agreed budget. Efforts form a major component of the budget. The efforts for
the project would basically be the sum of efforts put in by the total number of
people assigned to the project. It is important for the company to complete
80
the project within the agreed budget to make business sense. Hence Effort
Variation was selected for tracking.
5.2.5 Creating the questionnaire
This encompassed study of few CSAT survey templates and arriving at an
appropriate one. While doing so it was very important to have the appropriate
Service parameters for the survey questionnaire. These parameters should
help in arriving at an objective assessment of the CSAT.
All the dimensions for service quality that will impact customer satisfaction
needs to be considered for CSAT questions.
The dimensions considered for the CSAT questionnaire was required to
broadly cover the following areas:
• How do we perform in Planning & Control activities?
• How is the Quality of our deliverables?
• Do we have adequate Business Knowledge?
• Do we have adequate Technical Knowledge?
• Do we have the Communication Channels setup with the customer to
discuss project related matters?
• Is our Team cohesive and work as a team to solve customer issues?
• Do we have customer focus?
Each of the above dimensions was further discussed below.
1. Under ‘Overall Planning & Control’ the following parameters were
identified.
81
• Meeting agreed deadlines
• Change Management
• Issue Management
2. Under ‘Overall Quality of Deliverables’ the following parameters were
identified.
• Analysis and Design
• Deliverables for Acceptance Testing
• Implemented Systems and Enhancements
3. Under ‘Overall Business Knowledge’ the following parameters were
identified.
• Knowledge/Understanding of Customer Business
• Awareness of Customer issues/problems
• LTPL Team produces quality work
4. Under ‘Overall Technical Knowledge’ the following parameters were
identified.
• Vendor’s Team display adequate system knowledge.
• Vendor’s Team is proficient in all system technical areas.
5. Under ‘Overall Communications’ the following parameters were
identified.
• Effective Project Status Reporting
• Effective Project Progress meetings
• Accessibility of vendor’s team
82
• Vendor’s team maintained professional approach throughout the
project
6. Under ‘Overall Teamwork’ the following parameters were identified.
• Vendor functions as part of your team
• Common and recognized goals understood by everybody
• Feeling of team spirit and partnership
• Vendor’s Team inspires trust & confidence
7. Under ‘Overall Customer Focus’ the following parameters were
identified.
• Respond to issues in an expected time frame.
• Pro-active in identifying problems & improvements
• Added value provided by the Vendor
A 5 point scale was used for the measurement of the various parameters.
The questionnaire was designed after initial discussion with concerned
officials of a couple of IT vendors. The questionnaire was then pilot tested
against a few respondents before the actual administration.
The blank CSAT template is attached in the Annexure.
83
5.2.6 What is the period of study?
Once the research topic was identified, the period of the study was approx.
thirteen months. i.e. August 2007 to May 2009. The detailed timeline is
shown in Table 4.
This timeline included:
• Defining the data to be collected i.e. Metrics and CSAT questionnaire
• Creating the format for the data capture i.e. Microsoft Excel based for
Metrics and Microsoft word based CSAT survey form)
• Collecting the monthly metrics data,
• Collecting the completed CSAT survey questionnaire and
• Providing a summary of the overall CSAT.
84
Table 1 - Timeline for the execution of this research
Timeline ����
���� Tasks
Aug
2007
Sept
2007
Oct
2007
Nov
2007
Dec
2007
Jan
2008
Feb
2008
Mar
2008
Apr
2008
May
2008
Jun
2008
Aug
2008
Sep
2008
Research problem formulation 30
Literature survey (WIP) 120
Preparing the Research design 40
Sample design 30
Data collection 140
Data analysis 40
Interpretation 15
Report preparation 80
85
5.3 Sample Design
Criteria for selecting the sample for CSAT survey and for collecting project
metrics were as follows:
• For administering the CSAT survey, all the customer managers who
outsourced work to IT vendors formed the population.
• For collecting the project metrics data, all the projects executed for the
above customers formed the population.
The exact sample used for the LTPL study is given in section 6.2
5.4 Data Collection
In the current study, there are two types of data that was collected i.e. CSAT
responses and project Metrics data. Both these are primary data.
CSAT was administered to all the twelve customer managers every quarter.
Their responses for two quarters are attached in Table 5 and Table 6 and these
were used for analysis in this study.
Project Metrics data was collected every month. The five identified metrics was
collected every month for all the projects executed for the customer during that
month. This data was collected by the 5th of the following month. The monthly
metrics are attached in the Annexures.
The project metrics data for the months of August, September, October,
November and December 2007 was used for the analysis in this study.
86
CHAPTER 6
CASE STUDY OF LIONBRIDGE
87
6.0 CASE STUDY OF LIONBRIDGE
For the purpose of this study, a Case study of Lionbridge Technologies Pvt. Ltd.
was taken.
6.1 About LIONBRIDGE
Lionbridge Technologies Pvt., Ltd., is the Indian subsidiary of Lionbridge
Technologies Inc., a software services company based out of Waltham, in the
USA. Lionbridge is the leading provider of translation and localization services.
Company Overview
Lionbridge Technologies, Inc., listed in NASDAQ stock exchange, is a provider of
globalization and testing services. Lionbridge combines global onshore, near
shore and off shore resources with proven program management methodologies
to serve as an outsource partner throughout a clients product and content
lifecycle - from development to globalization, testing and maintenance.
Lionbridge was founded with a vision to enable international market success for
our clients. Today, with a presence in more than 26 countries, it is drawing on
this heritage and local knowledge to achieve a significant market advantage.
Organizations in the United States, Europe, and Asia see globalization as a key
to their success and are making Lionbridge their partner of choice. To see the
positive market response to its global scale and technology-based delivery
model, one need only look at several of their top accounts–from leading Internet
search companies to mobile device manufacturers. More than 20 clients spend
more than $5 million each with Lionbridge annually. Lionbridge has signed
several new, multi-million dollar clients including engagements with a leading
88
U.S. financial institution, a multinational pharmaceutical company, a large
European industrial organization, and an international travel company.
As the leader of its industry, Lionbridge enjoys the privilege of working with the
world’s most innovative organizations. Each year, more than 450 companies rely
on Lionbridge to accelerate their delivery of applications and content across
international markets.
Company Facts
Some of the important company facts are given below.
• Founded: 1996
• Headquarters: Waltham, Massachusetts, United States
• Employees: 4,500
• Locations: Solution Centres in 26 countries
• Revenue: $452 million in 2007 and $461 million in 2008
Liobridge had two main SBUs viz.:
• Translation, Localization and Content development Services:
• Development and Testing Services
A. Translation and Localization Services
Lionbridge is the leading provider of translation and localization
services. Lionbridge helps global companies meet their customers’ language
and cultural preferences by adapting a wide range of deliverables/ services as
in the Table 1.
B. Development and Testing Services
89
In addition to language services, Lionbridge also specializes in development
and testing. The development solutions include application development and
maintenance, product engineering, and content development.
Lionbridge’s award-winning testing services, offered under the VeriTest®
brand, ensure the quality, interoperability, usability, and performance of
software, hardware, Web sites, and content. VeriTest also provides third-party
product certification programs for major global technology platforms.
A key differentiator for Lionbridge is its global footprint, which enabled
Lionbridge to deliver superior service to customers through local contacts and
resources.
Lionbridge employs more than 4,600 specialists, including linguists, project
managers, engineers, subject matter experts, content developers, and quality
assurance professionals, in 26 countries.
Lionbridge works with a network of 25,000 independent translation partners.
An additional 1,600 development and testing professionals work in its solution
centers in India, China, and Eastern Europe.
Representative Clients
The list of representative clients of Lionbridge is given in Table 2.
Industry Focus
The industry focus sectors of Lionbridge are as given in Table 3. Lionbridge has
significant experience in these sectors.
Areas of Expertise
Lionbridge specialize in delivering the following services around the world:
90
• Translation and Localization
• Internationalization
• Interpretation
• Testing
• Product Certification
• eLearning and Content Development
Table 2 – Services provided by Lionbridge
• Products
• Documentation
• Web Sites
• Software
• Marketing Materials
• Multimedia
• Training
• eLearning Content
Source: Author’s analysis of Lionbridges’ data
Table 3 – Representative client list of Lionbridge
• Computer Associates
• Sony Ericsson
• Expedia
• Golden Living
• HP
• IBM
• EMC
• Merck
• Microsoft
• Motorola
• Nokia
• Oracle
• Pearson
• Porsche
• Thomson
• Volvo
Source: Author’s analysis of Lionbridges’ data
91
Table 4 – Industry focus of Lionbridge
• Automotive
• Consumer
• Education
• Financial Services
• Government
• Life Sciences
• Manufacturing
• Technology
• Telco/Mobile
Source: Author’s analysis of Lionbridges’ data
It was decided jointly along with the Lionbridge’s senior management, to select
one customer to conduct the CSAT survey. Based on the experience of this
exercise, further decision will be taken whether to extend this to other customers.
One of Lionbridge’s large customer was selected for the research study. This
customer was a very large player in the education sector in USA and UK and has
several software products which catered to the schools. This customer
outsourced their application maintenance and development to Lionbridge. The
health of this customer engagement was studied using the project Metrics data
and CSAT Survey tool.
The value of the outsourced product maintenance & development work to
Lionbridge was approx. $16 million annually. Lionbridge deployed about 425 staff
members to work on multiple projects for this customer.
92
The customer deputed 12 product managers to oversee the outsourced work,
few of them were based in USA and few in UK. Their good references to other
managers in their company would recommend new work to be outsourced to
Lionbridge. Lionbridge technologies Pvt. Ltd. [LTPL], the Indian subsidiary is the
fulfillment section of Lionbridge Technologies Inc.
6.2 Sample selection
The target population to be covered under this study was jointly decided along
with the senior management of LTPL. The selected customer had twelve
customer managers who were on day to day interaction with the LTPL project
teams. All the twelve customer managers were selected for administering the
CSAT survey. These twelve respondents were contacted. The questionnaire was
administered quarterly over the period of time. For this study purpose data from
two quarterly CSAT responses were taken.
For the project metrics data, all the projects executed monthly during August and
December 2007 were taken as sample.
93
CHAPTER 7
DATA ANALYSIS & FINDINGS
94
7.0 DATA ANALYSIS AND FINDINGS
7.1 Data Analysis
The CSAT responses from each of the twelve customer managers for two
quarters were tabulated separately and studied in detail for each of the
parameters.
The parameters which got rated 1 and 2 have been highlighted and are the
improvement areas.
The identified five project metrics for each of project which was under execution
during the identified month were tabulated. Control limits were set for each of the
five metrics as in table 7.
Metrics within acceptable limits were shown in Green legends. Metrics which
needs to be under observation were shown in Yellow legend. Metrics which were
under Amber were in danger and needed immediate attention.
7.1.1 Analysis of the CSAT Responses
The individual CSAT responses were captured into an EXCEL sheet. The
subtotals were generated for each major parameter and finally the average score
was arrived. This final average is the CSAT score. This was the overall CSAT for
the period.
Table 5 is the CSAT consolidate responses for the CSAT survey administered to
twelve customer managers in September 2007. The names of the CSAT
respondents have been withheld to maintain confidentiality. The scores of 2 and
95
1 are areas of improvement. The CSAT score obtained in the September 2007
survey was 3.29.
Table 6 is the CSAT consolidate responses for the CSAT survey administered to
twelve customer managers in November 2007. The scores of 2 and 1 are areas
of improvement. The CSAT score obtained in the Nov 2007 survey was 4.00.
The interpretation of the CSAT scores is as follows:
5 - Consistently surpasses expectations
4 – Exceeds expectations
3 – Always meets expectations
2 – Occasionally does not meet expectations
1 – Frequently does not meet expectations
96
Table 5 - CSAT Survey responses for Sep 2007
Parameter Responses from 12 CSAT respondents
Sr.
No.
1 2 3 4 5 6 7 8 9 10 11 12
1 Meeting agreed deadlines 4 4 3 2 4 2 4 4 4 4 4 3
2 Change Management 4 3 2 2 3 3 4 4 4 4 3 3
3 Issue Management 4 3 2 2 2 1 3 4 4 4 3 3
Overall Planning and Control 4 3 2 2 3 1 3 4 4 4 3 3
4 Analysis and Design 1 5 1 1 3 2 2 3 4 4 3 3
5 Deliverables for Acceptance Testing 3 3 3 2 3 3 4 3 4 4 4 3
6 Implemented Systems and Enhancements 3 4 3 2 3 3 3 3 4 4 3 3
Overall Quality of Deliverables 3 4 3 2 3 3 4 3 4 4 4 3
7 Knowledge/Understanding of Customer Business 2 4 2 3 2 1 3 3 4 4 3 3
8 Awareness of Customer issues/problems 3 4 1 2 3 1 4 3 4 4 3 2
9 Team produces quality work 3 4 2 2 3 2 3 3 4 4 3 3
Overall Business Knowledge 2 4 2 3 3 1 3 3 4 5 3 2
10 Respond to issues in an expected time frame. 3 4 3 3 3 2 4 3 4 5 3 3
11 Pro-active in identifying problems &
improvements 2 4 1 1 2 2 4 3 4 5 2 3
12 Added value provided by Lionbridge 2 4 3 2 2 3 4 3 4 5 4 3
Overall Customer Focus 2 4 3 3 3 2 4 3 4 5 3 3
13 Effective Project Status Reporting 3 4 3 3 3 2 2 4 4 5 3 3
14 Effective Project Progress meetings 3 4 3 3 2 2 3 4 4 5 3 3
97
15 Team accessibility 4 3 2 5 5 1 4 4 4 5 3 3
16 Team maintained professional approach 4 4 3 4 4 3 5 5 4 5 4 4
Overall Communications 3 4 3 3 3 2 4 4 4 4 3 3
17 Liobridge functions as part of your team 3 4 3 5 4 2 5 4 5 4 4 3
18 Common and recognised goals understood by
everybody 2 4 3 4 4 3 3 3 4 4 4 3
19 Feeling of team spirit and partnership 3 4 3 4 3 3 4 4 5 4 4 3
20 Lionbridge Team inspires trust & confidence 3 5 2 2 3 2 4 3 4 4 3 3
Overall Teamwork 3 5 3 4 3 3 4 3 4 5 4 3
21 Lionbridge Team display adequate system
knowledge. 2 4 1 4 3 2 3 3 4 5 4 3
22 Lionbridge Team are proficient in all system
technical areas. 1 4 1 1 3 2 2 3 3 5 2 2
Overall Technical Knowledge 2 4 1 1 3 2 3 3 4 5 3 3
Overall Rating for Lionbridge (CSAT score) 3 4 3 2 3 2 3.5 4 4 5 3 3
Avg. CSAT score 3.29
98
Table 6 - CSAT Survey responses for Nov 2007
Parameter Responses from 12 CSAT respondents
Sr.
No.
1 2 3 4 5 6 7 8 9 10 11 12
1 Meeting agreed deadlines 4 4 3 2 4 2 4 5 4 4 4 4
2 Change Management 4 3 2 2 3 3 4 4 4 4 4 3
3 Issue Management 4 4 2 2 2 2 3 5 4 4 3 4
Overall Planning and Control 4 4 2 2 3 2 3 5 4 4 4 4
4 Analysis and Design 2 3 2 2 3 3 3 4 4 4 4 4
5 Deliverables for Acceptance Testing 3 4 3 2 3 3 4 4 4 4 4 4
6 Implemented Systems and Enhancements 3 4 3 2 3 3 3 4 4 4 3 4
Overall Quality of Deliverables 3 4 3 2 3 3 4 4 4 4 4 4
7 Knowledge/Understanding of Customer Business 3 4 2 3 3 3 3 4 5 5 4 4
8 Awareness of Customer issues/problems 3 4 2 2 3 3 4 4 5 4 4 3
9 Team produces quality work 3 4 2 2 3 3 3 4 5 5 4 4
Overall Business Knowledge 3 4 2 3 3 3 3 4 5 5 4 4
10 Respond to issues in an expected time frame. 3 4 3 3 3 3 4 5 5 5 4 4
11 Pro-active in identifying problems &
improvements 3 4 2 2 3 3 4 4 4 5 3 4
12 Added value provided by Lionbridge 3 4 3 2 3 3 4 5 5 5 4 3
Overall Customer Focus 2 4 3 3 3 3 4 5 5 5 4 4
13 Effective Project Status Reporting 3 4 3 3 3 2 3 4 4 5 3 3
99
14 Effective Project Progress meetings 3 4 3 3 2 3 3 5 5 5 4 4
15 Team accessibility 4 3 3 5 5 3 4 5 5 5 4 4
16 Team maintained professional approach 4 4 3 4 4 3 5 5 5 5 4 4
Overall Communications 3 4 3 3 3 3 4 5 5 4 4 4
17 Lionbridge functions as part of your team 3 4 3 5 4 3 5 4 5 4 4 3
18 Common and recognized goals understood by
everybody 3 4 3 4 4 3 3 4 5 5 4 4
19 Feeling of team spirit and partnership 3 4 3 4 3 3 4 5 5 5 4 4
20 Lionbridge Team inspires trust & confidence 3 4 2 4 3 3 4 4 4 4 3 4
Overall Teamwork 3 5 3 4 3 3 4 5 5 5 4 4
21 Lionbridge Team display adequate system
knowledge. 3 4 3 4 4 3 3 5 5 5 4 4
22 Lionbridge Team are proficient in all system
technical areas. 3 4 3 3 4 3 3 5 4 5 4 4
Overall Technical Knowledge 3 4 3 3 3 3 3 5 5 5 4 4
Overall Rating for Lionbridge 3.5 4.5 3 3 4 3 4 5 5 5 4 4
Avg. Rating 4.00
100
The second dataset used was the metrics for the Reporting period i.e. monthly
data from August to December 2007.
7.1.2 Analysis of Metrics Data
The metrics data for the five reporting months i.e. August to December 2007 was
analyzed and the trend for the % of projects in Amber (refer to Table 14) with
respect to the schedule variance were plotted as indicated in the Figure 7.
Figure 7 - Trends of the projects in Amber.
Source: Author’s analysis
The downward trend line indicates that the percentage of the projects in Amber is
reducing which is good sign. The upward trend line of the projects in Amber
indicates that attention is needed and there are issues in the project execution.
101
The trend shown in table 7 indicates a downwards trend hence with respect to
Project schedules, most of the projects are faring well within the control limits
month on month.
7.2 Findings From The Study
This sections summarizes the findings of the CSAT scores and Project Metrics
data from the Study.
• The CSAT score obtained in the September 2007 survey was 3.29.
• The CSAT score obtained in the November 2007 survey was 4.00.
• Downward trend shown for the number of projects in Amber with respect
to Schedule Variance. This means that the number of projects are getting
delivered on time is improving month on month.
The above indicates that whenever the project deliveries are made on time, the
CSAT scores for that period are better. This means that there is a positive
relationship between these CSAT score and project metrics.
102
CHAPTER 8
CONCLUSIONS &
RECOMEDATIONS
103
8.0 CONCLUSIONS AND RECOMMENDATIONS
This current research study is an attempt to create a framework for evaluating
the health of a customer engagement for an IT outsourcing provider. The
framework will encompass quantitative and qualitative indicators. This
encompassed creation of survey questionnaire to measure CSAT and
identification of the project Metrics data.
Based on the above, the key objective of the research was to investigate the
relationship between the quantitative project Metrics data and qualitative
measurement of CSAT.
This will be used for arriving at the health of the customer engagement.
CSAT Score
The Overall Customer Satisfactions score which is a finite number between 1
and 5 which is quantifiable will be taken as the CSAT Index. As this is a finite
number, it gives an indication of the overall customer satisfaction. Hence the
CSAT will not depend on the perception of the person concerned.
The Parameters which got rated at 1 and 2 were the improvement areas.
The CSAT is planned to be administered to key customer managers once every
quarter. The overall CSAT score was planned to be shared with the customer.
Metrics Data
The metrics were captured monthly and shared with the customer in the monthly
reviews so as to reflect the status of the projects. The metrics trend for 3
consecutive months will indicate the health of the projects and this along with the
CSAT score gives an objective indicator of the health of the engagement. This
104
health indicator is from live projects and hence will not be based on the
perception of the manager concerned and also not be completely biased on any
recent incident on the projects.
This has helped in managing the customer perceptions to a large extent.
The metrics can be further refining and improvements so that it suits all the
customer’s requirements.
This research involved usage of CSAT Survey to provide qualitative indicators
and defined few critical metrics to provide quantitative measures. It also involved
study of the metrics data for supporting the CSAT scores. Together it helped in
arriving at the quantitative Customer Satisfaction indicator.
8.1 Summary Of Objectives And Outcomes
The table below is the tabulation of each of the objectives for which this study
was conducted and the outcome for each of these.
Table 7 - Summary of Objectives And Outcomes
No. Objectives Outcome
1 To understand Customer
Satisfaction
Through the article reviews, Satisfaction is
customers’ evaluation of a product or
service in terms of whether that product or
service has met their needs and
expectations. Failure to meet the needs
and expectations is assumed to result in
dissatisfaction with the product or service.
2 To understand the Through the literature reviews and
105
relationship between CSAT
and loyalty
validation using project data, it can be
easily established that there is a very
strong linkage between CSAT and loyalty.
3 To create a CSAT survey
tool for Qualitative
measurement of Customer
Satisfaction (CSAT score)
A framework for measuring CSAT was
created. This framework was used to arrive
at the Overall Customer Satisfactions score
which is a finite number between 1 and 5, is
quantifiable and will be taken as the CSAT
Index. As this is a finite number, it gives an
indication of the overall customer
satisfaction. Hence the CSAT will not
depend on the perception of the person
concerned.
Quarterly CSAT surveys were conducted
and analysis of the findings of two such
surveys was conducted in this study.
4 To define key metrics for
quantitative measurement to
support the above qualitative
CSAT score.
Minimum set of metrics that would be
required for any project was identified. The
trend of these metrics indicates the
performance of the delivery teams. Hence
this information will be backup information
for the CSAT score.
The five key metrics identified in this study
were: Schedule Variance, Effort Variance,
Internal Defects, External Defects and
DRE.
5 To study the relationship The first CSAT survey in Sep 2007 score
was 3.29 and the same for Nov 2007 was
106
between Project
performance and CSAT
4.00. The project performance in the above
time shows that an improvement in the
schedule variance metric.
This means that when project deliveries are
met, the CSAT scores are better. Hence
there is a positive relationship between
these two parameters.
8.2 Limitations Of Current Research
The population of the customers of any IT sourcing company will be limited and
hence the getting the ideal sample size will be an issue.
The population for the projects for the customer for whom the CSAT is being
conducted should be big. This may not be possible most of the time. Hence
setting trends and control limits may be difficult.
8.3 Recommendations
The framework created in this study can be used for assessing the health of the
customer engagement. In the current study the Schedule variance is used for
analysis. The other four metrics can analyzed for assessing the project
performance. These project metrics need not be limited to five. These could be
further extended based on the project needs. More metrics can be added and
one such example is ‘Turn around Time’ or TAT. This metrics is very important
for Support type of projects.
107
ANNEXURES
108
ANNEXURE 1 - CSAT
QUESTIONNAIRE TEMPLATE
109
CSAT QUESTIONNAIRE TEMPLATE
The template used for the CSAT survey is given below.
CUSTOMER SATISFACTION SURVEY
CUSTOMER INFORMATION
Name : ________________________ Project:________________________
Date: dd/mm/yyyy
Dear Manager,
Providing excellent service is important to us, and we would like to ask you to
take a few minutes of your time and provide us some feedback on our recent
performance. The enclosed Customer Satisfaction Survey is intended to provide
an opportunity for LTPL to measure current satisfaction levels. Information from
this survey will be used to develop and implement improvements where required.
Your feedback is a key element in helping us improve our operating
performance.
110
The survey has been designed to evaluate our performance in Planning, Quality,
Business Knowledge, Customer Focus, Communications, Teamwork, and our
Technical Ability.
As a key customer of LTPL, we request you to complete the questionnaire and
provide your valuable feedback. There is a section for you to provide any
additional comments that you feel are appropriate. Your response will help us to
serve you better.
Please return your completed survey by close of business on, DD/MM/ YYYY.
Thank you for supporting our improvement process.
111
LIONBRIDGE PERFORMANCE
For each of the following statements, please indicate the extent to which
Lionbridge has performed to your expectations by underlining the relevant
number. Please use the last page of this survey for comments regarding
Lionbridge's performance.
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines N/A 1 2 3 4 5
Change Management
Issue Management
Overall Planning and Control
Analysis and Design
Deliverables for Acceptance Testing
Implemented Systems and Enhancements
Overall Quality of Deliverables
Knowledge/Understanding of Customer Business
Awareness of Customer issues/problems
Lionbridge Team produces quality work
Overall Business Knowledge
Respond to issues in an expected time frame.
Pro-active in identifying problems & improvements
Added value provided by the Lionbridge
Overall Customer Focus
Effective Project Status Reporting
Effective Project Progress meetings
Lionbridge Team accessibility
Lionbridge Team maintained professional approach throughout the
project
Overall Communications
112
Lionbridge functions as part of your team
Common and recognized goals understood by everybody
Feeling of team spirit and partnership
Lionbridge Team inspires trust & confidence
Overall Teamwork
Lionbridge Team display adequate system knowledge.
Lionbridge Team is proficient in all system technical areas.
Overall Technical Knowledge
Overall Rating for Lionbridge
113
COMMENTS
Please use the following space for any additional comments regarding your
experience with Lionbridge. Your specific comments are extremely important to
our continuous improvement efforts.
Thank you for your time and co-operation in completing this survey.
114
ANNEXURE 2 - CSAT
QUESTIONNAIRE RESPONSES
115
CSAT QUESTIONNAIRE RESPONSES
The CSAT survey was administered to the twelve customer managers. Their
names have not been disclosed due to confidentiality.
The individual responses of the customer managers are listed below.
PARTICIPANT 1
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 4
Change Management 4
Issue Management 4
Overall Planning and Control 4
Analysis and Design 1
Deliverables for Acceptance Testing 3
Implemented Systems and Enhancements 3
Overall Quality of Deliverables
Knowledge/Understanding of Customer Business 2
Awareness of Customer issues/problems 3
Lionbridge Team produces quality work 3
Overall Business Knowledge 2
Respond to issues in an expected time frame. 3
Pro-active in identifying problems & improvements 2
Added value provided by the Lionbridge 2
Overall Customer Focus 2
Effective Project Status Reporting 3
Effective Project Progress meetings 3
Lionbridge Team accessibility 4
116
Lionbridge Team maintained professional approach throughout the
project
4
Overall Communications 3
Lionbridge functions as part of your team 3
Common and recognized goals understood by everybody 2
Feeling of team spirit and partnership 3
Lionbridge Team inspires trust & confidence 3
Overall Teamwork 3
Lionbridge Team display adequate system knowledge. 2
Lionbridge Team is proficient in all system technical areas. 1
Overall Technical Knowledge 2
Overall Rating for Lionbridge 3
COMMENTS
I thought the survey needed improvement. Some of the line items leading up to
the Overall rating line seemed to be for the wrong topic.
The biggest issue at the PDC for NovaNET is on the technical side.
The PDC’s problem solving ability for technical issues falls short.
The PDC is not strong on analysis skills for problem solving and/or design.
It is often difficult to have a technical conversation with the PDC because of the
mismatch in skill sets between MDC and PDC.
There is a lack of technical leadership – the emphasis is on management,
planning and control, and communication.
117
PARTICIPANT 2
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 4
Change Management 3
Issue Management 3
Overall Planning and Control 3
Analysis and Design 5
Deliverables for Acceptance Testing 3
Implemented Systems and Enhancements 4
Overall Quality of Deliverables 4
Knowledge/Understanding of Customer Business 4
Awareness of Customer issues/problems 4
Lionbridge Team produces quality work 4
Overall Business Knowledge 4
Respond to issues in an expected time frame. 4
Pro-active in identifying problems & improvements 4
Added value provided by the Lionbridge 4
Overall Customer Focus 4
Effective Project Status Reporting 4
Effective Project Progress meetings 4
Lionbridge Team accessibility 3
Lionbridge Team maintained professional approach throughout the
project
4
Overall Communications 4
Lionbridge functions as part of your team 4
Common and recognized goals understood by everybody 4
Feeling of team spirit and partnership 4
Lionbridge Team inspires trust & confidence 5
Overall Teamwork 5
Lionbridge Team display adequate system knowledge. 4
118
Lionbridge Team is proficient in all system technical areas. 4
Overall Technical Knowledge 4
Overall Rating for Lionbridge 4
COMMENTS
The Math09 Team has greatly benefited from having the Lionbridge employees
on it. Rajesh Beloskar brings a strong understanding of the base system we are
building on. His technical abilities, integration expertise and good attitude have
been invaluable to the team. Chaitanya Vasireddy has stepped beyond her role
as a Tester and leveraged her global system knowledge to fill in many of the
gaps in setting up products, loading content and figuring out how to resolve
issues related to content. Without her contributions the project would not be on-
schedule. The Test team at the PDC has done a professional and thorough job,
as we’ve grown to expect, in finding problems and reporting them in a timely and
descriptive way.
119
PARTICIPANT 3
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 3
Change Management 2
Issue Management 2
Overall Planning and Control 2
Analysis and Design (depends on perspective, we’d like to have
them do this but they really don’t have the skill sets. – and maybe
now we don’t expect them to do this at all)
1
Deliverables for Acceptance Testing 3
Implemented Systems and Enhancements 3
Overall Quality of Deliverables 3
Knowledge/Understanding of Customer Business 2
Awareness of Customer issues/problems 1
Lionbridge Team produces quality work 2
Overall Business Knowledge 2
Respond to issues in an expected time frame. 2
Pro-active in identifying problems & improvements 1
Added value provided by the Lionbridge 3
Overall Customer Focus 2
Effective Project Status Reporting 3
Effective Project Progress meetings 3
Lionbridge Team accessibility 2
Lionbridge Team maintained professional approach throughout the
project
3
Overall Communications 3
Lionbridge functions as part of your team 3
Common and recognized goals understood by everybody 3
Feeling of team spirit and partnership (weakness on Pearson part) 3
Lionbridge Team inspires trust & confidence 2
120
Overall Teamwork 3
Lionbridge Team display adequate system knowledge. 1
Lionbridge Team is proficient in all system technical areas. 1
Overall Technical Knowledge 1
Overall Rating for Lionbridge 2.
5
COMMENTS
In general, the PDC team skill set is seen as junior, or inexperienced in nature.
This manifests in recurring junior level coding issues. Also, there is a
demonstrated weakness in the area of design & architecture. Most
implementations seem to not take“system-wide” approaches, and seem narrowly
focused in most instances so there are often integration problems that should
have easily been avoided if the team has a design plan. It seems that unless
MDC staff spells out very specific implementation patterns and guidelines to
follow – there are problems. To date most projects are not organized in a
manner that MDC can provide detailed design plans for every component – so
we have problems.
It has not been uncommon to observe“two-minded” responses to questions about
these types of issues. On the one hand, when a recurring “junior” type problems
are observed, the PDC response seems to come back from PDC that either MDC
did not provide the required guidance or the team was not given time to do it
correctly. On the other hand, the repeatedly indicated that MDC does not allow
the PDC proper opportunity to control their own destiny and take the lead on
design and implementation.
121
To date, in most of these instances that I have looked into, the PDC team did not
escalate that they believed the “Correct” design & approach was being sacrificed
because of time or other constraints. So, the MDC management & lead teams at
the MDC have had no basis to know that a sub-optimal implementation was
being executed by the PDC.
Taken together with the complaint that MDC does not provide proper technical
guidance as needed, and not uncommon recurring incidence of poorly designed
implementations – it is hard to rate the PDC well in this area.
122
PARTICIPANT 4
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 2
Change Management 2
Issue Management 2
Overall Planning and Control 2
Analysis and Design 1
Deliverables for Acceptance Testing 2
Implemented Systems and Enhancements 2
Overall Quality of Deliverables 2
Knowledge/Understanding of Customer Business 3
Awareness of Customer issues/problems 2
Lionbridge Team produces quality work 2
Overall Business Knowledge 3
Respond to issues in an expected time frame. 3
Pro-active in identifying problems & improvements 1
Added value provided by the Lionbridge 2
Overall Customer Focus 3
Effective Project Status Reporting 3
Effective Project Progress meetings 3
Lionbridge Team accessibility 5
Lionbridge Team maintained professional approach throughout the
project
4
Overall Communications 3
Lionbridge functions as part of your team 5
Common and recognized goals understood by everybody 4
Feeling of team spirit and partnership 4
Lionbridge Team inspires trust & confidence 2
Overall Teamwork 4
Lionbridge Team display adequate system knowledge. 4
123
Lionbridge Team is proficient in all system technical areas. 1
Overall Technical Knowledge 1
Overall Rating for Lionbridge 2
COMMENTS
Lionbridge as an organization are severely hindered by the lack of technical
resources (weak Java and DB resources). Weak Test resources (inability to write
test case etc…).
The team members though do work very hard and there is good feeling of a team
with Lionbridge. I also feel the Lionbridge staff does care deeply for the project
and work very hard (and long hours) to try and make the project a success. But
the technical issues resulting from inexperienced staff are a constant problem.
124
PARTICIPANT 5
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 4
Change Management 3
Issue Management 2
Overall Planning and Control 3
Analysis and Design 3
Deliverables for Acceptance Testing 3
Implemented Systems and Enhancements 3
Overall Quality of Deliverables 3
Knowledge/Understanding of Customer Business 2
Awareness of Customer issues/problems 3
Lionbridge Team produces quality work 3
Overall Business Knowledge 3
Respond to issues in an expected time frame. 3
Pro-active in identifying problems & improvements 2
Added value provided by the Lionbridge 2
Overall Customer Focus 3
Effective Project Status Reporting 3
Effective Project Progress meetings 2
Lionbridge Team accessibility 5
Lionbridge Team maintained professional approach throughout the
project
4
Overall Communications 3
Lionbridge functions as part of your team 4
Common and recognized goals understood by everybody 4
Feeling of team spirit and partnership 3
Lionbridge Team inspires trust & confidence 3
Overall Teamwork 3
Lionbridge Team display adequate system knowledge. 3
125
Lionbridge Team is proficient in all system technical areas. 3
Overall Technical Knowledge 3
Overall Rating for Lionbridge 3
COMMENTS
This review was of the overall PSN PDC Team including both Development and
Test. On a whole if I was to rate them separately I would have given the test
team slightly higher marks. Overall all I find the teams are effective and that if
you provide clear requirements and expectations they will meet them.
126
PARTICIPANT 6
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines
Mesa Dev drives completion towards deadlines. Delivery dates
are met through compensating for PDC delivery in schedule.
2
Change Management
PDC needs to improve in this category. Change management in
manually managed at the direction of Mesa. Ongoing audits are
conducted during builds to ensure this is complete.
3
Issue Management
Advanced notice needs to improve. PDC depends on Mesa
Development for issue management.
1
Overall Planning and Control 2
Analysis and Design
Inconsistent. There have been some very positive results. Other
times analysis does not consider the larger picture. There have
been deficiencies in installer skills which has resulted in Mesa
providing analysis that the PDC should be providing. Poor job of
exception handling.
2
Deliverables for Acceptance Testing
Consistent acceptable performance.
3
Implemented Systems and Enhancements 3
Overall Quality of Deliverables
Need improvement towards consistency and quality of early
deliverables. Aggregated iterations are acceptable.
3
127
Knowledge/Understanding of Customer Business
Lack of understanding or product, configuring product, deployment
models.
1
Awareness of Customer issues/problems
Need to improve emphasis, attention, and urgency on items with
high priority status as indicated by Pearson Development.
1
Lionbridge Team produces quality work 2
Overall Business Knowledge 1
Respond to issues in an expected time frame.
Response and urgency is driven by Pearson Development. In
addition, the PDC team will over-promise and under-deliver.
2
Pro-active in identifying problems & improvements
Overall not proactive.
2
Added value provided by the Lionbridge
Heritage history, knowledge, and documentation is apparent at
PDC.
3
Overall Customer Focus
Requests for resource back-up plans, onsite consistency, maturity
of team members, and resistance towards addressing resource
issues. Otherwise, performance is sufficient.
2
Effective Project Status Reporting
Improving. Still need improvement in addressing risks, providing
real critical feedback to Pearson Development.
2
Effective Project Progress meetings
Feedback from concalls needs to be more details, commitments
need to be kept, and driving communication with offshore team
needs to improve.
2
128
Lionbridge Team accessibility
Team availability is limited. Weekly meeting is scheduled during
their work day. Attendance has been an issue. Communication
needs to expand beyond emails after weekly status meeting.
1
Lionbridge Team maintained professional approach throughout the
project
Professionalism is consistent and acceptable.
3
Overall Communications 2
Lionbridge functions as part of your team
Team interactions are positive. However, consistently resistance
towards making effective changes. Escalation ends with Pradeep
and Shilpa.
2
Common and recognized goals understood by everybody
Lack of urgency on high priority items. Improvements can be
implemented by both the PDC and Pearson Development.
3
Feeling of team spirit and partnership
Team interaction reflects good team spirit and partnership.
Escalations and interactions to PDC management reflect the
separation of Lionbridge and Pearson as two separate companies.
3
Lionbridge Team inspires trust & confidence
Actions and priorities will slip if Pearson Dev is not completely
engaged to ensure that priorities are met.
2
Overall Teamwork 3
Lionbridge Team display adequate system knowledge.
Lacking team members with certain product knowledge and
technical skills (web skills).
2
Lionbridge Team are proficient in all system technical areas.
Installers have been a major issue.
2
129
Overall Technical Knowledge
Need more object oriented design knowledge. Programming in
Java is very procedural. Major features of the languages
(exception handling) are not used.
2
Overall Rating for Lionbridge 2
COMMENTS
This assessment could very easily improve to a 4 (meeting/exceeding
expectations) with improvements in the following categories:
Product understanding – better training to ensure that team members have a
sufficient understanding of the product.
Management feedback – escalated issues are addressed in collaboration
between Lionbridge and Pearson Development towards constructive solutions.
Technical Skills – improvements in technical skills such as web development and
installers.
Communication – PDC project management to ensure that work is sufficiently
planned, coordinated between groups (Development, Test, Documentation),
critical actions addressed and resolved with urgency, and that weekly priorities
and technical direction is sufficiently communicated to offshore team.
Team member availability to address issues would also be helpful.
130
PARTICIPANT 7
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 4
Change Management 4
Issue Management 3
Overall Planning and Control 3
Analysis and Design 2
Deliverables for Acceptance Testing 4
Implemented Systems and Enhancements 3
Overall Quality of Deliverables 4
Knowledge/Understanding of Customer Business 3
Awareness of Customer issues/problems 4
Lionbridge Team produces quality work 3
Overall Business Knowledge 3
Respond to issues in an expected time frame. 4
Pro-active in identifying problems & improvements 4
Added value provided by the Lionbridge 4
Overall Customer Focus 4
Effective Project Status Reporting 2
Effective Project Progress meetings 3
Lionbridge Team accessibility 4
Lionbridge Team maintained professional approach throughout the
project
5
Overall Communications 4
Lionbridge functions as part of your team 5
Common and recognized goals understood by everybody 3
Feeling of team spirit and partnership 4
Lionbridge Team inspires trust & confidence 4
Overall Teamwork 4
Lionbridge Team display adequate system knowledge. 3
131
Lionbridge Team are proficient in all system technical areas. 2
Overall Technical Knowledge 3
Overall Rating for Lionbridge 3.
5
COMMENTS
None
132
PARTICIPANT 8
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 4
Change Management 4
Issue Management 4
Overall Planning and Control 4
Analysis and Design 3
Deliverables for Acceptance Testing 3
Implemented Systems and Enhancements 3
Overall Quality of Deliverables 3
Knowledge/Understanding of Customer Business 3
Awareness of Customer issues/problems 3
Lionbridge Team produces quality work 3
Overall Business Knowledge 3
Respond to issues in an expected time frame. 3
Pro-active in identifying problems & improvements 3
Added value provided by the Lionbridge 3
Overall Customer Focus 3
Effective Project Status Reporting 4
Effective Project Progress meetings 4
Lionbridge Team accessibility 4
Lionbridge Team maintained professional approach throughout the
project
4
Overall Communications 4
Lionbridge functions as part of your team 4
Common and recognized goals understood by everybody 3
Feeling of team spirit and partnership 4
Lionbridge Team inspires trust & confidence 3
Overall Teamwork 4
Lionbridge Team display adequate system knowledge. 3
133
Lionbridge Team are proficient in all system technical areas. 3
Overall Technical Knowledge 3
Overall Rating for Lionbridge 4
COMMENTS
The PDC continues to show its strength in management, planning and control,
and communication. The PDC
SIS team has taken steps to improve on the technical side. They provided the
analysis and design and implementation of a brand new installer for windows,
and analysis and design for launching existing multimedia content through a web
browser. These 2 projects provided evidence of technical improvements.
134
PARTICIPANT 9
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 4
Change Management 4
Issue Management 4
Overall Planning and Control 4
Analysis and Design 4
Deliverables for Acceptance Testing 4
Implemented Systems and Enhancements 4
Overall Quality of Deliverables 4
Knowledge/Understanding of Customer Business 4
Awareness of Customer issues/problems 4
Lionbridge Team produces quality work 4
Overall Business Knowledge 4
Respond to issues in an expected time frame. 4
Pro-active in identifying problems & improvements 4
Added value provided by the Lionbridge 4
Overall Customer Focus 4
Effective Project Status Reporting 4
Effective Project Progress meetings 4
Lionbridge Team accessibility 4
Lionbridge Team maintained professional approach throughout the
project
4
Overall Communications 4
Lionbridge functions as part of your team 5
Common and recognized goals understood by everybody 4
Feeling of team spirit and partnership 5
Lionbridge Team inspires trust & confidence 4
Overall Teamwork 4
Lionbridge Team display adequate system knowledge. 4
135
Lionbridge Team are proficient in all system technical areas. 3
Overall Technical Knowledge 4
Overall Rating for Lionbridge 4
COMMENTS
The PCS Team at Lionbridge is willing to looking into new approaches, technical
solutions, and take the lead role in investigating and implementing untried
concepts. The PCS Team at Lionbridge is very open to suggestions that may
lead to improvements either in application components or overall system
development.
Lionbridge and Chandler PCS members are definitely a TEAM! We all work
together. Success is a Team effort.
136
PARTICIPANT 10
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 4
Change Management 4
Issue Management 4
Overall Planning and Control 4
Analysis and Design 4
Deliverables for Acceptance Testing 4
Implemented Systems and Enhancements 4
Overall Quality of Deliverables 4
Knowledge/Understanding of Customer Business 4
Awareness of Customer issues/problems 4
Lionbridge Team produces quality work 4
Overall Business Knowledge 5
Respond to issues in an expected time frame. 5
Pro-active in identifying problems & improvements 5
Added value provided by the Lionbridge 5
Overall Customer Focus 5
Effective Project Status Reporting 5
Effective Project Progress meetings 5
Lionbridge Team accessibility 5
Lionbridge Team maintained professional approach throughout the
project
5
Overall Communications 4
Lionbridge functions as part of your team 4
Common and recognized goals understood by everybody 4
Feeling of team spirit and partnership 4
Lionbridge Team inspires trust & confidence 4
Overall Teamwork 5
Lionbridge Team display adequate system knowledge. 5
137
Lionbridge Team are proficient in all system technical areas. 5
Overall Technical Knowledge 5
Overall Rating for Lionbridge 5
COMMENTS
None
138
PARTICIPANT 11
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadline:
the most critical deadline was delivering 1.7.1 for beta testing by
4/1. This was successfully accomplished. Completion of an
advanced acceptance build was delayed and successful
conclusion of development rework and test confirmation are taking
longer than expected. However, the most important beta deadline
was met.
4
Change Management:
for the most part this seems to be acceptable. From time to time
issues have been re-introduced into builds. This could benefit
from improvement but has been satisfactory in Q3-07.
3
Issue Management:
team response has been positive towards resolving issues.
Improvements can be made in proactively anticipating issues and
concisely presenting options to Pearson for addressing issues.
3
Overall Planning and Control 3
Analysis and Design:
Q3-07 focused more on completing development and testing and
less on design. Analysis and design of solutions for memory
virtualization and report performance testing exceeded
expectations. However, some solutions still required deferral or
further consultation.
3
139
Deliverables for Acceptance Testing:
Engineering accepted the beta build. The last minute printing
issue was resolved quickly. Many known issues still exist and the
beta process is just beginning.
4
Implemented Systems and Enhancements:
Q3-07 concentrated on development and test completion more
than system implementations and enhancements.
3
Overall Quality of Deliverables 4
Knowledge/Understanding of Customer Business:
Tasks related to this aspect were not significant during Q3-07.
3
Awareness of Customer issues/problems:
Issue recognition and resolution was satisfactory during Q3-07.
3
Lionbridge Team produces quality work:
work quality was satisfactory during Q3-07. This is substantiated
by the acceptance of the beta build. The grade applied to this
question could increase or decrease based on feedback during
beta testing. Quantitative feedback will be available Q4-07.
3
Overall Business Knowledge 3
Respond to issues in an expected time frame:
During Q3-07 the team was attentive to responding to issues in a
timely manner.
3
Pro-active in identifying problems & improvements:
This improved towards the end of Q3-07. However, particularly
during Jan and Feb there were many concalls devoted to
addressing bottlenecks with hardware, work sequencing, and
priority work that could adversely affect the schedule. These
issues should have been pro-actively addressed.
2
140
Added value provided by the Lionbridge:
During Q3-07 there were 3 FTE Pearson employees and 32-48
Lionbridge employees working on SMe. The PDC team shared
significant responsibility for delivering the beta build on time.
4
Overall Customer Focus 3
Effective Project Status Reporting :
this has definitely improved with the concall actions/issues list and
the defect metrics. All of this has aided in the ability to manage
this effort. Focus for future improvements in proactively identifying
risks and issues and presenting options for addressing.
3
Effective Project Progress meetings:
the weekly concall effectiveness has improved. The weekly CCB
has needed more advanced preparation in updated comments and
status. In the last few weeks this step has been taken to reduce
the duration of this meeting and its effectiveness.
3
Lionbridge Team accessibility:
Team is accessible when required. Attendance and timeliness for
reporting to meetings have been very positive. This is an
improvement from Q2-07.
3
Lionbridge Team maintained professional approach throughout the
project :
The team compiled a re-staffing plan to meet an advanced
schedule and successfully achieved the primary milestone of
delivering an accepted build for 1-Apr beta.
4
Overall Communications 3
Lionbridge functions as part of your team:
Co-ordination and collaboration between the Lionbridge leads and
their Pearson counterparts has been very positive.
4
141
Common and recognized goals understood by everybody:
During Q3-07 the Lionbridge team successfully responded to the
critical Pearson goals for advancing the schedule and delivering a
successful build for beta on 1-Apr.
4
Feeling of team spirit and partnership:
During Q3-07 the Lionbridge team successfully responded to the
critical Pearson goals for advancing the schedule and delivering a
successful build for beta on 1-Apr.
4
Lionbridge Team inspires trust & confidence:
The Lionbridge team put significant effort into achieving the goals
established by Pearson. There are communication challenges
inherent in offshore collaboration which creates concern that the
message intended is delivered and received. In addition, there
was a period of time where it appeared that Development and Test
were two distinct groups instead of one team. This has improved
in recent weeks but was a concern throughout Q3-07.
3
Overall Teamwork 4
Lionbridge Team display adequate system knowledge:
There are many team members with significant SMe product
knowledge.
4
Lionbridge Team are proficient in all system technical areas:
Technical knowledge is sufficient in some areas but there have
been issues in others. Advanced ‘C’ and Build are two critical
areas where there needs to be improvement in technical
knowledge.
2
Overall Technical Knowledge 3
Overall Rating for Lionbridge 3
142
COMMENTS
All comments included in this survey are applicable to Q3-07. For the period of
Q3-07 the Lionbridge performance on the SASI project met expectations. This
represents an improvement from the survey completed Sep-07. In the Sep-07
survey four points were identified for improving the overall score to a 4
(meeting/exceeding expectations). Those four points with comments for each
are:
1. Product understanding – better training to ensure that team members have a
sufficient understanding of the product.
2. Management feedback – escalated issues are addressed in collaboration
between Lionbridge and Pearson Development towards constructive
solutions.
a. This area improved towards the end of the Q2-07. For much of Q1-07
management feedback was hindered by the separation of development,
test, and doc into three distinct groups. This feedback was provided to
Lionbridge management and communication is now driven through the
project manager as the single point of communication.
3. Technical Skills – improvements in technical skills such as web development
and installers.
a. New resources have been added to the team which will hopefully
improve the technical skills feedback. Existing issues with advanced ‘C’
and Build continue to exist. The process for build creation needs to
improve and be automated to reduce the amount of effort required and
the potential for manual errors.
4. Communication – PDC project management to ensure that work is sufficiently
planned, coordinated between groups (Development, Test, Documentation),
critical actions addressed and resolved with urgency, and that weekly
143
priorities and technical direction is sufficiently communicated to offshore
team. Team member availability to address issues would also be helpful.
a. Communication clearly improved with the implementation of the concall
tracker and the defect summaries initiated and implemented by the PDC
staff. Communication improvements were hindered for part of Q3-07
with Development, Test, and Documentation operating as three distinct
groups. This should improve with the project manager as the single
point of communication.
144
PARTICIPANT 12
Scale
N/A 1 2 3 4 5
Below Meets Exceeds
Expectations
Meeting agreed deadlines 3
Change Management 3
Issue Management 3
Overall Planning and Control 3
Analysis and Design 3
Deliverables for Acceptance Testing 3
Implemented Systems and Enhancements 3
Overall Quality of Deliverables 3
Knowledge/Understanding of Customer Business 3
Awareness of Customer issues/problems 3
Lionbridge Team produces quality work 3
Overall Business Knowledge 2
Respond to issues in an expected time frame. 3
Pro-active in identifying problems & improvements 3
Added value provided by the Lionbridge 3
Overall Customer Focus 3
Effective Project Status Reporting 3
Effective Project Progress meetings 3
Lionbridge Team accessibility 3
Lionbridge Team maintained professional approach throughout the
project
4
Overall Communications 3
Lionbridge functions as part of your team 3
Common and recognized goals understood by everybody 3
Feeling of team spirit and partnership 3
Lionbridge Team inspires trust & confidence 3
Overall Teamwork 3
145
Lionbridge Team display adequate system knowledge. 3
Lionbridge Team are proficient in all system technical areas. 2
Overall Technical Knowledge 3
Overall Rating for Lionbridge 3
COMMENTS
Lionbridge has not been able to assist with the tuning of the PSN application
(Weblogic). It would be good if we could have this work done before we put the
build in the performance environment and catch performance issues more
quickly.
146
ANNEXURE 3 – PROJECT
METRICS DATA
147
PROJECT METRICS DATA
The captured metrics may vary and hence to help interpret the metrics, upper
and lower limits have been assigned. The limits for various metrics have been
listed in Table 4. This table indicates the color legends for the control limits set
for the metrics. The control limits for Schedule and Effort Variations are studied
closely.
Table 8 - Legends
Metrics Green Yellow Amber Gray
Schedule Variance >= 0 & <= 3 > 3 and <= 5 Above 5 Blank or <0
Effort Variance >= 0 & <= 5 > 5 and <= 10 Above 10 Blank or <0
DRE >= 98 >= 90 and < 98 < 90 Blank or <0
Internal Defects >= 0 & <= 25 > 25 and <= 50 Above 50 Blank or <0
External Defects >= 0 & <= 3 > 3 and <= 5 Above 5 Blank or <0
As indicated in the above table, if the value of Schedule Variation Metric is
between 0 and 3 then it is indicated in ‘Green’ color. If the value of a metric is
between 3 and 5 then it is indicated in ‘Yellow’ color and if the value of the
metric is greater than 5 then it is indicated in ‘Amber’ color.
Similarly, if the value of Effort Variation Metric is between 0 and 5 then it is
indicated in ‘Green’ color. If the value of a metric is between 5 and 10 then it is
indicated in ‘Yellow’ color and if the value of the metric is greater than 10 then it
is indicated in ‘Amber’ color.
In both the cases if the value of the metrics is blank or less than zero then it is
indicated in ‘Gray’ color.
148
DRE value is greater than 98 is indicated in ‘Green’ color. If this value is
between 90 and 98 then it is indicated in ‘Yellow’ and ‘Amber’ if this is less than
90
If the Internal defects greater than 50 then it is indicated in ‘Amber’. If this count
less than 25 then it is indicated by ‘Green’ and ‘Yellow’ if the count is between
35 and 50.
If the External defects greater than 5 then it is indicated in ‘Amber’. If this count
less than 3 then it is indicated by ‘Green’ and ‘Yellow’ if the count is between 3
and 5.
Table 9 - Metrics data for August 2007
Program Project
%
Schedule
Variation
%
Effort
Variation
Internal
Defects
External
defects
Defect
Removal
efficiency
SASI 11.04 -6.41 13 0 100.00
SRC 5.29 -1.25 21 0 100.00
SM NG Reading
SM NG LMS
Sme v 1.7 -0.33 0.39 707 0 100.00
PAPA 0.00 -0.36 50 3 94.00
PCS -0.10 3.32 65 0 100.00
NovaNET V16 0.00 -4.30 35 0 100.00
PSN PSN
Curriculum
ADMIN
As indicated in Table 5, six projects were executed during the month of August
2007.
Analysis of Schedule Variance
Two out of six projects were in Amber on schedule variance. i.e. 33% of the
projects were in Amber. Rest of the 4 projects were within the control limits.
149
Effort Variance
The efforts variance for all projects was less than 5% hence all projects fared
well on the Efforts Variation.
External Defects
On one project there were defects leaked to customer and hence internal
testing process needed to be improved on this project.
DRE
The testing process on one of the projects needed improvement. The internal
testing process for rest of the projects was good. This was indicated by The
DRE which was 100% for most of the projects, which means that most of the
defects were caught in the internal testing.
Table 10 - Metrics data for September 2007
Program Project% Schedule
Variation
%
Effort
Variation
Internal
Defects
External
Defects
Defect
Removal
efficiency
SASI 7.46 2.54 58 6 91.00
SRC 5.61 -4.76 42 0 100.00
SM NG Reading 2.45 -2.31 0 0 0.00
SM NG LMS 2.28 3.65 325 0 100.00
Sme v 1.7 0.99 -4.71 43 0 100.00
PAPA 0.00 -1.55 28 2 100.00
PCS -0.27 -0.28 7 0 100.00
NovaNET V16 5.81 -4.73 196 0 100.00
PSN PSN 3.47 2.73 31 0 100.00
Curriculum
ADMIN
As indicated in the above table, there were nine projects executed during the
month of November 2007. Three out of the nine projects were in Amber on
150
schedule variance. i.e. 33% of the projects were in Amber and one project was
in Yellow.
Schedule Variance
Three out of nine projects were in Amber and one project was in Yellow on
schedule variance. i.e. 33% of the projects were in Amber and 11% in Yellow.
Overall there was an area of concern on the performance of this metric for the
above period as 44% of projects were greater than 3% control limit.
Effort Variance
The efforts variance for all projects was less than 5% hence all projects fared
ok on the Efforts Variation.
External Defects
There were external defects in two of the projects. This number was zero for
the rest of the projects.
DRE
The internal testing process for two projects needed improvements.
151
Table 11 - Metrics data for October 2007
Program Project
%
Schedule
Variation
%
Effort
Variation
Internal
Defects
External
defects
Defect
Removal
efficiencySASI 18.85 7.46 13 1 93.00
SRC 7.40 -4.17 50 0 100.00
SM NG Reading -3.23 -0.10 0 0 0.00
SM NG LMS 1.62 -7.34 690 0 100.00
Sme v 1.7 3.75 -2.12 25 0 100.00
PAPA 0.00 -0.05 44 0 100.00
PCS 0.48 -1.46 169 0 100.00
NovaNET V16 -4.58 -6.74 134 0 100.00
PSN PSN 1.40 -1.33 41 0 100.00
Curriculum
ADMIN
As indicated in the above table, there were nine projects executed during the
month of October 2007. Two out of the nine projects were in Amber on
schedule variance. i.e. 22% of the projects were in Amber.
Schedule Variance
Two out of nine projects were in Amber on schedule variance. i.e.22% of the
projects were in Amber and one project was in Yellow.
Effort Variance
One of the nine projects had had more than 5% on the efforts variance. Other
eight projects fared within 5% on the Efforts Variation.
External Defects
This number was zero for most of the projects which means that there were no
defects in the systems released to customers during the above period.
152
DRE
The internal testing process for most of the projects was very good. This was
indicated by The DRE which was 100% for most of the projects, which means
that most of the defects were caught in the internal testing.
Table 12 - Metrics data for November 2007
Program Project
%
Schedule
Variation
%
Effort
Variation
Internal
Defects
External
defects
Defect
Removal
efficiency
SASI 18.76 -1.24 61 5 92.00
SRC 7.09 -5.19 12 0 100.00
SM NG Reading -1.91 -2.05 0 0 0.00
SM NG LMS -1.76 -5.64 0 0 0.00
Sme v 1.7 0.89 0.05 712 0 100.00
PAPA 0.20 -5.82 0 0 0.00
PCS 0.76 -0.13 0 0 0.00
NovaNET V16 0.13 1.05 59 0 100.00
PSN PSN -0.30 -1.17 80 0 100.00
Curriculum
ADMIN
As indicated in the above table, there were nine projects executed during the
month of September 2007. Two out of the nine projects were in Amber on
schedule variance. i.e. 22% of the projects were in Amber.
Schedule Variance
Two out of nine projects were in Amber on schedule variance. i.e.22% of the
projects were in Amber. Rest of the projects were within 5% on schedule
variance
153
Effort Variance
The efforts variance for all projects was less than 5% hence all projects fared
well on the Efforts Variation.
External Defects
On one project there were defects leaked to customer and hence internal
testing process on this project needed to be improved. This defects leaked
count was zero for the rest of the projects.
DRE
The testing process on one of the projects needed improvement. The internal
testing process for rest of the projects was good. This was indicated by The
DRE which was 100% for most of the projects, which means that most of the
defects were caught in the internal testing.
154
Table 13 - Metrics data for December 2007
Program Project
%
Schedule
Variation
%
Effort
Variation
Internal
Defects
External
Defects
Defect
Removal
efficiency
SASI -0.50 -2.87 0 0 0.00
SRC 7.71 -0.71 41 0 100.00
SM NG LMS 0.00 4.30 833 0 100.00
Sme v 1.7 -1.23 7.22 87 0 100.00
PAPA 0.00 -2.05 16 1 100.00
PCS 0.04 0.00 20 0 100.00
NovaNET V16 0.00 -5.02 264 0 100.00
PSN PSN -2.23 2.74 50 0 100.00
e1 e1 0.22 5.51 139 0 100.00
Curriculum
ADMIN
As indicated in the above table, there were nine projects executed during the
month of December 2007. One out of the nine projects was in Amber on
schedule variance. i.e. 11% of the projects were in Amber.
Schedule Variance
One out of nine projects was in Amber on schedule variance. i.e. 11% of the
projects were in Amber. Rest of the projects were within 5% on schedule
variance.
Effort Variance
One of the nine projects had more than 5% on the efforts variance. Other eight
projects fared within 5% on the Efforts Variation.
External Defects
There were external defects in one project. This number was zero for the rest
of the projects.
155
DRE
The internal testing process for one project needed improvements.
Table 14 - Trend Data for the Projects in Amber.
The table below is the trends of projects in Amber.
Period %tage of projects in Amber
Aug ‘07 33
Sep’ 07 33
Oct ‘07 22
Nov ‘07 22
Dec ‘07 11
156
ANNEXURE 4 –
BIBLIOGRAPHY
157
BIBLIOGRAPHY
No. Author, Literature / Book / Article, Publication
1. Anonymous, . Outsource Your Noncore Functions. (2009, February).
Principal's Report, 09(2), 5-7. Retrieved October 11, 2009, from
ABI/INFORM Trade & Industry. (Document ID: 1639890091).
2. Anderson, N.. (2009, September). There can be no substitute for meeting
customers' expectations. MicroScope,18. Retrieved October 11, 2009,
from ABI/INFORM Trade & Industry. (Document ID: 1867847791).
3. YongKi Yoon, & Kun Shin Im. (2005). An Evaluation System for IT
Outsourcing Customer Satisfaction Using the Analytic Hierarchy
Process. Journal of Global Information Management, 13(4), 55-78.
Retrieved October 11, 2009, from ABI/INFORM Global. (Document
ID: 905476031).
4. Manning, M.. (2009, September). THE MOMENT OF TRUTH. Training
Journal,75. Retrieved October 12, 2009, from ABI/INFORM Global.
(Document ID: 1857935301). 5. Hsiuij Rebecca Yen. (2005). An attribute-based model of quality
satisfaction for Internet self-service technology. The Service Industries
Journal, 25(5), 641-659. Retrieved October 11, 2009, from ABI/INFORM
Global. (Document ID: 862970851).
6. Boris Bartikowski, & Sylvie Llosa. (2004). Customer Satisfaction
Measurement: Comparing Four Methods of Attribute Categorisations. The
Service Industries Journal, 24(4), 67-82. Retrieved October 11, 2009,
from ABI/INFORM Global. (Document ID: 719608421).
7. Humphry Hung, & Y H Wong. (2007). Organisational Perception of
Customer Satisfaction: Theories and Evidence. The Service Industries
Journal, 27(4), 495. Retrieved October 11, 2009, from ABI/INFORM
Global. (Document ID: 1284655281).
158
8. Hu, H., Kandampully, J., & Juwaheer, T.. (2009). Relationships and
impacts of service quality, perceived value, customer satisfaction, and
image: an empirical study. The Service Industries Journal, 29(2), 111.
Retrieved October 11, 2009, from ABI/INFORM Global. (Document
ID: 1648537101).
9. Yang, C., Yang, K., Yeh, T., & Pai, F.. (2009). Methods for determining
areas for improvement based on the design of customer surveys. The
Service Industries Journal, 29(2), 143. Retrieved October 11, 2009, from
ABI/INFORM Global. (Document ID: 1648537121).
10. Nick Johns. (2001). Importance-performance analysis using the profile
accumulation technique. The Service Industries Journal, 21(3), 49-63.
Retrieved October 11, 2009, from ABI/INFORM Global. (Document
ID: 79168593).
11. Keiningham, T., Aksoy, L., Cooil, B., & Andreassen, T.. (2008). Linking
Customer Loyalty to Growth. MIT Sloan Management Review, 49(4), 51-
57. Retrieved October 11, 2009, from ABI/INFORM Global. (Document
ID: 1520405501).
12. Broetzmann, Scott M, Kemp, John, Rossano, Mathieu, & Marwaha,
Jay. (1995). Customer satisfaction - Lip service or management
tool? Managing Service Quality, 5(2), 13. Retrieved October 11, 2009,
from ABI/INFORM Global. (Document ID: 6675688).
13. Yang, C., Yang, K., Yeh, T., & Pai, F.. (2009). Methods for determining
areas for improvement based on the design of customer surveys. The
Service Industries Journal, 29(2), 143. Retrieved October 11, 2009, from
ABI/INFORM Global. (Document ID: 1648537121).
14. Ton van der Wiele, Paul Boselie, & Martijn Hesselink. (2002). Empirical
evidence for the relationship between customer satisfaction and business
performance. Managing Service Quality, 12(3), 184-193. Retrieved
October 11, 2009, from ABI/INFORM Global. (Document ID: 266829621).
159
15. Mary K Pratt. Computerworld. Framingham: Jul 14, 2008. Vol. 42, Iss. 29;
pg. 35, 1 pgs. Quality Over Quantity.
16. Nigel Hill. Nigel Hill. Customer Satisfaction Measurement: how not to do
it, how to do it and why it should be done.
17. J L Herskett, W E Sassar and L A Schlesinger. The Service Profit Chain
(New York: Free Press 1997)
18. H A J Menezes, J Serbin, Xerox Corporation. The Customer Satisfaction
Program Case No. 591-055 (Boston-Harvard Business School 1991).
19. Cheon, M.J., Grover, V., & Teng, J.T.C. (1995). Theoretical perspective
on the outsourcing of information systems. Journal of Technology
Information, 10, 209-219.
20. Gelbstein, E. (2002). Outsourcing. In H. Bidgoli, et al. (Eds.),
Encyclopedia of information systems (pp. 428-430).
21. Grover, V., Cheon, M.J., & Teng, J.T.C (1996). The effect of service
quality and partnership on the outsourcing of information systems
functions. Journal of Management Information System, 72(4), 89-116.
22. McFarlan, F. W. & Nolan, R.L. (1995). How to manage an IT outsourcing
alliance. Sloan Management Review, 36(2), 9-23.
23. Anonymous. PR Newswire. New York: Jun 2, 2008. Outsourcing Trends,
Metrics & Insights.
24. Anonymous. Inc. Boston: Sep 2009. Vol. 31, Iss. 7; pg. 65, 4 pgs. HOW
TO: Know Your Customer Better. Inc, 31(7), 65-68. Retrieved October
12, 2009, from ABI/INFORM Global. (Document ID: 1849312041).
25. Zeithaml, Berry and Parasuraman Customer Expectations of Service
26. Valarie A. Zeithaml, Services Marketing
27. Philip Kotler, Marketing Management
28. Outsourcing Institute website - www.outsourcing.com
29. James R Evans and William M Lindsay, The Management And Control Of
Quality
160
ANNEXURE 5 – COPIES OF
FEW ARTICLES REVIEWED
161
COPIES OF FEW ARTICLES REVIEWED
As a part of this study several articles were studied. Copies of few of these
articles are attached in this section.
162
Article 1
THE MOMENT OF TRUTH
163
THE MOMENT OF TRUTH
Margo Manning. Training Journal. Ely: Sep 2009. pg. 75, 1 pgs
Abstract (Summary)
While delivering a customer care workshop to a team of managers, including a
training manager, I posed the simple question: "How do you know if your
customer is delighted?" The responses were varied: turnover, repeat business
and the usual key performance indicators.
Full Text
(713 words)
Copyright Fenman Limited Sep 2009
Margo Manning tells us why we can't afford to ignore customer care
Quality or quantity, are they mutually exclusive?
For a lot of training companies and departments, there appears to be a
measure based on either or: quality or quantity! It is not restricted to just
training; we can also include service desks, call centres, customer services and
many more departments and companies.
Within some of the companies I have worked, there is a real need to measure
success purely on statistics. Training managers will show the number of people
put through training over a period of time; they may also show some statistics
from 'happy sheets'. Call centres may report on the number of calls answered,
164
time to answer, missed calls and first time fixes. Customer services may report
on the number of complaints, number resolved, response time etc.
How many companies do you know ask the ultimate question of are we getting
the service or produce right?
The question is: "Would you recommend this company/department to a friend
or colleague?"
If the answer was yes, surely you would know you were on the right tracks
(quality would fare in this). If the answer was no, you would be doing
something wrong (possibly choosing quantity over quality). I fully appreciate
that there is more to this, however a yes or no does give you a good indication
of the customer experience.
While delivering a customer care workshop to a team of managers, including a
training manager, I posed the simple question: "How do you know if your
customer is delighted?" The responses were varied: turnover, repeat business
and the usual key performance indicators. I enquired if anyone ever thought to
ask the customer the ultimate question and the responses were: "We don't
measure the customer's delight", "We measure statistics that we can report on"
and the old faithful: "If we can't report it, we don't measure it".
So, asking the team "What are your KPIs?", one replied that "my team, for the
most part, answered all calls within the ten-second target". I asked: "What were
the first time fix rate statistics?" and he responded: "That was not so great;
however they [the service desk] did answer 98 per cent of their calls within ten
seconds."
Is this a clear case of quantity over quality? What is the cost of choosing
quantity? To name but a few: a bad reputation, an increased volume of calls
165
due to unhappy return callers and, I would imagine, very frustrated service
desk engineers.
Why were they on the customer care programme? The answer was "to offer a
better service". How is it possible to offer a better customer experience and
delight the customer when you don't know what he thinks of your current
service?
I asked the training manager directly how she measured customer delight and
she said she measured the obvious, including cost per head. She found my
question about return on investment difficult to answer. Asking her about
participants, their managers and the business's take on the development
provided, she responded rather sheepishly: "We don't have time to ask; even if
we did, we would not have time to do anything different."
Is it possible that once again quantity outdoes quality?
What needs to change?
Ask your customer the ultimate question: "Would you recommend this
company to a friend or colleague?" If the customer responds with a yes, find
out what you are doing well and roll this out to all your customers. If the
customer responds with a no, find out what you are doing wrong, what would
be the right thing to do and do it.
We need to stop seeing customers as a service level agreement and KPI. Let's
start seeing them as individuals who have options, one of which is to go
elsewhere. Internal customers have options as well and one of those is to bring
in externally-sourced providers.
166
Why is it, when our goal is to keep our business successful, that we don't ask
the all-important question of the all-important people - our customers? Why do
we continue to place more importance on quantity rather than quality?
[Author Affiliation]
Margo Manning is a director of learning and development company Bute Ltd.
She can be contacted on +44 (0)8700 420864 or at margo.manning@buteltd.
com
167
Article 2
Outsource Your Noncore Functions
168
Outsource Your Noncore Functions
Anonymous. Principal's Report. New York: Feb 2009. Vol. 09, Iss. 2; pg. 5, 3 pgs
Abstract (Summary)
Gains access to world-class capabilities. Outsourcing means specialization.
Outsourcing providers and specialty consulting firms can bring worldwide,
worldclass capabilities and expertise to a firm. Using world-class advertising
and public relations (PR) firms is one example of how to leverage their
expertise as your own. PR consultants also have easier access to editors of
national publications and can place articles in client-read magazines easier.
Full Text
Copyright Institute of Management & Administration Feb 2009
Knowledge and expertise no longer have to be homegrown. They can be
contracted for. Outsourcing is an effective tool for redefining and re-energizing
an organization. It helps A/E firms focus on their core business by assuming
that some or all of their noncore functions can be outsourced for the following
good reasons:
� Outsourcing leverages relationships instead of amassing resources;
� Outsourcing delivers value instead of directing activities; and
� Outsourcing ensures results, since it is under contract.
169
The effective and efficient use of an A/E firm's marketing and business
development personnel is a major element for achieving firmwide success.
Firm after firm has come to the realization that investing in and maintaining in-
house specialty departments may not make dollars and sense in a recession.
Relying upon outsourcing organizations and specialty consultants to supply
critical support services provides access to a wealth of intellectual capital
without investing in the personnel, equipment, or infrastructure. From payroll
processing to accounting, IT services to recruiting, market research to
corporate video presentations, outsourcing is a cost-effective method of
acquiring these services.
Typically, two types of shared services are outsourced: transactional services
and professional services. Transactional services are repetitive and generic,
i.e., administration, accounting, payroll, and benefits processing. Professional
services, marketing, and business development, on the other hand, are
knowledge-based and must be served by professional services consultants.
By dividing all business activities into four categories, you can identify
outsourcing opportunities offering the best long-term value and lowest-cost
options:
1. Peripheral functions provide no competitive advantage to a firm and are
not essential to the core business. These functions are easily
outsourced and produce minimal risk.
2. Support functions are essential but are not a core activity. Failure in this
area would cause serious damage to the business. In this case, a
manager's time and resources could be better spent on activities
fundamental to the business, recruiting this expertise externally.
3. Strategic functions provide an actual or potential source of competitive
advantage. One reason for outsourcing strategic functions is to achieve
170
better value from the strategic thinking of others and the added
capabilities they bring.
4. Core functions are the primary activity of the business and should never
be outsourced for any reason.
In weighing the benefits and costs associated with outsourcing, you may want
to consider the reasons firms commonly cite for outsourcing noncore functions
and the benefits they expect to receive, as reported by the Outsourcing
Institute (www.outsourc ing.com).
• Improves a firm's focus. Outsourcing allows the firm to focus on its core
activities, while support services are assumed by an outside provider,
such as a specialty consultant. It can enable an organization to
accelerate its growth and success through expanded investment in
areas offering the greatest competitive advantage. A Web site
consultant, who designs and maintains your Web site, is a good
example.
• Gains access to world-class capabilities. Outsourcing means
specialization. Outsourcing providers and specialty consulting firms can
bring worldwide, worldclass capabilities and expertise to a firm. Using
world-class advertising and public relations (PR) firms is one example of
how to leverage their expertise as your own. PR consultants also have
easier access to editors of national publications and can place articles in
client-read magazines easier.
• Shares the risks. Outsourcing providers and specialty consultants
makes investments not on behalf of just one firm but on behalf of all its
main clients, which eliminate the necessity of investment in equipment
or other inventory to provide these special services by the firm itself.
171
• Using subconsultants as an outsourcing opportunity. Outsourcing can
involve the transfer of assets from the firm to the outside provider. Using
MBE/WBE subconsultants provides a valuable resource, as each is a
specialist in their particular discipline-expertise that the A/E firm does
not have to maintain in-house. Working with specialty subconsultants is
essential in gaining government contracts.
• Reduces and controls operating costs. Firms outsource because they do
not have access to the required resources from within, and they don't
want the expense to build these resources from the ground up. This
includes access to services such as photography, printing, direct mail
fulfillment, and video and computer-based presentations. While some
large A/E firms have incorporated such facilities in-house in the past,
many have since abandoned them in favor of outsourcing them to
specialty photographers, audio/video providers, printers, and mail
houses.
• Offers resources not available internally. Access to an outside provider's
lowercost structure may result in greater economy of scale and is one of
the most compelling tactical reasons for using outsourcing to
professional services consultants. This includes services for market
research, client image surveys, client satisfaction surveys, and other
strategic research for the firm. It extends to graphic design, corporate
identity, annual reports, brochures, and other promotional material.
How to determine what the firm should outsource? The question then becomes
what and how much to outsource. To arrive at an answer that makes sense for
your organization, ask yourself these questions:
• If you were starting the company today, would you elect to perform this
function internally?
172
• Are you so good at performing this activity that others would hire you to
do it for them (insourcing)?
If you answer "no" to these questions it's time to think about outsourcing.
This information was adapted from an article by Karol White, Karol White
Consulting, New York City, a former managing director of
PricewaterhouseCoopers. She can be reached at 646-414-2141.
For those seeking to access the Deltek webinar "Secrets of Business
Development for Senior Executives" referred to in the January 2009 cover story
"Industry Leaders Discuss Ways to Generate New Business in an Uncertain
Economy," visit http://www.deltek.com/specialoffers/GenerateNewBusiness.
asp.
173
Article 3
Creating a customer-focused culture: some
practical frameworks and tools
174
Creating a customer-focused culture: some
practical frameworks and tools
Steve Macaulay, Graham Clark. Managing Service Quality. Bedford: 1998. Vol. 8,
Iss. 3; pg. 183
Abstract (Summary)
Many companies have customer service departments which act as a barrier
between the company and the customer. Is customer service represented at
the top? How credible is a customer satisfaction index? What effect does this
have on customer service? This article discusses a tool for assessing the
degree of customer focus within an organization's culture and provides a series
of checklist questions.
Full Text
(3545 words)
Copyright MCB UP Limited (MCB) 1998
Steve Macaulay: Management Development Consultant, Cranfield School of
Management, Bedford, England
Graham Clark: Senior Lecturer in Operations Management, Cranfield School of
Management, Bedford, England
What do a gourmet restaurant, a car service centre and the Inland Revenue
have in common? Their stated objective is to provide a service to their
customers, but less obvious is their own particular service culture, which
175
underpins this and may or may not support service delivery. Culture is a vital
ingredient in the service delivery process, yet for all that people talk about it, it
seems to have such an intangible, hard-to-identify property. It is often defined
as the "way we do things" or the "collective mindset". It is almost like "magic
dust"; when you have it you are head and shoulders above the rest, when you
don't you're left struggling to catch up. Virgin, British Airways and First Direct all
seem to have that special something that sets them apart in the customers'
eyes and even enables them to branch out into new businesses. This article
looks at how to get, and how to keep, a customer-focused culture - and, in
particular, how to understand the key components of the culture in your
organization. This understanding is the first step towards improvement and
success in this area.
What should you expect from a service-oriented culture?:
- Everyone takes responsibility for sorting out customer issues. There isn't any
buck-passing. Processes flow smoothly from one department to another.
- The environment is open: people aren't covering-up mistakes or withholding
information from each other and the customer.
- Service delivery is consistent, it doesn't depend on whether you are served by
a particular person. Measurement and performance management is often part
of this, agreed by consent, not imposed from above.
- There is a common language and shared assumptions, which rule out ghettos
and "turf" wars. In our experience this means conflicts and different
assumptions are worked through, not hidden away. Personal contact is made
with the customer: even in the briefest of interactions an appropriate warmth or
intimacy is created.
176
- Leaders and opinion formers are visible in personally demonstrating
customer-centric behaviour.
- Regular feedback is sought from the customer and the results acted upon.
Culture can be very resistant to change. Recently one of the authors contacted
a number of newly-established direct telephone insurance companies. These
were often off-shoots of rather slow, customer-indifferent groups. Even though
all the technology was in place the "old" culture pervaded - unfriendly service
and cumbersome ways of dealing with the customer. Why First Direct have
done so well in telephone banking is that they recruited customer service staff,
not those with a banking background.
Customers quickly form a lasting view of how they are dealt with, what Jan
Carlzon described as the "moment of truth". It is formed by the quality of the
"smile" you receive from reception or the telephone and how easy, or difficult, it
is to do business with that organization. For example, a customer satisfaction
study 18 months ago into the RAC Motoring Service revealed people who
called in felt the service was unfriendly. Some organizations seem to make the
whole process seem a miserable and time-consuming experience.
What is it that makes the difference? You can identify some very solid tangibles
- technology, training, the product itself and how the service is designed, for
example. Yet what makes service special is the people. Recruitment certainly
plays a part: anyone who had a good experience at a restaurant or hotel and
then has returned a year or so later, only to be disappointed by completely
different staff, will attest to that. Theme restaurant TGI Fridays goes to
enormous lengths to recruit people who fit the lively service culture, using
selection tests, and an assessment process including an entertaining "audition"
against clearly defined criteria. It rejects far more applicants that it appoints,
searching for an important set of qualities. Research defines the competencies
177
of front line service employees very much in terms of the ability to listen and
empathise with the customer; even under pressure.
In the space of five years, Birmingham Midshires Building Society transformed
its organization with the introduction of a highly customer-focused service
strategy which included a culture development programme. Nurturing and
sustaining competent service delivery is what a service culture is all about,
based on a credo of everyone "going the extra mile" for the customer.
When employees are asked to write down the most important measures of
company success, do customers and profit come out top and what relationship
do they see between the two? Underpinning the behaviour customers
experience is a clear set of values, which people live by. Many organizations
have "gone through" values programmes and display a set of words in public
places. Far fewer make the "public" and "private" values consistent. At the
customer service department of software company SCO, staff group meetings
were held to thrash out customer values which were translated into
performance measures that staff themselves had identified. Kwikfit displays
signs everywhere that say they aim for 100 per cent customer delight. Its
culture genuinely supports these statements - everyone is reminded from day
one that the customer comes first, and how this can be delivered in practice. It
is reinforced by same-day telephone surveys of customer reactions. Harvester
Restaurants hold induction days to introduce new employees to the service
culture and encourages them to make a personal commitment to action.
Culture is often reinforced repeatedly. British Airways is one of an increasing
number of service leaders to have adopted a balanced business scorecard
which seeks to put customer focus on an equal footing with finance, employee
satisfaction and operational measures. Events are held to drive home the
178
service message. Hi-fi retailer Richer Sounds has frequent lively team
meetings to motivate and focus customer delivery.
Service culture checklist
The following checklist summarises our experience of some of the things you
need to do if you are to promote a service culture:
- Recognise the substantial impact of culture on service delivery.
- Identify your organization's culture, its strengths and weaknesses.
- It is easier to build on the culture you have than attempt a complete
demolition. This requires a recognition that all culture has its good points and
enablers, as well as bad points and blockers. We were recently reminded of
this in our work with a privatised rail company. An "old hand" engineer said to
us, "If only my managers would recognise it wasn't all bad in British Rail".
- Be prepared to "let go" of aspects of your organization's culture which, though
valuable in the past, are no longer useful.
- Think carefully what behaviour is rewarded in your organization and what
impact this has on customer satisfaction.
- Strengthen and reinforce the customer-focused culture through organised
events and the day-to-day management of service performance.
- Actions speak louder than words - leaders must walk the talk, demonstrate
customer service in big and small ways. Most leaders fall short here and then
wonder why other people don't take seriously their exhortations on the
customer.
179
- Listen to the customer - everyone says they do but few regularly do so and
take on board the messages.
How to strengthen your service culture
Cranfield School of Management has developed a powerful diagnostic tool to
pinpoint the degree of customer focus in an organization. It is called the Culture
Web (see Figure 1) and at the centre is your view of the customer and the
world. It is revealed by examining, in service teams or management groups, a
series of seven interlocking components that together make up culture.
Using the Culture Web as a framework for talking about culture we can identify
aspects of organizational life that help or hinder a customer-responsive culture.
The paradigm and mindset
How organizations view the world will clearly have a direct impact on the way
that their people deal with customers. A recent radio programme investigated
the trend for London restaurants to charge for "no shows", no matter what the
reason for cancellation. An owner defended the practice, but suggested that
they might be able to review the practice once customers had "learnt how to
behave properly". Consider the impact this owner's attitude might have on the
way his staff viewed the customer.
Behind attitudes to the customer are assumptions that often go unchallenged
because they appear to have worked for us. It is only when a newcomer joins
the organization or a competitor changes the rules of the game that these basic
assumptions are reviewed.
Direct service operations provide a good example. Until recently the paradigm
of the insurance industry was an unchallenged belief that most business would
be carried out through branches or brokers. Direct Line challenged that
180
paradigm, recognising that customers wanted the flexibility of dealing without
an intermediary at times that suited them.
We frequently come across two mindsets which kill good service:
- "It would be all right if it weren't for the customers". Sadly, it seems that many
organizations' psyches contain a belief that customers are trouble. A true belief
that customer service really is a major priority is harder to find than it should
be.
- "We know what the customers want". This is potentially a more dangerous
assumption. It tends to breed a blinkered attitude, making it impossible to pick
up the signals that the market may be changing while we are not.
Key questions:
- Is the real focus on satisfying customer needs - or delivering what we do (and
have always done)? For example, some business schools and training
consultancies have fallen into the trap of delivering their successful
programmes year after year instead of helping organizations and individuals to
change and develop.
- Is there a belief that strong customer relationships are essential for long-term
survival and profit, or is short-term profit the overriding goal? (This leads to an
internal focus on costs, not the customer.)
- What are the central preoccupations of the organization? Where does the
customer fit in? What is the CEO obsessed with - for example is it innovation?
Profit? Status symbols?
181
Organization
The way a company is organised can have a profound effect on how easy it is
to do business with. Many companies have customer service organizations
which act as a barrier between the company and the customer. Customer
complaints may be handled "efficiently" by this group, but very few people meet
and deal with customers or hear what really upsets them. In contrast,
Birmingham Midshires Building Society makes a principle of handing
complaints to the people who process the work rather than employing a
separate complaints department.
Much of the value in recent business process re-engineering (BPR) projects
has been a greater emphasis on linking together processes and activities
which contribute to customer value rather than functional expertise and
generation of customer focus rather than operational focus. An example is the
creation of teams to handle the entirety of a group of customer transactions. At
Cigma in Greenock, a company which handles employee benefits for corporate
clients, there used to be functional groups, handling sales, service and credit
control separately. Today, client-focused teams have led to quality and
productivity improvements.
Key questions:
- Why has your organization structure evolved to its present form? Is it driven
by functional or regional requirements or by processes which are customer
focused?
- Do your organizational structures prevent customers getting in touch with
those who make decisions in the organization?
182
- If you were the customer, does the organizational structure make it easy for
you to do business?
Power structures
- This aspect of the organization's culture is particularly important when it
comes to changing the way things are done. Power structures may have
nothing to do with the way the organization chart is drawn: some individuals
appear to have far more power than their status would suggest, either because
of the force of their personality or because they exert some power based on
expertise. The problem arises when this works against the customers'
interests: some employees delight in withholding information from customers
because this seems to gives them perverse satisfaction. Managers must be
alert to this, as it is often a signal that these employees feel undervalued by
customers and organization alike.
An individual manager is able to block much needed change because it may
dilute his or her traditional power base. In working with a water company it
became clear that a major factor in the delivery of a customer service strategy
was the extent to which a group of relatively junior engineers would buy into a
new customer emphasis. This group had the ability to re-interpret and prevent
anything happening which didn't fit their views. The implementation of the
service strategy therefore had to allow time for building understanding and
commitment from this group.
Of course, power can be used very positively to engender customer focus. No
one can dispute the influence of Sir Colin Marshall at British Airways or Sir
Tom Farmer at Kwik Fit in driving through improvements in customer service.
Key questions:
183
- Who has the power? Is customer service represented at the top?
- How much power do front-line employees have over service delivery? Who
has decision-making power?
- In designing new services, which is most important - the voice of the
customer or the influence of empire builders?
Controls
Most of us are aware of the importance of measuring and managing service
performance. Tony Hughes, operations director of Bass Taverns' frequently
uses the quote "What gets measured gets managed, but what gets rewarded
gets done". In reality, a Cranfield survey suggests financial measures far
outweigh any other measure in most companies.
Positive examples may be provided by those companies which have
consistently rewarded people for customer satisfaction performance. A
company that has unswervingly recognised customer satisfaction performance
is Rank Xerox, which has consistently measured its performance against four
key performance indicators: customer satisfaction, employee satisfaction,
market share, and return on assets. It has built up its business by putting the
priority very much in that order, with the firm belief that satisfying customers
and employees leads to enhanced market share and therefore return on
assets. Avis has consciously designed non-financial measures, such as
telephone friendliness, which are weighed as equally important as "hard" sales
and financial data. BT Mobile has a wide ranging "dashboard" approach in its
company performance indicators: key measures are employee satisfaction and
internal quality.
184
Kwik Fit has focused the efforts of branch managers and staff on profitable
customer service by removing branch management of inventory control and
administration and putting them centrally. To maintain a business and
customer focus, each group of branches had its own "profit and loss" account
and measures against the mission of "100 per cent customer delight".
A prime control is often headcount - senior managers often use it to manage
cost, but it does not relate well to business development. One water company
imposed strict headcount restrictions which led to a succession of temporary
service staff, and hence deterioration of service from demotivated short-term
staff.
Key questions:
- Which control system or key performance indicator is the one which people
look at before everything else? How credible is a customer satisfaction index?
What behaviour do you observe as a result? Does this enhance or detract from
customer focus?
- Are controls for internal purposes? Are they for the benefit of the board or
accounts function? Whose benefit is it for?
- Are controls centralised/decentralised? Loose or tight? What effect does this
have on customer service?
Routines and rituals
What are the patterns of behaviour in the organization which impact on how we
deal with the customer? These are the activities which are not necessarily in
the company procedure manual but nevertheless have special significance for
the organization. They might range from the "informal system" such as ways of
185
getting round red tape, to celebrations of success such as pub nights or
parties.
The managing director of Credit Card Sentinel, Philip Williams, sends thank
you cards to people that have done well, a relatively inexpensive exercise
which has tremendous returns in employee satisfaction. Avis practises "visible
management", where head office managers spend time each year in the field
helping to rent cars.
Other routines and rituals erode the relationship with the customer; they
become a way of dealing with the difficulties of the job at the expense of the
customer. A example is provided by hospitals who employ rituals such as
asking everyone to put on pyjamas and night clothes even when their condition
does not require it. One interpretation of this ritual may be to establish an
ordered regime, with uniformed doctors and nurses in charge.
In the middle of an appointment to buy a child's first school uniform the sales
assistant announced she was going to her tea break. What does it say about
the importance of the customer?
Key questions:
- What routines do we have which send unfriendly messages to the customer?
- Have rituals developed from past practice which today work against the
customer?
- How often are customer successes celebrated and rewarded? What about
failures and problems being dealt with?
Symbols
186
Symbols are very important in organizational life. Who has a parking space or
a fitted carpet in their office (or who has an office at all) become major talking
points. They take on a life of their own, as anyone who has been involved with
a company car scheme will testify! Symbols may be a human role model, so
certain charismatic leaders may become symbols of change. Richard Branson
of Virgin would surely be a good example of a bold, unstuffy champion of the
customer.
The power of symbols became obvious in a security alarm company. In this
case sales staff were given cars, service engineers drove vans. It sent a clear
message that service was the poor relation. It was changed by providing
service engineers with estate cars which could be for private as well as
company use.
In Bass Taverns, certificates of competence were introduced to service
providers with some fanfare and became a symbol of success. Staff self-
esteem began to rise as they felt better about their contribution, and ownership
of customer issues increased, which showed itself in customer satisfaction and
business performance.
For anyone involved in an automobile accident, there is a lengthy and
inconvenient process to get your car repaired, involving questions, arguments
and forms. Recently an American insurance company has changed these
rituals - it simply videos the accident damage and resolves the matter in hours
with minimal paper work.
Key questions:
- What do our everyday symbols say to our customers and the way we think
about them? For example, the high, forbidding reception desk, the Post Office
queue, the sleepy bookshop with a cat snoozing in a corner.
187
- Do past symbols link in with our current business strategy and how we want
to be perceived?
- What contractory messages do we send? For example, we have seen a
notice expressing the importance customer satisfaction prominent behind
chatting receptionists and unanswered phones.
Stories
What stories people tell say a lot about customer attitudes in the organization.
These are sometimes called the "war stories" and are generally told to new
starters. They may be positive ("we're ahead of the competition, this is a good
place to work") or negative ("welcome to the mad house, don't take any risks,
keep your head down").
In some companies, stories circulate of staff who have been disciplined for
giving relatively small refunds because they didn't follow company red tape.
Some organizations have successfully influenced their service culture by
communicating a new set of stories linking acts of customer service to
business success: a Federal Express employee taking the initiative by hiring a
helicopter to get an essential package to a customer; an employee of Marriott
Hotels travelling all across town late at night to buy a familiar brand of hot
chocolate for a sleepless guest.
Key questions:
- What do company stories reflect about your treatment of your customers?
- How would stories reflect successful service "heroes" or role models?
188
- What are the themes of the stories which circulate? Do customers feature, if
so how?
Actions for a customer-focused culture
Organizational culture is always evolving; it is being reinforced, created and re-
created every time an employee represents their company to the customer.
The best way to understand your company culture is through dialogue with
other people in the organization and with customers.
The culture web provides an accessible framework for this conversation.
Usually it helps to have an external facilitator to guide the discussion and to
challenge, clarify assumptions and test what people mean .
We recommend a stepped process which will produce a plan of action:
- Map the existing culture using the service culture web.
- Review what you have put in this web and talk over with colleagues what
helps and hinders customer ownership, responsiveness and focus.
- Agree what needs to change to strengthen a service mindset.
- Look for actions to take in each aspect of the web that makes up the whole
service culture.
- Track change, using agreed measures.
189
Article 4
Quality Over Quantity
190
Quality Over Quantity
Mary K Pratt. Computerworld. Framingham: Jul 14, 2008. Vol. 42, Iss. 29; pg. 35, 1 pgs
Abstract (Summary)
Rick Franckowiak, director of the technology office at Johnson & Johnson
Pharmaceutical Research & Development LLC (J&JPRD), led an application
support project that brought a strategic shift in how services are delivered,
trimming the number of vendors while also increasing the quality of services
and cutting costs. Much of the work needed for a successful outcome was
done internally. Franckowiak says his team also used the project to gradually
introduce offshore services and to develop and implement more detailed
metrics to measure success and customer satisfaction. The team also had to
push the vendor for continual process improvement, which the contract
specified, says application support manager Frank Drust.
Full Text
(703 words)
Copyright Computerworld, Inc. Jul 14, 2008
This drug firm's approach to application support uses more service-level
metrics and fewer vendors. By Mary K. Pratt
RICK FRANC KOWIAK and his staff were facing rising costs for application-
support services that, despite the burgeoning price tag, could have been better.
So the team took action.
191
Franckowiak, director of the technology office at Johnson & Johnson
Pharmaceutical Research & Development LLC (J&JPRD), led an application
support project that brought a strategic shift in how services are delivered,
trimming the number of vendors while also increasing the quality of services
and cutting costs.
"Making a switch from a head count to servicelevel approach was a major
change, both culturally and in terms of process. But now, service levels are up,
and costs are down significantly," Franckowiak says.
Management of product support services is becoming increasing complex,
important and costly, says Bob Igou, an analyst at Gartner Inc. "IT
organizations are highly challenged to free up some money to do the new stuff
that the lines of business want and still keep last year's stuff running and upto-
date," he says.
Therefore, companies are trying to improve the management of their support
services to rein in costs and improve customer satisfaction. "They're paying big
bucks to get software support, and they're engaging with their vendors and
saying, 'What are we getting for this money?' " Igou says.
When J&JPRD started the application-support project, Franckowiak's
technology office oversaw a portfolio of more than 90 business applications.
The company had five major vendors providing support, with contracts focused
on the number of individual contractors rather than overall service levels,
Franckowiak says.
The four-member application support function team started the project by
examining different approaches to improve management. It opted to go with
just one vendor, charged with managing to a specified service level,
Franckowiak explains.
192
IN-HOUSE WORK
Yet vendor selection was only part of the process. Much of the work needed for
a successful outcome was done internally. Over two years, the team had to
build consensus around the project within both the IT department and the
business divisions, says application support manager Bart Leplae;
communication was essential to success.
Leplae says team members also categorized applications as "gold," "silver" or
"bronze" based on their importance to the business. Gold applications require
the quickest resolution times.
Franckowiak says his team also used the project to gradually introduce
offshore services and to develop and implement more detailed metrics to
measure success and customer satisfaction.
Despite its ultimate success, the project presented some lessons to be learned.
For example, the IT team came to recognize the importance of having the
vendor place the right employees in key management positions, Leplae says.
The team also had to push the vendor for continual process improvement,
which the contract specified, says application support manager Frank Drust.
"We shouldn't be making all the recommendations. At first we were, but we had
to push the vendor a little bit more; we wanted them to be proactive," he says.
Now, four years into the five-year contract, Drust says the process is running
smoothly enough to allow that to happen.
