I'm a Chief Experience Officer - Giarte

I’M A CHIEF EXPERIENCE OFFICER

USER EXPERIENCE

SLA

xLA

02 USER EXPERIENCE WWW.GIARTE.COM

USER EXPERIENCE

I’m a Chief Experience Officer

Written by Marco Gianotten

In the past, innovations like the PC and mobile phone first made their entrance on the business market. Only much later, following substantial price decreases, they became commonplace among consumers. Now the reverse is true: companies mainly follow trends that emerge in the world of consumer technology. The consequence: end users place high demands on Corporate IT and behave like business-consumers. Many CIOs realise that user experi-ence (UX) will play a prominent role within IT performance management. This chapter discusses the why and how.

We live in an economy that increasingly

revolves around perception: what customers

feel, think and experience is the driving force.

Social media, comparison sites and user

reviews feed the experience economy and

change the way companies and customers

interact, not only digitally but also physically.

In the retail world, the credo is increasingly

‘bricks and clicks’: strong growth in online

retail is regularly associated with investments

in the conversion of counters, shops, show-

rooms and bank branches into experience cen-

tres and flagship stores. The experience

economy might look like a fad, but the way in

which managers’ measure the performance of

their own organisations has already changed

significantly. At more and more companies,

customer service is no longer judged in terms

of efficiency (for example, the number of min-

utes per customer contact or costs per FTE)

but in terms of customer experience. Many

companies rely on the Net Promoter Score: the

extent to which customers recommend the

company or product to friends and colleagues.

IT has a major impact on customer experienceUser eXperience (UX) plays a prominent role in

the services and extends to the design, the

marketing and the management of customer

channels, such as websites and contact centres.

Services and products may in no way harm the

customer experience. Designing, testing and

improving the usability of websites is common-

place in the world of e-commerce. Companies

do their utmost to help customers as much as

they can during the orientation and sales pro-

cess and actively try to avoid ‘shopping cart

abandonment’. In e-commerce, there is a direct

relationship between usability and commercial

ratios, such as average revenue per order and

repeat visits. Mobilising positive customer


experience produces sales. The growing influ-

ence of IT on our daily lives as consumers

makes an ever increasing impression on our

user experience. If online banking is briefly

unavailable, this immediately causes negative

emotions: that are socially contagious. The

problems with ING’s Dutch internet banking on

3 April 2013 produced more than twenty-

thousand tweets (#ING) in just a few hours.

The employee is also a consumer In the experience economy, consumers hate it

when companies do not do what they promise,

if helpdesks do not really help or are hard to

reach, or if the information provided is inade-

quate or appears to be incorrect. The experi-

ence economy also hurtles into the world of

business IT. The modern consumer is not only

found in the high streets or online stores, but

also within your own organisation. There they

are ‘dressed’ as employees and end users, or the

internal IT customers. As end users, they adopt

more and more consumer behaviour: IT sys-

tems should ‘just work properly’ and the IT

department ‘should be easy to do business

with’. End users hate it when they have to call

back because the problem is not completely

solved, because they have to keep on repeating

what the problem is, or because they are always

faced with people who are not empowered to

solve the problem.

What does the end user actually want?End users expect user-friendly business appli-

cations and portals, a decisive service desk and

smart mobile apps that allow them to monitor

or control key processes. As organisations dis-

cover that working entails more than bricks,

brains and bytes (the key elements of the New

Way of Working, short NWoW), the expecta-

tions of IT are raised. With another way of

working for employees, the emphasis shifts to

behaviour: the conduct, the content of the

work, working together in multidisciplinary

teams and learning by sharing knowledge in the

workplace with co-workers. The performance

of IT is becoming increasingly important here:

it affects the personal productivity of employ-

ees. Users (and their managers) are therefore

becoming more critical and verbal: they con-

User experience is more than just technology. The technical performance of systems is often different from what users actually experience. There may be a considerable difference between the absolute availa-bility of core systems (usually far above 99 percent on the basis of 168 hours per week) and the perception of the users. What do they notice in practice of less than 1 percent technical downtime?

Health insurer Agis measured how its IT users experience availability. Their expe-rience was much worse than one would suspect from the technical availability. Every minute of downtime was magnified, especially during peak periods; systems’ slow responsiveness was seen by users as a form of unavailability.

Poor usability of business applications leads to longer learning curves, lower labour productivity and dissatisfaction. Usability sometimes also comes under pressure with off-the-shelf software. For example, Microsoft was harshly criticised during the introduction of its software package for Windows 8 because of the high learning curve and an inconsistent user experience: Windows 8 works diffe-rently on a tablet than on a desktop.


sider properly working IT as a matter of

hygiene; problems and obstacles in its use have

now become dissatisfiers. If users complain

about the sluggishness of a core application

such as CRM on their tablets, the IT depart-

ment does not stand on strong ground if it

argues that ‘the application is performing prop-

erly according to monitoring in the data centre’.

There is no point relying on user experience if

the negative user experiences are downplayed

or dismissed with the argument that ‘the SLA

has been complied with’. In other words: the

most important step is to recognise that expe-

rience, and thus the perception of the end user,

is the truth.

What do IT organisations do?IT organisations are until now used to measur-

ing and managing mainly on the basis of techni-

cal performance. With classical office

automation, that was hardly ever a problem: IT

organisations – or their service providers in the

case of outsourcing – had their own servers,

networks, workstations and applications. If an

application on a PC was too slow or hard to

access, it was fairly easy to figure out the tech-

nical causes because IT had control over the

entire chain. Nowadays more and more func-

tionality is virtualised: delivered to notebooks,

tablets and smartphones – purchased by

employees themselves – by means of web appli-

We are the 99%

The end-users

many CIOs realise that user experience will play a prominent

role within performance management


cations or with mobile apps. With trends like

Bring Your Own Device (BYOD) and mobile

internet, IT loses its absolute control over,

for example, workplace hardware, operating

systems on devices and networks. As a result,

control over user experience also declines. But

even without this control over the technology

you have to be accountable for managing and

improving user experience.

Classic SLAs are for technocrats The good news is that more and more CIOs

recognise the importance of user experience.

More and more IT departments are conducting

research into user satisfaction. Nevertheless,

customer satisfaction and user friendliness are

often the poor relation in IT. After years of

relying on ‘hard’ data, it is not easy to give

space to subjective indicators. IT departments

have difficulty initiating the appropriate

The new measure for satisfaction. Who dares? In its new outsourcing contracts, ABN AMRO has decided to rely on four KPIs that improve the cooperation between all parties in all the multi-vendor outsourced IT service supply-chains: aka as collaborative KPIs. Two of the four KPIs are subjective in nature: business satisfaction and project excellence. Business satisfaction is the annual score that key deci-sion makers award the IT. Project excellence is the evaluation of completed projects by business sponsors. The satisfaction is mea-sured on three points: a) quality of the deliv-ered functionality, b) lead time and c) value for money.

A few years ago, DSM decided to use KPIs in which the voice of the end user is paramount. As a single point of contact, it makes no sense to close an incident administratively (closing a ticket) with the aim of reaching agreements about the average resolution time of the SLA, while the problem is not properly resolved for the user. The opinion of the IT user has become overriding: after the ticket has been closed, he assesses whether he is satisfied with the solution by means of an online survey.

In an updated contract with KPN, ING has decided to work with a ‘new style’ SLA: an experience level agreement (XLA). XLAs are characterised by the lack of penalties, the focus on continuous improvement and a closed loop. If an individual user has a bad experience following a service moment (inci-dent handling, delivery from service cata-logue, answer to a question), he or she is called back to resolve the complaint. Even if the cause of the complaint lies with the solu-tion groups of third parties, KPN declares ownership of the entire chain for resolving incidents. This closed circle should lead to a culture of improvement (instead of a claim culture) where the user experience (rather than technical performance) occupies cen-tre stage. As with ABN AMRO, ING also has a collaborative KPI for all parties (thus includ-ing ING itself): the satisfaction of employ-ees about their workplace is expressed by a so-called happiness ratio. This is the ratio of users who rate their workplace with a 7 or more to those who award a 5 or less. The value 6 is neutral. The ratio is increased by one point each year.


improvement processes on the basis of results.

The methods used do not always provide any

useful control information; sometimes manag-

ers do not know which buttons they should

press. For example, little is known about which

leading indicators ultimately affect the lagging

indicator user satisfaction.

Focussing on user experience means that

Service Level Agreements (SLAs) must be

adapted to the new reality: not only managing

on the basis of technical control ratios, but

also on the basis of output. On the one hand,

this output affects the impact of IT on the

business and, on the other hand, in the percep-

tion of customers and users. In outsourcing,

new agreements – in the direction of XLAs,

eXperience Level Agreements – will also

have to be made.

User satisfaction 2.0: hard and soft User experience is becoming increasingly

important with both the design and functional-

ity of management and the corresponding ser-

vice. The measurement of user satisfaction

about the workplace or business applications

will have to be richer and deeper in order to

achieve a targeted improvement of quality.

It is important to combine soft data (opinions;

answers to open and closed questions) with

hard data (from IT Service Management sys-

tems, such as the active directory, service desk

tooling and CMDB) in order to identify causes

of discontent. In this way, differences become

clear between, for example, locations, type of

users and type of workplaces. The combined

data ensures that you can now provide the

answers. For the different categories of inci-

dents, what is the critical time period after

which the satisfaction plummets? Is there a

connection with working days or periods within

one day? And with the type of users? In addi-

tion to measuring the user experience, it is also

important to have access to tooling with which

user transactions can be measured for each

business process by means of a (mobile) device

and divided into components, such as render

time of the device, response time of the infra-

structure and the end-to-end transaction time.

Sentiments and discussions: the social part of UXBy analysing the open comments in satisfaction

research (with the aid of language technology),

the sentiments about corporate IT can be made

clearly visible and connections found within

Focussing on user experience means that SLAs must be adapted to the

new reality


Net Promoter Score In recent years, the Net Promoter Score (NPS) has successfully established itself in the row of top KPIs that (top) managers work with. The NPS indicates the extent to which custo-mers are willing to recommend a company or organisation. It is now ten years since the NPS was conceived by Frederick Reichheld. He saw the NPS as a solution for the problem that a high level of customer satisfaction did not appear to predict the loyalty of custo-mers. A disadvantage of working with the Net Promoter Score is that it is not exclusive; the figure is the total of the percentage of fans that awards a 9 or 10 minus the percentage of detractors that awards a 6 or less. The mid-dle group of passives is not included in the NPS score. An NPS of -5 can consist of 45% promoters, 10% passives and 50% detractors, but can also consist of 6% promoters, 88% passives and 11% detractors. These are sub-stantially different results with an identical final score. In addition, managers tend to opt for ‘low hanging fruit’: problem areas where

you can relatively quickly and easily imple-ment improvements in order to quickly realise higher NPS scores. NPS also takes no account of the effect of outliers: an average favourable NPS score produces a distorted image if the 5% most dissatisfied customers furiously stir things up on social media. In other words, the way in which the NPS is used also says some-thing about the management culture within an organisation.

Furthermore, considerable doubts are expressed among consumers about the value of the NPS. Kumar et al (2008) published an article in the Harvard Business Review about the results of an analysis carried out among 9,900 customers of a telecom company. It emerged from this that 81% of telecom cus-tomers said they would recommend their provider but only 30% of them actually did so. No more than 8% of those who received the recommendation eventually became a profi-table customer of the telecom company.

The good news is that UX can clearly be managed and that the results will contribute to a stronger IT-business alignment


multinationals too. The answer not only lies in

measuring and clever analysis; the physical dia-

logue with users is also important to encourage

empathy. More and more IT organisations use

customer panels to collect feedback. Rabobank

regularly organises IT user panels with a mod-

erator; IT professionals are present as listeners

and feedback (active listening) what they have

learned and would do differently in the services.

UX with Net Promoter Score: sense of non-sense for IT?There are various methods and sources avail-

able for monitoring the customer experience.

Some CIOs have implemented the Net

Promoter Score to measure the extent to which

users would recommend their own IT organisa-

tion. The advantage of the NPS is that this met-

ric is part of the marketing and sales

management lingua franca. However, there are

also a number of disadvantages attached to

working with NPS scores. Managers often do

not know what they can do to improve the rec-

ommended score. Furthermore, the NPS score

can be misleading and the question is also what

IT end users can and should do with the rec-

ommendation question, since they do not have

to deal with a free market with multiple provid-

ers for their IT.

Measuring user experience with Customer Effort Score In addition to the Net Promoter Score, a lot of

attention has been paid to a new metric during

the past five years or so. This is, more than the

NPS, focused on the service that companies

provide to their customers: the Customer

Effort Score (CES). The CES indicates the

amount of personal effort made by a customer

to get something done by the company. The

CES is applicable to proposals, questions,

orders, returns and changing data. The CES is a

five-point scale with the result being a number

between 1 and 5; the lower the number, the less

the effort. The CES is perfectly suitable as a

customer experience metric for IT organisa-

tions and can also be used in combination with

user satisfaction.

Another method that is widely used to monitor

the user experience is recording the user’s

opinion at ‘moments of truth’ (such as after the

delivery of a service or the resolution of an

incident) and then enriching this with ticket

data (such as the start and stop time, where the

elapsed time is the actual resolution time) in

order to identify causes of dissatisfaction in the

service chain. The recording of these opinions

during ‘moments of truth’ – often by means of

automated research following a telephone call

or email exchange – has already been used for

some time in the case of services provided to

consumers.

What is the ambition of the I T organisation? A service entity that mainly looks at the tech-

nology or a team that successfully contributes

to the growth and future of the business?

Those who have an eye for the latter, must be

open to matters such as (end) user experience

and experience level agreements. The good

news is that UX can clearly be managed and

that the results will contribute to a stronger

IT-business alignment.

GIARTEP.O. Box 8901000 AW Amsterdam The NetherlandsTelephone +31 (0)20 622 [email protected]

Documents

I'm a Chief Experience Officer - Giarte