Best Practices Roundtable: Quality Assurance and
Analytics
Sean Murphy, Director, Product Marketing, Speech
Analytics, Genesys
Patrick Botz, VP Workforce Optimization, VPI
Lou Cockerham, QA Product Specialist, inContact
Ann Thayer, CallFinder Account Manager – Speech
Analytics Solutions
Copyright Voice Print International, Inc. 2013
Optimization Best Practices
Top 5 Quality Assurance
©2014 Voice Print International, Inc. All Rights Reserved.
2
Founded: 1994 by current CEO, reputation for innovation
Headquarters: Camarillo, California, US-based R&D
Customers: More than 1,000 customers globally
Solutions: Contact Center and Back Office Workforce Optimization
Call & ScreenRecording
Speech & Data Analytics
Quality Assurance
Virtual Call Agents
Workforce ManagementPerformance Management
E-learning
VPI @ a Glance
Traditional Quality Monitoring
Quality of Customer-Agent Interactions
Wors
t P
ractice B
est P
ractic
e
Traditional
Random
Monitoring
High Value Low Value High Value
Traditional Approach to QA
Top Strategies to Improve Call Quality
48%
37%
26%
24%
19%
11%
0% 20% 40% 60%
Tie Quality Scores to Training
Use Analytics to Find Higher Value Calls
Automate QA Workflow
Speed Up QA Feeback to Agents
Ramp Up Calibration Sessions
Use Auto-scoring via Speech Analytics
Source: Call Center IQ Performance & Operations Report. 2014
Allow agents to flag calls
For coaching without scoring
To compete in ‘Olympics’ or ‘Academy Awards’
To create a Best Practices Call Library
Praise exemplary calls
Tie Quality Scores to Training1
Tie Quality Scores to Training
Assign Coaching Based on QA Scores
Send Message Based on QA Scores
to Find Higher Value Calls
Using Analytics
2
Types of Analytics
Account #:
Case ID:
Reason:
Model:
Notes:
New Case Commit
374982
289-820
Billing Issue
New Product A
Online PaymentIssues
11/28/2011 1:20:40 PM
Capture
Key Data
& Events
Organization:
Full Name:
Title:
Phone:
Address (Primary):
Email Address:
Reason Code:
Notes:
Account Number:
Case ID:
Voice Calling Plan:
Data Plan:
Model(s):
Shipping Preference:
Case Status:
Contact Details Order HistorySupport History Order Entry A/R
374982
Billing Issue
Customer having difficulty making online payments. Using Internet Explorer 9 Web Browser.
Advancetech, Inc.
Daniel Lee
Director of Operations
(602) 859-2904
27 Rockford DrivePhoenix, AZ 85042United States
289-820
Unlimited US and Canada
Unlimited Web and Email
FedEx 2-day
Tier 1
New Product A
SubmitCancel
Preparation: Creating Category ‘Buckets’
“Transfer to supervisor”
“Speak with someone else”
“Someone above you”
“In your department”
“Cancel”
“Confusing”
“Frustrating”
“Expensive”
“Difficult to use”
“Allworld Cellular”
“GlobalCom”
“Competitor is cheaper”
ESCALATION
REQUEST
NEGATIVE
SENTIMENTCOMPETITIVE
MENTION
2 TO 5
MINUTESCANCELLED
ACCOUNT
NEW
PRODUCT
Automate QA Workflow
60%+evaluator
productivity
Upsell Attempt-No Sale
3
BEST PRACTICE #3Speed Up QA Feedback to Agents
Encourage Feedback and Comments
Display QA Scores in Tickers & Scorecards
1
3
4
Calibration success depends on:
1. Monitoring Evaluation Form
2. Quality Standards Definitions
3. Calibration Process and Owner
Ramp Up Calibration Sessions5
Monitoring Evaluation Form
Should reflect what is important to your company and customers
Determine what knowledge, skills and abilities you are going to
evaluate
Organize questions in the flow of a call
Ideally, questions should be linear Yes, No or N/A
If using variable rating scales, define each grade in writing
Train all on pre-defined scoring benchmarks
Weight more critical questions heavier
Keep forms short and targeted
Design Best Practices
Outlines requirements for each question providing examples of behaviors
for each scoring range, if it’s an auto fail and what deems and auto fail.
Quality Standards Definition Document
Calibrated
5 standard
deviation
points
Schedule Calibration Calls
Evaluators Evaluate Calls Independently
Calibration Owner
Analyzes results calculates
Standard Deviation
Group Discussion
Analysts, manages & agents
discrepancies in evaluations.
Consensus Scoreor Re-Evaluate Call
Weekly until calibrated,
then monthly
Calibration Process and Owner
Thank You! Questions?
Patrick BotzVP of Workforce Optimization
Corporate Headquarters
160 Camino Ruiz
Camarillo, CA 93012
Direct: 1.800.200.5430 x5214
Email: [email protected]
www.VPI-corp.com
Learn best practices from 30,000+ others:
Easily Discover the Conversations
Goodbye, Traditional QA.
Hello, Speech Analytics!
The New Best Practice in Quality Assurance
Easily Discover the Conversations
• Manual monitoring practices:
– Time intensive
– Resource limited
– Difficult to identify coach-worthy calls
– Prohibits complete review and analysis of performance
– Lacks ability to benchmark performance and measure KPIs consistently
• Limited ability to document with accuracy
• Inadequate and statistically invalid sample sets
Challenges of Traditional Monitoring
Easily Discover the Conversations
Practices Are Changing
• Contact centers are pivotal in the customer journey
• Quality assurance programs need to change to meet the demands
of customers today
– It takes 12 positive customer experiences to make up for one negative
experience
– Consumers are 2x more likely to share their bad customer service experiences
than they are to talk about positive experiences
• Every phone call with a customer is elemental to the success of a
business
Easily Discover the Conversations
Speech Analytics is Transforming QA
• Speech analytics eliminates manual, time-consuming quality
monitoring:
– Rapid, automated drill-down to the most relevant, critical calls
– Efficient discovery of every conversation
– Unlimited ability to categorize calls by topics, agents, products, etc.
• Transforms knowledge from a shot in the dark to a direct hit!
Easily Discover the Conversations
Impact on the Bottom Line
• Small improvements in the quality of the
customer experience can have enormous
financial impact:
– 40% say the overall customer experience is a key driver
to spend more with a company
– A 10% increase in customer retention levels result in a
30% increase in the value of the company
– Companies that are in tune with customers' needs
experience a 28% year-over-year increase in revenue
Easily Discover the Conversations
• Recent article on speech analytics applied in the insurance industry
– Speech analytics revealed that once the word “ridiculous” appeared in an
interaction, the customer canceled four out of five times
– 80% more likely to change their insurer within three months
Practical Applications for QA
Easily Discover the Conversations
Results with Speech Analytics
• Other results directly related to implementing speech analytics in the
healthcare segment:
– 59% increase to inbound lead conversion rates
– 100% compliance with TCPA regulations, and reduction in litigation risk
– 90% reduction in hours to analyze conversations and identify successful script
transitions and timing
Easily Discover the Conversations
Misperceptions of Speech Analytics
• Introducing a new technology is too complex
• Resources to manage the complex technology
are limited
• Capital investment in hardware and
software is too high
The reality and advancements made
within the last few years tell a
different story…
Easily Discover the Conversations
Speech Analytics Facilitates QA, CX
• Redefine and expand QA programs:
– Introduction of cloud-based solutions
– More efficient and affordable
• Cloud-based solutions are prevalent and allow more ease of use
• Results are obtained within weeks of implementation vs months
• Vast improvements to the customer experience and bottom line
revenue
Easily Discover the Conversations
Impact of Insufficient QA Monitoring
• Consider the opportunity cost of not adopting new best practices
and speech analytics:
It is estimated that businesses & contact centers
lacking speech analytics technology fail to capture over 90%
of all customer communications, leaving an enormous
gap in business knowledge.
• Contact center speech analytics seats grew 26.0% from July 2013 to
the end of May 2014
• Estimated that the speech analytics market will grow by 20% in 2014
Easily Discover the Conversations
CallFinder’s Speech Analytics Solution
• Cloud-based call recording and speech analytics solution
– Affordable, flexible and scalable
– No hardware or software to install
• Fast audio processing
– Indexes thousands of calls within minutes
• One-to-one onboarding process with a specialist
– From goal identification, to search creation and refining to ensure optimal results
• 24 years of experience building telecom and marketing technology
solutions
• Providing clients with technology to improve their internal or external
contact centers
Easily Discover the Conversations
Ann Thayer
CallFinder - Account Manager
800-514-0436
www.linkedin.com/in/athayer
Let’s Connect!
Improving Quality Management with Interaction Analytics (Speech & Text Analytics)
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.32
Genesys Continuous Workforce Optimization
Front Office
Back Office
WorkforceOptimization
WorkDistribution
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.33
Challenges of Traditional Quality Management
Sample Validity
Typical:
• Less than 1% sample rate
• Results not very reliable
• Agents challenge findings
• Trending is poor
Ideal:
• Statistically valid sample, reliable findings
Objectivity of Findings
Typical
• Definitions are vague, multi-component, not linked to behaviors
• Difficult to get different evaluators of call to agree
Ideal
• Behaviorally anchored: Events defined by specific, observable behaviors
• Results highly consistent and reliable
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.34
Average Agent Score Using Analytics-Driven QM
Average Agent Score Using Traditional QM
Build Trust and LoyaltyCustomer Appreciation 45%Preferred Card 22%Encourage Usage 20%Average 29% 84%
Add ValueMention Cash Back 43%Mention Sweepstakes 24%
Mention Automatic Payment Plan 13%Mention Website 23%Mention Company Newsletter 13%Mention Customer Account 13%Average 21% 43%
The Solution: Interaction Analytics-Driven QM
POC Results: Major Financial Services Company
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.35
Analytics-Driven QM Best Practices
• Specific Agent Skills
• Build Rapport
• Contact Reason/Type
• Product Issue
• Billing Issue
• Etc…..
Topic Mapped Phrases
• How are you today?
• What’s the weather like there?
• I understand your frustration
Build Rapport
Product Issue• I have a problem with product “x”
• “x” function isn’t working
• How can I do “x” with the product?
Billing Issue• I have a question about my bill
• Why am I being charged for….
• You’re overcharging me
Step 1: Define all quality criteria and conversation topics as phrases
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.36
Analytics-Driven QM Best Practices
Step 2: Categorize all interactions according to the topics discussed
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.37
Analytics-Driven QM Best Practices
Step 3: Measure Key Performance Indicators (KPI) by topic
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.38
Analytics-Driven QM Best Practices
Agent with most repeat calls
Agent with low repeat call rate
Step 4: Measure KPI by topic for each Agent
Agent 1’s
Agent 2’s
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.39
AHT
CSAT
FCR
NPS
Pro
duct
know
ledg
e
Bill
ing
Issu
e
Pac
kage
or
Pla
n
Sys
tem
s
navi
gatio
n
Analytics-Driven QM Best Practices
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.39
•Step 5: Precisely Target Training, Coaching and eLearning Accordingly
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.40
Analytics-Driven QM Success Story
Process Description: Concentrix is a major Outsourcer with 37,000
Agents in 36 Countries Concentrix handles end-to-end Customer Care for
one of the largest North American Cell Phone Service Providers
Business ChallengesClient focus to increase customer loyalty and
satisfaction while reducing cost to serve.
Business pressure to:• Increase CSAT & Issue Resolution performance• Reduce Average Handle Time (AHT)
Solution DeployedSpeech Analytics Pilot followed by full deployment
Key Results• Improved CSAT Top and Bottom Box• Reduced Average Handling Time (AHT)
Account info
Industry Telecommunications
Lines of Business Customer Care
Speech Analytics
Engagement Type Pilot and FullDeployment
Benefits Realized (3 Months post deployment)
Customer Satisfaction (CSAT)CSAT Improvement 3rd Month CSAT Improvement 6th Month
Average Handle Time (AHT)AHT Reduction 4th MonthAHT Reduction 6th MonthAHT Reduction 9th Month
9.85%11.35%
115 seconds (19.6%)127 seconds (21.7%)136 seconds (23.2%)
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.41
Analytics-Driven QM Success Story
HRRG Success Story Background
• Collection agencies are required to introduce every call to a right party contact with a mini-Miranda statement
• “This is an attempt to collect a debt. Any information obtained will be used for that purpose.”
• Speaking the mini-Miranda is a legal requirement
• HRRG wanted to measure the extent to which agents were delivering the mini-Miranda and improve performance of those who were not performing to standard
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.42
Performance Before Automatic Monitoring
• In FL and CA, 17% of agents likely in compliance, 83% out of compliance
0%
20%
40%
60%
80%
100%
akm axe cxv dam dds dxj epr gmh jaj jxb kjc mlr mle nmc rms snm tdd tsr vmb
Agent Compliance: California
0%
20%
40%
60%
80%
100%
ajm
amu
cng
cxr
djd
edm fb
j
fxr
gxb
gxl
gxv
jas
jdr
jfm
jgm jja jm
f
jxt
klj lij lxf
mm
m
mth
mya
nd
a
np
c
nw
s
nxa
oxm paw pxh rc
g
rdr
rws
rxh
sjg
sjm sx
j
txw vl
j
wxl
Agent Compliance: Florida
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.43
After Two Months of Automatic Monitoring
0%
20%
40%
60%
80%
100%
akm axe cxv dam dds dlc dxj epr gmh jaj jxb mlr mjb mle nmc rms snm tdd tsr vmb
Agent Compliance: California
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
ajm
amu
cng
cxr
djd
edm
exz
fbj
fxr
gxb
gxl
gxv
hd
s
jas
jdr
jfm
jgm jja jm
f
jxt
klj lij lxf
mm
m
mth
mya
nd
a
np
c
nw
s
nxa
oxm pap
paw pxh rc
g
rdr
rws
rxh
sjg
sxj
tww vl
j
wjh
wxl
Agent Compliance: Florida
In FL and CA, vast majority of agents in compliance
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.44
Dramatic Gains
Compliance improved from 35% to 87% in Florida and 59% to 99% in California in two months
17%
27%
37%
47%
57%
67%
77%
87%
97%
Feb-11 Mar-11 Apr-11 May-11 Jun-11 Jul-11 Aug-11 Sep-11 Oct-11 Nov-11
Florida
California
© 2013, Genesys Telecommunications Laboratories, Inc. All rights reserved.45
How Interaction Analytics Improves QM
• Interaction Analytics can automatically evaluate 100% of conversations
• By removing human subjectivity, evaluations become more objective
• Interaction Analytics best practice is to define quality criteria as phrases
• Look for a solution that reliably recognizes phrases across all conversations
• For more information, please visit Genesys.com
or email me at [email protected]
Insert Cover Image Here
Title Goes HereLou CockerhamQA Product Specialist
inContact
47
To ensure consistency when evaluating agents
48
Helps to identify the variance betweenQuality and Operations
49
Helps to identify opportunities in call resolution and process improvement
50
Gaining agent buy-in and understanding of the Quality Program
51
Allowing upper management to make sound decisions based off of customer feedback
52
Helps to provide an overall understanding of the Quality Program
53
Outlines the set expectation of an agent within a customer contact
54
Do ensure consistency and discuss action planning to increase customer experience
55
Helps to identify opportunities and gain agentbuy-in
56
Training our ears to evaluate the true customer experience
57
Identify opportunities and successes through trending analysis to help improve the overall customer experience
– Cloud technology pioneer – 100%
focused since 2003
– Unparalleled customer satisfaction
– Supported 100% by inContact
experts
– 2,600+ implementations
– 150,000 agents
We Live the Cloud and We LOVE WFO
58
Best Practices Roundtable: Q&A
Sean Murphy, Director, Product Marketing, Speech Analytics, Genesys [email protected]
Patrick Botz, VP Workforce Optimization, VPI [email protected]
Lou Cockerham, QA Product Specialist, inContact [email protected]
Ann Thayer, CallFinder Account Manager – Speech Analytics Solutions [email protected]