Decisions Management use case : Telecom Customer Support Automation

Telecom Customer Support

Process Automation

[email protected]

mailto:[email protected]

iSencia Belgium NV 2

Setting the scene

We work with Belgium's largest telecom provider

Traditional telecom services : • voice, internet, IDTV, mobile

Nomadic services : • personal cloud, streaming, video-on-demand, gaming, …

Millions of customers


Setting the scene

Technical support & repair• 15K calls / day• 10K field interventions• 700 call center operators• 2500 field technicians

After-sales, billing support• 30K calls / day• 1500 call center operators


Setting the scene

The network is the service. Geographically distributed network, multitude of street cabinets, devices, …

Many variations of complex products and line technologies

Rapidly evolving products and services

Long half-life of old products and devices


What was in the past


Old-style repair support process

Customer calls 0800-...

Gets an operator (after a while)

Operator tries to understand what the customer's problem is

And starts consulting all kinds of BSS/OSS support systems, launching network measurements, retrieving equipment states...

Manual analysis of the information

On-the-spot diagnosis and resolution decision

Call

ManualCollectdata

ManualAnalyse and

diagnose

Field intervention

Rem.intervention Solu

tion


This doesn't help either

Large organization with many separate teams

Each historically had own tools and methods

Customers get sent around between teams

Operators and technicians are often subcontracted

=> Volumes i.o. customer satisfaction

BSS/OSS systems owned by many application teams

Mix of legacy systems, new technologies, packages, in-house dvp ...


Issues

Customer often fails to correctly describe problem

Manual data retrieval and line measurements are slow

Customer and operator are blocked waiting for results

Approach and results depend on individual skills

Long average call handling times

Risk for errors & inconsistent support decisions

Technician's tools inconsistent with operator decision

Low first-time-right ratio

Low customer satisfaction

Huge learning curve and high recurrent training costs


Improving the support processes


Where do we want to go?

Customer Experience• Consistent support answers and decisions• Improved FTR

Simplification• Integrated view on customer, services, network• Automate steps where possible• Support the interactions with the customer

Efficiency• Always access to up-to-date best practices & formalized knowledge• Shorter handling times• Less rework, training, knowledge transfer


How to get there

DARE : Diagnosis And Repair Engine

Single system for automated and interactive support processes • Call Center Operators for retail and professional customers• Field Technicians

• OLO Operators and Technicians for regulated and commercial products

• Customer Self Repair• Proactive repair, automated line profiling, ...

Consolidated, up to date business logic with 360°customer view • Frequent business logic updates

• Data collection and remote repair in the network, even in home equipments• Integrated data views and interactive guidance


How to feed it

Call center coaches and field intervention experts extractsupport&repair cases and best practices

Organize support tasks :• Automated data collection• Interactive forms• Analysis modules• Diagnostic decisions• Advises for (automated) repair actions


How to feed it

Define and build a library of :• Data connectors to OSS/BSS systems• Analysis modules• Reusable sub-processes

Define domain-specific support processes :• reusing connectors and modules• specializing diagnostic decisions


Example

Refactoring the end-2-end process for repair support

X

DARE

IVR

1

2

34

4


Pretroubleshooting & troubleshooting

While customer on queue :

• automated pre-troubleshooting

• data collection• rules-based problem analysis• root-cause diagnosis• identify and prioritize troubleshooting processes• advise on possible repair actions


Pretroubleshooting & troubleshooting

When the operator gets the customer call :

• diagnostic report available• detailed underlying data• guided interactive troubleshooting


Diagnostic report


Interactive guidance


Decision examples

IVR

Filtering and routing decisions based on• Global outages, planned works in the area ?• Customer info : Bad payer? SLA? …• Customer's products• Language


Decision examples

Pre-troubleshooting process

• Check customer's ordered products• Measure the physical line characteristics• Read network configuration, device status• Reach to customer modem and other CPEs• Check service configurations FIA, IDTV, VOIP, …• Compare configuration in the network and service management

systems with expected values(e.g. activation of TV packages, VOIP features, ...)


Decision examples

Troubleshooting process

• Is the PTS diagnosis what the customer is calling for?• Can the issue be solved by the operator?• Must a ticket be created?• Is a field intervention required?


And now a bit more technical...


DARE system overview

IVRService

GatewayBIQ

E2E testingSALY UTS/eTS

Callbackmanager

Backends

NPS

ABRTechnical situation

TBF

ProcodacHlog & XDSL

parameters

SAFEexpected attenuation

ROSY(SALY GUI)

4TELDSLAM

BAS

ABD

FH

NFH

PSTECustomer facing

serv/Assigned prod

LAMLocal Address Management

SGWModem Interaction

HomerSkynet accounts

SOCCABISSkynet

login/password

DARE

DARE Rules Engine DARE Process Engine

Pretrouble shooting

DARE Manager

Syst. Admin

2

STB

HGW

IMS

NGN

UMS

…

Product catalogPILA

LLDOFA WFM

Operator Cockpit

CLI/CLE1

Troubleshooting AdvisorConsole

DATAMINERSTB Info


DARE modules

Built on iSencia'sPasserelle EDM

Orange boxes : extensions

Passerelleprocess engine

Jboss Drools rules engine

Asset repository

Processeditor

App

licat

ion

mod

ules

Backendadapters

Role-basedsecurity

Process scheduler

ApplicationPreferences

Executiontraces

Sup

port

mod

ule

s

WebServices

(SOAP & REST)

OperatorConsole

AdminManagerIn

terf

ace

laye

r

Backendsimulator

Diagnosticprocesses

Analysisrules

Task form entry

raw data views

End-user Console

Task form entry

WebServices

(SOAP & REST)


Asset repository

• Process models, rules packages, images, translations … • 2-digit versioning schema• Packaging/grouping of related groups of assets• Root-level : Project• Simple preparation, activation, roll-back of new project releases• Export/import of project packages between test envs & PROD


Backend adapters

• >150 adapters• Typically slow tasks (2s – 2min)• Asynch/synch bridge with internal buffering, resource pools

(minimize internal resource blocking)• Configurable throttling settings to protect backends from overload• New/updated adapters can be uploaded without downtime


Application preferences

• Stored in DB• Web UI to import/export, CRUD, …• DARE modules & services can be reconfigured “live”• Used for analysis thresholds, refdata,

backend connection settings etc


Some usage scenarios


Collaborative process definition

Take advantage of :• Web-based access

• Role-based security

• Fully-featured graphical model editor• Asset versioning, personal workspace,

model locking

• Activate,update,rollbacksets of related assets atomically

• Test & simulation modesPasserelleprocess engine

Drools rules engine

Asset repository

Workfloweditor

App

licat

ion

mo

dule

sBackendadapters

Role-basedsecurity

Process scheduler


Executiontraces

Su

ppo

rtm

odul

es

WebServices

(SOAP & REST)

OperatorConsole

AdminManagerIn

terf

ace

laye

r

Backendsimulator

Workflowprocesses

Analysisrules

Task form entry

raw data views

End-user Console

Task form entry


(Don't) work at night

For automated/pro-active/batch runs• Schedule runs day-and-night

• Not just remote in location, but also in time !

• Integrate automated self-diagnostics in your workflows

• Automated notifications when needed

• Role-based security for secure supportand collaboration

• Connect to your sysadmin monitoring toolsvia adapters, web-services, event logs etc


Drools rules engine

Asset repository

Workfloweditor

App

licat

ion

mod

ules

Backendadapters

Role-basedsecurity

Process scheduler


Executiontraces

Sup

port

mo

dule

s

WebServices

(SOAP & REST)

OperatorConsole

AdminManagerIn

terf

ace

laye

rBackendsimulator

Workflowprocesses

Analysisrules

Task form entry

raw data views

End-user Console

Task form entry


Execution monitoring & support

The platform enables :• Browse active workflows • Integrated execution tracing,

provenance inforaw data views, graphical execution traces,Task form replay ...

• Secure pause/stop/resume• Collaboration between business

analysts and IT support

• Testing options : mock testing, replay etc


Drools rules engine

Asset repository

Workfloweditor

App

licat

ion

mod

ules

Backendadapters

Role-basedsecurity

Process scheduler


Executiontraces

Sup

port

mo

dule

s

WebServices

(SOAP & REST)

OperatorConsole

AdminManagerIn

terf

ace

laye

rBackendsimulator

Workflowprocesses

Analysisrules

Task form entry

raw data views

End-user Console

Task form entry


Traceability approach

• Each process execution gets unique ID, communicated to client apps

• Each task's lifecylce is stored• With backend data,

analysis results etc• Internal monitoring,

timing & error statistics


Execution browser & raw data views


Graphical execution trace


Finally, log files!

• All those application features are nice but...

• In the end sys admins want log files!

• Errors, timeouts etc get logged with specific error codes• Monitoring tools have their own rule-based decisions to react on error

frequencies, assign tickets to application teams...


Lessons learned


Graphical tools: Sales pitch vs facts

• Graphical process models, rules-based logic …often presented as a way to let non-developers build business solutions

• We've noticed that this works for contained scopes• But for larger contexts, with long-term maintenance,

performance requirements etc …?


Graphical tools: Sales pitch vs facts

What works best in our situation :• Close collaboration• Process experts formalize the definitions• Functional experts turned developers maintain :

• Interactive processes with simpler decision rules• Threshold values for complex decision modules• Reference data

• IT team maintains :• Complex, high-throughput flows, modules• Builds adapters• & coaches the business developers


Agility and Testing

• Built for fast implementation & maintenance cycles.But errors have large impacts.

• Gradually evolved to traditional release-bound test cycles for non-minor changes

• Live changes still allowed for :• Threshold changes• Ref data• Urgent rules fixes


Data issues

• Data distributed across many applications, databases...• For repair processes we need live data from

network, devices, ongoing provisioning flows, …• Confronted with many intermediate layers & conventions• Backend systems all have different backgrounds, data

structures• Data quality issues become visible in your projects

(as in all integration solutions)• Where to find the authorative data source

(batch replications, plain duplication, ...)


Data issues

• We store lots of data per diagnostic run(approx 150M data items / day)

• Maintain 6 weeks of results in the operational DB• Older weekly partitions off-loaded to DWH• Decision analytics done there


Organization issues

• Meeting culture, budget allocations etc all slow down the real implementation work

• Many different teams are impacted• They may see your integration work as a threat• End-users will see your work as threat

(your decision automation project is paid forby cost-cutting programs)


Thank you

Software

Decisions Management use case : Telecom Customer Support Automation