Earley Executive Roundtable on Data Analytics - Metrics for Measuring the Customer Experience and Digital Marketing Success

Copyright © 2015 Earley Information Science1 Copyright © 2015 Earley Information Science

Earley Executive Roundtable

Series on Data Analytics

Metrics for Measuring the Customer

Experience and Digital Marketing Success

June 10, 2 015

Presented by

Seth Earley

CEO

Earley Information ScienceClick to watch a recording of this presentation

http://info.earley.com/earley-roundtable-3-metrics-measuring-customer-experience-digital-marketing-success

Copyright © 2015 Earley Information Science2

Today’s Agenda

• Welcome & Housekeeping

– Session duration & questions

– Session recording & materials

– Take the survey!

• Introduction

– Seth Earley (@sethearley)

• Panelist Introductions

– Dr. Asuman Suenbuel, Senior Technical Advisor, Global Startup Focus Program, SAP

– Gary Parilis, Chief Research Officer, Greenwich Associates

– Pratibha Vuppuluri, Founder & Principal, KeyInsite

– Stuart Williams, Vice President, TBR @s2_Williams

• Panel Discussion

• Questions & Answers

• Join the conversation: #earleyroundtable


Seth Earley, Founder & CEO, Earley Information Science

[email protected]@sethearley

• Over 20 years experience in data science and technology, content and

knowledge management systems, background in sciences (chemistry)

• Current work in cognitive computing, knowledge and data management

systems, taxonomy, ontology and metadata governance strategies

• Co-author of Practical Knowledge Management from IBM Press

• Editor of Data Analytics Department IEEE IT Professional Magazine

• Member of Editorial Board Journal of Applied Marketing Analytics

• Former Co-Chair, Academy of Motion Picture Arts and Sciences, Science and

Technology Council Metadata Project Committee

• Founder of the Boston Knowledge Management Forum

• Former adjunct professor at Northeastern University

• Guest speaker for US Strategic Command briefing on knowledge networks

• AIIM Master Trainer – Information Organization and Access

• Course Developer and Master Instructor for Enterprise IA and Semantic

Search

• Long history of industry education and research in emerging fields


Metrics for Measuring the Customer Experience and Digital Marketing Success

Core Concepts


Marketing Technology Ecosystem - 2014

5


Marketing Technology Ecosystem - 2015

6


Fast Evolving Landscape

7

January 2014:

947 Companies

January 2015:

1876 Companies


Making Sense of Digital Marketing Technology

• The purpose of marketing technology is to engage with users no matter

what stage of their lifecycle they may be in

• They may not know anything about you

• They may not know they have a need or want that you can satisfy

• They may be considering making a purchase but not know where to start

• They may know what they want and simply researching their options

8

How do we engage with prospects and customers?

How do we measure engagement?


Measuring engagement

Engagement is emotional

Engagement is logical

Engagement is contextual

Engagement depends on perspective

Engagement is about meeting needs throughout the

customer journey

How do we support the customer throughout their journey?

With blog posts, reviews, FAQ’s, how to videos, testimonials, product

specifications, user stories, events, communities, comparisons, instructions,

articles, account inquiries, customer support, service requests, etc.

9


Measuring engagement

• Analytics can measure the level of engagement at each step of the journey

• We can measure the effectiveness of digital content as well as internal

processes that support the journey

DIGITAL BODY LANGUAGE


The Customer Journey and Supporting

Enterprise Processes

Learn Buy Get Use Pay Support

Marketing Sales Distribution Service Finance Support

Customer Journey

Enterprise Processes: Departments/Functional Areas

• Event Management • Webinar tools• Promotion

management • Social media• Marketing resource

management

• Inventory Management

• Supply chain • Logistics and

Distribution • Point of sale

systems

• Ecommerce • CRM• Web content

management • Sales Management • Marketing resource

management

• Knowledge base/ Unsupervised support

• On line documentation/ help systems

• Ecommerce • CRM• Billing system• Web content

management• ERP/ Accounting • Credit card

authorizations/ EFT

• CRM• Knowledgebase/

Unsupervised support

• On line documentation/ help systems

• Call center call tracking

• Trouble ticketing

Key Concept: Each system and process will have application metrics, quality metrics, process metrics and financial metrics. However differences in attributes, data models and taxonomies will impact the ease of linking

internal process with external customer metrics.


Measuring here

(macro level -

outcomes)

Measuring here

(micro level -

effects)

Data Sources

Working here

(product data,

taxonomy, search,

on-boarding

workflow, etc.)

Enterprise Strategy

Business Unit Objectives

Market Share

Time to market Wallet Share

Business ProcessesAttrition programs Customer Satisfaction

WebAnalytics

CRM Data

Search Metrics

Processes enable

objectives

L I N K

A G

E

New accounts

Revenue Growth

Data supports (and

measures)

processes

Objectives align

with strategy

Linking “Digital Body Language” to outcomes

CEO: “Show me how web content will increase revenue.”

Conversions

Data Scorecards

Process Scorecards

Business Outcome Scorecards


Today’s Panel of Experts

Asuman Suenbuel, Gary Parilis, Pratibha Vuppuluri, Stuart

Williams


Dr. Asuman Suenbuel

Senior Technical

Advisor, Global Startup

Focus Program

SAP

• Asuman has been with SAP since 2004 in Palo Alto. As part of

her R&D activities, she held in 2009/10 a Guest Professor

position at the University of Potsdam, Germany teaching

Concurrency in Enterprise Systems.

• Prior to her current role, she served as Senior Director &

Technology Advisor in SAP’s Office of the CTO, where she led

innovative research and development in a wide range of topics

including IoT, big data analytics, NLP, multi-core programming

and more.

• Asuman holds a doctorate degree in computer science from the

Technical University Berlin, she studied at the University of

Passau, Technical University Berlin and did her Master’s Thesis

in ICSI and UC Berkeley in 1998.

• She holds more than a dozen patents, is co-author and author

of numerous publications, keynote speaker at conferences,

session chair at various conferences, past head of Industry

Chair Formal Methods Europe. She also received numerous

awards for scientific contributions e.g. in 2002, from the

European Association of Software Technology.

• She speaks several languages German, Turkish, French,

English and she is a passionate researcher & IT professional.


Metrics for Customer Experience

• Derived from the technical beauty and

complexity of a product.

• Simplicity: The complexity needs to be

well hidden from the user (the product must

be presented to the user in the simplest

way and solve the users problem);

– No “Umberto Eco’s Library”

– SAP’s new term “running simple”. QUEST

example

• Satisfaction: does the product solve the

user’s problem, keeps data save. Does it

do what it promises.

• Differentiation: State of the art versus

trendsetting aspect.


Metrics for measuring

• Time spent with a particular product to solve a

particular problem P .

• A positive experience with a certain product

will lead to solving a customer’s problem or

perform the task quickly (without rephrasing

or defining analogies).

• Consequently, a customer will have to spend

less time with the tool/product to solve a

particular task. Example: one click travel

reimbursement that will automatically pull all

receipts and prepare it for review versus

individual import of receipts.

• The repetitive use of a product for solving

another problem/task is a positive indicator


Metrics for Digital Marketing Success

• Digital product presentation:

– What problem is being solved and how? What makes me special in

this domain? Trendsetting: how is it different from the rest?

– Technical set up:

• Average page views per visit

• Scientific reputation

• How is a product presented, is it self explaining?

• Average cost per page view

• Instructions/guidelines/implementation steps require time, not everyone

is willing to spent time

• How much time to spent to digest the guidelines?

– Careful consideration of of review page; may negatively contribute

to a product; users usually go to a review site to share negative or

extremely positive emotions.

– Aggressive marketing might not always work, considered as

nuisance (filter out ad).


Gary Parilis

• 19 years in the market research/analytics and customer loyalty field both on the client and provider side

• Currently Chief Research Officer at Greenwich Associates, the market leader in research-based consulting for the financial industry. Leads R&D initiatives and heads the analytics & modeling function, as well as providing methodological guidance across the firm

• Concentrating largely on development of offerings designed to maximize ROI

• Also focused on advancements in segmentation for the financial industry

• Ten years at TNS, one of the world’s largest custom market intelligence firms, including 5+ years Senior Vice President, leading the Marketing Science function in North America

• Previously specialized in market research and customer experience analytics, on the provider side (four years dedicated to IBM) and on the client side (five years at Lucent Technologies)

• Earned an M.S. and Ph.D. in Social Psychology from Rutgers University, and held hybrid academic/administrative positions

Chief Research Officer

Greenwich Associates


POV – Gary Parilis

Don’t lose sight of this: the objective is not to optimize

your models; it’s to improve the customer experience

Modeling is only useful if it leads to action


POV – Gary Parilis

Don’t overcomplicate

– Paraphrasing George E.P. Box: All models are wrong; some are

useful

– An obsessive focus on model fit can make your models more wrong

and less useful; find the middle ground

Which of these models should you trust?

More

Generalizable

Better

Fit

21

CultureChange

BehaviorChange

Unified Vision

Coordinated Reporting &

Analysis

Basic Measurement

• Siloed research

• No connection between data and tangible action steps

• Standardized reporting/analysis across divisions & functions

• Still no CEM strategy

• Dedicated CEM leadership

• Consistent communication

• Common metrics

• Specific actionplanning

• Front line staff engaged

• ROI case solidifies

• CEM is a way of doing business

• Daily behaviorschange culture

• Technology is an enabler

1

5

4

3

2

The CEM Maturity Path

Customer Experience Management

becomes more effective as programs

mature

Copyright © 2015 Greenwich Associates


Pratibha Vuppuluri

• Over ten years experience in the financial services industry

including seven years in the Healthcare, Private Equity Secondary

Market and Technology Investment Banking space at both UBS and

Deutsche Bank.

• Founder & Principal at KeyInsite Inc, a marketing analytics

company, and a Co-Founder of NariNetwork, a woman-focused

digital media company.

• Currently serves on the Board of Screenpro (a Consumer

electronics accessories company) and mindHive (Marketing mobile

app company). She is also a guide at the Resolution Project.

• Authored and published an economic thesis on “The Impact of

Negative Economic News Coverage on Consumer Confidence”.

• Graduated from Cornell University with a B.S. in Applied

Economics and Management with Distinction in Honors Research

and completed her Masters in International Affairs (International

Finance and Economic Policy) at Columbia University.

Founder & Principal

KeyInsite

23

Analytics approach: (1) Predict information attention/popularity (2) Data mine conversations /

networks (web / social) (3) Optimize target audience

Point of View: Trend - Digital Customer

Optimization

Acquire

Activate

Retain

Identifying the buyer persona.Map and measure relationships and flows between interactions with the customer, URLs and other connected information/knowledge entities.Analyze and understand networks and the participants in relation to location, clusters, connectors, leaders bridges, etc. Take insights from the analysis performed to identify an accurate/close enough buyer persona . Develop a strong call to action for achieving the desired preliminary objectives. Metrics: new signups, application downloads, signups to newsletters/blogs, etc.Create the identified call to action from the acquisition step based on target demographics tastes and lifestyle.

Identify open rates for email marketing initiatives, click through rates etc to derive customer’s experience in terms of their affinity towards the product/service.Use A/B testing, Propensity models, Correlations, Multi-Variate testing, Collaborative filtering etc to derive the significance of the call to action phase and also to derive the effectiveness of the campaign.Recommended products through previous purchase history or segment attribution.

Discount programs, where participants receive instant discounts/benefits at the point of sale.Based on activity pattern, segment loyal consumers and target messages to promote the brand online for active offers.


Stuart Williams

• Serves as vice president of research, engaging TBR clients to

promote the value of business insight that helps them improve

business performance, capture opportunity and increase customer

engagement.

• He leads a team that translates client needs into actionable expert

analysis, sets the overall research and syndicated agenda, and

crafts effective deliverables with TBR executives, directors and

senior analysts.

• Brings more than 20 years of experience as an IT decision maker

and business adviser to IT vendors.

• An expert on technology commercialization, business strategy and

competitive analysis Stuart’s research focuses on IT business

models and innovation at the intersection of vendor go-to-market

strategies and enterprise customer behavior, budgeting and

satisfaction.

• Widely quoted in the press, including The Wall Street Journal,

Fortune, USA Today, The Street, ZDNet and CNET.

• Received a B.A. in communication at the University of New

Hampshire and an M.S. in management of technology from the

Whittemore School of Business & Economics.

Vice President

Technology Business

Research


POV – Stuart Williams

• Businesses need CEO commitment, focus on maturity

• First order metrics:

– Productivity measures (e.g. time saved, efficiency)

– Volume measures (leads generated, pages viewed, impressions)

– Financial measures (e.g. opportunities created, revenue generated,

ROI)

– Customer measures (e.g. NPS, Walletshare, TLV, cost-to-acquire)

• Second order metrics

– IT maturity and change management

– Business process change

– Internal cultural change


Discussion


Thank you to our sponsors/producers

www.computer.org/itpro

www.informationdevelopmentworld.com

www.thecontentwrangler.com

http://www.tbri.com

http://www.computer.org/itpro

http://www.informationdevelopmentworld.com/

http://www.thecontentwrangler.com/

http://www.tbri.com/


http://www.henrystewartpublications.com/ama

Applied Marketing Analytics is the major new professional journal publishing in-depth, peer-

reviewed articles on all aspects of marketing analytics. Guided by an expert Editorial Board

each quarterly 100-page issue – published both in print and online – features detailed,

practical articles written by and for marketing analytics professionals on innovative thinking,

strategies, techniques, software and applied research showing how major brands are

collecting, interpreting and acting on marketing analytics, both around the world and across

varied digital and non-digital marketing channels.

10% off - use code “Earley” when you subscribe.

To subscribe with the discount, either

Email:

Simon Beckett [email protected]

Or call:

800-633-4931 (in the US/Canada)

+44 207 092 3465 (in the rest of the world)

http://www.henrystewartpublications.com/ama

mailto:[email protected]


For more information

• IT Professional Magazine - www.computer.org/itpro Next issue focuses on Analytics

• Chiefmartech.com blog: http://chiefmartec.com/2012/07/agile-marketing-in-a-single-

whiteboard-sketch/

• Example marketing stacks http://chiefmartec.com/2015/05/marketers-really-passionate-

marketing-stacks/

• Beckon (marketing dashboard provider) white papers http://www.beckon.com/resources/

• Agile digital marketing at enterprise scale http://www.earley.com/knowledge/white-paper/8-

principles-agile-digital-marketing-enterprise-scale

• Applying science to the art of digital merchandizing http://www.earley.com/knowledge/white-

paper/applying-science-art-digital-merchandising

• Marketing analytics resources http://blog.hubspot.com/marketing/marketing-analytics-

resources (KissMetrics, Occam’s Razor, Google Analytics Academy, online courses)

http://www.computer.org/itpro

http://chiefmartec.com/2012/07/agile-marketing-in-a-single-whiteboard-sketch/

http://chiefmartec.com/2015/05/marketers-really-passionate-marketing-stacks/

http://www.beckon.com/resources/

http://www.earley.com/knowledge/white-paper/8-principles-agile-digital-marketing-enterprise-scale

http://www.earley.com/knowledge/white-paper/applying-science-art-digital-merchandising

http://blog.hubspot.com/marketing/marketing-analytics-resources


Next Session: June 17th 1pm EDT

Using Business Analytics to Drive Higher ROI and

0rganizational Change

Joanna SchlossBusiness Intelligence and Analytics Evangelist, Dell Software

Geoff WoollacottPrincipal Analyst/Practice Manager, Technology Business Research

Phillip KemelorSenior Manager, Advanced Analytics –Digital Analytics, Ernst & Young LLP

Bruce DaleyPrincipal Analyst, Tractica


Earley Information Science helps

organizations establish a strong

information architecture and

content management foundation

Specializing in making information more findable,

useable and valuable to drive digital commerce

innovation, enhance customer experience, and

improve operational efficiency and effectiveness.

Realize your digital transformation vision

with EIS.

Earley Information Science

(EIS)A trusted information integrator

Founded – 1994

Headquarters – Boston, MA

www.earley.com

Seth Earley, CEO

Email: [email protected]

Twitter: @sethearley

LinkedIn: www.linkedin.com/in/sethearley

mailto:[email protected]

http://www.linkedin.com/in/sethearley


A Broad Spectrum of Business Solutions

DIGITAL BUSINESS SOLUTIONS

B2C Digital Commerce

• Product Curation for a World-Class Product Catalog

• Site Merchandising Taxonomy & Attribute Design

• Information Architecture for Shopper Context

B2B Digital Commerce

• Product Search & Findability

• Product Information Management

• Product Knowledge Management

Digital Workplace

• Enterprise Content & Records Management

• Information Architecture

• Enterprise Knowledge Management


EIS Reference Architecture


The Customer Journey and Digital Technology

Inbound H H M L M L

Outbound H H L M M H

Analytics H H H H M H

Integration H H H M M H

Transaction N/A N/A H N/A N/A N/A

Learn Choose Purchase Use Maintain Recommend


Outbound

Campaign Mgt

(ABC)

(DEF)

(GHI)

H H H H M H

Email marketing

(JKL)

H H M H M H

Social media

(MNO)

(PQR)

(STU)

H H L H L H

Soc media listening

(VWX)

(YZ

H H L H H H

GREENLegend Green

Yellow

Red

Tan

Appropriate functionality given current maturity and business objectives

Acceptable but not optimal functionality

Below required capabilities

Less critical functionality given current state and business strategy

YELLOW

RED

TAN

YELLOW

RED

TAN

REDRED

YELLOW YELLOW YELLOW

GREENYELLOW

RED RED

YELLOW

GREEN

Current Technology Stack

Maturity and capability leveraging current technology stack*

* Source: Interviews and Maturity Model

YELLOW YELLOWYELLOW

GREEN GREEN GREEN

YELLOWYELLOW

Learn Choose Purchase Use Maintain Recommend


Technologies Supporting the Digital Journey

Stage Goals Technologies Example Stack

Learn Build awareness through

advertising, campaigns, word

of mouth, web site content

Content management,

campaign management, SEO,

email, social media

Adobe AEM, Bronto Mail,

Ion, Google ads, FB, Twitter,

Instagram, DigiMind,

Luminoso

Choose Help customers learn about

the products and try to get

them into the store to try

All of above plus more

personalized content, on site

search

Above plus Salesforce.com,

Hubspot, Act-On

Purchase Transact with the customer

or get them to retail store.

Ecommerce tools (catalog,

shopping cart, order

management etc.)

Demandware, Hybris, Digital

River

Use Provide more personalized

content based on the

purchase, engage in social

media, answer questions

Chat, personalized email

messages and web content,

knowledge base, social media

tools

Adobe AEM, Bronto,

Hubspot, DigiMind,

Luminoso

Maintain Same goals as above plus

troubleshooting

Same as above with heavier

emphasis on reminders,

troubleshooting

Endeca, Hana, Magento,

Hubspot

Recommend Further support a positive

experience with events,

promotions, community

development

Greater use of email,

campaigns and heavy

emphasis on social media

listening and participation

Adobe AEM, Bronto, Google

ads, FB, Twitter, Instagram,

DigiMind, Luminoso

36


Digital Maturity

Stage

Capability

Stage 1

Unmanaged

Stage 2

Nascent

Stage 3

Evolving

Stage 4

Harmonized

Stage 5

Choreographed

Commerce

Experience

No connection of

promotions to on site

experience. Inability to

select and filter

Mobile friendly search,

browse and purchase,

Promotional content

surrounding targeted

customer through paid

and earned media

Shopping cart retrieval

with targeted just in time

offers based on past

behaviors, cross sell and

up sell driven by data

relationships and

merchandiser strategy

Agile promotions,

bundles, personalized

recommendations based

on customer data and

behavior

Custom vehicle design

and pricing with order

flowing to manufacturing

with flexible financial

models to compensate

dealer

Digital Asset

Management

Static & fragmented

content, poor digital asset

control, hand-crafted

channels

Content ownership

defined, aggregation from

multiple sources, lifecycle

monitored

Content tagging/reuse

moderate, digital assets

managed, editorial

guidelines in place

Content & assets

coordinated across

enterprise, channels, by

audience

Dynamic content

presented according to

device, context, geo-

location & segment

Product Information

Management

Poor data quality, manual

validation, limited

transformation

Category hierarchies &

attributes identified, semi-

automated data quality

processes

Business rules built into

product data quality,

manual hierarchy

mapping to downstream

systems

Data quality feedback to

upstream sources, docs to

data engineering

workflows and integration

Consistent cross-channel,

cross enterprise

mapping, integration with

domain analytics

Content and Site

Architecture

Poor site navigation, no

ability to search,

confusing content or

selection, content not

aligned with user needs,

disconnected from

shopping function

Integration of content with

ecommerce functionality,

faceted search baaed on

customer needs and

driven by personas and

use cases, content

strategy specifically

designed to assist

selection of vehicle and

accessories

Semantic search for

curated video assets and

knowledge base access,

configuration of custom

vehicle, tuned attributes

for faceted search

Adaptive content based

on attribute model that

considers demographic,

psychographic, social

graph and web site

behaviors to provide just

in time content., Avatar

interface to structured

content to answer

questions

Predictive analytics driven

personalized offers and

experience. Real time

integration with dealer

network, social media,

social graph data, third

party data, web site click

streams, single view of

customer data

Site Search

Basic site search without filtering, manual dictionary updates, no coordination with SEO

Curated site search categories, aligned with navigation, SEO-friendly landing pages

Increased precision, SEO synonyms in taxonomy, search dimensions optimized

Attributes drive comparisons & cross-sell, associative relationships drive ways to shop & up-sell

Multi-device search, mobile geo-location, 3rd

party site search is competitive advantage

Governance None in place

Brand, merchandiser

print & ecommerce

fiefdoms

Centralized, managed &

funded

Competencies support

maintenance processes &

workflow

Integrated practices

operationalized, impact

analysis is proactive37


Evolving Digital Maturity

Stage

Capability

Stage 1

Unmanaged

Stage 2

Nascent

Stage 3

Evolving

Stage 4

Harmonized

Stage 5

Choreographed

Commerce

Experience

No connection of

promotions to on site

experience. Inability to

select and filter

Mobile friendly search,

browse and purchase,

Promotional content

surrounding targeted

customer through paid

and earned media

Shopping cart retrieval

with targeted just in time

offers based on past

behaviors, cross sell and

up sell driven by data

relationships and

merchandiser strategy

Agile promotions,

bundles, personalized

recommendations based

on customer data and

behavior

Custom vehicle design

and pricing with order

flowing to manufacturing

with flexible financial

models to compensate

dealer

Digital Asset

Management

Static & fragmented

content, poor digital asset

control, hand-crafted

channels

Content ownership

defined, aggregation from

multiple sources, lifecycle

monitored

Content tagging/reuse

moderate, digital assets

managed, editorial

guidelines in place

Content & assets

coordinated across

enterprise, channels, by

audience

Dynamic content

presented according to

device, context, geo-

location & segment

Product Information

Management

Poor data quality, manual

validation, limited

transformation

Category hierarchies &

attributes identified, semi-

automated data quality

processes

Business rules built into

product data quality,

manual hierarchy

mapping to downstream

systems

Data quality feedback to

upstream sources, docs to

data engineering

workflows and integration

Consistent cross-channel,

cross enterprise

mapping, integration with

domain analytics

Content and Site

Architecture

Poor site navigation, no

ability to search,

confusing content or

selection, content not

aligned with user needs,

disconnected from

shopping function

Integration of content with

ecommerce functionality,

faceted search baaed on

customer needs and

driven by personas and

use cases, content

strategy specifically

designed to assist

selection of vehicle and

accessories

Semantic search for

curated video assets and

knowledge base access,

configuration of custom

vehicle, tuned attributes

for faceted search

Adaptive content based

on attribute model that

considers demographic,

psychographic, social

graph and web site

behaviors to provide just

in time content., Avatar

interface to structured

content to answer

questions

Predictive analytics driven

personalized offers and

experience. Real time

integration with dealer

network, social media,

social graph data, third

party data, web site click

streams, single view of

customer data

Site Search

Basic site search without filtering, manual dictionary updates, no coordination with SEO

Curated site search categories, aligned with navigation, SEO-friendly landing pages

Increased precision, SEO synonyms in taxonomy, search dimensions optimized

Attributes drive comparisons & cross-sell, associative relationships drive ways to shop & up-sell

Multi-device search, mobile geo-location, 3rd

party site search is competitive advantage

Governance None in place

Brand, merchandiser

print & ecommerce

fiefdoms

Centralized, managed &

funded

Competencies support

maintenance processes &

workflow

Integrated practices

operationalized, impact

analysis is proactive38

Data & Analytics

Earley Executive Roundtable on Data Analytics - Metrics for Measuring the Customer Experience and Digital Marketing Success