1 A/R/T ForumJune 12, 2006 Copyright © 2006 Indiana University Customer Tracking Professor Ray Burke E.W. Kelley Professor of Business Administration Director,

1A/R/T Forum June 12, 2006 Copyright © 2006 Indiana University

Customer Tracking

Professor Ray BurkeE.W. Kelley Professor of Business Administration

Director, Customer Interface Lab

Indiana University

Alex LeykinPhD Student

Department of Computer Science

Indiana University

222A/R/T Forum June 12, 2006

The Evolution of Marketing Intelligence

• Wave IBrand and Category Management

• Wave IICustomer Relationship Management

• Wave IIICustomer Experience Management


Wave I – Brand and Category Management

• Enabling Technologies

UPC barcode scanning

• Causal Variables

Price, promotions, displays, feature ads,

product assortments, shelf space

• Performance Measures

Sales, market share, gross margin,

sales/square-foot, turn rate, GMROII


Wave II – Customer Relationship Management

• Enabling TechnologiesCustomer loyalty cards, credit/debit cards

• Causal VariablesWave I, plus:

Customer attributes (geodemographics),

purchase history, targeted promotions

• Performance MeasuresCustomer retention, customer loyalty, share-of-customer, lifetime customer value, ROC curves


Wave III – Customer Experience Management

• Enabling TechnologiesReal-time customer tracking

(RFID, GPS, video, clickstream)

• Causal VariablesWave II, plus:

Store layout, displays, navigation aids, music,

product adjacencies, service levels, queues/crowding

• Performance MeasuresStore traffic, shopping path, aisle penetration,

dwell time, product interaction, conversion rate


Key Customer Touchpoints

• Store Entrance and Window Displays

• Lead Fixtures and Merchandising

• End-of-Aisle Displays

• High Volume / Margin Departments

• Customer Service Desk

• Checkout


Observational Measures

Engagement:Examination of signs, displays, circulars

Category dwell time

Salesperson contact

Product/package/display interaction

Conversion:Aisle penetration

Purchase conversion rate

Product price/margin (absence of incentive)

Shopping basket size

Returns


Benefits of Computer Tracking

Breadth of Coverage:Census of customers/items (e.g., for security, inventory)

24/7 tracking (time of day/crowding analysis)

Potential to track entire store (path analysis)

Scalable to multiple stores (benchmarking, experiments)

Speed:Real time data (e.g., for staffing, replenishment)

Data Integration:Link path, penetration, conversion data to consumer

demographics, shopping basket, purchase history


Computer Tracking Solutions:Tracking Carts with Infrared/RFID Sensors

101010A/R/T Forum June 12, 2006

Computer Tracking Solutions:Tracking Carts with Infrared/RFID Sensors

• LimitationsOnly applicable in retail stores using carts and/or baskets (e.g., grocery, mass retail)

Only tracks customers who choose to use carts/baskets, losing “fill-in” shoppers

Unable to track customers who leave carts. May overestimate perimeter traffic, dwell times

No measure of gaze direction or package interaction

No information on group size or behavior

111111A/R/T Forum June 12, 2006

Computer Tracking Solutions:Tracking Shoppers with Video Cameras

121212A/R/T Forum June 12, 2006

Computer Tracking Solutions: Tracking Shoppers with Video Cameras

131313A/R/T Forum June 12, 2006

Automatic Behavior Analysis

141414A/R/T Forum June 12, 2006

Incoming Store TrafficInitial Direction Distribution

Aisle26%

Aisle37%

Aisle113%

Checkout Area30%

MainAisle44%

Average Store Traffic by Hour of Day

0

20

40

60

80

100

120

140

12am

-1am

1am

-2am

2am

-3am

3am

-4am

4am

-5am

5am

-6am

6am

-7am

7am

-8am

8am

-9am

9am

-10a

m

10am

-11a

m

11am

-12p

m

12pm

-1pm

1pm

-2pm

2pm

-3pm

3pm

-4pm

4pm

-5pm

5pm

-6pm

6pm

-7pm

7pm

-8pm

8pm

-9pm

9pm

-10p

m

10pm

-11p

m

Total Store Traffic

500

550

600

650

700

750

800

850

900

950

3/4/

2005

3/6/

2005

3/8/

2005

3/10

/200

5

3/12

/200

5

3/14

/200

5

3/16

/200

5

3/18

/200

5

3/20

/200

5

3/22

/200

5

3/24

/200

5

3/26

/200

5

3/28

/200

5

3/30

/200

5

4/1/

2005

Collection Period 3/3/05 - 4/2/05

Store Entry and Traffic Patterns

151515A/R/T Forum June 12, 2006

Post Period

Pre Period

Aisle Penetration

161616A/R/T Forum June 12, 2006

Category Dwell Time

171717A/R/T Forum June 12, 2006

Computer Tracking Solutions: Tracking Shoppers with Video Cameras

• Limitations

Cameras have a limited field of view and work best in smaller stores (e.g., specialty retail stores, drug stores, convenience stores, banks)

Tracking entire customer path requires multiple cameras with overlapping views

Occlusions (e.g., shelving, signage, other customers) and shadows can interfere with tracking

Difficult to distinguish between employees and customers

181818A/R/T Forum June 12, 2006

Tracking - System Overview

Detection Tracking Activity Recognition

The tracking system works in three steps:

191919A/R/T Forum June 12, 2006

Tracking – Background Subtraction

• Each background pixel is represented as a stack of values

• To decide if a new pixel is a part of the background, a lookup is performed through the full stack and if no matches are found the pixel is considered to be a “foreground pixel”

codebook

codeword

202020A/R/T Forum June 12, 2006

Tracking - Blobs

• The result of background subtraction is a binary bitmap

• Foreground regions corresponding to moving people are represented as blobs (in red)

212121A/R/T Forum June 12, 2006

Tracking – Camera Model

• Parallel lines and the heights of objects in the scene are used to determine the camera’s location and field-of-view

• The camera model permits the translation from world coordinates to image coordinates and back

222222A/R/T Forum June 12, 2006

Tracking – Detecting Heads

The head is usually the least occluded part of the human body. Therefore, to reliably detect multiple people within one blob, we look at their head locations:

1. Estimate the height of each vertical line of the blob

2. Find a number of local maxima in the resulting histogram

232323A/R/T Forum June 12, 2006

Tracking – Detecting Heads (cont.)

242424A/R/T Forum June 12, 2006

Tracking – Probabilistic Modeling

At each instant in time, the tracking system attempts to find the model of the scene which:

Best fits the current observation (what’s in the image)

Is consistent with the model from the last observation

The system estimates the following parameters for each person:

• body width and height (cm)

• current location on the ground (X and Y)

• color histogram

252525A/R/T Forum June 12, 2006

Tracking – Sampling Dynamics

To construct a new model, we randomly apply a number of “jump-diffuse” mutations to the old model

Then the likelihood of the new model is evaluated

• Add body

• Delete body

• Move

• Change height

• Change width

• Change position

• Switch ID

Jump Steps Diffuse Steps

262626A/R/T Forum June 12, 2006

Tracking - Results

272727A/R/T Forum June 12, 2006

Tracking Example: Camera View

282828A/R/T Forum June 12, 2006

Tracking Example: Store View

292929A/R/T Forum June 12, 2006

Insights from Observational Research

• Store Entry

Shoppers take time and space to adjust to the in-store environment

Identify “recognition points” where consumers slow down and start observing

Provide answers and solutions, including signs, circulars, baskets, cash/wrap

303030A/R/T Forum June 12, 2006

Insights (cont.)

• Traffic Flow

Identify dominant pathways through the store

Angle and direction of approach determines best position/orientation for signs and displays.

The greater the speed of approach, the shorterthe message

Facilitate incoming access to destination products, outgoing access to impulse items

313131A/R/T Forum June 12, 2006

Insights (cont.)

• Penetration and Purchase Conversion

Low penetration categories may require additional navigational aids, new product displays, merchandising, and/or changes in store layout to improve traffic flow

Categories with low purchase conversion rates may indicate weaknesses in product assortment, pricing, or presentation

333333A/R/T Forum June 12, 2006

The Original Men’s Section

343434A/R/T Forum June 12, 2006

Men’s Style Center - Outfits

353535A/R/T Forum June 12, 2006

Men’s Style Center – Product Table

363636A/R/T Forum June 12, 2006

Making It Easier for Men to Shop

Enhanced product display drives category traffic and sales:

85% increase in product fixture interaction

44% increase in unit sales

38% increase in dollar sales

373737A/R/T Forum June 12, 2006

Insights (cont.)

• Crowding

Provide sufficient aisle width for displays, carts, strollers, crowds

Reposition fixtures or product displays to eliminate bottlenecks

Avoid crowding in categories requiring extended decision times

383838A/R/T Forum June 12, 2006

Insights (cont.)

• Checkout

Measure queue lengths and waiting time to flag problems with line management, checkout process and customer service

Reduce waiting time by opening more lines, eliminating price checks, speeding up credit authorization, and employing self checkout

"Black Friday" Boosts Store Traffic...

0

200

400

600

800

1000

1200

1400

1600

6-7AM 7-8AM 8-9AM 9-10AM 10-11AM

11-12PM

12-1PM 1-2PM 2-3PM 3-4PM 4-5PM 5-6PM 6-7PM 7-8PM 8-9PM 9-10PM

11/28/2003

11/21/2003

Source: Burke 2005

… But Not Purchase Conversion Rate

0%

10%

20%

30%

40%

50%

60%

6-7AM 7-8AM 8-9AM 9-10AM 10-11AM

11-12PM

12-1PM 1-2PM 2-3PM 3-4PM 4-5PM 5-6PM 6-7PM 7-8PM 8-9PM 9-10PM

11/28/2003

11/21/2003

Source: Burke 2005

“Black Friday”

424242A/R/T Forum June 12, 2006

Challenges

• Creating the Digital Store

• Employee Identification

• Tracking Customer Groups

• Measuring Focus of Attention

434343A/R/T Forum June 12, 2006

Challenges: Creating the Digital Store

• We need an accurate record of:

Store layoutProduct placementVisual environmentStore events

• A digital representation of the store is created from store floor plans and planograms, and extracted from video

444444A/R/T Forum June 12, 2006

Challenges: Employee Identification

• If employees wear uniforms, the system can single them out by using a “color fingerprint”

• In other retail contexts, employees can be identified behaviorally (e.g., tracks that walk behind registers or into stockroom)

454545A/R/T Forum June 12, 2006

Challenges: Who is shopping together?

• After recording each customer’s path on the floor map, we compute time-space cross-correlations between pairs of paths

• This correlation is used as a metric in a clustering process to detect shopper groups

464646A/R/T Forum June 12, 2006

Challenges: Tracking Customer Groups

• Goal: In 1 hour video segment detect shopper groups

• We treat customers as swarming agents, acting according to simple rules (e.g. stay together with swarm members)

5

1 6

10

474747A/R/T Forum June 12, 2006

Challenges: Swarming

• The actors that best fit this model signal a Swarming Event

• Multiple swarming events are further clustered with fuzzy weights to find out shoppers in the same group

11

1213

484848A/R/T Forum June 12, 2006

Challenges: Measuring Focus of Attention

By exploiting a number of visual cues, such as walking speed, shoulder position and facial color, we approximate the angle of the customer’s gaze

494949A/R/T Forum June 12, 2006

Summary of Tracking Insights

1. Track customer path

2. Measure category penetration, dwell time, and conversion

3. Measure line queues and crowding

4. Cluster shoppers based on path similarity

• Evaluate store layout and product adjacencies

• Manage in-store communication, product assortment, and pricing

• Manage service levels, staffing

• Behavioral segmentation

505050A/R/T Forum June 12, 2006

Resources

Questions?

[email protected]

Indiana University’s Center for Retailing:

The Third Wave of Marketing Intelligence

Available online at www.kelley.iu.edu/retail

Documents

1 A/R/T ForumJune 12, 2006 Copyright © 2006 Indiana University Customer Tracking Professor Ray Burke E.W. Kelley Professor of Business Administration Director,