Case History di un importante contesto

retail: la new customer experience per

la GDOAnna Monreale – Ricercatrice Università di Pisa

Our digital traces ….

• We produce an unthinkable amount of data while running

our daily activities.

• How can we manage all these data? Can we get an

added value from them?

2

Who’s got the benefits of big data so far?• A few latifundists of data

• GAFA

o Profiling for behavioral advertising and target marketing

• NSA

o Profiling for discovering potential threats to homelandsecurity

o Mass surveillance

A new deal on personal data

DEMOCRATIZING BIG DATA

A user-centric ecosystem for Big Data

• Engage people in the creation and use of big data and

knowledge, by empowering individuals with self-knowledge

• Incentivize individual to participate, to align own self-interest

with broader societal goals

• Based on transparency, trust & privacy, peer-to-peer

networks

Personal Data Store (PDS)

LivLab Livorno Case Study

Un Laboratorio di innovazione

• Goal: improve customers self-

awareness with respect to shopping

and mobility dimensions involving

actively customers.

A change of perspective

Organization-centric visiono Companies use customer data for profiling

and promoting products and services

LivLab Vision: User-Centric modelWe analyze user data and provide to customers:

o A dashboard to visualize and monitor the data

o Indicators on the shopping and mobility habits

o The possibility to have a comparison with the collectivehabits

o A service to increase the self-awareness in terms of mobility and shopping

What is LivLab

WHO

o More than 100 Coop customers

o In the Livorno area

o Having a Smartphone Android

WHAT

o Apps for mobility and shopping data visualization and

monitoring

o Apps for gamification

o Web platform based on Persona Data Store schema

Why Shopping and Mobility Data

• Analyze purchases

• Analyze bought products

• Analyze shopping frequency

Shopping BehaviorTypical Shopping

Basket

Shopping

Systematicity

Indicator

• Analyze shops visited

• Analyze typical shopping days

• Analyze shopping time

Mobility Behavior Typical visit behavior

Mobility

Systematicity

Indicator

Why Shopping and Mobility Data

• Understanding individual and collective

phenomena

• Indentifying factors having an impact on the life

styles and wellbeing

• Increasing the self-awareness in the daily choises

• Design and provide services helping in improving

the daily life quality

We need Data Mining for PDS

• Individual data mining for extracting added value from

individual personal data describing human activities

• Collective vs Individual with knowledge sharing: can

be achieved comparable levels of self-awareness

through individual users' models sharing?

• Collective - Individual interaction: it is possible to find

a completely new type of knowledge by exploiting the

mutual interaction between individual and collective

models?

Personal Perspective on Shopping

19.30 Sun

17.50 Wed

11.50 Tue

17.30 Mon

08.40 Mon

Profitability & PredictabilityIndicators

Profitability & Predictability Indicators

• Goal: What is the relationship between the

regularity of a customer’s behavior and her

profitability for a shop?

• Solution: individually measure customers retails to

evaluate how much is systematic their shopping

behavior.

SystematicCustomer

CasualCustomer

BRE & STRE Calculus

• Step 1: identify representative

baskets patterns w.r.t. basket

composition - shop and time.

• Step 2: classify each basket with

a representative basket patter.

• Step 3: calculate BRE and STRE

by using the frequencies and the

entropy formula

17.50 Wed

WeekDays, Late Afternoon

BRE STRE

BRE: Basket Revealed Entropy

STRE: Spatio-Temporal Revealed Entropy

These measures tell us respectively how unpredictable is the

basket composition and the visiting pattern of a given customer.

Customers Classification• 71,172,672 readings • 56,448 customers• 84,362 distinct products

• Year 2012• Livorno province (23 shops)• At least a shop per month

Avg Expenditure

Tot Expenditure

BRE

STRE

Products of Systematic Customers

Vegetables and fruit are the items most shopped by the

customers which are in the intersection of the systematic

sets both for BRE and STRE.

Towards a Personal Cart Assistant

Towards a Personal Cart Assistant

• Goal: Which are the products the custoemr is

going to add to her current shopping list or cart?

• Solution: individually understand which are the

typical shopping patterns of each customer and

exploit them to predict/suggests how the

shopping list could be completed.

Typical Shopping Patterns

Shopping History

Shopping Patterns

Autofocus Clustering Algorithm

Extract Personal Representative Baskets

Personal Cart Assistant

Current Basket

RecommendedProducts

Most Similar Basket

LivLab Web Application

Personal Mobility Data

Personal Mobility Data

Users’ IMN pointCoop shop

Personal Mobility Data

Users’ IMN pointCoop shop

Personal Mobility Data

Users’ IMN pointCoop shop

Personal Mobility Data

Users’ IMN pointCoop shop

Personal Shopping Data

Where I Am?

Your typical trip takes 30 minutes,

like 5,71% of the other customers

Your basket compoition is quite predictible,

like 10% of the other customers

The shop of and the time of your

purchases are not easily predictible,

like 15,88% of the other customers

Mobile Applications

LivLab Spese Application• Details on the last

shopping session• History of customer

purchaises

• Which kind of

customer are you?

(bio/eco/etc..)

LivLab Gamification• Maintaining active customer’s interest

• Game: Find the Jolly Product seguendo dei suggerimenti!

• Award: fidelity points!

LivLab Gamification: InGreen

• Crowdsourcing for collecting data about product

packaging

• Game: For each product

1. Write a review

2. Provide the type of packaging (glass, plastic,

paper, etc.)

3. Provide the weight of packaging

• Award: fidelity points!

• Utility:

• Estimation of the garbage in the customer

shopping

• Definition of indicators about the impact of the

customer shopping on the environment

LivLab Smart Shopping List• A service providing a pre-compiled shopping list based on the

customer history• Pre-compiled shopping based on the season

• Pre-compiled shopping based on customer systematic purchases

• Pre-compiled shopping based on the last purchases

• Suggestions based on you typical purchaising and current

discounts

Collective Perspective

Nowcasting GDP & Well-Being

Nowcasting GDP & Well-Being

• Goal: Estimating well-being observing customers retails

through a measure called sophistication.

• GDP (Gross Domestic Product): is the market value of all

goods and services produced within a country in a given

period of time.

• GDP is thought to capture average prosperity

• Can we estimate GDP? Can we nowcast it?

Customers–Products & Sophistication• Customers are sophisticated if they purchase

sophisticated products.

• Products are sophisticated if they are bought

by sophisticated customers.

Relation between GDP and customers sophistication (left) and product purchased (right)

Coop Flu Trend

Coop Flu Trend• Goal: Can we predict Flu by analyzing changes in

purchases behaviors?

• Do different behaviors exist based on flu peaks? Do we

observe changes in purchases in these periods?

• Product segments can work as proxy for prediction

Flu Trend• Epidemiological data

from 2004/05 to 2014/15

• ~900 physicians and

pediatricians

• Weekly reporting of

cases of flu syndrome

• Cases divided by age

group and by type of

risk

• Reports from the 42nd

week of the year until

the last week of April of

the following year

Flu incidence in Italy2004/05 – 2014/15 seasons

Weeks

Cas

es x

100

0 as

sist

ed

Flu Detection Workflow

• Identify the products having

adoption trend similar to the flu

trend

• Identify the customers that buy

them during the flu-peak

• Identify the sentinels: frequent

baskets of such customers during

the peak

• Use the sentinels as control set

for the following year flu peak

46

Knowledge Discovery

& Data Mining Lab

http://kdd.isti.cnr.it

47

Knowledge Discovery

& Data Mining Lab

http://kdd.isti.cnr.it

Master Universitario Di II Livello

Big Data Technology

Big Data Sensing & Procurement

Big Data Mining

Big Data Story Telling

Big Data Ethics

Il Master Big Data ha l’obiettivo di formare “data scientists”, dei

professionisti dotati di un mix di competenze multidisciplinari

che permettono non solo di acquisire dati ed estrarne conos-

cenza, ma anche di raccontare “storie” attraverso questi dati, a

supporto delle decisioni, della creatività e dello sviluppo di

servizi innovativi, e di saper gestire le ripercussioni etiche e

legali dei Big Data, che spesso contengono informazioni

personali e suscitano problematiche relative alla privacy, alla

trasparenza, alla consapevolezza.

Aree di innovazione socio-economica:

Big Data for Social Good

Big Data for Business

Big Data Analytics E Social Mining

SoBigData

it

Master Big Data and Social Mining 2016

Mid-Term Workshop Aula Gerace, Dipartimento di Informatica, Università di Pisa

Edificio C, 2 Piano, Largo B. Pontecorvo 3, Pisa

Agenda

10:00 – 10:30   Accoglienza, stato di avanzamento del master, presentazione nuova Edizione

10:30 – 13:00   Presentazioni dei progetti degli allievi

13:00 – 14:00   Pranzo & networking

14:00 – 15:00   Presentazioni dei progetti degli allievi

15:00 – 17:00   Discussione e networking fra allievi e tutor aziendali ed accademici

11Novembre

2016

Key publications• F Giannotti, M Nanni, F Pinelli, D Pedreschi. Trajectory pattern mining. ACM SIGKDD 2007

• F Giannotti, D Pedreschi. Mobility, data mining and privacy: Geographic knowledge discovery. Springer,

2008

• A Monreale, F Pinelli, R Trasarti, F Giannotti. WhereNext: a location predictor on trajectory pattern

mining. ACM SIGKDD 2009

• S Rinzivillo, D Pedreschi, M Nanni, F Giannotti, N Andrienko, G Andrienko. Visually driven analysis of

movement data by progressive clustering. Information Visualization 7 (3-4), 225-239. 2008

• D Wang, D Pedreschi, C Song, F Giannotti, AL Barabasi. Human mobility, social ties, and link prediction.

ACM SIGKDD 2011

• F Giannotti, M Nanni, D Pedreschi, F Pinelli, C Renso, S Rinzivillo, R Trasarti. Unveiling the complexity

of human mobility by querying and mining massive trajectory data. The VLDB Journal 20(5) 2011

• R Trasarti, F Pinelli, M Nanni, F Giannotti. Mining mobility user profiles for car pooling. ACM SIGKDD

2011

• M Coscia, G Rossetti, F Giannotti, D Pedreschi. Demon: a local-first discovery method for overlapping

communities. ACM SIGKDD 2012

• D Pennacchioli, M Coscia, S Rinzivillo, F Giannotti, D Pedreschi. The retail market as a complex

system. EPJ Data Science 3 (1), 1-27 (2014)

• A Monreale, S Rinzivillo, F Pratesi, F Giannotti, D Pedreschi. Privacy-by-design in big data analytics and

social mining. EPJ Data Science 3 (1), 1-26 (2014)

• Luca Pappalardo, Filippo Simini, Salvatore Rinzivillo, Dino Pedreschi, Fosca Giannotti & Albert-László

Barabási. Returners and explorers dichotomy in human mobility. Nature Communications 6, Article number:

8166 (2015) doi:10.1038/ncomms9166 (2015)

Key publications• M Coscia, G Rossetti, F Giannotti, D Pedreschi. Demon: a local-first discovery method for

overlapping communities. ACM SIGKDD 2012

• S Rinzivillo, S Mainardi, F Pezzoni, M Coscia, D Pedreschi, F Giannotti. Discovering the

geographical borders of human mobility. KI-Künstliche Intelligenz 26 (3) 2012

• D Pennacchioli, M Coscia, S Rinzivillo, D Pedreschi, F Giannotti. Explaining the Product Range

Effect in Purchase Data. IEEE BIGDATA 2013

• B Furletti, L Gabrielli, C Renso, S Rinzivillo. Analysis of GSM Calls Data for Understanding User

Mobility Behavior. IEEE BIGDATA 2013

• L Milli, A Monreale, G Rossetti, D Pedreschi, F Giannotti, F Sebastiani. Quantification trees.

IEEE ICDM 2013

• Giusti, Marchetti, Pratesi, Salvati, Pedreschi, Giannotti, Rinzivillo, Pappalardo, Gabrielli. Small

area model based estimators using Big Data Sources. Journal of Official Statistics, 31(2) 2015.

• Furletti, Gabrielli, Garofalo, Giannotti, Milli, Nanni, Pedreschi, Vivio. Use of mobile phone data to

estimate mobility flows. Measuring urban population and intercity mobility using big data in an

integrated approach. Italian Symposium on Statistics, 2014.

• Luca Pappalardo, Maarten Vanhoof, Zbigniew Smoreda, Dino Pedreschi,Fosca Giannotti.

Human Mobility and Economic Development. IEEE BIG DATA (2015).

Vision papers• F Giannotti, D Pedreschi, A Pentland, P Lukowicz, D Kossmann, J

Crowley, D Helbing. A planetary nervous system for social mining and collective awareness. The European Physical Journal Special Topics 214 (1), 49-75, 2012

• J van den Hoven, D Helbing, D Pedreschi, J Domingo-Ferrer, FGiannotti . FuturICT—The road towards ethical ICT. The European Physical Journal Special Topics 214 (1), 153-181, 2012

• M Batty, KW Axhausen, F Giannotti, A Pozdnoukhov, A Bazzani, M Wachowicz. Smart cities of the future. The European PhysicalJournal Special Topics 214 (1), 481-518, 2012

Thank you