Case History di un importante contesto
retail: la new customer experience per
la GDOAnna Monreale – Ricercatrice Università di Pisa
Our digital traces ….
• We produce an unthinkable amount of data while running
our daily activities.
• How can we manage all these data? Can we get an
added value from them?
2
Who’s got the benefits of big data so far?• A few latifundists of data
• GAFA
o Profiling for behavioral advertising and target marketing
• NSA
o Profiling for discovering potential threats to homelandsecurity
o Mass surveillance
A new deal on personal data
DEMOCRATIZING BIG DATA
A user-centric ecosystem for Big Data
• Engage people in the creation and use of big data and
knowledge, by empowering individuals with self-knowledge
• Incentivize individual to participate, to align own self-interest
with broader societal goals
• Based on transparency, trust & privacy, peer-to-peer
networks
Personal Data Store (PDS)
LivLab Livorno Case Study
Un Laboratorio di innovazione
• Goal: improve customers self-
awareness with respect to shopping
and mobility dimensions involving
actively customers.
A change of perspective
Organization-centric visiono Companies use customer data for profiling
and promoting products and services
LivLab Vision: User-Centric modelWe analyze user data and provide to customers:
o A dashboard to visualize and monitor the data
o Indicators on the shopping and mobility habits
o The possibility to have a comparison with the collectivehabits
o A service to increase the self-awareness in terms of mobility and shopping
What is LivLab
WHO
o More than 100 Coop customers
o In the Livorno area
o Having a Smartphone Android
WHAT
o Apps for mobility and shopping data visualization and
monitoring
o Apps for gamification
o Web platform based on Persona Data Store schema
Why Shopping and Mobility Data
• Analyze purchases
• Analyze bought products
• Analyze shopping frequency
Shopping BehaviorTypical Shopping
Basket
Shopping
Systematicity
Indicator
• Analyze shops visited
• Analyze typical shopping days
• Analyze shopping time
Mobility Behavior Typical visit behavior
Mobility
Systematicity
Indicator
Why Shopping and Mobility Data
• Understanding individual and collective
phenomena
• Indentifying factors having an impact on the life
styles and wellbeing
• Increasing the self-awareness in the daily choises
• Design and provide services helping in improving
the daily life quality
We need Data Mining for PDS
• Individual data mining for extracting added value from
individual personal data describing human activities
• Collective vs Individual with knowledge sharing: can
be achieved comparable levels of self-awareness
through individual users' models sharing?
• Collective - Individual interaction: it is possible to find
a completely new type of knowledge by exploiting the
mutual interaction between individual and collective
models?
Personal Perspective on Shopping
19.30 Sun
17.50 Wed
11.50 Tue
17.30 Mon
08.40 Mon
Profitability & PredictabilityIndicators
Profitability & Predictability Indicators
• Goal: What is the relationship between the
regularity of a customer’s behavior and her
profitability for a shop?
• Solution: individually measure customers retails to
evaluate how much is systematic their shopping
behavior.
SystematicCustomer
CasualCustomer
BRE & STRE Calculus
• Step 1: identify representative
baskets patterns w.r.t. basket
composition - shop and time.
• Step 2: classify each basket with
a representative basket patter.
• Step 3: calculate BRE and STRE
by using the frequencies and the
entropy formula
17.50 Wed
WeekDays, Late Afternoon
BRE STRE
BRE: Basket Revealed Entropy
STRE: Spatio-Temporal Revealed Entropy
These measures tell us respectively how unpredictable is the
basket composition and the visiting pattern of a given customer.
Customers Classification• 71,172,672 readings • 56,448 customers• 84,362 distinct products
• Year 2012• Livorno province (23 shops)• At least a shop per month
Avg Expenditure
Tot Expenditure
BRE
STRE
Products of Systematic Customers
Vegetables and fruit are the items most shopped by the
customers which are in the intersection of the systematic
sets both for BRE and STRE.
Towards a Personal Cart Assistant
Towards a Personal Cart Assistant
• Goal: Which are the products the custoemr is
going to add to her current shopping list or cart?
• Solution: individually understand which are the
typical shopping patterns of each customer and
exploit them to predict/suggests how the
shopping list could be completed.
Typical Shopping Patterns
Shopping History
Shopping Patterns
Autofocus Clustering Algorithm
Extract Personal Representative Baskets
Personal Cart Assistant
Current Basket
RecommendedProducts
Most Similar Basket
LivLab Web Application
Personal Mobility Data
Personal Mobility Data
Users’ IMN pointCoop shop
Personal Mobility Data
Users’ IMN pointCoop shop
Personal Mobility Data
Users’ IMN pointCoop shop
Personal Mobility Data
Users’ IMN pointCoop shop
Personal Shopping Data
Where I Am?
Your typical trip takes 30 minutes,
like 5,71% of the other customers
Your basket compoition is quite predictible,
like 10% of the other customers
The shop of and the time of your
purchases are not easily predictible,
like 15,88% of the other customers
Mobile Applications
LivLab Spese Application• Details on the last
shopping session• History of customer
purchaises
• Which kind of
customer are you?
(bio/eco/etc..)
LivLab Gamification• Maintaining active customer’s interest
• Game: Find the Jolly Product seguendo dei suggerimenti!
• Award: fidelity points!
LivLab Gamification: InGreen
• Crowdsourcing for collecting data about product
packaging
• Game: For each product
1. Write a review
2. Provide the type of packaging (glass, plastic,
paper, etc.)
3. Provide the weight of packaging
• Award: fidelity points!
• Utility:
• Estimation of the garbage in the customer
shopping
• Definition of indicators about the impact of the
customer shopping on the environment
LivLab Smart Shopping List• A service providing a pre-compiled shopping list based on the
customer history• Pre-compiled shopping based on the season
• Pre-compiled shopping based on customer systematic purchases
• Pre-compiled shopping based on the last purchases
• Suggestions based on you typical purchaising and current
discounts
Collective Perspective
Nowcasting GDP & Well-Being
Nowcasting GDP & Well-Being
• Goal: Estimating well-being observing customers retails
through a measure called sophistication.
• GDP (Gross Domestic Product): is the market value of all
goods and services produced within a country in a given
period of time.
• GDP is thought to capture average prosperity
• Can we estimate GDP? Can we nowcast it?
Customers–Products & Sophistication• Customers are sophisticated if they purchase
sophisticated products.
• Products are sophisticated if they are bought
by sophisticated customers.
Relation between GDP and customers sophistication (left) and product purchased (right)
Coop Flu Trend
Coop Flu Trend• Goal: Can we predict Flu by analyzing changes in
purchases behaviors?
• Do different behaviors exist based on flu peaks? Do we
observe changes in purchases in these periods?
• Product segments can work as proxy for prediction
Flu Trend• Epidemiological data
from 2004/05 to 2014/15
• ~900 physicians and
pediatricians
• Weekly reporting of
cases of flu syndrome
• Cases divided by age
group and by type of
risk
• Reports from the 42nd
week of the year until
the last week of April of
the following year
Flu incidence in Italy2004/05 – 2014/15 seasons
Weeks
Cas
es x
100
0 as
sist
ed
Flu Detection Workflow
• Identify the products having
adoption trend similar to the flu
trend
• Identify the customers that buy
them during the flu-peak
• Identify the sentinels: frequent
baskets of such customers during
the peak
• Use the sentinels as control set
for the following year flu peak
46
Knowledge Discovery
& Data Mining Lab
http://kdd.isti.cnr.it
47
Knowledge Discovery
& Data Mining Lab
http://kdd.isti.cnr.it
Master Universitario Di II Livello
Big Data Technology
Big Data Sensing & Procurement
Big Data Mining
Big Data Story Telling
Big Data Ethics
Il Master Big Data ha l’obiettivo di formare “data scientists”, dei
professionisti dotati di un mix di competenze multidisciplinari
che permettono non solo di acquisire dati ed estrarne conos-
cenza, ma anche di raccontare “storie” attraverso questi dati, a
supporto delle decisioni, della creatività e dello sviluppo di
servizi innovativi, e di saper gestire le ripercussioni etiche e
legali dei Big Data, che spesso contengono informazioni
personali e suscitano problematiche relative alla privacy, alla
trasparenza, alla consapevolezza.
Aree di innovazione socio-economica:
Big Data for Social Good
Big Data for Business
Big Data Analytics E Social Mining
SoBigData
it
Master Big Data and Social Mining 2016
Mid-Term Workshop Aula Gerace, Dipartimento di Informatica, Università di Pisa
Edificio C, 2 Piano, Largo B. Pontecorvo 3, Pisa
Agenda
10:00 – 10:30 Accoglienza, stato di avanzamento del master, presentazione nuova Edizione
10:30 – 13:00 Presentazioni dei progetti degli allievi
13:00 – 14:00 Pranzo & networking
14:00 – 15:00 Presentazioni dei progetti degli allievi
15:00 – 17:00 Discussione e networking fra allievi e tutor aziendali ed accademici
11Novembre
2016
Key publications• F Giannotti, M Nanni, F Pinelli, D Pedreschi. Trajectory pattern mining. ACM SIGKDD 2007
• F Giannotti, D Pedreschi. Mobility, data mining and privacy: Geographic knowledge discovery. Springer,
2008
• A Monreale, F Pinelli, R Trasarti, F Giannotti. WhereNext: a location predictor on trajectory pattern
mining. ACM SIGKDD 2009
• S Rinzivillo, D Pedreschi, M Nanni, F Giannotti, N Andrienko, G Andrienko. Visually driven analysis of
movement data by progressive clustering. Information Visualization 7 (3-4), 225-239. 2008
• D Wang, D Pedreschi, C Song, F Giannotti, AL Barabasi. Human mobility, social ties, and link prediction.
ACM SIGKDD 2011
• F Giannotti, M Nanni, D Pedreschi, F Pinelli, C Renso, S Rinzivillo, R Trasarti. Unveiling the complexity
of human mobility by querying and mining massive trajectory data. The VLDB Journal 20(5) 2011
• R Trasarti, F Pinelli, M Nanni, F Giannotti. Mining mobility user profiles for car pooling. ACM SIGKDD
2011
• M Coscia, G Rossetti, F Giannotti, D Pedreschi. Demon: a local-first discovery method for overlapping
communities. ACM SIGKDD 2012
• D Pennacchioli, M Coscia, S Rinzivillo, F Giannotti, D Pedreschi. The retail market as a complex
system. EPJ Data Science 3 (1), 1-27 (2014)
• A Monreale, S Rinzivillo, F Pratesi, F Giannotti, D Pedreschi. Privacy-by-design in big data analytics and
social mining. EPJ Data Science 3 (1), 1-26 (2014)
• Luca Pappalardo, Filippo Simini, Salvatore Rinzivillo, Dino Pedreschi, Fosca Giannotti & Albert-László
Barabási. Returners and explorers dichotomy in human mobility. Nature Communications 6, Article number:
8166 (2015) doi:10.1038/ncomms9166 (2015)
Key publications• M Coscia, G Rossetti, F Giannotti, D Pedreschi. Demon: a local-first discovery method for
overlapping communities. ACM SIGKDD 2012
• S Rinzivillo, S Mainardi, F Pezzoni, M Coscia, D Pedreschi, F Giannotti. Discovering the
geographical borders of human mobility. KI-Künstliche Intelligenz 26 (3) 2012
• D Pennacchioli, M Coscia, S Rinzivillo, D Pedreschi, F Giannotti. Explaining the Product Range
Effect in Purchase Data. IEEE BIGDATA 2013
• B Furletti, L Gabrielli, C Renso, S Rinzivillo. Analysis of GSM Calls Data for Understanding User
Mobility Behavior. IEEE BIGDATA 2013
• L Milli, A Monreale, G Rossetti, D Pedreschi, F Giannotti, F Sebastiani. Quantification trees.
IEEE ICDM 2013
• Giusti, Marchetti, Pratesi, Salvati, Pedreschi, Giannotti, Rinzivillo, Pappalardo, Gabrielli. Small
area model based estimators using Big Data Sources. Journal of Official Statistics, 31(2) 2015.
• Furletti, Gabrielli, Garofalo, Giannotti, Milli, Nanni, Pedreschi, Vivio. Use of mobile phone data to
estimate mobility flows. Measuring urban population and intercity mobility using big data in an
integrated approach. Italian Symposium on Statistics, 2014.
• Luca Pappalardo, Maarten Vanhoof, Zbigniew Smoreda, Dino Pedreschi,Fosca Giannotti.
Human Mobility and Economic Development. IEEE BIG DATA (2015).
Vision papers• F Giannotti, D Pedreschi, A Pentland, P Lukowicz, D Kossmann, J
Crowley, D Helbing. A planetary nervous system for social mining and collective awareness. The European Physical Journal Special Topics 214 (1), 49-75, 2012
• J van den Hoven, D Helbing, D Pedreschi, J Domingo-Ferrer, FGiannotti . FuturICT—The road towards ethical ICT. The European Physical Journal Special Topics 214 (1), 153-181, 2012
• M Batty, KW Axhausen, F Giannotti, A Pozdnoukhov, A Bazzani, M Wachowicz. Smart cities of the future. The European PhysicalJournal Special Topics 214 (1), 481-518, 2012
Thank you