Mandar R. Mutalikdesai Head of Data Semanticsmandar@dataweave.in

Data is the new Fabric

COMPETITIVE INTELLIGENCE IN RETAIL

BIG DATA AND DATA PRODUCTS

"Big data": publicly available as well as within intranetsHuge opportunities, because data = insightNeed of the hour: data democratization---intra-, inter-, and trans-firewallMany challenges: aggregating data, cleaning data, normalizing data, learning patterns and insights, presentationThis is where data products come in!

BUILDING DATA PRODUCTS

• Aggregate large amounts of data publicly available on the web, and serve it in readily usable forms

• Serve actionable data through APIs, Visualizations, and Dashboards• Provide reporting and analytics layer on top of datasets and APIs

Build data products that provide timely actionable insights for businesses and consumers by aggregating and analyzing public data on

the web

SOME CHALLENGES

Data acquisition: crawling and extraction infrastructureData is uncertain and incompleteData is human understandable, not machine understandableData storage and retrievalAnalytics: more data (often) beats better algorithmsProduct UX

Pricing Date Open Government Data

Social Media Data

Attributes

AttributesBig Data Platform

Unstructured, spread across sources and temporally changing

JSON, PDF, XML, DOC, XLS, HTML and much more

Data APIs

Dashboards

Data Services

API Feeds

Visualizations and Widgets

DATAWEAVE PLATFORM

AGGREGATION AND EXTRACTIONExtraction Layer: Offline Extraction of Factual Data

Aggregation Layer: Distributed Crawler Infrastructure

Public Data on the Web

NORMALIZATION

Nomalization Layer

Extraction LayerOffline extraction of factual data

Remove Noise Fill Gaps in Data

Represent Data Clustering

Machine Learning

Techniques

Knowledge Base

DATA STORAGE AND SERVING

Serving Layer

Indexes Views

Filters Pre-Computed Results

Highly Responsive

Serving Layer

Distributed Data Storage Crawl Snapshots Processed Data Clustered Data

DashboardsVisualizationsData APIs Reports

RETAIL ANALYTICS

eCommerce Big Data made easy!

Data science made easy for every retail/brand manager Make better decisions on pricing, competitive strategy, product coverage

with full visibility.

PRICEWEAVE

Real-time Retail Insights Platform across Geographies

Pricing and Assortment Analytics Product and Catalog Intelligence

Price Monitoring, Availability & Historical

Analysis

Gap Analysis & Coverage Along Multiple

Dimensions

Product Similarity Based on Multiple Attributes

Catalog Spread & Depth in Key Product

Categories

PRICING INTELLIGENCE

Compare and monitor product prices, availability, offers across competitors in real-time

• Improve margins by responding to pricing opportunities proactively• Increase sales through competitive offers, promos etc. (e.g. free shipping) • Drive price parity across distribution channels. Detect pricing violations.• Develop long term strategies with pricing trends & insights from the past

ASSORTMENT INTELLIGENCE

Benchmark and analyze your SKU coverage to identify gaps and strengths in your catalog

• Increase coverage by focusing on a wider set of customers• Identify & promote products, brands, categories where one has a unique

advantage• Discover product or category gaps in catalog• Reduce customer acquisition costs

PRODUCT SIMILARITY MAP

Compare products based on features, tech specs, pricing and brand attributes. Identify similar and substitute products

• Fulfil customer demand for a given product with a compelling offer on a substitute to gain share

• Cover out-of-stock products with similar ones• Compare competitor coverage, regardless of SKU similarity

COLOR VARIANT ANALYSIS

Decipher trends in colors, patterns, and variants for apparel, footwear and lifestyle products. Use colors,

fabric, and style for assortment analysis.

• Spot current and predict future trends in colors and patterns• Ensure most popular colors and sizes are stocked in line with demand• Gain insights from social channels about brands, clothing trends

PROMOTION ANALYSIS

Promote the right products. Enhance stock availability of popular products. Understand competitor promotion strategy.

• Gain understanding on products that are being featured by competitors • Discounts, coupons and price points at which the promoted products

are being offered • Get insights into competitors’ product rankings and promotions on a

ongoing basis • Ensure the products customers are interested are always in stock and

attractively priced

ALERTS AND NOTIFICATIONS

Set alerts on any attributes that need tracking, in real time. Get informed on new product introductions, hot deals, and price

slashes, as they happen.

• Get insights in to competitor’s strategy• Discover pricing violations & unauthorized discounts on your brand• Keep tabs on the market trends, in real time

CUSTOM REPORTS

Generate custom reports pivoted around: SKU, sub- category, category, brand, price, geography and

other attributes

• Pick your priorities to pivot the reports & gain actionable insights• Understand key trends relative to products, brands, categories and

geography• Point-in-time snapshots to understand market offers for product/

category• Compare snapshots for trends / gaps

HOW IT WORKS

Data Acquisition Deep Web crawling

• Daily update of 12 million product prices

• 50 million data points monitored

• 3 TB of data processed on a daily basis

Data Extraction and Normalization

• Extract relevant data points

• Clean data stored

• Normalized records based on knowledge bases

• Noise cancellation

Machine Learning and Information Retrieval

• Classify products into product types

• Clustering same products (& similar products)

• Multi parametric clustering

Data Representation

• Dashboards

• Visualizations

• APIs

• Alerts and Notifications

• Reports

• Integration

THANK YOU