DFW Big Data talk on Mahout Recommenders

1©MapR Technologies 2013- Confidential

Introduction to MahoutAnd How To Build a Recommender

Me, Us

Ted Dunning, Chief Application Architect, MapRCommitter PMC member, Mahout, Zookeeper, DrillBought the beer at the first HUG

MapRDistributes more open source components for HadoopAdds major technology for performance, HA, industry standard API’s

TonightHash tag - #dfwbd #maprSee also - @ApacheMahout @ApacheDrill

@ted_dunning and @mapR

Requested Topic For Tonight

What is Mahout? What makes it different? How can big data technology solve impossible problems? How is big data affecting the world?

What is MapR? What is MapR doing? How does MapR’s technology work? How are customers making use of MapR? How can anyone make use of MapR to solve problems?

Oh … Also This

Detailed break-down of a live machine learning system running with Mahout on MapR

With code examples

I may have to summarize

just a bit

Part 1:5 minutes of math

Part 2:12 minutes: I want a pony

Part 3:A working example

What Does Machine Learning Look Like?

O(κ k d + k3 d) = O(k2 d log n + k3 d) for small k, high qualityO(κ d log k) or O(d log κ log k) for larger k, looser quality

But tonight we’re going to show you how to keep it simple yet powerful…

Comparison of Three Main ML Topics

Recommendation: – Involves observation of interactions between people taking action (users)

and items for input data to the recommender model– Goal is to suggest additional appropriate or desirable interactions– Applications include: movie, music or map-based restaurant choices;

suggesting sale items for e-stores or via cash-register receipts

Part 1:A bit of math

(the math of bits)

Mahout Math

Goals are– basic linear algebra,– and statistical sampling,– and good clustering,– decent speed,– extensibility,– especially for sparse data

But not – totally badass speed– comprehensive set of algorithms– optimization, root finders, quadrature

Matrices and Vectors

At the core:– DenseVector, RandomAccessSparseVector– DenseMatrix, SparseRowMatrix

Highly composable API

Important ideas: – view*, assign and aggregate– iteration

m.viewDiagonal().assign(v)

Assign? View?

Why assign?– Copying is the major cost for naïve matrix packages– In-place operations critical to reasonable performance– Many kinds of updates required, so functional style very helpful

Why view?– In-place operations often required for blocks, rows, columns or diagonals– With views, we need #assign + #views methods– Without views, we need #assign x #views methods

Synergies– With both views and assign, many loops become single line

Examples

double alpha; a.assign(alpha);

a.assign(b, Functions.chain( Functions.plus(beta), Functions.times(alpha));

More Examples

The trace of a matrix

Set diagonal to zero

Set diagonal to negative of row sums

Examples

m.viewDiagonal().zSum()

Examples

m.viewDiagonal().assign(0)

Examples

Set diagonal to negative of row sums excluding the diagonal

m.viewDiagonal().assign(0)

Vector diag = m.viewDiagonal().assign(0);diag.assign(m.rowSums().assign(Functions.MINUS));

Clustering and Such

Streaming k-means and ball k-means– streaming reduces very large data to a cluster sketch– ball k-means is a high quality k-means implementation– the cluster sketch is also usable for other applications– single machine threaded and map-reduce versions available

SVD and friends– stochastic SVD has in-memory, single machine out-of-core and map-reduce

versions– good for reducing very large sparse matrices to tall skinny dense ones

Spectral clustering– based on SVD, allows massive dimensional clustering

Mahout Math Summary

Matrices, Vectors– views– in-place assignment– aggregations– iterations

Functions– lots built-in– cooperate with sparse vector optimizations

Sampling– abstract samplers– samplers as functions

Other stuff … clustering, SVD

Part 2:How recommenders work

(I still want a pony)

Recommendations

Behavior of a crowd helps us understand what individuals will do

Recommendations

Alice got an apple and a puppy

Charles got a bicycle

Charles

Recommendations

Alice got an apple and a puppy

Charles got a bicycle

Bob got an apple

Charles

Recommendations

What else would Bob like??

Charles

Recommendations

What if everybody gets a pony?

Now what does Bob want??

Charles

Log Files

Charles

Log Files

Log Files and Dimensions

Things u1 Alice

BobCharles

History Matrix

Charles

✔ ✔ ✔

✔ ✔

Cooccurrence Matrix

Indicator Matrix

id: t4title: puppydesc: The sweetest little puppy ever.keywords: puppy, dog, pet

indicators: (t1)

Problems with Raw Cooccurrence

Very popular items co-occur with everything– Welcome document– Elevator music

That isn’t interesting– We want anomalous cooccurrence

Recommendation Basics

Coocurrence

t3 not t3

t1 2 1

not t1 1 1

Spot the Anomaly

Root LLR is roughly like standard deviations

A not A

B 13 1000

not B 1000 100,000

A not A

not B 0 2

A not A

not B 0 10,000

A not A

B 10 0

not B 0 100,000

0.44 0.98

2.26 7.15

A Quick Simplification

Users who do h (a vector of things a user has done)

Also do r

User-centric recommendations(transpose translates back to things)

Item-centric recommendations(change the order of operations)

A translates things into users

Symmetry Gives Cross Recommentations

Conventional recommendations with off-line learning

Cross recommendations

For example

Users enter queries (A)– (actor = user, item=query)

Users view videos (B)– (actor = user, item=video)

ATA gives query recommendation– “did you mean to ask for”

BTB gives video recommendation– “you might like these videos”

The punch-line

BTA recommends videos in response to a query– (isn’t that a search engine?)– (not quite, it doesn’t look at content or meta-data)

Real-life example

Query: “Paco de Lucia” Conventional meta-data search results:– “hombres del paco” times 400– not much else

Recommendation based search:– Flamenco guitar and dancers– Spanish and classical guitar– Van Halen doing a classical/flamenco riff

Real-life example

Hypothetical Example

Want a navigational ontology? Just put labels on a web page with traffic– This gives A = users x label clicks

Remember viewing history– This gives B = users x items

Cross recommend– B’A = label to item mapping

After several users click, results are whatever users think they should be

Nice. But we can do better?

things

thingtype 1

thingtype 2

Part 3:What about that worked example?

http://bit.ly/18vbbaT

SolRIndexerSolR

IndexerSolrindexing

Cooccurrence(Mahout)

Item meta-data

Indexshards

Complete history

Analyze with Map-Reduce

SolRIndexerSolR

IndexerSolrsearchWeb tier

Item meta-data

Indexshards

User history

Deploy with Conventional Search System

Objective Results

At a very large credit card company

History is all transactions

Development time to minimal viable product about 4 months

General release 2-3 months later

Search-based recs at or equal in quality to other techniques

Summary

Input: Multiple kinds of behavior on one set of things

Output: Recommendations for one kind of behavior with a different set of things

Cross recommendation is a special case

Objective Results

At a very large credit card company

History is all transactions

Development time to minimal viable product about 4 months

General release 2-3 months later

Search-based recs at or equal in quality to other techniques

Me, Us

Ted Dunning, Chief Application Architect, MapRCommitter PMC member, Mahout, Zookeeper, DrillBought the beer at the first HUGtdunning@{apache.org,maprtech.com} ted.dunning@gmail.com

MapRDistributes more open source components for HadoopAdds major technology for performance, HA, industry standard API’s

TonightHash tag - #dfwbd #maprSee also - @ApacheMahout @ApacheDrill

@ted_dunning and @mapR

DFW Big Data talk on Mahout Recommenders

Technology

Learning Apache Mahout Classification Related/PDFs and Books... · 2015-05-24 · The entropy matrix Summary 2. Apache Mahout Introducing Apache Mahout Algorithms supported in Mahout

Mahout classification presentation

Intelligent IoT Stack An Apache Based, IoT.pdfApache Spark + Apache Mahout Apache Flink CCO Matrices from Mahout. Scenario 2 Edge Device Side. In Core Recommenders Simple app on in-vehicle

mahout introduction

Apache Mahout

MAHOUT classifier tour

mahout feather

News from Mahout

Intro to Mahout

Mahout part2

Leveraging Solr and Mahout

Introducing Apache Mahout

Logistic Regression using Mahout

Introduction to Mahout

Mahout Scala Bindings and Mahout Spark Bindings for Linear ...Mahout Scala Bindings and Mahout Spark Bindings for Linear Algebra Subroutines WorkingNotesandManual Dmitriy Lyubimov∗

Hands on Mahout!

Learning Apache Mahout Classification - DropPDF1.droppdf.com/files/0ZaSP/learning-apache-mahout-classification-by... · Learning Apache Mahout Classification Credits ... , with PDF

New Directions in Mahout's Recommenders

Mahout Interview Questions

Critiquing-based recommenders: survey and emerging