DFW Big Data talk on Mahout Recommenders

1©MapR Technologies 2013- Confidential

Introduction to MahoutAnd How To Build a Recommender


Me, Us

Ted Dunning, Chief Application Architect, MapRCommitter PMC member, Mahout, Zookeeper, DrillBought the beer at the first HUG

MapRDistributes more open source components for HadoopAdds major technology for performance, HA, industry standard API’s

TonightHash tag - #dfwbd #maprSee also - @ApacheMahout @ApacheDrill

@ted_dunning and @mapR


Requested Topic For Tonight

What is Mahout? What makes it different? How can big data technology solve impossible problems? How is big data affecting the world?


Also

What is MapR? What is MapR doing? How does MapR’s technology work? How are customers making use of MapR? How can anyone make use of MapR to solve problems?


Oh … Also This

Detailed break-down of a live machine learning system running with Mahout on MapR

With code examples


I may have to summarize


I may have to summarize

just a bit


Part 1:5 minutes of math


Part 2:12 minutes: I want a pony


Part 3:A working example


What Does Machine Learning Look Like?


What Does Machine Learning Look Like?

O(κ k d + k3 d) = O(k2 d log n + k3 d) for small k, high qualityO(κ d log k) or O(d log κ log k) for larger k, looser quality

But tonight we’re going to show you how to keep it simple yet powerful…


Comparison of Three Main ML Topics

Recommendation: – Involves observation of interactions between people taking action (users)

and items for input data to the recommender model– Goal is to suggest additional appropriate or desirable interactions– Applications include: movie, music or map-based restaurant choices;

suggesting sale items for e-stores or via cash-register receipts




Part 1:A bit of math

(the math of bits)


Mahout Math

Goals are– basic linear algebra,– and statistical sampling,– and good clustering,– decent speed,– extensibility,– especially for sparse data

But not – totally badass speed– comprehensive set of algorithms– optimization, root finders, quadrature


Matrices and Vectors

At the core:– DenseVector, RandomAccessSparseVector– DenseMatrix, SparseRowMatrix

Highly composable API

Important ideas: – view*, assign and aggregate– iteration

m.viewDiagonal().assign(v)


Assign? View?

Why assign?– Copying is the major cost for naïve matrix packages– In-place operations critical to reasonable performance– Many kinds of updates required, so functional style very helpful

Why view?– In-place operations often required for blocks, rows, columns or diagonals– With views, we need #assign + #views methods– Without views, we need #assign x #views methods

Synergies– With both views and assign, many loops become single line


Examples

double alpha; a.assign(alpha);

a.assign(b, Functions.chain( Functions.plus(beta), Functions.times(alpha));


More Examples

The trace of a matrix

Set diagonal to zero

Set diagonal to negative of row sums


Examples




m.viewDiagonal().zSum()


Examples





m.viewDiagonal().assign(0)


Examples



Set diagonal to negative of row sums excluding the diagonal


m.viewDiagonal().assign(0)

Vector diag = m.viewDiagonal().assign(0);diag.assign(m.rowSums().assign(Functions.MINUS));


Clustering and Such

Streaming k-means and ball k-means– streaming reduces very large data to a cluster sketch– ball k-means is a high quality k-means implementation– the cluster sketch is also usable for other applications– single machine threaded and map-reduce versions available

SVD and friends– stochastic SVD has in-memory, single machine out-of-core and map-reduce

versions– good for reducing very large sparse matrices to tall skinny dense ones

Spectral clustering– based on SVD, allows massive dimensional clustering


Mahout Math Summary

Matrices, Vectors– views– in-place assignment– aggregations– iterations

Functions– lots built-in– cooperate with sparse vector optimizations

Sampling– abstract samplers– samplers as functions

Other stuff … clustering, SVD


Part 2:How recommenders work

(I still want a pony)


Recommendations

Behavior of a crowd helps us understand what individuals will do


Recommendations

Alice got an apple and a puppy

Charles got a bicycle

Alice

Charles


Recommendations

Alice got an apple and a puppy

Charles got a bicycle

Bob got an apple

Alice

Bob

Charles


Recommendations

What else would Bob like??

Alice

Bob

Charles


Recommendations

What if everybody gets a pony?

Now what does Bob want??

Alice

Bob

Charles


Log Files

Alice

Bob

Charles

Alice

Bob

Charles

Alice


Log Files

u1

u3

u2

u1

u3

u2

u1

t1

t2

t3

t4

t3

t3

t1


Log Files and Dimensions

u1

u3

u2

u1

u3

u2

u1

t1

t2

t3

t4

t3

t3

t1

t1

t2

t3

t4

Things u1 Alice

BobCharles

u3u2

Users


History Matrix

Alice

Bob

Charles

✔ ✔ ✔

✔ ✔

✔ ✔


Cooccurrence Matrix

1 2

1 1

1

1

2 1


Indicator Matrix

✔


Indicator Matrix

✔

id: t4title: puppydesc: The sweetest little puppy ever.keywords: puppy, dog, pet

indicators: (t1)


Problems with Raw Cooccurrence

Very popular items co-occur with everything– Welcome document– Elevator music

That isn’t interesting– We want anomalous cooccurrence


Recommendation Basics

Coocurrence

t3 not t3

t1 2 1

not t1 1 1


Spot the Anomaly

Root LLR is roughly like standard deviations

A not A

B 13 1000

not B 1000 100,000

A not A

B 1 0

not B 0 2

A not A

B 1 0

not B 0 10,000

A not A

B 10 0

not B 0 100,000

0.44 0.98

2.26 7.15


A Quick Simplification

Users who do h (a vector of things a user has done)

Also do r

User-centric recommendations(transpose translates back to things)

Item-centric recommendations(change the order of operations)

A translates things into users


Symmetry Gives Cross Recommentations

Conventional recommendations with off-line learning

Cross recommendations


For example

Users enter queries (A)– (actor = user, item=query)

Users view videos (B)– (actor = user, item=video)

ATA gives query recommendation– “did you mean to ask for”

BTB gives video recommendation– “you might like these videos”


The punch-line

BTA recommends videos in response to a query– (isn’t that a search engine?)– (not quite, it doesn’t look at content or meta-data)


Real-life example

Query: “Paco de Lucia” Conventional meta-data search results:– “hombres del paco” times 400– not much else

Recommendation based search:– Flamenco guitar and dancers– Spanish and classical guitar– Van Halen doing a classical/flamenco riff


Real-life example


Hypothetical Example

Want a navigational ontology? Just put labels on a web page with traffic– This gives A = users x label clicks

Remember viewing history– This gives B = users x items

Cross recommend– B’A = label to item mapping

After several users click, results are whatever users think they should be


Nice. But we can do better?


users

things


users

thingtype 1

thingtype 2



Part 3:What about that worked example?


http://bit.ly/18vbbaT


SolRIndexerSolR

IndexerSolrindexing

Cooccurrence(Mahout)

Item meta-data

Indexshards

Complete history

Analyze with Map-Reduce


SolRIndexerSolR

IndexerSolrsearchWeb tier

Item meta-data

Indexshards

User history

Deploy with Conventional Search System


Objective Results

At a very large credit card company

History is all transactions

Development time to minimal viable product about 4 months

General release 2-3 months later

Search-based recs at or equal in quality to other techniques


Summary

Input: Multiple kinds of behavior on one set of things

Output: Recommendations for one kind of behavior with a different set of things

Cross recommendation is a special case


Objective Results

At a very large credit card company

History is all transactions

Development time to minimal viable product about 4 months

General release 2-3 months later

Search-based recs at or equal in quality to other techniques


Me, Us

Ted Dunning, Chief Application Architect, MapRCommitter PMC member, Mahout, Zookeeper, DrillBought the beer at the first HUGtdunning@{apache.org,maprtech.com} [email protected]

MapRDistributes more open source components for HadoopAdds major technology for performance, HA, industry standard API’s

TonightHash tag - #dfwbd #maprSee also - @ApacheMahout @ApacheDrill

@ted_dunning and @mapR

mailto:tdunning@%7Bapache.org,maprtech.com

mailto:[email protected]

Technology

DFW Big Data talk on Mahout Recommenders