Algorithms beyond the limits of Machine Learning:Combining Machine-learning with Expert-Human Judgment for Better PersonalizationEric Colson @ericcolson | Data Driven NYC | March 2016

Recommender Systems

Different Capabilities

Find the EigenvectorsTask 1: Task 2:

Find the leopard print dress

Image example inspired from this blog post by Artem Khurshudov: http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.html

5

Different Capabilities

x <- eigen()

z <- find_dress()

Human Computation

Photo credit: http://gridtalk-project.blogspot.com/2010/09/when-computers-were-human.html

9

1 2 3

Recommender Systems

75% of videos watched from recommendations

50% of connections driven by recommendations

100% of merchandise sold via recommendations

35% of sales driven from recommendations

g ()

Machines• Rote calculations• Qualify• Rank

Expert-Humans• Cognition, improvise• Curation, social norms • Relationships

m()

Coordinating Machines & Humans

Logistics• Physical Delivery• Convenience• Feedback

Queues

The Algorithmh()

z()

12345678913216567891321658523612345321656789132165

5234532165428816789173333

Additive Contributions

s(h, m)

hm

Assumes: s(h), s(m) > 0s(h) ≠ s(m)

= s(h) + s(m) + s(hm)> [ s(h) OR s(m) ]

Training Machines

m(.)

Training humans

h(.)

Other benefits of combining• Scale• Feedback• Controlled Variation• Specialization

“My husband is returning home from a tour in Iraq – he is disabled (PTSD). I would love something to wear for a very special date night.”

Specialization

More human humans

Thanks!