Modern Machine Learning: New Challenges and

Connections

Maria-Florina Balcan

Machine Learning is Changing the World

“A breakthrough in machine learning would be worthten Microsofts” (Bill Gates, Microsoft)

“Machine learning is the hot new thing”(John Hennessy, President, Stanford)

“Web rankings today are mostly a matter of machine learning” (Prabhakar Raghavan, VP Engineering at Google)

The World is Changing Machine Learning

New applications Explosion of data

Modern applications: massive amounts of raw

data.Only a tiny fraction can be annotated by human experts.

Billions of webpages ImagesProtein sequences

Modern ML: New Learning Approaches

Modern applications: massive amounts of raw

data.

Modern ML: New Learning Approaches

Learning AlgorithmExpert

• Semi-supervised Learning, (Inter)active Learning.

Techniques that best utilize data, minimizing need for expert/human intervention.

Modern applications: massive amounts of

data distributed across multiple locations.

Modern ML: New Learning Approaches

Modern ML: New Learning Approaches

• scientific data

Key new resource communication.

• video data

E.g.,

Modern applications: massive amounts of

data distributed across multiple locations.

The World is Changing Machine Learning

Resource constraints. E.g.,

• Computational efficiency (noise tolerant algos) • Communication

• Human labeling effort

• Statistical efficiency

• Privacy/Incentives

New approaches. E.g.,

• Semi-supervised learning

• Distributed learning

• Interactive learning

• Multi-task/transfer learning

• Never ending learning

• Community detection

d log(1/)

A2

equilibriad(1/)

ÂErr(h)

The World is Changing Machine Learning

My research: foundations, robust algorithms.

Applications: vision, comp. biology, social networks.

New approaches. E.g.,

• Semi-supervised learning

• Distributed learning

• Interactive learning

• Never ending learning

• Community detection

• Multi-task/transfer learning

Machine Learning is Changing the WorldMy research: MLT way to model and tackle

challenges other fields. E.g.,

Game TheoryApproximation Algorithms

Matroid Theory

Machine Learning Theory

Discrete Optimization

Mechanism Design

Control Theory

• Machine Learning Lenses in Other Areas

Outline of the talk

• Modern Learning Paradigms

• Interactive Learning

• Submodularity, implications to Matroid Theory, Algorithmic Game Theory, Optimization

;

V

• Discussion, Other Exciting Directions

Supervised Learning• E.g., which emails are spam and which are important.

Not spam spam

• E.g., classify objects as chairs vs non chairs.

Not chair

chair

Labeled Examples

Learning Algorithm

Expert / Oracle

Data Source

c* : X ! {0,1}

h : X ! {0,1}

(x1,c*(x1)),…, (xm,c*(xm))

• Algo sees (x1,c*(x1)),…, (xm,c*(xm)), xi i.i.d. from D

Distribution D on X

Statistical / PAC learning model

(x1,…,x

m)

+

+

--++--

-

• Does optimization over S, finds hypothesis h 2 C.• Goal: h has small error, err(h)=Prx 2 D(h(x) c*(x))

• c* in C, realizable case; else agnostic

14

Two Main Aspects in Classic Machine Learning

Algorithm Design. How to optimize?Automatically generate rules that do well on observed

data.

Generalization Guarantees, Sample Complexity

Confidence for rule effectiveness on future data.

E.g., Boosting, SVM, etc.

Interactive Machine Learning

• Active Learning

• Learning with more general queries; connections

Active Learning

face

O

O

O

Expert Labeler

raw data

Learning Algorith

m

Unlabeled

data

Classifier

not face

Active Learning in Practice

• Text classification: active SVM (Tong & Koller, ICML2000).

• e.g., request label of the example closest to current separator.

• Video Segmentation (Fathi-Balcan-Ren-Regh, BMVC 11).

w

+-

Exponential improvement.

• Sample with 1/ unlabeled examples; do binary search.-

Active: only O(log 1/) labels.

Passive supervised: (1/) labels to find an -accurate threshold.

+-

Active Algorithm

• Canonical theoretical example [CAL92, Dasgupta04]

Provable Guarantees, Active Learning

• First noise tolerant algo [Balcan, Beygelzimer, Langford,

ICML’06]

[Hanneke07, DasguptaHsuMontleoni’07, Wang’09 , Fridman’09, Koltchinskii10, BHW’08, BeygelzimerHsuLangfordZhang’10, Hsu’10, Minsker’12, Ailon’12, …]

Generic (any class), adversarial label noise.

Substantial follow-up on work.

Lots of exciting activity in recent years.

My work:

Provable Guarantees, Active Learning

• First computationally efficient algos[Balcan-Long COLT’13] [Awasthi-Balcan-Long STOC’14]

Margin Based Active Learning

• Realizable: exponential improvement in label complex. over passive [only O(d log 1/) labels to find w error ].

[Balcan-Long COLT’13] [Awasthi-Balcan-Long STOC’14]

Learning linear separators, when D logconcave in Rd.

• Agnostic: poly-time AL algo outputs w with err(w) =O(´), ´ =err(best lin. sep).

+

+

--++--

-

• Unexpected Implications for Passive Learning!

• Computational complexity: improves significantly on the best guarantees known for passive [KKMS’05], [KLS’09].

• Sample complexity: resolves open question about ERM.

Margin Based Active-Learning, Realizable Case

Draw m1 unlabeled examples, label them, add them to W(1).iterate k = 2, …, s

• find a hypothesis wk-1 consistent with W(k-

1).• W(k)=W(k-1).• sample mk unlabeled samples x

satisfying |wk-1T ¢ x| · k-1

• label them and add them to W(k).

w1

1

w2

2

w3

Log-concave distributions: log of density fnc concave.• wide class: uniform distr. over any convex set, Gaussian,

etc.

Theorem If then err(ws)· D log-concave in Rd.

after

Active learning Passive learning

rounds using

label requests label requests

unlabeled examples

labels per round.

Margin Based Active-Learning, Realizable Case

Analysis: Aggressive Localization

Induction: all w consistent with W(k), err(w) · 1/2k

w

w*

Analysis: Aggressive Localization

Induction: all w consistent with W(k), err(w) · 1/2k

wk-1

w

k-1

w*

Suboptimal

wk-1

w

k-1

w*

Analysis: Aggressive Localization

Induction: all w consistent with W(k), err(w) · 1/2k

wk-1

w

k-1

w*

· 1/2k+1

Analysis: Aggressive Localization

Induction: all w consistent with W(k), err(w) · 1/2k

wk-1

w

k-1

w*

· 1/2k+1

Enough to ensure

Need only labels in round k.

Localization in concept space.

Margin Based Active-Learning, Agnostic Case

Draw m1 unlabeled examples, label them, add them to W.

Localization in instance space.

iterate k=2, …, s • find wk-1 in B(wk-1, rk-1) of small

tk-1 hinge loss wrt W.• Clear working set.• sample mk unlabeled samples x

satisfying |wk-1 ¢ x| · k-1 ;• label them and add them to W.

end iterate

• Analysis: exploit localization & variance analysis control the gap between hinge loss and 0/1 loss in each round.

Improves over Passive Learning too!

Passive Learning

Prior Work Our Work

Malicious

Agnostic

Active Learning[agnostic/malicious]

[KLS’09]

NA

[KLS’09]

err(w)err(w)

err(w) err(w)

err(w)

Improves over Passive Learning too!

Passive Learning

Prior Work Our Work

Malicious

Agnostic

Active Learning[agnostic/malicious]

Info theoretic optimal

[KKMS’05]

[KLS’09]

[KKMS’05]

NAInfo theoretic optimal

Slightly better results for the uniform distribution case.

err(w)

err(w)

err(w)

err(w)

err(w)

err(w)

Useful for active and passive learning!

Localization both algorithmic and analysis tool!

Important direction: richer interactions with the expert.

Learning AlgorithmExpert

Fewer queries

Natural interaction

Better Accuracy

Learning Algorith

m

raw data

Expert

Labeler

Classifier

New Types of InteractionClass Conditional

Query

Mistake Query

Learning Algorithm

raw data

Expert Labeler

)Classifier

dog cat penguin

wolf

Class Conditional & Mistake Queries

• Used in practice, e.g. Faces in IPhoto.• Lack of theoretical understanding.

• Realizable (Folklore): much fewer queries than label requests.

[Balcan-Hanneke COLT’12]

Tight bounds on the number of CCQs to learn in the presence of noise (agnostic and bounded noise)

Our Work

Important direction: richer interactions with the expert.

Learning AlgorithmExpert

Fewer queries

Natural interaction

Better Accuracy

3.5

2.4

9.1

Interaction for Faster Algorithms

E.g., clustering protein sequences by function

Expert an expensive computer program.

Voevodski-Balcan-Roglin-Teng-Xia, [UAI 10, SIMBAD 11, JMLR’12]

• Strong guarantees under natural stability condition: produce accurate clusterings with only k one-vs-all queries.

• BLAST to computes similarity to all other objects in dataset (one-vs-all queries).

• Better accuracy than state of the art techniques using limited info on standard datasets (pFam and SCOP datasets).

Voevodski-Balcan-Roglin-Teng-Xia [UAI 10, SIMBAD 11, JMLR’12]

Better accuracy than state of the art techniques using limited info on standard datasets (pFam and SCOP datasets).

Interaction for Faster Algorithms

Interactive Learning

• First noise tolerant poly time, label efficient algos for high dim. cases.[BL’13] [ABL’14]

• Learning with more general queries.

• Active & Differentially Private [Balcan-Feldman, NIPS’13]

[BH’12]

• Communication complexity, distributed learning.

Cool Implications:

• Sample & computational complexity of passive learning

[Balcan-Blum-Fine-Mansour, COLT’12] Runner UP Best Paper

Summary:

Related Work:

Learning Theory Lenses on Submodularity

Submodular functions• Discrete fns that model laws of diminishing returns.

• Optimization, operations research

• Algorithmic Game Theory

• Machine Learning

[Lehman-Lehman-Nisan’01], ….

[Bilmes’03] [Guestrin-Krause’07], …

• Social Networks

[Kleinberg-Kempe-Tardos’03]

[Balcan&Harvey, STOC 2011 & Necktar Track, ECML-PKDD 2012]

Novel structural results

New Lenses on Submodularity

Learning Submodular Functions

Alg. Game TheoryEconomics

Matroid Theory

Discrete Optimization

Applications: Economics, Social Networks, …

Submodular functions

xS Tx

+

+

Large improvement

Small improvement

For T µ S, xS, f(T [ {x}) – f(T) ¸ f(S [ {x}) – f(S)

• V={1,2, …, n}; set-function f : 2V ! R submodular if

E.g.,

+

Submodular functions

• Concave Functions Let h : R ! R be concave. For each S µ V, let f(S) = h(|S|)

• Vector Spaces Let V={v1,,vn}, each vi 2 Rn.

For each S µ V, let f(S) = rank(V[S])

More examples:

• Coverage Fns Let A1,,An be sets.For each S µ V, let f(S) = ||

Learning submodular functions

• Influence Function in Social Networks• V = set of nodes.• f(S) = expected influence when S is the originating set of nodes in the difusion.

• Valuation Functions in Economics • V = all the items you sell.• f(S) = valuation on set of items S.

PMAC model for learning real valued functions

Distribution D on 2[n]

Labeled Examples

Learning Algorithm

Expert / Oracle

Data Source

Alg.outputs f : 2[n] R+g :2[n] R+

(S1,f(S1)),…, (Sm,f(Sm))

• Algo sees (S1,f(S1)),…, (Sm,f(Sm)), Si i.i.d. from D, produces g.

Probably Mostly Approximately Correct

• With probability ¸ 1-± we have PrS[g(S) · f(S)· ® g(S)]¸ 1-²

Learning submodular functions [BH’11]

No algo can PMAC learn the class of submodular fns with approx. factor õ(n1/3).

Theorem: (General lower bound)

Corollary: Matroid rank fns do not have a concise, approximate representation.

Surprising answer to open question in Economics of

Theorem: (General upper bound)

Poly time alg. for PMAC-learning (w.r.t. an arbitrary distribution) with an approx. factor ®=O(n1/2).

• Even if value queries allowed; even for rank fns of matroids.

Noam NisanPaul Milgrom

A General Lower Bound

Construct a hard family of matroid rank functions.

A1 A2 ALA3

X

X X

Low=log2n

High=n1/3

X

… … …. ….

L=nlog log n

No algo can PMAC learn the class of non-neg., monotone, submodular fns with an approx. factor õ(n1/3).

Theorem

Vast generalization of partition matroids.

Exploit Additional Properties

• E.g., product distribution

• BH’11, Lipschitz submodular

• FV’13 general submodular

• Learning valuation fns from AGT and Economics.

• Learning influence function in social networks.

Learning Valuation FunctionsTarget dependent learnability for classes from Algorithmic Game Theory and Economics

Additive µ OXS µ GS µ Submodular µ XOS µ Subadditive

XOR

OR

({1}, $2) ({2}, $5)

Romania SwitzerlandOR

({2}, $6)({1}, $10)({3}, $5)

[Balcan-Constantin-Iwata-Wang, COLT 2012]

Theorem: (Poly number of XOR trees)

O(n²) approximation in time O(n1/²).

Learning the Influence FunctionInfluence function in social networks: coverage

function.

[Du, Liang, Balcan, Song 2014]

• Maximum likelihood approach; promising performance

• Memetracker Dataset, blog data cascades : “apple and jobs”, “tsunami earthquake”, “william kate marriage”

[Balcan&Harvey, STOC 2011 & Necktar Track, ECML-PKDD 2012]

Novel structural results

New Lenses on Submodularity

Learning Submodular Functions

Algo Game TheoryEconomics

Matroid Theory

Discrete Optimization

Follow-up work: Algo Game Theory, Machine Learning, Discrete Optimization, Privacy.

Discussion. Other Exciting Directions

• Foundations for Modern ML and Applications

Themes of My Research

Game TheoryApprox.

Algorithms

Matroid Theory

Machine Learning Theory

Discrete Optimization

Mechanism Design

Control Theory

• Learning Lens on Other Fields, Connections

Foundations for Modern Paradigms

Resource constraints. E.g.,

• Computational efficiency (noise tolerant algos) • Communication

• Human labeling effort

• Statistical efficiency

• Privacy/Incentives

New approaches. E.g.,

• Semi-supervised learning

• Distributed learning

• Interactive learning

• Never ending learning

• Community detection

• Multi-task/transfer learning

Analysis Beyond the Worst Case• Identify ways to capture structure of real world

instances, analysis beyond the worst case.

• E.g., circumvent NP-hardness barriers for objective based clustering (e.g., k-means, k-median)

• Perturbation resilience: small changes to input distances shouldn’t move optimal solution by much.

• Approximation stability: near-optimal solutions shouldn’t look too different from optimal solution.

0.7

2.1

[BBG’09] [BB’09] [BRT’10] [BL’12]

Analysis Beyond the Worst Case

• Understand characteristics of practical tasks that makes them easier to learn.

• Develop learning tools that take advantage of such characteristics.

Major Direction:

• Identify ways to capture structure of real world instances, analysis beyond the worst case.

Understanding & Influencing Multi-agent Systems

What do we expect to happen?

More realistic: view agents as adapting, learning entities.Major Question: Can we “guide” or “nudge” dynamics to high-quality states quickly [using minimal intervention]?

Traditional: analyze equilibria (static concept)

• Foundations for Modern ML and Applications

Themes of My Research

Game TheoryApprox.

Algorithms

Matroid Theory

Machine Learning Theory

Discrete Optimization

Mechanism Design

Control Theory

• Learning Lens on Other Fields, Connections