Upload
brennan-matthews
View
49
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Label Embedding Trees for Large Multi-class Tasks. Samy Bengio Jason Weston David Grangier. Presented by Zhengming Xing. Outline. Introduction Label Trees Label Embeddings Experiment result. Introduction. Large scale problem: the number of example Feature dimension Number of class. - PowerPoint PPT Presentation
Citation preview
Label Embedding Trees for Large Multi-class Tasks
Samy Bengio Jason WestonDavid Grangier
Presented by Zhengming Xing
Outline
• Introduction• Label Trees• Label Embeddings• Experiment result
Introduction
Main idea: propose a fast and memory saving multi-class classifier for large dataset based on trees structure method
Large scale problem:
the number of example Feature dimensionNumber of class
Introduction Label Tree:
Label Predictors:
Indexed nodes:
Edges:
Label sets:
The root contain all classes, and each child label set is a subset of its parent
K is the number of classes
Disjoint tree: any two nodes at the same depth cannot share any labels.
IntroductionClassifying an example:
Label TreesTree loss
I is the indicator function
is the depth in the tree of the final prediction for x
Label treeLearning with fixed label tree: N,E,L chosen in advanceGoal: minimize the tree loss over the variables F
Given training data
Relaxation 1
Relaxation 2
Replace indicator function with hinge loss and
Label treeLearning label tree structure for disjoint tree
Treat A as the affinity matrix and apply the steps similar to spectral clustering
Basic idea: group together labels into the same label set that are likely to be confused at test time.
Label embeddingsis a k-dimensional vector with a 1 in the yth position and 0 otherwise
Problem : how to learn W, V
define
solve
Label embeddingsMethod 1:
The same two steps of algorithm 2Learn V
Learn W
minimize
minimize
Label embedding
Combine all the methods discussed above
Method 2: join learn W and V
minimize
minimize
ExperimentDataset
Experiment
Experiment