Embed Size (px)
Citation preview
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
MultisensoryPerception
Alex Teichman
August 26, 2009
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
The Object Recognition Problem
How much does this look like a hand?
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
The Object Recognition Problem
Where is the coke can?
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
The Object Recognition Problem
What is the object in the box?
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Some Difficulties Among Many
Labels are painful to acquire.
Generalization to data not in your training set can bearbitrarily hard.
Descriptors are generally suited to a particular task.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Some Difficulties Among Many
Labels are painful to acquire.
Generalization to data not in your training set can bearbitrarily hard.
Descriptors are generally suited to a particular task.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Some Difficulties Among Many
Labels are painful to acquire.
Generalization to data not in your training set can bearbitrarily hard.
Descriptors are generally suited to a particular task.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Boosting - Intuition
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Boosting - Intuition
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Boosting - Math
Training examples (ym,~xm) for all m = 1..M, ym ∈ {−1,+1}.
minH
∑m
exp(−ymH(~xm)) (1)
H(~x) =∑
t
ht(~x) (2)
ht(~x) =
{at if dist(~x , ~xt) < θt
0 otherwise(3)
Stagewise optimization:minht+1
∑m exp (−ym (H(~xm) + ht+1(~xm)))
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Boosting - Math
Training examples (ym,~xm) for all m = 1..M, ym ∈ {−1,+1}.
minH
∑m
exp(−ymH(~xm)) (1)
H(~x) =∑
t
ht(~x) (2)
ht(~x) =
{at if dist(~x , ~xt) < θt
0 otherwise(3)
Stagewise optimization:minht+1
∑m exp (−ym (H(~xm) + ht+1(~xm)))
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Boosting - Math
Training examples (ym,~xm) for all m = 1..M, ym ∈ {−1,+1}.
minH
∑m
exp(−ymH(~xm)) (1)
H(~x) =∑
t
ht(~x) (2)
ht(~x) =
{at if dist(~x , ~xt) < θt
0 otherwise(3)
Stagewise optimization:minht+1
∑m exp (−ym (H(~xm) + ht+1(~xm)))
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Boosting - Math
Training examples (ym,~xm) for all m = 1..M, ym ∈ {−1,+1}.
minH
∑m
exp(−ymH(~xm)) (1)
H(~x) =∑
t
ht(~x) (2)
ht(~x) =
{at if dist(~x , ~xt) < θt
0 otherwise(3)
Stagewise optimization:minht+1
∑m exp (−ym (H(~xm) + ht+1(~xm)))
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Image Descriptors Library
Goal: Plug-and-play image descriptors.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Image Descriptors Library
See descriptors 2d package
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Hand Detection
Histogram of Oriented Gradient descriptors (3 differentparameter settings)
Superpixel Color Histogram descriptors (4 differentsuperpixel sizes)
1 hour to train.
Collaboration with Romain and Alex S.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Hand Detection
Histogram of Oriented Gradient descriptors (3 differentparameter settings)
Superpixel Color Histogram descriptors (4 differentsuperpixel sizes)
1 hour to train.
Collaboration with Romain and Alex S.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Hand Detection
Histogram of Oriented Gradient descriptors (3 differentparameter settings)
Superpixel Color Histogram descriptors (4 differentsuperpixel sizes)
1 hour to train.
Collaboration with Romain and Alex S.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Hand Detection
Histogram of Oriented Gradient descriptors (3 differentparameter settings)
Superpixel Color Histogram descriptors (4 differentsuperpixel sizes)
1 hour to train.
Collaboration with Romain and Alex S.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Video Label Propagation
Label a video given just a few frame labels.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Mechanical Turk Feedback Loop
Implementing the feedback loop of the hands training methodologyon mechanical turk with a bottles dataset.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Online Learning
mina
∑m
exp(−ymH(~xm))
Can solve this for all weak classifier responsessimultaneously.Constant space for any number of training examples.
Maintain the gradient and Hessian as a stream of labels comesinto the classifier:
g =1
M
∑m
−ymIm
H =1
M
∑m
ImITm
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Moving Forward
Labels are painful to acquire.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Generalization to data not in your training set can bearbitrarily hard.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Descriptors are generally suited to a particular task.
Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Moving Forward
Labels are painful to acquire.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Generalization to data not in your training set can bearbitrarily hard.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Descriptors are generally suited to a particular task.
Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Moving Forward
Labels are painful to acquire.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Generalization to data not in your training set can bearbitrarily hard.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Descriptors are generally suited to a particular task.
Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Moving Forward
Labels are painful to acquire.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Generalization to data not in your training set can bearbitrarily hard.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Descriptors are generally suited to a particular task.
Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Moving Forward
Labels are painful to acquire.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Generalization to data not in your training set can bearbitrarily hard.
Put classifier predictions on Mechanical Turk, getcorrections, feed back into boosting.
Descriptors are generally suited to a particular task.
Use a large library of descriptors and allow boosting tochoose which are most useful through the optimization.
MultisensoryPerception
AlexTeichman
The ObjectRecognitionProblem
Boosting
ImageDescriptorsLibrary
Applications
MovingForward
Thanks!
