Attentional CascadeImprovements

Research by Jeffrey A. Edlund and Greg S. GriffinLearning Systems CS 156b, Mar 9 2006

Simple Training Sets

Suggested Improvements

1. Dual Cascade– When: data is balanced– Why: faster

2. Fade Cascade– When: data is unbalanced– Why: fewer training examples

required

Dual Cascade: What Is It

NHU=4 NHU=8 NHU=12

€

⇒

€

⇒

€

⇒

€

⇓

€

⇓

€

⇓

Early Rejection

Dual Cascade: What Is It

NHU=4 NHU=8 NHU=12

€

⇒

€

⇒

€

⇒

€

⇓

€

⇓

€

⇓€

⇑

€

⇑

€

⇑

Early Rejection

Early Acceptance

Unbalanced Data

Balanced Data

Dual Cascade: Pros & Cons

• Major Advantage: speed– ~ 2 - 4 times faster on equally balanced data sets– Even faster if you have more positive examples– Little or no increase if positive examples are rare

• e.g. Viola and Jones

• Minor Disadvantage: accuracy– Small increase in out-of-sample error

Dual Cascade: Applications

Fade Cascade AddressesThe Big Problem:

ntest =16,000

x ~ 2

x ~ 10

Fade Cascade

• Instead of throwing out points that are clearly positive or clearly negative, we reduce the weights for those points and renormalize.

Balanced Data

Unbalanced Data

Fade Cascade: Pros & Cons

• Major Advantage: fewer training examples– ~ 2 - 4 times less training data required, for the

same out-of-sample error

• Minor Disadvantage: less efficient training– All data points are now used for training, at all

levels of the cascade (we’re weighting them instead of dropping them)

– But: we now require less training data

Suggested Improvements

1. Dual Cascade– When: data is balanced– Why: faster

2. Fade Cascade– When: data is unbalanced– Why: fewer training examples

required

Unbalanced Data

Balanced Data

Unbalanced Data

Preliminary!

Balanced Data

Preliminary!

“Smart” Cascade

• Improved performance on both balanced and unbalanced datasets?

• We don’t know yet!