46
AB I N J O:U C L C R I Hannes Schulz, Hauke Strasdat, and Sven Behnke University of Freiburg Institute of Computer Science Nov 29, 2007

The ball is not just orange

  • Upload
    cijat

  • View
    1.151

  • Download
    2

Embed Size (px)

DESCRIPTION

How to find balls in images of a moving humanoid robot using neural networks.Talk from RoboCup Workshop 07 in Pittsburgh.

Citation preview

Page 1: The ball is not just orange

A B I N J O: U C L C R I

Hannes Schulz, Hauke Strasdat, and Sven Behnke

University of FreiburgInstitute of Computer Science

Nov 29, 2007

Page 2: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

M

The ball

is small,

is easy to confuse with other objects

is the most important object on the field:You cannot play sensibly without knowing its position

Page 3: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

M

The ball

is small,

is easy to confuse with other objects

is the most important object on the field:You cannot play sensibly without knowing its position

Page 4: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

M

The ball

is small,

is easy to confuse with other objects

is the most important object on the field:You cannot play sensibly without knowing its position

Page 5: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

M

The ball

is small,

is easy to confuse with other objects

is the most important object on the field:You cannot play sensibly without knowing its position

. We should put a lot of effort into finding the single real ball.

Page 6: The ball is not just orange

O

1 I B N-B

2 F B C

3 C B C

4 E

Page 7: The ball is not just orange

O

1 I B N-B

2 F B C

3 C B C

4 E

Page 8: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

Page 9: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

T E C

Page 10: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

L C

Page 11: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

M B

Page 12: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

CW L

Page 13: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

C R

Page 14: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

Page 15: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

H F

Page 16: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

O O

Page 17: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

O O

Page 18: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

F

Page 19: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

F

Page 20: The ball is not just orange

O

1 I B N-B

2 F B C

3 C B C

4 E

Page 21: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C E YUV S

Actual ball color

Wider, brownish color

. Allows for motion blur

Page 22: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C E YUV S

Two ellipses for “orange”

Actual ball color

Wider, brownish color

. Allows for motion blur

Page 23: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C E YUV S

Two ellipses for “orange”

Actual ball color

Wider, brownish color

. Allows for motion blur

Page 24: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

S C I

orange

white

green

(-candidate)

64 : 1

YUV Camera Image

Page 25: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

S C I

orange

white

green

(-candidate)

64 : 1

1. Find Maximum

2. Find Weighted Mean

Page 26: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

S C I

orange

white

green

(-candidate)

64 : 1

cut corresponding area

Page 27: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

S C I

orange

white

green

(-candidate)

64 : 1

cut corresponding area

Box size depends on position in image

Page 28: The ball is not just orange

O

1 I B N-B

2 F B C

3 C B C

4 E

Page 29: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

P C

. Projection changes with ellipses: Robust to changes inlighting conditions

Page 30: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

P C

. Projection changes with ellipses: Robust to changes inlighting conditions

Page 31: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

P C

. Projection changes with ellipses: Robust to changes inlighting conditions

Page 32: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

P L

YUV-Image Y-Image Subsampled Y-Image

. Subtraction of mean: Robust to changes in lighting conditions

Page 33: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H L F

Page 34: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H L F

Page 35: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H L F

+

-

- -

-

+

-

- -

-

+

-

- -

-

+

-

- -

-

Page 36: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

N N C

+

-

- -

-

1: Ball

0: No Ball

Page 37: The ball is not just orange

O

1 I B N-B

2 F B C

3 C B C

4 E

Page 38: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 06 – T

H520 MHz ARM PocketPC

VGA HTC Camera

D S160 balls

440 non-balls

divided randomly in training set(80%) and test set (20%)

P (T S)100% of distractors classifiedcorrectly

1 ball out of 32 not recognized

Page 39: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 06 – T

H520 MHz ARM PocketPC

VGA HTC Camera

D S160 balls

440 non-balls

divided randomly in training set(80%) and test set (20%)

P (T S)100% of distractors classifiedcorrectly

1 ball out of 32 not recognized

Page 40: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 06 – R T

H520 MHz ARM PocketPC

VGA HTC Camera

R T T

Robot decides autonomouslywhich object to approach.

Page 41: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 2007

H1.33 GHz PC

WVGA µEye Camera

Page 42: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 2007

S D D S273 balls

548 non-balls

training set 62%, validation set13%, test set 25%

varying lighting conditions

Page 43: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 2007

A SAvg Luminance

ball non-ball

Avg Orange-Greennessball non-ball

D S273 balls

548 non-balls

training set 62%, validation set13%, test set 25%

varying lighting conditions

Page 44: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 2007

A SAvg Luminance

ball non-ball

Avg Orange-Greennessball non-ball

D S273 balls

548 non-balls

training set 62%, validation set13%, test set 25%

varying lighting conditions

R T S91.1% accuracy

76.6% if stimuli flipped up/down.Drop suggests dependency ongradient.

88.2% if lighting in testset differs.Classifier seems indifferent.

Page 45: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C C

Classifier

Task Neural Net Linear Classifier KNN (k = 5)

Regular 91.1% 86.5% 88.5%Flipped 74.0% 70.0% 76.6%

Page 46: The ball is not just orange

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C

The ball has properties aside from being orange

These properties are exploited by our Neural NetworkClassifiers

Changes in Lighting conditions can be dealt with by projectionto lines in YUV-space and Luminance.

The method introduced here can be generalized to othersmall-sized objects on the field.