Embed Size (px)
Citation preview
Matthias Wimmer, Bernd Radig, Michael Beetz
Chair for Image UnderstandingComputer Science
Technische Universität München
Adaptive Skin Color Classification
2005-12-18 2/10Technische Universität MünchenMatthias Wimmer
Motivation
Skin color detection supports… face model fitting
mimic recognition person identification gaze estimation fatigue detection (e.g. vehicle)
hand tracking gesture recognition action recognition supervising work
challenge our approach results outlook
2005-12-18 3/10Technische Universität MünchenMatthias Wimmer
Challenge Skin color depends on image conditions:
illumination: light source, light color, shadow, shading,… camera: type, settings,… visible person: ethnic group, tan,…
Skin color occupies a large area within color space
challenge our approach results outlook
2005-12-18 4/10Technische Universität MünchenMatthias Wimmer
Observations Skin color varies greatly between images. Skin color varies slightly within an image.
Basic idea learn image specific skin color characteristics parameterize a skin color classifier accordingly
skin color of image1 skin color of image2
challenge our approach results outlook
2005-12-18 5/10Technische Universität MünchenMatthias Wimmer
Our approach
Offline step: learn the skin color mask
specific for any face detector
Online steps: Step 1: detect the image specific skin color model
using the face detector using the skin color mask
Step 2: adapt a skin color classifier Step 3: calculate the skin color image
challenge our approach results outlook
2005-12-18 6/10Technische Universität MünchenMatthias Wimmer
Skin color model & skin color classifier detect the face extract the skin color pixels skin color model:
mean values: μr, μg, μbase
standard deviations: σr, σg, σbase
adaptive skin color classifier:skin := lowr ≤ r ≤ highr
lowg ≤ g ≤ highg lowbase ≤ base ≤ highbase
learn the boundslowr := μr – 2σr
highr := μr + 2σr . . . . . . . . .
challenge our approach results outlook
2005-12-18 7/10Technische Universität MünchenMatthias Wimmer
Results better results for
colored persons exact shape outline detection of facial parts:
eyes, lips, brows,…
correctly detected pixels: non-adaptive approach: 90.4% 74.8%
40.2% adaptive approach: 97.5% 87.5%
97.0% improvement: 7.9% 17.0%
141.3%
challenge our approach results outlook
2005-12-18 8/10Technische Universität MünchenMatthias Wimmer
Conclusion Challenge: much variation within skin color
illumination, camera, visible person skin color occupies a large area within color space
We propose a way to reduce those variations exploit an image specific skin color model adapt a skin color classifier to that skin color model
We proved our approach using a simple but real-time capable skin color classifier comparison: non-adaptive ↔ adaptive
challenge our approach results outlook
2005-12-18 9/10Technische Universität MünchenMatthias Wimmer
Ongoing research Learn skin color mask for other face detectors Specialize more powerful skin color classifiers Recognize other feature images/color images
lip color image tooth color image eye color image hair color image eye brow color image
example: lip color detection
challenge our approach results outlook
2005-12-18 10/10Technische Universität MünchenMatthias Wimmer
Thank you !
2005-12-18 11/10Technische Universität MünchenMatthias Wimmer
Overview Motivation / Challenge Our approach
extracting skin color pixels adapt skin color classifier
Results Conclusion Outlook
2005-12-18 12/10Technische Universität MünchenMatthias Wimmer
Challenge (2): non-skin color pixels Skin color pixels have to be separated from non-
skin color pixels. Areas of skin color and
non-skin color overlap. Color can not make a
distinctive separation.
challenge our approach results outlook
2005-12-18 13/10Technische Universität MünchenMatthias Wimmer
Basic idea learn image specific skin color characteristics parameterize classifier accordingly with those
characteristics
2005-12-18 14/10Technische Universität MünchenMatthias Wimmer
Offline: Learn the skin color mask face image database with labeled skin color pixels skin color mask: array with 24 x 24 cells
Computational steps:
1. detect the face in every image
2. every cell is assigned the relative number of labeled skin color pixels at its position
3. apply threshold
1. 2. 3.
challenge our approach results outlook
2005-12-18 15/10Technische Universität MünchenMatthias Wimmer
Step 1: Detect the image specific skin color model
detect the face extract the skin color pixels normalized RGB color space:
base = R + G + B
r = R / base
g = G / base
skin color model: mean values: μr, μg, μbase
standard deviations: σr, σg, σbase
challenge our approach results outlook
2005-12-18 16/10Technische Universität MünchenMatthias Wimmer
Step 2: Adapt a skin color classifier non-adaptive skin color classifier:
skin := 0.35 ≤ r ≤ 0.5 0.2 ≤ g ≤ 0.7 200 ≤ base ≤ 740
adaptive skin color classifier:skin := lowr ≤ r ≤ highr
lowg ≤ g ≤ highg lowbase ≤ base ≤ highbase
learn the bounds via the skin color model mean value and standard deviationlowr := μr – 2σr
highr := μr + 2σr . . . . . . . . .
linear function:lowr := aμr + bμg + cμbase + dσr + eσg + fσbase + g
. . .
challenge our approach results outlook
2005-12-18 17/10Technische Universität MünchenMatthias Wimmer
Adaptive skin color classifier non adaptive skin color classifier:
skin := 0.35 ≤ r ≤ 0.5 0.2 ≤ g ≤ 0.7 200 ≤ base ≤ 740
adaptive skin color classifier:skin := lowr ≤ r ≤ highr lowg ≤ g ≤ highg
lowbase ≤ base ≤ highbase
learn the bounds out of the skin color model
2005-12-18 18/10Technische Universität MünchenMatthias Wimmer
Related work Feedback of information from
high level vision components to low level vision components
challenge our approach results outlook
