View
214
Download
0
Tags:
Embed Size (px)
Citation preview
© Copyright OMRON Corporation Sensing Technology Research Center
Evaluation Result
of the Interim Software
Sep.2, 2002
OMRON SENSING TECHNOLOGY RESEARCH CENTER
Omron CorporationSensing Technology Research Center
Vision-based Human Understanding Technologies Laboratory
To: Hong Kong Baptist University
© Copyright OMRON Corporation Sensing Technology Research Center
Thanks
• Thanks to your considerable efforts, better results obtained with this interim software.
© Copyright OMRON Corporation Sensing Technology Research Center
How to Evaluate
21
3
421
3
6
45
Eye(open)Mouth
1.In2.Out3.Upper4.Lower
1.left2.right3.top of upper lip4.bottom of upper lip 5. top of lower lip6. bottom of lower lip
Evaluation Criteria: ・ Distance between Ground truth and Detected position. (Normalized the width of face to 100 pixels) (See bellow figures: 4 points of open eye, 3 points of close eye and 6 points of mouth)Software version: ・ MD09Aug2002.zipUsing Program: ・ OMRON use one function (Fixed_DetectFeature) . ( To know the potential performance of the ASM Algorithm for mouth )
Ground Truth
Detected Position
21 4
1.In2.Out4.Lower
Eye(close)
OMRON employed the following evaluation criteria for the outline detection of the facial organ .
© Copyright OMRON Corporation Sensing Technology Research Center
Evaluation Data Sets
OMRON evaluated with following Data Sets.
INC_EYEC INC_EYEO INC_MA INC_MI
StateNumber of Images
Number returned the result
Error or not return result
INC_EYEC Wide opened Eye 146 145 1(*)
INC_EYEO Closed Eye 147 147 0
INC_MA Shapes of Japanese vowel “a” 146 146 0
INC_MI Shapes of Japanese vowel “I” 147 147 0
Omron_a1fb1Normal face:
But high resolution image116 72 44(*)
*1) The list of these errors : See appendix
© Copyright OMRON Corporation Sensing Technology Research Center
Summary of Evaluation
[Eye]- The number of error detections of right eye is more than the left eye. I guess that the different model or algorithm is used for the left and right eye detection.- The detection accuracy falls down due to lower eyelid, upper eyelid and eye blow.- I guess the main reason is the error of facial feature point detection.
[Mouth]- Top of upper lip and bottom of lower lip is more accurate.- The opened mouth, i.e. “A” and “I”, should be improved. The variety of the active shape models is important for robustness.- The detection accuracy falls down due to teeth and nose. [ etc ] - Many face and feature point detection error occurred with high resolution images.
© Copyright OMRON Corporation Sensing Technology Research Center
Suggestion
Our goal is to construct the principle algorithm of the state estimation of facial organ until February 2003. It is important for us to develop the ASM algorithm for the outline detection of facial organ as soon as possible.
- We need to investigate the cause of failure in detecting the outline of facial organ. We should consider the errors of ASM algorithm and the others separately. This is the reason why we should evaluate the outline detection of facial organ using fixed corners of eyes and mouth.
- In order to realize the last goal until February, OMRON want you to begin to concentrate the research of the state estimation of facial organ.
© Copyright OMRON Corporation Sensing Technology Research Center
Correct Examples
Closed eye
Wide opened Eye
shape of ”A” Mouth
shape of ”I” Mouth
© Copyright OMRON Corporation Sensing Technology Research Center
Error Example:Closed Eye
I guess the main reason is the error of facial feature detection
Eye Brow Lower eyelid
Too short
Etc.
-> The length of left and right eye feature points differ greatly.
INC_EYEC
© Copyright OMRON Corporation Sensing Technology Research Center
Error Examples:Wide opened Eye
Opened eye is mistaken as closed eye.
Influence of the lower eyelid
Etc.
Nonexistent relation of between outline of eye and outline of black eye
Eye blow
Identify the inside eye and the black eye
INC_EYEO
© Copyright OMRON Corporation Sensing Technology Research Center
Error Examples:shape of ”A” Mouth
The left and right points of mouth are fixed landmark points.
Can’t spread the model Upper-lower is wrong. This cases is too much.
Influence of the ChinDon’t leach the Upper-upper.Influence of the teeth
INC_MA
© Copyright OMRON Corporation Sensing Technology Research Center
Error Examples: shape of ”I” Mouth
Influence of the nose. (Noise of nose)thin Upper-lip
Influence of teeth?Can’t stretch the Upper-lower and Lower-Upper
INC_MIThe left and right points of mouth are fixed landmark points.
© Copyright OMRON Corporation Sensing Technology Research Center
Error Examples: high resolution image (no.1)
These images were saved by JPG format. If I convert JPG to BMP, we got slightly different results. But Results is not so good.
omron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
The accuracy of facial feature detection is low.Please check your software of face detection and facial feature detection.
Error Example: high resolution image (no.2)omron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 3pix (Left eye)Result
Detection Rate within 3pix(LeftEye)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Out In Upper Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 3pix (Right eye)Result
Detection Rate within 3pix(RightEye)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
In Out Upper Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 3pix (Mouth)Result
Detection Rate within 3pix(Mouth)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Left Right Top of Upper Bottom of Upper Top of Lower Bottom of Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 5pix (Left eye)Result
Detection Rate within 5pix(LeftEye)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Out In Upper Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 5pix (Right eye)Result
Detection Rate within 5pix(RightEye)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
In Out Upper Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 5pix (Mouth)Result
Detection Rate within 3pix(Mouth)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Left Right Top of Upper Bottom of Upper Top of Lower Bottom of Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 7pix (Left eye)Result
Detection Rate within 3pix(LeftEye)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Out In Upper Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 7pix (Right eye)Result
Detection Rate within 3pix(RightEye)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
In Out Upper Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Detection Rate within 7pix Mouth)Result
Detection Rate within 3pix(Mouth)
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
90.0
100.0
Left Right Top of Upper Bottom of Upper Top of Lower Bottom of Lower
%
INC_MAINC_MIINC_EYECINC_EYCOomron_a1fb1
© Copyright OMRON Corporation Sensing Technology Research Center
Speed(Only outline detection of facial organResult
Average Time(Only facial organ detection)
0.0
5.0
10.0
15.0
20.0
25.0
30.0
35.0
40.0se
c
INC_MA 2.8 2.0 26.5 INC_MI 2.7 1.9 19.6 INC_EYEC 2.0 1.2 22.8 INC_EYCO 3.0 2.2 24.9 omron_a1fb0- F1 3.1 2.3 35.8
Left Right Mouth
*) Pentium4 2.2GHz
© Copyright OMRON Corporation Sensing Technology Research Center
The List of Error FileFixed_DetectFeature(‘XXX\INC35\omron-ct\EYEC\
930077M3INC1EYECG0_000');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0066F1B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0004F1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\0070M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0028F1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1011M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0029F1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1013M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0044M1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1017M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0071M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1021M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0073M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1028M1B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\1076M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1029F2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0011M1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1030M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0014M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1033M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0018M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1042M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0025F1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1045M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0036F3B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1049M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0038F1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1050M3B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0039F1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1051M1B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0040F1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1058M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0043M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1059M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0047M1B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1064M2B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0048M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1067M1B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0049M2B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1068M1B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0055M4B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1069F1B1A1F6');
Fixed_DetectFeature('XXX\OmronDB\a1fb1\0063M4B1A1F6'); Fixed_DetectFeature('XXX\OmronDB\a1fb1\1070M1B1A1F6');