View
219
Download
0
Category
Tags:
Preview:
Citation preview
Face Recognition and Retrieval in VideoBasic concept of Face Recog. & retrieval And their basic methods.
C.S.E.Kwon Min Hyuk
True? False?Q1 : in recently, face recognition
researches focus on video-based rather than still image-based (O / X)
Q2 : There is three approaches; (O/ X)◦1. key-frame based◦2. Temporal model based◦3. image set based
Face variation and expression make face recognition difficult. (O/X)
Intro Q1 : Why do we need Face recogni-
tion system?◦ Increasing request to search specific peo-
ple related video contents
◦ Can be applied at security, human-com-puter inter action etc.
IntroQ2: What is recent trend of approach
to Face recog.?
◦ Traditionally , focused on Still image-based appr.
◦ Recently , focused on Video-based appr.
◦We can extract more information from video than that of still image.
General steps for Face recognitionWhere is face located in video
frame?◦ We should look for which part of the frame
is face. Face detecting and Tracking.
Recognizing face◦There is some basic approach for
Face Recog. Key fame-based approach Temporal Model-based approach Image set-based approach
Face detectionUsing statistical geometric model
◦ From the frame Extract appearance fea-tures such as edge, intensity, color(histogram)
To evolve the face detector by using machine learning tech.◦Adaboost◦Neural Network◦Support Vector Machine
Face trackingFace detection’s limit.
◦It detect only frontal or near frontal view.
Tracking face is needed to handle large head motions.
Face tracking◦Difficulties◦Method to solve
Face trackingDifficulty 1 : There is
◦Face appearance variation◦3D motion◦Background change
Method to solve: face online boosting◦Using tracked images in previous
frames.◦Applying current result to tracking
seq. for next frame. (real-time updat-ing feedback)
example in next slide
Face tracking(con’d)Difficulty 2: The adaptive tracker
can adapt to non-targets.
Method to solve : add basal appear-ance of target ◦ Teaching the tracker about some basal ap-
pearance of the target. ◦ Basal appearance : Image set of various
target condition (face expression, pose etc)
After face detecting and trackingWe can determine the part of
frame where face is located.
Now, we can get to face recogni-tion.
Face RecognitionBasic steps for Face Recog.
◦1. get weak evidence in individual frame.
◦2. collect that evidence over time.◦3. lead(determine) reliable result.
Three approaches◦1. key-frame approach◦2. temporal model-based approach◦3. image set-based approach
Key-frame based ap-proachTreat each video as a collection
of images. Basic steps of the approach.
◦1. input data(still images, video)◦2. from data, extract images of the
target. Extracted images are called key-frames
or examplars.
◦3. matching them with all or subset of other video sequence(where the target is).
Key-frame based ap-proach(con’d)How can we get some ‘good’ key-
frame from input data?◦By image-based recognition◦In each frame, probe the nose and
eyes’ triangular structure.
If it is in the frame, then face recognition is performed. And key-frame is extracted.
Key-frame based ap-proach(con’d)
◦Applying K-Means clustering Cluster means a group whose elements
have some common property. This algorithm is grouping some data ob-
servations into one of cluster which has nearest mean.
Key-frame based ap-proach(con’d)Other algorithms
◦ Isomap algorithm◦ Combination of majority and probabilistic
voting.◦ And so on. (I’ll skip the details.)
Finally, all or subset of video sequence will be compared(matched) with extracted ‘good’ key-frame to determine recognition.
Temporal Model Based approachTo handle face dynamics
◦Ex: face expression(non-rigid)or head movement(rigid)
Using temporal sequence(continuous coherent)◦Ex> Using whole sequence of chang-
ing face dynamics as a image set.
Temporal Model Based ap-proach(con’d)Basic methods
◦Matching the face Trajectory. Trajectory means the moving face’s
path(orbit) through in surfaces. Two(model and object) trajectory dis-
tance accumulates recognition evidence over time.
Temporal Model Based ap-proach(con’d)Other method
◦Trained statistical face model Using density estimation.
◦Probabilistic approach Using time-series state space variables
◦Hidden Markov Model Fusing pose and person-discriminant fea-
tures.
I’ll skip all of details.
Image set based approachThis approach uses
◦both image collected over consecutive time
(similar with temporal image set)
◦And independent still image set (similar with key-frame)
Combination of both temporal and key-frame based approaches.
Two major approaches.◦Statistical modal-based◦Mutual subspace-based
Image set based ap-proach(con’d)Image set classification
◦Non-parametric sample based Compare representative images of each
image sets
◦Parametric model-based In terms of probabilistic, compare two
distributionsof each image set.
Image set based ap-proach(con’d)Statistical Model-based
◦To determine recognition, consider similarity of two manifolds manifold is large group (more than clus-
ter)which contains several cluster.
Drawback ◦Need to solve the difficult parameter
estimation problem.
Image set based ap-proach(con’d)Mutual Subspace-Based
Model(MSM)◦To determine Similarity between im-
age sets measure by the smallest principal angles
between subspaces.
CMSM is expansion of MSM◦Assume more constraints.
Face RetrievalIt is difficult to recognize face in the
uncontrolled condition like face dy-namics , light intensity, hair styles
◦There is two applications1. Person Retrieval 2. Cast listing
Person RetrievalFace recognition tech. are ap-
plied.Basic method
◦Using head model (with multiple tex-ture map)
◦Step1. rendering(extract or gener-ate) face images
◦Step2. identifying target face.◦Step3. updating the texture map of
the model.
Cast listing (cast : actors or charac-ters in film)
Automatic cast listing is interest-ing problem.
Based on face recognition◦Because face is repeatable cue in
the filmUsing image clustering method
For accuracy, Treat clothing ap-pearance additional cues for clustering.
Challenges and Future di-rectionDatabases.
◦Constructed in lab. enviro. Not a real world.
◦Limited face appearance of variation.Low-quality Video data
◦Exist lots of noise hard to filter out.Computational Cost
◦Face recognition requires quite high power devices.
Recommended