Embed Size (px)
Citation preview
Outlines• Motivation • ProblematicIntroduction
• hand gesture analysis approaches• vision based gesture taxonomies and
representations Resume
• challenges• analysis of existing literatureComparisons
• Our proposition: last year PFE of my collegues example
conclusion2/30
Introduction• Motivation : Appear of advanced input systems such us
gesture tracking. Usage of hand gesture in gaming helpful for disabled people. • Problematic: this paper investigates about various
application domain that uses hand gesture it shows a progress report on static and
dynamic gesture recognition throw gesture taxonomies , representation and recognition techniques.
3/30
Hand gesture analysis approaches
Glove based analysis
- glove : give as the finger flexion
- sensors mechanical or optical : attached to the glove and act like transducer
- sensors magnetic or acoustic : to determine the relative position of the hand
vision-based analysis
- Three dimensional model of human hand
- cameras : matches the images of the hand to the model
- Estimating parameters : used to detect a gesture classifications
analysis of drawing gestures
- Stylus : input device based on the recognition of written text
Resume: hand gesture analysis approaches
5/30
• Enabling technology for HCI1. CONTACT-BASED DEVICES: based on physical interaction of users with the device interfaces several detectors are attached to the device So we have several types of devices 1) The mechanical devices: are based on motion sensors 2) The haptic-primed devices :are based on multi-touch screen devices 3) The ultrasonic devices: are based on motion trackers that contains
sonic emitters 4) inertial devices are based : on the magnetic earths' field we have 3
solutions:
the use of wi controller
independent of the target
system
detecting normal
gesture using an
accelerometer
a user- intuitive
system that represent a
multimodal of personalized gestures .
Resume: hand gesture analysis approaches:
6/30
CYBERFORCE SYSTEM CYBERGRASPCYBERGLOVE
Suit that captureThe body gestures
Mechanical devices
7/30
INFRARED CAMERA FISH EYE CAMERA: TIME OF FLIGHT CAMERA
STEREO VISION BASED CAMERA PANTILZOOM CAMERA:
Resume: hand gesture analysis approaches:vision –based recognition camera
types
8/30
• Enabling technology for HCi.2. VISION-BASED DEVICES: based on specific cameras , hand markers several types of cameras depends on their use :a. INFRARED CAMERA : used for night visionb. FISH EYE CAMERA: used with minimum variations c. TIME OF FLIGHT CAMERA: for depth information d. STEREO VISION BASED CAMERA: deliver 3D global
informatione. PANTILZOOM CAMERA: to precisely identify details in a
captured scene We have 2 types of hand markers a. REFLECTIVE: passive as they shine only when strobes hit themb. LIGHT –EMITTING DIODES : active as they flash in sequence
Resume: hand gesture analysis approaches:
9/30
Hand vision based gesture taxonomies
and representations
Gestures are an important aspect of human interaction, both interpersonally and in the context of Human-Machine Interfaces.
A gesture is a form of non-verbal communication in which visible bodily actions communicate particular messages, either in place of speech or together and in parallel with words.
Gestures include movement of the hands, face, or other parts of the body.
What are gestures ?
11/30
• A way for computers to understand human body language
• Enables humans to interface with machines and interact naturally without any mechanical devices
• Interpreting human gestures via mathematical algorithms
What is gesture recognition ?
12/30
What are vision-based gesture taxonomies?
Theorical reseachs
dynamicStatic
13/30
Emblems : Familiar gestures that vary across different cultures (ex: the sign of « ok »)
Regulators : Regulate, modulate and maintain the flow of speech during a conversation (Ex:
interruption)Adaptors :
Postural changes and other movements frequently made to feel more comfortable (ex : legs and arms positionAffective display :
Body movements that display a certain affective state (ex : emotions)Illustrators :
Pointing to something that you are discussing about. It reinforces what you are saying.
What are types of gestures ?
14/30
The input devices
• WIRED GLOVES
• DEPTH AWARE CAMERAS
• STEREO CAMERAS
• CONTROLLER-BASED GESTURES
15/30
Gesture Recognition
System
User interface display
Standard web camera
user Randering & Updating object
Image capture
Image input
Hand movement
System recognition architecture A basic working of the gesture
recognition system 16/30
17/30
3D model-based algorithms
Appearance-based models
The representation models
18/30
3D model based :• Used heavely for computer animation• Created of 3D complicated surfaces
The representation models
Inconvenient :Use heavy 3D data
Advantage :Update the model parameters
19/30
2. Apprearance based model:• Performs an average shape from point sets• Sequences as gesture templates.
(Parameters for this method are either the images
themselves)
The representation models
20/30
• Hidden Markov Model (HMM) : is a joint statistical model for an ordered sequence of variables.
• Finite-state machine (FSM) : development tool for approaching and solving problems and as a formal way of describing solutions for later developers
The representation models
21/30
The gesture reconition techniques
• K-means : This classification searches for statistically similar groups in multi-spectral space.
• K-nearest neighbors (K-NN) : classify objects according to the closest training examples in the feature space.
• Template matching : determine similarities between two entities (points, cures, or shapes)
• Support vector machine (SVM) : to nonlinearly map input data to where the data to separate them.
• Dynamic time warping (DTW) : used to find the optimal alignment of two signals by calculating the distance between each possible pair of points out of two signals
22/30
Virtual controllers:-
Remote control:-Through the use of gesture recognition, remote control with the wave of a hand of various devices is possible.
For systems where the act of finding or acquiring a physical controller could require too much time, gestures can be used as an alternative control mechanism. Controlling secondary devices in a car, or controlling a television set are examples of such usage.
Uses Of Gesture Recognition
23/30
Uses Of Gesture RecognitionSocially assistive robotics :
Sign language recognition:
By using proper sensors worn on the body of a patient and by reading the values from those sensors, robots can assist in patient rehabilitation. The best example can be stroke rehabilitation.
Just as speech recognition can transcribe speech to text, certain types of gesture recognition software can transcribe the symbols represented through sign language into text.
24/30
Comparison • Challenges :Handle a degree of freedom: huge variability of 2D
appearance depends on camera view point , different silhouette scale and many resolution for temporal dimension .
The trade off between : performance , cost , according balancing to the type of the application , real time performance scalability and user independence
The ability to analyze images of different hand gestures under different lighting and background conditions
25/30
• Existing literature
Comparison
26/30
• Existing Product & software
Comparison
27/30
o literatures
Representation technique :based on computer-aided design through a wired model of the object
Difficulty : the system can handle only a limited number of shapes
Advantage: allows for real-time object representation along with minimal computing effort
3D model Representation
technique : segments the potential region with an object of interest from the given input sequence.
Difficulty : sensitive to viewpoint changes and thus cannot provide precise spatial information
Advantage: uses global and local feature extraction approaches (has high precision)
appearance
o products : • For the products mentioned
previously it should be modified in terms of cost-effectiveness, robustness, and user acceptability
• these commercial products are still in the initial phases.
Comparison : recapitulation
28/30
Conclusion
29/30
Thanks for
your attention
