8/7/2019 Vision Based Gesture in Embedded Devices

1/20

COMPUTERVISION & GESTURE

RECOGNITION

-- Mukesh Kumar

8/7/2019 Vision Based Gesture in Embedded Devices

2/20

Introduction

Input devices

How System Works

Computer Vision Frameworks

OpenCV

Challenges

Few Apps Idead

Demo

Conclusions

OUTLINE

8/7/2019 Vision Based Gesture in Embedded Devices

3/20

1. INTRODUCTION

Computer vision is the science where machine

is able to extract information from an image that

is necessary to solve some task. It enables

computer to understand visual input

The image data can be acquired in many forms

and from many kind of sources, such as video

sequences, views from multiple cameras, or

multi-dimensional data from a medical scanner,

video, normal 2D cameras, infrared cameras,

radars or specialized sensors.

8/7/2019 Vision Based Gesture in Embedded Devices

4/20

GESTURE RECOGNITION

Gesture recognition is a language

technology with the goal of interpreting

human gestures via mathematical algorithms

Gesture recognition can be conducted with

techniques from computer vision and image

processing.

8/7/2019 Vision Based Gesture in Embedded Devices

5/20

8/7/2019 Vision Based Gesture in Embedded Devices

6/20

Two common technologies for hand gesture

recognition

y glove-based method

Using special glove-based device to extract hand posturey vision-based method

3D Depth data Model

Normal Color Camera / Appearance Model

8/7/2019 Vision Based Gesture in Embedded Devices

7/20

Data Glove Based Gesture Recognition

8/7/2019 Vision Based Gesture in Embedded Devices

8/20

VISION-BASED GESTURE RECOGNITION

8/7/2019 Vision Based Gesture in Embedded Devices

9/20

HOW IT WORKS

Recognition + Gesture recognition

The classical problem in computer vision is that of determining whether or not theimage data contains some specific object, feature, activity, gesture, optical character,Motion e.t.c.

This can be achieved using following steps :

Image Acquisition :Image Acquisition is done to generate a 2D image, 3D depth data, Imagesequence from video feeds at real time or from any other sources usingspecific hardware.

Pre processing of ImagesNoise reduction, Contrast Enhancements,Gray scale conversion, Matrix of

Image creation, Histogram comparison, Image color Inversion.Few of these steps can also be done during Image Acquisition phase insidethe cameras.

Feature ExtractionLine,Edge, Corner Detection

Blobs and shape detection and extraction

Color recognition, like skin color

8/7/2019 Vision Based Gesture in Embedded Devices

10/20

High Level ProcessingHigh Level Processing is done on the extracted features from the image for Gesture, pose,

Motion detection.

Pixel Classification

ImageCorrelation

Facial Recognition

Hand detectionFeature Tracking

In this phase Neural Network andArtificial intelligence is used to teach the system about the

gesture, kind of motions, faces.Adaptive algorithms are used to adapt the system for working under

different kind of scenarios and environments which learn itself and become more intelligent system after

learning from the various kind of inputs.

8/7/2019 Vision Based Gesture in Embedded Devices

11/20

8/7/2019 Vision Based Gesture in Embedded Devices

12/20

Training samples

Training samplesy Negative samples: images that must

not contain object representations. We

collected 500 random images as negative

samples.

y Positive samples: hand posture images

that are collected from humans hand, orgenerated with a 3D hand model. For each

posture, we collected around 450 positive

samples. As the initial test, we use the white

wall as the background.

8/7/2019 Vision Based Gesture in Embedded Devices

13/20

AVAILABLE FRAMEWORKS

Few Computer Vision Frameworks available:

OpenNI Works with Microsoft Kinect and other 3D sensors generating

3D depth Images

OpenCV (started by Intel) Have a C/C++ interface for complex Image

Processing and Applying

Neural network Algorithm. Works with 2D or 3Dimage samples or live video feeds.

Aforge.Net Same as OpneCV, with a .NET interface. Have complex

Image Processing and Neural network Algorithm, genetic algorithms and

machine learning libraries for Applications. Subset of this project has a

Glyph recognition system.

NokiaCV Library

EmbedCV - An Embeddable Computer Vision Library

The OpenSURF Computer Vision Library

8/7/2019 Vision Based Gesture in Embedded Devices

14/20

OPENCVINTEL OPENSOURCECOMPUTER VISIONLIBRARY

OpenCVis a library of programming functions mainly aimed at real

time computer vision, developed by Intel.

y It is free for use under the open source BSD license.

y The library is cross-platform.

8/7/2019 Vision Based Gesture in Embedded Devices

15/20

8/7/2019 Vision Based Gesture in Embedded Devices

16/20

MODULES

OpenCVOpenCV FunctionalityFunctionalityyy more than 350 algorithmsmore than 350 algorithms

8/7/2019 Vision Based Gesture in Embedded Devices

17/20

8/7/2019 Vision Based Gesture in Embedded Devices

18/20

CHALLENGES

Accuracy of gesture recognition software.

Image noise - not necessary be under consistent

lighting, or in the same location. Items in the

background or distinct features of the users maymake recognition more difficult.

Hardware Requirement like 3D depth Mapper.

Processing Power required for real time Vision.

8/7/2019 Vision Based Gesture in Embedded Devices

19/20

EyeSight's hand-waving, gesture-based UIy a simple hand gesture over the phone will silence incoming calls while in a

meeting, scroll between photos in the photo gallery, skip tracks in the media

player

8/7/2019 Vision Based Gesture in Embedded Devices

20/20

FEWAPPS IDEAS