HAND GESTURE RECOGNITION BASED ON SHAPE PARAMETERS

By:Nithinkumar Jephin JoseBhavith PSachin kumar

Guided by:Mrs. Aswini P

IntroductionHand Gesture recognition system

provides us an innovative, natural, user friendly way of interaction with the computer.

has a wide area of application including Human machine interaction, Sign Language, Immersive game technology etc.

a real time system for hand gesture recognition on the basis of detection of some meaningful shape based features like orientation, centre of mass (centroid), status of fingers and thumb in terms of raised or folded and their respective location in image.

Algorithm is implemented in Matlab v7.10

Generated bits are transmitted to microcontroller serially

We use this hand gestures for◦ Sign Language Recognition◦ Human Machine Interaction

INPUT IMAGE

PROCESSING & SEGMENTATION

ORIENTATION DETECTION

FEATURE EXTRACTION

CLASSIFICATION & BITSGENERATION

HAND GESTURE INTERPRETATION

CENTROID, THUMB,FINGER REGION DETECTION

Block Diagram of Gesture Recognition Algorithm

Converting video file into images at predefined intervals. Image might be RGB format.

More sensitive to different light conditions

Input Image

RGB Image

YCbCr Image

K-Means Algorithm YCbCr Binary Image

Image segmentation

Iteratively minimizes the sum of distances from each object to its cluster centroid until the sum cannot be decreased further.

The result is a set of clusters that are well separated from other clusters and compacted in their own cluster.Images is having uniform plain background consist of only one hand object,

Two Clusters 1. Hand object 2. Background

pixel value = 1 pixel value = 0

Segment 1Segment 2

Orientation Detection

Identify whether hand is in vertical or horizontal.

Compute length to width ratio of bounding box

ratio>1 hand position is vertical

ratio<1 hand position is horizontal.

Scan image from bottom to top or left to right and check white pixel in

the boundary

White pixel bottom hand position is vertical

White pixel left side hand position is horizontal.

ratio= thisBB(4)/thisBB(3);if ratio>1 disp('vertical image'); vorientation=1;else disp('horizontal image'); vorientation=0;end

Centroid

Calculate the CENTROID for partitioning the hand into

two halves

Finger portionNon finger region

stat = regionprops(in3,'centroid');xcent=ceil(stat.Centroid(1));ycent=ceil(stat.Centroid(2));

Feature Extraction

To detect presence/absence of thumb.

Thumb can reside at left most side or right most side of all fingers.

Crop the bounding box into two regions by taking 33% pixel width

from each side.

Green boundary left side Blue boundary right side

Count total number of white pixels in binary

image.

Count number of pixels in each box.

If less than 7% of total white pixels exist in any

of the right box or left box, we consider that

thumb is present in that box only.

Thumb Detection

Finger Detection For getting total number of fingers raised.

Generating an exact boundary on image.

Finding out the first white pixel at the bottom of the image

mm=floor(st.BoundingBox(2)+st.BoundingBox(4));col=find(in3(mm,:),1,'first');row=mm;

For getting total number of fingers raised.

Boundary is find out using a function “ bwtraceboundary”

boundary = bwtraceboundary(in3,[row, col],'N');

Bits Generation Plotting the variations in distance from bottom.

Finding out individual peaks

Determination of maximum peak.

Threshold value generation 75% of maximum peak

Threshold value

Highest peak

Elimination of insignificant peaks

Peaks above threshold “1”

Peaks below threshold “0”

Generation of 5 bit binary digit.

for example : 0 0 1 1 1

Thus generated bits represent a unique gesture

• used to recognize alphabet in Sign Language

• used as an input to control a machin

Generation of Alphabet

Alphabets are generated by comparison with data base

for ifin=1:size(FDB,1) if (sum(abs(FDB(ifin,:)-Fingers(1,:)))== 0) disp(CDB(ifin)); end end

For example : input image be

0 1 0 0 1

Sign Language Recognition

0 1 0 0 1 –0 0 0 0 10 1 0 0 0

0 + 0 + 0 + 0 + 1 = 1 Not matching

Jump to next row0 1 0 0 1 –

0 1 0 0 10 0 0 0 0 0 + 0 + 0 + 0 + 0 = 0

Matching

Generating alphabet

Graphical User Interface

Gesture Controlled Robot

RFEncoder

Generated Bits

Transmitter

Antenna

Transmitter Block Diagram

Receiver

RF Decoder

Motor Driver

ICMotors

Antenna

Receiver Block Diagram

Power Supply

Transmitter Circuit

Receiver Circuit