MEMS ACCELEROMETER BASED NONSPECIFIC – USER HAND GESTURE RECOGNITION

WELCOME

MEMS ACCELEROMETER BASED

NONSPECIFIC – USER HAND GESTURE RECOGNITION

GRACE ABRAHAM

ROLL . NO : 25

S7 ECE

SNMIMT.

26 /7/ 2013 Dept. of ECE 2

CONTENTS• INTRODUCTION

• GESTURE MOTION ANALYSIS

• SENSING SYSTEM

• SYSTEM WORK FLOW

• GESTURE SEGMENTATION

• MODEL 1 : BASED ON SIGN SEQUENCE AND HOPFIELD NETWORK.

• MODEL 2 : BASED ON VELOCITY INCREMENT

• MODEL 3 : BASED ON SIGN SEQUENCE AND TEMPLATE MATCHING.

• EXPERIMENTAL RESULTS

• ADVANTAGES

• APPLICATIONS

• CONCLUSION

26/7/2013 Dept. of ECE 3

MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

INTRODUCTION• Human – Machine Interactions

• Physical Gestures

• Gesture Recognition

7 Hand Gestures MEMS Accelerometer A Micro Inertial Measurement Unit ( IMU)

2 Methods

Approaches

o Vision – Based ( Limitation ) o Accelerometer Based


o Template Matchingo Statistical Matching

26/7/2013 Dept. of ECE 4

• 3 Gesture Recognition Models

• 7 Gestures

• Inputted to MEMS 3 – Accelerometer

• Gesture Segmentation Algorithm

• 100’s of data to 8 number code

• Gesture Recognition

Sign Sequence & Hopfield Based

Velocity Increment Based

Sign Sequence & Template Matching Based

Up, down, left, right, tick, circle, cross

26/7/2013 Dept. of ECE 5


BLOCK DIAGRAM

SENSING CHIPDATA

PROCESSINGGESTURE

SEGMENTATIONGESTURE

RECOGNITION

GESTURE INPUT

RECOGNISED GESTUREOUTPUT

26/7/2013 Dept. of ECE 6


GESTURE MOTION ANALYSIS

26/7/2013 Dept. of ECE 7

Fig 1 : Coordinate System

Fig 2 : Gesture up motion decomposition

• Motion in vertical plane ( x – z plane)

• Accelerations on x-z plane

• Up Gesture

Up Gesture

Circle Gesture

o X axis : no accelerationo Z axis : negative – positive – negative

o X axis : positive – negative – positive o Z axis : negative – positive – negative - positive

Velocity zero at pt. 1 & 2 Sign changes at pt. 3 & 4


Fig 3 : Predicted velocity and acceleration in the z-axis

Fig 4 : Real acceleration plot

• One Axis – up & down, left & right

• Two Axis – tick, circle, cross (complex)

• Acceleration changes in z axis

• Real acceleration is the same with theprediction

• Unique acceleration pattern

1 to 3 :-ve; V changes from 0 to max. at3 3 to 4 :+ve; V changes from -ve to +ve& max. at pt 4 4 to 1 :-ve; V changes from +ve to zero

26/7/2013 Dept. of ECE 8


SENSING SYSTEM

Fig 5 : Sensing System

• MEMS 3 – axes acceleration sensing chip

• Data management chip

• Bluetooth Wireless data chip

26/7/2013 Dept. of ECE 9


• MEMS ???

• ACCELEROMETER ???

Micro Electro-Mechanical Systems

Combination of mechanical functions & electronical functions on same chip

Micro fabrication technology

Electromechanical device

Measure acceleration forces

26/7/2013 Dept. of ECE 10


26/7/2013 Dept. of ECE 11

SYSTEM WORK FLOWMEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

Fig 6 : Motions of seven gestures

GESTURE SEGMENTATION

• DATA ACQUISITION

Horizontally place sensing device

Time interval not less than 0.2sec

Perform Gestures as shown in Fig 6

26/7/2013 Dept. of ECE 12


• GESTURE SEGMENTATION

DATA PREPROCESSING

2 Processes

o Remove vertical axis offsets by subtracting data points from mean value

o Filter to eliminate noise

26/7/2013 Dept. of ECE 13


26/7/2013 Dept. of ECE 14

Fig. 3.2. Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-circle-cross.Fig. 3.2. Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-circle-cross.Fig. 3.2. Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-circle-cross.Fig. 3.2. Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-circle-cross.

Fig 7 : Segmentation of a seven-gesture sequence in the order up-down-left-right-tick-

circle-cross.

SEGMENTATION

o Find terminal points

o We need

o 2 x n matrices generated

o Compare max. acceleration b/w terminal points with its mean value

o No. of columns = No. of gestures

Amplitude of points

Point separation

Mean value

Distance from nearest intersection

Sign variation b/w 2 successive points


MODEL ONE : GESTURE RECOGNITION BASED ON SIGN

SEQUENCE AND HOPFIELD NETWORK

26/7/2013 Dept. of ECE 15


Fig. 4.1 . Sign sequence generationFig. 4.1 . Sign sequence generationFig. 4.1 . Sign sequence generationFig. 4.1 . Sign sequence generation

Fig 8 : Sign sequence generation

GESTURE RECOGNITION

• FEATURE EXTRACTION

Examine the sign of the first mean point

of a gesture

Store in gesture code

Detect no. of sign changes

Store the alternate signs in sequence in

the gesture code

Code for the gesture in fig 8 is 1, -1, 1, -1

26/7/2013 Dept. of ECE 16


• GESTURE ENCODING

Max. no. of signs for 1 gesture on 1 axis is 4

Eight numbers in one gesture code

Hopfield network can take only 1 & -1 as inputs

+ve, -ve sign and zero are encoded as “1 1”, “-1 -1” and “1 -1”

Each gesture has a unique 16 - number code

26/7/2013 Dept. of ECE 17


• HOPFIELD NETWORK AS ASSOCIATIVE MEMORY

Recovery mechanism

Weight matrix is constructed

sp - Pattern to be storedP - Number of patterns I - Identity matrix

npTPp

p

P sPIssw 1,1,)(1

qsv )0(

qqTpp

p

p Psssswvu

)()0()1(

1

1

)1()( nwvnu))(sgn()1( nuv

))(sgn()( nunv

,

outputnv )(

26/7/2013 Dept. of ECE 18


• GESTURE COMPARISON

Gesture code is compared with the standard gesture codes

Difference b/w the two codes is calculated

Smallest difference indicates the most likely gesture

26/7/2013 Dept. of ECE 19


Table 1: Standard patterns for the seven gestures

26/7/2013 Dept. of ECE 20


MODEL TWO: GESTURE RECOGNITION BASED ON VELOCITY INCREMENT

• Model deals with complex gestures

• Area bounded by acceleration curve & x axis

• Partitioned areas with alternate signs

• Normalization of area sequence,

- Normalized area

- Original area, - Max. area

maxA

AA

original

norm

Increase/decrease in velocity

normA

originalAmaxA

26/7/2013 Dept. of ECE 21


Fig. 9 : Acceleration partition

• To avoid misalignment due to noise

• Compare velocity increment

– Two area sequences compared– Comparison result

• Gesture with min. Value recognized

Imagining curve has mass Obtain center of mass Two curves are aligned to coincide their centers of masses

Subtracting 2 area sequence vectors

nnd

nn

nn

AAAAAAA

AAAAAS

AAAAAS

'

2

'

21

'

1

''

1

'

3

'

2

'

12

1........3211

.....

......

,,

21, ss

dA

25/7/2013 Dept. of ECE 22


25/7/2013 Dept. of ECE 23

WORK FLOW CHART MEMS Accelerometer Based Nonspecific – User Hand Gesture Recognition

MODEL 3:GESTURE RECOGNITION BASED ON SIGN SEQUENCE &

TEMPLATE MATCHING

• Similar to model one

• No encoding of sign sequence in to combinations of -1s & 1s

• Not limited to specific users

Table 2: Gesture codes for model 3

25/7/2013 Dept. of ECE 24


EXPERIMENTAL RESULTS

• ACCURACY Model III > I > II

• PERFORMANCEModel III > I > II

• Model III has an overall mean accuracy of 95.6%

Table 7.1.Comparison of gesture recognition accuracy(%) of three models

Table 3 : Comparison of gesture recognition accuracy(%) of 3 models

25/7/2013 Dept. of ECE 25


ADVANTAGES

User friendly

Gesture patterns are not critical

Noise filtering is not required

User doesn’t require any advanced training

Low power, compact and robust sensing

25/7/2013 Dept. of ECE 26


APPLICATIONS

Character recognition in 3-D space

To control a tv set

Virtual keyboard

Immersive game technology

For socially assistive robotics

25/7/2013 Dept. of ECE 27


CONCLUSION

Sensor data collection, segmentation & recognition

Sign sequence of gesture is extracted

100’s of data to code of 8 numbers

Code compared with standard patterns

25/7/2013 Dept. of ECE 28


REFERENCES• WEBSITES

ieeexplore.ieee.org/ www.analog.com

MEMS Accelerometer Based Nonspecific-User Hand Gesture Recognition , IEEE SENSORS JOURNAL, VOL. 12, NO. 5, MAY 2012.

S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, “Hand-written character recognition using MEMS motion sensing technology,” in Proc.IEEE/ASME Int. Conf. Advanced Intelligent Mechatronics, 2008

• BOOKS REFERED

25/7/2013 Dept. of ECE 29


25/7/2013 Dept. of ECE 30

THANK YOU


25/7/2013 Dept. of ECE 31

QUERIES ????.....


Engineering

MEMS ACCELEROMETER BASED NONSPECIFIC – USER HAND GESTURE RECOGNITION