MICRO ELECTRICAL MECHANICAL SYSTEM ACCELEROMETER BASED NONSPECIFIC-USER HAND GESTURE RECOGNITION

8/12/2019 MICRO ELECTRICAL MECHANICAL SYSTEM ACCELEROMETER BASED NONSPECIFIC-USER HAND GESTURE RECOGNITION

1/37

MICRO ELECTRICAL MECHANICAL SYSTEM

ACCELEROMETER BASED NONSPECIFIC-USER

HAND GESTURE RECOGNITION

PRESENTED BY

APARNA C BHADRAN

S7 CSE

B.Tech

1


2/37

INDEX

1. INTRODUCTION

2. GESTURE MOTION ANALYSIS

3. SYSTEM SENSING OVERVIEW

4. GESTURE SEGMENTATION

5. GESTURE RECOGNITION BASED ON SIGN

SEQUENCE AND HOPFIELD NETWORK

6. GESTURE RECOGNITION BASED ON

VELOCITY INCREMENT

2


3/37

7.GESTURE RECOGNITION BASED ON

SIGN SEQUENCE AND TEMPLATE

MATCHING

8.EXPERIMENTAL RESULTS

9.CONCLUSION

10.REFERENCES

3


4/37

INTRODUCTION

The increase in human-machine interactions

has made user interface technology more

important.

MEMS-Micro Electrical Mechanical System.

Physical gestures will1. greatly ease the interaction process.

2. enable humans to more naturally command computers

or machines.

4


5/37

Examples:

telerobotics

character-recognition controlling a television set remotely

enabling a hand as a 3-D mouse .

Many existing devices can capture gesturessuch asjoystick

trackball

touch tablet.

5


6/37

The technology employed for capturinggestures can be

Relatively expensive.

So Micro Inertial Measurement Unit is used.

Two types of gesture recognition methods

Vision-based Accelerometer.

Due to the limitations of vision based such as unexpected optical noise,

slower dynamic response

6


7/37

Recognition system is implemented based on

MEMS acceleration sensors.

The acceleration patterns are not mapped into

-velocity-displacement

not transformed into frequency domain.

But are recognized in time domain.

7


8/37

There are three different gesture recognition

models:1) sign sequence and Hopfield based gesture recognition

model

2) velocity increment based gesture recognition model

3) sign sequence and template matching based gesture

recognition model

8


9/37

GESTURE MOTION ANALYSIS

Gesture motions are in the vertical plane i.e x-z plane.

The alternate sign changes of acceleration onthe two axes are required to differentiate any

one of the 7 gestures: up,

down,

left,

right,

tick,

circle,

cross

9


10/37

the gesture uphas

the acceleration on z-axis in the order:

negativepositivenegative

no acceleration on x-axis

10


11/37

1 3: acceleration on z-axis is negative

velocity changes from zero to a maximum

value at 3

acceleration at point 3 is zero.

11


12/37

3 4: acceleration on z-axis is positive;

velocity changes from negative to positive and

is maximum at point 4, where acceleration

becomes zero.

4 1: acceleration on z-axis is negative;

velocity changes from positive to zero.

acceleration and velocity become zero at point

1

12


13/37

13


14/37

SENSING SYSTEM OVERVIEW

1. Sensor Description

The sensing system utilized for hand motion

data collection :

14


15/37

2. SystemWork Flow

1. The system is switched on

2. The accelerations in three perpendiculardirections are detected by the MEMS sensors.

3. Transmitted to a PC via Bluetooth protocol.

4. The data is passed through a segmentationprogram

5. the processed data are recognized by a

comparison program to determine the presented

gestures.

15


16/37

GESTURE SEGMENTATION

A. Data Acquisition

The sensing devices should be held

horizontally during the whole data collection

process.

The time interval between two gestures should

be no less than 0.2 seconds

16


17/37

The gestures should be performed as

17


18/37

B. Gesture Segmentation

1. Data Preprocessing:

Raw data received from the sensors are

preprocessed by two 2 processes:

a) vertical axis offsets are removed in the

time-sequenced data

b) a filter is applied to the data sets to

eliminate high-frequency noise data

18


19/37

2.Segmentation:

The purpose is to find the terminal points of

each gesture in a data set .

The conditions of determining the gesture

terminal points are

a) amplitude of the points

b) point separation

c) mean value

d) distance from the nearest intersection

e) sign variation between two successive points.

19


20/37

all these five conditions are checked separately on x- and z-

axes acceleration data.

two matrices are generated for each of gesture sequence data

20


21/37

GESTURE RECOGNITION BASED ON SIGN

SEQUENCE

AND HOPFIELD NETWORK

1) Feature Extraction:

The gestures which motions on one axis areseparated from those which involve 2D

motions.

21


22/37

22


23/37

The gesture code is 1,-1,1,-1

2) Gesture Encoding:

Before recognition the obtained gesture code should be encoded

it can be restored later by Hopfield network.

The maximum number of signs for one gesture onone axis is four.

so if the x and z axes sign sequences arecombined, there will be totally eight numbers in

one gesture code. the input for Hopfield network can only be 1 or

-1.

23


24/37

we encoded the positive sign, negative sign

and zero using the following rules:

1 1 represents positive sign;

- 1 -1 represents negative sign;

1 -1 represents zero.

24


25/37

25


26/37

3) Hopfield Network as Associative Memory:

The involvement of Hopfield network has

made more fault tolerant.

The network can retrieve the patterns that has

been stored previously.

26


27/37

4) Gesture Comparison:

After gesture code restoration, each gesture

code is compared with the standard gesture

codes.

The comparison is made by calculating the

difference between the two codes

Smallest difference indicates the most likely

gesture.

27


28/37

GESTURE RECOGNITION BASED

ON VELOCITY INCREMENT

The acceleration of a gesture on one axis is

partitioned firstly according to the signs.

28


29/37

Due to the intensity variance of each gesture,

an area sequence should be normalized

29


30/37

after normalization: the area sequences are not compared immediately

They are processed by using an algorithm analogous to

center of mass.

The final step

- to compare the velocity increment sequence

The gesture, which has the minimum value

can be recognized.

30


31/37

GESTURE RECOGNITION BASED ON SIGN

SEQUENCE AND TEMPLATE MATCHING

The recognition algorithm of this model is

very similar to that of model one except that no Hopfield network is used

All the sign sequences are represented by - 1,

1 and 0

31


32/37

32


33/37

EXPERIMENTAL RESULTS

33


34/37

Model III has the highest accuracy among the

three models, while the performance of Model

II is the worst of the three. The test results shown in Table III are based on

72 test samples.

34


35/37

CONCLUSION

To enhance the performance : improve the segmentation algorithm .

Moreover, other features of the motion data

may be utilized for pattern classification in

future work.

35


36/37

REFERENCES

[1] T. H. Speeter, Transformation human hand motion for telemanipulation,Presence, vol. 1, no. 1, pp. 6379, 1992.

[2] S. Zhou, Z. Dong, W. J. Li, and C. P. Kwong, Hand-written char-

acter recognition using MEMS motion sensing technology, inProc.IEEE/ASMEInt.Conf. Advanced Intelligent Mechatronics, 2008, pp.

14181423.

[3] J. K. Oh, S. J. Cho, and W. C. Bang et al., Inertial sensor based recognition of3-D character gestures with an ensemble of classifiers, pre-

sented at the 9th Int. Workshop on Frontiers in Handwriting RecogniTion,2004.

[4] W. T. Freeman and C. D. Weissman, TV control by hand gestures,

presented at the IEEE Int. Workshop on Automatic Face and Gesture

Recognition, Zurich, Switzerland, 1995.

36


37/37

THANK YOU

37

Documents

MICRO ELECTRICAL MECHANICAL SYSTEM ACCELEROMETER BASED NONSPECIFIC-USER HAND GESTURE RECOGNITION