Hand Gesture Recognition In Real Time Using IR Sensor · Hand Gesture Recognition In Real Time Using IR Sensor Rohith H R 1, Shiva Gowtham 2, Sharath Chandra A S 3 1,2,3 R V College

Hand Gesture Recognition In Real Time Using IR Sensor

Rohith H R1, Shiva Gowtham2, Sharath

Chandra A S3

1,2,3R V College Of Engineering

Bengaluru, India

Electonics Department

April 14,15 - 2017

Abstract

Communication is a major challenge to people with

hearing loss and people who are aphonic. Due to

these disabilities there exists a communication gap

between the hearing population and those with the

disability. A wearable glove based system fitted

with IR sensors can be used in bridging this gap in

communication. The gloves are used to recognize the

hand gestures of the disabled people and convert

those gestures into meaningful messages in real

time. With the amount of bend made by the fingers

continuous data stream is obtained as output from

Infrared (IR) sensor. IR sensors change the output

voltage depending on the strength of the received

signal. Change in bend is converted to change in the

electrical voltage by IR sensors. The output from IR

sensors is processed by a microcontroller and a

corresponding message is displayed. The hardware

components used in this project are Microcontroller

(Norton), IR sensors, LCD display, switch, buzzer

and power supply. The software components are

Embedded C, Kiel U vision. The main purpose of

this device is to bridge the gap in communication

and help the people with disability to overcome the

challenge and communicate more effectively.

International Journal of Pure and Applied MathematicsVolume 114 No. 7 2017, 111-121ISSN: 1311-8080 (printed version); ISSN: 1314-3395 (on-line version)url: http://www.ijpam.euSpecial Issue ijpam.eu

111

Keywords—IR sensor ;an–machine interfaces;

Gestures recognition, wearable sensors; motion

detection.

1. Introduction

According to Albert Mehrabian's book Silent Messages, based on

extensive research, communication on a daily basis is made up of

93% non-verbal communication and only 7% verbal

communication. But realistically non-verbal communication can

vary anywhere between 60 to 90 percent. Therefore irrespective

of having a disability or not, gestures play a major role in

communication. People with hearing loss and people who are

aphonic communicate with the help of gestures called Sign

language. Sign language chiefly uses manual communication like

hand signs and gestures to mediate a message. There are many

different sign languages. Every country have their own sign

language, and few countries have more than one. People without

the disability don’t feel the need to learn the sign language until

a need arises, and hence majority of the people are not aware of

communicating through sign language. So it becomes hard to

communicate effectively with people having hearing loss. Hence

there is a need for a system which can bridge this gap in

communication. There are 2 approaches. which are used in

recognizing. and interpreting the gestures. vision .based glove

based systems.

Fig1: Flex sensor attached to glove in the hand is used to perform

Vision based systems provide more .non contact and natural

solution. to obtain information. of its surroundings. and hand

International Journal of Pure and Applied Mathematics Special Issue

112

gestures. Among many hand recognition phases the 3 main

among them are tracking, detection and recognition. Due to its

complexity, system. requires. an high level operating. algorithm.,

and hence making the design difficult. Even if the system is

feasible an enclosure in which measures are taken to provide a

environment that meets certain requirements, it lacks.

processing. speed and accuracy.. Glove based gestures

recognition systems is less complex and provides more accuracy.

Glove based systems make use of sensors. fixed to the gloves. to

capture the movement. of fingers.. This paper proposes a

wearable glove based system that can interpret the hand

gestures of people with disability and convert those gestures into

meaningful messages that can be displayed on a screen.

2. Methodology

Hand Talk [3] a glove equipped with IR sensor which translates

finger and hand movement into words or sentence. Gloves

equipped with IR sensor recognise hand movements and then

transmit data to either phone or display on an LCD or it could

either be input to a speaker. The IR sensor data are either

converted to text or to voice. Person with no knowledge about

sign language can read from the LCD display or listen from the

cell phone what the dumb or deaf person is saying in sign

language. The main advantage of Hand Talk design was its cost

effectiveness, long life and simplicity to create. interactive glove

which helps to improve life of deaf and dumb.

Fig 2: Block diagram of Hand-Talk gloves[2]


113

To start with research, once we obtain a electrical voltage

depending on hand movement. Different hand gestures are

interpreted by finger. Components used in Hand Talk system [1]

are: 1.Microcontroller 2 Motion sensor(IR Sensor )3. LCD display

4. Smart Phone 5. Speaker. Motion Sensors could be either an,

flex sensor or an IR sensor.In a Hand Talk gloves equipped with

accelerometer based on the amount of bent of the finger the

signals are interpreted into words. Where as in flex sensor

depending upon the change in resistance due to bent the signals

are interpreted. Infrared sensor [2] is a that emits to sense

things around it. IR measures heat of objects and also detect the

motion. These types of sensors are capable of measuring only

infrared .radiation,. In infrared sensor an IR LED could be used

as emitter .and IR photodiode as detector. But in IR sensor the

detector has to be sensitive to wavelength emitted by emiter

When .IR light falls on the photodiode., Resistances and output

voltages changes with change in magnitude infrared light

received. Figure 3 explains [7] the working of IR sensor with no

obstacle in front of it sender. continuously sends infrared rays

and since there is no obstacle to reflect back the infrared rays,

sensor do not detect infrared rays. When a gloves is equipped

with IR sensor the emitter is placed in the back of the finger and

detector of the sensor is placed at front or at the nails part of

gloves. When the emitter of IR sensor sends infrared radiation it

is received by the detector placed in front of gloves. In case the

person is wearing the gloves and bends the finger to

communicate with other person then the infrared radiation

emitted would not be received the detector of the IR sensor

equipped on the gloves.

Fig 3: Working of IR sensor with no obstacle in front[7]

And Figure 4 explains the working of IR sensor with obstacle in

front of it. Sender continuously sends infrared rays and since and

when there is an obstacle to reflect back the infrared rays, sensor

detects infrared rays.


114

Figure 4: Working of IR sensor with no obstacle in front[7]

In the context of this project ,when a dumb/deaf person wears

Hand Talk gloves and doesn’t bend his fingers then there would

be no obstacle to interrupt the infrared rays. But when the

person bends his finger the infrared rays would not reach the

receiver due to interference by obstacle. It is programmed in

microcontroller such a way that when the receiver receives

infrared rays it is read as binary bit 1 and when receiver do not

receive infrared rays it is read as binary bit 0. Innovati’s LCD

2x16 connections are very simple , and is used in many LCD

applications it could be controlled Innovati’s basic commander.

LCD 2x16 has 2 lines where each line can display upto 16

characters. In this LCD cursor control command is used to

display on the screen.[1]

3. Working Of System

Hand Talk gloves is worn by deaf/dumb person. The IR sensors

are fixed on each of the finger of the gloves. The IR sensors

mounted on gloves can detect hand gesture made by t person

wearing gloves. 5V supply is required for system to operate hence

not much of a problem to worry about getting a shock. And then

the IR sensors convert these movements ofand into electrical

signal. A ADC is used which converts analog to digital signal.

The voltage for every movement of hand is recorded and is

programmed in such a way in microcontroller as a predefined

value. The IR sensor is given as input to microcontroller(Norton)

to compare to predefined value. After the comparison by

microcontroller, the voltage difference measured obtained would

be within a fixed range as programmed and dumped in the

microcontroller.


115

Fig 5:Working and flowchart of Hand-Talk gloves with IR sensor as switch[2]

And in the microcontroller using Arduino software it would be

programmed in such a way that for every voltage there would be

a sentence that is to be displayed on LCD.

Microcontroller(Norton) compares the voltage produced due to

hand movement with predefined voltage and displays the

sentence in that voltage range.

4. Output

Software protous simulation is used LCD and keypad

interfacing. When a finger is bent by a deaf/ dumb or a normal

person then corresponding word stored in microcontroller for

communication would be displayed in LCD so that the

deaf/dumb people can convey their messages to others.

Fig 6:LCD with keypad interfacing[5]

The Compilation for the above design using keil software is

shown in Figure 7


116

Fig 7:Compilation output of LCD With Keypad Interfacing using keil software

In figure 8 it shows that for various bends of the IR sensor the

various voltage will be produced which is measured using

oscilloscope and given as input data for microcontroller.

Figure 8: IR sensor output analysis for various bends

If a deaf/dumb or normal person, wants to communicate some

message with the normal person he/she has to bend a finger so

that corresponding word or sentence would be displayed on LCD

screen and the message will be conveyed to normal person when

he reads it from LCD. From the Figure 9 we can see that when a

finger is bent corresponding word or sentence stored in

microcontroller ie “I Need Doctor” is displayed on LCD display.

This output normal person can read the message in LCD.


117

Fig 9: LCD display displaying output based on finger bent

The entire system of Handtalk gloves equipped with IR sensor.

And IR sensor is connected to microcontroller which is

programmed using the Arduino software to display words or

sentence on the LCD display. Te whole system is shown in

Figure 10.

Fig 10 : Complete model of Hand Talk gloves equipped with

IR sensor

5. Applications

This Hand Talk gloves system [2] is cheap, cost efficient and

portable. The system uses simple techniques .It helps deaf and

dumb people in public sectors, working areas, marking areas for

communicating with others[5]. This gloves plays major role in

various fields such as Biometrics, Robotics, Musical instrument

by replacing physical buttons with hand gestures, Automatic

control in industries. It can also be used for computer gaming as

a simulator. It is also loaded with touch sensors on the thumb,

the fingers and the palm which allows the wearer to use the


118

keyboard. Another application can be found in the field of fire

extinguishing by fire extinguishers when members of a team

can’t even see each other person, these gloves will be able to

transmit signals through simple hand gestures. The Hand Talk

gloves have symbols and icons on them that light up according to

the signal received.

6. Conclusion

Dumb and deaf person rely majorly on interpreters and sign

language for communication. However, in everyday life

dumb/deaf people find it difficult in finding interpreters for

communication as they are not available everywhere, also high

costs and also scheduling qualified interpreters. Also this gloves

would help improving quality of life of dumb or deaf significantly.

The data obtained from glove equipped with IR sensor was used

to extract word for every ASL. System provides an identity and

moral support to dumb and deaf people. It can be extended to

display any number of words, which requires further

modification of the gloves. The Hand Talk will be further

extended by developing a smart phone app and speech

recognition(Voice apr9600) IC which would enable reliable

communication between Hard of Hearing and the Visually

Impaired people.

7. Acknowledgement

Our sincere thanks to our guide Mr Kiran V and Mrs Rohini S

Hallikar (Assistant Professor) Department of Electronics and

Communication for all the help and support the project“hand

gesture recognition system using IR sensor” from January 2017

to April 2017.

8. Reference

[1] K Anetha, Rejina, Parvin J, PG Scholar, Dept. of Embedded

Systems, Dr N G.P.,Institute, of Technology, Coimbatore, India,

International Journal of Innovative Research in Computer and

Comunication Engineering, (An ISO : 3297; 2007 Certified

Organization) Vol no 2, Special Issue : 3, July 2014

[2] A.A.Markad , A.L.Borkar, Student, Asst Professor, .Department of

Electronics and, Telecommunication, M S S C E T., Jalna ,

Maharashtra,, International journal of Electrical and Electronic

Engineers,Volume. No 8, Issue 01., Janaury-June 2016


119

[3] Ajinkya Raut, ,Vineeta Singh, ,Vikrant Rajput,,Ruchika Mahale

PES MCOE, PUNE, The International Journal of ,Engineering

and Science Volume No 1 ,,Issue 2,, Pages 19-25., 2012,.ISSN:

2319 – 1813,, ISBN: 2319 – 1805

[4] Angelo M. Sabatini ,LauraDipietro,. Senior Member at IEEE,

and Paol Dario., Fellow., IEEE transactions on systems.,man and

cybernetics , part c:.applications .and reviews.,volume No .38,N)

4,July,2008

[5] J. Thilagavathy Assistant professor, A. Jeyapaul Murugan ,

Department of ECE Dr.Sivanthi Aditanar College of

Engineering, International. journal Of engineering research and

technology., ISSN:. 2378-0171, Volume No: 03 Issue 2, March :

2016

[6] J. Segen and S. Kumar, “Human-computer Interaction using

gesture recognition and 3D hand tracking,” Proc. of ICIP’98, vol.

3, Chicago,Oct.1998, pp. 188-192

[7]D. Kim, S. Lee, and J. Paik, “Active shape model-based gait

recognition using infrared images,”.International Journal of

Signal Processing, Image Processing and Pattern Recognition,

vol. 2, no.4, Dec..2009

[8]Z.Zhang., Z.Ren. and .J. Meng, “ROBUST part-:Based Hand

Gesture Recognition. Using. Kinect. Sensor.,” in IEEE.

Transactions. on multimedia, vol 15., no.15., pp/.1110-1120,

August. 2013.

[9] Kalpattu S Abhishek,Physics Department and material

science,Hong Kong uni,Hong. Kong., 2016 IEEE

InternationalConference. on Electron. Devices. and Solid.

StateCircuits.

[10]M. Elmahgiubi, M. Ennajar, N. Drawil, M. S. Elbuni ,"Sign.

Language Translator. and Gesture. Recognition",computer and

Information. technology 201 Global. Summit. on Sousse,1-6, 2016


120

121

122

Documents

Hand Gesture Recognition In Real Time Using IR Sensor · Hand Gesture Recognition In Real Time Using IR Sensor Rohith H R 1, Shiva Gowtham 2, Sharath Chandra A S 3 1,2,3 R V College