62
1 CHAPTER 1 INTRODUCTION

CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

  • Upload
    others

  • View
    6

  • Download
    0

Embed Size (px)

Citation preview

Page 1: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

1

CHAPTER 1

INTRODUCTION

Page 2: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

2

1. INTRODUCTION

Nowadays, robots are used in the different domains ranging from search and

rescue in the dangerous environments to the interactive entertainments. The more

the robots are employed in our daily life, the more a natural communication with

the robot is required. Since the introduction of the most common input computer

devices not a lot have changed. This is probably because the existing devices are

adequate. It is also now that computers have been so tightly integrated with

everyday life, that new applications and hardware are constantly introduced. The

means of communicating with computers at the moment are limited to keyboards,

mice, light pen, trackball, keypads etc. These devices have grown to be familiar but

inherently limit the speed and naturalness with which we interact with the

computer. On the other hand, hand gesture, as a natural interface means, has been

attracting so much attention for interactive communication with robots in the

recent years.

Vision based interfaces are feasible and at the present moment the computer

is able to “see”. Hence users are allowed for richer and user-friendlier man-

machine interaction. This can lead to new interfaces that will allow the deployment

of new commands. Plenty of time will be saved as well. Recently, there has been a

surge in interest in recognizing human hand gestures. In this context, vision based

hand detection and tracking techniques are used to provide an efficient real time

interface with the robot. However, the problem of visual hand recognition and

tracking is quite challenging.

Page 3: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

3

Many early approaches used position markers or colored gloves to make the

problem of hand recognition easier, but due to their inconvenience, they cannot be

considered as a natural interface for the robot control. The recent vision based

approaches do not need any extra hardware except a camera.

Hand-gesture recognition has various applications like computer games,

machinery control, wheelchair and thorough mouse replacement. One of the most

structured sets of gestures belongs to sign language. In sign language, each gesture

has an assigned meaning .Computer recognition of hand gestures may provide a

more natural-computer interface. Hand gestures can be classified in two categories:

static and dynamic. A static gesture is a particular hand configuration and pose,

represented by a single image. A dynamic gesture is a moving gesture, represented

by a sequence of images. We will focus on the recognition of static images.

Interactive applications pose particular challenges. The response time should be

very fast. The user should sense no appreciable delay between when he or she

makes a gesture motion and when the computer responds. The computer vision

algorithms should be reliable and work for different people. The main purpose of

using gestures is that it provides a more natural way of controlling and provides a

rich and intuitive form of interaction with robots. This mainly involves Image

Processing and Machine Learning for the system or application development.

Beyond this, it also requires some kind of hardware for interfacing with the system

for gesture control.

Page 4: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

4

CHAPTER 2

EXISTING SYSTEMS

Page 5: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

5

2. EXISTING SYSTEMS

Many systems exist that are used for controlling the robot through gestures.

Some gesture recognition systems involve, adaptive color segmentation, hand

finding and labeling with blocking, morphological filtering, and then gesture

actions are found by template matching and skeletonizing. This does not provide

dynamicity for the gesture inputs due to template matching. Another system uses

machine interface device to provide real-time gestures to the robot. Analog flex

sensors are used on the hand glove to measure the finger bending, also hand

position and orientation are measured by ultrasonic for gesture recognition. And in

another approach, gestures are recognized using Microsoft Xbox 360 Kinect(C).

Kinect gathers the color and depth information using an RGB and Infra-Red

camera respectively. This system though is not very cost effective.

Page 6: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

6

CHAPTER 3

PROPOSED SYSTEM

Page 7: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

7

3. PROPOSED SYSTEM

The proposed system, using which the user can navigate the wireless robot

in the environment using various gestures commands. The main objective is to

provide reliable and a more natural technique for the user to navigate a wireless

robot in the environment using gestures. In this system, user operates the robot

from a control station that can be a laptop or a PC with a good quality in-built

webcam or external webcam. This webcam is used to capture real time video

stream of hand gestures to generate commands for the robot. Gesture commands

are given using hand palm. Irrespective of the gesture technique used, robot is

moved in all possible directions in the environment using four possible types of

commands which are Forward, Backward, Right and Left. Image frame is taken as

an input and processed using Image Processing techniques. Processed image is

then used to extract the gesture command. This gesture command can have one of

the four possible commands.. Generated signal is stored in the file at the control

station. Zigbee on the robot accesses this file to transmit the signals from the

control station to the robot. As soon as the Zigbee gets command from the control

station, it is passed to the PIC microcontroller. PIC takes this signal as input from

the Zigbee and generates some output signals that are passed to the motor driver.

This output signal generation depends on the gesture input, for every four possible

gesture input, different output signal is generated. The motor driver is used to drive

the DC motors of the robot. It takes digital signals as the input from the PIC and

gives these signals as an output to the DC motors. Once a command signal is given

to the robot, it continues to move in that direction till the next command is given or

any obstacle comes in the path.

Page 8: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

8

3.1 BLOCK DIAGRAM

3.1.1 TRANSMITTER SECTION

Fig 3.1 Block diagram of transmitter section

WEBCAM PC ZIGBEE

TRANSMITTER

POWER

SUPPLY

Page 9: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

9

Webcam:

Webcam is an image capturing device. The webcam is used to capture an

image from a moving frame of live images. Only when an image is captured it can

be processed to generate a suitable command signal.

Zigbee transmitter:

Zigbee transmitter modulates the encoded data with carrier frequency of

12.4 GHz by using Frequency Shift Keying modulation. After the modulation the

encoded signal is transmitted through the antenna to receiver section Zigbee.

PC:

The webcam is connected to PC where the image is processed using the

MATLAB software. PC is also used for giving keyboard inputs and to view the

feature extraction values.

Power supply:

This block consists of a transformer, rectifier, filter, regulator IC, load. This

is used to supply power to the Zigbee module.

Page 10: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

10

3.1.2 RECEIVER SECTION

Fig 3.2 Block diagram of receiver section

ZIGBEE RECEIVER

DECODER PIC

MICROCONTROLLER

RELAY DRIVER

DC MOTOR

Page 11: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

11

Zigbee receiver:

The receiver Zigbee receives the serial data from the transmitter Zigbee.

Decoder:

In the decoder the encoded signal is decoded into original signal as per

transmitted. It converts the serial data into the parallel and given to the PIC

microcontroller.

PIC microcontroller:

PIC microcontroller is used for controlling the motor driver based on the

received command signal. It controls the relay driver of the dc motor to drive the

robot in specified direction.

Relay driver:

Driver is used to increase the current level of micro controller output signal

for the motor. Relay is an electronic switch which is used to give the corresponding

supply either positive or negative to the motor.

Dc motor:

A linear dc motor directly produces force and motion in a straight line. So

they are used to drive the robot.

Page 12: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

12

3.2 CIRCUIT DIAGRAM

3.2.1 RECEIVER

Fig 3.3.Circuit diagram of receiver section

Page 13: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

13

The PIC microcontroller is supplied with 5V from the power supply circuit

given to the pins 11 and 32. Ground is connected to pins 31 and 12. The reset

circuit is connected to the pin 1 of the PIC controller.

LCD is connected to the PIC controller at port D, a bidirectional i/o port

having 8 pins. The 8 data bits pins(7 to 14) of LCD are connected to port D.

Contrast adjustments are made by connecting variable resisters to pin 3 of LCD.

Supply and ground are given to the pins 1 and 2 respectively.

Port B is a bidirectional port of 8 pins which are used as interrupt pins. This

can be software programmed for internal weak pull ups for all the inputs. The relay

circuit which is built up with a dc motor connected to LM298 IC is connected to

this port at pins 38 and 39. The pins 2 and 7 of the IC are connected to PIC

controller. The dc motor is connected to pins 3 and 5 of LM298 IC.

Port C of the PIC is connected to the Zigbee module. The pin 3 in Zigbee is

connected to the pin 25 which is the USART asynchronous transmit or

synchronous clock pin of the bidirectional port C of PIC. The Zigbee is powered at

pin 1 with 3.3V and grounded at pin 10.

Page 14: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

14

CHAPTER 4

HARDWARE AND SOFTWARE DESCRIPTION

Page 15: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

15

4.1 HARDWARE DESCRIPTION

4.1.1 WEBCAM

Webcam is an image capturing device. The webcam is used to capture an

image from a moving frame of live images. Only when an image is captured it can

be processed to generate a suitable command signal.

4.1.2 MICROCONTROLLER (PIC Controller)

When PIC gets command signal from the Zigbee, it is having HTTP headers

sent by web page with a tagged signal. Signal is read character by character and

appended in the string. Every time after appending the character, PIC checks for

the tagged word in the string. For every iteration it checks the substring of tagged

word at the end. If ‘sig’ is the tagged word in the signal, then program check for

substring sig at the end of a string in each iteration loop. As it gets a tagged word at

the end of the string, it terminates the loop for reading the signal character by

character. Then it reads only the next character which is an actual command signal

generated by the gestures. Depending on this command, signal is sent to L293D

motor driver through the digital pins of the PIC. Four digital pins of the PIC is set

as input to the L293D PIC, two pins on both sides. It has four possible methods as

forward(), backward(), right(), left(). Depending on the command signal, a

particular method is called for every iteration. Each method is defined with a

specified command to make each digital pin HIGH or LOW.

4.1.3 ZIGBEE CC2500

Zigbee is a wireless transmission device. Thus it works as a RF transmitter.

The command signals generated from the image processing software has to be sent

to the PIC microcontroller at the receiver end which is remote. Hence by using

Page 16: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

16

Zigbee we can transmit the serial data wirelessly. Zigbee CC2500 is used because

it has low power consumption and its’ cost is significantly lesser than other Zigbee

modules. Since Zigbee is a transceiver it is also used in receiver section to receive

the serial data.

4.1.4 MOTOR DRIVER (L298 IC Circuit)

It has four input pins two on each side of the PIC. All `these four pins are

connected to the digital pins of an PIC and four output pins are connected to DC

motors of the Robot. Enable pins are used to enable input/output pins on both the

sides of PIC and Vcc is used for supplying external power to the DC motors. Both

the motors on the same side of the robot move in the same direction at a time. So

positive ends of both motors are put in output pin 1 of PIC and negative ends of

same motors are put into output pin 2, same thing is done for other side of PIC too.

Vcc pin is be used to provide external power supply to the DC motors.

Operation Relay 1 Relay 2 Relay 3 Relay 4

Forward 0 1 1 0

Reverse 1 0 0 1

Left 1 0 1 0

Right 0 1 0 1

Stop 1 1 1 1

Table 4.1 Relay state

Page 17: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

17

4.1.5 DC MOTOR

A DC motor is mechanically commutated electric motor powered from

direct motor (DC). The stator is stationary in space by definition and therefore so is

its current. The current in the rotor is switched by the commutator. DC motors

better suited for equipment ranging from 12V DC systems in automobiles to

conveyor motors, both which require fine speed control for a range of speeds

above and below the rated speeds. The speed of a DC motor can be controlled by

changing field current.

4.1.6 RS232 CABLE

In telecommunications, RS-232 is a standard for serial communication

transmission of data. It formally defines the signals connecting between

a DTE (data terminal equipment) such as a computer terminal, and a DCE (data

circuit-terminating equipment or data communication equipment), such as

a Zigbee. The RS-232 standard is commonly used in computer serial ports. It

allows serial communication for interfacing the computer and the Zigbee module.

Page 18: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

18

4.2 SOFTWARE DESCRIPTION

4.2.1 FLOWCHART

IMAGE ACQUISITION

(IMAGE CAPTURED VIA WEBCAM

OF SIZE 360*160)

RESING OF IMAGE (256*256)

GRAYSCALE IMAGE TRANSFORM

MAXIMUM INTENSITY IMAGE

GENERATION OF BINARY IMAGE

EDGE DECTECTION OF IMAGE

(CANNY ALGORITHM)

FEATURE EXTRACTION

GENERATION OF COMMAND

SIGNAL FOR GESTURES

RECEVING COMMAND SIGNALS

WIRELESSLY (ZIGBEE RX)

Page 19: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

19

Fig 4.1 Flow diagram

WIRELESS TRANSMISSION OF

COMMAND SIGNALS (ZIGBEE TX)

COMMAND SIGNALS ARE READ

USING PIC CONTROLLER

RELAY CONNECTED TO PIC IS

USED TO DRIVER THE MOTOR

COMMAND SIGNALS ARE

RECEIVED WIRELESSLY (ZIGBEE

RX)

Page 20: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

20

4.2.2 IMAGE ACQUISITION

Image acquisition in image processing can be broadly defined as the action

of retrieving an image from some source, usually a hardware-based source, so it

can be passed through whatever processes need to occur afterward. Performing

image acquisition in image processing is always the first step in the workflow

sequence because, without an image, no processing is possible. The image that is

acquired is completely unprocessed and is the result of whatever hardware was

used to generate it, which can be very important in some fields to have a consistent

baseline from which to work. One of the ultimate goals of this process is to have a

source of input that operates within such controlled and measured guidelines that

the same image can, if necessary, be nearly perfectly reproduced under the same

conditions so anomalous factors are easier to locate and eliminate.

4.2.3 IMAGE SIZING AND CONVERSION

The captured image is resized into pixels of range 256*256 and then the

resized image is converted into grayscale image for further processing.

4.2.4 IMAGE THRESHOLDING

In grayscale image there are 255 gray intensity levels are present. For image

segmentation process we require a binary image in which there are only two

intensity levels. So a threshold value is set and the intensity levels above that are

taken as dark image and below it are transformed to white image. Thus the image

has been transformed to a binary image which makes easy for the processing for

image segmentation.

Page 21: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

21

4.2.5 IMAGE SEGMENTATION

Edge detection is the name for a set of mathematical methods which aim at

identifying points in a digital image at which the image brightness changes sharply

or, more formally, has discontinuities. The points at which image brightness

changes sharply are typically organized into a set of curved line segments

termed edges. The same problem of finding discontinuities in 1D signal is known

as step detection and the problem of finding signal discontinuities over time is

known as change detection.

Edge detection is a fundamental tool in image processing, machine

vision and computer vision, particularly in the areas of feature

detection and feature extraction.

Canny edge detection:

The Canny edge detector is an edge detection operator that uses a multi-

stage algorithm to detect a wide range of edges in images. It was developed

by John F. Canny in 1986.

The Process of Canny edge detection algorithm can be broken down to 5 different

steps:

1. Apply Gaussian filter to smooth the image in order to remove the noise

2. Find the intensity gradients of the image

3. Apply non-maximum suppression to get rid of spurious response to edge

detection

4. Apply double threshold to determine potential edges

5. Track edge by hysteresis: Finalize the detection of edges by suppressing all

the other edges that are weak and not connected to strong edges.

Page 22: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

22

4.2.6 FEATURE EXTRACTION

The purpose of feature extraction is to reduce the original dataset by

measuring certain properties of features that distinguish on input pattern. Feature

extraction is a special form of dimension reduction. When the input data to an

algorithm is very large to be processed, then the input data will be transformed into

a reduced set of features which also known as feature vector. Transforming the

input image to feature vector is known as feature extraction. Feature extraction

removes distracting variance from dataset so that downstream classifiers or

regression estimators perform better. The area where feature extraction ends and

classification, or regression, begins is necessarily murky: an ideal feature extractor

would simply map the data to its class labels, for the classification task. In this

paper six statistical features namely Mean, Variance, Skewness, Kurtosis, Energy,

and Entropy are extracted. These feature vectors are used to create a training set in

for the classification process.

Mean - For a data set, the arithmetic mean is equal to the sum of the values

divided by the number of values The mean is the arithmetic average of a set of

values, or distribution; however, for skewed distributions, the mean is not

necessarily the same as the middle value (median), or the most likely (mode).

Variance - The variance (σ2), is defined as the sum of the squared distances

of each term in the distribution from the mean (μ), divided by the number of terms

in the distribution (N).

Page 23: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

23

Skewness - If it is possible to divide the histogram at the center into two

identical halves, wherein each half is not a mirror image of the other, is called as

skewness. A measure of skewness is a single value that indicates the degree and

direction of asymmetry

Entropy -Entropy is a statistical measure of randomness that can be used to

characterize the texture of the input image. E = entropy(I) returns E, a scalar value

representing the entropy of grayscale image I. Entropy is a statistical measure of

randomness that can be used to characterize the texture of the input image. Entropy

is defined as,

-sum (p.*log2 (p))

where p contains the histogram counts returned from imhist

%entropy %equivalent mat lab code

Etp = entropy (ima)

Energy -Energy returns the sum of squared elements in the Grey Level Co

Occurrence Matrix (GLCM). Energy is also known as uniformity. The range of

energy is [0 1]. Energy is 1 for a constant image. Energy is also known as 84

uniformity of ASM (angular second moment) which is the sum of squared

elements from the GLCM.

%energy %equivalent mat lab code

Egy = sum (h_norm)

Kurtosis- Kurtosis, K measures the Peakness or flatness of a distribution

relative to a normal distribution. Kurtosis is a measure of how outlier-prone a

distribution is. Kurtosis is any measure of the "peakedness" of the probability

distribution of a real-valued random variable. In a similar way to the concept of

skewness, kurtosis is a descriptor of the shape of a probability distribution and, just

as for skewness; there are different ways of quantifying it for a theoretical

distribution.

Page 24: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

24

%kurtosis %Equivalent mat lab code

Ku = kurtosis (double (ima (:)))

Area -The area is a measure of the size of the foreground of the image. The

area is the number of pixels in the image. Area is a quantity that expresses the

extent of a two-dimensional surface or shape in the plane. Area can be understood

as the amount of material with a given thickness that would be necessary to fashion

a model of the shape, or the amount of paint necessary to cover the surface with a

single coat.

It is the two-dimensional analog of the length of a curve (a one-dimensional

concept) or the volume of a solid (a three-dimensional concept).The area of a

shape can be measured by comparing the shape to squares of a fixed size.

%area %equivalent mat lab code

Ar = bwarea (double (ima (:)))

Sum -Summation is the operation of adding a sequence of numbers; the

result is their sum or total. If numbers are added sequentially from left to right, any

intermediate result is a partial sum, prefix sum, or running total of the summation.

%Sum %equivalent mat lab code

Bag = sum (sum (ima))/ (sze (1)*sze (2))

%concatenate the features

vv = [ag;ahg;egy;etp;sd;co_v;m1;va;ku;ar;bag]/100.

After extracting these features of segmented mass/tumor, then the dataset has to be

constructed in the proper format, so that it can be given to the standard classifier

tools.

Page 25: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

25

CHAPTER 5

SIMULATION RESULTS

Page 26: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

26

5. 1 SIMULATION RESULTS

STEP 1: Webcam is used to capture real time video stream of hand gestures.

The captured image frame of size (360*120) is displayed as shown below in the

fig 5.1,

Fig 5.1 Image captured using webcam

STEP 2: The captured image is resized to (256*256). Its shown in the fig5.2

below,

Fig 5.2 Resized image

Page 27: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

27

STEP 3: The sized image is converted into grayscale image of same size

which is shown the fig5.3 below,

Fig 5.3 Gray image

STEP 4: Thresholding is done on this grayscale image for the recognition of

hand palm. Initially minimum threshold value is set to a certain constant. This

value can be used to threshold an image and thus to increment the value till the

system detects only one single blob of white area without any noise. The

thresholded image is shown in the fig5.4 below,

Page 28: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

28

Fig 5.4 Maximum intensity image

STEP 5: The thresholded image is converted into binary image for clear

hand gesture recognition. The image is as shown in the fig5.5 below,

Fig 5.5 Binary image

Page 29: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

29

STEP 6: The binary gesture image whose edges are detected using edge

detection method. In this method canny edge detection algorithm is used rather

than sobel, prewitt and Roberts edge detection technique because of its silent

features. The edge detected image is shown in the fig5.6 below,

Fig 5.6 Edge detected image

Page 30: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

30

5.2 COMMAND SIGNALS

The command signals are generated using parameters such as mean,

variance, skewness, entropy, energy, sum, area and kurtosis. The detection of

number of peaks and edge calculation is done using edge detection algorithms like

canny. The direction of movement of the robot is based on our hand gestures

(number of fingers open). if the number of fingers detected is one, the robot moves

forward; if two, the robot moves backward; if three, it moves left; if four, it moves

right; if five, the robot stops.

Other than hand gestures, keyboard inputs are also used to control the robot.

S.NO COMMAND ROBOT MOVEMENT

1 *f Forward

2 *v Reverse

3 *l Left

4 *r Right

5 *s Stop

Table 5.2 List of command signals

Page 31: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

31

FINAL HARDWARE MODULE

TRANSMITTER

Page 32: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

32

RECEIVER

Page 33: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

33

CHAPTER 6

CONCLUSION AND FUTURE SCOPE

Page 34: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

34

6.1 CONCLUSION

The proposed system based on PIC microcontroller is found to be more

compact, user friendly and less complex, which can readily be used in order to

perform several tedious and repetitive tasks. Though it is designed keeping in mind

about the need for industry, it can be extended for other purposes such as

commercial & research applications.

This project is highly efficient in places where the Robots are to be

controlled only by the authorized persons. Hence this project is applicable for real

time applications such as defense, rescue and military purpose.

6.2 FUTURE SCOPE

This project can be used in various applications with some modifications.

1. This robot is just a prototype and for real time applications the DC Motors

can be replaced by PMDC MOTORS or AC SERVO MOTORS for carrying

heavy loads inside the industries.

2. The project design, if implemented with a wheel chair, it can be used by

differently able people to move around.

3. Since the robot is provided with a wireless camera, it can be used for

surveillance purpose.

4. Regarding the software part, the type and weight carried by the robot can be

incorporated in the data base by adding suitable sensors and devices.

5. This project can be used for exploration of outer space.

Page 35: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

35

REFERENCES

[1] Sturman D., Zeltzer D., “A survey of glove-based input”, IEEE Transactions on

Computer Graphics and Applications, Vol. 14, No.1, Jan. 1994, pp. 30-39.

[2] Pavloic V.I., Sharma., Huang T.S., “Visual interpretationof hand gestures for

human-computer interaction: A review”. IEEE Transaction Pattern Analysis and

Machine Intelligence, Vol 19, July 1997, pp.677-695.

[3] Trivino G., Bailador., “Linguistic description of human body posture using

fuzzy logic and several levels of abstraction”, IEEE conference on Computational

intelligence for measurement systems and applications, Ostuni, Italy 27-29 Jun,

2007, pp. 105-109.

[4] Quek F.K.H., “Toward a vision-based hand gesture interface”, proceedings of

the virtual reality system technology conference, 1994,ppl. 17-29.

[5] Do J. and et al, “Advanced soft remote control system using hand gestures”,

MICAI(Advances in Artificial Intelligence)2006, LNAI, vol. 4293, 2006,pp. 745-

755.

[6] Premaratne P. and Nguyen Q., “Consumer electronics control system based on

hand gesture movement invariants”, IET Computer Vision, vol. 1 , no. 1, Mar.

2007, pp. 35-41.

[7] Kohler M., “Vision based remote control in intelligent home environments”, 3D

Image Analysis and Synthesis, 1996,pp. 147-154.

Page 36: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

36

[8] Bretzner L., Laptev I., Lindeberg T., Lenman S. and Sundblad Y., “A

Prototype system for computer vision based human computer interaction”,

Technical report ISRN KTH/NA/P-01/09-SE,2001.

[9] Sawah A.E., and et al., “A framework for 3D hand tracking and gesture

recognition using elements of genetic programming”, 4th Canadian conference on

Computer and robotic vision, Motreal, Canada, 28-30 May, 2007,pp. 495-502.

[10] Kim H., Fellner D.W., “Interaction with hand gesture for a back-projection

wall”, Proceedings of Computer Graphics International, 19 Jun, 2004, pp. 395-402.

[11] N.D. Binh, E. Shuichi and T. Ej ima, “Real-Time hand tracking and Gesture

Recognition sustem”, Oroc. Of GVIP Conference, 2005.

[12] Y.Fang, J.Cheng, K.Wang and H.Lu, “Hand Gesture Recognition Using Fast

Multi-Scale Analysis”, Proc. Of international Conference in Image and Graphics,

pp. 694-698, 2007.

Page 37: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

37

APPENDICES

Page 38: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

38

APPENDIX 1

PIC16F877

ARCHITECTURE OF PIC16F877

Page 39: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

39

PIN DIAGRAM OF PIC 16F877:

Page 40: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

40

PIN OUT DESCRIPTION

Legend: I = input O = output I/O = input/output P = power

— = Not used TTL = TTL input ST = Schmitt Trigger input

Page 41: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

41

Legend: I = input O = output I/O = input/output P = power

— = Not used TTL = TTL input ST = Schmitt Trigger input

Page 42: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

42

FEATURES PIC16F877:

The PIC16FXX series has more advanced and developed features when

compared to its previous series. The important features of PIC16F877 series is

given below.

General Features

o High performance RISC CPU.

o ONLY 35 simple word instructions.

o All single cycle instructions except for program branches which are two cycles.

o Operating speed: clock input (200MHz), instruction cycle (200nS).

o Up to 368×8bit of RAM (data memory), 256×8 of EEPROM (data memory),

8k×14 of flash memory.

o Pin out compatible to PIC 16C74B, PIC 16C76, PIC 16C77.

o Eight level deep hardware stack.

o Interrupt capability (up to 14 sources).

o Different types of addressing modes (direct, Indirect, relative addressing

modes).

Page 43: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

43

o Power on Reset (POR).

o Power-Up Timer (PWRT) and oscillator start-up timer.

o Low power- high speed CMOS flash/EEPROM.

o Fully static design.

o Wide operating voltage range (2.0 – 5.56)volts.

o High sink/source current (25mA).

o Commercial, industrial and extended temperature ranges.

o Low power consumption (<0.6mA typical @3v-4MHz, 20µA typical @3v-

32MHz and <1 A typical standby).

Peripheral Features

o Timer 0: 8 bit timer/counter with pre-scalar.

o Timer 1:16 bit timer/counter with pre-scalar.

o Timer 2: 8 bit timer/counter with 8 bit period registers with pre-scalar and post-

scalar.

o Two Capture (16bit/12.5nS), Compare (16 bit/200nS), Pulse Width Modules

(10bit).

Page 44: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

44

o 10bit multi-channel A/D converter

o Synchronous Serial Port (SSP) with SPI (master code) and I2C (master/slave).

o Universal Synchronous Asynchronous Receiver Transmitter (USART) with 9

bit address detection.

o Parallel Slave Port (PSP) 8 bit wide with external RD, WR and CS controls

(40/46pin).

o Brown Out circuitry for Brown-Out Reset (BOR).

Key Features

o Maximum operating frequency is 20MHz.

o Flash program memory (14 bit words), 8KB.

o Data memory (bytes) is 368.

o EEPROM data memory (bytes) is 256.

o 5 input/output ports.

o 3 timers.

o 2 CCP modules.

o 2 serial communication ports (MSSP, USART).

o PSP parallel communication port

Page 45: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

45

o 10bit A/D module (8 channels)

Analog Features

o 10bit, up to 8 channel A/D converter.

o Brown Out Reset function.

o Analog comparator module.

PIC START PLUS PROGRAMMER:

The PIC start plus development system from microchip technology provides

the product development engineer with a highly flexible low cost microcontroller

design tool set for all microchip PIC micro devices. The picstart plus development

system includes PIC start plus development programmer and mplab ide.

The PIC start plus programmer gives the product developer ability to program user

software in to any of the supported microcontrollers. The PIC start plus software

running under mplab provides for full interactive control over the programmer.

Page 46: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

46

APPENDIX 2

ZIGBEE MODULE

Page 47: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

47

CC2550:

DESCRIPTION:

The CC2550 is a low-cost 2.4 GHz transmitter designed for very low-power

wireless applications. The circuit is intended for the 2400-2483.5 MHz ISM

Page 48: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

48

(Industrial, Scientific and Medical) and SRD (Short Range Device) frequency

band.

The RF transmitter is integrated with a highly configurable baseband

modulator. The modulator supports various modulation formats and has a

configurable data rate up to 500 kBaud.

The CC2550 provides extensive hardware support for packet handling, data

buffering and burst transmissions.

The main operating parameters and the 64-byte transmit FIFO of CC2550

can be controlled via an SPI interface. In a typical system, the CC2550 will be used

together with a microcontroller and a few passive components.

FEATURES:

RF Performance

o Programmable output power up to +1 dBm

o Programmable data rate from 1.2 to 500 kBaud

o Frequency range: 2400 - 2483.5 MHz

Analog Features

o OOK, 2-FSK, GFSK, and MSK supported

Page 49: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

49

o Suitable for frequency hopping and multichannel systems due to a fast settling

frequency synthesizer with 90 us settling time

o Integrated analog temperature sensor

Digital Features

o Flexible support for packet oriented systems: On-chip support for sync word

insertion, flexible packet length, and automatic CRC handling

o Efficient SPI interface: All registers can be programmed with one "burst"

transfer

o Optional automatic whitening of data

Low-Power Features

o 200 nA SLEEP mode current consumption

o Fast startup time: 240 us from SLEEP to TX mode (measured on EM design

[3])

o 64-byte TX data FIFO (enables burst mode data transmission)

General

Page 50: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

50

o Few external components: Complete on chip frequency synthesizer, no

external filters needed

o Green package: Ro HS compliant and no antimony or bromine

o Small size (QLP 4x4 mm package, 16 pins)

o Suited for systems compliant with EN 300 328 and EN 300 440 class 2

(Europe), FCC CFR47 Part 15 (US), and ARIB STDT66 (Japan)

o Support for asynchronous and synchronous serial transmit mode for backwards

compatibility with existing radio communication protocols

Page 51: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

51

APPENDIX 3

LIQUID CRYSTAL DISPLAY (LCD):

LCD (Liquid Crystal Display) screen is an electronic display module and

find a wide range of applications. A 16x2 LCD display is very basic module and is

very commonly used in various devices and circuits. These modules are preferred

over seven segments and other multi segment LEDs. The reasons being: LCDs are

economical; easily programmable; have no limitation of displaying special &

even custom characters (unlike in seven segments), animations and so on.

A 16x2 LCD means it can display 16 characters per line and there are 2 such lines.

In this LCD each character is displayed in 5x7 pixel matrix. This LCD has two

registers, namely, Command and Data.

The command register stores the command instructions given to the LCD. A

command is an instruction given to LCD to do a predefined task like initializing it,

clearing its screen, setting the cursor position, controlling display etc. The data

register stores the data to be displayed on the LCD. The data is the ASCII value of

the character to be displayed on the LCD. Click to learn more about internal

structure of a LCD.

Page 52: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

52

PIN DIAGRAM:

Page 53: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

53

PIN DESCRIPTION:

Pin

No Function Name

1 Ground (0V) Ground

2 Supply voltage; 5V (4.7V – 5.3V) Vcc

3 Contrast adjustment; through a variable resistor VEE

4 Selects command register when low; and data register

when high

Register

Select

5 Low to write to the register; High to read from the register Read/write

6 Sends data to data pins when a high to low pulse is given Enable

7

8-bit data pins

DB0

8 DB1

9 DB2

10 DB3

11 DB4

12 DB5

13 DB6

14 DB7

15 Backlight VCC (5V) Led+

16 Backlight Ground (0V) Led-

Page 54: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

54

APPENDIX 4

POWER SUPPLY

BLOCK DIAGRAM:

Fig 8.1 Block diagram of power supply

The ac voltage, typically 220V rms, is connected to a transformer, which

steps that ac voltage down to the level of the desired dc output. A diode rectifier

then provides a full-wave rectified voltage that is initially filtered by a simple

capacitor filter to produce a dc voltage. This resulting dc voltage usually has some

ripple or ac voltage variation. A regulator circuit removes the ripples and also

remains the same dc value even if the input dc voltage varies. This voltage

regulation is usually obtained using one of the popular voltage regulator IC units.

Page 55: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

55

CIRCUIT DIAGRAM:

Circuit diagram of power supply

WORKING PRINCIPLE:

Transformer:

The potential transformer will step down the power supply voltage (0-230V)

to (0-15V and 0-9V) a level. If the secondary has less turns in the coil then the

primary, the secondary coil's voltage will decrease and the current or AMPS will

increase or decreased depend upon the wire gauge. This is called a STEP-DOWN

transformer. Then the secondary of the potential transformer will be connected to

the rectifier.

Page 56: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

56

Bridge rectifier:

When four diodes are connected as shown in figure, the circuit is called as

bridge rectifier. The input to the circuit is applied to the diagonally opposite

corners of the network, and the output is taken from the remaining two corners.

Let us assume that the transformer is working properly and there is a

positive potential, at point A and a negative potential at point B. the positive

potential at point A will forward bias D3 and reverse bias D4.

The negative potential at point B will forward bias D1 and reverse D2. At

this time D3 and D1 are forward biased and will allow current flow to pass through

them; D4 and D2 are reverse biased and will block current flow. The path for

current flow is from point B through D1, up through Load, through D3, through

the secondary of the transformer back to point B.

One-half cycle later the polarity across the secondary of the transformer

reverse, forward biasing D2 and D4 and reverse biasing D1 and D3. Current flow

will now be from point A through D4, up through Load, through D2, through the

secondary of transformer, and back to point A. Across D2 and D4. The current

flow through Load is always in the same direction.

Page 57: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

57

In flowing through Load this current develops a voltage corresponding to

that. Since current flows through the load during both half cycles of the applied

voltage, this bridge rectifier is a full-wave rectifier. One advantage of a bridge

rectifier over a conventional full-wave rectifier is that with a given transformer the

bridge rectifier produces a voltage output that is nearly twice that of the

conventional half-wave circuit.

This bridge rectifier always drops 1.4Volt of the input voltage because of the

diode. We are using 1N4007 PN junction diode, its cut off region is 0.7Volt.

So any two diodes are always conducting, total drop voltage is 1.4 volt.

Filter:

If a Capacitor is added in parallel with the load resistor of a Rectifier to form

a simple Filter Circuit, the output of the Rectifier will be transformed into a more

stable DC Voltage.

At first, the capacitor is

charged to the peak

value of the rectified

Waveform.

Beyond the peak, the capacitor is discharged through the load until the time

at which the rectified voltage exceeds the capacitor voltage. Then the capacitor is

charged again and the process repeats itself.

Page 58: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

58

IC voltage regulators:

Voltage regulators comprise a class of widely used ICs. Regulator IC units

contain the circuitry for reference source, comparator amplifier, control device, and

overload protection all in a single IC. IC units provide regulation of either a fixed

positive voltage, a fixed negative voltage, or an adjustably set voltage.

A fixed three-terminal voltage regulator has an unregulated dc input voltage,

it is applied to one input terminal, a regulated dc output voltage from a third

terminal, with the second terminal connected to ground.

The series 78 regulators provide fixed positive regulated voltages from 5 to 24

volts. Similarly, the series 79 regulators provide fixed negative regulated voltages

from 5 to 24 volts.

This is a regulated power supply circuit using the 78xx IC series. These

regulators can deliver current around 1A to 1.5A at a fix voltage levels. The

common regulated voltages are 5V, 6V, 8V, 9V, 10V, 12V, 15V, 18V, and 24V. It

is important to add capacitors across the input and output of the regulator IC to

improve the regulation.

Page 59: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

59

APPENDIX 5

WEBCAM

A webcam is a video camera that feeds or streams its image in real time to

or through a computer to computer network. When "captured" by the computer, the

video stream may be saved, viewed or sent on to other networks via systems such

as the internet, and email as an attachment. When sent to a remote location, the

video stream may be saved, viewed or on sent there. Unlike an IP camera (which

connects using Ethernet or Wi-Fi), a webcam is generally connected by

a USB cable, or similar cable, or built into computer hardware, such as laptops.

The term 'webcam' (a clipped compound) may also be used in its original sense of

a video camera connected to the Web continuously for an indefinite time, rather

than for a particular session, generally supplying a view for anyone who visits

its web page over the Internet. Some of them, for example, those used as

online traffic cameras, are expensive, rugged professional video cameras.

Page 60: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

60

APPENDIX 6

RS232

RS232 COMMUNICATION:

In telecommunications, RS-232 is a standard for serial binary data

interconnection between a DTE (Date Terminal Equipment) and a DCE (Data

Circuit-Terminating Equipment). It’s commonly used in computer serial ports.

Scope of the Standard:

The Electronic Industries Alliance (EIA) standard RS-232-C [3] as of 1969

defines

Electrical signal characteristics such as voltage levels, signaling rate, timing

and slew-rate of signals, voltage withstand level, hort-circuit behavior,

maximum stray capacitance and cable length.

Interface mechanical characteristics, pluggable connectors and pin

identification

Functions of earth circuit in the interface connector.

Standard subsets of interface circuits for selected telecom appliances.

The standard does not define such elements as character encoding (for

example, ASCII, Baudot or EBCDIC), or the framing of characters in the data

stream (bits per character, start/stop bits, parity). This standard does not define

protocols for error detection or algorithms for data compression.

Page 61: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

61

The standard does not define bit rates for transmission, although the standard

says it is intended for bit rates lower than 20,000 bits per second. Many modern

devices can exceed this speed (34,400 and 57,6000 bit/s being common, and

115,200 and 230,400 bit/s making occasional appearances) while still using RS-

232 compatible signal levels.

Details of character format and transmission bit rate are controlled by the

serial port hardware, often a single integrated circuit called a UART that converts

data from parallel to serial form. A typical serial port includes specialized driver

and receiver integrated circuits ti convert between internal logic levels and RS-232

compatible signal levels.

CIRCUIT WORKING DESCRIPTION:

In this circuit the MAX 232 IC is used as logic level converter. The

MAX232 dual driver/receiver that includes a capacitive voltage generator to

supply EIA 232 voltage levels from a single 5V supply. Each receiver converts

EIA-232 to 5V TTL/CMOS levels. Each driver converts TLL/COMS input levels

into EIA-232 levels.

Page 62: CHAPTER 1 INTRODUCTION - WordPress.com · robot in the environment using gestures. In this system, user operates the robot from a control station that can be a laptop or a PC with

62

In this circuit the micro controller transmitter pin is connected in the

MAX232 T2IN pin which converts input 5V TTL/CMOS level to RS232 level.

Then T20UT pin is connected to reviver pin of 9 pin D type serial connector which

is directly connected to PC.

In PC the transmitting data is given to R2IN of MAX232 through

transmitting pin of 9 pin D type connector which converts RS232 level to 5V

TTL/CMOS level. The R2OUT pin is connected to receiver pin of the

microcontroller. Likewise the data is transmitted and received between the

controller and PC or other devices vice versa.

Circuit diagram of RS232