Touchless writer

Touchless Writer

Software Development Project-II Course No: CSE-3200

Developed By Supervised ByBikash Karmokar Dr. Kazi Md. Rokibul Alam0707019

&Md. Kibria Siddiquee0707024

Department of Computer Science and EngineeringKhulna University of Engineering & Technology

Inspiration

First of all we would like to pay glowing tribute to the Language Movement martyrs who sacrificed their lives for the mother tongue in 1952.

From that inspiration we developed a software for

Bengali language

TOUCHLESS WRITER

What is it ?

Touchless introduces a new way of interacting with the computers by means of object tracking through webcams for Bengali character writing.

What is it ?

Here data is inserted for writing purpose using webcam without use of keyboard or on-screen keyboard by mouse.

Needed Tools

Webcam Pen with a head of red color/any colored

object Windows Platform Avro Keyboard installed .Net Framework 3.5

Tracking Process

Capturing video using aforge .net From the tracking environment it first

detect the red colored object and mark it with a rectangle using EuclideanColorFiltering.

Getting (x , y) coordinate of the rectangle and putting pixel at that point on a white panel.

So finally we get 33*30 pixel bitmap image which is ready for neural network input.

Tracking Process

After writing

Training network Recognize character Speak out character

Training Completion

Recognized Result

Saving written letter

Mechanism of recognition

So how neural network works ?

Neural network phases

Retrieve data

Feature Extraction

Training

Testing

RETRIEVE DATA

Total number of input = 33*30 = 990 pixels

FEATURE EXTRACTION

In this method it scans the binary image until it finds the boundary. The searching follows according to the clockwise direction.

FEATURE EXTRACTION

For any foreground pixel p, the set of all foreground pixels connected to it is called connected component containing p.

FEATURE EXTRACTION

The pixel p and its 8-neighbors are shown in Figure 4. Once a white pixel is detected, it checks another new white pixel and so on.

FEATURE EXTRACTION

FIG: pixel p with its 8 adjacentAfter feature extraction our input will

approximately reduced to 67%

p

TRAINING PHASE

TESTING

In this phase we will test the network by giving some patterns. We match it with every trained pattern and find out the pattern that gives highest match and lowest match also.

Development Tools

Visual studio 2008 XML Avro Keyboard installed Aforge .Net

Limitations

Due to brightness and contrast sometimes webcam can hardly detect the expected color.

Because of the similarity of tracking environment background color and object color the writing panel gets unexpected pixels.

Limitations

As we draw character using object movement it is not properly drawn as like as original character, sometimes it becomes totally different from the original. For that reason neural network can’t understand or recognize the original character and it outputs wrong character as input value or character.

Future plan

Add facility for writing for both Bangla and English

Add facility to make the software capable of running without the help of keyboard and mouse.

Adding printing capabilities of written text.

Future plan

Adding written text reading capabilities in Bangla.

Adding capabilities of tracking more than one object and take several decisions depending on object combinations

References

Microsoft Press Microsoft Visual C Sharp 2008 Step by Step

Beginners C#.net 2005 Worx Publication Professional C#.net 2005 Wrox Publication MSDN Library www.c-sharpcorner.com www.codeproject.com www.aforgenet.com 

Thanks to all