HandTalk

Design & Architecture

Feb 15th, 2008

Status Update

Found sweet new module for the glove – does exactly what we need

But now have to work with less than ideal number of I/O pins!

Going to talk to Motorola contact to see if we can get sweeter modules, sensors or phone

Sensors need power – almost solved that problem

Glove Bluetooth Data Acquisition Module

Nokia 95

Flex Sensors A2D Bluetooth Module

Bluetooth Module

Gesture Recognition (Java2ME)

Text Speech Engine

Quantized data

Analog Digital

Translated Gesture

USER

AUDIENCE

Ges

ture

Speech

Architecture

Use Cases

Power On User activates application on cellphone which sends wake

command over Bluetooth to glove module Glove module LED turns green indicating device is active Cellphone application sends command for glove to stream data

and processes it as it arrives Recognize Gesture (Fist/Open Palm for now)

User makes gesture, causing a combination of readings to be sent

If measurement pattern matches a known gesture, the appropriate word is voiced through Text-to-Speech engine

Sleep Mode If user turns off application, sleep command is sent to glove LED on glove module turns off indicating it is in sleep mode

Risks & Mitigation

- Glove module has ability to power off sensors solution works if we power sensors through module

- Relay activated by general purpose I/O pin on module

Putting powered sensors to sleep might require separate switch

- Use 2 flex sensors for each of 3 fingers and 1 each for the other 2

- Use 1 sensor for each finger and 3D accelerometer Requires rethinking of objective

Too few I/O pins(8 as opposed to ideally 10+)

- Define acceptable thresholds- Expand voltage range of sensors using more

complicated setup- Machine learning

Difficult to match sensor readings to known patterns

Glove module specifically does what we needBluetooth complicated to implement

MitigationRisks