Embed Size (px)
Citation preview
PhonePoint Pen: Using Mobile Phones to Write
in Air
Romit Roy ChoudhuryAssistant Professor
Noting small pieces of information,
quickly and effortlesslycan be useful
So, what are the options ...
Buy Milk
State of the Art
Sticky notes … organization is a nightmareorganization is a nightmare not handynot handy
Typing on keyboard
tiny keyboard sizestiny keyboard sizes small inter-key spacingsmall inter-key spacing
Audio recording cannot sketch informationcannot sketch information time consuming to browse through voice messagestime consuming to browse through voice messages
So, need a solution that is
Easy to use
Always-with-me
Allows sketching
Searchable
Proposed Approach: PhonePoint Pen
User can write messages in air holding the phone like a penholding the phone like a pen
Use built-in accelerometer
capture hand movementcapture hand movement
Decode text and image sent to user’s email address for future use sent to user’s email address for future use
Use Cases
Mashing with cameras superimposing text on picturessuperimposing text on pictures
Sketching exchanging figures between caller/calleeexchanging figures between caller/callee
One handed use e.g., carrying a suitcase e.g., carrying a suitcase
Duke
University
Prototype
Current prototype implemented on Nokia N95 mobile phones
several design challenges emerge …
Design Challenges (1)
Lack of a Gyroscope Accelerometers only measure linear Accelerometers only measure linear acceleration acceleration Linear Movements – Rotation Ambiguity Linear Movements – Rotation Ambiguity
Proposed Approach: • Hold Phone in Non-rotating GripHold Phone in Non-rotating Grip• Determine Angular Orientation during Determine Angular Orientation during the the
pause pause
P
Design Challenges (2)
Background Vibration (Noise) Sensitive AccelerometersSensitive Accelerometers Significant Jitter by hand vibrationsSignificant Jitter by hand vibrations
Proposed Approach: • Smooth the accelerometer readings with Smooth the accelerometer readings with
moving average over last n (=7) readingsmoving average over last n (=7) readings• Suppress acceleration values < 0.5 Suppress acceleration values < 0.5
P
2/ sm
Design Challenges (3)
Computing Displacement Erroneous Acceleration ReadingErroneous Acceleration Reading Ambiguity when acceleration is zeroAmbiguity when acceleration is zero
Proposed Approach: • Detect Pause Using Moving WindowDetect Pause Using Moving Window• Reset Velocity to Zero in Between StrokesReset Velocity to Zero in Between Strokes
P
Design Challenges (4)
Lifting Pen from the Paper e.g., dotting the “i” and crossing the “t”e.g., dotting the “i” and crossing the “t”
Proposed Approach: • Impulse on the Z axis during the liftImpulse on the Z axis during the lift• Off the Record – On the RecordOff the Record – On the Record
P
Drawing a Rectangle
RawRaw Accelerometer ReadingAccelerometer Reading
Noise Smoothing using Noise Smoothing using Moving AverageMoving Average
Background Noise Background Noise SuppressionSuppression
Velocity Plot after Velocity Plot after Avoiding Velocity DriftsAvoiding Velocity Drifts
Final RectangleFinal Rectangle
Raw Accelerometer Data for M and iRaw Accelerometer Data for M and i
More PhonePen words …
Ongoing Work
Lowercase character recognition
Smaller hand movements in air
Increased robustness and reliability
Swipe your credit card and sign in air …
Our vision:
