Ultra-Low Power Gesture Recognition System

Bryce Kellogg, Vamsi Talla, Shyam Gollakota

Beyond Mouse and Keyboard

New Forms of Interaction

Limited to line of sight andrequires significant power

Our Idea

TV Cellular Wi-Fi

Leverage changes in ambient signals for always on gesture recognition

AllSee

First gesture system that runs without batteries• Leverages ambient TV and RFID signals

3 – 4 orders of magnitude less power

Integrated with mobile phones• Works through pockets

AllSee Hardware Architecture

RF Energy Harvester

Antenna

Digital Logic(MSP430)

WirelessReceiver

40 µW

AllSee Hardware Architecture

RF Energy Harvester

Antenna

Digital Logic(MSP430)

WirelessReceiver

40 µW

Consumes > 100 mW of powerImpractical for always on gestures

Challenge: Radios Drain Batteries

How do we access wireless signals without traditional radios?

Solution: Ambient Backscatter ReceiverUses only passive components

Amplitude

Zero power consumption

AllSee Hardware Architecture

RF Energy Harvester

Antenna

Digital Logic(MSP430)

40 µW

WirelessReceiver

AllSee Hardware Architecture

RF Energy Harvester

Antenna

Digital Logic(MSP430)

WirelessReceiver

40 µW

Challenge: Designing Low-Power Classifier

Our receiver only provides amplitude - Prior solutions use phase (Doppler, AoA)

Can’t run computationally intensive operations- No machine learning or even multiplication

Solution: Amplitude Library of GesturesAm

plitu

de

Time

Push versus Pull

Generalizing to Multiple Gestures

Uses only add, shift, and compare operations

AllSee Hardware Architecture

RF Energy Harvester

Antenna

Digital Logic(MSP430)

WirelessReceiver

40 µW

1 mW

Solution: Optimize AllSee’s Workflow

Sleep Sampling Detection

200 Hz

Classification

System power consumption: 30 µWResponse Time: 80 µs

Evaluation

Classification Accuracy

5 Participants8 Gestures20 RepetitionsTotal 800 Gestures

0.25% not detected94% correctly classified

Measured over 24 hours while 12 people share the workspace

False Positives

11.1 per hour

Start: 0.083 per hour

Through the Pocket

90% correctly classified

Conclusion

First gesture system that runs without batteries• Leverages ambient TV and RFID signals

3 – 4 orders of magnitude less power