Toward Subtle Intimate Interfaces for Mobile Devices

Using an EMG Controller

Enrico CostanzaMedia Lab Europenow at MIT Media Lab

Samuel A. InversoMedia Lab Europe

Rebecca AllenMedia Lab Europe

Outline

• Motivation: Subtle and hands-free interaction• EMG as a solution• An EMG-based controller• Design approach• Formal user study• Conclusion

• Mobile interaction is often in public spaces• Subtle interfaces: do not disrupt the environment• Intimate interaction: only for the user• Ringing vs. vibrating alert

Importance of Subtlety in Mobile Interfaces

• Speech recognition and evident gesturing can be inappropriate

Importance of Subtlety in Mobile Interfaces

Design for Hands-free Interaction

Eyeglass displays

Design for Hands-free Interaction

• Electrical signal from muscle activity• Can measure isometric activity:

subtle or no movement• Surface Electrodes (EKG-like)• Non-contact sensing (future)

Electromyogram (EMG) as a Novel Solution

• Prosthesis control• Input devices for disabled users• Affect sensing• Music expression

EMG in CHI(Related Work)

• EMG and movement are not always related • Tanaka & Knapp report this as a limitation• We think it is an advantage!

EMG and MovementLimitation or Advantage?

• EMG greatest potential for mobile HCI• Sense subtle gestures• Example: brief contraction of the bicep

Motionless Gestures

• Self-contained in armband• Integration with Bluetooth devices

(e.g. Phones and PDAs)• No calibration for individual users

EMG-based Controller

The gesture should be:• Natural to perform• Different from normal muscle activity

User centered iterative approach:1. Select muscle & generic gesture definition

(non-detailed description to subjects)2. Definition refinement, model and algorithm3. Tuning

Design Process

• Realistic controlled environment: subjects walked around obstacles in trafficked walkway

• 10 subjects• Audio stimuli and feedback• Is training avoidable? (minimal feedback)• Push the limit: short and long contractions

Formal User Study

• 96% correct recognition• No false positives• No training necessary in 7 out of 10 cases• Cannot distinguish short and long contractions

across different subjects

Results

• EMG can be successfully used (96%)• Generally no training required• No calibration across users

Discussion

• Cannot distinguish short and long contractions• Subjective definition of “short” and “long”

Discussion

• Test in more complex scenarios• Measure subtleness• Improve algorithm• Use more muscles (alphabet definition)

Future Work

• Subtle interaction for mobile devices• New reason to use EMG in CHI• Motionless gestures• It works (96% correct recognition no false positives)

Summary and Conclusion