View
1.644
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Master's students use concepts from my (Jeff Funk) forthcoming book (Technology Change and the Rise of New Industries) to analyze the technical and economic feasibility of new human-computer interfaces (e.g., touch, gestures, voice, neural interfaces). See my other slides for details on concepts, methodology, and other new industries.
Citation preview
1
The Future Of Human-Computer Input Interfaces
Stephanie BudimanJohnny Cham
Karthik NandakumarOsbert Poniman
Mauhay Mary Esther Samson
2
Computing Anytime Anywhere!
Motivation: In today’s world, easy access to information and computing is required anytime and anywhere
3
HCI
Human-Computer Interface (HCI)
• HCI is the technology that connects man and machine
• Robust HCIs are needed to enable ubiquitous computing
We focus only on input interfaces in this presentation
Human Computer
Thoughts
ActionInput
InterfaceAction
Recognition
Task Execution
Understanding
Output Interface
Sensory Perception
Rendering
4
Technology Paradigms for Input Interfaces
Graphical User Interfaces
Command Line Interfaces
Batch Interfaces
Natural User Interfaces Neural Interfaces
5
Value Proposition
Improvements in component technologies
AccuracyThroughputAffordability
User ExperienceEase of UseSociabilityMobility
CLI
GUI
Natural UI
Neural Interfaces
Low Medium HighLow
Medium
High
6
Key Components of Input Interfaces Human-Computer Input Interfaces
Neural Interfaces
Natural UI
Speech
Micro phone
Neural electrodes/
sensors
GestureTouch
3D Camera
Touch sensor
Tracking & Recognition
Software
Materials/Nanotechnology
Signal Processing Hardware (Semicon-ductors)
7
Speech Interfaces
Key Components
Microphone
Automated Speech Recognition (ASR) and Natural Language Understanding (NLU) Software
Possible methods of improvement are
• Increase Signal to Noise Ratio (SNR) from microphone
• Achieve human-level performance on ASR/NLU tasks
Key value that needs improvement is Accuracy
8
Microphone Technology
Electret Condenser Microphone (ECM)
SNR: 55-58 dB
MEMS Digital Microphone
SNR: 61 dBFlatter frequency response
Smaller size (CMOS fabrication)
Noise levels in a microphone is close to the thermal noise limit Analog Devices, MEMS Microphone Technology, October 2010
9
Microphone: Method of ImprovementMicrophone array can mitigate background noise & interference1
1. LOUD project from MIT Computer Science and Artificial Intelligence Laboratory, 2005
2. Microphone Array project in MSR: Approach and Results, Microsoft Research, June 2004
Noise suppression algorithms can increase SNR by 18dB with just 4 microphones in an array2
Improvement in SNR Corresponding decrease in Word Error Rate
10
Automated Speech Recognition (ASR)
“Increase in vocabulary sizes needs exponential increase in computing power due to potential combinatorial explosions”
L. Rabiner, “Challenges in Speech Recognition”, NSF Symp. on Next Gen. ASR, 2003
11
ASR Accuracy Improvements
ASR accuracy is acceptable only in some niche applications
WO
RD
ER
RO
R R
ATE (
%)
NIS
T B
ench
mark Te
st Histo
ry – M
ay 2
009
YEAR
12
ASR Accuracy in Text Dictation
• Stand-alone speech interfaces may be useful for tasks like dictation
• Speech as an important modality in multimodal user interfaces (e.g., Microsoft Kinect) may be the future
* http://blogs.msdn.com/b/sprague/archive/2004/10/22/246506.aspx
*
13
Touch Interfaces
Key Component
• Touch screen
Microsoft SurfaceiPad Touch
Key values that need improvement are Throughput and Affordability
14
Comparing Touch UI with Mouse• Fitt’s Law: Time required to move to a target area displayed
on screen is proportional to index of difficulty (distance to the target/width of the target area)
Touch UI has better throughput than mouse for more difficult tasks
Muller, L.Y.L., “Multi-touch displays: design, applications & performance evaluation”, 2008
15
Improving Throughput of Touch UI
B. Buxton, “Multi-Touch Systems”, Microsoft Research, 2007
Single Touch• 1960, IBM• 1972, Plato IV Computer• 1992, Simon (IBM):1st smart
phone
Single Touch with Pen Input• 1991, Digital Desk• 1996, Palm Pilot• Pen writes, touch
manipulates
Multi-touch• 2001, Fingerworks• 2005, Apple
Interaction based on large multi-touch screens can increase the throughput of touch user interfaces
16
Touch Screen Technologies
Infrared (IR)
Resistive
Surface Acoustic Wave
Surface Capacitive
Source: 3M Touch System Website
17
Comparison of Touch TechnologiesTouch
technology Capacitive Resistive InfraredSurface
Acoustic Wave
Transparence Very good 90%
75%-85% Very good 90%
Very good 90%
Resolution Good Good Limited due to spacing of IR
sensors
Good
Display size Up to 21” Up to19” Up to 60” Up to 30”
Touch method Human touch Can use any pointing device
Can use any pointing device
Finger, gloved, hand or soft tip
Touch Reliability
Unlimited 50 million touches
Unlimited 100 million touches
Cost (15”Monitor) $680 $695 $1356 $820
Source: Frost & Sullivan, Advances in Haptics & Touch Technology, 2010
18
Future of Touch: Tangible Bits
Manipulating digital objects through physical objects
19
Gesture Interfaces
Key Components
3D Camera (image sensor)
Tracking, Recognition &
Gesture Understanding
Software
Key values that need improvement are Accuracy, Throughput and Affordability
20
Gesture UI: Methods of ImprovementRequired improvements in image sensor technology
Accuracy
• Higher spatial resolution (number of pixels)
• Robustness to lighting changes (high pixel sensitivity)
• More accurate depth sensing (lower depth error)
Throughput
• Higher frame rate (temporal resolution)
Affordability
• Smaller pixel size reduces price per pixel
21
Image Sensor Technologies
CMOS-based image sensors are also expected to follow Moore’s Law in size and cost scaling
T. Suzuki, “Challenges of Image-Sensor Development”, ISSCC, 2010
As pixel size decreases, resolution improves (more pixels per area), but sensitivity decreases because the “light available per pixel” will become less
Back illuminated CMOS technology provides better trade-off between pixel size and sensitivity than traditional charge coupled device (CCD)-based image sensors
22
Camera Technology Improvements
T. Suzuki, “Challenges of Image-Sensor Development”, ISSCC, 2010
Reducing pixel-size (green square) and improving sensitivity (Yellow circle ) miniaturized cameras without reducing quality
23
Camera Price Improvements
K. Wiley, “Digital Photography”, www.keithwiley.com
Number of pixels (resolution) has increased, while price per pixel has decreased
Year
Pric
e pe
r pi
xel f
or a
12
MP
cam
era
24* R. Lange, “3D Time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology”, PhD Thesis, 2000
3D Image Sensor Technology• Time of flight (ToF)
cameras require high temporal resolution* (70 picoseconds for 1cm depth resolution)
• CMOS technology now has the required temporal resolution to enable production of affordable ToF 3D cameras
• 3D cameras will improve the accuracy of gesture UI
Cost-effective 3D image sensors are now becoming available (e.g., Microsoft Kinect ~ 150 USD)
Dep
th E
rro
r/D
ista
nce
Distance to image
25
Neural InterfacesKey Component
Brain scanning device
Key values that need improvement are Accuracy, Throughput and Affordability
Required improvements in brain scanning technology
Accuracy – Higher spatial resolution
Throughput – Higher temporal resolution
Affordability – Size and better materials
26
SPECT
EEG
1936 1950 1972 19751968
CT Scan
1983
MEG
1991
fMRINIRS
1973
MRI PET
US$2.9M
US$5M-7M US$1M-1.5M
US$250K US$2.4M US$0.5M-3M
US$180K- 250K>US$30K
Baranga, A. B.-A. (2010). "Brain's Magnetic Field: a Narrow Window to Brain's Activity". Electromagnetic field and the human body workshop, (pp. 3-4).
Key Brain Scanning Technologies
Electro Encephalo Graphy
Magneto Encephalo Graphy
27
Comparison of Technologies
• Ideally, a non-invasive technology with high spatial resolution and high temporal resolution is required
• Additionally, the technology must be affordable and portable in order to be useful in HCI applications
Gerven, M. v., et al., “The Brain-Computer Interface Cycle”, J. Neural Eng, 2009
Invasive
Non-invasive
Neuron can fire ~0.1mm (spatial) & ~10 ms (temporal)
28
Spatial Resolution Improvement
While spatial resolution is important for accuracy, high temporal resolution is critical for user interfaces
R. Kurzweil, “The Singularity is Near”, 2005
29
ElectroEncephaloGraphy (EEG)
• Non-invasive interface using electrodes to pick up brain signals
• Key limitation: Poor spatial resolution
Increasing number of EEG electrodes may provide limited improvement in spatial resolution and higher SNR
J. Malmivuo, “Comparison of the Properties of EEG and MEG”, Intl J of Bioelectromagnetism , 6 (1), 2004
30
MagnetoEncephaloGraphy (MEG)
Baranga, A. B.-A. (2010). "Brain's Magnetic Field: a Narrow Window to Brain's Activity". Electromagnetic field and the human body workshop, (pp. 12).
fT
31
MEG: Improvements in Millimeter-scale Atomic Magnetometer
Target: ~100fT and <100Hz
J. Kitching, et al., “Uncooled, Millimeter-Scale Atomic Magnetometers”, IEEE Sensors 2009 Conference (pp. 1844-1846)
A: 2004B: 2007C: 2007D: 2009
32
Further Scope for Improvement• Hybrid technologies
– Enable high levels of accuracy in diagnosis that individual modalities cannot offer.
• E.g: MEG, EEG and MRI1 or PET/CT, PET/MRI2
• Better shielding for noise reduction
– Low frequency noise: use flux-entrapment shields
– High frequency noise: use lossy magnetic shields based on induced eddy currents
• Reduce costs
– Open-source: OpenEEG
[1] Baranga, A. B.-A. (2010). "Brain's Magnetic Field: a Narrow Window to Brain's Activity". Electromagnetic field and the human body workshop, (pp.15).
[2] Stommen, J. (2011, Mar 25). "Superior capabilities boost hybrid imaging, says report“, Medical Device Daily
33
Opportunities in Input InterfacesMicrophone
Multitouch Sensors
3D camera & motion sensors
Haptic Devices
Neural Sensors
Speaking
Touch Events/
Gestures
Body/head movements
Free hand gestures
Facial expression
Eye gaze
Hand pressure
Brain activity
Speech Recognition &
Language Understanding
Touch/Gesture Recognition & Understanding
Neural Signal Analysis &
Understanding
User Interface System Design & Integration
Multimodal Fusion
Human Factors
Engineering
Context Aware
Services
34
Market Segments for Input Interfaces
Natural & Neural User Interfaces
SmartphonesTablet/
Wearable PCs
Consumer Electronics
TelevisionGames
Virtual Reality
Entertainment
Healthcare
ProstheticsTele-medicine Military
E-commerce
Security
AircraftAutomotive
Transportation
Tele-control
Robotics
Virtual / Collaborative
Learning
EducationGroup InterfaceTelepathic System
Telecommunications
35
Conclusions
• Ubiquitous computing requires new HCI paradigms
• Natural and Neural Interfaces are the future of human-computer input interfaces
• Touch interfaces have already diffused into the mainstream; speech and gesture interfaces are becoming more accurate and affordable
• Neural interfaces requires development of more accurate, cheap, and portable sensors
• Numerous entrepreneurial opportunities are available both in technology development & customization
36
THANK YOU!
Q&A?