Embed Size (px)
DESCRIPTION
Keynote address by Mark Billinghurst at the IUI 2014 conference in Haifa Israel, February 27th, 2014.
Citation preview
Using Augmented Reality to Create Empathic Experiences
Mark Billinghurst
The HIT Lab NZ, University of Canterbury
February 27th 2014
Courtesy Matt Rettig, CMU
Processing Power
Operate
Experience
Adapt
Beyond the Desktop
Intelligent User Interfaces AI + HCI: User Interface involving some
elements of Artificial Intelligence Computer having model of user/domain
First IUI Conference in 1997 Readings in IUI (Wahster 1998)
Microsoft Clippy (1997) MS Office Intelligent User Interface
Intelligent User Interfaces
Multiple Intelligences
Frames of Mind: The Theory of Multiple Intelligences Howard Gardner (1983)
Multiple Intelligences
Emotional Intelligence Emotional Intelligence - Why it
can matter more than IQ Goldman (1995)
Identify, assess, and control the emotions of oneself, of others, and of groups
Foundations of Emotional Intelligence
Empathy
Empathy
Empathy vs. Intelligence Intelligence: the power of one’s brain, divided
into many categories and used in numerous ways. Empathy: the power of one’s heart, expressing
one’s true emotions with themselves, those around them, and their own world.
IQ vs EQ?
Mirror Neurons
Neuron that fires both when an animal acts and it observes the same action performed by another Giacomo Rizzolatti, Univ. of Palma (1980s/90s)
Empathic Computing 1. Computing systems that can understand your feelings and emotions 2. Computing systems that help you better understand the feelings of others
Affective Computing
Ros Picard – MIT Media Lab http://affect.media.mit.edu
Appliances That Make You Happy
Jun Rekimoto – Univ. Tokyo Smile detection + smart devices
Can we develop interfaces that allow us to be more empathetic to others?
adsf
Empathy Computing Requirements
Basic Requirements Making the technology transparent
Empathy Definition Seeing with the eyes of another Hearing with the ears of another Feeling with the heart of another
Augmented Reality
1977
2008
Using AR for Empathy
Augmented Reality can: Remove technology barriers Enhance communication Change perspective Share experiences Enhance interaction in real world
Communication Seams
Technology introduces artificial seams in the communication (eg separate real and virtual space)
Task Space
Communication Space
Removing Barriers: Shared Space
Face to Face interaction, Tangible AR metaphor - ~3,000 users (Siggraph 1999)
Easy collaboration with strangers Users acted same as if handling real objects
Billinghurst, M., Poupyrev, I., Kato, H., & May, R. (2000). Mixing realities in shared space: An augmented reality interface for collaborative computing. In Multimedia and Expo, 2000. ICME 2000. 2000 IEEE International Conference on (Vol. 3, pp. 1641-1644).
Enhancing Face to Face Communication
Virtual Viewpoint Visualization
Mogilev, D., Kiyokawa, K., Billinghurst, M., & Pair, J. (2002, April). AR Pad: An interface for face-to-face AR collaboration. In CHI'02 extended abstracts on Human factors in computing systems (pp. 654-655).
AR Pad Handheld AR device AR shows viewpoints Users collaborate easier Show communication cues
Changing Perspective CamNet (1992)
British Telecom Wearable Teleconferencing
audio, video Remote collaboration
Sends task space video Similar CMU study (1996)
cut performance time in half
WACL: Remote Expert Collaboration
Wearable Camera/Laser Pointer Independent pointer control Remote panorama view
WACL: Remote Expert Collaboration
Remote Expert View Panorama viewing, annotation, image capture
Kurata, T., Sakata, N., Kourogi, M., Kuzuoka, H., & Billinghurst, M. (2004, October). Remote collaboration using a shoulder-worn active camera/laser. In Wearable Computers, 2004. ISWC 2004. Eighth International Symposium on (Vol. 1, pp. 62-69).
View Through Google Glass
Always available peripheral information display Combining computing, communications and content capture
Ego-Vision Collaboration
Google Glass camera + processing + display + connectivity
Current Collaboration on Glass
First person remote conferencing/hangouts Limitations
Single POV, no spatial cues, no annotations, etc
Sharing Space: Social Panoramas
Capture and share social spaces in real time Enable remote people to feel like they’re with you
Key Technology
Google Glass Capture live image panorama (compass + camera) Capture spatial audio, live video
Remote device (desktop, tablet) Immersive viewing, live annotation
Capturing Space: Real World Capture
Hands free AR Portable scene capture (color + depth)
Projector/Kinect combo, Remote controlled pan/tilt
Remote expert annotation interface
Remote Expert View
Capturing Behaviours
3 Gear Systems Kinect/Primesense Sensor Two hand tracking http://www.threegear.com
Skeleton Interaction + AR
HMD AR View Viewpoint tracking
Two hand input Skeleton interaction, occlusion
Ghostman
Use AR to capture and overlay your actions into remote persons space Eg remote therapy
Looking to the Future
What’s Next?
Brain to Brain Control
Rajesh Rao, University of Washington First Brain to Brain control
System Architecture
Scaling Up
Seeing actions of millions of users in the world Augmentation on city/country level
AR + Smart Sensors + Social Networks
Track population at city scale (mobile networks) Match population data to external sensor data
medical, environmental, etc
Mine data to improve social services
Research Challenges How to convey emotion? How to measure empathy? Interface/interaction models? How to communicate emotion? Scaling up to city/country scale?
Conclusion
Harvard Grant Study $20 million, 75 years study
268 Harvard graduates 456 disadvantaged people Led by George Valliant
What makes us happy? warmth of relationships throughout
life have the greatest positive impact on "life satisfaction".
“The seventy-five years and twenty million dollars expended on the Grant Study points to a straightforward five-word conclusion: Happiness is love. Full stop.” George Valliant
Conclusions Empathic Computing
Sharing what you see, hear and feel
AR Enables Empathic Experiences Removing technology Changing perspective Sharing space/experience
Many directions for future research
More Information
• Mark Billinghurst – Email: [email protected]
– Twitter: @marknb00
• Website – http://www.hitlabnz.org/
![State of Augmented Reality, Virtual Reality and Mixed Reality · State of Augmented Reality, Virtual Reality and Mixed Reality [Microsoft Hololen] [Ready Player One] Augmented Reality](https://img.pdfslide.us/doc/110x75/5f82ab6da2d89130b90d78c7/state-of-augmented-reality-virtual-reality-and-mixed-reality-state-of-augmented.jpg)
