Multi-touch, Multi-display, and Perceptual

Research @

St Andrews Human-Computer Interaction

Laboratory (SACHI)

Miguel Nacenta

University of St Andrews

mans@st-andrews.ac.uk

http://nacenta.com

UNIVERSITY OF ST ANDREWS, SCOTLAND

UNIVERSITY OF ST ANDREWS, SCOTLAND

SACHI

ST ANDREWS HUMAN-COMPUTER

INTERACTION RESEARCH GROUP

Prof. Aaron Quigley Dr. Per Ola Kristensson Dr. Tristan Henderson Dr. Uta Hinrichs

Information

Visualization,

MDEs

Text Input,

Crowdsourcing,

Gestures

Networks,

Privacy

Interactive

Public

Displays

http://www.acm.org/UIST http://www.its2013.org

OCTOBER 6-11, ST ANDREWS

ACM User Interface Software and Technology ACM Interactive Tabletops and Surfaces

SACHI RESEARCH

Edge Interaction

Haptic Tabletop Puck

E-conic

Attention Switching

Diff Displays

Continuous Gesture Recognition

Memorability of Gestures

Applications (LADDIE)

Jan 2013

Multi-Touch

Multi-Display, Multi-user, Perception

Gesture Input

EDGE-INTERACTION: PROBLEM

Jan 2013

Amount of data grows continuously

Relational data often visualized in graphs

Large graph visualizations cluttered & difficult to explore

MULTI-TOUCH INTERACTION TECHNIQUE SET

Jan 2013

TouchPlucking

TouchPinning

TouchStrumming

TouchBundling

PushLens

SOLUTION: MULTI-TOUCH INTERACTION

TECHNIQUE SET

Jan 2013

PROBLEM: HAPTIC (FORCE) FEEDBACK

Very simple (vibration)

Or, Very Expensive

Jan 2013

HAPTIC TABLETOP PUCK

Jan 2013

HAPTIC TABLETOP PUCK

Jan 2013

PROBLEM: SIZE, ORIENTATION AND PERSPECTIVE

IN MULTI-DISPLAY ENVIRONMENTS

Jan 2013

E-Conic: Viewer-aware interface

Jan 2013

PROBLEM: ATTENTION SWITCH IN MULTI-

DISPLAY ENVIRONMENTS

Multiple-screens means more pixels

But distributing the visual interface has costs

We studied perceptual and cognitive consequences of distributing the interface

Jan 2013

STUDY OF ATTENTION SWITCH IN MDES

Visual attention switch is cognitively costly

Relevant for ‘Second Screen’ experiences

16

Mobile Hybrid Large Display

PROBLEM: WHAT HAS CHANGED?

Jan 2013

DIFF DISPLAY:

GAZE-AWARE CHANGE HIGHLIGHTING

Jan 2013

PROBLEM: REMOTE GESTURE RECOGNITION

Difficult to define many gestures

Current recognizers: single-hand

Still evolving area

Jan 2013

CONTINUOUS GESTURE RECOGNITION

A bimanual template-matching gesture interface for the Kinect

Assigns a posterior probability distribution over all template classes while the user is articulating the gesture

PROBLEM: GESTURE SET MEMORABILITY

Gestures to invoke commands in non-desktop

interfaces (e.g., browser)

Next Page

Save Bookmark, etc…

But which ones can be

remembered best?

Self-generated gestures

Pre-designed gestures

Jan 2013

STUDY OF THE MEMORABILITY OF

GESTURE SETS

Jan 2013

PROBLEM: HOW TO KEEP PEOPLE ENGAGED

WITH INFORMATION IN PUBLIC SETTINGS?

Increase in the amount of available public

displays

“Display blindness” is starting to emerge

What kind of interface and content is

appropriate for public settings?

Working with Historic Scotland

Jan 2013

LADDIE

Jan 2013

AND MUCH MORE…

Jan 2013

A

P

F

M R

rsα β

nT

ProjectorImage-Plane

SELECTED PAPERS

Jan 2013

Schmidt, S., Nacenta, M.A., Dachselt, R., and Carpendale, S. A set of multi-touch graph interaction

techniques. ACM International Conference on Interactive Tabletops and Surfaces, ACM (2010), 113–116.

Marquardt, N., Nacenta, M.A., Young, J.E., Carpendale, S., Greenberg, S., and Sharlin, E. The Haptic Tabletop

Puck: tactile feedback for interactive tabletops. Proceedings of the ACM International Conference on

Interactive Tabletops and Surfaces, ACM (2009), 85–92.

Ledo, D., Nacenta, M.A., Marquardt, N., Boring, S., and Greenberg, S. The HapticTouch toolkit: enabling

exploration of haptic interactions. Proceedings of the Sixth International Conference on Tangible, Embedded

and Embodied Interaction, ACM (2012), 115–122.

Nacenta, M.A., Sakurai, S., Yamaguchi, T., et al. E-conic: a perspective-aware interface for multi-display

environments. Proceedings of the 20th annual ACM symposium on User interface software and technology,

ACM (2007), 279–288.

Rashid, U., Nacenta, M.A., and Quigley, A. The cost of display switching: a comparison of mobile, large display

and hybrid UI configurations. Proceedings of the International Working Conference on Advanced Visual

Interfaces, ACM (2012), 99–106.

Kristensson, P.O., Nicholson, T.F.W. and Quigley, A. 2012. Continuous recognition of one-handed and two-

handed gestures using 3D full-body motion tracking sensors. In Proceedings of the 17th ACM International

Conference on Intelligent User Interfaces (IUI 2012). ACM Press: 89-92.

Nacenta, M.A., Kamber, Y, Qiang, Y and Kristensson, P.O. Memorability of Pre-designed and User-defined

Gesture Sets (2013). Full paper in Proceeding of the thirty-first annual ACM SIGCHI conference on Human

factors in computing systems CHI ’13 (to appear).

CREDIT TO STUDENTS AND COLLABORATORS

TOO… Campbell Cromar (ADS Scotland)

Carl Gutwin (Prof. U Saskatchewan)

Ehud Sharlin (Prof. U Calgary)

Jakub Dostal (PhD St Andrews)

Jim Young (Prof. U Manitoba)

Nic Marquardt (PhD U Calgary)

Raimund Dachselt (Prof. U Magdeburg)

Satoshi Sakurai (PhD U Osaka)

Saul Greenberg (Prof. U Calgary)

Sebastian Schmidt (Diplom U Magdeburg)

Sheelagh Carpendale (Prof. U Calgary)

Sriram Subramanian (Prof. U Saskatchewan)

Tokuo Yamaguchi (Ph D U Osaka)

Umar Rashid (PhD St Andrews)

Yemliha Kamber (MSc St Andrews)

Yizhou Qiang (MSc St Andrews)

Yohei Miki (Msci U Osaka)

Yoshifumi Kitamura (Prof. U Osaka)

Yuichi Itoh (Prof U osaka)

Jan 2013

SACHI – UIST – ITS – MIGUEL

sachi.st-andrews.ac.uk

@SACHI_research

www.acm.org/uist/

@UIST2013

www.its2013.org

@ITS_2013

www.nacenta.com

@miguelnacenta

Jan 2013