View
374
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Vision-based approaches are a promising method for indoor navigation, but prototyping and evaluating them poses several challenges. These include the effort of realizing the localization component, difficulties in simulating real-world behavior and the interaction between vision-based localization and the user interface. In this paper, we report on initial findings from the development of a tool to support this process. We identify key requirements for such a tool and use an example vision- based system to evaluate a first prototype of the tool.
Citation preview
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Tool Support for Prototyping Interfaces for Vision-Based Indoor Navigation
Andreas Möller1, Christian Kray2, Luis Roalter1, Robert Huitl1, Stefan Diewald1, Matthias Kranz3
1Technische Universität München, Germany
2University of Münster, Germany 3Luleå University of Technology, Department of Computer Science,
Electrical and Space Engineering, Luleå, Sweden
1st Workshop on Mobile Vision and HCI held at MobileHCI 2012, San Francisco
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Background and Motivation
• Location information still the most important contextual information
• Indoor localization is a hot topic and useful for a lot of scenarios: – Airports – Hospitals – Conference venues – Large environments
• Various indoor localization methods possible: – WLAN/cell based localization – Sensor based localization – Beacon based localization – Vision based localization
21.09.2012 Luis Roalter 2
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
21.09.2012 Luis Roalter 3
Why use vision based localization?
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Why vision based localization?
• Use existing features in the environment
• Modern devices are equipped with a camera and powerful processor
• No additional augmentation needed
21.09.2012 Luis Roalter 4
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
21.09.2012 Luis Roalter 5
Which challenges will appear?
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Challenges of Vision-Based Localization
• No fixed device needed – Use your own smartphone without any optional accessories – Hold the device as you like (landscape, portrait, to the ground, to the wall)
• Quality of visual features – Repeating furniture in one hallway – Wall of the same paint
• Orientation of the user’s smartphone – Features are typically found in eye height (windows, adverts, posters)
• In a typical pose, user holds the device in a way that it points at the floor
21.09.2012 Luis Roalter 6
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Challenges in HCI
• User interfaces need to support/address those problems: – Deal with accuracy – Corrective actions if not enough features are available
• Hard to evaluate (would require already functional localization, but interfaces goes hand in hand with algorithms)
• Little/no experience with developing user interfaces for vision-based systems
• There is a need for adequate prototyping tools!
21.09.2012 Luis Roalter 7
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
21.09.2012 Luis Roalter 8
Prototyping. What do we need?
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Prototyping Indoor Navigation Interfaces
• Requirements:
– Prototyping in early development
– Ease change of the location (e.g. choose a new environment)
– Evaluation of usability, efficiency and effectivity
– Controlled conditions of localization conditions (e.g. location and orientation accuracy)
– Evaluation of individual features in real-world scenarios
21.09.2012 Luis Roalter 9
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
The Wizard of Oz Approach
Experimenter‘s Phone(Server)
Subject‘s Phone(Client)
Send image directly OR
send image ID of local database Local Image DB
Trigger UI events
21.09.2012 Luis Roalter 10
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
User Interface Concept
• Use the camera of the smartphone
• Enable the usage of: – Virtual Reality (VR) with panorama images (compare to
Google StreetView) taken in advance – Augmented Reality (AR) with a live view of the environment
• In both cases the user needs to hold the camera device towards interesting regions.
• How to motivate the user to hold up the smartphone?
21.09.2012 Luis Roalter 11
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Augmented Reality • Display the live camera-image
• Draw information within the image
• Hold the smartphone in eye-height
• Live view of the environment
• Ease match from to the real environment
21.09.2012 Luis Roalter 12
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
21.09.2012 Luis Roalter 13
Virtual Reality • Preloaded panorama
images
• Similar images to the real environment
• Holding the smartphone to the ground
• No orientation change needed
• User has to match real and virtual reality
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Mockup
21.09.2012 Luis Roalter 14
• Testing the augmentation content
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
What if the algorithm needs a feature-rich region?
21.09.2012 Luis Roalter 15
• Motivate the user to look at the phone to follow the instructions (in both AR and VR conditions!)
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Theoretical Evaluation
• Evaluation based on the initial requirements – Use the prototype at early project stage (even before localization is
implemented) – Testing AR and VR interface concepts on a real smartphone – Inaccuracy possible due to manual transmission of images
– Different motivation screens to point upon a feature-rich region
• All initial requirements are met.
21.09.2012 Luis Roalter 16
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
• 5 subjects evaluating the mockup system • Predefined route through the building:
– About 100m – Dataset recorded with LadyBug
Camera system – Corridors and a hall including doors
Practical Evaluation
21.09.2012 Luis Roalter 17
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Findings of the first evaluation
• AR considered as nice and convenient • AR Overlay was not always accurate, felt disturbing
(in case of orientation error)
• VR had offset in location • But user can deal better with inaccuracy, matching panoramas to the real
environment
21.09.2012 Luis Roalter 18
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Interface Design and Usability
• Future research questions:
– Which visualization deals better with varying conditions?
– Automatic vs. manual adjustments by the user?
– Preferences for proactive behavior?
– Are VR and AR sufficient for navigation tasks?
21.09.2012 Luis Roalter 19
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Ongoing related work
• Test the application within a large environment (a larger route with multiple turns)
• Use dataset of a complete building recorded in advance (for VR)
• Evaluate the user‘s experience with the application user-interface
• Survey with 30 participants
21.09.2012 Luis Roalter 20
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Current Implementation
21.09.2012 Luis Roalter 21
Subject’s Screen (top) Drawing of images in AR or VR mode, connected to the server device
Experimenter’s Screen (right) Send specific images to the client
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Path trough the building
21.09.2012 Luis Roalter 22
• Including hallways, corridors and different turns
• Evaluate the user’s experience in case of errors
• Current research topic and related user study will be published soon
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Summary
• All initial requirements are met – Testing of the prototype without having an actual localization – Different views and augmentations are possible
• A larger study with a longer way is currently evaluated with the same prototype
• Full functional prototype framework available (for public on request)
• Wizard-of-Oz approach can be replaced step by step
21.09.2012 Luis Roalter 23
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Map
21.09.2012 Luis Roalter 24
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Thank you for your attention! Questions?
? ? 21.09.2012 Luis Roalter 25
[email protected] www.vmi.ei.tum.de/team/luis-roalter.html
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
Paper Reference
• Please find the associated paper at: https://vmi.lmt.ei.tum.de/publications/2012/mobivis_v02_preprint.pdf
• Please cite this work as follows: • Andreas Möller, Christian Kray, Luis Roalter, Stefan Diewald, Rorbert Huitl,
Matthias Kranz. 2012. Tool Support for Prototyping Interfaces for Vision-Based Indoor Navigation In: Workshop on Mobile Vision and HCI (MobiVis) on MobileHCI 2012, San Francisco, USA, September 2012
21.09.2012 Luis Roalter 26
Technische Universität München Distributed Multimodal Information Processing Group Prof. Dr. Matthias Kranz
If you use BibTex, please use the following entry to cite this work:
21.09.2012 Luis Roalter 27
@INPROCEEDINGS{Mobivis12moeller, author={Andreas M\"{o}ller and Christian Kray and Luis Roalter and Stefan Diewald and Robert Huitl and Matthias Kranz}, title={{Tool Support for Prototyping Interfaces for Vision-Based Indoor Navigation}}, booktitle={{Proceedings of the Workshop on Mobile Vision and HCI (MobiVis). Held in Conjunction with Mobile HCI}}, year={2012}, month=sep, location={San Francisco, USA}, abstract={Vision-based approaches are a promising method for indoor navigation, but prototyping and evaluating them poses several challenges. These include the effort of realizing the localization component, difficulties in simulating real-world behavior and the interaction between vision-based localization and the user interface. In this paper, we report on initial findings from the development of a tool to support this process. We identify key requirements for such a tool and use an example vision- based system to evaluate a first prototype of the tool.}, }