Upload
dennis-mosley
View
212
Download
0
Tags:
Embed Size (px)
Citation preview
Research Area B
Leif Kobbelt
Communication
System
Interface
Research Area B
2
Research Area B
3
Interface
System
Communication
A
C
D
• definition of „application“ within UMIC• find new and relevant applications (killer app)
• combine existing technology
• identify application profiles
• types of data
• amount of data
• latency requirements
• input / output devices
• golden demo• good : convey the message
• bad : no basis research, commercial competitors
4
Research Area B
Future Mobile Applications
5
fund
amen
tal
algor
ithm
s syste
m
desig
n
evalu
ation
com
mer
cializ
ation
Future Mobile Applications
6
fundam
enta
l
algorit
hms
syst
em
desig
n
eval
uatio
n
com
mer
cializ
ation
Future Mobile Applications
7
fundam
enta
l
algorit
hms
syst
em
desig
n
eval
uatio
n
com
mer
cializ
ation
communication
it security
computer graphics
computer vision
Future Mobile Applications
8
fundam
enta
l
algorit
hms
syst
em
desig
n
eval
uatio
n
com
mer
cializ
ation
software engineering
security
computer graphics & vision
interface design
Future Mobile Applications
9
fundam
enta
l
algorit
hms
syst
em
desig
n
eval
uatio
n
com
mer
cializ
ation
graphical UIs
prototypes
user studies
...
Visual Computing for Future Mobile Applications
Bastian Leibe
Target Scenario: Pedestrian Navigation
11
Aachen Cathedral
Mobile visual search Simply point the camera to any object/building of interest. Images are transmitted to a central server for recognition.
Target Scenario: Pedestrian Navigation
12
Aachen Cathedral
Mobile visual search Simply point the camera to any object/building of interest. Images are transmitted to a central server for recognition. Object-specific content is sent back to for visualization on the
mobile phone (mobile AR).
LocalizeMe Demo
13
P. Steingrube, T. Weyand, T. Sattler, A. Schmitz, B. Leibe, L. Kobbelt
Mobile Service Structure
14
Localization Service
Image Database
3D Model
Render Server
Information Service
Cultural Database
Internet
Compound
Application
MobileClient
ServerServerServer
User Interface
Localization: Large-Scale Image Matching
How can we perform this matching step efficiently?
15
Database with thousands (millions) of images
??
Mobile photo
Localization: Large-Scale Image Matching
16
Database with thousands (millions) of images
Mobile photo
Local features(~1000 per image) …
“Visual vocabulary”(~1M feature clusters)
Shortlist of candidate matches(~100 images)
Shortlist of candidate matches(~100 images)
Matching (nearest neighbors in 128D space)
Localization: Geometric Verification
17
Mobile photo (for each image in shortlist)
xAj
xBj
XjAssumption: corresponding 3D structure
T. Sattler, B. Leibe, L. Kobbelt, SCRAMSAC: Improving RANSAC’s Efficiency with a Spatial Consistency Filter. International Conference on Computer Vision, 2009.
Find a rigid geometric transformation to verify that the matched features correspond to the same 3D structure. Problem: efficient processing with many outliers.
Summary: Visualization Service
18
Image
Local Feature
Extractor
Local Feature
Database
Image Database
Feature Matching
ImageImageCandidate Matches
Geometric Verificatio
n
DetermineLocation
On mobile device
On server side
Mobile Service Structure
19
Localization Service
Image Database
3D Model
Render Server
Information Service
Cultural Database
Internet
Compound
Application
MobileClient
ServerServerServer
User Interface
World-Scale Mining for Content Creation
20
e.g. Wikipedia match
Mining geotagged images Extracted Image clusters
Automaticannotation
& verification
Frequent tags
Example: Automatic Landmark Detection
21
Matched images for Aachen city hall (subset)
T. Weyand, B. Leibe
How Does This Scale?
Feasibility study Pairwise matching on 500,000 geotagged images of Paris How many matching images can we find at a certain location?
Touristic sites and central roads are well-covered.
22
T. Weyand, B. Leibe
Mobile Service Structure
23
Localization Service
Image Database
3D Model
Render Server
Information Service
Cultural Database
Internet
Compound
Application
MobileClient
ServerServerServer
User Interface
Virtual City Model
24
Floor Plan Map (2D)
Street Graph (2D)
Height Field(3D)
Synthetic Textures
Photographic
Textures
Building Model
Landscape Model
Optimized Octree Data
Structure
Estate Plan(2D)
Results: Virtual Aachen Model
25
G. Fabritius, J. Kraßnigg, L. Krecklau, C. Manthei, A. Hornung, M. Habbecke, L. Kobbelt
Alternative 1: Mobile Rendering
26
Rendering quality ona regular PC
C. Schreder, A. Schmitz, L. Kobbelt
Mobile rendering Very limited memory! Need to precompute
octree structure Dynamic transmission
of geometry & textures
Alternative 2: Stream Rendering
27
Rendering on server
Internet
Current prototype Transmit images in
UDP packages Tradeoff: compression
vs. framerate
Mobile Service Structure
28
Localization Service
Image Database
3D Model
Render Server
Information Service
Cultural Database
Internet
Compound
Application
MobileClient
ServerServerServer
User Interface
Application Study: Looking Through Time
How did Aachen’s cathedral look in past centuries? Camera phone as a “magic lens” to reveal past building states How should the interface be designed for such an application? Mock-up prototype (using Motion Capture system for tracking) Evaluation in user studies
29
T. Palm, J. Borchers
Research in Novel Interaction Techniques
30
T. Karrer, M. Weiss, M. Wittenhagen, G. Herkenrath, J. Borchers
TWEND Twisting and bending as new
interaction gestures in futuremobile devices
E.g. for interaction with anelectronic map or an e-book.
PocketDRAGON Directly drag objects along their
movement trajectory to preciselynavigate to a specific point in avideo sequence
Mobile implementation madepossible through communicationwith server