Embed Size (px)
Citation preview
Creativity SupportIn Projection-based Augmented EnvironmentsBruno Simões, Federico Prandi, and Raffaele de Amicis (et.al.)F. Graphitech
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
Framework
Problem Statement
Introduction &Context
Conclusions12
3
4
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
AgendaPath of Enlightenment
LowCost
CreativityCommunication
Culture
Content CreationCollaboration
Crowdsourcing
to develop new creative tools (based on computer vision techniques and pico-projection) that can transform the surrounding environment into an user interface
Introductionc-Space Objective
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
LowCost
CreativityCommunication
Culture
Content CreationCollaboration
Crowdsourcing
to develop new creative tools (based on computer vision techniques and pico-projection) that can transform the surrounding environment into an user interface
Introductionc-Space Objective
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
What is Creativity?A. Einstein
“seeing what everyone else has seen, andthinking what no one else has thought”
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
IntroductionWhat is Creativity?
What is Creativity?Betty Edwards
“creativity is the ability to find new solutionsto a problem or new modes of expression;thus it brings into existence something newto the individual and to the culture.”
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
IntroductionWhat is creativity?
Presentation and InteractionCrowdsourcing and gamified data collection 4D Reconstruction
Introductionc-Space State-of-Art
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
Introductionc-Space Objective
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
What About Creativity
Introductionc-Space Objective
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
What About Creativity• Knowledge transfer: How mechanisms implemented promote interaction between users• Germinality: how well does it open up to new perspectives• Does it really new ways of expression
Introductionc-Space State-of-Art Limitations
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
Limitations:- Simple user experience (AR)- 2DoF (storylines) - Ergonomics- Embarrassment factor- …
to develop new creative tools (based on computer vision techniques and pico-projection) that can transform the surrounding environment into an user interface
Culture LowCost
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
Crowdsourcing
Creativity
Content CreationCollaboration
Communication
Introductionc-Space Objective
to develop new creative tools (based on computer vision techniques and pico-projection) that can transform the surrounding environment into an user interface
Display limitation ?
Projection-Based Augmented RealityUniting physical and virtual objects
“combination of physical reality with computergenerated data projected directly within or on theuser's physical space”
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
IntroductionProjection-based AR
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
IntroductionProjection-based AR
Projection Mapping vs Traditional ProjectionUses the projection surface as a larger screenCustomizes the appearanceof an object/environment
Spatial AR
uses real physical objects to both represent and interact with computer-generated
information
Tangible User Interfaces (TUI)+
IntroductionPossible Advantages
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
• Real objects can be physically handled • Does not require see-through displays or to wear a head-mounted display• Multi-user experience • Facilitates co-located collaboration• Ergonomics, e.g. might reduce physical stress
1. Visual creation of augmented environments and automatic recomputationof projection maps on camera movementsProjective augmented reality using mobile devices raises calibration issues due to mobility that must be addressed.
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
Research ProblemProjection-based AR
2. Automatic Background Color CompensationUnfortunately, without accommodating for the different background colors, the user may be subject to viewing inaccurate colors and reduced contrast
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
Research ProblemProjection-based AR
3. Visual Pollution, Privacy & Human SafetyThe invasiveness of projected content can lead to "visual pollution" or bring annoyance to other people in the vicinity. Additionally projected content can blind people.Projective interfaces exposes the end-user to the risk of projecting sensitive information, e.g. PM popups.
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
Research ProblemProjection-based AR
4. Find effective new forms of user interactionWe want the surrounding environment itself to be transformed into an interface for the users to interact through gestures, wearables or simply by moving around.
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
Research ProblemProjection-based AR
Collaboration and Information Exchange!
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
FrameworkResearch Problem 1
Solution to visually create new augmented environments and to automatically re-compute projection maps on camera movements
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
FrameworkPPP Setup
Marker-less tracking, perspective correction, and safety monitoringcomputer generated content real worldPico-projector
processing unit + WIFICamera
projection
Setup
(Client
)
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
FrameworkPPP Setup
Server
Client-S
erver
real world
Normative policies, collaboration features
descriptors, video samples, 3D mapping setups, visible objects
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
FrameworkMobile Editor
Setup
project
ion ma
pping
Mapping handlersVisual Object
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
FrameworkPPP Procedure
Client-S
erver
Device
AROutput
QueryState
Object
UpdateApplication
Logic
RenderGraphics
Cloud/DeviceTarget DatabaseCamera CameraFrame
Pixel FormatConversion
ConvertedFrame
User-DefinedTargets
ImageTargets
FrameMarkers
Multi-ImageTargets
CylinderTargets
WordTargets
VirtualButtons
Detect New ObjectsTrack Detected ObjectsEvaluate Virtual Buttons
RenderCameraPreview
ServerServer
Projector
calibration& perspective correction
collaboration
BCC Descriptors Stack
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
FrameworkBackground Color Compensation
Render
ing
Lights Data
ColorCorrectionSystem
CameraFrameBuffer
ColorCorrectionFrameBuffer
LightDetectionSystem
Gamma function Estimation, Homer H. Chen
How can we create an easy-to-use normative policy to automatically regulate the use and the power of PPP technology?e.g. streets, where drivers or passengers could be temporarily blinded by the projection.
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
FrameworkResearch Problem 3
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
FrameworkResearch Problem 3
imagga
Automated image tagging and keywording
During the initial user trials, a small group of participants reported:1. No difficulty in learning the techniques that were proposed to setup a 3D projection mapping2. They found appealing the ability to combine information in a shared workspace.3. Found useful the feature to download content to their smartphones from projected animated QR codes4. Picking sometimes was difficult to archive
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
ConclusionsTest Case Results
1. Current pico-projectors have a luminance between 5 and 100 lumens. Hence, these projectors can only be used indoors or outdoors at night.2. The use markerless descriptors under certain conditions can be an issue:1. Poor illumination of the environment2. Surfaces with few features, e.g. white halls, etc.3. Projection without auto-focus can be disruptive
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
ConclusionsBiggest Limitations
1. Blending multiple projections into a seamless projection2. Restrict the projection in such way that it will never hit people in the face, but while keeping the projecting around the person.3. Better way to impose normative policies: executed at a real time frame rate.4. Explore interaction concept with drones, e.g. Wristwatch Drone.5. Drop the back-server requirement when the use of QR codes is possible
c-SpaceGrant agreement no: 611040
fostering new creative paradigms based on recording and sharing “casual” videos through the internet.The 2nd International Conference on Augmented and Virtual Reality (SALENTO AVR 2015)
ConclusionsFuture Work
