Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Dr.-Ing. Dipl.-Kfm. Christoph Runde
Franziska Jöckel
Virtual Dimension Center (VDC)
Fellbach
Auberlenstr. 13
70736 Fellbach
www.vdc-fellbach.de
Distributed Collaborative Virtual &
Augmented Reality VDC-Whitepaper
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
1. Introduction
2. Basics
2.1 Definition
2.2 Benefits
2.3 Fundamental Concepts
2.4 Network Architectures
2.5 Challenges and Solutions
3. Areas of Application
4. Market Overview
5. Summary & Outlook
Structure
19.09.2019 2
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
"Distributed work" is becoming more and more important: a study by Forrester
Research revealed that more than 80 percent of employees already work
regularly with people who are at a different location (eg home office, other
office building, another company location abroad, etc.).1
Interdisciplinary communication is becoming increasingly important: change in
the working environment (digitization, new ways of working, etc.); increasing
complexity of tasks, idea and decision-making processes; domain-specific
know-how as undocumented experiential knowledge
“Traditional alternatives" for distributed, interdisciplinary collaboration:
o On-site meetings: Disadvantage of high travel costs and times
o Remote meetings in the form of telephone and video conferencing:
especially when discussing 3D content lack of visualization options, lack of
group awareness
1 https://www.wiwo.de/erfolg/management-der-zukunft/arbeitsform-der-zukunft-so-fuehren-sie-virtuelle-teams/19385778.html
Starting Situation (1/2)
19.09.2019 3
Introduction
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Virtual (VR), augmented (AR) and mixed (MR) reality technologies enable
interactive, remote collaboration, and they are becoming increasingly
important in the enterprise context
Head-mounted displays (HMDs) are becoming cheaper and more powerful
The VR HMD market has been split equivalently until just a few years ago,
and now there is a spillover of consumer solutions with increased
performance in industrial applications; even game engines are now used
industrially A scenario in which multi-user environments / virtual rooms are
also used for meetings in the corporate context is conceivable
Technological trends such as the integration of eye tracking, face capture and
gesture recognition into current VR systems are an indicator that
interpersonal aspects, including emotions, should be mapped in the future
Starting Situation (2/2)
19.09.2019 4
Introduction
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Collaborative virtual / augmented environments are
virtual spaces where multiple users work together in
real time
In comparison “multi-user-VR": does not define, if an
interaction with each other or against each other
(especially in gaming context) takes place
Environments can be completely virtual (VR) or a
combination of virtual and real world (AR)
The goal is to bring people together in a collaborative
virtual environment, preferably from multiple locations,
in order to exchange experiences and ideas
Participants can be represented by avatars
Participants can communicate with each other
(linguistically, gesture) and interact with other
participants or virtual objects
Definition „Collaborative virtual / augmented environments“
19.09.2019 5
Image: Facebook
Social VR-platformFacebook Spaces
Image: Spatial.is
AR-collaboration, realistic avatarrepresentation bySpatial
Basics
Image: MiddleVR
VR productpresentation;connection ofdifferent devices
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Enables immersive meetings and workshops over
long distances
Enables exchange of information in real time, in
particular useful regarding visualization information of
3D models
Support of corporate groups and internationalization
activities
Reduction of travel costs and times
Acceleration of development cycles
Reduced error rates through increased and early
involvement of experts
Accelerated reactivity to incidents
Greatest benefit especially if the focus is on the
interdisciplinary planning of a complex object
Benefits
19.09.2019 6
Image: Eversheim 1995, S. 17
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
In most cases, a collaborative virtual environment
(CVE) system consists of several remote processes
that share the same set of data
The processes are usually distributed among many
(remote) computers connected through a network
A distributed system usually has the following
characteristics:
o Virtual environment data is replicated
o Passing on updates is time-critical
o Updates are asynchronous (eg non-blocking)
The replication of the data is important for performance reasons
Each computer that participates in the CVE simulation typically
renders the scene 30 to 100 times per second
The part of the scene being rendered must be read from
memory and sent to the rendering device
Distributed virtual environments: fundamental concepts (1/2)
19.09.2019 7
Image: Theoktisto, Fairén
Components of CVEs; yellow highlightedelements are referring directly tocollaboration aspects
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Latency: the delay at which data sent to receiver
arrives, depending on the data line (telephone, LAN,
satellite), distance
Bandwidth: amount of data that can be transmitted per
unit of time
Reliability:
o Loss of information
o Corruption of data
o Visibility of information by strangers
Network protocol: Rules by which two applications
communicate
o Data formats, semantics (meaning of the message), error
handling
o Many protocols in practical use: ftp, http, realtime audio,
etc.
o For communication between applications, many protocols
are used simultaneously
Distributed virtual environments: fundamental concepts (2/2)
19.09.2019 8
Virtual environmentwith 3 avatarsImage: Fraunhofer IPA
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Hardware compatibility
Data security, data hosting
Data preparation of the models to be visualized
Supported data formats
Mapping of physical laws
Communication: language, gestures, facial expressions
Interaction possibilities
Avatar representation and configuration
Usability / user interface
User management
Maximum number of users
Documentation of work results
Costs
Further important criteria regarding collaborative VR/ AR solutions
19.09.2019 9
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Core idea of distributed virtual environments:
o All participants are present in the same environment
o State must be consistent for all participants
Trade-off consistency-performance:
o Information is generated on one host and mirrored on another
o Outdated information through latency in the transmission
o Without synchronization, mirrored information can only be trusted to a
limited extent
o Consistent states can only be achieved by synchronization, but these only
without having high update rates Contradiction between speed and
consistency
Thus, distributed virtual environments are either fast-changing dynamic
worlds or consistent worlds that provide identical information to all hosts
Distributed virtual environments: State management (1/2)
19.09.2019 10
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Basically there are three approaches to state management:
1. Central information management
o State of the virtual environment is managed centrally
o Synchronized writing access
o Centre is the bottleneck, but can also distribute targeted
2. High-frequency information distribution
o Goal: Fast updates at the expense of network-wide consistency
o States of all hosts are sent completely and frequently to all other hosts
o Problem: Prevent multiple hosts from simultaneously manipulating the same object (eg via lock
manager)
3. State prediction
o Idea: Each host constructs approximations of the actual state between updates
o 2 phases: Prediction (dead reckoning) and convergence (in the actual state) after update
o Use of laws of physics, collision detection
19.09.2019 11
Distributed virtual environments: State management (2/2)
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
19.09.2019 12
Distributed virtual environments: Network architectures (1/2)
The most common network architectures:
Centralized Server: Centralized Primaries Modelo Based on the primary server replication model
o Primary scene is on a computer (server)
o All other computers are called clients and they only hold
the scene backup replicas
o When a client wants to update data, he must send an
update request to the server
o If the update is accepted after the scene consistency
check, the primary replica is updated and the message is
sent to all backup replicas to update their values
Peer-to Peer: Active Replicationo All computers replicate all collaborative scene data,
keeping the scenes fully synchronized
o Performance is limited by the slowest computer in the
system
Centralizedprimariesconsistencymodel
ActivereplicationconsistencymodelImage: Peciva, 2007
Image: Peciva, 2007
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
19.09.2019 13
Distributed Server (cf. concept Distributed
Scene Graph):
– Distributed Primaries Modelo Primary scene is distributed on the computers
o Different parts of the scene belong to different
computers and each computer is allowed to write
only in its part of the scene
– Data Ownershipo Primaries are allowed to migrate between computers
more flexible than Distributed Primaries Model
o If the computer owns a data item, it can read and
overwrite its values; the update will be sent to all
other computers
o Another computer may request access to data item
and if approved, also read and overwrite it
Distributed primariesconsistency model
Data ownershipconsistency model
Image: Peciva, 2007
Image: Peciva, 2007
Distributed virtual environments: Network architectures (2/2)
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Technology:
Costs and selection of technology
Compatibility software and hardware
Bandwidth, latency
Privacy / data security
Creation of 3D data
User requirements:
Build up know-how
Physical restriction, cyber sickness
Social acceptance
Further:
Structure of a VR meeting (preparation,
(temporal) procedure, etc.)
Uncertain benefit
Challenges regarding the implementation of collaborative V/AR applications
19.09.2019 14
Image: VDC Fellbach
Results of theDesign ThinkingWorkshop „Collaborative VR/ AR“
Basics
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Within the company
Live and promote a culture of change
Hiring VR specialists
Inteligent data policy
Ensure access to VR equipment
External
Set up of test environments / experimental rooms
Definition of standards
Publication of success stories
19.09.2019 15
Image: VDC Fellbach
Test environmentat VDC Fellbach
Image: VDC Fellbach
Image: VDC Fellbach
Approaches to implementing collaborative V/AR applications
Basics
Test environmentat VDC Fellbach
Test environmentat VDC Fellbach
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Usage scenarios are diverse and just as
different
Variety of possible properties require
careful selection and testing
Main application fields:
1. Entertainment: Multi-player games (not
included in the following)
2. Presentation
3. Distance training
4. Assistance / Remote Support /
Teleoperations
Areas of application
19.09.2019 16
Image: Innoactive
VR training in logistics @ Audi with Innoactive
Image: ESI
Image: Re´flekt
IC.IDO enablescollaborative andimmersive Design Reviews
Remote Service Support by meansof AR technology
Areas of Application
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
19.09.2019 17
Area of application: Presentation
Virtual, interactive presentation of a product
Complex and large products presentable
Securing a common understanding, reducing
misunderstandings
Enables easy demonstrating of variants
In product development, design reviews
(distributed development on the same subject,
intuitive and interactive exploration, testing
prototype savings, product quality improvement,
lead time reduction)
Marketing/ sales, market research (accelerate
sales processes, product reviews)
Layout planning, virtual inspections
Image: rumii
Image: WeAre
VR meeting / conceptpresentation(rumii)
VR productpresentation withannotations(WeAre)
Image: vr-on
Layout planning: virtual inspectionof a factory (vr-on)
Areas of Application
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
19.09.2019 18
Area of application: Training
Learning levels: explain, accompany,
examine
Introduction of documentation, simulators
Simulation scenarios
Visualization of the hidden
Training of dangerous situations
Training of processes in the team
Shortening downtime
Training already during planning / without
occupancy of the object / safe with
scenario technique
e.g. product trainings, process-oriented
trainings (eg assembly, surgery, etc.)
Image: University of Southern California
Assistant Steve explains andissuesinstructions
Image: vr-on
VR producttraining (vr-on)
Image: realworld-one
Training session(realworld-one)
Areas of Application
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
19.09.2019 19
Plant training simulator ofthe Virtual Reality und Multimedia Park Turin:Definition of scenarios,processing of scenarios,linked 2D and 3D-perspective
Image: Virtual Reality and Multimedia Park, Turin
Area of application: Training
Areas of Application
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
19.09.2019 20
Area of application: Assistance/ Remote Support
Use of a 3D environment as a knowledge
management platform to support remote
service personnel
Linking 3D environment to real system:
online 3D quickly conveys perspective of the
colleague on site
Use online 3D where cameras can not be
used (harsh environment, signal latencies)
On-site service staff can be supported with
3D information by AR systems
Image: Fh-IPA
Remote configuration of a machine tool
Image: Fh-IPA
Service technician in head office cansupport the on-site technician byoverlayingadditional information via a AR display
Image: tepcon
AR solution forsupport of servicingpurposes
Areas of Application
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
VR / AR use cases have been segmented according to their implementation
complexity and achievable benefits for the automotive industry, manufacturing
and supply sectors
Remote expert / support solutions can be found in all three segments in
the “Must Do" segment - bringing with them a high level of benefit
combined with a low level of implementation complexity
Remote collaboration for design agreements and similar is classified as a
“Must Do“ as well (with particularly high benefit in the automotive sector)
AR is generally perceived as more beneficial, though more complex than VR
applications
Repair and maintenance are at the heart of implementation efforts: using real-
time visualization and information to complete a maintenance or repair task
Areas of application: Evaluation (1/2)
19.09.2019 21
Areas of Application
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
19.09.2019 22
Image: Capgemini
Classification of „Remote Expert“ and „Remote Collaboration“ solutions as„Must Do“ (high benefitwith relatively lowcomplexity) – Example forthe automotive industry
Areas of application: Evaluation (2/2)
Areas of Application
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Presentation/ Design Review / Training
(VR / AR):
ESI
Innoactive
Lightshape
MiddleVR
Realworld One
Solid White
TechViz
vr-on
WeAre
WorldViz
VR-Meeting platforms:
Altspace VR
Facebook Spaces
VRChat
Market overview – Companies in this technological field
19.09.2019 23
Remote expert/ Assistance
applications (AR):
Atheer
CDM Tech
Ilogic
Re´flekt
Tepcon
TruPhysics
TZM
Wikitude
XM Reality
Market Overview
A detailed overview (over 60 solutions) of VR / AR
collaboration solutions can be found on the VDC
intranet.
A collection of specific application examples and
success stories will be published in the course of 2019.
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Communication:
o (Spatial) audio
o Recognition of viewing direction,
gesture
o Virtual laser pointer
o Virtual whiteboards
o Speech-to-text
o 3D drawing function
Import function of 3D objects, pdfs,
etc.
Variation of the meeting environment
Configuration of avatars
Viewpoint navigation
Screenshots
Example functions of collaborative V/ AR solutions
19.09.2019 24
Interaction with 3D models:
o Exploded view
o Cross section
o Coloring
o Measuring
o Moving/ Rotating
o Annotations
Structure/ organisation:
o Access management, permissions
o Role management (moderator/
presenter, participants, etc.): sharing
perspective, teleport groups, muting,
etc.
o Documentation of the sessions
Market Overview
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Use cases of distributed, collaborative VR / AR:
o Remote collaboration for presentation, meetings,
trainings
o Remote expert solutions
Top benefits:
o According to Capgemini: increasing efficiency and
productivity, increasing safety, saving time, reducing
complexity
o According to Deloitte: reduction of travel costs, increase
of flexibility and efficiency of work processes
Remote expert solutions are particularly promising
The magnitude of revenues generated in the future
depends heavily on developments in the hardware
sector (HoloLens & Co.)
Summary
19.09.2019 25
Summary
Image: Deloitte
VR/ AR study ofCapgemini andDeloitte
Image: Capgemini
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
19.09.2019 26
Outlook
Acquisition and transmission of gesture
as it is an important communication channel:
eg pointing direction, shrugging
Acquisition and transmission of facial
expressions important for face to face
communication: interpretation
Realistic avatars by scanning the
participants 3D projections in virtual or
remote conference rooms
Haptic feedback in specific interactions
increased immersion
Image: TU Chemnitz
Image: heise.de
Gesture recognitionin front of Powerwall (forscene navigation)
Kinect recordsmovements of mouthand eyebrows andthereby animatesavatars : wrinkling, nodding, smiling, lipmovements
Image: spatial.is
Realistic avatars, developed based on a 2D picture
Outlook
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Basics Areas of ApplicationIntroduction Market Overview Outlook
Sources:
WeAre Blogeintrag (2019):„How to conduct remote meetings more effectively“, verfügbar unter: https://weare-rooms.com/blog/
vr-on Whitepaper (2019): „Virtual Reality Kollaboration“
P. Ivanov (2017): „So leiten Sie virtuelle Teams“, verfügbar unter: https://www.wiwo.de/erfolg/management-der-zukunft/arbeitsform-der-
zukunft-so-fuehren-sie-virtuelle-teams/19385778.html
Capgemini Studie (2018): „Augmented and Virtual Reality in Operations: A guide for investment”, verfügbar unter:
https://www.capgemini.com/wp-content/uploads/2018/09/AR-VR-in-Operations1.pdf
Deloitte, Fraunhofer FIT, Bitkom (2016): „Head Mounted Displays in deutschen Unternehmen. Ein Virtual, Augmented und Mixed Reality
Check“ , verfügbar unter: https://www2.deloitte.com/content/dam/Deloitte/de/Documents/technology-media-telecommunications/Deloitte-
Studie-Head-Mounted-Displays-in-deutschen-Unternehmen.pdf
J. Peciva Dissertation (2007): „Active Transactions in Collaborative Virtual Environments“
VDC Fellbach Whitepaper (2013): „Collaborative Virtual Engineering“, verfügbar unter: https://www.vdc-
fellbach.de/fileadmin/assets/Publikationen/Whitepaper/2013_VDC-Whitepaper_Collaborative_Virtual_Engineering.pdf
V. Theoktisto, M. Fairén: „Enhancing Collaboration in Virtual Reality Applications”
Further literature:
S. Ren, F. D. Mckenzie (2015):„Collaborative Virtual Environment For Engineering Laboratory“
D. Livingstone, G. Creechan, J. Scullion (2013): „Learning about Collaborative Virtual Environments by Creating Collaborative Virtual
Environments“
J. Mütterlein, T. Hess (2018): „Specifics of Collaboration in Virtual Reality: How Immersion Drives the Intention to Collaborate“
C. Toinard (2000): „Patterns for Collaborative Virtual Reality”
E. Langbehn (2017): “Influence of Avatar Appearance on Presence in Social VR”
R. Rabätje, S. Menzel, M. Wochnig (2019): „Herausforderungen bei kollaborativen Anwendungen mit VR-HMDs“
D. Lake, H. Liu, M. Bowman (2010): “Distributed scene graph to enable thousands of interacting users in a virtual environment”
19.09.2019 27
Sources & Literature
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
VDC-members working in this technological field:
19.09.2019 28
© Competence Centre for Virtual Reality and Cooperative Engineering w. V. – Virtual Dimension Center (VDC)
Thank you very much for your interest!
You are interested in this topic and you are looking for contact persons/
implementation partners? Please contact us.
Dr.-Ing. Dipl.-Kfm. Christoph Runde
Virtual Dimension Center (VDC)
Fellbach
Auberlenstr. 13
70736 Fellbach
www.vdc-fellbach.de