View
2
Download
0
Category
Preview:
Citation preview
Virtual and Augmented Reality – 2017
MIECT, MIEET, MEI
Introduction to course
Universidade de Aveiro
Departamento de Electrónica,
Telecomunicações e Informática
Realidade Virtual e Aumentada Beatriz Sousa Santos, 2017/2018
2
Virtual and Augmented (Mixed) Reality
• Has been expanding from a research field into a commercially viable
technology.
• is highly interdisciplinary, combining work in diverse disciplines:
- more technology oriented research:
computer graphics, display technology, computer vision,
sensors, signal processing and computer networking.
and
- human centered research:
interaction techniques, design, human factors,
wearable computing and mobile computing.
• The growing interest in VAR applications is creating new challenges
for research in all of these areas
3
What is Virtual Reality?
“For better or worse, the label virtual reality stuck to this particular branch of computer graphics.
I define a virtual reality experience as any in which the user is effectively immersed in a responsive virtual world. This implies user dynamic control of viewpoint.”
“A high-end user-computer interface that involves real-time simulation and interaction through multiple sensorial channels (vision, sound, touch, smell, taste)”.
(Fred Brooks, 1999)
4
(Burdea et al., 2004)
Virtual Reality in practice for years – examples
Industry
Jaguar / Land Rover
Ford
http://www.youtube.com/watch?v=j3qcnvgVlNk
https://www.youtube.com/watch?v=BEFfp2QhHZ
U
5
Medicine
http://www.youtube.com/
watch?v=zJmrcEM-uvA
7
Entertainment
http://www.oculusvr.com/
Oculus Rift
2014; ~300 USD
8
http://www.youtube.com/watch?v=N8uuDT5AYts
“AR is a novel human machine interaction tool that overlays computer-
generated information on the real world environment. The information display
and image overlay are context-sensitive, which means that they depend on the
observed objects.” (Ong et al., 2008)
10
What is Augmented Reality?
• According to Azuma Augmented Reality systems have the following
characteristics:
“1) Combine real and virtual
2) Interactive in real time
3) Registered in 3D”
11
http://www.youtube.com/watch?v=8UxWkZtUKaI
Early Augmented Reality – examples
Lego
12
Ray Ban Virtual Mirror
13
http://www.youtube.com/watch?v=Ag7H4YScqZs
http://www.ray-
ban.com/usa/virtual-mirror
Augmented Reality Sandbox
14
http://idav.ucdavis.edu/~okre
ylos/ResDev/SARndbox/
http://www.techradar.com/reviews/wearables/microsoft-hololens-1281834/review
The Future of AR?
Hololens
2016; ~3000 USD
https://www.microsoft.com/microsoft-hololens/en-us
R. Furlan,”The future of augmented reality: Hololens - Microsoft's AR headset
shines despite rough edges”, IEEE Spectrum, 2016, 53 (6), p. 21
Mixed Reality: “Continuum”
“Augmenting natural feedback to the operator with simulated cues”
16
(Milgram et al., 1994)
Mixed Reality (MR)
Real Envinoment Augmented Reality Augmented Virtuality Virtual Reality
(Steinicke et a., 2009) http://www.youtube.com/watch?feature=endscreen&v=UgQfo7eNFdw&NR=1
Team
Beatriz Sousa Santos; Paulo Dias
(bss@ua.pt; paulo.dias@ua.pt)
18
Topics
• Introduction to Virtual Reality (VR) and Augmented Reality (AR)
Definition, historical perspective and evolution, examples
• Modeling of Virtual Environment (VE)
• Input and output devices, tracking, cameras
• 3D user interfaces and interaction techniques
• Human factors in VR and AR
• Frameworks for VR and AR
• Traditional and emerging applications
http://sweet.ua.pt/bss
20
Lectures and Lab classes (tentative)
1 (15/09/17) - Introduction to the course
Frameworks para RV e RA / VR and AR Frameworks
2 (22/09/17) - Introduction to Virtual and Augmented Reality
Paper presentation Guidelines
Unity
3 (29/9/17) - Introduction to Virtual and Augmented Reality (cont.)
Unity
4 (6/10/17) - Input Devices / Paper presentation
Unity
5 (13/10/17) – Output Devices (visual) - Stereoscopy / Paper presentation
Unity - Practical assignment
6 (20/10/17) - Output Devices (other) - Paper presentation
Unity – Practical assignment
21
7 (27/10/17) - Interaction in VR and AR - Paper presentation
Unity or Introduction OpenCV
8 (3/11/17) - Cameras and world registration - Paper presentation
Camera calibration with OpenCV
9 (10/11/17) - Human Factors in VR and AR - Paper presentation
Toolkits for AR (ARToolkit and others)
10 (17/11/17) - VR and AR Hardware
Hardware Demo
11 (24/11/17) - Assignments: mid-term demo and presentation
12 (15/12/17) - Evaluation methods
Unity – Practical assignment
Assignments: demo and presentation (tbd)
22
Lectures and Lab classes
• ~1h30 min – lecture; including paper presentation and discussion
• ~1h 30 min – lab
• 2 sessions devoted to presentation and demo of the assignments
Unity, OpenCV, ARToolkit ....
23
Main bibliography
• Jerald, J., The VR Book: Human-Centered Design for Virtual Reality, ACM and
Morgan & Claypool, 2016
• Schmalstieg, D., Hollerer, T., Augmented Reality: Principles and Practice (Usability).
Addison-Wesley Professional, 2016
• Craig, A., Sherman, W., Will, J., Developing Virtual Reality Applications:
Foundations of Effective Design, Morgan Kaufmann, 2009
• Burdea, G., P. Coiffet, Virtual Reality Technology, 2nd ed., Wiley, 2003
• Bowman, D., E. Kruijff, I. Poupyrev, J. LaViola, 3D User Interfaces: Theory
and Practice, Addison Wesley, 2004
• Anthes, C., M. Wiedemann, D. Kranzlmüller, “State of the Art of Virtual Reality
Technology”, IEEE Aerospace Conference, 2016.
• Journal and Conference Papers
24
Other bibliography
• Kim, G., Designing Virtual Reality Systems: The structured approach, Springer 2005
• Mullen, T., Prototyping Augmented Reality, Sybex, 2011
• Furht (ed.), Handbook of Augmented Reality, Springer, 2011
• Stanney, K.(ed.), Handbook of Virtual Environments, Design, Implementation and Applications, Lawrence Erlbaum Associates, 2002
• Journal and Conference Papers
25
- Exam – 40%
- Programming assignment – 20% + 30%
- Paper presentation and discussion – 10%
• Working students must decide on the type of assessment and inform us until September, 30
Assessment
26
Practical assignment (general characteristics)
• Groups of two students
• Mini-project
• Development of a simple application using specific devices and
libraries
• Existing code may be used, provided that it is explicitly identified
and its source adequately cited
28
This year’s possible “client”:
29 http://www.tsf.pt/portugal/saude/interior/hospital-rovisco-pais-e-central-na-recuperacao-de-
doentes-com-avc-4486208.html
Our devices
30
Analyze and present a conference/ journal paper:
1 student – short paper (~4 pages)
2 students– long paper (8+ pages)
Each student must:
- Decide if he/she will present alone or with a colleague
- select two papers
or send preferences to (bss@ua.pt)
until September, 30
- Propose another paper
- Read the paper presentation guidelines (presentation guidelines)
- Make a 12 mim (short paper) or a 25 min (2 students - long paper) presentation
- Send the slides to (bss@ua.pt)
31
Papers to read, present and discuss
http://ismar.vgtc.org/
32
http://ieeexplore.ieee.org/Xplore/
http://dl.acm.org/
http://www.springer.com
/computer/image+proce
ssing/journal/10055
And other
journals or
conferences
Discuss papers
• After each paper presentation:
- All students vote on their colleagues presentation
(through googleforms)
- Two students will discuss the paper
33
https://docs.google.com/forms/d/1rXAb2LlJTaj74-
xWMhhYMwWcNoUlzBUiyY9gZp2OFT0/viewform
• Students’ profile?
• Questions?
35
Interesting recent papers and links
• Rod Furlan,”The future of augmented reality: Hololens - Microsoft's
AR headset shines despite rough edges”, IEEE Spectrum, 53 (6), p.
21, DOI: 10.1109/MSPEC.2016.7473143
36
Recommended