26
Virtual and Augmented Reality: Applications and Issues in a Smart City Context A/Prof Stuart Perry, Faculty of Engineering and IT, University of Technology Sydney

Virtual and Augmented Reality: Applications and Issues in ... · VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT 23 ... Proposal for low-cost scanning

  • Upload
    vuthien

  • View
    241

  • Download
    0

Embed Size (px)

Citation preview

Virtual and Augmented Reality: Applications and Issues in a Smart City Context

A/Prof Stuart Perry, Faculty of Engineering and IT, University of Technology Sydney

2

Overview

VR and AR Fundamentals

How can VR and AR enable Smart Cities of the future?

Challenges of Human Vision for VR/AR

Other Challenges for VR and AR in Smart Cities

Our Research

Conclusions

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

3

VR and AR Fundamentals

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

4

Virtual Reality

Artificial content replaces the real world.

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

http://maxpixel.freegreatpicture.com/photo-1663612

5

Augmented Reality

Artificial content is displayed in the context of the real world.

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

https://upload.wikimedia.org/wikipedia/commons/thumb/a/ae/ZSpace_200_with_students.jpg/1280px-ZSpace_200_with_students.jpg

6

Mixed Reality

Artificial content is displayed in the context of the real world and is directly affected by

the real world and might inform decisions about actions in the real world.

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

https://www.flickr.com/photos/caseorganic/4336163331

7

VR/AR Hardware

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Gear VR

Tablets and Phones

BPhone

8

How can VR and AR Enable Smart Cities of the Future?

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

9VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Tourism/Entertainment Navigation Education

Hạ Long Bay (Vietnamese: Vịnh Hạ Long, IPA: [Vịnh HạLong] ( listen)) is a UNESCO World Heritage Site and popular travel destination in Quảng Ninh Province, Vietnam.

Security

Disturbance

Vào !!!Blah Blah Blah BlaBlah Blah

Blah Blah Blah Blah Blah Blah

Blah Blah Blah Blah Blah Blah

Blah Blah

h

Data Visualisation and Urban Planning

https://commons.wikimedia.org/wiki/File:Scale_model_of_2020_Shanghai_at_Urban_Planning_Exhibition_Center.jpg

10

Challenges of Human Vision for VR/AR

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

11VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

The vergence-accommodation conflict is a major cause of visual discomfort and is inherent to most existing

3D/VR/AR systems.

• Careful design of content and device can help, but not fix entirely.

• Motion makes it worse.

• Differences in individuals mean that it can’t be easily fixed for everybody!

http://informationdisplay.org/IDArchive/2008/July/ConsequencesofIncorrectFocusCuesinStereoDis.aspx

Reproducing a Realistic Sense of Depth (I)

12VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Reproducing a Realistic Sense of Depth (II)

Standard 3D systems such as 3D TV, VR headsets and CAVE systems exploit only stereopsis to produce the sense of 3D.

• But this is only a single part of a very complicated system of cues that humans use to create 3D images.

• Except for special cases stereopsis in humans only works up to about 6m.

• Within this range and beyond humans use a number of monocular and binocular ques to infer 3D structure.

Understanding these cues will help us to build better systems that create an accurate feeling of depth given the

technological constraints.

By Diliff - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=5403298

13VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Reproducing Complex Materials (I)

https://www.flickr.com/photos/hiyashi/15545618831

https://commons.wikimedia.org/wiki/File:Foglio_-_David_Patchen_9416.jpg

Material Appearance

• How do we realistically represent the complex interactions of materials and light in the

real world?

• In augmented and mixed reality we need to have objects respond to a real environment.

14VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Reproducing Realistic Materials (II)

https://commons.wikimedia.org/wiki/File:Mori_Uncanny_Valley.svg

The “uncanny valley”. The

relationship between the

similarity of an object to a

human and the sense of

familiarity or ease of our

response to it.

https://commons.wikimedia.org/wiki/File:Repliee_Q2_face.jpg

15VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Reproducing Realistic ColourThere is a lot about colour in 2D that we still don’t understand well.

These illusions are not

clearly understood in

2D and they aren’t

present in most 2D

models of colour.

Do they happen in 3D

too?

16VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Reproducing Realistic Light

https://www.flickr.com/photos/wonderlane/12167006583

https://commons.wikimedia.org/wiki/File:Wynn_Hotel_Interior_-

_Outdoor_Pool_(21118189499).jpg

Lighting Virtual Environments

• How do we represent the range of lighting conditions in real life?

• How do we transition across environments with different light levels and colour

temperatures?

17VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Controlling Attention and Gaze

https://www.flickr.com/photos/stevendepolo/6228420376

How does depth, texture, colour and 3D structure work together to attract attention and

guide the reaction to content in VR/AR environments?

Salience and visual attention has an important impact on perceptions of quality, storytelling

and the sense of immersion.

18

Other Challenges for VR and AR in Smart Cities

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

19VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Interaction and Haptic Feedback

https://commons.wikimedia.org/wiki/File:ManusVR_Glove_2016.png

https://www.flickr.com/photos/bagogames/25845851080

• How do we control and

interact with VR/AR content?

• Pseudo-haptic illusions

• Avoiding fatigue

https://commons.wikimedia.org/wiki/File:Tom_Cruise_(Not_really).jpg

20VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Latency and Environment ComplexityReal life content can be extremely complex.

We need to compress content in ways that exploit perceptual aspects of the

human visual system.https://commons.wikimedia.org/wiki/File:Normal_map_example.png

21VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Content Creation

Creating artificial realistic 3D content is getting easier, but remains complex

and time consuming.

How can we make this easier and more intuitive?

https://commons.wikimedia.org/wiki/File:Working_with_Nodes_Blender.PNG

22VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

Importing Content

https://commons.wikimedia.org/wiki/File:VIUscan_handheld_3D_scanner_in_use.jpg

https://www.flickr.com/photos/merlijnhoek/17202022610

Still very difficult to capture objects with

complicated structure, material appearance,

or objects that may move during scanning.

23VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

UTS:PILAB: Who are we?

Perceptual Quality

•Virtual Reality

•Augmented Reality

•2D Image and Video Quality

•Lightfield Technologies

Material Appearance and Perception in 3D

Image Processing

New 3D scanning technologies focused on the capture of material properties and dynamic objects

Research Interests Image and Multimedia

Processing

Object Detection and Recognition

Colour Science and Human Vision

Perceptual Quality and Material Appearance

Psychophysics and Subjective Testing

Machine Learning, Scene Understanding and Image Matching

Salience and affective computing

3D scanning, modelling and processing

Skills

Prof Myriam

Amielh

Ind. A/Prof

Don Bone

A/Prof Stuart

Perry

Dr Min Xu

Dr Wenjing

Jia

Ind. Prof

Barry Drake

A/Prof Richard

Xu

Dr Eva Cheng

Dr Daniel

Franklin

Dr Nico Pietroni

Multimedia and Quality of Experience

3D Geometry Processing

Biomedical Image Processing

Dr Zenon

Chaczko

24VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

UTS:PILAB VR/AR Current Research

PILab Existing

Strengths

High fidelity scanning

of 3D Objects

Perceptually Pleasing

Compression of 3D

Content

Modelling and

Improvement of

Quality of 3D

Experience

Advanced Mesh

processing

PhD Project underway for

compression of 3D data for

VR applications

PhD project started on

innovative scanning

techniques to capture and

render material appearance

information.

Examining options

for projects in the

area of VR for

education

Proposal for low-cost scanning

solutions for everyday users

PhD project underway on

data visibility and

interaction in VR/AR

environments

PhD Project underway on

developing technology for 3D

scanning and mesh

parameterisation of

deformable objects

Representation inJPEG

PLENO point cloud Ad hoc

Group

PhD Project underway on 3D rendering

of urban environments

25

Conclusions

Smart Cities of the future should exist for humans, not machines

VR and AR technologies have a role to merge the real and data worlds

in ways that we can only imagine

However, these technologies must consider the human element to be

useful and effective

If we don’t understand the human element, we will never be able to

create a bright future for our children

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

26

Thank you!

VIRTUAL AND AUGMENTED REALITY: APPLICATIONS AND ISSUES IN A SMART CITY CONTEXT

https://upload.wikimedia.org/wikipedia/commons/thumb/a/ae/ZSpace_200_with_students.jpg/1280px-ZSpace_200_with_students.jpg