A Framework for Perceptual Studies in Photorealistic

Augmented RealityMartin Knecht1, Andreas Dünser2,

Christoph Traxler1, Michael Wimmer1 and Raphael Grasset3

2 HIT Lab NZ

University of Canterbury

3 HIT Lab NZ / ICG

University of Canterbury /Graz University of Technology

1 Institute of Computer Graphicsand Algorithms

Vienna University of Technology

Martin Knecht 2

Introduction

MotivationPerceptual IssuesFrameworkPilot StudyLimitationsFuture WorkConclusion

Martin Knecht 3

Motivation

Is this a photograph or computer generated?What visual factors affect your decision? Rademacher P., et al. (2001)

Courtesy of Rademacher et al. (2001)

Martin Knecht 4

Motivation

People are very good in judging if an image looks photo-realistic or notBut process behind it not fully understood!

In augmented reality (AR) virtual objects should be rendered in a photorealistic way

Goal: A framework that allows us to study photorealistic rendering techniques

at real-time frame rates

Martin Knecht 5

Perceptual Issues

Kruijff E., et al. (2010) distinguish between two types of problems:

Technological limitationsPerceptual nature

Proposed framework focuses on perceptual issues

Martin Knecht 6

Perceptual Pipeline

Perceptual pipeline used to classify issuesKruijff E., et al. (2010)

Environment

Capturing

Augmentation

Display Device

User

AR Framework

Martin Knecht 7

Capturing / Augmentation Stages

Capturing: Covers the process of converting the optical image to a digital one Kruijff E., et al. (2010)

Image resolution, lens distortion,exposure, color correctness,…

Augmentation: Deals with adding the virtual objects into the augmented sceneKruijff E., et al. (2010)

Registration errors, occlusion, rendering,…

Martin Knecht 8

Framework

Our contribution:Framework to study photorealistic rendering techniquesInteractive tasks with global illuminationThree different rendering modesFast prototyping of experiments (XML)A pilot study to test framework

Martin Knecht 9

Framework

Differential instant radiosity is used to render the scenes Knecht M., et al. (2010)

Rendering modes can be changed during run-time

Mode A: shadows / indirect illuminationMode B: shadows / indirect illuminationMode C: shadows / indirect illumination

Martin Knecht 10

Rendering Mode A

Different rendering modes are supportedNo shadowsNo indirect illumination

Martin Knecht 11

Rendering Mode B

Different rendering modes are supportedWith shadowsNo indirect illumination

Martin Knecht 12

Rendering Mode C

Different rendering modes are supportedWith shadowsWith indirect illumination

Martin Knecht 13

Pilot Study

Pilot study to evaluate our frameworkTwenty-one participants (15 male, 6 female)Age between 19 – 595 tasks

Martin Knecht 14

Task 1: Estimate distance between real and virtual cubes

Pilot Study

Martin Knecht 15

Task 2: Place real cube at position of virtual one

Pilot Study

Martin Knecht 16

Task 3: Place virtual cube at position of real one using computer keyboard

Pilot Study

Martin Knecht 17

Task 4: Grab and lift real cube

Pilot Study

Martin Knecht 18

Task 5: Grab and lift virtual cube - real cube was used for tactile feedback

Pilot Study

Martin Knecht 19

Pilot Study

Duration: 30 to 60 minutesInterview followed afterwardsMeasured

Distance error for tasks 1, 2, 3Time for tasks 2, 3, 4, 5

Analyzed data using non-parametric Friedman testsResult: No significant effect of rendering

technique found

Martin Knecht 20

Pilot Study

Martin Knecht 21

Pilot Study

Martin Knecht 22

Pilot Study

Distance estimation perceived easier by 6 participants in left/right and height direction than in depth

Although no significant effect was found

Martin Knecht 23

Pilot Study

Participants made heavy use of occlusion cue to place cubes in Task 2 & 3 – (19 out of 20)

Martin Knecht 24

Pilot Study

7 participants preferred manipulation with a computer keyboard (Task 2 vs. Task 3).

Martin Knecht 25

Limitations

No support for stereo renderingNo distance information of environmentOnly video see-through HMDs are supportedTone-mapping still needs manual fine tuningNo support for mobile devices

Martin Knecht 26

Future Work

Enhance rendering frameworkSupport for stereo renderingAdd camera artifacts

New studies with new tasksTasks without occlusion informationUse chin rest to reduce movementDesign new tasks

Martin Knecht 27

Conclusion

We proposed a research test-bed for perceptual studies – still work in progress

Different rendering modes allow new kind of experiments

We showed the results of a pilot study using our framework

Thank you for your attention!What features would you like to have in such a framework?

28Martin Knecht