Virtual ExperiencesVirtual ExperiencesCourses and ResearchCourses and Research

Benjamin LokBenjamin Lok

March 22March 22ndnd, 2005, 2005

OutlineOutline

TopicsTopics• Interactive Computer GraphicsInteractive Computer Graphics• Human Computer InteractionHuman Computer Interaction• Virtual RealityVirtual Reality

JobsJobs ResearchResearch SkillsSkills

Interactive Computer GraphicsInteractive Computer Graphics

A d ob eP h otosh op

W eb D es ig n

G rap h ics D es ig n

V id eo G am es

F lig h tS im u la to rs

S c ien tificV isu a liza tion

D ataV isu a liza tion

A p p lica tion s

V irtu a lR ea lity

R esearch

R ea l-T im e

TVL og os

A n im ation

M ovieS p ec ia l E ffec ts

N on R ea l-T im e

3 D G rap h ics

C om p u ter G rap h ics

MotivationMotivation

We want to understand how to do:We want to understand how to do:

MotivationMotivation

Why should I take this course?Why should I take this course? An excuse to build that An excuse to build that

program/game/project program/game/project you always wantedyou always wanted

A Computer Science A Computer Science course that involves a course that involves a variety of skills (other variety of skills (other than coding!)than coding!)• ArtArt• PsychologyPsychology• SoundSound• CreativityCreativity

To play with fun toysTo play with fun toys To show off your projects!To show off your projects!

Human Computer InteractionHuman Computer Interaction What is a user interface?What is a user interface? Why do we care about design?Why do we care about design?

We see this all the time. We see this all the time. • What’s good about the design of this error box?What’s good about the design of this error box?

The user knows there is an errorThe user knows there is an error• What’s poor about the design of this error box?What’s poor about the design of this error box?

DiscouragingDiscouraging Not enough informationNot enough information No way to No way to resolveresolve the problem (instructions or contact the problem (instructions or contact

info)info)

My ChoiceMy Choice

iPod by Apple iPod by Apple ComputersComputers

Pros:Pros:• portableportable• powerpower• ease of useease of use• # of controls# of controls

Cons:Cons:• scratches easilyscratches easily• no speech for car no speech for car

useuse• proprietaryproprietary

HCI CommunityHCI Community Academics/Industry Academics/Industry

Research Research • TaxonomiesTaxonomies• TheoriesTheories• Predictive modelsPredictive models

Experimenters Experimenters • Empirical dataEmpirical data• Product designProduct design

Other areas (Sociologists, Other areas (Sociologists, anthropologists, anthropologists, managers)managers)• MotorMotor• PerceptualPerceptual• CognitiveCognitive• Social, economic, ethicsSocial, economic, ethics

HCI ToolsHCI Tools SoundSound 3D3D AnimationAnimation VideoVideo DevicesDevices

• Size (small->very large)Size (small->very large)• Portable (PDA, phone)Portable (PDA, phone)• PlasticityPlasticity

Context Context sensitive/awaresensitive/aware

PersonalizablePersonalizable UbiquitousUbiquitous

Usability RequirementsUsability Requirements

Goals:Goals:• UsabilityUsability• UniversalityUniversality• UsefulnessUsefulness

Achieved by:Achieved by:• PlanningPlanning• Sensitivity to user Sensitivity to user

needsneeds• Devotion to Devotion to

requirements requirements analysisanalysis

• TestingTesting

Bad InterfacesBad Interfaces EncumberingEncumbering ConfusingConfusing SlowSlow Trust (ex. windows Trust (ex. windows

crashing)crashing) What makes it hard?What makes it hard?

• Varies by cultureVaries by culture• Multiple platformsMultiple platforms• Variety of usersVariety of users

Think of a game Think of a game you’ve played with a you’ve played with a bad interfacebad interface• UNIXUNIX

ExampleExample Five fastest places to click on for a Five fastest places to click on for a

right-handed user?right-handed user?

ExampleExample What affects time?What affects time?

Virtual Reality DefinitionVirtual Reality Definition

What is virtual reality?What is virtual reality?• Virtual – Virtual –

Being in essence or effect, but not in factBeing in essence or effect, but not in fact Example VRAMExample VRAM

• Reality –Reality – The state or quality of being real. Something The state or quality of being real. Something

that exists independently of ideas concerning that exists independently of ideas concerning it. Something that constitutes a real or it. Something that constitutes a real or actual thing as distinguished from something actual thing as distinguished from something that is merely apparent.”that is merely apparent.”

• What was the first VR?What was the first VR?

What was the first VR?What was the first VR?

Ivan Sutherland’s Ivan Sutherland’s The Ultimate DisplayThe Ultimate Display

““Don’t think of that thing as a Don’t think of that thing as a screen, think of it as a screen, think of it as a window, a window through window, a window through which one looks into a which one looks into a virtual world. The challenge virtual world. The challenge to computer graphics is to to computer graphics is to make that virtual world look make that virtual world look real, sound real, move and real, sound real, move and respond to interaction in respond to interaction in real time, and even feel real time, and even feel real.”real.”

Key Elements of Virtual Reality Key Elements of Virtual Reality ExperienceExperience

Virtual World - content of a given Virtual World - content of a given mediummedium• screen play, script, etc.screen play, script, etc.• actors performing the play allows us to actors performing the play allows us to

experience the virtual worldexperience the virtual world

Immersion – sensation of being in an Immersion – sensation of being in an environmentenvironment• mental immersion – suspension of mental immersion – suspension of

disbeliefdisbelief• physical immersion – bodily entering physical immersion – bodily entering

the mediumthe medium• Related to Related to presencepresence – (mentally – (mentally

immersed) the participant’s sensation immersed) the participant’s sensation of being in the virtual environment of being in the virtual environment (Slater)(Slater)

Walking Experiment atUNC – Chapel Hill

Augmented RealityAugmented Reality

A combination of a A combination of a real scene viewed real scene viewed by a user and a by a user and a virtual scene virtual scene generated by a generated by a computer that computer that augments the augments the scene with scene with additional additional information.information.

Ultrasound Visualization Research atUNC – Chapel Hill

All Virtual Objects All Real Objects

Immersive TechnologyImmersive Technology Head-mounted Head-mounted

DisplayDisplay• Optical SystemOptical System• Image Source (CRT Image Source (CRT

or LCD)or LCD)• Mounting ApparatusMounting Apparatus• EarphonesEarphones• Position TrackerPosition Tracker

Immersive TechnologyImmersive Technology Multi-screen Projection Multi-screen Projection

of stereoscopic images of stereoscopic images (CAVE)(CAVE)

Immersive TechnologyImmersive Technology

Single large Single large stereoscopic stereoscopic displaydisplay• Projection-basedProjection-based• Head-trackedHead-tracked• Possible tracking of Possible tracking of

hands and arms.hands and arms.• Brings virtual Brings virtual

objects into the objects into the physical worldphysical world

User’s perspectiveUser’s perspective

SettingSetting Objects in worldObjects in world Other participantsOther participants Active/PassiveActive/Passive

• Factory SimulationFactory Simulation• Architectural Architectural

WalkthroughWalkthrough

Immersive Virtual Characters for Immersive Virtual Characters for Educating Medical Communication Educating Medical Communication

SkillsSkillsJ. Hernendez, A. Stevens, D. S. LindJ. Hernendez, A. Stevens, D. S. Lind Department of Surgery (College of Medicine)Department of Surgery (College of Medicine)

M. DuersonM. Duerson Department of Community Health and Family Medicine (College Department of Community Health and Family Medicine (College of Medicine) of Medicine)

K. Johnsen, A. Raij, R. Dickerson, B. LokK. Johnsen, A. Raij, R. Dickerson, B. LokDepartment of Department of Computer and Information Science and Computer and Information Science and Engineering (College of Engineering)Engineering (College of Engineering)

R. FerdigR. FerdigCollege of EducationCollege of Education The University of Florida, Gainesville, FLThe University of Florida, Gainesville, FL

J. Jackson, M. ShinJ. Jackson, M. ShinDepartment of Department of Computer Science Computer Science The University of North Carolina at Charlotte, Charlotte, NCThe University of North Carolina at Charlotte, Charlotte, NC

Sebastian Foti, Juan Cendan, Peggy WernerSebastian Foti, Juan Cendan, Peggy WernerAssociated FacultyAssociated Faculty

Future WorkFuture Work Future collaboratorsFuture collaborators

• UF College of Education UF College of Education • Medical College of Georgia Medical College of Georgia

(MCG)(MCG) Now n=20, future n=120 Now n=20, future n=120

students / year (MCG)students / year (MCG) Evaluate VP vs SPEvaluate VP vs SP Non-verbal communicationNon-verbal communication

• Body leanBody lean• Eye gazeEye gaze

Real Speech vs Text to Real Speech vs Text to Speech Speech

Negative training transferNegative training transfer

Driving Application: Driving Application: Engineering Design EvaluationEngineering Design Evaluation

At assembly design stageAt assembly design stage::• Several constructed parts Several constructed parts • Many CAD models for parts yet Many CAD models for parts yet

to be constructedto be constructed• Unverified assembly plansUnverified assembly plans

How can we verify the assembly How can we verify the assembly plan?plan?

To be built in 2008We only have this in 2005

MotivationMotivation::• Finding design flaws earlier will Finding design flaws earlier will

save time and moneysave time and money• Virtual environments have Virtual environments have

difficulty simulating hands-on difficulty simulating hands-on assemblyassembly

• Support team based interactionSupport team based interaction

Capturing the Physical ModelCapturing the Physical ModelScanningScanning Define color markersDefine color markers

Interaction in ME

Reconstruction from Reconstruction from multiple scansmultiple scans

Skill SetSkill Set ““Oh I don’t want to program, I just want to create Oh I don’t want to program, I just want to create

stuff”stuff” Architect or Car analogyArchitect or Car analogy Programming -> tool to create what is in your Programming -> tool to create what is in your

imaginationimagination Programming proficiency goal:Programming proficiency goal: be able to be able to

code anything that you imaginecode anything that you imagine To do what you saw before we use:To do what you saw before we use:

• C/C++, OpenGL, many different librariesC/C++, OpenGL, many different libraries Math, math, and yet more math!!!Math, math, and yet more math!!!

• Differential equations (simulations, physics engines), Differential equations (simulations, physics engines), linear algebra (transforms, 3D manipulations), statisticslinear algebra (transforms, 3D manipulations), statistics

Graduate SchoolGraduate School Programming Team Lead at RockStar GamesProgramming Team Lead at RockStar Games

• Worked on Midnight Club 2Worked on Midnight Club 2 Renderman Developer at PixarRenderman Developer at Pixar

• Worked on the Jellyfish of Nemo and The IncrediblesWorked on the Jellyfish of Nemo and The Incredibles Software Engineer for Interactive Media for DisneySoftware Engineer for Interactive Media for Disney

• Worked on Toon Town, online capable titlesWorked on Toon Town, online capable titles Worked at 989 Sports, EA Sports, Freedom Force, etc.Worked at 989 Sports, EA Sports, Freedom Force, etc. What did they all have in common?What did they all have in common?

• Published scientific journals in computer sciencePublished scientific journals in computer science• Graduate school in computer scienceGraduate school in computer science• Worked on efficient collision detection, particle systems, Worked on efficient collision detection, particle systems,

curved surfaces, virtual reality, etc.curved surfaces, virtual reality, etc.• All had All had

GRE scores in the top 10%GRE scores in the top 10% Went to a top graduate school (admission is tough)Went to a top graduate school (admission is tough) Strong math and science and coding Strong math and science and coding Why would a CS professor want to work with you?Why would a CS professor want to work with you?

Interested? Take the following…Interested? Take the following…

CAP4730 Computational Structures in CAP4730 Computational Structures in Computer Graphics (Fall ’05)Computer Graphics (Fall ’05)

CAP4930/6930 Design and Creation of CAP4930/6930 Design and Creation of Virtual Environments (Fall ’05)Virtual Environments (Fall ’05)

CAP4930/6930 Human-Computer CAP4930/6930 Human-Computer Interaction (Spring ’06?)Interaction (Spring ’06?)

Computational Geometry (Spring ’06)?Computational Geometry (Spring ’06)? Computer Simulation Concepts (Fall ’05)Computer Simulation Concepts (Fall ’05)

Virtual Experiences Group 2006Virtual Experiences Group 2006 PhD StudentsPhD Students

• Kyle JohnsenKyle Johnsen• John QuarrelsJohn Quarrels• Andrew RaijAndrew Raij• Xiyong WangXiyong Wang• Aaron KotranzaAaron Kotranza• Brent RossenBrent Rossen

UndergraduatesUndergraduates• Harold RodriguezHarold Rodriguez• Joshua HortonJoshua Horton

AlumniAlumni• Robert Dickerson (BS)Robert Dickerson (BS)• Sayed Hashimi (BS)Sayed Hashimi (BS)• Art Homs (BS)Art Homs (BS)• Cyrus Harrison (MS)Cyrus Harrison (MS)• George Mora (MS) George Mora (MS) • Andrew Joubert (BS)Andrew Joubert (BS)• Samuel Preston (BS)Samuel Preston (BS)• John Samuelsen (BS)John Samuelsen (BS)

Thank you! Questions?

Virtual Experiences Group 2004Virtual Experiences Group 2004 PhD Students (4)PhD Students (4)

• Kyle JohnsenKyle Johnsen• John QuarrelsJohn Quarrels• Andrew RaijAndrew Raij• Xiyong WangXiyong Wang

Undergraduates (4)Undergraduates (4)• Robert DickersonRobert Dickerson• Sayed HashimiSayed Hashimi• Andrew JoubertAndrew Joubert• Art HomsArt Homs

Alumni (4)Alumni (4)• Cyrus Harrison (MS)Cyrus Harrison (MS)• George Mora (MS) George Mora (MS) • Andrew Joubert (BS)Andrew Joubert (BS)• Samuel Preston (BS)Samuel Preston (BS)• John Samuelsen (BS)John Samuelsen (BS)

Thank you! Questions?