Part 16: Social and Collaborative Virtual Environments · Social and Collaborative Virtual Environments Virtuelle Realität Wintersemester 2006/07 Prof. Bernhard Jung ... Email, bulletin

Virtuelle Realität

1

Part 16:

Social and Collaborative Virtual Environments

Virtuelle RealitätWintersemester 2006/07

Prof. Bernhard Jung

Prof. B. Jung Virtuelle Realität, WS 2006/07

IBM Ups Investment In Virtual World

IBM is set to invest $10 million over the next twelve months to increase its presence in the market for technologies that enable so-called virtual worlds such as Second Life, a 3-D Web environment in which people from various walks doff their everyday identities to assume online alter egos, a company spokesman said Friday. As part of the effort, IBM will expand its use of Second Life, which now has more than one million users, for virtual meetings with employees and business partners. Next week, company CEO Sam Palmisano will go "in world" to hold a virtual town hall on IBM's Second Life island. The company recently held a similar gathering with the press and analysts, and has hosted a virtual party for IBM alumni inside the online world. It's also building a 3-D replica of China's famed Palace Museum inside Second Life, which will be open to cyber tourists. IBM's virtual reality ambitions go far beyond Second Life, however, a spokesman says. …

[ http://www.techweb.com/wire/ebiz/193700609 November 10, 2006]

Virtuelle Realität

2


Social Virtual Environments

Virtual RealityReal-time computer graphics, audio, haptics3D user interface"Presence": sense of "being there"

Social Virtual Environmentssense of "being with others"collaborative work, meetings, games, entertainment, …


Overview

Conversation with othersGroupwareNew communication technologiesCollaborative Virtual Environments

"Ubiquitous Virtual Reality"

Further InformationJ. Preece, Y. Rogers & H. Sharp. Interaction Design – Beyound Human-Computer

Interaction. John Wiley & Sons. 2002. Alan Dix, Janet E. Finlay, Gregory D. Abowd, Russell Beale. Human-Computer

Interaction, 3rd Edition. Prentice Hall, 2004.T. Pfeiffer, M. Weber & B. Jung: Ubiquitous Virtual Reality - Accessing Shared Virtual

Environments through Videoconferencing Technology. Proceedings Theory and Practice of Computer Graphics Conference. Eurographics, 2005, pp. 209-216. Available at: www.informatik.tu-freiberg.de/~mweber

Virtuelle Realität

3


Conversation with others

Various mechanisms and ‘rules’ we follow to hold a conversationmutual greetings

A: Hi thereB: Hi!C: HiA: All right?C: Good, how’s it going?A: Fine, how are you?C: OKB: So-so. How’s life treating you?


Conversational rules

turn-taking to coordinate conversationA: Shall we meet at 8?B: Um, can we meet a bit later?

A: Shall we meet at 8?B: Wow, look at him?A: Yes what a funny hairdo!B: Um, can we meet a bit later?

Back channeling to signal to continue and followingUh-uh, umm, ahh

Virtuelle Realität

4


More conversational rules

farewell ritualsBye then, see you, see you later….

implicit and explicit cuese.g. looking at watch, fidgeting with coat and bags explicitly saying “Oh dear, must go, look at the time, I’m late…”


Breakdowns in conversation

When someone says something that is misunderstoodSpeaker will repeat with emphasis:A: “this one?”B: “no, I meant that one!”

(also a case of deixis: this and that refer to the visible situation)Also use tokens:Eh? Quoi? Huh? What?

Virtuelle Realität

5


What happens in technology-mediated conversations?

Do same conversational rules apply?Are there more breakdowns?How do people repair them?

Phone?Email?Instant messaging?SMS texting?Video conferencing?


Design implications

How to support conversations when people are ‘at a distance’ from each other

Many applications have been developedEmail, videoconferencing, videophones, instant messaging, chatrooms, collaborative virtual environments, media spaces

How effective are they?

Do they mimic or extend existing ways of conversing?

Virtuelle Realität

6


What is groupware?

Software specifically designedto support group workingwith cooperative requirements in mind

NOT just tools for communicationGroupware can be classified by

when and where the participants are workingthe function it performs for cooperative work


The Time/Space Matrix

Classify groupware by: when the participants are working,

at the same time or not where the participants are working,

at the same place or not

Common names for axes:time:

synchronous/asynchronousplace:

co-located/remote

differentplace

sameplace

sametime

differentplace

Virtuelle Realität

7


Time/Space Matrix (ctd)

differentplace

sameplace

sametime

differenttime

face-to-faceconversation telephone

post-it note letter


Classification by Function

Cooperative work involves:Participants who are workingArtefacts upon which they work(e.g. documents)

participants

artefacts of work

control andfeedback

P P

A

communication

understanding

direct

Virtuelle Realität

8


What interactions does a tool support?

participants

artefacts of work

control andfeedback

P P

A

communication

understanding

direct

meeting and decisionsupport systems

– common understanding

computer-mediatedcommunication

– direct communicationbetween participants

shared applicationsand artefacts

– control and feedbackwith shared work objects


Time/space matrix revisited

co-located remote

synchronous

asynchronous

co-authoring systems,shared calendars

argumentationtools

email andelectronic

conferences

shared work surfaces and editorsshared PCs and windows

video conferences,video-wall, etc.

Collaborative VEsmeeting rooms

CAVEs

Second Life

Hologram Messaging(still SciFi)

Virtuelle Realität

9


Synchronous computer-mediated communication

Conversations are supported in real-time through voice and/or typingExamples include video conferencing, chatrooms, collaborative virtual environments

BenefitsCan keep more informed of what is going on Video conferencing allows everyone to see each other providing some support for non-verbal communicationChatrooms can provide a forum for shy people to talk more

Problems:Video lacks bandwidth so judders and lots of shadowsDifficult to establish eye contact with images of othersPeople can behave badly when behind the mask of an avatar


Video conferencing: wall, kiosk, booth

Virtuelle Realität

10


Will video be a success using G3 mobile phones?

The VP-210" VisualPhone: a mobile video phone developed by the japanese company Kyocera Corporation Source: http://www.kyocera.co.jp/news/1999/9905/0003-e.asp

• Will the judder, sudden jerks and shadowsdisappear?

• Will it be possible to establish eye contactand read lips on such a small image?

• Will people find it socially acceptable totalk to an image of someone in the palm oftheir hands?


Asynchronous computer-mediated communication

Communication takes place remotely at different timesEmail, bulletin boards, newsgroups, online forums, blogs, …Recipients of email:

direct in To: fieldcopies in Cc: field

delivery identical – difference is social purposeBenefits include:

Read any place any timeFlexible as to how to deal with itPowerful, can send to many peopleCan make saying things easier

Problems include:FLAMING!!!SpammingMessage overloadFalse expectations as to when people will reply

Virtuelle Realität

11


txt is gr8

instant messaging1996 – ICQ small companynow millionsmore like conversation

SMSpeople adapt their communication styles to medium

y is it we al lv shrt msgsoriginally a feature of internal management protocolshort messages (160 chars) and text with numbersno-one predicted mass adoption!!now phones with cameras for MMS

Hi, u there

want to meet later

yeh, had a good night last night?uhu


Computer-mediated communication combined with other activity: Meeting and decision support systems

In design, management, and research, we want to:generate ideasdevelop ideasrecord ideas

Primary emphasis: common understanding

Three types of systemsargumentation tools

asynchronous co-locatedrecording the arguments for design decisions

meeting roomssynchronous co-locatedelectronic support for face-to-face meetings

shared drawing surfacessynchronous remoteshared drawing board at a distance

ArgTool

Virtuelle Realität

12


Typical meeting room

sharedscreen


Meeting rooms

synchronous co-located

electronic support for face-to-face meetingsindividual terminals (often recessed)large shared screen (electronic whiteboard)special softwareU or C shaped seating around screen

Various modes:brainstorming, private use, WYSIWIS

WYSIWIS – ‘what you see is what I see’all screens show same imageany participant can write/draw to screen

Virtuelle Realität

13


Meeting capture

use ordinarywhiteboarddetector andspecial pensLCD projectionon whiteboardlow-cost alternativeto dedicated meeting room


New communication technologies

Move beyond trying to support face-to-face communication Provide novel ways of interacting and talking

Examples include:SMS texting via mobile phonesOnline chatting in chatrooms Media spacesCollaborative virtual environments

Virtuelle Realität

14


Hypermirror(Morikawa and Maesako, 1998)

allows people to feel as if they are in the same virtual place even though in physically different spaces

(woman in white sweater is in a different room to the other three)

People in different places are superimposed

on the same screento make them appear as if

in same space


Creating personal space in Hypermirror

2) Two in this room are invadingthe ‘virtual’ personal spaceof the other person by appearing to bephysically on top of them

3) Two in the room move apart to allow person in other space more ‘virtual’personal space

Virtuelle Realität

15


Everyone happy


Clearboard (Ishii et al, 1993)

ClearBoard - transparent board that shows other person’s facial expression on your board as you draw

Virtuelle Realität

16


Collaborative Virtual Environments (CVEs)

meet others in a virtual worldparticipants represented – embodiment artefacts too …

computer (e.g. spreadsheet) and ‘real’ (virtually) objectstext?

consistent orientation or easy to read

MUDs (Multi-user domains)2D/3D places to meet on the webusers represented as avatars


Collaborative Virtual Environments

The rooftop garden in BowieWorld, a Collaborative Virtual environment (CVE), supported by Worlds.com. Users take part by “dressing up” as an avatar. There are 100s of avatars to choose from, including penguins and real persons. Once an avatar has entered a world they can explore it and chat to other avatars.

Source: www.worlds.com

Virtuelle Realität

17



Second Life (SL) is a privately owned, partly subscription-based 3-D virtual world, made publicly available in 2003 by San Francisco-based Linden Lab,[2]and founded by former RealNetworks CTO Philip Rosedale. Users can visit this virtual world almost as if it were a real place. They explore, meet new people, participate in individual and group activities, and, if they decide to visit often, they learn new skills and mature socially (in the sense of learning the culture of the virtual world). Though sometimes referred to as a game, Second Life does not have points, scores, winners and losers, levels, and end-strategy, or most of the other characteristics of a game. Source: http://en.wikipedia.org/wiki/Second_Life


Videogames: Virtual Thievery Criminals break into accounts, steal artifacts and sell them for thousands of dollars.

Dec. 11, 2006 issue - Keep a close eye on your magic wand, or somebody will steal it. World of Warcraft and other Massively Multiplayer Online Role-Playing Games have recently become the target of criminals who seek in-game currency, or gold, because of its real-world value. Cyberthieves break into players' accounts, steal the artifacts and characters acquired during the course of the game, and sell them—sometimes for thousands of dollars. Whereas gold farmers in China have sought to acquire gold legitimately by playing WOW for hours on end, theft is a relatively new phenomenon, says Mikko Hyppönen, chief research officer at F-secure, a cybersecurity firm in Helsinki. "We really didn't see this a year ago."…One reason criminals have gone for game crime is because it's less risky than breaking into online bank accounts. Chances are your local police won't even be able to keep a straight face when you report that someone stole your magical potions.Thieves also benefit from lax virtual-property laws in the United States, where most of these games are made. Designers are reluctant to push for legal recognition of virtual property for fear of being held liable for theft. "Game designers certainly don't want to facilitate hacking, but they often are concerned about their own potential liability for a loophole that creates a hack," says John Fairfield, associate professor of law at Indiana University. "So therefore they don't share the incentives to publicize and enforce as strongly as the owner of the virtual property would." The game companies argue that they own the virtual property, not the players. "Because players don't own it, players can't fight back when it's stolen," says Fairfield. "And that makes players easy victims." Until the matter of ownership gets cleared up, this new crime wave will be tough to stop.

http://www.msnbc.msn.com/id/15994153/site/newsweek/ Dec. 11, 2006 issue

Virtuelle Realität

18





Virtuelle Werkstatt – Collaborative Virtual Prototyping in VRUniversität Bielefeld, Universität zu Lübeck

Virtuelle Realität

19



PASION - Psychologically Augmented Interaction Over NetworksUniversität Bielefeld http://www.techfak.uni-bielefeld.de/ags/wbski/PASION/

"Interactive Social Displays im virtuellenRaum erlauben die Interaktion mittels Bild, Ton und erweitertenMöglichkeitenzwischen Individuenund Gruppen."

Ubiquitous Virtual Reality

Accessing Shared Virtual Environments through Videoconferencing Technology

Thies Pfeiffer,SFB 360, University of Bielefeld, Germany

Matthias Weber, and Bernhard JungTechnical University of Freiberg, Germany

Talk presented atEurographics Conference on Theory and Practice of Computer Graphics, 2005

Virtuelle Realität

20


What if...

... one could communicate to distant places?telegraphy, phone, chat

... one could communicate on the road?mobile phones, sms

... one could see each other in remote communication?


Videoconferencing Solutions

DesktopWebCam & MicrophoneNetMeeting, iChat, GnomeMeeting

Stand-AlonePolycom VSX 3000D-Link DVC-2000TelePortec Teleporter(3D, Volumetric Display) VSX 3000

DVC-2000Teleporter

Virtuelle Realität

21


What if...



... one could really see each other?videoconferencing

... the remote place is not real?


Videoconferencing in VR

2D-Videoimage:AliceStreet

3D-Avatars:blue-cColiseum

…

(see www.kolabora.com/news/2004/02/20/threedimensional_virtual_conferencing_rooms_become.htm)

(see Gross et al., blue-c: ASpatially Immersive Display and 3DVideo Portal for Telepresence,Siggraph 2003)

(see Baker et al., Computation and Performance Issues InColiseum, An Immersive Videoconferencing System,ACM Multimedia 2003)

Virtuelle Realität

22


What if...



... one could really see each other?videoconferencing

... the other place is not real?shared virtual environments

... the communication partner is not real?chatter bots, telephony systems


Chatting Avatars

(Permanent Exhibition at the Heinz-Nixdorf Museum, developedby the AI group at University of Bielefeld, picture taken fromhttp://www.techfak.uni-bielefeld.de/ags/wbski/labortag2002.html)

Virtuelle Realität

23


What if...

everything is possible?

Distant mobile communicationAudio and videoNatural communicationStandard clientsSeamless transition betweenhuman-human and human-machine interaction

When one could interact with an artificial system using


Motivation

Using standard software and protocolsUsing different platformsSupporting different input modalities using video, audio and data

→ Using videoconferencing technology

Goal: viewing and controlling Virtual Environments (VEs) ubiquitously

Virtuelle Realität

24


History of Videoconferencing

1924 - Two-Way Videophone demonstration by Herbert Eugene Ives

(b&w, 1929 color)1936 - First official Videophone connection

(Berlin-Leipzig, Deutsche Reichspost) 1956 - Picturephone® by AT&T1971 - First trans-atlantic videophone (Ericsson)1992 - INRIA Videoconferencing System

(IP-based software)1996 - ITU-T standard H.323 v11996 - Microsoft NetMeeting v2.0

(with video) 2000 - First MPEG-4 streaming video cell phone (Samsung)2001 - First trans-atlantic tele surgery2004 - ITU-T recommends H.264 video codec


Videoconferencing

Connecting people with each otherChannels for multi-modal communication

Audio: speech, soundsVideo: gestures, facial expressionsData: text-chat

BidirectionalPoint-to-point or multipointDifferent protocols:

H.323, SIP, proprietary (e.g. Skype)

Virtuelle Realität

25


3D Conferencing

Connecting people within VEsAll conferencing participants use special purpose VR software to share the VEMixed reality


Videoconferencing in AR/VR

Studierstube(Vienna University of Technology, Austria):

video transferred via Videoconferencing (H.323)special protocol for other data, like trackingspecial client needed that computes trackingno standard software

(see Istvan Barakonvi, W. Frieb, Dieter Schmalstieg, Augmented Reality Videoconferencing for CollaborativeWork, Proc. of the 2nd Hungarian Conference on Computer Graphics and Geomtery, Budapest, Hungary, May 2003)

Virtuelle Realität

26


Concept

Thin ClientStandard videoconferencing software

Thick ServerRuns the VEProvides Videoconferencing Gateway

Sending content to participantsUtilizing natural communication interface

Accepts or initiates calls


Concept (2)

Virtuelle Realität

27


Example Application Scenarios

Information DesksTechnical Support

Educational SystemsUnified Messaging ServicesInteractive EntertainmentCooperative Work...


Virtuelle Realität

28


Implementation

Research prototype of a videoconferencing interfaceServer is an extension of an existing application for virtualconstruction via natural speechTested client platforms include Windows XP / NetMeeting, Linux / GnomeMeeting and Pocket PC / PocketBone


Server

Virtual construction application (in-house)VR framework Avango (OpenSource)

SGI OpenGL PerformerOpenH323 (OpenSource)ESMERALDAspeech recognition software (in-house)

Virtuelle Realität

29


Scenegraph (1)


Scenegraph (2)

Virtuelle Realität

30


Scenegraph (3)


Scenegraph (4)

Virtuelle Realität

31


Scenegraph (5)


Types of Communication

Human – humanAudio: speech, soundsVideo: mimics, facial expressions, gesturesText: chat

Machine – humanAudio: Speech-synthesis, soundsVideo: images, movies, real-time graphicsText: status messages

Human – machineAudio: speech-recognitionText: typed natural language, commandsVideo: computer-vision, face-detection & recognition, gesture recognition

Virtuelle Realität

32


Open Questions

Robust Speech RecognitionEvaluation: speech recognisers <> codecs

Visual InteractionGesturesFace Recognition / Tracking

Usability studiesDifferent devicesDifferent combinations of modalities


Conclusion

Videoconferencing interface to VEsThin client solutionScales from low to high-end devices

(cell phone, handheld, PC, standalone)Supports broad range of applicationsUses natural interface

→ Make your VE ubiquitously accessible!

Documents

Part 16: Social and Collaborative Virtual Environments · Social and Collaborative Virtual Environments Virtuelle Realität Wintersemester 2006/07 Prof. Bernhard Jung ... Email, bulletin