8
Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka. Cloning and Teleporting Avatars Across Workstations and Mobile Devices in Collaborative Virtual Environments: Clipboard Operations for Virtual Reality Owen Noel Newton Fernando , Guo Saito , Uresh Duminduwardena , Yoshie Tanno , and Michael Cohen Spatial Media Group; University of Aizu; Aizu-Wakamatsu, Fukushima-ken 965–8500; Japan Email: d8052101,s1100143,mcohen @u-aizu.ac.jp 2-3-7, Shimizunuma, Miyaginoku; Sendai, Miyagi-ken 983–0845; Japan Email: [email protected] Dialog Telekom Ltd.; 475, Union Place; Colombo 2; Sri Lanka Email: [email protected] Abstract— Audio windows are, in analogy to graphical win- dowing systems, an idiom for managing multiple streams of information, as in teleconferences, chatspaces, or virtual concerts. Audio windowing systems are especially appropriate on small screen devices for practical applications in ubiquitous computing. We have developed a multiplatform implementation of audio win- dowing systems to control source sink transmissions in collabo- rative virtual environments (CVEs), for workstations and mobile devices. Clipboard operations have been implemented to allow teleporting (cut/paste) and cloning (copy/paste) avatars in both interfaces. The workstation- and mobile-based interfaces encourage multiple spaces, which enables multipresence-enabled conferencing features in collaborative virtual environment. These interfaces are integrated with other CVE clients, interoperating with a heterogeneous groupware suite, including stereographic panoramic browsers, spatial audio backends, and speaker arrays. Our main goal is to explore how to extend audio window systems with narrowcasting (selection) and multipresence to support useful conferencing capabilities. I. I NTRODUCTION Our group is researching CVEs, collaborative virtual environments: multimodal (auditory, visual, haptic) interfaces and applications for tele¨ existence and artificial reality group- ware [1], [2]. CVEs allow users to share experience through a set of common objects, regardless of geographical separation, across computer networks. Users are provided with symbolic or figurative embodiments called “avatars” that may convey identity, presence, position (location and orientation), different perspectives of respective users, and activities to others. Prac- tical systems allow users to delegate these avatars to interact with the world and to communicate with one another using multimedia (audio, video, graphics, text, etc.). Vision and audition are the two primary human senses for obtaining information about the outside world, and rich interfaces need both modes of information. Visual windowing systems allow multiple applications to share display resources; audio windowing systems could bring order to the cacophony of multiple simultaneous sound sources. Audio windowing is conceived of as a front end, or user interface, to an audio system with a spatial sound backend. Interfaces for computers and portable devices mainly present information visually. While it is often convenient to display information on monitors or large screen displays, vi- sual interfaces have limitations as the display shrinks, because size corresponds intimately to the amount of information that can be conveyed. The size of the devices is critical when developing portable devices like mobile phones, since most people prefer to use small and light devices. One efficient way to decrease the size of a portable device is to decrease the size of the visual display or, more radically, completely remove it. This is possible if information can be conveyed aurally instead of visually. Even though graphical displays of the portable devices are smaller, visual information is still useful, so portable devices like mobile phones need both visual and auditory information. Our interfaces [3] convey both modes of information for workstations and mobile devices. A. Related Research Benford et al. [4], [5], [6], [7], [8] derive a model for aware- ness and interaction in virtual environments. Their “spatial model of interaction” describes interaction based on physical properties of a space. Thus, the abilities to see and to hear are affected by distance, direction, and possible obstructions. The key awareness abstractions in the spatial model are “aura,” “fo- cus,” and “nimbus.” There are many ways of mapping stereo- typical scenarios into equivalent configurations supported by narrowcasting idioms we developed. A direct analogy between nimbus and source “visibility” (audibility, etc.) and between focus and sink attention allows an equivalence. Benford et al. developed their model for two participants, but audio window- ing systems are designed for more than two participants, so an analogy between nimbus and source as well as focus and sink is a coarse projection of a much more complicated relation. There are differences between, for instance, soloing a source 133

Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

  • Upload
    others

  • View
    0

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka.

Cloning and Teleporting AvatarsAcross Workstations and Mobile Devices

in Collaborative Virtual Environments:Clipboard Operations for Virtual Reality

Owen Noel Newton Fernando�, Guo Saito

�, Uresh Duminduwardena

�, Yoshie Tanno

�, and Michael Cohen

��Spatial Media Group; University of Aizu; Aizu-Wakamatsu, Fukushima-ken 965–8500; Japan

Email: � d8052101,s1100143,mcohen � @u-aizu.ac.jp�2-3-7, Shimizunuma, Miyaginoku; Sendai, Miyagi-ken 983–0845; Japan

Email: [email protected]�Dialog Telekom Ltd.; 475, Union Place; Colombo 2; Sri Lanka

Email: [email protected]

Abstract— Audio windows are, in analogy to graphical win-dowing systems, an idiom for managing multiple streams ofinformation, as in teleconferences, chatspaces, or virtual concerts.Audio windowing systems are especially appropriate on smallscreen devices for practical applications in ubiquitous computing.We have developed a multiplatform implementation of audio win-dowing systems to control source � sink transmissions in collabo-rative virtual environments (CVEs), for workstations and mobiledevices. Clipboard operations have been implemented to allowteleporting (cut/paste) and cloning (copy/paste) avatars inboth interfaces. The workstation- and mobile-based interfacesencourage multiple spaces, which enables multipresence-enabledconferencing features in collaborative virtual environment. Theseinterfaces are integrated with other CVE clients, interoperatingwith a heterogeneous groupware suite, including stereographicpanoramic browsers, spatial audio backends, and speaker arrays.Our main goal is to explore how to extend audio window systemswith narrowcasting (selection) and multipresence to supportuseful conferencing capabilities.

I. INTRODUCTION

Our group is researching CVEs, collaborative virtualenvironments: multimodal (auditory, visual, haptic) interfacesand applications for teleexistence and artificial reality group-ware [1], [2]. CVEs allow users to share experience through aset of common objects, regardless of geographical separation,across computer networks. Users are provided with symbolicor figurative embodiments called “avatars” that may conveyidentity, presence, position (location and orientation), differentperspectives of respective users, and activities to others. Prac-tical systems allow users to delegate these avatars to interactwith the world and to communicate with one another usingmultimedia (audio, video, graphics, text, etc.).

Vision and audition are the two primary human sensesfor obtaining information about the outside world, and richinterfaces need both modes of information. Visual windowingsystems allow multiple applications to share display resources;audio windowing systems could bring order to the cacophonyof multiple simultaneous sound sources. Audio windowing is

conceived of as a front end, or user interface, to an audiosystem with a spatial sound backend.

Interfaces for computers and portable devices mainlypresent information visually. While it is often convenient todisplay information on monitors or large screen displays, vi-sual interfaces have limitations as the display shrinks, becausesize corresponds intimately to the amount of information thatcan be conveyed. The size of the devices is critical whendeveloping portable devices like mobile phones, since mostpeople prefer to use small and light devices. One efficient wayto decrease the size of a portable device is to decrease the sizeof the visual display or, more radically, completely removeit. This is possible if information can be conveyed aurallyinstead of visually. Even though graphical displays of theportable devices are smaller, visual information is still useful,so portable devices like mobile phones need both visual andauditory information. Our interfaces [3] convey both modes ofinformation for workstations and mobile devices.

A. Related Research

Benford et al. [4], [5], [6], [7], [8] derive a model for aware-ness and interaction in virtual environments. Their “spatialmodel of interaction” describes interaction based on physicalproperties of a space. Thus, the abilities to see and to hear areaffected by distance, direction, and possible obstructions. Thekey awareness abstractions in the spatial model are “aura,” “fo-cus,” and “nimbus.” There are many ways of mapping stereo-typical scenarios into equivalent configurations supported bynarrowcasting idioms we developed. A direct analogy betweennimbus and source “visibility” (audibility, etc.) and betweenfocus and sink attention allows an equivalence. Benford et al.developed their model for two participants, but audio window-ing systems are designed for more than two participants, so ananalogy between nimbus and source as well as focus and sinkis a coarse projection of a much more complicated relation.There are differences between, for instance, soloing a source

133

Page 2: Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka.

and muteing its compliment.

B. Formalization and Privacy

The suite of inclusion and exclusion narrowcast com-mands for sources and sinks are like analogs of burningand dodging (shading) in photographic processing. The du-ality between source and sink operations [9] is tight, andthe semantics are identical: an object is inclusively enabled

by default unless it is explicitly excluded (with

������ ��� ��� �������� �������� �!� �"� �# ��$&%'�)( ), or peers are explicitly included (with

������ ����� �"� �*)��+&��,-�/.0*21'+&143�5���� ��� �!��� ��� �$6���'�)( #87 ,)1)('%:9 # �8;)<>=�$6?�@��2( ) when the respective avataris not. Such narrowcasting functions, which filter stimuliby explicitly blocking out and/or concentrating on selectedentities, can be applied not only to other users’ sinks forprivacy, but also to one’s own sinks for selective attendanceor presence.

“Privacy” has two interpretations, the first association beingthat of avoiding “leaks” of confidential information, protectingsecrets. But a second interpretation means “freedom fromdisturbance,” in the sense of not being bothered by irrelevanceor interruption. Our audio windowing systems feature nar-rowcasting operations that manage privacy in both senses, byfiltering duplex information flow through an articulated con-ferencing model called audio windows in analogy to graphicalwindows.

C. Multipresence and Autofocus

Humans are indivisible, so one cannot physically be in twoor more places at the same time. A unique feature of oursystem is the ability of a human pilot to delegate multipleavatars simultaneously, increasing quantity of presence; suchmultipresence enables us to overcome some fundamental con-straints of human condition. Our virtual environment interfacesencourage multipresence, by supporting sources and sinks inmultiple places simultaneously— allowing, for example, a userto monitor several spaces at once.

If the sinks are in separate conference rooms, each sourceis localized only with respect to the collocated sink. If mul-tiple sinks share a single conference room, an “autofocus”algorithm is employed by anticipating “the rule of the firstwavefront,” the tendency to perceive multiple identical simul-taneous sources from different locations as a single fusedsource. Rather than adding and averaging the contribution ofeach source [10], [11] to possibly multiple sinks, our systemlocalizes each source only with respect to its respective bestsink.

D. Multiple Spaces

Dix et al. [12] described their experiences of spaces inthe Equator project (www.equator.ac.uk), in particularthe way in which multiple spaces, both virtual and physical,can co-exist. People and objects may have locations in andrelationships to both physical space and one or more virtualspaces, and these different spaces together interact to give an

overall system behavior and user experience. Our interfacesexplore interactions between virtual and physical spaces toallow users to be in multiple places simultaneously. Ourworkstation- and mobile-based interfaces support multiplespaces to enhance multipresence-enabled conferencing capa-bilities in collaborative virtual environment.

Multiple spaces are supported via launching multiple in-stances of the workstation-based interface. An arbitrary num-ber of applications can be run, for example, corresponding tohome, school or virtual concert spaces. Upon launching aninstance of the application, a user provides a space name,upon which are based all the networked attributes of thatspace, including channel numbers, position parameters, andnarrowcasting attributes. This approach allow users to virtuallyinhibit arbitrary number of spaces without any difficulties.

Multiple spaces are supported directly within the mobile-based interface (there being no multitasking on a mobilephone operating system). Area-division multiplexing of thegraphical display is used for the mobile interface to displaymultiple spaces. In consideration of the small screen displayof mobile devices, the mobile interface currently supports onlytwo virtual rooms. An extension to our research is planning amultiwindowing system with an arbitrary number of spaces,to allow users to virtually inhibit more than two.

E. Clipboard Operations

We have developed clipboard operations for workstation-and mobile-based interfaces. The clipboard operations, enabledby dynamic deletion and creation of avatars, can be usedfor teleporting (cut/paste) and cloning (copy/paste) incollaborative virtual environments. These operations allowteleported and cloned avatars to convey narrowcasting at-tributes across multiple spaces.

In our workstation-based interface, the system clipboardcan be used for teleporting and cloning. For instance,one workstation-based application (associated with a homespace) can cut or copy avatars and paste into an-other workstation-based application (associated with a schoolspace), if the same user runs both applications on one machine.If another workstation-based application runs on a differentmachine connected to the same session (for instance, a schoolspace), it should communicate with the newly pasted avatar.

Our mobile interface can use clipboard operations for tele-porting and cloning in two spaces to enhance conferencingcapabilities in CVE. These operations done by using differentkeys of mobile phones as shown in Table III.

Deletion and creation of avatars enable teleporting andcloning in distributed applications and heterogeneous sessions.for example, when a user cuts or deletes avatars in onespace, sends message to the server to delete avatars in thesession according to the space name and channel number.Symmetrically, when a user pastes an avatar in anotherspace, sends messages to the server to create an avatar in thesession according to the space name and channel number withnarrowcasting attributes.

134

Page 3: Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka.

TABLE I

ROLES OF ACBEDGFIH�JK&LMN AND APO�QERLSM5N : AN ARBITRARY NUMBER OF AVATARS CAN BE INSTANTIATED AND ASSOCIATED WITH USERS AT RUNTIME. ICONIC

AND FIGURATIVE ATTRIBUTES OF NARROWCASTING FUNCTIONS EXTEND AVATARS TO DENOTE THE INVOKED FILTERS.

Source Sink

Function radiation/transmission receptionLevel amplification sensitivity

Direction OUTput INputInstance speaker listener

Transducer loudspeaker microphone or dummy-headOrgan mouth earTool megaphone or loud-hailer (bullhorn) ear trumpet

Exclude mute deafen

Inhibit in TVU Con WX YZX[Y

Inhibit Self in Multiplicity

reflexive (thumb up) (thumbs back)

Inhibit Other in Multiplicity

transitive (thumb down) (thumbs up)

Include select (solo or cue) attend : confide and harken

Assert in TVU Con \X ]^X8]

Assert Target in Multiplicity

explicit (megaphone) (ear trumpets)

Assertion side-effect in Multiplicity

implicit (translucent hand) (translucent hands)

135

Page 4: Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka.

F. Groupware Architecture

We have designed and implemented an architecture andframework [13] to support a collaborative virtual environment[14], allowing distributed users to share multimodal virtualworlds [15]. The CVE architecture, sketched in Fig. 1 is basedupon a client/server model, and its main transaction sharesthe state of virtual objects and users (avatars) by effectivemulticast via replicated-unicast of position (translation, rota-tion) parameters and narrowcasting attributes to client peersin a session. The mobile interface is integrated with ourCVE through a “servent” (server/client hybrid) HTTP _ TCP/IP

gateway. The client/server architecture enables multimodalcommunication, platform independence, and easy networkconnectivity, as components are built with Java.

Fig. 1. CVE Architecture: groupware suite.

In this article, we describe two audio windowing inter-faces: for workstations, “Multiplicity,” and for 2.5- and ` ��a -generation mobile devices, “ bdc Con,” with figurative and iconicavatars, respectively. Our audio windowing systems are inte-grated with multimodal groupware suite to enhance and enrichthe user experience. All the controls from these interfaces aremulticast to all the other (generally heterogeneous) clientsin a session, synchronizing state, including narrowcastingattributes. These two interfaces are described in the followingsections.

II. MULTIPLICITY: JAVA3D WORKSTATION-PLATFORMED MULTIPERSPECTIVE INTERFACE

A. Implementation

We have implemented a workstation-based narrowcastinginterface (as shown in Figs. 2 & 3, 5(a), 5(b), and 6(a)), Mul-tiplicity [16], using Java3D (java.sun.com/products/java-media/3D/). An arbitrary number of avatars can beinstantiated and associated with users at runtime. Attributesof narrowcasting functions, summarized by Table I, extend thefigurative avatars to denote the invoked filters. Multiplicity

can be configured at runtime to display multiple perspec-tives from various standpoints, including exocentrically fromvarious strategically placed cameras and egocentrically (bothendocentric & tethered) with respect to a selected avatar.

Each avatar, in general in our environment, can act asboth sink and source. Every source in this environment canidentify and be associated with a best sink by an autofo-cus algorithm. Various kinds of “selection” are used in ourvirtual environment. A singleton selection (multiple avatarsnot simultaneously selectable) is used to determine the targetfor locally generated motion commands and the standpointand orientation for endo- and ego-centric visual perspectives( View 0 and View 1 buttons in Fig. 3). Further, the se-lect and self commands are used, along with the othernarrowcasting attributes, to resolve privacy operations.

The Multiplicity application discovers which of the possiblyseveral “self”-designated avatar delegates (represented by starsover their heads) is most sensitive (visualized by arrowswhich fly from source e best sink as shown in Fig. 3),directionalizes the sources accordingly, and composites theoverlaid soundscapes for display to each user via headphone,nearphones, stereo speakers, or speaker array. A “phantomsources” feature, described in the next section, is used tologically separate listening and viewing positions, allowing theinterface a fluid perspective.

B. Phantom Sources

In the movie “Multiplicity” [17], Doug Kinney is a man withtoo many tasks and not enough time to complete them. Whenhe feels too much pressure, his temper explodes with profoundresults. He wishes he could be in more than one place at atime. A medical researcher provides the “miracle” by offeringDoug an exact duplicate of himself. Being in two places atonce offers some advantages, but problems arise when Doug#2 can’t handle the stress of overwork any better than Doug #1.Doug winds up as four slightly different versions, each of thememphasizing some facet of his character, and complicationsmultiply.

Considering workload of people in real life, they might wishto be in more than one place at once to finish their workproperly. There is no way to do this in reality, but virtual realityinterfaces can be used to enhance working environments,having virtually inhibit users’ representatives in collaborativevirtual environment. Our audio windowing systems allow usersto virtually inhibit more than one place at once.

In the absence of multipresence of our Multiplicity interface,an egocentric display (from the viewpoint of a particularavatar) naturally and correctly spatializes sound sources. Anissue arises when a CVE is extended by multipresence, allow-ing a single human to designate multiple avatars as associatedwith the user: exocentric camera positions will not have avatar-centric audio displays. A Java3D listening point is implicitlyassociated with the virtual camera position (viewpoint). Ourtechnique is to transparently relocate sources to compensatefor the selected viewpoint. The multipresence extension, mo-tivated by desire to reduce granularity of control and refined by

136

Page 5: Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka.

Fig. 2. The Mixels panel displays and controls state of each avatar, including selection for motion commands and perspective, autofocused sinks, narrowcastingattributes, designation as “self,” and position.

Fig. 3. Java3D narrowcasting interface with phantom sources: Avatars in green (second from the right) and blue (rear) with stars overhead act as sinks,listener delegates for the user, and avatars in red (front left) and white (rightmost) are sources. Though the best sink for both sources is the green avatar inthis particular configuration, the best sink depends in general on the position of the sinks and sources and the narrowcasting attributes. Each source’s arrowsfly to its respective best sink. This space’s sources are heard before and behind the green avatar. Visual phantom sources are drawn silently displaced relativeto the selected avatar (blue in this example), while auditory phantom sources are invisibly displaced relative to the virtual camera. (The red cube and whitesphere straddling the blue avatar represent the red and white sources, reflecting the displacement from the green sink.) The sound heard by the avatars isindependent of the viewpoint (virtual camera position).

137

Page 6: Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka.

an autofocus function, works-around the Java3D assumption ofcoupling of listening and viewing points.

Phantom sources are used to control superposition of sound-scapes relative to a selected viewpoint. Relative displacementfrom sources e sinks can be used to display phantom sourcesfrom alternate locations, exocentrically visibly and endocentri-cally auditorilly. Logical separation of viewpoint and listeningpoint is used to overcome Java3D assumptions and makethe interface more fluid. An extra benefit of the phantomsource displacement is the accommodation of a rotatable stereospeakers axis (including longitudinal as well as transversalarrangement).

The fictional Point-of-View Gun in the movie “The Hitch-hiker’s Guide to the Galaxy” [18] is a very unusual weapon.Put simply, when you fire this gun at someone, they instantlyunderstand things from your perspective. It was invented by agroup of housewives who had become utterly sick of endingevery domestic argument with the words “You just don’tget it, do you?” The gun was immediately in huge demandby marketing agencies, governments, and religions, but wasdeemed to be too dangerous in the wrong hands and so waslocked away in a secret location to prevent it from ever beingused again. The Point-of-View Gun is not yet available onEarth!

Virtual reality user interfaces need this kind of feature toallow users to practice empathy and compassion on their own.Our Multiplicity interface supports these empathic featuresvisually and auditory with phantom sources. When a userselects an avatar in the left-most column of the Mixels panel,the visual and auditory superposition of the multipresentsinks’ soundscapes are drawn relative to the selected avatar.Visual phantom sources show the relative displacements ofthe sound sources from the respective best sinks, as shownin Fig. 3. Auditory phantom sources reflecting the relativepositions of each source with respect to its respective bestsink are heard relative to the currently selected virtual camera(exocentric visual perspective). The consolidated soundscapeheard by the user, composed of phantom auditory sources, doesnot change with Mixel panel-selected avatar. Users can seeand hear different avatars’ perspectives, including their ownmultipresent avatars or someone else’s. Also, if a user pressesthe View 0 visual perspective button, then the user can see afirst-person perspective of the selected avatar, which might betheir own or someone else’s avatar’s first-person perspective.

III. b5c CON: (I f PPLI DOJA) MOBILE DEVICE-PLATFORMED

DYNAMIC MAP

A. I f ppli: I-mode with Java

I-mode’s Java-based i f ppli service allows users of compat-ible terminals to download software and content from about athousand web sites. Users can use the downloaded applicationsand content whenever they want, without having to reconnectto the internet. Launched in January 2001, i f ppli is the resultof wireless technology research conducted by NTT DoCoMowith Sun Microsystems. I f ppli uses the Java platform devel-oped by Sun for consumer electronics and built-in devices,

Fig. 4. NTT DoCoMo i-mode MIDlet (iappliTool for DoJa) running “ gPh Con.”Workstation commands are synchronized with mobile interfaces Home andSchool spaces. Quasi-realtime synchronization with a CVE server motivatesthe use of “ghost icons,” shown as outlines in the Home space, to distinguishlocal and session states of avatars. In this example, in the Home space, #2 isattended and #3 is selected, so their complements (in the same space) areimplicitly deafened & muted respectively; #1 is simultaneously implicitlydisabled (deafened & muted) and also selected for rotation (as indicatedby its “halo”). Meanwhile, in the School space, #0 is muted and #3 isdeafened.

as well as i-mode’s extended library, which was developedjointly by the two companies. Since the data processing powerof mobile phones is far less than that of PCs, i-mode usesa version of Java called “KVM” that runs on systems withrelatively low processing power. A key feature of the KVM

environment is that its security functions are tighter than thoseof standard Java.

B. Implementation

We have designed and implemented a mobile telephoneinterface (as shown in Figs. 4 and 6(b)) [19] for use inCVEs. Programmed with J2ME (java.sun.com/j2me),our application runs on (NTT DoCoMo (www.nttdocomo.com)) i f ppli mobile phones, as illustrated by Fig. 4. Featuringselectable icons with one rotational and two translationaldegrees of freedom, the b5c Con 2.5D dynamic map interfaceis used to control position, sensitivity, and audibility ofavatars in a groupware session. Its isosceles triangle icons

138

Page 7: Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka.

(a) School space in Multiplicity (b) Home space in Multiplicity

Fig. 5. Both instances (corresponding to School and Home spaces) of Multiplicity run by the same user on one machine use the system clipboard for teleporting(cut/paste) and cloning (copy/paste). In this example, avatars #2 and #3 have been cut from the School space along with their narrowcasting attributesand pasted into the Home space, as avatars #4 (front left green) and #5 (front right blue).

(a) Home space in Multiplicity on different machine (b) Home and School spaces of gPh Con (Mobile Interface) inthe sessions

Fig. 6. Distributed and heterogeneous (workstation and mobile) interfaces run in the sessions: allow create and delete operations for teleporting and cloning. Inthis example, avatars #4 (front left green) and #5 (front right blue) are newly created avatars according to space name and channel number with narrowcastingoperations. Avatar #4 is selected and #5 is attended, so their complements (comprising all the other avatars) are implicitly muted and implicitlydeafened respectively.

139

Page 8: Cloning and Teleporting Avatars Across Workstations and Mobile … · 2005-12-13 · Proceedings of the International Conference on Information and Automation, December 15-18, 2005,

Proceedings of the International Conference on Information and Automation, December 15-18, 2005, Colombo, Sri Lanka.

are representations of symbolic heads in an orthographicprojection, including narrowcasting attributes shown in Table I,which operations are shown in Table II. The icons respond toteleporting & cloning operations, shown in Table III. b5c Conhas vibrational cues to signal mode changes and successfultransmission/reception (which feedback is especially importantin wireless communication).

TABLE II

MNEMONIC INITIALS OF NARROWCASTING OPERATIONS ON THE

ALPHANUMERIC KEYPAD USED TO TOGGLE SELECTION SET ATTRIBUTES.

attendABC2

deafenDEF3

muteMNO6

select (solo)PQRS

7

sink/selfGHI4

In our bdc Con application, narrowcasting attributes’ graphicaldisplays are triply encoded— by position (before the “mouth”for mute and select, straddling the “ears” for deafenand attend), symbol (‘+’ for assert & ‘–’ for inhibit, asshown in Table I), and color (green for assert & red and yellowfor inhibit). The attributes are not mutually exclusive, and theencoding dimensions are orthogonal.

TABLE III

KEYS USED TO INVOKE CLIPBOARD AND UTILITY HISTORY OPERATIONS.

undo 1 1redo 1 2cut 1 3copy 1 4paste 1 5delete 1 6

The autofocus feature is also applied to the bdc Con interface.bdc Con can display best sinks corresponding to each source. Weuse two methods to visualize autofocus: drawing a circle abovethe corresponding icons, and flashing corresponding icons withsame color.

IV. CONCLUSION

We have developed idioms for selective attention, privacy,and presence across multiple spaces: narrowcasting for au-dio windowing systems, whether via workstation or a no-madic device like a mobile phone. The platform-agnosticdeployment of the audio narrowcasting operations encouragesthe modernization of office- and mobile-based conferencing,leveraging session integration across spaces and anticipatingmultipresence enabled by higher bandwidth and more durablemobile connectivity. Clipboard operations have been appliedto avatar presence to allow teleporting (cut/paste) andcloning (copy/paste).

REFERENCES

[1] N. A. Bolhassan, M. Cohen, O. N. N. Fernando, T. Kamada, W. L.Martens, H. Osaka, and T. Yoshikawa, ““Just Look At Yourself!”:Stereographic Exocentric Visualization and Emulation of StereographicPanoramic Dollying,” in Proc. ICAT: Int. Conf. on Artificial Realityand Tele-Existence, University of Tokyo, Dec. 2002, pp. 146–153,vrsj.t.u-tokyo.ac.jp/ic-at/ICAT2003/papers/02146.pdf.

[2] M. Cohen, T. Azumi, Y. Yatsuyanagi, M. Sasaki, S. Yamaoka, andO. Takeichi, “Networked Speaker Array Streaming Back to Client: theWorld’s Most Expensive Sound Spatializer?” in Proc. ICAT: Int. Conf.on Artificial Reality and Tele-Existence, Tokyo, Dec. 2002, pp. 162–169, vrsj.t.u-tokyo.ac.jp/ic-at/ICAT2003/papers/02162.pdf.

[3] O. N. N. Fernando, K. Adachi, U. Duminduwardena, M. Kawaguchi, andM. Cohen, “Audio Narrowcasting and Privacy for Multipresent Avatarson Workstations and Mobile Phones,” IEICE Trans. on Information andSystems, vol. E89-D, no. 1, Jan. 2006.

[4] S. Benford, J. Bowers, L. E. Fahlen, J. A. Mariani, and T. Rodden,“Supporting Cooperative Work in Virtual Environments,” Comput. J.,vol. 37, no. 8, pp. 653–668, 1994.

[5] C. Greenhalgh and S. Benford, “Massive: A collaborative virtualenvironment for teleconferencing,” ACM Trans. on Computer-HumanInteraction, vol. 2, no. 3, pp. 239–261, Sept. 1995.

[6] S. Benford, J. Bowers, L. Fahlen, C. Greenhalgh, J. Mariani, andT. Rodde, “Networked virtual reality and cooperative work,” vol. 4, no. 4,pp. 364–386, 1995, ISSN 1054-7460.

[7] C. Greenhalgh and S. Benford, “Boundaries, awareness and interactionin collaborative virtual environments,” in WET-ICE ’97: Proc. 6th Int.Wkshp. on Enabling Technologies: Infrastructure for CollaborativeEnterprises. IEEE Computer Society, 1997, pp. 193–198, ISBN 0-8186-7967-0.

[8] S. Benford, C. Greenhalgh, G. Reynard, C. Brown, and B. Koleva,“Understanding and Constructing Shared Spaces with Mixed-RealityBoundaries,” ACM Trans. on Computer-Human Interaction, vol. 5, no. 3,pp. 185–223, Sept. 1998.

[9] M. Cohen, “Exclude and include for audio sources and sinks: Analogs ofmute & solo are deafen & attend,” Presence: Teleoperators and VirtualEnvironments, vol. 9, no. 1, pp. 84–96, Feb. 2000, ISSN 1054-7460;www.u-aizu.ac.jp/ i mcohen/welcome/publications/ie1.pdf.

[10] S. DiPaola and D. Collins, “A 3D Virtual Environment for SocialTelepresence,” in Proc. Western Computer Graphics Symp., Vernon, BC,Canada, Mar. 2002.

[11] N. Tsingos, “Perceptual audio rendering of complex virtual environ-ments,” in Proc. of SIGGRAPH, San Diego, Aug. 2004, pp. 249 – 258.

[12] A. Dix, A. Friday, B. Koleva, T. Rodden, H. Muller, C. Randell,and A. Steed, “Managing multiple spaces,” in Spaces, Spatiality andTechnology, ser. The Computer Supported Cooperative Work Series,P. Turner and E. Davenport, Eds., vol. 5. Springer, 2005, ISBN 1-4020-3272-2.

[13] T. Kanno and M. Cohen, “An architecture for collaborative virtualenvironments,” 3D Forum: J. of Three Dimensional Images, vol. 16,no. 1, pp. 166–174, Mar. 2002, ISSN 1342-2189.

[14] E. F. Churchill, D. N. Snowdon, and A. J. Munro, Eds., CollaborativeVirtual Environments: Digital Places and Spaces for Interaction, ser.Computer Supported Cooperative Work (CSCW). Springer, 2001,ISBN 1-85233-244-1.

[15] M. Cohen and W. L. Martens, “Spatial Media Research at the Universityof Aizu,” JVRSJ: J. of the Virtual Reality Society of Japan, vol. 6, no. 2,pp. 52–57, Sept. 2001, ISSN 1342-6680.

[16] O. N. N. Fernando, K. Adachi, and M. Cohen, “Phantom sources forseparation of listening and viewing positions for multipresent avatarsin narrowcasting collaborative virtual environments,” in Proc. MNSA:Int. Wkshp. on Multimedia Network Systems and Applications (inconjunction with ICDCS: 24th Int. Conf. on Distributed ComputingSystems), Hachioji, Tokyo, Mar. 2004, pp. 170–175, ISBN 0-7695-2087-1.

[17] H. Ramis and C. Miller, “Multiplicity,” 1996, Columbia Pictures Corp.[18] G. Jennings and D. Adams, “The Hitchhiker’s Guide to the Galaxy,”

2005, Touchstone Pictures.[19] O. N. N. Fernando, M. Cohen, U. C. Dumindawardana, and

M. Kawaguchi, “Duplex narrowcasting operations for multipresentgroupware avatars on mobile devices,” IJWMC: Int. J. of Wireless andMobile Computing (Special Issue on Mobile Multimedia Systems andApplications), vol. 1, no. 5, 2005, ISSN 1741-1084, 1741-1092.

140