Accommodating Global Policies and User Preferences in Computer Supported Collaborative Systems...

Accommodating Global Policies andUser Preferences in Computer Supported Collaborative Systems

Student:

Helmuth Trefftz

Advisors:

Prof. Rick Mammone

Prof. Ivan Marsic

Rutgers University

April, 2001

Agenda

• Motivation• Related Work• Thesis Statement• Experimental Setup• Mathematical Model• Preliminary Results• Future Work• Conclusions

Motivation (1)

• We are multi-modal

Motivation (1)

• We are multi-modal• We acquire information

through multiple modalities (sensory channels):– Sight– Sound– Smell– ..

Motivation (1)

• We are multi-modal• We acquire information through

multiple modalities (sensory channels): :– Sight– Sound– Smell– ..

• Most intense experiences involve multiple modalities (love)

Motivation (2)

• Computer Systems become more and more multimodal:

Motivation (2)

• Computer Systems become more and more multimodal:– e-mail, chat

Motivation (2)

• Computer Systems become more and more multimodal:– e-mail, chat– chat with cartoons (Microsoft)

Motivation (2)

• Computer Systems become more and more multimodal:– e-mail, chat– chat with cartoons (Microsoft)– CuSeeMe

Motivation (2)

• Computer Systems become more and more multimodal:– e-mail, chat– chat with cartoons (Microsoft)– CuSeeMe– NetMeeting, Face Mail– …

• Integration of text, voice, video

Motivation (2)

• Face Mail:• Input

– Text + “:- ):

• Output– Synthesized voice– Animated character

• Multiple Info. Rep.!

Motivation (3)

• Multiple levels of information representation

Motivation (3)

• Multiple levels of information representation– Some years ago: browsers showed

a low-resolution image first, then refined

Motivation (3)

• Multiple levels of information representation– Some years ago: browsers showed

a low-resolution image first, then refined

– In Virtual Environments: distant objects are represented with lower Level Of Detail

Motivation (3)

• Multiple levels of information representation– Some years ago: browsers showed

a low-resolution image first, then refined

– In Virtual Environments: distant objects are represented with lower Level Of Detail

– “FaceMail”: you type text, output: voice + avatar

Motivation (4)

• Multiple dimensions of information representation

Motivation (4)

• Multiple dimensions of information representation– Text: easier to store, easier to

search

Motivation (4)

• Multiple dimensions of information representation– Text: easier to store, easier to

search– Sound: hands-free, ‘warmer’

interaction

Motivation (4)

• Multiple dimensions of information representation– Text: easier to store, easier to

search– Sound: hands-free, ‘warmer’

interaction– Text -> Sound: MS Whisper– Sound -> Text: IBM Via-voice

Motivation (6)

• Multiple dimensions of information representation (VSI):

• Company I used to work for.• Product: multi-modal

representation of information for distance learning.

• Channels: video + sound + text + animations.

Motivation (6)

• Multiple levels of information representation (VSI)– Video + Sound (engaging)

Motivation (6)

• Multiple levels of information representation (VSI)– Video + Sound (engaging)– Graphics + Animations (concepts)

Motivation (6)

• Multiple levels of information representation (VSI)– Video + Sound (engaging)– Graphics + Animations (concepts)– Text (search)

Motivation (6)

• After months of separate work in video, text, graphics

Motivation (6)

• After months of separate work in video, text, graphics

• Integrated product was… NOT SMOOTH!

Motivation (6)

• After months of separate work in video, text, graphics

• Integrated product was… NOT SMOOTH!

• Elaborate animations made video choke!

Motivation (6)

• Integration: find the appropriate space in a multi-dimensional space:

Video + Audio

Graphics

Motivation (7)

• Multiple values in each dimension:

Motivation (7)

• Multiple values in each dimension:– Video: resolution, frames per

second

Motivation (7)

• Multiple values in each dimension:– Video: resolution, frames per

second– Sound: sample resolution, sample

rate

Motivation (7)

• Multiple values in each dimension:– Video: resolution, frames per

second– Sound: sample resolution, sample

rate– In Networked Virtual

Environments: update rate

Motivation (7)

• Multiple values in each dimension:– Video: resolution, frames per

second– Sound: sample resolution, sample

rate– In Networked Virtual

Environments: update rate– Text: ?

Motivation (8)

• Higher values in each dimension:

Motivation (8)

• Higher values in each dimension:– More space to store

Motivation (8)

• Higher values in each dimension:– More space to store– More CPU cycles to process

Motivation (8)

• Higher values in each dimension:– More space to store– More CPU cycles to process– More bandwidth to transmit

Motivation (9)

• Computers in a Collaborative System

Adapted from: http://www.intel.com/intel/museum/25anniv/hof/moore.htm

generation

performance

i i+1i - 1

Motivation (9)

• Computers in a Collaborative System:

• Disparities among consecutive generations will grow larger in:– Processor speed– Memory– Bandwidth

Motivation (9)

• How to cope with these disparities?

Slower computers Faster computers

Motivation - Summary

• Multiple modalities - multiple dimensions

Motivation - Summary

• Multiple modalities - multiple dimensions

• Multiple qualities of information representation - multiple points

Motivation - Summary

• Multiple modalities - multiple dimensions

• Multiple qualities of information representation - multiple points

• Performance - limits the “valid” hyper space

Related Work (1)

• CVEs in Distance Education in Colombia

Related Work (1)

• CVEs in Distance Education in Colombia– Collaborative Virtual Environment

as learning experience

Related Work (1)

• CVEs in Distance Education in Colombia– Collaborative Virtual Environment

as learning experience– AVALON: (Carlos Correa)

• Multiple modalities:– Avatars, VRML worlds, OpenGL– Voice: multicast version of “Speak

freely” (Francisco Cardona, now at Swiss Federal Institute of Technology (EPFL) )

Related Work (1)

• CVEs in Distance Education in Colombia– Used in a real learning

environment (Environmental Issues class at Eafit University).

– Improvement in learning measured with the “Teaching for Understanding” model (Harvard)

Related Work (2)

• Michael Macedonia - Ph.D. thesis (1995)– Partition the space in hexagonal

spaces– Each hexagonal space is a

multicast group– Implemented in DIS (Distributed

Interactive Simulation) and SIMNET (Simulator Networking)

Read Thesis Statement

Related Work (2)

• Thesis Statement:“Virtual environment software

architectures can exploit wide area multicasting communications and

entity relationships to partition the virtual world and enable the

development of scalable distributed interactive simulations for military

projects”.(Michael Macedonia, 1995)

Related Work (3)

• Michael Capps - Ph.D. thesis (2000)– Have multiple representations for

each object– Define which representation to

use based on:• Quality• Importance• Cost

Our Thesis Statement

“It is possible to express global policies and individual users’ preferences in

collaborative multimodal systems as a formal mathematical model.

Solving this mathematical model, which includes performance measurements taken at each

participating node, guarantees enforcement of the policies while

allowing individual users to adjust their interaction with the system.”

Thesis Statement

Assumptions

• Collaborative System• Diverse degrees of computing

power• Nodes have limited computing

power• Shared information: varying

levels of fidelity in time/space dimensions

Experimental Setup (1)

• Distributed visualization system• Users share a Visualization Data

Set• “Video” of the other

participants • Telepointers• One user can manipulate the

object at a time

Experimental Setup (2)

• User Interface

Experimental Setup (3)

• Information Dimensions (variables):

Experimental Setup (3)

• Information Dimensions (variables):– Video

Experimental Setup (3)

• Information Dimensions (variables):– Video– Object Movements

Experimental Setup (3)

• Information Dimensions (variables):– Video– Object Movements– Telepointer Movements

Experimental Setup (3)

• Information Dimensions (variables):– Video– Object Movements– Telepointer Movements– Graphic complexity used locally

for the visualized data set

Experimental Setup (3)

• Information Dimensions (variables):– Video– Object Movements– Telepointer Movements– Graphic complexity used locally

for the visualized data set– Rendered object frames per

second

Experimental Setup (4)

• Independent variables:– Video– Object Movements– Telepointer Movements– Graphic complexity used locally

for the visualized data set

• Dependent variable:– Rendered object frames per

second

Experimental Setup (5)

• Mapping from independent to dependent variable:

Experimental Setup (5)

• Mapping from independent to dependent variable:

• “Performance Mapping”

Experimental Setup (5)

• Mapping from independent to dependent variable:

• “Performance Mapping”

• Individual for each machine• Determined before the

collaborative session running a simulated session

Experimental Setup (6)

• Architecture:

Server

Client 1 Client 2 Client n

Experimental Setup (6)

• Switchboard analogy

Mathematical Model (1)

• Measurable way to express:– User preferences– Minimum performance levels – Maximum level of service that can

be offered• Server processing cycles• Available bandwidth

Mathematical Model (1)

• User preferences: (i)

• Objective Function.

})30/50(.)20/25(.)20/0()3/25(.)20/0{( FMTGVMax

Mathematical Model (1)

• Minimum performance levels: (ii)– V >= 1– G >= 1– T >= 5– M >= 1– F >= 2

Mathematical Model (1)

• Maximum level of service: (iii)– Sum (Vi) <= 100

– Sum (Ti) <= 150

– Sum (Mi) <= 120

– Gi and Fi have only local impact

Mathematical Model (1)

• Can be solved locally (at each machine):– Individual Objective Function +

type (ii) restrictions

• Require a global solution:– Type (iii) restrictions + Global

Objective Function

• Possibility to solve the first part in parallel.

Mathematical Model (2)

• Combining the three types of equations

• A “LINEAR PROGRAMMING” OPTIMIZATION PROBLEM

Mathematical Model (2)

• Combining the three types of equations

• A “LINEAR PROGRAMMING PROBLEM”

• Each local solution:– Valid search space enforcing

global policies– Optimize local Objective Function

allowing each user to adjust her interaction

Mathematical Model (2)

• Global Solution:– Determine (and enforce) top

limit on service handled by the server and the network

Mathematical Model (3)

• Challenges:– In classic Linear Programming

variables take continuous values.– In our problem: discrete values.– Parallel processing of partial

solutions– Once a solution is found, if a

condition changes, the next solution should be found without restarting the computation.

Mathematical Model (3)

• Possible solutions:• Use integer programming• Use piece-wise linear

programming• Non-linear optimization

techniques

Preliminary results (1)

• Server:– Accepts TCP connections for meta-

information – A predefined multicast group for

each “socket” in the switchboard

TCP

UDP multicastTimer

UDP unicast

Preliminary results (2)

• Client:– Establishes a TCP connection for

subscription messages– Receives simulated updates from

the server

TCP

UDP (mcast)

TimerSS

UDP (unicast)

Preliminary results(3)

• Computers:– Dragonfire: Pentium II @ 350 MHz,

64MB main memory– Morlak: Pentium II @ 400 MHz,

256 MB main memory

Preliminary results(3)

• 3D Models:– Guts: 27,202 vertices– Skull: 1,210 vertices

Preliminary results(4)

• “Idle” performance:

SkullGuts

Dragonfire

Morlak0

20

40

60

80

100

No remote updates

Preliminary results(5)

• Effect of video-events frequency:

No

upda

tes

1000

ms

500

ms

200

ms

100

ms

Morlak GutsMorlak Skull0

1020304050607080

Video Updates

Preliminary results(5)

• Effect of object-movement-events frequency:

No

upda

tes

1000

ms

500

ms

200

ms

100

ms

Morlak GutsMorlak Skull0

102030405060708090

Object Updates

Preliminary results(5)

• Note that video events have more impact on the local performance than object movement events.

Preliminary results(6)

• Resulting performance mapping (Skull):

1

2

3

4

5

1

2

3

4

5

40

50

60

70

80

90

Object Updates

Object x Video Morlak Skull

Video Updates

Avera

ge F

PS

Preliminary results(6)

• Resulting performance mapping (Guts):

1

2

3

4

5

1

2

3

4

5

4

4.5

5

5.5

6

6.5

7

Object

Object x Video Morlak Guts

Video

Avera

ge F

PS

Future Work (1)

• Server: – Receive and cache (not queue)

messages.

Future Work (1)

• Server: – Receive and cache (not queue)

messages.

• Client:– Implement frame grabbing– Send updates

Future Work (2)

• Math:– Local solution for type (i) and type

(ii) equations– Global solution involving type (iii)

equations– Choose solver:

• Initially: “brute force” (search through the finite search space)

• Linear programming?• Non-linear programming?

Conclusions (1)

• Mathematical model provides objective solution for two types of conflicting situations:

• Diverse Computing power of the nodes

• Global policies and users’ preferences

Conclusions (2)

• Assigning different multicast groups to the different values the variables can take make the solution scalable.

Thank you!