Distributed Graphics: Two Case Studies Jian Huang, CS 594, Spring 2002

Distributed Graphics: Two Case Studies

Jian Huang, CS 594, Spring 2002

Distributed Graphics

• Two major categories:– Data can be replicated– Data can not be replicated

• Dependent on the choice– Drastically different approach– Specific pros and cons

This Lecture

• Look at a sample system in each category:– A Distributed 3D Graphics Library, Blair MacIntyre

and Steven Feiner, SIGGRAPH’2002

– Deep Compression for Streaming Texture Intensive Animations, Daniel Cohen-Or, Yair Mann, Shachar Fleishman, SIGGRAPH’1999

• Lots of research issues are currently still open problems

Distributed Graphics• Systems for distributing the shared graphical state

of multi-display, multi-person, distributed, interactive applications

Computer Supported Corporative Work (CSCW) and Distributed Virtual Environments (DVE)

High Level Libraries

• Have excellent programming libraries for stand-alone 3D applications: Inventor, Performer, Java3D, …

• No such libraries for distributed graphics applications, although some DVE systems have been proposed:– A mechanism to distribute application state (either a custom

solution or using a general-purpose lib)– Maintain the state of the graphical display separately and locally

• A “dual databases” approach– Synchronizing the dual database is complex, tedious, and error-

prone

General Goal• To use a “single database” approach for distributed 3D

graphics library• Repo-3D is an object-oriented, high-level graphics package

– The objects used to create the graphical scenes are directly distributable from the programmer’s perspective

– The objects reside in one large distributed shared memory (DSM)

– The underlying system automatically replicates the fine-grained objects across as many processes as needed

– Updates to objects are automatically reflected in all replicas, with any required objects automatically distributed as needed.

Different between Distributed and Monolithic Applications

• Distributed control– In a non-trivial distributed application, different

components need to be notified of changes to the distributed state

• Interactivity– Network latency bandwidth limitations make updates to

distributed state much slower. For performance, need to perform some operations locally

• Local variations– There are times when a shared graphical scene may need

to be modified locally

Repo-3D• Repo-3D addresses these problems in two ways:

– Notification object: associated with any replicated object. Invoke notification object’s methods when the replicated object is updated.

– Local variation: to allow the properties of a graphical object to be modified locally, and parts of the scene graph to be locally added, removed, or replaced.

Motivation• Authors’ fundamental belief:

– With uniform high-level support for distributed programming in the languages and toolkits, prototyping and experimenting with distributed interactive applications can be (almost) as simple as multi-threaded programming in a single process

• Useful for:– True with exploration and prototyping phase

– Object-oriented programming (high data transparency)

Client-Server Data Sharing Inappropriate for Distributed Graphics

• Client-server data sharing (e.g. CORBA,

Modular-3 Network Objects, Java RMI) impossible for high frame-rate distributed graphics

• Data replication is necessary

• Update of shared/replicated data should be asynchronous

Underlying Infrastructure• Developed in Modula-3 language: descendant of

PASCAL, and heavily influenced Java

• In particular, Modula-3:– strong type safety

– exception handling

– concurrency

– object-oriented programming

– automatic garbage collection

Distributed Objects

• Within the paradigm of distributed shared memory (DSM)

• Three types of objects– Simple objects: normal local data obj

– Remote objects: client-server semantics. All method invocations go across the network to the original object

– Replicated objects: replicated at all locations, need to be consistently updated

Remote Objects

• The Network Object package provides support for remote objects

• It implements – distributed garbage collection

– exception propagation back to the calling site

– automatic marshalling and unmarshalling of method arguments

– return values of virtually any data type between heterogeneous machine architectures

Replicated Objects• The Replicated Object package supports replicated objects.

Each process can call any method of an object it shares

– All operations on an instance of an object are atomic and serializable

– All operations are performed in the same order on all copies of the object

– If two methods are invoked simultaneously, the order of invocation is nondeterministic

– Full replication: no partial replication

Update Scheme for Replicated Objects

• Each method is marked either read or update• Only update methods trigger an update event• Use a sequencer process (maybe a remote process)

to enforce the same sequence of operations for a replicated object

• May have multiple sequencers in the distributed system

• All replicas of an object, including the one invoked this update method, executes the update after receiving update msg from sequencer

Anim-3D

• Graphical objects (GO): – all entities of a graphical scene: geometry, light, …– GOs can be grouped in any Directed Acyclic Graph

(DAG)– Each GO is associated with a set of properties

• Properties: – behavior of GO– Defined by a name and a value:

• say, TextureReconstruction, nearest neighbor

• Input event call-backs

GO• GO hierarchy:

• GO property of g, is inherited by all g’s descendants, unless overridden

• But, property is always attached to separate nodes• Change of a property won’t affect its siblings

Property

• 4 behavior types for each property:– Constant– Synchronous– Asynchronous– Dependent

Basic Design of Repo-3D

• The interface is programmatical and declarative– No graphics scene file format: all scenes exist

as a Repo-3D program– Scenes can be updated with a new Repo-3D

program as a replicated object

Replicated Objects?

• GO: 21 GO types, all replicated, except RootGO, which corresponds to window to be rendered into

• Properties: most of the Modula-3 objects that comprise a property are not replicated– Properties with a constant binding between name and

value

– All names

– Behaviors and requests

• Input callbacks are not replicated

Local Variations

• Needed to maintain interactivity and local modifications to shared scene graph

• Local state: state added to each object – only accessible to the local copies – not passed to remote processes when the object is copied to create a

new replica– Typically: additions, deletions, and replacements to the properties or

children

• New methods that manipulate the local variations are added to the GOs as non-update method

• Repo-3D combines both the global and local state when creating the graphical scene

Local Variations - property

• Property variations. – 3 methods to set, unset, and get the global property list

attached to a GO– methods to manipulate local variations: – add or remove local properties (overriding the value

normally used for the object)– hide or reveal properties (causing the property value of

the parent node to be inherited)– flush the set of local variations (removing them in one

step) or atomically apply them to the global state of the object

Local Variation - children

• Child variations– 5 methods to add, remove, replace, retrieve, and flush

the set of children contained in a group node

– Additional ones for local variation: • add a local node

• remove a global node locally

• replace a global node with some other node locally

• remove each of these local variations

• flush the local variations (remove them all in one step)

• atomically apply the local variations to the global state

Example

Streaming Texture Intensive Applications

• The problem faced by all network-based applications is bandwidth

• A network-based compression scheme is a two-part compressed sequence– header download– streaming

• MPEG takes advantage of frame-to-frame coherency only, still too big for streaming

• Need to reduce to 2KB/sec bandwidth

Introduction

• Synthetic animations have higher potential to be compressed

• because we know more :– geometry– texture data– (temporal) viewing information

Background

• An MPEG video sequence consists of intra-frames (I), predictive (P) and interpolated frames (B)

• The compression ratio is depends on the successful motion estimation

• For synthetic animation, model-based motion estimation could provide further improvement

Background (2) : A different approach [Levoy]

• Render the animation on-the-fly at both ends of communication rather than compress post-rendered images

• Sever: render high and low quality images, compute the residual error between them, and transmit the residual image compressed.

• Client: render only the low quality images and to add the transmitted residual images.

Background (3) [Levoy]

• Require pre-computing the residual images and downloading entire model and animation sequence before streaming

• Not clear how to treat texture intensive animations

• Other related techniques use imposters and sprites

View-dependent Texture Streaming

•All you need: streaming the environment requires the server to transmit the animation script and visible parts of the model.

•No need for the whole texture

•It’s better to use nearby views as textures rather than using the original textures

• Nearby views are advantageous as texture because:– they have an “almost” one-to one

correspondence with the current view and therefor

– they are post-rendered and may include various global illumination effects.

View-dependent Texture Streaming(2)

View-dependent Texture Streaming(3)

• The reuse could be either forward or backward

• Occasionally need to request new texture from server from pre-computed future frames

• Always need to consider geometry streaming (visibility, gaps …)

View-dependent Texture Streaming(4)

The Texture Quality Factor

• Texture quality factor: a local scaling factor to estimate that locally shrink or expand when mapped onto target image.

• 2-Norm of matrix A

• sqrt(max Eigenvalue)

The Texture Quality Factor (linear case)

The Texture Quality Factor (non-linear case)

The Texture Quality Factor (non-linear case)

Geometry Streaming

• Geometry stream gets a higher priority than view-dependent texture stream

• Use stencil buffer to identify visibility information for each frame

Bandwidth (compared with MPEG)

Two Reference Views

Three In-Between Views

Results

More results

More results