On-demand transmission of 3D models over lossy networks · 2020. 7. 31. · level (UEP/SR), and a transport protocol integrated with TCP-friendly rate control (TFRC). Next, we brieﬂy

ARTICLE IN PRESS

0923-5965/$ - se

doi:10.1016/j.im

�Correspondfax: +1912 966

E-mail addr

[email protected]

Signal Processing: Image Communication 21 (2006) 396–415

www.elsevier.com/locate/image

On-demand transmission of 3D models over lossy networks

Dihong Tian, Ghassan AlRegib�

School of Electrical and Computer Engineering, Georgia Institute of Technology, 210 Technology Circle, Savannah, GA 31407-3039, USA

Received 7 September 2005; accepted 25 January 2006

Abstract

Three-dimensional (3D) meshes are used intensively in distributed graphics applications where model data are

transmitted on demand to users’ terminals and rendered for interactive manipulation. For real-time rendering and high-

resolution visualization, the transmission system should adapt to both data properties and transport link characteristics

while providing scalability to accommodate terminals with disparate rendering capabilities. This paper presents a

transmission system using hybrid unequal-error-protection and selective-retransmission for 3D meshes which are encoded

with multi-resolutions. Based on the distortion-rate performance of the 3D data, the end-to-end channel statistics and the

network parameters, transmission policies that maximize the service quality for a client-specific constraint is determined

with linear computation complexity. A TCP-friendly protocol is utilized to further provide performance stability over time

as well as bandwidth fairness for parallel flows in the network. Simulation results show the efficacy of the proposed

transmission system in reducing transmission latency and providing smooth performance for interactive applications. For

example, for a fixed rendering quality, the proposed system achieves 20–30% reduction in transmission latency compared

to the system based on 3TP, which is a recently presented 3D application protocol using hybrid TCP and UDP.

r 2006 Elsevier B.V. All rights reserved.

Keywords: Distributed 3D graphics; Interaction; Multi-resolution compression; Unequal error protection; Selective retransmission; TCP-

friendly rate control; Multimedia streaming

1. Introduction

Internet-based multimedia applications are ex-panding from streaming video/audio to distributed3D graphics, driven by growing demands of variousapplications such as electronic commerce, colla-borative CAD, medical and scientific visualization,and virtual environments. Interaction is one of thekey aspects of a distributed graphics application.

e front matter r 2006 Elsevier B.V. All rights reserved

age.2006.01.003

ing author. Tel.: +1 912 966 7937;

7928.

esses: [email protected] (D. Tian),

ech.edu (G. AlRegib).

Most of the distributed graphics applicationsrequire certain level of interaction with the objectsinvolved. Some applications such as immersiveenvironments and shared reality thoroughly involveusers in the interaction. Response time, which is thelatency between the user input and the response(e.g., scenes displayed on the user’s terminal) fromthe system, is one of the major considerations indesigning high-performance distributed graphicssystems. In contrast to specific 3D systems thatassume all models are locally available and areessentially designed as stand-alone systems, distrib-uted graphics applications often require on-demandexchange of 3D data in a networked environment,and impose requirements of real-time response and

.

www.elsevier.com/locate/image

ARTICLE IN PRESSD. Tian, G. AlRegib / Signal Processing: Image Communication 21 (2006) 396–415 397

smooth performance over time on the interaction.In addition, 3D data differ from general mediacontent for it requires rendering capability on theuser terminal. Sending the same dimension of datato user terminals with disparate rendering capabil-ities may result in significant difference in responsetime between the different clients. Scalability of the3D data, as well as the transmission, is thereforedesirable. All the aforementioned requirements, notonly have promoted the use of high-performancecomputing systems and distributed platforms, butalso call for careful considerations on ways ofreducing transmission latency, providing scalability,and maintaining high-resolution visualization whennetwork delays and random data losses are in-volved.

Multi-resolution compression of 3D meshes[1,16,19–21,23,24] is a partial solution to providescalability for 3D data. Using multi-resolutionencoders, the server can select the appropriateresolution for a particular client according to itsquality requirement, or initially sends a coarserepresentation of the 3D model to the client forquick reconstruction and rendering, and thentransmits refinement layers which allow the clientto gradually increase model fidelity toward higherresolutions. Although, such methods are successfulin exploring the space and time efficiency of 3Ddata, higher efficiency can be accomplished byaddressing the effects of network behaviors. Inparticular, to display 3D scenes on the user’sterminal with satisfactory quality and in real time,the impact of packet losses and transmission delayson the decoding process need to be explored.Typically, reliable or error-resilient transmissioncan be achieved by pre-processing techniques suchas data partitioning [7,17,27], post-processing tech-niques such as error-concealment, and network-oriented techniques such as forward error correc-tion [2,6,4] and retransmission techniques[3,5,8,9,14]. All these techniques address efficienttransmission of 3D data separately. Yet an appro-priate combination of such techniques is desired toachieve better performance. Interaction and trade-offs among the selected techniques need to beinvestigated, taking into account the distortion-rateperformance of the 3D data and the networkcharacteristics.

In this paper, a hybrid mechanism of unequal-error-protection and selective-retransmission is pro-posed for multi-resolution meshes. Hierarchicaldata batches of the multi-resolution mesh are

protected preferentially according to their distor-tion-rate performance, network parameters, andchannel statistics estimated by the transport layer.To minimize response time in interaction, theproposed mechanism is designed to have linearcomputational complexity. In addition, by integrat-ing TCP-friendly congestion control into the sys-tem, the proposed mechanism achieves smoothperformance over time as well as bandwidth fairnessfor co-existing applications in the network. Simula-tion results show the efficacy of the proposedmechanism. For instance, compared with a recentlypresented 3D application protocol referred to as3TP [5], the proposed system achieves 20–30%reduction in transmission latency while deliveringthe same level of rendering quality.

The main contributions of the presented work aresummarized as follows:

�
We analyze intensively the property of multi-resolution 3D meshes and present a TCP-friendlytransmission system for multi-resolution meshesincluding a novel and meaningful quality mea-sure. � Given a distortion constraint, we derive a
simplified expression (with assumptions) of theoptimal FEC code that minimizes the expectedtransmission latency when combined with re-transmissions.
� Observing a semi-infinite space for finding the
theoretical optimal solution, we propose anextended steepest decent search algorithm whichquickly reaches the local optima in the solutionspace.
� Based on the above results for quality-critical
scenarios, we further develop a time-criticalstreaming algorithm which significantly decreasesthe receiving distortion upon a strict delayconstraint.

The rest of the paper is organized as follows. Therelevant prior art that addresses channel effects inmulti-resolution mesh transmission is summarizedin the next section. Major aspects of the proposedmesh transmission system are described in Section 3,and Section 4 presents a detailed study of the hybridunequal-error-protection and selective-retransmis-sion mechanism. Test results in simulated networkenvironments are given in Section 4. Finally,Section 5 concludes the paper and summarizesfuture work.

ARTICLE IN PRESS

1For those methods that do not require grouping data into

batches to achieve high compression ratio such as a wavelet-

based scheme [19], we still consider this ‘‘batched’’ organization

as a general representation of multi-resolution mesh data for two

reasons: (i) isolated refinement operations in the multi-resolution

mesh perform small and localized changes which normally do not

result in perceivable distortion, and (ii) the refinement data will

D. Tian, G. AlRegib / Signal Processing: Image Communication 21 (2006) 396–415398

2. Prior art

In this section, we briefly discuss transmissionsystems that have been proposed in the literature[2–9,14,17,27]. According to the adopted error-resilient mechanism, these systems can be categor-ized into pre-processing, error correction, andtransport-layer protocols. In MPEG-4, pre-proces-sing-based error resiliency is achieved by datapartitioning [7,17,27], i.e., partitioning the bitstreaminto segments and decoding each segment indepen-dently. In [14], the partitions are ordered in a treestructure according to their interdependencies andtransmitted accordingly.

Using forward error correction (FEC) in provid-ing error resiliency to multi-resolution 3D mesheshas been investigated by AlRegib et al. [2,4,6],where unequal-error-protection (UEP) for differentresolutions of the compressed mesh is proposed. In[4,6], bit-allocation algorithms are presented todistribute source and channel coding bits under atotal bit budget. The similar concept is alsoexploited in this work for forward packet-lossresilience, but our work differs from those earlierapproaches in an essential aspect: instead of solelyusing FEC, the proposed solution jointly considersforward packet-loss resilience and feedback-basedretransmission. As a result, the system supportsreliable mesh transmission and can guarantee acertain quality level required by the application.Moreover, the proposed mechanism determines theoptimal tradeoff between forward packet-loss re-siliency and retransmission such that the delay isminimal with respect to the distortion constraint.

The transport control protocol (TCP) and theuser datagram protocol (UDP) are usually used forerror-sensitive and delay-sensitive streams, respec-tively. Recently, hybrid TCP/UDP transport proto-cols such as 3TP have been proposed to stream 3Dmodels [3,5,9]. The general idea is to send importantdata reliably using TCP and the remaining, lessimportant, data using UDP. In spite of theeffectiveness of these algorithms, they do not fullyexplore the space or time efficiency of the lossychannel as they utilize solely feedback-based re-transmission without considering its interactionwith error control coding. In addition, rate controlis not considered when using UDP, which makes theapplication irresponsive to congestion and unfair-ness to other streams in the network. The lack of arate control mechanism also results in large varia-tion of the transmission throughput, and therefore,

of the receiving quality over time. Finally, anessential difficulty of using hybrid TCP and UDPis the synchronization between the two transportassociations. In contrast to the hybrid TCP/UDPtransport protocols, the transmission mechanismproposed in this paper employs a unified transportlayer, and regulates data retransmission at theapplication level. Transmission latency is furtherreduced by incorporating unequal error protectionwith retransmission. A TCP-friendly rate controlprotocol is implemented in the presented transmis-sion system, which provides both smoothness andresponsiveness to the application.

3. System overview

In this section, we provide an overview on the 3Dmesh transmission system that is under considera-tion, and introduce the most important systemparameters. As can be seen in Fig. 1, the proposedsystem has three major components: a 3D meshcodec, a hybrid unequal-error-protection and selec-tive-retransmission mechanism at the applicationlevel (UEP/SR), and a transport protocol integratedwith TCP-friendly rate control (TFRC). Next, webriefly describe the system components and thecorresponding parameters. Toward the end of thesection, we present a simulation environment whichwill be used to test the performance of the proposedsystem.

3.1. The codec component

The 3D mesh codec implemented in the presentwork closely follows the compressed progressive

mesh algorithm proposed by Pajarola and Ros-signac [21] although the proposed transmissionsystem is general enough to incorporate othermulti-resolution compression methods, as will bediscussed later in this section. A compressed meshstream is composed of a base mesh, M0, and L

enhancement batches, fBigi¼1;...;L, each of whichencodes a set of vertex-split [16] operations whichtransfer the triangulated mesh surface to a higherresolution.1 Sequentially, batch Bi refines the

ARTICLE IN PRESS

Encodedbatches

Receivedbatches

Fig. 1. The proposed transmission system consists of three components.

D. Tian, G. AlRegib / Signal Processing: Image Communication 21 (2006) 396–415 399

resolution of mesh Mi�1 to higher resolution Mi

until the full resolution is reached. Each resolutionMi, differs from the full resolution mesh by certainerror (distortion), Di, with a certain bit-rate, Ri. Ameasure of such distortion that properly reflects theperceptual quality of different resolutions is im-portant. In this paper, we measure such distortionby introducing a metric with a similar formulationto the peak signal-to-noise ratio (PSNR) which iscommonly used in imaging. In particular, for amesh with multiple resolutions fMigi¼0;...;L whereM0 is the base mesh and ML is the full resolution,we define the quality of mesh Mi as

PSNRm920 log10EmaxðM0;MLÞ

ErmsðMi;MLÞ(dB), (1)

where EmaxðÞ and ErmsðÞ are the measured maximumand root-mean-square surface distances between thecorresponding pairs of meshes, respectively. Detailson this metric are given in Appendix A. Thesedistances can be calculated using the fast Metro tool[10] in practice. In addition, an empirical studypresented in [26] and our further discussion inAppendix A show that using this PSNR-like metrichas the following advantages: (i) it provides mean-ingful measurement on visual quality of differentresolutions, (ii) it normalizes the distance andprovides unified quantities for describing qualityof multiple models that are in the same 3D scene but

(footnote continued)

eventually be packetized when they are transmitted over the

network.

with different coordinate grids, and (iii) thenumerical range of the metric is close to that ofPSNR in imaging, which brings convenience forevaluating the quality of 3D models based onhuman’s subjective experience. These aspects aregenerally important for quality control in graphicsapplications and are especially vital for applicationsthat involve users in distributed and interactive 3Dpresentations.

For a compressed multi-resolution mesh, trans-formation from a coarser representation to a finerresolution includes decoding two parts of informa-tion: connectivity and geometry. Connectivity in-formation encodes the cut-edges for performingvertex-split operations, which refine the topology ofthe mesh surface on the contrary of edge-collapses[16]; geometric data record the position of thecollapsed vertex, and is compressed using vertexprediction followed by entropy coding for theprediction error [21]. If the connectivity is notdecodable because of information loss, the trans-formation will stop because the vertex-split processcannot continue without knowing the cut-edges. Onthe other hand, it is not the case when any geometryinformation is missing because geometry containsonly prediction errors of vertex positions, which donot prevent the decoding process from proceedingto the next level but introduce additional distortion.It should be mentioned that in a wavelet-based meshcompression scheme such as [18,19], all connectivityinformation except for the coarsest representation iseliminated by converting the input mesh to a semi-regular mesh before compression, in which case only

ARTICLE IN PRESSD. Tian, G. AlRegib / Signal Processing: Image Communication 21 (2006) 396–415400

vertex-position data are encoded in the enhance-ment bitstream.

For the codec using edge-collapse and vertex-splitas basic operations, both connectivity and geometryare included in enhancement batches. Intuitively,one may consider that connectivity information ismore important than geometry and should betreated differently in transmission. However,our empirical study (Appendix B) shows that,because of error propagation, decoding connectivitywithout refining geometry tends to amplify thegeometric error and results in degradation insteadof improvement on visual quality. In other words,although connectivity and geometry have differentimpacts on the decoding process for the multi-resolution mesh, they are equally important in thesense of preserving visual quality. Therefore, whileunequal error resilience is desired for different databatches from an optimal quality point of viewregarding their unequal rate-distortion perfor-mance, information encoded within a batch shouldbe treated in a unified manner. Importantly, thisfact provides generality to the proposed system toincorporate all 3D model codecs that are eitherbased on edge-collapse/vertex-split operations orwavelets.

In all present multi-resolution mesh codingschemes, the base mesh M0 has both connectivityand geometry information and has to be correctlyreceived or the rendering process will not be ableto start (infinite distortion), while it in generalhas a fairly small fraction (1–2% or less) ofthe entire bitstream. Regarding this, to simplifynotation (avoid differentiating the base meshand the enhancement batches), and for the easeof performance presentation (avoid presentinginfinite distortion), in the rest of the paper weassume that a coarse representation has beenreceived by the client through a reliable channel,and focus our discussion on the transmission ofenhancement data.

3.2. The transmission component

The L enhancement batches, fBigi¼1;...;L, havedifferent rate-distortion performance and are trea-ted intelligently to achieve the best quality. In thispaper, best quality is interpreted as either minimizedtransmission latency t under a distortion constraintDmax or minimized decoding distortion Dd within alimited time frame tmax. Given network statisticsreported by the transport layer, the sending

application protects the batches with unequal-rateforward error correction (FEC) codes and/orretransmission according to their respective costs,and determines the optimal tradeoff that minimizesthe transmission delay t under the constraint ofDmax or vice versa. This hybrid unequal-error-protection and selective retransmission mechanism(UEP/SR) is the core contribution of this work. Toillustrate, consider the situation where a distortionconstraint Dmax is imposed. A set of w batches,fB1;B2; . . . ;Bwg, are selected under a criterion thattheir overall bit-rate is minimal while their resultingresolution has a distortion level below the con-straint, i.e., DdpDmax. According to the discussionin Section 3.1, these selected batches need to bereliably transmitted, or the decoding distortion ofthe received mesh will not be satisfied (if any batchesamong the selected ones are not correctly received,sending higher batches would not help decreasingthe decoding distortion). Moreover, instead ofsolely using feedback-based retransmission toachieve reliable transmission, unequal-rate FECcodes are concurrently used to reduce the potentialretransmission cost, and computation is performedto find the optimal FEC codes that minimizethe expected transmission delay, EðtÞ, with thecondition that undecodable batches among theselected ones will be retransmitted. Detailed discus-sion on the transmission mechanism is presented inSection 4.

The Reed–Solomon (RS) code is employed forFEC. We assume an ðn; kÞ RS code with a block sizeof n packets (i.e., n� packet�size symbols) includingk ðkonÞ information packets (code rate k=n).Considering a lossy channel, an RS code with ðn�kÞ parity-check packets will be able to recover thesame number of packet losses. Hence the errorprobability of a batch using RS code ðn; kÞ is

Pðn; kÞ ¼ Prf4ðn� kÞ losses out of ng. (2)

If the random packet loss is modeled by a Bernoulliprocess, for example, it is easy to get

Pðn; kÞ ¼ 1� Prfpðn� kÞ losses out of ng

¼ 1�Xn�k

i¼0

n

i

!pið1� pÞn�i, ð3Þ

where p is the packet-loss rate. Similarly, thecalculation of (2) can be performed for moresophisticated channel models, for which a fewresults have been found in the literature. For atwo-state Markov channel model, for example, one


may refer to the derivation presented in [15]. As willbe shown in Section 4, such results can be easilyintegrated in the computation of transmissiondecisions, and the simulation results show that eventhe simple approximation (3) performs fairly wellwith the proposed algorithms.

3.3. The transport layer

UDP streams suffer from the lack of congestioncontrol mechanism that prevents them from beingreasonably fair2 when competing for bandwidthwith TCP-based traffic, as TCP throttles itstransmission rate against the network congestion.A TCP-friendly system should regulate its datasending rate according to the network condition,typically expressed in terms of the packet size s, theround-trip time r, and the packet-loss rate p. Ideally,the TCP throughput equation is suitable in describ-ing the steady-state sending rate of a TCP-friendlyflow [11]:

T ¼s

rffiffiffiffiffiffiffiffiffiffi2p=3

pþ 4rð3

ffiffiffiffiffiffiffiffiffiffi3p=8

pÞpð1þ 32p2Þ

, (4)

where a recommended choice of the retransmissiontimeout, tRTO ¼ 4r, has already been integrated.

A TFRC protocol based on Eq. (4) has beenspecified in [13]. TFRC is a receiver-based mechan-ism designed for applications that use a fixed packetsize and vary their sending rate in packets persecond in response to congestion. The receiverperiodically (once per round-trip time) sends feed-back reports to the sender, containing the informa-tion that allows the sender to adjust its sending rate.Rate control using TFRC provides bandwidthfairness for parallel flows in the network as well asnetwork stability, thus avoiding congestion collapse.In addition, TFRC’s rate fluctuation is much lowerthan TCP, making it more appropriate for stream-ing applications that desire constant receivingquality. The TFRC implemented in the proposedmesh transmission system is slightly modified sothat each data packet is acknowledged by thereceiver [22]. The ACKs are used to infer theround-trip time and detect packet losses in order toperform selective retransmission for undecodablebatches.

2A flow is ‘‘reasonably fair’’ if its sending rate is generally

within a factor of two of the sending rate of a TCP flow under the

same condition [13].

3.4. Simulation environment

In this section, we present a simulation environ-ment on which all our reported results will be based.

(1) Network environment: Simulation in this paperis performed using ns-2 [25]. Fig. 2 shows thesimulated topology. The link between R1 and R2 isthe bottleneck and the bandwidth is shared by f

parallel flows. Random early detection (RED) [12]gateways are deployed at R1 and R2 to improveboth fairness and performance of the flows. All theparameters are listed in Table 1.

(2) Test models: Simulation results, unless other-wise noted, are obtained using the following models(courtesy of Cyberware, Inc): HORSE (39 698 faces),DINOSAUR (28 096 faces), VENUS HEAD (67 170faces), and BALLJOINT (68 530 faces). All the modelsare quantized with 12 bits3 and are encoded togenerate 10 enhancement batches. Their rate-dis-tortion performance is plotted in Fig. 3.

4. The proposed transmission mechanism

In a hybrid TCP/UDP protocol as reviewed inSection 2, transmission latency is reduced by usingUDP while distortion is reduced by TCP, i.e.,feedback-based retransmission. Error control cod-ing is an alternative to retransmission to improvethe channel utilization and provide quality control.Although information theory has shown that soleuse of feedback or error control coding can achievethe channel capacity, real systems often haveconstraints that invalidate ideal assumptions andcall for joint considerations on both approaches.Typically, there are many situations where thereceiving application has a maximally allowed levelon either the transmission latency or the renderingdistortion. In such cases, retransmission and errorcontrol coding should be treated intelligently so thatthe lowest distortion level or the minimum trans-mission latency is obtained while satisfying thecorresponding constraint. In this section, we studyin detail the hybrid unequal-error-protection andselective-retransmission mechanism for multi-reso-lution meshes. This proposed mesh transmission

3In general, using 12–14 bits for quantizing 3D models with

moderate dimensions introduces invisible distortion [21]. It is

worthwhile to be pointed out, however, that the test models and

the quantization parameter herein are selected for the illustration

purpose. Our proposed transmission mechanism has no depen-

dency on these parameters.

ARTICLE IN PRESS

Table 1

Summary of simulation parameters

Packet size 1000/500bytes

ACK size 40 bytes

Bottleneck delay 40ms

Bottleneck bandwidth 1Mbps

Bandwidth of side links 10Mbps

Delay of side links 10ms

Simulation duration 100–200 s

TCP window 20

RED parameters

Min threshold 5 packets

Max threshold 0.5*buffer size

Buffer size 1000 packets

q_weight 0.002

0 1 2 3 4 5 6 7 8x 104

10

15

20

25

30

35

40

45

50

Size of enhancement data (bytes)

PS

NR

m (

dB)

BalljointDinosaurHorseVenus Head

Fig. 3. Rate-distortion performance for four test models.

S1 C1

Sf

S3

C2S2

3D server(TFRC)

R1 R2

Cf

C3

Back-groundtraffic(TCP)

RED

10Mbps, 10m

s

Bottleneck

1.0 Mbps, 40ms

3D client

10M

bps,

10m

s

10M

bps,

10m

s 10Mbps, 10m

s

TCPclientsRED

Fig. 2. Simulation topology in ns-2. There are totally f parallel flows sharing the bottleneck bandwidth. All the background traffic is FTP

traffic transmitted over TCP.


mechanism is referred to as REP (retransmissionand error protection) hereafter.

4.1. Distortion-constrained transmission

We first look at the problem of minimizing thetransmission latency with a given distortion con-straint, Dmax. For a multi-resolution mesh stream, ifa lower batch is not decoded correctly, perceptualquality cannot be improved by decoding higherbatches. Henceforth, satisfying Dmax is equivalent toselecting the least number of batches that need to betransmitted reliably. Denoted by w, this leastnumber of batches is expressed as

w ¼ minfxjDdðxÞpDmaxg, (5)

where DdðxÞ denotes the decoding distortion of thefirst x batches.

Once the number of selected batches is deter-mined, the challenge is to find the optimal tradeoffbetween forward error protection and retransmis-sion such that the user-requested distortion level canbe satisfied with minimum transmission latency. Todo so, an optimal distribution of parity-checkpackets for the selected batches is computed, takinginto account their unequal rate-distortion perfor-mance and potential retransmission costs. Theselected data are then protected with thecorresponding parity-check packets, and is trans-mitted (with possible retransmissions) until thebatches are correctly decoded. A simple illustrationof this transmission mechanism is presented inFig. 4, where w batches are selected to be reliablytransmitted with a calculated distribution of the

ARTICLE IN PRESS

RS RS

RS RS

Batch 1Batch 2 Batch χ Batch 1 Batch 2 Batch χ

RS

Batch 2 Batch Remaining dataRemaining data

2

Batch L

∗∗

∗

∗Batch L ... ... ... ...

Selecteddata

Fig. 4. A simple demonstration on the proposed hybrid UEP/SR method, where the packets marked with ‘RS’ are parity-check packets

and ‘*’ indicates packet losses during transmission. Batch 2 is retransmitted because it is among the selected w batches and has not been

correctly received.


parity-check packets (noted with ‘RS’). Packetsmarked with ‘�’ indicate the losses occurred duringtransmission. As depicted in Fig. 4, Batch 2 is notdecodable due to packet losses and is retransmittedin order to satisfy the distortion constraint. Afterthe w batches are correctly received, the remainingdata are transmitted (with or without error resi-lience, provided that the requested distortion levelhas been met).

Using sw and cw to represent the vectors(with length w) of source and parity-checkpackets, respectively, for the selected w batches,the expectation of the transmission latency t isgiven as

Eðtjsw; cwÞ ¼ Eðtsw þ tcw þ tRjsw; cwÞ

¼ Eðtsw þ tcw jsw; cwÞ þ EðtRjsw; cwÞ, ð6Þ

where Eð Þ represents the probabilistic expectation,and tR denotes the total latency incurred byretransmission; tsw , tcw denote the transmission costsfor all the source and parity-check packets, respec-tively, in the selected batches, i.e.,

tsw ¼ jssw j ¼Xwi¼1

ssw ðiÞ and tcw ¼ jscw j ¼Xwi¼1

scwðiÞ,

where ssw ðiÞ and scw ðiÞ correspond to the transmis-sion cost for the ith batch. Given the RS codesðsw; cwÞ, the packet-loss rate p and the round-triptime r, the steady-state transmission throughput T isdescribed by Eq. (4), and ssw , scw can be consideredas constant-value vectors. With the notations s0 ¼ðssw þ scw Þ and t0 ¼ tsw þ tcw ¼ js0j ¼ 1 � s0, we haveEðtsw þ tcw jsw; cwÞ ¼ Eðt0Þ, and Eq. (6) is further

expressed as

Eðtjsw; cwÞ ¼ Eðt0Þ þ EðtRjsw; cwÞ

¼ js0j þ Pðsw þ cw; cwÞ � ½s0 þ EðsRjsw; cwÞ�

¼ 1 � s0 þ Pðsw þ cw; cwÞ � s0

þ P2ðsw þ cw; cwÞ � s0 þ � � �

þ Pnðsw þ cw; cwÞ � s0 þ � � �

¼1

1� Pðsw þ cw; cwÞ� s0. ð7Þ

Note that in (7), EðtRjsw; cwÞ was expanded as asummation of recursive retransmission costs multi-plied by the corresponding probabilities, with anassumption that all processing cost upon data loss isignorable. Pðsw þ cw; cwÞ is the vector of batch errorprobabilities with each element computed accordingto (2), and Pnðsw þ cw; cwÞ is defined as

Pnðsw þ cw; cwÞ ¼ Pðn�1Þðsw þ cw; cwÞ

� Pðsw þ cw; cwÞ; n41.

The optimal distribution of parity-check packets,cwopt, is then given by

cwopt ¼ argmincw

Eðtjsw; cwÞ. (8)

Eqs. (7)–(8) with (5) provide the theoreticalsolution to the problem. Yet an operational solutionshould also take into account the computationcomplexity as exhaustive search will not be feasible(the choice of cw in (8) is arbitrary and therefore thesolution space is not a close space). To accommo-date real-time applications, a generalized steepest

decent algorithm is used in this work to findthe optimal (or a possibly suboptimal) solution.This fast algorithm starts with cw ¼ ;, i.e., no

ARTICLE IN PRESS

( |[0,0,0,0,0])E τ

( |[1,0,0,0,0])E τ( |[0,0,0,1,0])E τ ( |[0,0,0,0,1])E τ( |[0,1,0,0,0])E τ

( | [1,0,0,1,0])E τ ( |[0,0,0,2,0])E τ ( | [0,0,0,1,1])E τ( | [0,1,0,1,0]E τ

( | [1,1,0,1,0])E τ ( |[0,1,0,2,0])E τ ( | [0,1,0,1,1])E τ( |[0,2,0,1,0])E τ

Fig. 5. Illustration on the steepest decent algorithm for finding

cwopt. The double-circled nodes indicate the searching path of the

steepest decent algorithm, and the solid one represents the

optimal operating point which has the minimum expected

transmission delay.0 10 20 30 40 50

4.2

4.4

4.6

4.8

5

5.2

5.4

5.6

Number of parity− check packets (K)

Min

imum

exp

ecte

d de

lay

(sec

)

Fig. 6. The computed minimum expected delay versus the

number of parity-check packets that are added. The computation

is performed for the HORSE model with a setting of these

parameters: packet size s ¼ 500bytes, packet-loss rate p ¼ 2%,

and round-trip time r ¼ 100ms.

4We keep the same round-trip time in all the computations

based on the following fact: although the packet-loss rate

increases quickly as the network becomes more congested, the

round-trip time is observed with slight variation in all cases

because of the regulation by RED gateways and the rate control

mechanism.


parity-check packets are added at the initial point.At each iteration, the algorithm adds one moreparity-check packet to either one of the w batches,which results in w possibilities. It then finds amongstthe one that decreases the expected delay Eðtjsw; cwÞthe most, or stops the computation if Eðtjsw; cwÞincreases for all cases, where Eðtjsw; cwÞ is calculatedusing (7). Fig. 5 presents a simple illustration on thisfast algorithm. The double-circled nodes indicatethe searching path of the steepest decent algorithm,which have smaller expected transmission delaysthan their parents and siblings; the solid onerepresents the optimal operating point that has theminimum expected transmission delay, meaningthat all its descendants (skipped in the plot) havelarger expected delays. To simplify notation, thesource vector sw is ignored in Fig. 5 and only thevector of parity-check packets is indicated. FromFig. 5, it is apparent that the steepest decentalgorithm has a linear computational complexityOðK � wÞ, or more generally, OðK � LÞ, where K is thetotal number of parity-check packets that are added(the depth of the tree) and L is the number ofgenerated batches (the maximum width of the tree).

Fig. 6 presents a further illustration of thealgorithm. For the HORSE model (with 10 batches),it shows the computed minimum expected transmis-sion delay with respect to the total number ofparity-check packets that are added to the batches.In the computation, the packet size, the packet-lossrate, and the round-trip time are set to bes ¼ 500 bytes, p ¼ 2%, and r ¼ 100ms, respectively.As can be seen in Fig. 6, adding parity-checkpackets at the beginning quickly decreases theexpected transmission delay, which reaches a mini-mal at a certain point (K ¼ 9 in this case). There-

after, adding more parity-check packets graduallyincreases the expected transmission latency, imply-ing that the additional transmission for sendingparity-check packets has exceeded the potentialretransmission cost.

To study the performance of the algorithm fordifferent network conditions, we compute theoptimal number of parity-check packets withrespect to various packet-loss rates4 and fordifferent models. The results are shown in Fig. 7.The total number of parity-check packets that areneeded for the minimum expected transmissiondelay is plotted with the packet-loss rate varyingfrom 1% to 15%. As one may expect, when thenetwork becomes heavily congested (higher packet-loss rates), the retransmission cost increases, andFig. 7 shows that more parity-check packets areneeded to minimize the expected transmissionlatency. An approximately linear relation is ob-served between the number of parity-check packets(K) and the packet-loss rate (p), which implies thatthe computation complexity is linearly increased asthe packet-loss rate gradually increases, accordingto the expression OðK � LÞ. In Fig. 7, it is noted thatthe VENUS HEAD model requires more parity-check


packets (larger K) than the HORSE and DINOSAUR

models with the same packet-loss rate, simplybecause its batches are encoded with largersizes. In general, batch sizes can be reduced while

010 9 8 7 6 5 4

1

2

3

4

5

6

7

8

Number of reliably transmitted batches (X )

10 9 8 7 6 5 4


Ave

rag

e d

elay

(se

c)A

vera

ge

del

ay (

sec)

3TP

REP

3TP

REP

0

2

4

6

8

10

(a) (b

(c) (d

Fig. 8. Average transmission delays for sending various portions of data

flows in the network is f ¼ 12, and the packet sizes are s ¼ 1000bytes

00 4 8 12 16

10

20

30

40

50

60

70

Packet loss rate (%)

Num

ber

of p

arity

-che

ck p

acke

ts

HorseVenus HeadDinosaur

Fig. 7. The number of parity-check packets versus the packet-

loss rate. The packet size and the round-trip time in the

computation are set to be s ¼ 500bytes and r ¼ 100ms,

respectively.

a higher number of resolutions (larger L) will occurwith potential loss/reduction in compressionefficiency [23].

We now present simulation results of the pro-posed hybrid mechanism (REP) for distortion-constrained transmission, and compare it with the3TP [5]. For fair comparison, in 3TP, we replaceUDP with TFRC so the transport layer of 3TP isalso TCP-friendly. Thus, in both REP and 3TP, theremaining data other than the selected batches aretransmitted under the same mechanism, i.e., soleTFRC–UDP without FEC or retransmission. Forthis sake, in our presented results, we compare only

the transmission delays for the batches that areselected to be transmitted reliably under a givendistortion constraint, and are treated differently in3TP and REP (in particular, 3TP uses TCP andREP uses hybrid UEP/SR).

Fig. 8(a)–(d) plots the transmission delay withrespect to different numbers of batches that areselected to be reliably transmitted, determined uponthe distortion constraint. The delay results havebeen averaged over all received meshes during thesimulation period. The number of parallel flows in

10 9 8 7 6 5 4


10 9 8 7 6 5 4


Ave

rag

e d

elay

(se

c)A

vera

ge

del

ay (

sec)

3TP

REP

3TP

REP

0

2

4

6

8

10

12

0

5

10

15

20

25

)

)

reliably using REP and 3TP, respectively. The number of parallel

in (a) and (b) and s ¼ 500bytes in (c) and (d).

ARTICLE IN PRESS

02 3 4 6 8 10 12 15 20

2

4

6

8

10

Number of parallel flows (f )

2 3 4 6 8 10 12 15 20


2 3 4 6 8 10 12 15 20


2 3 4 6 8 10 12 15 20


Ave

rag

e d

elay

(se

c)A

vera

ge

del

ay (

sec)

Ave

rag

e d

elay

(se

c)A

vera

ge

del

ay (

sec)

3TP

REP

3TP

REP

3TP

REP

3TP

REP

0

9

8

7

6

5

4

3

2

1

0

2

4

6

8

10

12

0

1

2

3

4

5

6

(a)

(c)

(b)

(d)

Fig. 9. Delay results when the distortion constraint Dmax is 36 dB. The packet sizes are s ¼ 500bytes in (a) and (b) and s ¼ 1000bytes in (c)

and (d). (a) w ¼ 8, (b) w ¼ 7, (c) w ¼ 9, (d) w ¼ 8.

5This number depends on the rate-distortion curve of the

model.


the network is set to be f ¼ 12, which results in amoderate network congestion situation as reflectedby the measured packet-loss rate (p � 6–7%). Thepacket sizes are s ¼ 1000 bytes for Fig. 8(a) and (b)and s ¼ 500 bytes for Fig. 8(c) and (d), respectively.As can be seen from these plots, REP considerablyreduces the transmission latency compared to 3TP.Specifically, when the full resolution model isrequired by the receiving application, all theenhancement batches need to be reliably trans-mitted ðw ¼ 10Þ. Simply, 3TP returns to be sole TCPin this case. In contrast, REP still achievessubstantial delay reduction by using hybrid UEP/SR. For example, in Fig. 8(a) and (b) where thepacket size is s ¼ 1000 bytes, 27% reduction in theaverage delay is observed for the HORSE model and25% is observed for the VENUS HEAD model.

In Figs. 8(a)–(d), it is observed that as the portionof reliably transmitted data becomes smaller (i.e.,smaller w), which corresponds to a lower constrainton decoding distortion, the performance of REPgradually merges to 3TP. This behavior is antici-

pated because as the distortion constraint islowered, both mechanisms transmit most of thedata using TFRC–UDP without FEC or retrans-mission. Yet, considering a distributed onlinepresentation of 3D scenes, a small reduction in theaverage delay provided by REP may still result insignificant improvement on overall performancewhen a number of 3D meshes are transmitted ondemand.

In Fig. 9, variation of the average delay fordifferent network congestion situations is investi-gated. Suppose an upper bound of the decodingdistortion, DmaxX36 dB, is requested for the modelsto be rendered. To satisfy Dmax, w ¼ 7–9 enhance-ment batches5 are selected to be reliably transmittedaccording to Eq. (8) and Fig. 3. Fig. 9(a)–(d)presents the delay results of REP and 3TP for theselected batches for the test models. It is shown thatREP outperforms 3TP under most of the network


conditions, and the gain is especially significantwhen the network encounters moderate or heavycongestion (fX6 in the figures). For the networkwith light traffic load ðfp4Þ, REP and 3TP performsimilarly. When only two flows exist in the networkand the packet size is s ¼ 1000 bytes (Fig. 9(c) and(d)), one may note that 3TP provides slightly betterresults than REP. This is resulted from TCP’sadvantage of quick adaptation to the changes inavailable bandwidth compared with TFRC [13].

4.2. Delay-constrained transmission

So far, we detailed the proposed mechanism fordistortion-constraint applications. In this part, weconsider a time-critical situation where a delayconstraint tmax is imposed by the application and aminimum decoding distortion is desired. A hybridTCP/UDP protocol is not suitable for this problembecause of the automatic retransmission behavior ofTCP. Although having the application conserva-tively sends a small portion of data via TCP and theremainder via UDP may possibly satisfy the delayconstraint, it is difficult to perform the conservative

selection in TCP because it halves the transmissionrate in response to a single packet drop, whichresults in frequent abrupt changes in throughput. Incontrast, TFRC has lower variation of throughputover time [13], making it favorable in the delay-constraint application that is under consideration.

Utilizing the smooth sending rate variations inTFRC, we have developed in REP an operationalalgorithm with low computation complexity fordelay-constrained transmission. In particular, for agiven time frame, the algorithm selects the max-imum number of batches to transmit provided thattheir minimum expected transmission latency doesnot exceed the allowable time frame. The minimumexpected transmission latency is achieved by findingthe optimal tradeoff between UEP and retransmis-sion, and is computed using the proposed steepestdecent algorithm as shown in the previous subsec-tion. We consider the selected data (with thecorresponding RS codes) as a greedy conservative

solution at each transmission opportunity. Aftersending the selected source data and the corre-sponding parity-check packets, the server updatesits sending buffer according to the feedback. Thesame procedure is repeated before all the batchesare correctly received or the deadline has beenreached.

We denote the source bit-rate of the L batches byvector s, and use sðiÞ, tðiÞ to represent the sourcevector and the time instant for ith transmissionopportunity, respectively. Major steps of the algo-rithm are then presented as follows:

(i) i ¼ 0, sð0Þ ¼ s, tð0Þ ¼ 0;(ii) i ¼ i þ 1;

(iii) update sði�1Þ to sðiÞ by removing those batchesthat have been acknowledged;

(iv) if tði�1Þotmax, find ðw; cðiÞwoptÞ such that the

following conditions are satisfied:

1pwpL;

EðtðiÞjsðiÞw ; cðiÞwoptÞoa � ðtmax � tði�1ÞÞ; 0oap1;

if woL; then EðtðiÞjsðiÞwþ1; cðiÞwþ1;optÞ4a � ðtmax � tði�1ÞÞ;

8>><>>:

(9)

where cðiÞwopt ¼ argmin

cðiÞw

EðtjsðiÞw ; cðiÞw Þ is given by

Eqs. (7)–(8);

(v)send ðsðiÞw þ cðiÞwoptÞ using TFRC;

(vi) if tmax has not been reached, set tðiÞ with thecurrent time; REDO (ii).

Note that in step (iii), w 2 ½1;L� is the maximum
number of batches with corresponding RS codesðsðiÞw ; c
ðiÞwoptÞ whose expected transmission latency falls

within the remaining time frame, ðtmax � tði�1ÞÞ,multiplied by a fraction factor a. Ideally, we havea ¼ 1:0 given that smooth transmission throughputis provided by TFRC, in which case satisfaction of(9) can be considered as the greediest conservativeselection of data that is to be sent. In practice, amaybe chosen to be close (but not equal) to 1.0 for firstfew transmissions in order for the application to bemore robust to abrupt drops of data sending rateoccurred during transmission, thus avoiding largevariation of the decoding distortion among received3D models. The particular choice of the factor a is adesign issue. In our simulation, we have found thata simple choice with a ¼ 0:8 for the initial transmis-sion ði ¼ 1Þ and a ¼ 1:0 for the rest transmissionsði41Þ works fairly well.

Performance of the above algorithm has beeninvestigated compared with a heuristic mechanismthat is also based on TFRC. It utilizes the timeframe maximally by performing a simple scheme:the maximum number of batches that satisfy thetiming condition are first selected based on thetransmission throughput; the remaining time slot (if


any) is then filled with the first few batches that havethe least bit-rate but are generally most important.Pseudo-code of this simple filling algorithm (FA) isshown in the following, where T is the transmissionthroughput and RðwÞ denotes the total bit-rate of wbatches, fB1;B2; . . . ;Bwg.

while totmax and swa; dofind the maximum w 2 ½1;L� s.t. RðwÞ

Tptmax � t;

send sw ¼ fB1;B2; . . . ;Bwg using TFRC;

t ¼ tþ RðwÞT;

end while

Simulation results are presented in Figs. 10and 11, where the number of parallel flows in thesimulated network is set to be f ¼ 12 (moderatetraffic load) and the packet size is s ¼ 500 bytes.Fig. 10(a)–(d) plot the decoding distortion averagedover all received meshes with respect to various

151 2 3 4 5 6 7 9

20

25

30

35

40

45

50

Constrained transmission latency (sec)

1 2 3 4 5 6 8 11


Ave

rag

e d

isto

rtio

n (

dB

)

15

20

25

30

35

40

45

50

Ave

rag

e d

isto

rtio

n (

dB

)

FAREP

FAREP

(a) (b

(c) (d

Fig. 10. Average decoding distortion of received meshes for various dela

flows is f ¼ 12, the packet size is s ¼ 500bytes, and the resulting packet

(d) BALLJOINT.

delay constraints, i.e., different tmax. One can seethat REP greatly improves the decoding qualitywithin the same time frame compared to the FAalgorithm. In addition, as tmax increases, thedecoding quality provided by REP quickly arises,whereas for FA, the resulting effect is observed to beunpredictable. It is a natural consequence ofrandom losses as in the heuristic scheme, all theenhancement batches, except those few ones that areused to fill the remaining time slot, are treatedequally without error resilience.

In Fig. 11, we trace the decoding distortion ofevery received HORSE or VENUS HEAD mesh fortwo individual cases: (a) and (b) tmax ¼ 2 s, and (c)and (d) tmax ¼ 7 s. The (blue) triangles markedcurve denotes the PSNRm values of decodedmeshes in REP, while the (red) circles markedcorresponds to the FA mechanism. As expected,REP has much lower variation of the decodingquality in addition to a lower average distortion

1 2 4 6 8 10 12 15


1 2 3 4 5 7 9 12


15

20

25

30

35

40

45

50

Ave

rag

e d

isto

rtio

n (

dB

)

15

20

25

30

35

40

45

50

Ave

rag

e d

isto

rtio

n (

dB

)

FAREP

FAREP

)

)

y constraint in a typical network situation: the number of parallel

-loss rate is p ¼ 5:5%. (a) HORSE; (b) VENUS HHEAD; (c) DINOSAUR;

ARTICLE IN PRESS

35

30

25

20

15

10

50

45

45

40

35

30

25

20

15

40

35

30

25

20

15

100 5 10 15 20 25 30

0 20 40 60 80 100Index of decoded meshes

Index of decoded meshes0 5 10 15 20 25 30

Index of decoded meshes

0 20 40 60 80 100Index of decoded meshes

Dec

odin

g di

stor

tion

(dB

)D

ecod

ing

dist

ortio

n (d

B)

Dec

odin

g di

stor

tion

(dB

)

35

30

25

20

15

10

Dec

odin

g di

stor

tion

(dB

)

REPFA

REPFA

REPFA

(a) (b)

(c) (d)

Fig. 11. Trace of the decoding distortion for each received mesh: (a) and (b) tmax ¼ 2 s; (c) and (d) tmax ¼ 7 s. (a) HORSE, tmax ¼ 2 s; (b)

VENUS HEAD, tmax ¼ 2 s; (c) HORSE, tmax ¼ 7 s; (d) VENUS HEAD, tmax ¼ 7 s.


level (higher PSNRm) than FA. Difference in theaverage quality is especially significant when largertransmission latency is allowed. For example, inFig. 11(c) and (d), REP provides a median-levelquality of 39.68 dB (for HORSE) or 37.76 dB (forVENUS HEAD), whereas FA has only 29.07 or28.35 dB, correspondingly. Such difference in theaverage quality has actually been shown in Fig. 10.For the test models, average quality near to 40 dB istypically observed for REP when tmax ¼ 7–10 s,with a 10 dB or higher gain over the comparingheuristic. The perceptual quality difference of therendered models is as significant as reflected by thequantities, captured by Fig. 12(a)–(d). All the aboveresults, along with those presented in Section 4.1,have verified REP a successful transmission me-chanism for streaming 3D meshes over a lossynetwork.

5. Conclusions and future work

We have proposed a hybrid unequal-error-pro-tection and selective retransmission mechanism(REP) for streaming multi-resolution meshes overlossy networks with a goal of reducing responsetime and improving rendering quality in interactionfor distributed graphics applications. In particular,to minimize the transmission latency under adistortion constraint, a portion of multi-resolutionmesh data is selected to be reliably transmitted.Then, the best tradeoff between forward packet-lossresilience and retransmission is calculated in lineartime using a steepest decent algorithm. To maximizerendering quality within a limited time frame, wedeveloped an operational algorithm which has itscore based on the proposed steepest decent algo-rithm. These algorithms have low computational

ARTICLE IN PRESS

Fig. 12. Rendering quality near to 40 dB is obtained by REP when tmax ¼ 7–10 s, with a 10 dB or higher gain compared with the simple

heuristic (FA) under the same situation.


complexity, making them attractive for distributedgraphics applications.

TCP-friendly rate control (TFRC) is integrated inREP, which regulates the data sending rate inresponse to network congestion so that REP fairlycompetes for bandwidth with co-existing applica-tions in the network. In addition, because TFRChas much lower variation of throughput over timethan TCP, it is a favorable mechanism for REP toachieve smooth performance in interaction in time-critical situations.

REP is a network-oriented approach. It appro-priately selects the transport and application layertechniques including TCP-friendly rate control,forward error protection, and feedback-based re-transmission, and finds an optimal combination ofthem according to the characteristics of compressedmeshes and the lossy channel. Pre- or post-proces-sing methods such as data partitioning or errorconcealment are not included at this stage. Whilethese techniques can be incorporated relativelyindependently (for example, the data partitioningtechnique provides error resilience by dividing a 3Dscene/model into components, each of which can be

encoded, transmitted, and decoded separately fol-lowing the same process presented in the paper),exploration on their integration with the network-oriented approach is expected to further improveservice quality for distributed graphics applications.

As yet, REP assumes that packet losses areprimarily caused by buffer overflow (congestion) inwired networks. For streaming over wireless linkswhere packets can be corrupted by channel errors atthe physical layer, packet losses caused by bit errorsshould be considered. Not only rate control needs tobe elaborated accordingly, but also an accurate modelof the packet-loss process becomes important. Further-more, the feedback channel is also error-prone becauseof the fading effect, which complicates the process offinding optimal transmission decisions. All thesefeatures are to be addressed to improve the perfor-mance for streaming 3D data over wireless channels.

Appendix A. Measuring distortion for multi-

resolution meshes

The error between a simplified mesh surface S1

and its original S0 is generally defined as the

ARTICLE IN PRESS

Fig. 13. An example when the Hausdorff (maximum) distance fails to reflect the quality degradation of different resolutions: (a) full

resolution; (b) Emn ¼ 0:28, Erms ¼ 0:18, H ¼ 7:69; (c) Emn ¼ 0:42, Erms ¼ 0:36, H ¼ 7:69.

6Human ratings and naming times are two subjective measures

widely used in the experimental sciences of visual fidelity.


distance between corresponding sections of themeshes. Given a point v and a surface S, thedistance eðv;SÞ is defined as

eðv;SÞ ¼ minv02S

dðv; v0Þ,

where dðÞ is the Euclidean distance between twopoints in E3. The one-sided distance between twosurfaces S0, S1 is then defined as

EmaxðS1;S0Þ ¼ maxv2S1

eðv;S0Þ. (10)

Note that this definition of distance is not sym-metric, i.e., EmaxðS0;S1ÞaEmaxðS1;S0Þ in general.The Hausdorff distance is a two-sided distanceobtained by taking the maximum of EmaxðS0;S1Þ

and EmaxðS1;S0Þ, i.e., HðS1;S0Þ ¼ maxfEmax

ðS1;S0Þ;EmaxðS1;S0Þg. It has been observed thoughthat the two one-sided distances usually have closenumerical measures. In our definition, we preferablyuse (10) concerning that S0 is the full-resolutionmesh and S1 is its simplified representation.

Similarly, the mean distance Emn is defined as thesurface integral of the distance divided by the areaof S1:

EmnðS1;S0Þ ¼1

S1

ZS1

eðv;S0Þds. (11)

And the root-mean-square summary Erms squareseach of the summed distances before normalization:

ErmsðS1;S0Þ ¼1

S1

ZS1

e2ðv;S0Þds

� �1=2

. (12)

All the above three error measures, i.e., (10)–(12),can be computed using the Metro tool [10], and areempirically verified in [26] successful predictors of

visual quality as judged by human ratings andnaming times.6 In our experiments, we haveconfirmed that the mean and the mean-squaredsummaries properly measure the quality degrada-tion of different resolutions for a multi-resolutionmesh. However, it is noted that the maximumdistance (or the Hausdorff distance), althoughcommonly used in the literature, may notreflect the quality difference of consecutive levelsproperly. For example, a coarser resolution mayhave larger mean and mean-squared distancesummary than its finer version whereas the Haus-dorff distances for both levels are the same, asdepicted in Fig. 13.

Furthermore, while the mean and mean-squaredsurface distances work successfully in measuring thequality of simplified meshes, they are not definedwithin meaningful or unified numerical ranges. Thenumerical values of the measured distances dependon particular coordinate grids that the mesh verticeslie on, and hence may drastically vary betweenmeshes even in the same 3D scene or when thecoordinate system is simply scaled. For example, themeasured root-mean-square distance Erms ¼ 0:36 inFig. 13(c) does not convey any information abouthow much the mesh quality is degraded from its fullresolution. To overcome this inconvenience for 3Dapplications which essentially deal with visualization,we define a PSNR-like metric for the multi-resolutionmesh using the mean square distance summary. Fora mesh with multiple resolutions fMigi¼0;...;L, whereM0 is the base mesh and ML is the full resolution, we


define the quality of mesh Mi as

PSNRm920 log10EmaxðM0;MLÞ

ErmsðMi;MLÞ(dB), (13)

Fig. 14. Measured PSNRm values for five r

01

23

45

012

34

515

20

25

30

35

40

692 faces, 16.14 dB 5660 faces, 13.95 dB

X: ConnectivityY: Geometry

PS

NR

m (

dB)

(a)

(c)

(b)

Fig. 15. For the TRICERATOPS model: (a) decoding the mesh with sole co

quality; (b) and (c) the decoding distortion for different decoding patter

different impacts of connectivity and geometry on the decoder as well

geometry. For example, the two models shown in (a) correspond to th

where Emax and Erms are the measured maximumand mean-squared distances, respectively. Note thatthe distortion measure given by (13) no longerdepends on the coordinate grids of the mesh. More

esolutions of the TRICERATOPS model.

0 1 2 3 4 510

15

20

25

30

35

# of decoded geometry batches

PS

NR

m (

dB)

C = 0

C = 1

C = 2

C = 3

C = 4C : # of decoded connectivity batches

nnectivity data amplifies the geometric error and degrades visual

ns is measured by the PSNRm, which objectively demonstrates the

as the error-propagation effect of decoding connectivity without

e highest and lowest points on the Y-axis in (b), respectively.

ARTICLE IN PRESS

50

45

35

30

25

20

15

100 2 4 6 8

60

50

40

30

20

1010

10

50

40

30

20

1010

40

# of decoded geometry batches

0 2 4 6 8 10# of decoded geometry batches



PS

NR

m (

dB)

PS

NR

m (

dB)

PS

NR

m (

dB)

50

45

35

30

25

20

15

10

60

55

50

45

40

35

30

25

20

15

10

40

PS

NR

m (

dB)

50

45

35

30

25

20

15

10

40

PS

NR

m (

dB)

PS

NR

m (

dB)

86

42

0 02

46

8 10

Horse

Geometry

Geometry

Connectivity

Dinosaur

86

42

0 0 24

68

10

Connectivity

Fig. 16. Decoding distortion measured by PSNRm for different decoding patterns of the multi-resolution mesh. All the models are with 10

enhancement batches.



importantly, as shown in the paper, PSNRm providesmeaningful and approximately unified numericalvalues that well match human’s subjective experienceof using PSNR in imaging. Fig. 14 presents anotherexample, where five resolutions are generated fromthe TRICERATOPS model. One can see that themeasured PSNRm values meaningfully reflect theperceived difference of the resolutions compared tothe full-resolution mesh.

Appendix B. Connectivity versus geometry

Connectivity and geometry information havedifferent impacts on decoding multi-resolutionmeshes. Missing connectivity information stops thedecoding process because the vertex-split process isnot able to continue without knowing cut-edges;missing geometric data does not prevent the decoderfrom proceeding to the next level since the decodercan keep refining the topology using connectivityinformation with predicted vertex positions. Forbetter understanding on this procedure, one mayrefer to [16,21] for further details.

An intuition resulting from the above descriptionis that connectivity is more important than geome-try. Nonetheless, it is observed in our experimentsthat, because of error propagation, decoding con-nectivity without refining geometry tends to amplifythe geometric error and results in quality degrada-tion instead of higher resolution. From a visualiza-tion point of view, it implies that both connectivityand geometry are crucial in preserving the quality.This conclusion has been utilized in the paper fordesigning the transmission mechanism. As a sup-port, several results from our tests are presented inthis section.

Fig. 15 gives an example of decoding the fiveresolutions of the TRICERATOPS model. In Fig. 15(a),the base mesh with 692 faces and the decoded levelwith all faces added without geometric data arepresented. One can see that the perceptual distor-tion of the latter model is substantially largecompared with the base mesh even though it hasthe fully recovered topology. Note that the visualdifference is also properly reflected by the PSNRm

values.Fig. 15(b) presents more results for different

decoding patterns using the PSNRm metric. Eachdashed line in Fig. 15(b) denotes a decoding pathwith a certain number of connectivity batches,noted as C ¼ 0; 1; 2; . . . in the plot. For example,the line marked with C ¼ 0 in Fig. 15(b) is

horizontal because there is no connectivity informa-tion decoded, in which case all the geometry batchesare also not decodable. On the other hand, thepoints that align vertically in the plot correspond toa decoding path with the same number of geometrybatches and various connectivity. The base mesh inFig. 15(a), for example, corresponds to the highestpoint on the Y-axis in the plot, while the lowestpoint on the Y-axis corresponds to the mesh withfull connectivity as also shown in Fig. 15(a).Fig. 15(c) provides a three-dimensional perspectivefor the same results. The points on the diagonal ofthe mesh plot correspond to those on the envelopein Fig. 15(b), which have equal numbers ofconnectivity and geometry levels decoded. Con-formably, such a point has either higher qualitythan the points that are on the same line with it inthe X-dimension (same Y, larger X), or smaller dataset compared with the points that horizontallyconnect to them in the Y-dimension (same X, largerY). Similar results for the test models used in thepaper are also obtained, which are presented inFig. 16. All the results confirm that highest PSNRm

values are achieved by equally decoding connectiv-ity and geometry.

References

[1] P. Alliez, M. Desbrun, Progressive compression for lossless

transmission of triangle meshes, in: Proceedings of the 28th

Annual Conference on Computer Graphics and Interactive

Techniques, 2001, pp. 195–202.

[2] G. Al-Regib, Y. Altunbasak, An unequal error protection

method for packet loss resilient 3-D mesh transmission, in:

Proceedings of IEEE INFOCOM, vol. 2, 2002, pp. 743–752.

[3] G. Al-Regib, Y. Altunbasak, 3TP: an application-layer

protocol for streaming 3-D graphics, in: Proceedings of the

IEEE International Conference on Multimedia and Expo,

vol. 1, 2003, pp. 421–424.

[4] G. Al-Regib, Y. Altunbasak, J. Rossignac, An unequal error

protection method for progressively compressed 3D models,

IEEE Trans. Multimedia 7 (4) (2005) 766–776.

[5] G. Al-Regib, Y. Altunbasak, 3TP: an application-layer

protocol for streaming 3-D graphics, IEEE Trans. Multi-

media 7 (6) (2005) 1149–1156.

[6] G. Al-Regib, Y. Altunbasak, R.M. Mersereau, Bit allocation

for joint source and channel coding of progressively

compressed 3-D models, IEEE Trans. Circuits Syst. Video

Tech. 15 (2) (February 2005) 256–268.

[7] C.L. Bajaj, S. Cutchin, V. Pascucci, G. Zhuang, Error

resilient transmission of compressed VRML, Technical

Report, TICAM, The University of Texas at Austin, 1998.

[8] B.-Y. Chen, X. Nishita, Multiresolution streaming mesh

with shape preserving and QoS-like controlling, in: Proceed-

ings of Web3D, 2002, pp. 35–42.


[9] Z. Chen, B. Bodenheimer, F.J. Barnes, Robust transmission

of 3D geometry over lossy networks, in: Proceedings of

Web3D, 2003, pp. 161–172.

[10] P. Cignoni, C. Rocchini, R. Scopigno, Metro: measuring

error on simplified surfaces, in: Proceedings of Euro-

graphics, vol. 17(2), 1998, pp. 167–174.

[11] S. Floyd, M. Handley, J. Padhye, J. Widmer, Equation-

based congestion control for unicast applications, in:

Proceedings of ACM SIGCOMM, 2000, pp. 43–56.

[12] S. Floyd, V. Jacobson, Random early detection gateways for

congestion avoidance, IEEE/ACM Trans. Networking 1 (4)

(1993) 397–413.

[13] M. Handley, S. Floyd, J. Padhye, J. Widmer, TCP friendly

rate control (TFRC): protocol specification, Request

for Comments (RFC) 3448, The Internet Society, January

2003.

[14] A.F. Harris, R. Kravets, The design of a transport protocol

for on-demand graphical rendering, in: Proceedings of

NOSSDAV, 2002, pp. 43–49.

[15] U. Horn, K. Stuhlmuller, M. Link, B. Girod, Robust

internet video transmission based on scalable coding and

unequal error protection, Signal Process: Image Commun.

15 (1999) 77–94.

[16] H. Hoppe, Progressive mesh, in: Proceedings of ACM

SIGGRAPH, 1996, pp. 99–108.

[17] ISO/IEC JTC 1/SC 29/WG 11, Generic coding of

audio-visual objects—Part 2: Visual, ISO/IEC 14496-2,

1999.

[18] A. Khodakovsky, I. Guskov, Normal mesh compression, in:

Geometric Modeling for Scientific Visualization, Springer,

Berlin, 2002.

[19] A. Khodakovsky, P. Schroder, W. Sweldens, Progressive

geometry compression, in: Proceedings of ACM SIG-

GRAPH, 2000, pp. 271–278.

[20] J. Li, C.-C. Kuo, Progressive coding of 3D graphic models,

in: Proceedings of the IEEE, 1998, pp. 1052–1063.

[21] R. Pajarola, J. Rossignac, Compressed progressive meshes,

IEEE Trans. Visualization Comput. Graphics 6 (1) (2000)

79–93.

[22] R. Rejaie, M. Handley, D. Estrin, RAP: an end-to-end rate-

based congestion control mechanism for realtime streams in

the internet, in: Proceedings of IEEE INFOCOM, vol. 3,

1999, pp. 1337–1345.

[23] G. Taubin, 3D geometry compression and progressive

transmission, in: Proceedings of Eurographics, 1999.

[24] G. Taubin, A. Gueziec, W. Horn, F. Lazarus, Progressive

forest split compression, in: Proceedings of ACM SIG-

GRAPH, 1998, pp. 123–132.

[25] UCB/LBNL/VINT, network simulator ns (version 2),

Available on URL: hhttp://www.isi.edu/nsnam/ns/i.

[26] B. Watson, A. Friedman, A. McGaffey, Measuring and

predicting visual fidelity, in: Proceedings of ACM SIG-

GRAPH, 2001, pp. 213–220.

[27] Z. Yan, S. Kumar, C.-C. Kuo, Error-resilient coding of 3-D

graphic models via adaptive mesh segmentation, IEEE

Trans. Circuits Syst. Video Tech. 11 (7) (2001) 860–873.

http://www.isi.edu/nsnam/ns/

Documents

On-demand transmission of 3D models over lossy networks · 2020. 7. 31. · level (UEP/SR), and a transport protocol integrated with TCP-friendly rate control (TFRC). Next, we brieﬂy