Cross-layer design for mobile multimedia

Doug Young Suh Media Lab., Department of Electric and Radio Engineering

School of Electronic and Information, Kyung Hee University Suwon, Korea suh@khu.ac.kr

Abstract— Mobile multimedia is getting popular. It is expected that 70% of mobile Internet traffic will be video in 2013. Video traffic is so heavy that effective delivery will reduce total amount of Internet traffic as well as your communication fee. This paper argues that cross-layer design will enable the optimal QoE (Quality of Experience) and QoS (Quality of Service) controls for mobile multimedia services.

Keywords-Cross-layer design; QoS; QoE; multimedia

I. INTRODUCTION In your smart phone, signal strength of 3G channel and that

of WiFi channel are shown. It is a cross layer application which is familiar to everybody. Signal strength is measured in physical layer and is displayed by an application of application layer. They give us information about wireless channel condition and we decide an action according to the information. We feel that the channel condition is time varying and dependent on location. We can enjoy fast Internet surfing by locating yourself in the best WiFi zone. It would be very helpful that mobile multimedia service is controlled according to current channel condition, automatically. But, it is not possible yet for various reasons.

Goal of CLD : Mobile multimedia services include several players such as customer(or user), service provider, network provider, and manufacturer. CLD will let customer enjoy higher quality multimedia at lower communication fee or lower consumption of battery power. CLD will let service provider to transport more multimedia streams within less leased bandwidth. CLD will let network provider serve more users by using less channel resources such as bandwidth and buffer space. CLD will make manufacturer happy since CLD equipped smart phone will get better reputation in market. More aggressively speaking, CLD will reduce world-wise CO2 emission amount.

QoS/QoE : Currently, all the wireless standards argue that they support QoS. What does ‘support QoS’ mean? It means to satisfy or, at least, to try to satisfy different QoS requirements of different services. Service requirements can be categoried into 4 categories as shown Table 1[1]. They are different in bitrate as well as temporal QoS and semantic QoS requirements. Temporal QoS refers to delay and jitter while semantic QoS refers to bit error and packet loss. In Internet, bit error is not considered.

Conversational service is very sensitive to delay. For comfortable video phone service, end-to-end delay from

capturing image to display image should be less than 150ms. Streaming service like VOD(Video-On-Demand) allows 5~10 seconds buffering delay, but is very strict on packet loss since the users want to enjoy movie of good image quality. Interactive service like Web browsing may allow reaction within 2 seconds while interactive service like fighting game is very sensitive in delay. Download service is not sensitive to delay at all, but no packet loss is allowed.

QoE, degree of user satisfaction, can be said to result from QoS. QoS can be expressed by objective parameters (bitrate, delay, loss rate) while QoE is rather subjective like MOS(mean of scores). QoS parameters show trade-off fashion. For example, if you retransmit lost packets to reduce loss rate, then, delay and bitrate will become longer and higher, respectively.

Section 2 explains QoS tools in all protocol layers. It is followed by Section 3 which explains why they cannot have been used together. Section 4 provides possibilities of realization of cross layer design. Section 5 concludes this paper.

II. QOS TOOLS All network protocols provide QoS tools to enhance QoS.

As defined, ‘protocols’ are agreement between horizontal peers. For example, UDP(User Datagram Protocol) is transport layer protocol implemented in the same horizontal layer in the server and the client. Protocols in a layer are independent of those in the other layers. In order to enhance QoS of mobile multimedia, every layer provides its own QoS enhancement tools.

A. Video layer Rate control : Bitrate can be adaptively controlled by adapt

quantization level according to available bitrate. The higher bitrate is, the higher the quality of reconstructed video is. This is applicable only live or conversation services in which video capturing, encoding, transmitting, and decoding are concurrent. Rate also can be controlled for pre-coded video by using scalable video coding techniques by which a video sequence is encoded into more than one bitstreams progressively dependent. There are 3 kinds of scalability which provides multi-layered bitstreams different in spatial resolution, temporal resolution, quantization resolution, or combination of three.

Error propagation : Since a image frame is encoded as predicting from previously encoded frames forward and backward, error in an image propagates to subsequent frames. In order to avoid this propagation, Intra frames which is encoded by itself are periodically(usually, 0.5sec period).

This research was supported by the MKE(The Ministry of Knowledge Economy), Korea, under the ITRC(Information Technology Center) support program supervised by the NIPA(National IT Industry Promotion Agency) (NIPA-2011-(C1090-1111-0001))

Conference Proceeding 2012 21st Annual Wireless and Optical Communications Conference(WOCC)-April 19-21, Kaohsiung, Taiwan

978-1-4673-0941-7/12/$31.00 ©2012 IEEE -77-

Bitrate of Intra frame is much higher than those of predicted frames.

Data partitioning : Encoded video data can be sorted in terms of significance. In Intra frame, DC values are more important than the other information, while MV(Motion Vector) values are more important. By just using DC and MV values, one can reconstruct image of minimum quality. Remained transform data can also be partitioned in terms of frequency bands, among which lower frequency bands are more important.

Delay sensitive coding allows only forward prediction, since backward prediction causes delay. If backward prediction is used in 30Hz video(30 frames per seconds), more than 60ms delay occurs in the encoder.

B. Transport layer Roles of transport layer are end-to-end flow control and

end-to-end error control. Flow control assures that the server transmits data as much as the receiver can receive. Error control here handles packet losses. For both purposes, RTCP(Realtime Transport Control Protocol) is used. The sender sends RTCP-SR(Sender Report) packet periodically(say, every second) with time stamp. Upon receiving the packet, the receiver sends RTCP-RR(Receiver Report) packet with measured packet loss ratio and time spent in the receiver.

The server calculates RTT(round-trip-time) from time-stamps for flow control. Larger RTT is considered to be caused by network congestion or buffer fullness in the receiver, so that the server reduces bitrate, vice versa.

Lost packet can be retrieved by FEC(Forward Error Coding) or retransmission. Since Retransmission causes delay, FEC is more popular in realtime services. Limited number of retransmissions can be allowed if RTT is much smaller than delay constraints of the service. Parity ratio of FEC can be controlled by the packet loss ratio informed by RTCP-RR.

C. Network layer Network layer protocols determine actions in routers. There

are 3 kinds of QoS protocols in network layer, such as best effort(BE), per-class QoS, and per-flow QoS. Best effort means that network does not do anything for QoS. Most Internet packets are delivered in this category. IETF(Internet Engineering Task Force, ietf.org)’s per-class QoS protocol is diffServ, while that of per-flow QoS is intServ accompanied with RSVP(ReSource reserVation Protocol).

Per-class QoS : Both IPv4 and IPv6 headers have 8 bit long field for per class QoS. In IPv4 and IPv6, they are called as ToC(Type of Class) and TC(Traffic Class), respectively. In DSCP(diffServ CodePoint), the first 3 bits are used for selection of PHB(Per Hop Behavior), while next two bits are used for loss priority. The highest priority PHB, 101 is called EF(Expedite Forwarding) which is used for delivery of very high priority service like VoIP. PHBs of 100, 011, 010, and 001 are called AF(Assured Forwarding) 1~4, respectively, of medium level priority. 000 is assigned for BE while usages of 111 and 110 are not defined. PHBs is different in scheduling schemes. In the same PHB, loss rate is determined by loss

priority. As approaching congestion, packets of lower priority are discarded at higher probability. 11 is the highest while 00 is the lowest.

Per-flow QoS : Network resources are reserved along the path. Network resources include guaranteed bitrate and peak bitrate, and buffer spaces associated to two bitrate. RSVP(Resource reSerVation Protocol) is used for reservation for continuous streams. IntServ specifies how every packet of a stream can be identified in routers along the path. 5 tuples such as source/destination IP addresses, source/destination port numbers, and protocol are used for identification. An identified packet is served as agreed and written in the routing table. This is so complex process that number of simultaneous per-flow connections cannot be infinitely large in a router. This problem, so called, ‘scalability problem’ has impeded deployment of per-flow QoS in the open Internet.

D. MAC/PHY layer QoS in this layer is of high concern in these days, because

mobile multimedia services are getting popular, QoS of the last link dominates end-to-end QoS, and all wireless standards insist to support QoS. Basically, MAC/PHY layer provides flow control and error control.

Flow control also can be categorized into per-class and per-flow controls. IEEE802.11e, so-called, ‘WiFi’ provides 4 ACs(Access Categories) which are four queues of different queuing policies as a per-class QoS scheme. IEEE802.16m, so-called, ‘WiMAX’ as well as 3G UMTS and 4G LTE support both per-class and per-flow QoS protocols. Per-flow QoS, here is very feasible since there is no ‘scalability problem’ in the last hop so that we may reserve resources in the last hop.

Error control is very important in wireless link. 3GPP uses Raptor code as an FEC. In physical layer, hybrid ARQ(Automatic Repeat reQuest) which is combination of FEC and retransmission is very powerful error correcting tool. Finite retransmissions are allowed since RTT is less than few milli-seconds.

Error rate and bitrate can be adaptive by power control in physical layer. They show trade-off relationship and usually target error rate is fixed so that power control changes bitrate.

III. PROS AND CONS ON CLD (CROSS LAYER DESIGN) All protocol layers are supporting QoS and some tools

seem desirable if used together. For example, scalable video coding provides bitstreams of different significance and per-class QoS looks very helpful to deliver them accordingly. Combination of QoS tools in different layers is called cross-layer design, which seems promising, but has not been deployed widely.

A. Independence principle of OSI 7 layer model One of the major principles of conventional OSI(Open

System Interconnections) 7 layer model is “independence”. All 7 layers are independently designed, implemented, and maintained in condition that interfaces between two adjacent layers are kept as agreed. Even specialists for each layer are educated independently. It is very advantageous principle for

Conference Proceeding 2012 21st Annual Wireless and Optical Communications Conference(WOCC)-April 19-21, Kaohsiung, Taiwan

978-1-4673-0941-7/12/$31.00 ©2012 IEEE -78-

amount

satisfaction

cokepizza

coke

pizza

Equi-cost line with

10$

Optimal Choice

S3<S2<S1Equi-satisfaction

S1S2S3

parallel management of layers, even though it results in inefficiency, which has been negligible in wired networks.

Protocols which have been realized in hardware are substituted by software. Advances in software engineering technology enables to handle extremely sophisticated system. SDR, or Software Defined Radio research aims to make all 7 layers interoperable in one software package. This goal seems unrealistic. But, some results are implemented and deployed as cognitive radio in co-existing networks.

B. Shannon’s Separation Rule Shannon[2], ‘the father of digital communication’ proved

“separation principle”. It says that source coding which removes redundancy for data compression can be separated from channel coding which increase redundancy against information loss. (Source coding is an issue of video layer while channel coding is an issue of transport and MAC/PHY layers.) The rule is true in very ideal condition of one-to-one communication and unlimited complexity and delay.

In real-world, these conditions are not realistic. Multicast/broadcast is not one-to-one communication. Condition of mobile channel is time-varying and dependent on location. All real-time services do not wait infinite time for optimization. A multimedia stream is composed of sub-streams of different significance. JSCC(Joint Source Channel Coding) as breach of separation principle takes into account of these facts.

C. Over-provision CLD may save amount of required resources at best by

20~30% hopefully as results of sophisticated algorithms and endless discussion between experts of different layers. But, suppose that network provider provides 100% more resources at the same price. Then, the effort for realization of CLD becomes of no value. It has been almost so during the last 2 decades. You all can recognize that available bandwidth, either in wired network or wireless network has been increasing exponentially.

Even though they construct high-ways, there are always congested bottlenecks, which need traffic control. Advent of smart phone increases real-time multimedia service traffic drastically and, moreover, display size gets bigger and bigger. Mobile channel is said to suffer from 3 low 1 high, that is, low in battery, CPU, and bandwidth, but high in cost. Flat fee will be no more kept and usage-based fee will be popular so that resource saving will become bigger issue.

IV. EXAMPLES OF CLD APPROACH

A. Basics for CLD research Principle of diminishing marginal utility is originated from

economics. Utility(degree of satisfaction) of first sip of coke is higher than that of tenth sip. Utility of unit amount coke is getting down as drinking more coke and corresponds to stiffness of slope in Figure 1. As shown in the figure, utility of pizza also shows the same fashion. Suppose that you have 10$ for buying pizza and coke and maximize total utility of

coke and pizza. All combination of amounts of pizza and coke make a straight line so called “equi-cost line” in Figure 2. Trace of the same total utility will make a monotone decreasing curve so called “equi-satisfaction curve” as shown Figure 2 where vertical and horizontal axes correspond to amount of coke and pizza, respectively. The highest satisfaction is accomplished at the contact point of an equi-cost line and an equi-satisfaction curve.

Figure 1. satisfaction against amount, Figure 2. optimal point

Most parameters related to mobile multimedia follow ‘principle of diminishing marginal utility.’ For example, quality of video increases sharply as increasing bitrate at low bitrate while increasing rate goes down at high bitrate. For another example in physical layer, Shannon’s channel capacity shows the same fashion against SNR(Signal to Noise Ratio). Even though most of them show the same fashion, their units are different so that Lagrange equation which handles multiple variables is for optimization.

For example, in the following equation for video streaming service, in application layer (or video layer) video quality distortion D is to be minimized, and in physical layer required bitrate R and power consumption P is to be minimized. These variables D, R, and P are inter-correlated and have values within their own ranges.

) Do(subject t distortionquality Video : )P Po(subject t nComsumptio Power :

)1 PLR0 o(subject t Ratioss Packet Lo: ) Ro(subject t bitrate actual bitrate, available :, where

,),,,(

min

max

avail

2121

DDPPLR

RRRPλRλ,PLR)D(RPLRRJMinimize

avail

availavail

><

<<<

×+×+=ll

B. International Standards Video people work in MPEG and ITU-T for

standardization of multimedia compression and systems for storage and delivery while network people work in IETF, 3GPP, and IEEE802 series for standardization from physical layer to transport layer. In some senses, their scopes are overlapped, but still cross layer approaches have not been so active.

Currently, cross-layer activities in MPEG seem interesting. They analyzed current situation comprehensively in view of state-of-art technology, market, and future trends. MMT(MPEG Media Transport) standard which will be finalized in the end of 2013 takes cross layer design in consideration.

Conference Proceeding 2012 21st Annual Wireless and Optical Communications Conference(WOCC)-April 19-21, Kaohsiung, Taiwan

978-1-4673-0941-7/12/$31.00 ©2012 IEEE -79-

In its working draft, they defined bidirectional exchange of QoS information. Top-down QoS information provides application information to underlying network. Application layer has media specific information which is helpful for QoS control in the underlying network. It helps the underlying network to understand media specific data for various types of QoS control. Bottom-up QoS information provides the underlying network information to the application layer. The underlying network has the accurate information about time-varying network condition which helps QoS control in application layer faster and more accurately. There are many different the underlying networks. Therefore, to provide the underlying network information efficiently, the format of information should be understandable by the application layer.” They call for participation of experts who understand both sides such as multimedia and network.

V. CONCLUSION In the market, barriers against cross layer design are

collapsing. Cloud and CDN(Contents Delivery Network) allows more intelligence in the middle of network. DASH(Dynamic Adaptive Streaming over HTTP), as a part of MMT in MPEG and as a part of TS26.234 so called PSS(Packet Switched Service) in 3GPP exploits HTTP cache(proxy) functionalities in CDN. Currently, could is being used only for leasing storage and leasing bandwidth, but if leasing CPU in cloud is popular, more intelligent processing such as QoS and QoE controls by using cloud will be possible soon. Network neutrality with cloud and CDN will encourage

application developers, manufacturers, and service providers to develop more efficient multimedia services, for whose interoperability international standards on cross layer design should be established.

Green IT is another area in which cross layer design is important. Goal of Green IT is to reduce energy consumption with respect to IT devices and services. Horizontally, total energy consumption in user terminals, networks, data centers, and cloud can be optimized by comprehensive control. Vertically, energy consumption in a terminal can be optimized by cross layer control over CPU, memory, display, transmitter/receiver and so on.

Cross layer design has long history academically. Most researches have shown dramatic performance, but have been looked down because of lack of sense of reality. This paper says that in the next decades some of them will be realized in the market by virtue of paradigm shifts in standardization, market needs, network environments, complexity handling capacity, and, most of all, harmonization efforts of experts from different areas.

REFERENCES

[1] 3GPP Technical Specification “TS 22.105: Service and Service capabilities,” March 2002.

[2] Claude Shannon, “A Mathematical Theory of Communication,” Bell System Technical Journal, Vol. 27, pp. 379-423, July and October 1948.

Conference Proceeding 2012 21st Annual Wireless and Optical Communications Conference(WOCC)-April 19-21, Kaohsiung, Taiwan

978-1-4673-0941-7/12/$31.00 ©2012 IEEE -80-