12
Research Article Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media Playout Mingfu Li , 1,2 Chien-Lin Yeh, 1 and Shao-Yu Lu 1 1 Department of Electrical Engineering, School of Electrical and Computer Engineering, College of Engineering, Chang Gung University, Guishan District, Taoyuan City 33302, Taiwan 2 Neuroscience Research Center, Chang Gung Memorial Hospital, Linkou, Guishan District, Taoyuan City 33305, Taiwan Correspondence should be addressed to Mingfu Li; mfl[email protected] Received 9 November 2017; Accepted 22 January 2018; Published 15 February 2018 Academic Editor: Miroslav Voznak Copyright © 2018 Mingfu Li et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Quality of Experience (QoE) of video streaming services has been attracting more and more attention recently. erefore, in this work we designed and implemented a real-time QoE monitoring system for streaming services with Adaptive Media Playout (AMP), which was implemented into the VideoLAN Client (VLC) media player to dynamically adjust the playout rate of videos according to the buffer fullness of the client buffer. e QoE monitoring system reports the QoE of streaming services in real time so that network/content providers can monitor the qualities of their services and resolve troubles immediately whenever their subscribers encounter them. Several experiments including wired and wireless streaming were conducted to show the effectiveness of the implemented AMP and QoE monitoring system. Experimental results demonstrate that AMP significantly improves the QoE of streaming services according to the Mean Opinion Score (MOS) estimated by our developed program. Additionally, some challenging issues in wireless streaming have been easily identified using the developed QoE monitoring system. 1. Introduction Internet Protocol Television (IPTV) services [1] have been becoming increasingly popular among telecommunication companies. Network/service providers are concerned with not only the cost of constructing an IPTV service system [2] but also its service quality [3, 4]. In the past, Quality of Service (QoS) of network services is the main concern. However, QoS cannot accurately characterize the users’ perception. erefore, the concept of Quality of Experience (QoE) was proposed and studied recently [5–13]. e most accurate method to obtain QoE is doing subjec- tive tests [14] that are time-consuming and costly. erefore, we proposed a cost-effective and real-time QoE evaluation method in [5], where a hybrid QoE assessment scheme was employed. In the hybrid QoE assessment, subjective tests only need be performed occasionally for updating the parameters of QoS-QoE mapping functions. Hence, the cost of a hybrid QoE assessment scheme reduces significantly. In [7], the QoE measurement for P2P-based IPTV ser- vices was studied. Another paper [8] proposed a QoS/QoE mapping and adjustment model for the cloud-based multi- media infrastructure. Additionally, since streaming services over wireless networks have been becoming popular, the QoE enhancement algorithms, protocols, and evaluation methods for wireless streaming services were proposed [9–12]. In [13], Chen et al. presented a comprehensive survey of the evolution of video quality assessment methods and analyzed their characteristics, advantages, and drawbacks. Since the QoE evaluation for streaming services becomes more and more important, several QoE evaluation or moni- toring tools were developed [15–19]. A QoE monitoring tool for multimedia quality assessment in NS-3 network simulator was presented in [15]. However, since the objective and full- reference QoE assessment method is used in [15], it is costly and cannot accurately indicate the users’ perception in a real commercial video streaming system. e paper [16] devel- oped a testbed, which allows obtaining conclusive results regarding QoE in video-mediated group communication, for controlled experiments. In [17], Jeong and Ahn designed the QoE monitoring function for video services by extending ITU-T IPTV QoS/QoE metrics and integrating to OMA-DM Hindawi International Journal of Digital Multimedia Broadcasting Volume 2018, Article ID 2619438, 11 pages https://doi.org/10.1155/2018/2619438

Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

  • Upload
    others

  • View
    1

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

Research ArticleReal-Time QoE Monitoring System for Video StreamingServices with Adaptive Media Playout

Mingfu Li 12 Chien-Lin Yeh1 and Shao-Yu Lu1

1Department of Electrical Engineering School of Electrical and Computer Engineering College of Engineering Chang GungUniversityGuishan District Taoyuan City 33302 Taiwan2Neuroscience Research Center Chang Gung Memorial Hospital Linkou Guishan District Taoyuan City 33305 Taiwan

Correspondence should be addressed to Mingfu Li mflimailcguedutw

Received 9 November 2017 Accepted 22 January 2018 Published 15 February 2018

Academic Editor Miroslav Voznak

Copyright copy 2018 Mingfu Li et alThis is an open access article distributed under theCreative CommonsAttribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

Quality of Experience (QoE) of video streaming services has been attracting more and more attention recently Therefore in thiswork we designed and implemented a real-time QoE monitoring system for streaming services with Adaptive Media Playout(AMP) which was implemented into the VideoLAN Client (VLC) media player to dynamically adjust the playout rate of videosaccording to the buffer fullness of the client buffer The QoE monitoring system reports the QoE of streaming services in realtime so that networkcontent providers can monitor the qualities of their services and resolve troubles immediately whenever theirsubscribers encounter them Several experiments including wired and wireless streaming were conducted to show the effectivenessof the implemented AMP and QoE monitoring system Experimental results demonstrate that AMP significantly improves theQoE of streaming services according to the Mean Opinion Score (MOS) estimated by our developed program Additionally somechallenging issues in wireless streaming have been easily identified using the developed QoE monitoring system

1 Introduction

Internet Protocol Television (IPTV) services [1] have beenbecoming increasingly popular among telecommunicationcompanies Networkservice providers are concerned withnot only the cost of constructing an IPTV service system [2]but also its service quality [3 4] In the past Quality of Service(QoS) of network services is the main concern HoweverQoS cannot accurately characterize the usersrsquo perceptionTherefore the concept of Quality of Experience (QoE) wasproposed and studied recently [5ndash13]

Themost accurate method to obtain QoE is doing subjec-tive tests [14] that are time-consuming and costly Thereforewe proposed a cost-effective and real-time QoE evaluationmethod in [5] where a hybrid QoE assessment schemewas employed In the hybrid QoE assessment subjectivetests only need be performed occasionally for updating theparameters of QoS-QoE mapping functions Hence the costof a hybrid QoE assessment scheme reduces significantlyIn [7] the QoE measurement for P2P-based IPTV ser-vices was studied Another paper [8] proposed a QoSQoE

mapping and adjustment model for the cloud-based multi-media infrastructure Additionally since streaming servicesover wireless networks have been becoming popular the QoEenhancement algorithms protocols and evaluation methodsfor wireless streaming services were proposed [9ndash12] In[13] Chen et al presented a comprehensive survey of theevolution of video quality assessment methods and analyzedtheir characteristics advantages and drawbacks

Since the QoE evaluation for streaming services becomesmore and more important several QoE evaluation or moni-toring tools were developed [15ndash19] A QoE monitoring toolformultimedia quality assessment inNS-3 network simulatorwas presented in [15] However since the objective and full-reference QoE assessment method is used in [15] it is costlyand cannot accurately indicate the usersrsquo perception in a realcommercial video streaming system The paper [16] devel-oped a testbed which allows obtaining conclusive resultsregarding QoE in video-mediated group communication forcontrolled experiments In [17] Jeong and Ahn designed theQoE monitoring function for video services by extendingITU-T IPTVQoSQoEmetrics and integrating to OMA-DM

HindawiInternational Journal of Digital Multimedia BroadcastingVolume 2018 Article ID 2619438 11 pageshttpsdoiorg10115520182619438

2 International Journal of Digital Multimedia Broadcasting

protocol agent Although themonitoring service can improvethe user experience the presented QoE factors are still notgood enough In [18] the proposed concept Monitoringof Audio Visual Quality by Key Performance Indicators(MOAVI) is able to isolate and focus investigation set upalgorithms increase the monitoring period and guaranteebetter prediction of perceptual quality In [19] Ickin et aladded functionality to the VideoLAN Client (VLC) mediaplayer [20] to record a set of metrics from the user interfaceapplication-level network-level and the available sensors ofthe device Based on these metrics the proposed VLQoE aQoE instrumentation for video streaming can infer the QoEof streaming services on a smartphone

To improve the playout quality of video streamingservices Adaptive Media Playout (AMP) schemes [21 22]have been used In this work an AMP playout system isimplemented into the VLC media player which is a freeand open source cross-platformmultimedia player Since theplayout rate of the AMP system can be dynamically adjustedduring video playback the effects of AMP on QoE mustbe considered as well However almost all the developedQoE monitoring tools in the literature [15ndash19] did not takethe effects of AMP on QoE into account Thus in [23] weproposed the QoE monitoring system that takes the effectsof AMP on QoE into consideration Additionally the mostimportant feature of our proposed QoE monitoring systemis that it can derive an overall QoE for several QoS metricsusing the product form presented in [5] Our previous work[5] has shown that the product form performs much betterthan the conventional averaging techniques in deriving anoverall QoE of multiple QoS metrics Using the developedQoE monitoring program network providers can identifynetwork troubles and fix them in real time to improve thesystem performance and qualities

The novelties and contributions of this paper are summa-rized as follows First the implemented QoEmonitoring sys-tem takes the effects of AMP onQoE into consideration Sec-ond the developedQoEmonitoring systemderives an overallQoE from several QoS metrics such as the initial playoutdelay packet loss rate underflow time ratio and normalizedplayout rate Thirdly we implement the AMP scheme whichwas proposed in [22] into the VLC media player Finally theGroup of Pictures- (GOP-) based cumulative average jitter 119869119899which is used to determine the playback threshold 119875119899 of theproposed AMP is proposed to eliminate the effect of diverseframe sizes on the estimation of network jitter

The rest of this paper is organized as follows Sec-tion 2 describes the implementation of the AMP playoutsystem Implementation of the QoE monitoring system isthen introduced in Section 3 Section 4 conducts severalexperiments and evaluates the QoE performance of videostreaming services using the developed programs Finally theconcluding remarks are given in Section 5

2 Implementation of AMP

This section first describes the principles of AMP inSection 21 Then the implementation schemes available

programming tools and some challenging issues on mea-surements or estimations of key parameters are addressed inSection 22

21 Principles of Proposed AMP In this work we employthe freeware VLC media player to realize the AMP TheAMP is implemented according to our proposed scheme [22]whose principle is described briefly as follows In addition tothe conventional slowdown threshold 119871 a dynamic playbackthreshold 119875119899 and a speedup threshold 119867 are designed Ifthe buffer fullness is below 119871 the playout rate is reduced toavoid buffer underflow When the buffer fullness exceeds thespeedup threshold119867 the playout rate is increased to reducethe buffer fullness for avoiding buffer overflowThe DynamicPlaybackThreshold Algorithm (DPTA) in [22] is designed todynamically adjust the playback threshold 119875119899 which affectsthe initial playout delay under various network conditionsWhenever the current buffer fullness 119899 surpasses or equals119875119899 the playback starts immediately with the proper playoutrate 120583(119894 119899) which is determined by the following equation

120583 (119894 119899) =

119877119871 (119899) if 119899 lt 119871119877S (119894) if 119871 le 119899 le 119867119877119867 (119899) if 119899 gt 119867

(1)

where 119877119878(119894) is a random process and is estimated by theArrival Process Tracking Algorithm (APTA) presented in[22] That is 119877119878(119894) depends on the frame arrival process atthe client buffer and is limited between (1 minus 119903)1205830 and (1 +119903)1205830 as shown in Figure 1 Anyone interested in the detailedderivation of 119877119878(119894) and 119875119899 can refer to [22] The quadraticplayout rate functions 119877119867(119899) and 119877119871(119899) in [22] are definedas follows

119877119867 (119899) = (1 + 1199032) 1205830 minus (119873 minusmin 119899119873119873 minus 119867 )

2

(1199032 minus 119903) 1205830

119877119871 (119899) = (1 minus 1199031) 1205830 + (119899119871)2

(1199031 minus 119903) 1205830(2)

where 119873 is the uppermost threshold correlated to the buffersize at the client Notably the playout rate must be limitedsuch that rate variations are unnoticeable or acceptable byusersThe perception of a slowdown video is usually differentfrom that of a speedup video for users Therefore twodifferent restricted deviation ratios denoted by 1199031 and 1199032 for119877119871(119899) and 119877119867(119899) respectively are set for playout rates Thecorresponding playout rate function of the proposed AMPis given by Figure 1 Numerical results in [22] have shownthat the proposedAMPwith quadratic playout rates performsmuch better than several conventional playout systems withlinear playout rates and can adapt to different network loadconditions for reducing the initial playout delay

22 Realization of AMP VLC media player provides auseful VLC ActiveX package whose architecture is given inFigure 2 The main component of VLC ActiveX package isLibVLC which supports several properties such as input

International Journal of Digital Multimedia Broadcasting 3

(i n)

(1 + r2)0

(1 + r)0

(1 minus r)0

(1 minus r1)0

Warningzone

Warningzone

Safetyzone

0 L H N n

Figure 1 Playout rate function of the proposed AMP [22]

VLC user interface

MainLib VLC

Videooutput

Audiooutput

Decoder

Access

InputDemux

Figure 2 Architecture of VLC ActiveX package

audio output and video output The input property collectsall related parameters of the received video stream fromthe Network Interface Card (NIC) (Access) the networkdemultiplexer (Demux) and the video decoder (Decoder)as shown in Figure 2 Several useful statistical parametersin inputitem()stats() are listed in Table 1 Based on thesecollected parameters from the input property one can set andcontrol the playout rate of the VLCmedia player to realize theAMP function

According to the descriptions in Section 21 the playoutrate mainly depends on the buffer fullness 119899 (in frames) Toaccurately measure the number demux frames of demulti-plexed video frames one can directly analyze the packetsarriving at the NIC card By reading the ldquopayload unit startindicatorrdquo bit in the header of each 188-byte MPEG-2 TS(Transport Stream) packet as shown in Figure 3 one caneasily calculate the number demux frames of video framesreceivedWhen the ldquopayload unit start indicatorrdquo bit equals 1

Table 1 Receiving parameters in VLC inputitem()stats()

Parameters Meaningdemux read packets Number of packets receiveddemux read bytes Number of bytes receiveddemux bitrate Bit rate of the video streamaverage demux bitrate Average bit rate of the video streamdemux corrupted Number of corrupted framesdecoded audio Number of audio frames decodeddecoded video Number of video frames decodedlost pictures Number of frames lost

a new MPEG-2 TS video frame is arriving Hence the bufferfullness 119899 can be computed by the following equation

119899 = 119889119890119898119906119909 119891119903119886119898119890119904 minus 119889119890119888119900119889119890119889 V119894119889119890119900 (3)

After deriving the buffer fullness 119899 the playout rate of theAMP media player can be dynamically adjusted accordingto (1) The command to dynamically set the playout rate ofthe VLCmedia player is defined by the function varset(inputldquoraterdquo 120583(119894 119899)) Although the currently developed program isdesigned for the MPEG-2 TS format there is no difficulty toinclude other advanced video encoding technologies such asthe H264AVC and H265HEVC [24] in the future

The dynamic playback threshold 119875119899 [22] is determined by

119875119899 = 1198710 + (119871 minus 1198710) sdotmin 1 119869119899119888119879 (4)

where 1198710 and 119888 are design parameters and 119879 = 11205830 is theaverage frame interarrival time 119869119899 is the cumulative averagejitter and is defined by

119869119899 = (1 minus 1119899 minus 1) 119869119899minus1 +

1003816100381610038161003816119883119899minus1 minus 1198791003816100381610038161003816119899 minus 1 for 119899 gt 1 (5)

where 119883119899minus1 is the frame interarrival time between the(119899 minus 1)-st and the 119899th frames However video frames arecategorized into I B and P frames and different types offrames usually have very different frame sizes For examplethe frame size of an I frame is usually much larger thanthat of a B or P frame Hence the frame interarrival timebetween two successive I and B (or P) frames may becomerelatively large because of the long transmission time of anI frame compared with that between two successive B and Pframes Such variance of frame interarrival timesmaybe is notdue to the network jitter but the diversity of frame sizes Toreduce the impact of frame size variation on the estimation ofnetwork jitter 119869119899 and estimate 119869119899more accurately in this workthe average frame interarrival time within a GOP is usedHence the variable 119883119899minus1 in (5) is replaced by 119884119896minus1119866 where119896 = lfloor119899119866rfloor 119866 is the number of frames in a GOP and 119884119896minus1 isthe interarrival time between the I frames of the (119896minus1)-st andthe 119896th GOPs

3 Implementation of QoE Monitoring System

The procedures to implement the QoE monitoring systemare described as follows First we propose an architecture

4 International Journal of Digital Multimedia Broadcasting

188 bytes

Transportpacketstream

Header Header HeaderPayload Payload Payload

Syncbyte

Transporterror

indicator

Payloadunit startindicator

Transportpriority PID

Transportscrambling

control

Adaptationfield

control

Continuitycounter

Adaptationfield

8 1 1 1 13 2 2 4

Figure 3 MPEG-2 TS packet format

QoE monitoring system

QoE evaluation

QoEQoSreporting

NIC card AMP media player

QoS measurement(i) Underflow time ratio

(ii) Packet loss rate(iii) Initial playout delay(iv) Normalized playout rate

Figure 4 Architecture of the QoE monitoring system

for the QoE monitoring system Next several network-basedQoS metrics are defined and their measurement methods arepresented Finally the overall QoE evaluation method for theQoE monitoring system is introduced

31 Architecture of QoE Monitoring System The architec-ture of our proposed QoE monitoring system is plotted inFigure 4 It includes the QoS measurement engine QoEevaluation function and performance reporting windowThe QoS measurement engine is designed to collect theQoS metrics from the media player and the NIC cardThese obtained QoS metrics are used by the QoE evaluationfunction for producing the overall QoE of video streamingservices All instantaneous QoS metrics and QoE results arereported in real time and the curve of overall QoE versus timeis also displayed with a chart in the Graphic User Interface(GUI) for the monitoring purpose

32 Measurements of QoS Parameters In the proposed QoEmonitoring system the considered QoS metrics includeunderflow time ratio QoSu packet loss rate QoSl initial

Figure 5 Buffering mechanism in VLC player

playout delay QoSd and normalized playout rate QoSr Themeasurements of these parameters are described as follows

(a) Underflow time ratio QoSu whenever an underflowevent occurs the underflow time starts to accumulateuntil the playback restarts again At the same timethe number of underflow events is increased by 1Theunderflow time ratio equals the ratio of accumulatedunderflow time to the total playback time durationThe accumulated underflow time can be computedaccording to the value of Status() provided by VLCLua extension program as shown in Figure 5 Whenthe value of Status() changes into 2 the underflowtimer turns ON until the value of Status() becomes3The sojourn time at the state with Status() = 2 is theunderflow timeWhenever the underflow timer turnsOFF the accumulated underflow time is updated

(b) Packet loss rate QoSl the packet loss rate is obtainedby analyzing the number of receiving packets at the

International Journal of Digital Multimedia Broadcasting 5

NIC card or using the network protocol analyzer suchas Wireshark

(c) Initial playout delay QoSd initial playout delaystrongly depends on the buffering time set in VLCmedia player However in our implemented AMPthis parameter is directly determined by the DPTAalgorithm proposed in [22]

(d) Normalized playout rate QoSr whenever the playoutrate is adjusted the normalized playout rate is com-puted and saved However within each minute onlythe maximum and minimum normalized playoutrates are used in our design [5] That is the corre-sponding QoE of QoSr during every minute equalsthemean of these two individual QoEs resulting fromthe maximum and minimum normalized playoutrates

33 Overall QoE Evaluation Function The overall QoE eval-uation function adopts the integratedmultivariate QoE func-tion 119891119868(QoSuQoSlQoSdQoSr) and the individual QoS-QoE mapping functions 119891u(119909) 119891l(119909) 119891d(119909) and 119891r(119909) pro-posed in [5] to derive the overall QoE Based on the results of[5] they are given as follows

119891119868 (QoSuQoSlQoSdQoSr)

= 5 sdot 119891u (QoSu)5 sdot 119891l (QoSl)

5 sdot 119891d (QoSd)5

sdot 119891r (QoSr)5

(6)

119891u (119909) = 5119890minus571119909 (7)

119891l (119909) = 5119890minus1607119909 (8)

119891d (119909) = 5119890minus00416119909 (9)

119891r (119909) = 5119909894119890minus894(119909minus1) (10)

The scheme used in [5] consists of three steps to bedone beforehand First several Key Performance Indicators(KPIs) or QoS metrics such as the initial playout delaypacket loss rate underflow time ratio and playout ratewere selected Then several distorted videos were generatedbased on these selected QoS metrics Second subjective testswere conducted using human observers to rate the distortedvideos in terms of Mean Opinion Score (MOS) [25] TheACR subjective test method was adopted in [5] Howeverthe length of each distorted video clip is about 64 s ratherthan 10 s Each distorted video clip was viewed and ratedby no less than 30 viewers The title of the used video clipwhich is with the frame rate 30 fps resolution 1920 times 1080 pand video codec MPEG-2 is ldquoAvatarrdquo Thirdly using theregression approach the QoS-QoE mapping functions forindividual QoS metrics were obtained Finally the productform of individual QoS-QoE mapping functions [5 6] isused to derive the overall QoE of a video streaming serviceNotably although only four QoS metrics are considered in

Figure 6 VLC media player and Lua program

Figure 7 QoEMOS monitoring program

(6) the number of QoSmetrics can be arbitrarily extended asneeded

4 Experimental Results

In this section we first briefly introduce the GUIs andfunctions of developed AMP andQoEmonitoring programsSubsequently using the developedQoEmonitoring programseveral experiments are conducted to demonstrate the perfor-mance of AMP Finally we use the QoE monitoring programto identify some challenging issues of wireless streaming

41 QoE Monitoring System The developed AMP is basedon the VLC media player and its add-ons such as the Luaextension Figure 6 shows the VLC media player and itsLua extension program One can directly set the multicastIP of the streaming source and activate the AMP functionon the VLC Lua extension program Although in this paperthe AMP is implemented into the VLC media player thereis no difficulty to implement it into other popular mediaplayers such as the MS Windows Media Player or YouTubePlayer The QoE monitoring program which is developedusing the C++Builder and is shown in Figure 7 evaluates anddisplays the overall QoE The measured QoS and syntheticQoEMOS are displayed in real time on the monitoringwindow as shown in Figure 7 The corresponding QoEsof individual QoSs are displayed on the left side of themonitoringwindowwhile the curve of the overall QoE versustime is displayed on the right side of the monitoring windowas shown in Figure 7 By clicking the ldquoToolsrdquo on themenu barof the QoE monitoring program other detailed QoS metrics

6 International Journal of Digital Multimedia Broadcasting

Figure 8 Monitoring tools for video packets at the client

Figure 9 Experimental network scenario 1

Table 2 Information of test video 1

Parameter ValueVideo title The Big BangTheoryLength 593 secResolution 352 times 288 pixelsEncoding scheme MPEG-2 TSBit rate 1406 kbpsFrame rate 25 fpsGOP size 15Content size 995MB

such as the interarrival time of video frames normalizedplayout rate number of packet losses buffer fullness andthe jitter observed at the client can be displayed as shownin Figure 8 By clicking the interested QoS parameter suchas the ldquoMoveAvg Interarrivalrdquo the curve of the QoS versustime is immediately displayed on the monitoring window ofFigure 8

42 Performance Evaluation of AMP In order to demon-strate our AMP design and QoE monitoring system the firstexperimental network scenario is constructed as shown in

Figure 9 The server pumps the video stream into the firstrouter followed by a bottleneck link and the second routerFinally the disturbed video stream arrives at the receiverAll link capacities in Figure 9 are set to be 10Mbps ThePareto ON-OFF cross-traffic is injected into the bottlenecklink using theDistributed Internet TrafficGenerator (D-ITG)tool [26] Therefore the video stream is disturbed by thecross-traffic in the bottleneck link and thus packet losses andjitters may occur The video is encoded into the MPEG-2 TSat the streaming server and the RTPUDP streaming protocolis employed Table 2 lists the detailed information of the firsttest video and Table 3 lists the parameters of AMP The otherparameters not mentioned here are set the same as those in[22]

First the QoEs of two media playout systems with andwithout AMP are compared The video server pumps amulticast stream into the network and two independentclients simultaneously receive the same multicast stream atthe remote side One client uses the media player with AMPwhile the other one adopts the media player without AMPThe observed QoEs of these two systems with and withoutAMP under different cross-traffic rates 0 4 and 8Mbps aregiven in Figure 10 When the cross-traffic rate is not largerthan 4Mbps the overall QoE is almost over 4 no matterwhether the AMP is enabled or not This is because thejitter burstiness and losses of video packets are very smallunder a low cross-traffic rate Additionally under a low cross-traffic rate the observed QoE on the media player withoutAMP remains very stable while the QoE on the media playerwith AMP slightly oscillates in the first minute The reasonis explained as follows The media player without AMP canstart playback only when the buffer fullness exceeds 119871 = 100yielding the initial playout delay about 4 s and always playsthe video with the normal playout rate However the mediaplayer with AMP can start playback earlier but with a lowerinitial playout rate Since the initial playout rate of the mediaplayer with AMP is smaller than the normal playout ratethe observed QoE on the media player with AMP is slightlydegraded in the first minute as shown in Figure 10 Table 4shows the measured initial playout delays QoSd of the mediaplayers with and without AMP Because the initial playoutdelay in the media player with AMP is much smaller thanthat without AMP the corresponding QoE of QoSd of themedia player with AMP becomes better When the cross-traffic rate becomes 8Mbps the video stream is seriouslydisturbed by the cross-traffic so that the observed QoE at theclient becomes unacceptable (MOS lt 3) The serious QoEdegradation is mainly due to the increase of the number ofunderflow events and underflow timeHowever the observedQoE on the media player with AMP is still much better thanthat without AMP as shown in Figure 10

The corresponding buffer fullness and the playout rateat the client under the cross-traffic rate 6Mbps are shownin Figure 11 According to Figure 11 the buffer fullness ofthe system with AMP is more stable than that of the systemwithout AMP For the media player with AMP the underflowevents never happen because the buffer fullness never dropsto an unacceptable low level after the playback Althoughthe playout rate of the media player with AMP varies the

International Journal of Digital Multimedia Broadcasting 7

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5M

OS

Cross-traffic rate = 0 MbpsPlayer with AMPPlayer without AMP

Cross-traffic rate = 4 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Cross-traffic rate = 8 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Figure 10 Comparison of synthetic MOSsQoEs between the systems with and without AMP under various cross-traffic rates

Player with AMPPlayer without AMP

0

25

50

75

100

125

150

175

200

Buffe

r ful

lnes

s (fr

ames

)

20 40 60 80 1000Time (sec)

Player with AMPPlayer without AMP

20 40 60 80 1000Time (sec)

0

02

04

06

08

1

12

Nor

mal

ized

pla

yout

rate

Figure 11 Comparison of buffer fullness and normalized playout rates between systems with and without AMP (cross-traffic rate = 6Mbps)

8 International Journal of Digital Multimedia Broadcasting

Table 3 Parameters setting in AMP

Parameter 1198710 119871 119867 119873 119888 119903 1199031 1199032Value 30 100 200 500 04 015 03 03

Table 4 Initial playout delay QoSd and corresponding QoE

Cross-traffic rate Player with AMP Player without AMP(Mbps) QoSdQoE QoSdQoE0 1124 sec4776 4001 sec42334 1656 sec4667 4250 sec41898 3876 sec4255 4756 sec4102

deviation of the playout rate is never over 25 so that theQoEdegradation resulting from it is still acceptable However forthe media player without AMP the buffer fullness duringthe time interval (10 s 66 s) decreases almost linearly to zeroand leads to an underflow event yielding a significant QoEdegradation All results in Figures 10 and 11 demonstratethat the AMP significantly improves the QoE of streamingservices

43 Performance Monitoring for Wireless Streaming Videostreaming over wireless networks has become very popularrecently However it is still challenging to offer stream-ing services over wireless networks To clearly identify thechallenging issues of wireless streaming services in thissubsection the AMP function is not enabled in the mediaplayers of all receivers To evaluate the QoE of wirelessstreaming services the second network scenario shown inFigure 12 is considered and the first test video given in Table 2is used The video stream is multicast to two independentreceivers at the last hop via the IEEE 80211n wireless link toReceiver 1 and via the wired link to Receiver 2 Receiver 1 isaway from theWiFi AP by 2m and no obstacle between themFigure 13 shows the observed synthetic MOSs at Receivers1 and 2 under different cross-traffic rates using the QoEmonitor program developed in this paper By Figure 13 theQoE at Receiver 2 using a wired link is better than that atReceiver 1 using a wireless link Even though no cross-trafficexists the MOS of the video streaming service at Receiver1 via the wireless link is less than 4 According to the QoSparameters measured at Receivers 1 and 2 the difference ofQoE between them is mainly due to a higher packet loss ratein the wireless link

Subsequently to compare the performance of multicastand unicast the network scenario 3 shown in Figure 14 isconsidered The second video clip with the resolution 720times 480 p and bit rate 1936 kbps is used as shown in Table 5Different numbers of similar unicast streams are pumpedfrom the video server to the receivers using a wireless linkat the last hopThe synthetic MOS of the streaming service atthe receivers decreases significantly as the number of unicaststreams increases as shown in Figure 15 This is because thejitter burstiness and packet loss rate increase as the numberof unicast streams in the network grows yielding the QoEdegradation at the receivers Note that if themulticast scheme

Table 5 Information of test video 2

Parameter ValueVideo title Call Me Maybe (MV)Length 199 secResolution 720 times 480 pixelsEncoding scheme MPEG-2 TSBit rate 1936 kbpsFrame rate 30 fpsGOP size 15Content size 46MB

is used only one video stream is required to be pumped intothe network no matter how many receivers exist That is theQoE of video streaming services is irrelevant to the numberof receivers if the multicast scheme is employed Obviouslyunder the scenario that multiple users watch the same videostream the multicast scheme performs much better than theunicast one in terms of QoE

Finally to investigate the impact of transmission distanceand obstacles on the QoE of wireless streaming services thewireless network scenario 4 plotted in Figure 16 is consideredand the second test video given in Table 5 is used The videostream is multicast to several wireless receivers at differentpositions in our labs No cross-traffic is injected into thenetwork No obstacles exist between the wireless AP andthe receivers at Locations 1 and 2 However the receiver atLocation 1 is much closer to the AP than the receiver atLocation 2 isThere are cupboards and a wall between the APand the receiver at Location 3 The receiver at Location 4 isoutside the door of our lab and is isolated from the AP by awallThe other receiver at Location 5 is also isolated from theAP by a wall and cupboards All receivers receive the samemulticast stream simultaneously and their individual MOSsare estimated using the developed QoEmonitoring programThe results are shown in Figure 17 where the MOSs of thereceivers located at Locations 1 and 2 are the best becauseno obstacle blocks the wireless signals for them while theMOSs of the receivers located at Locations 4 and 5 are theworst because there exist cupboards and a wall between themand the AP The MOS of the receiver located at Location 3is better than those located at Locations 4 and 5 because thereceiver at Location 3 is much closer to the AP than thoseat Locations 4 and 5 are Accordingly the distance and thenumber of obstacles between the WiFi AP and the receiversstrongly affect the QoE of wireless streaming services

5 Conclusions

In this work anAMP and a real-timeQoEmonitoring systemfor video streaming services arerealized The AMP design is

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 2: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

2 International Journal of Digital Multimedia Broadcasting

protocol agent Although themonitoring service can improvethe user experience the presented QoE factors are still notgood enough In [18] the proposed concept Monitoringof Audio Visual Quality by Key Performance Indicators(MOAVI) is able to isolate and focus investigation set upalgorithms increase the monitoring period and guaranteebetter prediction of perceptual quality In [19] Ickin et aladded functionality to the VideoLAN Client (VLC) mediaplayer [20] to record a set of metrics from the user interfaceapplication-level network-level and the available sensors ofthe device Based on these metrics the proposed VLQoE aQoE instrumentation for video streaming can infer the QoEof streaming services on a smartphone

To improve the playout quality of video streamingservices Adaptive Media Playout (AMP) schemes [21 22]have been used In this work an AMP playout system isimplemented into the VLC media player which is a freeand open source cross-platformmultimedia player Since theplayout rate of the AMP system can be dynamically adjustedduring video playback the effects of AMP on QoE mustbe considered as well However almost all the developedQoE monitoring tools in the literature [15ndash19] did not takethe effects of AMP on QoE into account Thus in [23] weproposed the QoE monitoring system that takes the effectsof AMP on QoE into consideration Additionally the mostimportant feature of our proposed QoE monitoring systemis that it can derive an overall QoE for several QoS metricsusing the product form presented in [5] Our previous work[5] has shown that the product form performs much betterthan the conventional averaging techniques in deriving anoverall QoE of multiple QoS metrics Using the developedQoE monitoring program network providers can identifynetwork troubles and fix them in real time to improve thesystem performance and qualities

The novelties and contributions of this paper are summa-rized as follows First the implemented QoEmonitoring sys-tem takes the effects of AMP onQoE into consideration Sec-ond the developedQoEmonitoring systemderives an overallQoE from several QoS metrics such as the initial playoutdelay packet loss rate underflow time ratio and normalizedplayout rate Thirdly we implement the AMP scheme whichwas proposed in [22] into the VLC media player Finally theGroup of Pictures- (GOP-) based cumulative average jitter 119869119899which is used to determine the playback threshold 119875119899 of theproposed AMP is proposed to eliminate the effect of diverseframe sizes on the estimation of network jitter

The rest of this paper is organized as follows Sec-tion 2 describes the implementation of the AMP playoutsystem Implementation of the QoE monitoring system isthen introduced in Section 3 Section 4 conducts severalexperiments and evaluates the QoE performance of videostreaming services using the developed programs Finally theconcluding remarks are given in Section 5

2 Implementation of AMP

This section first describes the principles of AMP inSection 21 Then the implementation schemes available

programming tools and some challenging issues on mea-surements or estimations of key parameters are addressed inSection 22

21 Principles of Proposed AMP In this work we employthe freeware VLC media player to realize the AMP TheAMP is implemented according to our proposed scheme [22]whose principle is described briefly as follows In addition tothe conventional slowdown threshold 119871 a dynamic playbackthreshold 119875119899 and a speedup threshold 119867 are designed Ifthe buffer fullness is below 119871 the playout rate is reduced toavoid buffer underflow When the buffer fullness exceeds thespeedup threshold119867 the playout rate is increased to reducethe buffer fullness for avoiding buffer overflowThe DynamicPlaybackThreshold Algorithm (DPTA) in [22] is designed todynamically adjust the playback threshold 119875119899 which affectsthe initial playout delay under various network conditionsWhenever the current buffer fullness 119899 surpasses or equals119875119899 the playback starts immediately with the proper playoutrate 120583(119894 119899) which is determined by the following equation

120583 (119894 119899) =

119877119871 (119899) if 119899 lt 119871119877S (119894) if 119871 le 119899 le 119867119877119867 (119899) if 119899 gt 119867

(1)

where 119877119878(119894) is a random process and is estimated by theArrival Process Tracking Algorithm (APTA) presented in[22] That is 119877119878(119894) depends on the frame arrival process atthe client buffer and is limited between (1 minus 119903)1205830 and (1 +119903)1205830 as shown in Figure 1 Anyone interested in the detailedderivation of 119877119878(119894) and 119875119899 can refer to [22] The quadraticplayout rate functions 119877119867(119899) and 119877119871(119899) in [22] are definedas follows

119877119867 (119899) = (1 + 1199032) 1205830 minus (119873 minusmin 119899119873119873 minus 119867 )

2

(1199032 minus 119903) 1205830

119877119871 (119899) = (1 minus 1199031) 1205830 + (119899119871)2

(1199031 minus 119903) 1205830(2)

where 119873 is the uppermost threshold correlated to the buffersize at the client Notably the playout rate must be limitedsuch that rate variations are unnoticeable or acceptable byusersThe perception of a slowdown video is usually differentfrom that of a speedup video for users Therefore twodifferent restricted deviation ratios denoted by 1199031 and 1199032 for119877119871(119899) and 119877119867(119899) respectively are set for playout rates Thecorresponding playout rate function of the proposed AMPis given by Figure 1 Numerical results in [22] have shownthat the proposedAMPwith quadratic playout rates performsmuch better than several conventional playout systems withlinear playout rates and can adapt to different network loadconditions for reducing the initial playout delay

22 Realization of AMP VLC media player provides auseful VLC ActiveX package whose architecture is given inFigure 2 The main component of VLC ActiveX package isLibVLC which supports several properties such as input

International Journal of Digital Multimedia Broadcasting 3

(i n)

(1 + r2)0

(1 + r)0

(1 minus r)0

(1 minus r1)0

Warningzone

Warningzone

Safetyzone

0 L H N n

Figure 1 Playout rate function of the proposed AMP [22]

VLC user interface

MainLib VLC

Videooutput

Audiooutput

Decoder

Access

InputDemux

Figure 2 Architecture of VLC ActiveX package

audio output and video output The input property collectsall related parameters of the received video stream fromthe Network Interface Card (NIC) (Access) the networkdemultiplexer (Demux) and the video decoder (Decoder)as shown in Figure 2 Several useful statistical parametersin inputitem()stats() are listed in Table 1 Based on thesecollected parameters from the input property one can set andcontrol the playout rate of the VLCmedia player to realize theAMP function

According to the descriptions in Section 21 the playoutrate mainly depends on the buffer fullness 119899 (in frames) Toaccurately measure the number demux frames of demulti-plexed video frames one can directly analyze the packetsarriving at the NIC card By reading the ldquopayload unit startindicatorrdquo bit in the header of each 188-byte MPEG-2 TS(Transport Stream) packet as shown in Figure 3 one caneasily calculate the number demux frames of video framesreceivedWhen the ldquopayload unit start indicatorrdquo bit equals 1

Table 1 Receiving parameters in VLC inputitem()stats()

Parameters Meaningdemux read packets Number of packets receiveddemux read bytes Number of bytes receiveddemux bitrate Bit rate of the video streamaverage demux bitrate Average bit rate of the video streamdemux corrupted Number of corrupted framesdecoded audio Number of audio frames decodeddecoded video Number of video frames decodedlost pictures Number of frames lost

a new MPEG-2 TS video frame is arriving Hence the bufferfullness 119899 can be computed by the following equation

119899 = 119889119890119898119906119909 119891119903119886119898119890119904 minus 119889119890119888119900119889119890119889 V119894119889119890119900 (3)

After deriving the buffer fullness 119899 the playout rate of theAMP media player can be dynamically adjusted accordingto (1) The command to dynamically set the playout rate ofthe VLCmedia player is defined by the function varset(inputldquoraterdquo 120583(119894 119899)) Although the currently developed program isdesigned for the MPEG-2 TS format there is no difficulty toinclude other advanced video encoding technologies such asthe H264AVC and H265HEVC [24] in the future

The dynamic playback threshold 119875119899 [22] is determined by

119875119899 = 1198710 + (119871 minus 1198710) sdotmin 1 119869119899119888119879 (4)

where 1198710 and 119888 are design parameters and 119879 = 11205830 is theaverage frame interarrival time 119869119899 is the cumulative averagejitter and is defined by

119869119899 = (1 minus 1119899 minus 1) 119869119899minus1 +

1003816100381610038161003816119883119899minus1 minus 1198791003816100381610038161003816119899 minus 1 for 119899 gt 1 (5)

where 119883119899minus1 is the frame interarrival time between the(119899 minus 1)-st and the 119899th frames However video frames arecategorized into I B and P frames and different types offrames usually have very different frame sizes For examplethe frame size of an I frame is usually much larger thanthat of a B or P frame Hence the frame interarrival timebetween two successive I and B (or P) frames may becomerelatively large because of the long transmission time of anI frame compared with that between two successive B and Pframes Such variance of frame interarrival timesmaybe is notdue to the network jitter but the diversity of frame sizes Toreduce the impact of frame size variation on the estimation ofnetwork jitter 119869119899 and estimate 119869119899more accurately in this workthe average frame interarrival time within a GOP is usedHence the variable 119883119899minus1 in (5) is replaced by 119884119896minus1119866 where119896 = lfloor119899119866rfloor 119866 is the number of frames in a GOP and 119884119896minus1 isthe interarrival time between the I frames of the (119896minus1)-st andthe 119896th GOPs

3 Implementation of QoE Monitoring System

The procedures to implement the QoE monitoring systemare described as follows First we propose an architecture

4 International Journal of Digital Multimedia Broadcasting

188 bytes

Transportpacketstream

Header Header HeaderPayload Payload Payload

Syncbyte

Transporterror

indicator

Payloadunit startindicator

Transportpriority PID

Transportscrambling

control

Adaptationfield

control

Continuitycounter

Adaptationfield

8 1 1 1 13 2 2 4

Figure 3 MPEG-2 TS packet format

QoE monitoring system

QoE evaluation

QoEQoSreporting

NIC card AMP media player

QoS measurement(i) Underflow time ratio

(ii) Packet loss rate(iii) Initial playout delay(iv) Normalized playout rate

Figure 4 Architecture of the QoE monitoring system

for the QoE monitoring system Next several network-basedQoS metrics are defined and their measurement methods arepresented Finally the overall QoE evaluation method for theQoE monitoring system is introduced

31 Architecture of QoE Monitoring System The architec-ture of our proposed QoE monitoring system is plotted inFigure 4 It includes the QoS measurement engine QoEevaluation function and performance reporting windowThe QoS measurement engine is designed to collect theQoS metrics from the media player and the NIC cardThese obtained QoS metrics are used by the QoE evaluationfunction for producing the overall QoE of video streamingservices All instantaneous QoS metrics and QoE results arereported in real time and the curve of overall QoE versus timeis also displayed with a chart in the Graphic User Interface(GUI) for the monitoring purpose

32 Measurements of QoS Parameters In the proposed QoEmonitoring system the considered QoS metrics includeunderflow time ratio QoSu packet loss rate QoSl initial

Figure 5 Buffering mechanism in VLC player

playout delay QoSd and normalized playout rate QoSr Themeasurements of these parameters are described as follows

(a) Underflow time ratio QoSu whenever an underflowevent occurs the underflow time starts to accumulateuntil the playback restarts again At the same timethe number of underflow events is increased by 1Theunderflow time ratio equals the ratio of accumulatedunderflow time to the total playback time durationThe accumulated underflow time can be computedaccording to the value of Status() provided by VLCLua extension program as shown in Figure 5 Whenthe value of Status() changes into 2 the underflowtimer turns ON until the value of Status() becomes3The sojourn time at the state with Status() = 2 is theunderflow timeWhenever the underflow timer turnsOFF the accumulated underflow time is updated

(b) Packet loss rate QoSl the packet loss rate is obtainedby analyzing the number of receiving packets at the

International Journal of Digital Multimedia Broadcasting 5

NIC card or using the network protocol analyzer suchas Wireshark

(c) Initial playout delay QoSd initial playout delaystrongly depends on the buffering time set in VLCmedia player However in our implemented AMPthis parameter is directly determined by the DPTAalgorithm proposed in [22]

(d) Normalized playout rate QoSr whenever the playoutrate is adjusted the normalized playout rate is com-puted and saved However within each minute onlythe maximum and minimum normalized playoutrates are used in our design [5] That is the corre-sponding QoE of QoSr during every minute equalsthemean of these two individual QoEs resulting fromthe maximum and minimum normalized playoutrates

33 Overall QoE Evaluation Function The overall QoE eval-uation function adopts the integratedmultivariate QoE func-tion 119891119868(QoSuQoSlQoSdQoSr) and the individual QoS-QoE mapping functions 119891u(119909) 119891l(119909) 119891d(119909) and 119891r(119909) pro-posed in [5] to derive the overall QoE Based on the results of[5] they are given as follows

119891119868 (QoSuQoSlQoSdQoSr)

= 5 sdot 119891u (QoSu)5 sdot 119891l (QoSl)

5 sdot 119891d (QoSd)5

sdot 119891r (QoSr)5

(6)

119891u (119909) = 5119890minus571119909 (7)

119891l (119909) = 5119890minus1607119909 (8)

119891d (119909) = 5119890minus00416119909 (9)

119891r (119909) = 5119909894119890minus894(119909minus1) (10)

The scheme used in [5] consists of three steps to bedone beforehand First several Key Performance Indicators(KPIs) or QoS metrics such as the initial playout delaypacket loss rate underflow time ratio and playout ratewere selected Then several distorted videos were generatedbased on these selected QoS metrics Second subjective testswere conducted using human observers to rate the distortedvideos in terms of Mean Opinion Score (MOS) [25] TheACR subjective test method was adopted in [5] Howeverthe length of each distorted video clip is about 64 s ratherthan 10 s Each distorted video clip was viewed and ratedby no less than 30 viewers The title of the used video clipwhich is with the frame rate 30 fps resolution 1920 times 1080 pand video codec MPEG-2 is ldquoAvatarrdquo Thirdly using theregression approach the QoS-QoE mapping functions forindividual QoS metrics were obtained Finally the productform of individual QoS-QoE mapping functions [5 6] isused to derive the overall QoE of a video streaming serviceNotably although only four QoS metrics are considered in

Figure 6 VLC media player and Lua program

Figure 7 QoEMOS monitoring program

(6) the number of QoSmetrics can be arbitrarily extended asneeded

4 Experimental Results

In this section we first briefly introduce the GUIs andfunctions of developed AMP andQoEmonitoring programsSubsequently using the developedQoEmonitoring programseveral experiments are conducted to demonstrate the perfor-mance of AMP Finally we use the QoE monitoring programto identify some challenging issues of wireless streaming

41 QoE Monitoring System The developed AMP is basedon the VLC media player and its add-ons such as the Luaextension Figure 6 shows the VLC media player and itsLua extension program One can directly set the multicastIP of the streaming source and activate the AMP functionon the VLC Lua extension program Although in this paperthe AMP is implemented into the VLC media player thereis no difficulty to implement it into other popular mediaplayers such as the MS Windows Media Player or YouTubePlayer The QoE monitoring program which is developedusing the C++Builder and is shown in Figure 7 evaluates anddisplays the overall QoE The measured QoS and syntheticQoEMOS are displayed in real time on the monitoringwindow as shown in Figure 7 The corresponding QoEsof individual QoSs are displayed on the left side of themonitoringwindowwhile the curve of the overall QoE versustime is displayed on the right side of the monitoring windowas shown in Figure 7 By clicking the ldquoToolsrdquo on themenu barof the QoE monitoring program other detailed QoS metrics

6 International Journal of Digital Multimedia Broadcasting

Figure 8 Monitoring tools for video packets at the client

Figure 9 Experimental network scenario 1

Table 2 Information of test video 1

Parameter ValueVideo title The Big BangTheoryLength 593 secResolution 352 times 288 pixelsEncoding scheme MPEG-2 TSBit rate 1406 kbpsFrame rate 25 fpsGOP size 15Content size 995MB

such as the interarrival time of video frames normalizedplayout rate number of packet losses buffer fullness andthe jitter observed at the client can be displayed as shownin Figure 8 By clicking the interested QoS parameter suchas the ldquoMoveAvg Interarrivalrdquo the curve of the QoS versustime is immediately displayed on the monitoring window ofFigure 8

42 Performance Evaluation of AMP In order to demon-strate our AMP design and QoE monitoring system the firstexperimental network scenario is constructed as shown in

Figure 9 The server pumps the video stream into the firstrouter followed by a bottleneck link and the second routerFinally the disturbed video stream arrives at the receiverAll link capacities in Figure 9 are set to be 10Mbps ThePareto ON-OFF cross-traffic is injected into the bottlenecklink using theDistributed Internet TrafficGenerator (D-ITG)tool [26] Therefore the video stream is disturbed by thecross-traffic in the bottleneck link and thus packet losses andjitters may occur The video is encoded into the MPEG-2 TSat the streaming server and the RTPUDP streaming protocolis employed Table 2 lists the detailed information of the firsttest video and Table 3 lists the parameters of AMP The otherparameters not mentioned here are set the same as those in[22]

First the QoEs of two media playout systems with andwithout AMP are compared The video server pumps amulticast stream into the network and two independentclients simultaneously receive the same multicast stream atthe remote side One client uses the media player with AMPwhile the other one adopts the media player without AMPThe observed QoEs of these two systems with and withoutAMP under different cross-traffic rates 0 4 and 8Mbps aregiven in Figure 10 When the cross-traffic rate is not largerthan 4Mbps the overall QoE is almost over 4 no matterwhether the AMP is enabled or not This is because thejitter burstiness and losses of video packets are very smallunder a low cross-traffic rate Additionally under a low cross-traffic rate the observed QoE on the media player withoutAMP remains very stable while the QoE on the media playerwith AMP slightly oscillates in the first minute The reasonis explained as follows The media player without AMP canstart playback only when the buffer fullness exceeds 119871 = 100yielding the initial playout delay about 4 s and always playsthe video with the normal playout rate However the mediaplayer with AMP can start playback earlier but with a lowerinitial playout rate Since the initial playout rate of the mediaplayer with AMP is smaller than the normal playout ratethe observed QoE on the media player with AMP is slightlydegraded in the first minute as shown in Figure 10 Table 4shows the measured initial playout delays QoSd of the mediaplayers with and without AMP Because the initial playoutdelay in the media player with AMP is much smaller thanthat without AMP the corresponding QoE of QoSd of themedia player with AMP becomes better When the cross-traffic rate becomes 8Mbps the video stream is seriouslydisturbed by the cross-traffic so that the observed QoE at theclient becomes unacceptable (MOS lt 3) The serious QoEdegradation is mainly due to the increase of the number ofunderflow events and underflow timeHowever the observedQoE on the media player with AMP is still much better thanthat without AMP as shown in Figure 10

The corresponding buffer fullness and the playout rateat the client under the cross-traffic rate 6Mbps are shownin Figure 11 According to Figure 11 the buffer fullness ofthe system with AMP is more stable than that of the systemwithout AMP For the media player with AMP the underflowevents never happen because the buffer fullness never dropsto an unacceptable low level after the playback Althoughthe playout rate of the media player with AMP varies the

International Journal of Digital Multimedia Broadcasting 7

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5M

OS

Cross-traffic rate = 0 MbpsPlayer with AMPPlayer without AMP

Cross-traffic rate = 4 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Cross-traffic rate = 8 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Figure 10 Comparison of synthetic MOSsQoEs between the systems with and without AMP under various cross-traffic rates

Player with AMPPlayer without AMP

0

25

50

75

100

125

150

175

200

Buffe

r ful

lnes

s (fr

ames

)

20 40 60 80 1000Time (sec)

Player with AMPPlayer without AMP

20 40 60 80 1000Time (sec)

0

02

04

06

08

1

12

Nor

mal

ized

pla

yout

rate

Figure 11 Comparison of buffer fullness and normalized playout rates between systems with and without AMP (cross-traffic rate = 6Mbps)

8 International Journal of Digital Multimedia Broadcasting

Table 3 Parameters setting in AMP

Parameter 1198710 119871 119867 119873 119888 119903 1199031 1199032Value 30 100 200 500 04 015 03 03

Table 4 Initial playout delay QoSd and corresponding QoE

Cross-traffic rate Player with AMP Player without AMP(Mbps) QoSdQoE QoSdQoE0 1124 sec4776 4001 sec42334 1656 sec4667 4250 sec41898 3876 sec4255 4756 sec4102

deviation of the playout rate is never over 25 so that theQoEdegradation resulting from it is still acceptable However forthe media player without AMP the buffer fullness duringthe time interval (10 s 66 s) decreases almost linearly to zeroand leads to an underflow event yielding a significant QoEdegradation All results in Figures 10 and 11 demonstratethat the AMP significantly improves the QoE of streamingservices

43 Performance Monitoring for Wireless Streaming Videostreaming over wireless networks has become very popularrecently However it is still challenging to offer stream-ing services over wireless networks To clearly identify thechallenging issues of wireless streaming services in thissubsection the AMP function is not enabled in the mediaplayers of all receivers To evaluate the QoE of wirelessstreaming services the second network scenario shown inFigure 12 is considered and the first test video given in Table 2is used The video stream is multicast to two independentreceivers at the last hop via the IEEE 80211n wireless link toReceiver 1 and via the wired link to Receiver 2 Receiver 1 isaway from theWiFi AP by 2m and no obstacle between themFigure 13 shows the observed synthetic MOSs at Receivers1 and 2 under different cross-traffic rates using the QoEmonitor program developed in this paper By Figure 13 theQoE at Receiver 2 using a wired link is better than that atReceiver 1 using a wireless link Even though no cross-trafficexists the MOS of the video streaming service at Receiver1 via the wireless link is less than 4 According to the QoSparameters measured at Receivers 1 and 2 the difference ofQoE between them is mainly due to a higher packet loss ratein the wireless link

Subsequently to compare the performance of multicastand unicast the network scenario 3 shown in Figure 14 isconsidered The second video clip with the resolution 720times 480 p and bit rate 1936 kbps is used as shown in Table 5Different numbers of similar unicast streams are pumpedfrom the video server to the receivers using a wireless linkat the last hopThe synthetic MOS of the streaming service atthe receivers decreases significantly as the number of unicaststreams increases as shown in Figure 15 This is because thejitter burstiness and packet loss rate increase as the numberof unicast streams in the network grows yielding the QoEdegradation at the receivers Note that if themulticast scheme

Table 5 Information of test video 2

Parameter ValueVideo title Call Me Maybe (MV)Length 199 secResolution 720 times 480 pixelsEncoding scheme MPEG-2 TSBit rate 1936 kbpsFrame rate 30 fpsGOP size 15Content size 46MB

is used only one video stream is required to be pumped intothe network no matter how many receivers exist That is theQoE of video streaming services is irrelevant to the numberof receivers if the multicast scheme is employed Obviouslyunder the scenario that multiple users watch the same videostream the multicast scheme performs much better than theunicast one in terms of QoE

Finally to investigate the impact of transmission distanceand obstacles on the QoE of wireless streaming services thewireless network scenario 4 plotted in Figure 16 is consideredand the second test video given in Table 5 is used The videostream is multicast to several wireless receivers at differentpositions in our labs No cross-traffic is injected into thenetwork No obstacles exist between the wireless AP andthe receivers at Locations 1 and 2 However the receiver atLocation 1 is much closer to the AP than the receiver atLocation 2 isThere are cupboards and a wall between the APand the receiver at Location 3 The receiver at Location 4 isoutside the door of our lab and is isolated from the AP by awallThe other receiver at Location 5 is also isolated from theAP by a wall and cupboards All receivers receive the samemulticast stream simultaneously and their individual MOSsare estimated using the developed QoEmonitoring programThe results are shown in Figure 17 where the MOSs of thereceivers located at Locations 1 and 2 are the best becauseno obstacle blocks the wireless signals for them while theMOSs of the receivers located at Locations 4 and 5 are theworst because there exist cupboards and a wall between themand the AP The MOS of the receiver located at Location 3is better than those located at Locations 4 and 5 because thereceiver at Location 3 is much closer to the AP than thoseat Locations 4 and 5 are Accordingly the distance and thenumber of obstacles between the WiFi AP and the receiversstrongly affect the QoE of wireless streaming services

5 Conclusions

In this work anAMP and a real-timeQoEmonitoring systemfor video streaming services arerealized The AMP design is

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 3: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

International Journal of Digital Multimedia Broadcasting 3

(i n)

(1 + r2)0

(1 + r)0

(1 minus r)0

(1 minus r1)0

Warningzone

Warningzone

Safetyzone

0 L H N n

Figure 1 Playout rate function of the proposed AMP [22]

VLC user interface

MainLib VLC

Videooutput

Audiooutput

Decoder

Access

InputDemux

Figure 2 Architecture of VLC ActiveX package

audio output and video output The input property collectsall related parameters of the received video stream fromthe Network Interface Card (NIC) (Access) the networkdemultiplexer (Demux) and the video decoder (Decoder)as shown in Figure 2 Several useful statistical parametersin inputitem()stats() are listed in Table 1 Based on thesecollected parameters from the input property one can set andcontrol the playout rate of the VLCmedia player to realize theAMP function

According to the descriptions in Section 21 the playoutrate mainly depends on the buffer fullness 119899 (in frames) Toaccurately measure the number demux frames of demulti-plexed video frames one can directly analyze the packetsarriving at the NIC card By reading the ldquopayload unit startindicatorrdquo bit in the header of each 188-byte MPEG-2 TS(Transport Stream) packet as shown in Figure 3 one caneasily calculate the number demux frames of video framesreceivedWhen the ldquopayload unit start indicatorrdquo bit equals 1

Table 1 Receiving parameters in VLC inputitem()stats()

Parameters Meaningdemux read packets Number of packets receiveddemux read bytes Number of bytes receiveddemux bitrate Bit rate of the video streamaverage demux bitrate Average bit rate of the video streamdemux corrupted Number of corrupted framesdecoded audio Number of audio frames decodeddecoded video Number of video frames decodedlost pictures Number of frames lost

a new MPEG-2 TS video frame is arriving Hence the bufferfullness 119899 can be computed by the following equation

119899 = 119889119890119898119906119909 119891119903119886119898119890119904 minus 119889119890119888119900119889119890119889 V119894119889119890119900 (3)

After deriving the buffer fullness 119899 the playout rate of theAMP media player can be dynamically adjusted accordingto (1) The command to dynamically set the playout rate ofthe VLCmedia player is defined by the function varset(inputldquoraterdquo 120583(119894 119899)) Although the currently developed program isdesigned for the MPEG-2 TS format there is no difficulty toinclude other advanced video encoding technologies such asthe H264AVC and H265HEVC [24] in the future

The dynamic playback threshold 119875119899 [22] is determined by

119875119899 = 1198710 + (119871 minus 1198710) sdotmin 1 119869119899119888119879 (4)

where 1198710 and 119888 are design parameters and 119879 = 11205830 is theaverage frame interarrival time 119869119899 is the cumulative averagejitter and is defined by

119869119899 = (1 minus 1119899 minus 1) 119869119899minus1 +

1003816100381610038161003816119883119899minus1 minus 1198791003816100381610038161003816119899 minus 1 for 119899 gt 1 (5)

where 119883119899minus1 is the frame interarrival time between the(119899 minus 1)-st and the 119899th frames However video frames arecategorized into I B and P frames and different types offrames usually have very different frame sizes For examplethe frame size of an I frame is usually much larger thanthat of a B or P frame Hence the frame interarrival timebetween two successive I and B (or P) frames may becomerelatively large because of the long transmission time of anI frame compared with that between two successive B and Pframes Such variance of frame interarrival timesmaybe is notdue to the network jitter but the diversity of frame sizes Toreduce the impact of frame size variation on the estimation ofnetwork jitter 119869119899 and estimate 119869119899more accurately in this workthe average frame interarrival time within a GOP is usedHence the variable 119883119899minus1 in (5) is replaced by 119884119896minus1119866 where119896 = lfloor119899119866rfloor 119866 is the number of frames in a GOP and 119884119896minus1 isthe interarrival time between the I frames of the (119896minus1)-st andthe 119896th GOPs

3 Implementation of QoE Monitoring System

The procedures to implement the QoE monitoring systemare described as follows First we propose an architecture

4 International Journal of Digital Multimedia Broadcasting

188 bytes

Transportpacketstream

Header Header HeaderPayload Payload Payload

Syncbyte

Transporterror

indicator

Payloadunit startindicator

Transportpriority PID

Transportscrambling

control

Adaptationfield

control

Continuitycounter

Adaptationfield

8 1 1 1 13 2 2 4

Figure 3 MPEG-2 TS packet format

QoE monitoring system

QoE evaluation

QoEQoSreporting

NIC card AMP media player

QoS measurement(i) Underflow time ratio

(ii) Packet loss rate(iii) Initial playout delay(iv) Normalized playout rate

Figure 4 Architecture of the QoE monitoring system

for the QoE monitoring system Next several network-basedQoS metrics are defined and their measurement methods arepresented Finally the overall QoE evaluation method for theQoE monitoring system is introduced

31 Architecture of QoE Monitoring System The architec-ture of our proposed QoE monitoring system is plotted inFigure 4 It includes the QoS measurement engine QoEevaluation function and performance reporting windowThe QoS measurement engine is designed to collect theQoS metrics from the media player and the NIC cardThese obtained QoS metrics are used by the QoE evaluationfunction for producing the overall QoE of video streamingservices All instantaneous QoS metrics and QoE results arereported in real time and the curve of overall QoE versus timeis also displayed with a chart in the Graphic User Interface(GUI) for the monitoring purpose

32 Measurements of QoS Parameters In the proposed QoEmonitoring system the considered QoS metrics includeunderflow time ratio QoSu packet loss rate QoSl initial

Figure 5 Buffering mechanism in VLC player

playout delay QoSd and normalized playout rate QoSr Themeasurements of these parameters are described as follows

(a) Underflow time ratio QoSu whenever an underflowevent occurs the underflow time starts to accumulateuntil the playback restarts again At the same timethe number of underflow events is increased by 1Theunderflow time ratio equals the ratio of accumulatedunderflow time to the total playback time durationThe accumulated underflow time can be computedaccording to the value of Status() provided by VLCLua extension program as shown in Figure 5 Whenthe value of Status() changes into 2 the underflowtimer turns ON until the value of Status() becomes3The sojourn time at the state with Status() = 2 is theunderflow timeWhenever the underflow timer turnsOFF the accumulated underflow time is updated

(b) Packet loss rate QoSl the packet loss rate is obtainedby analyzing the number of receiving packets at the

International Journal of Digital Multimedia Broadcasting 5

NIC card or using the network protocol analyzer suchas Wireshark

(c) Initial playout delay QoSd initial playout delaystrongly depends on the buffering time set in VLCmedia player However in our implemented AMPthis parameter is directly determined by the DPTAalgorithm proposed in [22]

(d) Normalized playout rate QoSr whenever the playoutrate is adjusted the normalized playout rate is com-puted and saved However within each minute onlythe maximum and minimum normalized playoutrates are used in our design [5] That is the corre-sponding QoE of QoSr during every minute equalsthemean of these two individual QoEs resulting fromthe maximum and minimum normalized playoutrates

33 Overall QoE Evaluation Function The overall QoE eval-uation function adopts the integratedmultivariate QoE func-tion 119891119868(QoSuQoSlQoSdQoSr) and the individual QoS-QoE mapping functions 119891u(119909) 119891l(119909) 119891d(119909) and 119891r(119909) pro-posed in [5] to derive the overall QoE Based on the results of[5] they are given as follows

119891119868 (QoSuQoSlQoSdQoSr)

= 5 sdot 119891u (QoSu)5 sdot 119891l (QoSl)

5 sdot 119891d (QoSd)5

sdot 119891r (QoSr)5

(6)

119891u (119909) = 5119890minus571119909 (7)

119891l (119909) = 5119890minus1607119909 (8)

119891d (119909) = 5119890minus00416119909 (9)

119891r (119909) = 5119909894119890minus894(119909minus1) (10)

The scheme used in [5] consists of three steps to bedone beforehand First several Key Performance Indicators(KPIs) or QoS metrics such as the initial playout delaypacket loss rate underflow time ratio and playout ratewere selected Then several distorted videos were generatedbased on these selected QoS metrics Second subjective testswere conducted using human observers to rate the distortedvideos in terms of Mean Opinion Score (MOS) [25] TheACR subjective test method was adopted in [5] Howeverthe length of each distorted video clip is about 64 s ratherthan 10 s Each distorted video clip was viewed and ratedby no less than 30 viewers The title of the used video clipwhich is with the frame rate 30 fps resolution 1920 times 1080 pand video codec MPEG-2 is ldquoAvatarrdquo Thirdly using theregression approach the QoS-QoE mapping functions forindividual QoS metrics were obtained Finally the productform of individual QoS-QoE mapping functions [5 6] isused to derive the overall QoE of a video streaming serviceNotably although only four QoS metrics are considered in

Figure 6 VLC media player and Lua program

Figure 7 QoEMOS monitoring program

(6) the number of QoSmetrics can be arbitrarily extended asneeded

4 Experimental Results

In this section we first briefly introduce the GUIs andfunctions of developed AMP andQoEmonitoring programsSubsequently using the developedQoEmonitoring programseveral experiments are conducted to demonstrate the perfor-mance of AMP Finally we use the QoE monitoring programto identify some challenging issues of wireless streaming

41 QoE Monitoring System The developed AMP is basedon the VLC media player and its add-ons such as the Luaextension Figure 6 shows the VLC media player and itsLua extension program One can directly set the multicastIP of the streaming source and activate the AMP functionon the VLC Lua extension program Although in this paperthe AMP is implemented into the VLC media player thereis no difficulty to implement it into other popular mediaplayers such as the MS Windows Media Player or YouTubePlayer The QoE monitoring program which is developedusing the C++Builder and is shown in Figure 7 evaluates anddisplays the overall QoE The measured QoS and syntheticQoEMOS are displayed in real time on the monitoringwindow as shown in Figure 7 The corresponding QoEsof individual QoSs are displayed on the left side of themonitoringwindowwhile the curve of the overall QoE versustime is displayed on the right side of the monitoring windowas shown in Figure 7 By clicking the ldquoToolsrdquo on themenu barof the QoE monitoring program other detailed QoS metrics

6 International Journal of Digital Multimedia Broadcasting

Figure 8 Monitoring tools for video packets at the client

Figure 9 Experimental network scenario 1

Table 2 Information of test video 1

Parameter ValueVideo title The Big BangTheoryLength 593 secResolution 352 times 288 pixelsEncoding scheme MPEG-2 TSBit rate 1406 kbpsFrame rate 25 fpsGOP size 15Content size 995MB

such as the interarrival time of video frames normalizedplayout rate number of packet losses buffer fullness andthe jitter observed at the client can be displayed as shownin Figure 8 By clicking the interested QoS parameter suchas the ldquoMoveAvg Interarrivalrdquo the curve of the QoS versustime is immediately displayed on the monitoring window ofFigure 8

42 Performance Evaluation of AMP In order to demon-strate our AMP design and QoE monitoring system the firstexperimental network scenario is constructed as shown in

Figure 9 The server pumps the video stream into the firstrouter followed by a bottleneck link and the second routerFinally the disturbed video stream arrives at the receiverAll link capacities in Figure 9 are set to be 10Mbps ThePareto ON-OFF cross-traffic is injected into the bottlenecklink using theDistributed Internet TrafficGenerator (D-ITG)tool [26] Therefore the video stream is disturbed by thecross-traffic in the bottleneck link and thus packet losses andjitters may occur The video is encoded into the MPEG-2 TSat the streaming server and the RTPUDP streaming protocolis employed Table 2 lists the detailed information of the firsttest video and Table 3 lists the parameters of AMP The otherparameters not mentioned here are set the same as those in[22]

First the QoEs of two media playout systems with andwithout AMP are compared The video server pumps amulticast stream into the network and two independentclients simultaneously receive the same multicast stream atthe remote side One client uses the media player with AMPwhile the other one adopts the media player without AMPThe observed QoEs of these two systems with and withoutAMP under different cross-traffic rates 0 4 and 8Mbps aregiven in Figure 10 When the cross-traffic rate is not largerthan 4Mbps the overall QoE is almost over 4 no matterwhether the AMP is enabled or not This is because thejitter burstiness and losses of video packets are very smallunder a low cross-traffic rate Additionally under a low cross-traffic rate the observed QoE on the media player withoutAMP remains very stable while the QoE on the media playerwith AMP slightly oscillates in the first minute The reasonis explained as follows The media player without AMP canstart playback only when the buffer fullness exceeds 119871 = 100yielding the initial playout delay about 4 s and always playsthe video with the normal playout rate However the mediaplayer with AMP can start playback earlier but with a lowerinitial playout rate Since the initial playout rate of the mediaplayer with AMP is smaller than the normal playout ratethe observed QoE on the media player with AMP is slightlydegraded in the first minute as shown in Figure 10 Table 4shows the measured initial playout delays QoSd of the mediaplayers with and without AMP Because the initial playoutdelay in the media player with AMP is much smaller thanthat without AMP the corresponding QoE of QoSd of themedia player with AMP becomes better When the cross-traffic rate becomes 8Mbps the video stream is seriouslydisturbed by the cross-traffic so that the observed QoE at theclient becomes unacceptable (MOS lt 3) The serious QoEdegradation is mainly due to the increase of the number ofunderflow events and underflow timeHowever the observedQoE on the media player with AMP is still much better thanthat without AMP as shown in Figure 10

The corresponding buffer fullness and the playout rateat the client under the cross-traffic rate 6Mbps are shownin Figure 11 According to Figure 11 the buffer fullness ofthe system with AMP is more stable than that of the systemwithout AMP For the media player with AMP the underflowevents never happen because the buffer fullness never dropsto an unacceptable low level after the playback Althoughthe playout rate of the media player with AMP varies the

International Journal of Digital Multimedia Broadcasting 7

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5M

OS

Cross-traffic rate = 0 MbpsPlayer with AMPPlayer without AMP

Cross-traffic rate = 4 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Cross-traffic rate = 8 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Figure 10 Comparison of synthetic MOSsQoEs between the systems with and without AMP under various cross-traffic rates

Player with AMPPlayer without AMP

0

25

50

75

100

125

150

175

200

Buffe

r ful

lnes

s (fr

ames

)

20 40 60 80 1000Time (sec)

Player with AMPPlayer without AMP

20 40 60 80 1000Time (sec)

0

02

04

06

08

1

12

Nor

mal

ized

pla

yout

rate

Figure 11 Comparison of buffer fullness and normalized playout rates between systems with and without AMP (cross-traffic rate = 6Mbps)

8 International Journal of Digital Multimedia Broadcasting

Table 3 Parameters setting in AMP

Parameter 1198710 119871 119867 119873 119888 119903 1199031 1199032Value 30 100 200 500 04 015 03 03

Table 4 Initial playout delay QoSd and corresponding QoE

Cross-traffic rate Player with AMP Player without AMP(Mbps) QoSdQoE QoSdQoE0 1124 sec4776 4001 sec42334 1656 sec4667 4250 sec41898 3876 sec4255 4756 sec4102

deviation of the playout rate is never over 25 so that theQoEdegradation resulting from it is still acceptable However forthe media player without AMP the buffer fullness duringthe time interval (10 s 66 s) decreases almost linearly to zeroand leads to an underflow event yielding a significant QoEdegradation All results in Figures 10 and 11 demonstratethat the AMP significantly improves the QoE of streamingservices

43 Performance Monitoring for Wireless Streaming Videostreaming over wireless networks has become very popularrecently However it is still challenging to offer stream-ing services over wireless networks To clearly identify thechallenging issues of wireless streaming services in thissubsection the AMP function is not enabled in the mediaplayers of all receivers To evaluate the QoE of wirelessstreaming services the second network scenario shown inFigure 12 is considered and the first test video given in Table 2is used The video stream is multicast to two independentreceivers at the last hop via the IEEE 80211n wireless link toReceiver 1 and via the wired link to Receiver 2 Receiver 1 isaway from theWiFi AP by 2m and no obstacle between themFigure 13 shows the observed synthetic MOSs at Receivers1 and 2 under different cross-traffic rates using the QoEmonitor program developed in this paper By Figure 13 theQoE at Receiver 2 using a wired link is better than that atReceiver 1 using a wireless link Even though no cross-trafficexists the MOS of the video streaming service at Receiver1 via the wireless link is less than 4 According to the QoSparameters measured at Receivers 1 and 2 the difference ofQoE between them is mainly due to a higher packet loss ratein the wireless link

Subsequently to compare the performance of multicastand unicast the network scenario 3 shown in Figure 14 isconsidered The second video clip with the resolution 720times 480 p and bit rate 1936 kbps is used as shown in Table 5Different numbers of similar unicast streams are pumpedfrom the video server to the receivers using a wireless linkat the last hopThe synthetic MOS of the streaming service atthe receivers decreases significantly as the number of unicaststreams increases as shown in Figure 15 This is because thejitter burstiness and packet loss rate increase as the numberof unicast streams in the network grows yielding the QoEdegradation at the receivers Note that if themulticast scheme

Table 5 Information of test video 2

Parameter ValueVideo title Call Me Maybe (MV)Length 199 secResolution 720 times 480 pixelsEncoding scheme MPEG-2 TSBit rate 1936 kbpsFrame rate 30 fpsGOP size 15Content size 46MB

is used only one video stream is required to be pumped intothe network no matter how many receivers exist That is theQoE of video streaming services is irrelevant to the numberof receivers if the multicast scheme is employed Obviouslyunder the scenario that multiple users watch the same videostream the multicast scheme performs much better than theunicast one in terms of QoE

Finally to investigate the impact of transmission distanceand obstacles on the QoE of wireless streaming services thewireless network scenario 4 plotted in Figure 16 is consideredand the second test video given in Table 5 is used The videostream is multicast to several wireless receivers at differentpositions in our labs No cross-traffic is injected into thenetwork No obstacles exist between the wireless AP andthe receivers at Locations 1 and 2 However the receiver atLocation 1 is much closer to the AP than the receiver atLocation 2 isThere are cupboards and a wall between the APand the receiver at Location 3 The receiver at Location 4 isoutside the door of our lab and is isolated from the AP by awallThe other receiver at Location 5 is also isolated from theAP by a wall and cupboards All receivers receive the samemulticast stream simultaneously and their individual MOSsare estimated using the developed QoEmonitoring programThe results are shown in Figure 17 where the MOSs of thereceivers located at Locations 1 and 2 are the best becauseno obstacle blocks the wireless signals for them while theMOSs of the receivers located at Locations 4 and 5 are theworst because there exist cupboards and a wall between themand the AP The MOS of the receiver located at Location 3is better than those located at Locations 4 and 5 because thereceiver at Location 3 is much closer to the AP than thoseat Locations 4 and 5 are Accordingly the distance and thenumber of obstacles between the WiFi AP and the receiversstrongly affect the QoE of wireless streaming services

5 Conclusions

In this work anAMP and a real-timeQoEmonitoring systemfor video streaming services arerealized The AMP design is

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 4: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

4 International Journal of Digital Multimedia Broadcasting

188 bytes

Transportpacketstream

Header Header HeaderPayload Payload Payload

Syncbyte

Transporterror

indicator

Payloadunit startindicator

Transportpriority PID

Transportscrambling

control

Adaptationfield

control

Continuitycounter

Adaptationfield

8 1 1 1 13 2 2 4

Figure 3 MPEG-2 TS packet format

QoE monitoring system

QoE evaluation

QoEQoSreporting

NIC card AMP media player

QoS measurement(i) Underflow time ratio

(ii) Packet loss rate(iii) Initial playout delay(iv) Normalized playout rate

Figure 4 Architecture of the QoE monitoring system

for the QoE monitoring system Next several network-basedQoS metrics are defined and their measurement methods arepresented Finally the overall QoE evaluation method for theQoE monitoring system is introduced

31 Architecture of QoE Monitoring System The architec-ture of our proposed QoE monitoring system is plotted inFigure 4 It includes the QoS measurement engine QoEevaluation function and performance reporting windowThe QoS measurement engine is designed to collect theQoS metrics from the media player and the NIC cardThese obtained QoS metrics are used by the QoE evaluationfunction for producing the overall QoE of video streamingservices All instantaneous QoS metrics and QoE results arereported in real time and the curve of overall QoE versus timeis also displayed with a chart in the Graphic User Interface(GUI) for the monitoring purpose

32 Measurements of QoS Parameters In the proposed QoEmonitoring system the considered QoS metrics includeunderflow time ratio QoSu packet loss rate QoSl initial

Figure 5 Buffering mechanism in VLC player

playout delay QoSd and normalized playout rate QoSr Themeasurements of these parameters are described as follows

(a) Underflow time ratio QoSu whenever an underflowevent occurs the underflow time starts to accumulateuntil the playback restarts again At the same timethe number of underflow events is increased by 1Theunderflow time ratio equals the ratio of accumulatedunderflow time to the total playback time durationThe accumulated underflow time can be computedaccording to the value of Status() provided by VLCLua extension program as shown in Figure 5 Whenthe value of Status() changes into 2 the underflowtimer turns ON until the value of Status() becomes3The sojourn time at the state with Status() = 2 is theunderflow timeWhenever the underflow timer turnsOFF the accumulated underflow time is updated

(b) Packet loss rate QoSl the packet loss rate is obtainedby analyzing the number of receiving packets at the

International Journal of Digital Multimedia Broadcasting 5

NIC card or using the network protocol analyzer suchas Wireshark

(c) Initial playout delay QoSd initial playout delaystrongly depends on the buffering time set in VLCmedia player However in our implemented AMPthis parameter is directly determined by the DPTAalgorithm proposed in [22]

(d) Normalized playout rate QoSr whenever the playoutrate is adjusted the normalized playout rate is com-puted and saved However within each minute onlythe maximum and minimum normalized playoutrates are used in our design [5] That is the corre-sponding QoE of QoSr during every minute equalsthemean of these two individual QoEs resulting fromthe maximum and minimum normalized playoutrates

33 Overall QoE Evaluation Function The overall QoE eval-uation function adopts the integratedmultivariate QoE func-tion 119891119868(QoSuQoSlQoSdQoSr) and the individual QoS-QoE mapping functions 119891u(119909) 119891l(119909) 119891d(119909) and 119891r(119909) pro-posed in [5] to derive the overall QoE Based on the results of[5] they are given as follows

119891119868 (QoSuQoSlQoSdQoSr)

= 5 sdot 119891u (QoSu)5 sdot 119891l (QoSl)

5 sdot 119891d (QoSd)5

sdot 119891r (QoSr)5

(6)

119891u (119909) = 5119890minus571119909 (7)

119891l (119909) = 5119890minus1607119909 (8)

119891d (119909) = 5119890minus00416119909 (9)

119891r (119909) = 5119909894119890minus894(119909minus1) (10)

The scheme used in [5] consists of three steps to bedone beforehand First several Key Performance Indicators(KPIs) or QoS metrics such as the initial playout delaypacket loss rate underflow time ratio and playout ratewere selected Then several distorted videos were generatedbased on these selected QoS metrics Second subjective testswere conducted using human observers to rate the distortedvideos in terms of Mean Opinion Score (MOS) [25] TheACR subjective test method was adopted in [5] Howeverthe length of each distorted video clip is about 64 s ratherthan 10 s Each distorted video clip was viewed and ratedby no less than 30 viewers The title of the used video clipwhich is with the frame rate 30 fps resolution 1920 times 1080 pand video codec MPEG-2 is ldquoAvatarrdquo Thirdly using theregression approach the QoS-QoE mapping functions forindividual QoS metrics were obtained Finally the productform of individual QoS-QoE mapping functions [5 6] isused to derive the overall QoE of a video streaming serviceNotably although only four QoS metrics are considered in

Figure 6 VLC media player and Lua program

Figure 7 QoEMOS monitoring program

(6) the number of QoSmetrics can be arbitrarily extended asneeded

4 Experimental Results

In this section we first briefly introduce the GUIs andfunctions of developed AMP andQoEmonitoring programsSubsequently using the developedQoEmonitoring programseveral experiments are conducted to demonstrate the perfor-mance of AMP Finally we use the QoE monitoring programto identify some challenging issues of wireless streaming

41 QoE Monitoring System The developed AMP is basedon the VLC media player and its add-ons such as the Luaextension Figure 6 shows the VLC media player and itsLua extension program One can directly set the multicastIP of the streaming source and activate the AMP functionon the VLC Lua extension program Although in this paperthe AMP is implemented into the VLC media player thereis no difficulty to implement it into other popular mediaplayers such as the MS Windows Media Player or YouTubePlayer The QoE monitoring program which is developedusing the C++Builder and is shown in Figure 7 evaluates anddisplays the overall QoE The measured QoS and syntheticQoEMOS are displayed in real time on the monitoringwindow as shown in Figure 7 The corresponding QoEsof individual QoSs are displayed on the left side of themonitoringwindowwhile the curve of the overall QoE versustime is displayed on the right side of the monitoring windowas shown in Figure 7 By clicking the ldquoToolsrdquo on themenu barof the QoE monitoring program other detailed QoS metrics

6 International Journal of Digital Multimedia Broadcasting

Figure 8 Monitoring tools for video packets at the client

Figure 9 Experimental network scenario 1

Table 2 Information of test video 1

Parameter ValueVideo title The Big BangTheoryLength 593 secResolution 352 times 288 pixelsEncoding scheme MPEG-2 TSBit rate 1406 kbpsFrame rate 25 fpsGOP size 15Content size 995MB

such as the interarrival time of video frames normalizedplayout rate number of packet losses buffer fullness andthe jitter observed at the client can be displayed as shownin Figure 8 By clicking the interested QoS parameter suchas the ldquoMoveAvg Interarrivalrdquo the curve of the QoS versustime is immediately displayed on the monitoring window ofFigure 8

42 Performance Evaluation of AMP In order to demon-strate our AMP design and QoE monitoring system the firstexperimental network scenario is constructed as shown in

Figure 9 The server pumps the video stream into the firstrouter followed by a bottleneck link and the second routerFinally the disturbed video stream arrives at the receiverAll link capacities in Figure 9 are set to be 10Mbps ThePareto ON-OFF cross-traffic is injected into the bottlenecklink using theDistributed Internet TrafficGenerator (D-ITG)tool [26] Therefore the video stream is disturbed by thecross-traffic in the bottleneck link and thus packet losses andjitters may occur The video is encoded into the MPEG-2 TSat the streaming server and the RTPUDP streaming protocolis employed Table 2 lists the detailed information of the firsttest video and Table 3 lists the parameters of AMP The otherparameters not mentioned here are set the same as those in[22]

First the QoEs of two media playout systems with andwithout AMP are compared The video server pumps amulticast stream into the network and two independentclients simultaneously receive the same multicast stream atthe remote side One client uses the media player with AMPwhile the other one adopts the media player without AMPThe observed QoEs of these two systems with and withoutAMP under different cross-traffic rates 0 4 and 8Mbps aregiven in Figure 10 When the cross-traffic rate is not largerthan 4Mbps the overall QoE is almost over 4 no matterwhether the AMP is enabled or not This is because thejitter burstiness and losses of video packets are very smallunder a low cross-traffic rate Additionally under a low cross-traffic rate the observed QoE on the media player withoutAMP remains very stable while the QoE on the media playerwith AMP slightly oscillates in the first minute The reasonis explained as follows The media player without AMP canstart playback only when the buffer fullness exceeds 119871 = 100yielding the initial playout delay about 4 s and always playsthe video with the normal playout rate However the mediaplayer with AMP can start playback earlier but with a lowerinitial playout rate Since the initial playout rate of the mediaplayer with AMP is smaller than the normal playout ratethe observed QoE on the media player with AMP is slightlydegraded in the first minute as shown in Figure 10 Table 4shows the measured initial playout delays QoSd of the mediaplayers with and without AMP Because the initial playoutdelay in the media player with AMP is much smaller thanthat without AMP the corresponding QoE of QoSd of themedia player with AMP becomes better When the cross-traffic rate becomes 8Mbps the video stream is seriouslydisturbed by the cross-traffic so that the observed QoE at theclient becomes unacceptable (MOS lt 3) The serious QoEdegradation is mainly due to the increase of the number ofunderflow events and underflow timeHowever the observedQoE on the media player with AMP is still much better thanthat without AMP as shown in Figure 10

The corresponding buffer fullness and the playout rateat the client under the cross-traffic rate 6Mbps are shownin Figure 11 According to Figure 11 the buffer fullness ofthe system with AMP is more stable than that of the systemwithout AMP For the media player with AMP the underflowevents never happen because the buffer fullness never dropsto an unacceptable low level after the playback Althoughthe playout rate of the media player with AMP varies the

International Journal of Digital Multimedia Broadcasting 7

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5M

OS

Cross-traffic rate = 0 MbpsPlayer with AMPPlayer without AMP

Cross-traffic rate = 4 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Cross-traffic rate = 8 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Figure 10 Comparison of synthetic MOSsQoEs between the systems with and without AMP under various cross-traffic rates

Player with AMPPlayer without AMP

0

25

50

75

100

125

150

175

200

Buffe

r ful

lnes

s (fr

ames

)

20 40 60 80 1000Time (sec)

Player with AMPPlayer without AMP

20 40 60 80 1000Time (sec)

0

02

04

06

08

1

12

Nor

mal

ized

pla

yout

rate

Figure 11 Comparison of buffer fullness and normalized playout rates between systems with and without AMP (cross-traffic rate = 6Mbps)

8 International Journal of Digital Multimedia Broadcasting

Table 3 Parameters setting in AMP

Parameter 1198710 119871 119867 119873 119888 119903 1199031 1199032Value 30 100 200 500 04 015 03 03

Table 4 Initial playout delay QoSd and corresponding QoE

Cross-traffic rate Player with AMP Player without AMP(Mbps) QoSdQoE QoSdQoE0 1124 sec4776 4001 sec42334 1656 sec4667 4250 sec41898 3876 sec4255 4756 sec4102

deviation of the playout rate is never over 25 so that theQoEdegradation resulting from it is still acceptable However forthe media player without AMP the buffer fullness duringthe time interval (10 s 66 s) decreases almost linearly to zeroand leads to an underflow event yielding a significant QoEdegradation All results in Figures 10 and 11 demonstratethat the AMP significantly improves the QoE of streamingservices

43 Performance Monitoring for Wireless Streaming Videostreaming over wireless networks has become very popularrecently However it is still challenging to offer stream-ing services over wireless networks To clearly identify thechallenging issues of wireless streaming services in thissubsection the AMP function is not enabled in the mediaplayers of all receivers To evaluate the QoE of wirelessstreaming services the second network scenario shown inFigure 12 is considered and the first test video given in Table 2is used The video stream is multicast to two independentreceivers at the last hop via the IEEE 80211n wireless link toReceiver 1 and via the wired link to Receiver 2 Receiver 1 isaway from theWiFi AP by 2m and no obstacle between themFigure 13 shows the observed synthetic MOSs at Receivers1 and 2 under different cross-traffic rates using the QoEmonitor program developed in this paper By Figure 13 theQoE at Receiver 2 using a wired link is better than that atReceiver 1 using a wireless link Even though no cross-trafficexists the MOS of the video streaming service at Receiver1 via the wireless link is less than 4 According to the QoSparameters measured at Receivers 1 and 2 the difference ofQoE between them is mainly due to a higher packet loss ratein the wireless link

Subsequently to compare the performance of multicastand unicast the network scenario 3 shown in Figure 14 isconsidered The second video clip with the resolution 720times 480 p and bit rate 1936 kbps is used as shown in Table 5Different numbers of similar unicast streams are pumpedfrom the video server to the receivers using a wireless linkat the last hopThe synthetic MOS of the streaming service atthe receivers decreases significantly as the number of unicaststreams increases as shown in Figure 15 This is because thejitter burstiness and packet loss rate increase as the numberof unicast streams in the network grows yielding the QoEdegradation at the receivers Note that if themulticast scheme

Table 5 Information of test video 2

Parameter ValueVideo title Call Me Maybe (MV)Length 199 secResolution 720 times 480 pixelsEncoding scheme MPEG-2 TSBit rate 1936 kbpsFrame rate 30 fpsGOP size 15Content size 46MB

is used only one video stream is required to be pumped intothe network no matter how many receivers exist That is theQoE of video streaming services is irrelevant to the numberof receivers if the multicast scheme is employed Obviouslyunder the scenario that multiple users watch the same videostream the multicast scheme performs much better than theunicast one in terms of QoE

Finally to investigate the impact of transmission distanceand obstacles on the QoE of wireless streaming services thewireless network scenario 4 plotted in Figure 16 is consideredand the second test video given in Table 5 is used The videostream is multicast to several wireless receivers at differentpositions in our labs No cross-traffic is injected into thenetwork No obstacles exist between the wireless AP andthe receivers at Locations 1 and 2 However the receiver atLocation 1 is much closer to the AP than the receiver atLocation 2 isThere are cupboards and a wall between the APand the receiver at Location 3 The receiver at Location 4 isoutside the door of our lab and is isolated from the AP by awallThe other receiver at Location 5 is also isolated from theAP by a wall and cupboards All receivers receive the samemulticast stream simultaneously and their individual MOSsare estimated using the developed QoEmonitoring programThe results are shown in Figure 17 where the MOSs of thereceivers located at Locations 1 and 2 are the best becauseno obstacle blocks the wireless signals for them while theMOSs of the receivers located at Locations 4 and 5 are theworst because there exist cupboards and a wall between themand the AP The MOS of the receiver located at Location 3is better than those located at Locations 4 and 5 because thereceiver at Location 3 is much closer to the AP than thoseat Locations 4 and 5 are Accordingly the distance and thenumber of obstacles between the WiFi AP and the receiversstrongly affect the QoE of wireless streaming services

5 Conclusions

In this work anAMP and a real-timeQoEmonitoring systemfor video streaming services arerealized The AMP design is

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 5: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

International Journal of Digital Multimedia Broadcasting 5

NIC card or using the network protocol analyzer suchas Wireshark

(c) Initial playout delay QoSd initial playout delaystrongly depends on the buffering time set in VLCmedia player However in our implemented AMPthis parameter is directly determined by the DPTAalgorithm proposed in [22]

(d) Normalized playout rate QoSr whenever the playoutrate is adjusted the normalized playout rate is com-puted and saved However within each minute onlythe maximum and minimum normalized playoutrates are used in our design [5] That is the corre-sponding QoE of QoSr during every minute equalsthemean of these two individual QoEs resulting fromthe maximum and minimum normalized playoutrates

33 Overall QoE Evaluation Function The overall QoE eval-uation function adopts the integratedmultivariate QoE func-tion 119891119868(QoSuQoSlQoSdQoSr) and the individual QoS-QoE mapping functions 119891u(119909) 119891l(119909) 119891d(119909) and 119891r(119909) pro-posed in [5] to derive the overall QoE Based on the results of[5] they are given as follows

119891119868 (QoSuQoSlQoSdQoSr)

= 5 sdot 119891u (QoSu)5 sdot 119891l (QoSl)

5 sdot 119891d (QoSd)5

sdot 119891r (QoSr)5

(6)

119891u (119909) = 5119890minus571119909 (7)

119891l (119909) = 5119890minus1607119909 (8)

119891d (119909) = 5119890minus00416119909 (9)

119891r (119909) = 5119909894119890minus894(119909minus1) (10)

The scheme used in [5] consists of three steps to bedone beforehand First several Key Performance Indicators(KPIs) or QoS metrics such as the initial playout delaypacket loss rate underflow time ratio and playout ratewere selected Then several distorted videos were generatedbased on these selected QoS metrics Second subjective testswere conducted using human observers to rate the distortedvideos in terms of Mean Opinion Score (MOS) [25] TheACR subjective test method was adopted in [5] Howeverthe length of each distorted video clip is about 64 s ratherthan 10 s Each distorted video clip was viewed and ratedby no less than 30 viewers The title of the used video clipwhich is with the frame rate 30 fps resolution 1920 times 1080 pand video codec MPEG-2 is ldquoAvatarrdquo Thirdly using theregression approach the QoS-QoE mapping functions forindividual QoS metrics were obtained Finally the productform of individual QoS-QoE mapping functions [5 6] isused to derive the overall QoE of a video streaming serviceNotably although only four QoS metrics are considered in

Figure 6 VLC media player and Lua program

Figure 7 QoEMOS monitoring program

(6) the number of QoSmetrics can be arbitrarily extended asneeded

4 Experimental Results

In this section we first briefly introduce the GUIs andfunctions of developed AMP andQoEmonitoring programsSubsequently using the developedQoEmonitoring programseveral experiments are conducted to demonstrate the perfor-mance of AMP Finally we use the QoE monitoring programto identify some challenging issues of wireless streaming

41 QoE Monitoring System The developed AMP is basedon the VLC media player and its add-ons such as the Luaextension Figure 6 shows the VLC media player and itsLua extension program One can directly set the multicastIP of the streaming source and activate the AMP functionon the VLC Lua extension program Although in this paperthe AMP is implemented into the VLC media player thereis no difficulty to implement it into other popular mediaplayers such as the MS Windows Media Player or YouTubePlayer The QoE monitoring program which is developedusing the C++Builder and is shown in Figure 7 evaluates anddisplays the overall QoE The measured QoS and syntheticQoEMOS are displayed in real time on the monitoringwindow as shown in Figure 7 The corresponding QoEsof individual QoSs are displayed on the left side of themonitoringwindowwhile the curve of the overall QoE versustime is displayed on the right side of the monitoring windowas shown in Figure 7 By clicking the ldquoToolsrdquo on themenu barof the QoE monitoring program other detailed QoS metrics

6 International Journal of Digital Multimedia Broadcasting

Figure 8 Monitoring tools for video packets at the client

Figure 9 Experimental network scenario 1

Table 2 Information of test video 1

Parameter ValueVideo title The Big BangTheoryLength 593 secResolution 352 times 288 pixelsEncoding scheme MPEG-2 TSBit rate 1406 kbpsFrame rate 25 fpsGOP size 15Content size 995MB

such as the interarrival time of video frames normalizedplayout rate number of packet losses buffer fullness andthe jitter observed at the client can be displayed as shownin Figure 8 By clicking the interested QoS parameter suchas the ldquoMoveAvg Interarrivalrdquo the curve of the QoS versustime is immediately displayed on the monitoring window ofFigure 8

42 Performance Evaluation of AMP In order to demon-strate our AMP design and QoE monitoring system the firstexperimental network scenario is constructed as shown in

Figure 9 The server pumps the video stream into the firstrouter followed by a bottleneck link and the second routerFinally the disturbed video stream arrives at the receiverAll link capacities in Figure 9 are set to be 10Mbps ThePareto ON-OFF cross-traffic is injected into the bottlenecklink using theDistributed Internet TrafficGenerator (D-ITG)tool [26] Therefore the video stream is disturbed by thecross-traffic in the bottleneck link and thus packet losses andjitters may occur The video is encoded into the MPEG-2 TSat the streaming server and the RTPUDP streaming protocolis employed Table 2 lists the detailed information of the firsttest video and Table 3 lists the parameters of AMP The otherparameters not mentioned here are set the same as those in[22]

First the QoEs of two media playout systems with andwithout AMP are compared The video server pumps amulticast stream into the network and two independentclients simultaneously receive the same multicast stream atthe remote side One client uses the media player with AMPwhile the other one adopts the media player without AMPThe observed QoEs of these two systems with and withoutAMP under different cross-traffic rates 0 4 and 8Mbps aregiven in Figure 10 When the cross-traffic rate is not largerthan 4Mbps the overall QoE is almost over 4 no matterwhether the AMP is enabled or not This is because thejitter burstiness and losses of video packets are very smallunder a low cross-traffic rate Additionally under a low cross-traffic rate the observed QoE on the media player withoutAMP remains very stable while the QoE on the media playerwith AMP slightly oscillates in the first minute The reasonis explained as follows The media player without AMP canstart playback only when the buffer fullness exceeds 119871 = 100yielding the initial playout delay about 4 s and always playsthe video with the normal playout rate However the mediaplayer with AMP can start playback earlier but with a lowerinitial playout rate Since the initial playout rate of the mediaplayer with AMP is smaller than the normal playout ratethe observed QoE on the media player with AMP is slightlydegraded in the first minute as shown in Figure 10 Table 4shows the measured initial playout delays QoSd of the mediaplayers with and without AMP Because the initial playoutdelay in the media player with AMP is much smaller thanthat without AMP the corresponding QoE of QoSd of themedia player with AMP becomes better When the cross-traffic rate becomes 8Mbps the video stream is seriouslydisturbed by the cross-traffic so that the observed QoE at theclient becomes unacceptable (MOS lt 3) The serious QoEdegradation is mainly due to the increase of the number ofunderflow events and underflow timeHowever the observedQoE on the media player with AMP is still much better thanthat without AMP as shown in Figure 10

The corresponding buffer fullness and the playout rateat the client under the cross-traffic rate 6Mbps are shownin Figure 11 According to Figure 11 the buffer fullness ofthe system with AMP is more stable than that of the systemwithout AMP For the media player with AMP the underflowevents never happen because the buffer fullness never dropsto an unacceptable low level after the playback Althoughthe playout rate of the media player with AMP varies the

International Journal of Digital Multimedia Broadcasting 7

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5M

OS

Cross-traffic rate = 0 MbpsPlayer with AMPPlayer without AMP

Cross-traffic rate = 4 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Cross-traffic rate = 8 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Figure 10 Comparison of synthetic MOSsQoEs between the systems with and without AMP under various cross-traffic rates

Player with AMPPlayer without AMP

0

25

50

75

100

125

150

175

200

Buffe

r ful

lnes

s (fr

ames

)

20 40 60 80 1000Time (sec)

Player with AMPPlayer without AMP

20 40 60 80 1000Time (sec)

0

02

04

06

08

1

12

Nor

mal

ized

pla

yout

rate

Figure 11 Comparison of buffer fullness and normalized playout rates between systems with and without AMP (cross-traffic rate = 6Mbps)

8 International Journal of Digital Multimedia Broadcasting

Table 3 Parameters setting in AMP

Parameter 1198710 119871 119867 119873 119888 119903 1199031 1199032Value 30 100 200 500 04 015 03 03

Table 4 Initial playout delay QoSd and corresponding QoE

Cross-traffic rate Player with AMP Player without AMP(Mbps) QoSdQoE QoSdQoE0 1124 sec4776 4001 sec42334 1656 sec4667 4250 sec41898 3876 sec4255 4756 sec4102

deviation of the playout rate is never over 25 so that theQoEdegradation resulting from it is still acceptable However forthe media player without AMP the buffer fullness duringthe time interval (10 s 66 s) decreases almost linearly to zeroand leads to an underflow event yielding a significant QoEdegradation All results in Figures 10 and 11 demonstratethat the AMP significantly improves the QoE of streamingservices

43 Performance Monitoring for Wireless Streaming Videostreaming over wireless networks has become very popularrecently However it is still challenging to offer stream-ing services over wireless networks To clearly identify thechallenging issues of wireless streaming services in thissubsection the AMP function is not enabled in the mediaplayers of all receivers To evaluate the QoE of wirelessstreaming services the second network scenario shown inFigure 12 is considered and the first test video given in Table 2is used The video stream is multicast to two independentreceivers at the last hop via the IEEE 80211n wireless link toReceiver 1 and via the wired link to Receiver 2 Receiver 1 isaway from theWiFi AP by 2m and no obstacle between themFigure 13 shows the observed synthetic MOSs at Receivers1 and 2 under different cross-traffic rates using the QoEmonitor program developed in this paper By Figure 13 theQoE at Receiver 2 using a wired link is better than that atReceiver 1 using a wireless link Even though no cross-trafficexists the MOS of the video streaming service at Receiver1 via the wireless link is less than 4 According to the QoSparameters measured at Receivers 1 and 2 the difference ofQoE between them is mainly due to a higher packet loss ratein the wireless link

Subsequently to compare the performance of multicastand unicast the network scenario 3 shown in Figure 14 isconsidered The second video clip with the resolution 720times 480 p and bit rate 1936 kbps is used as shown in Table 5Different numbers of similar unicast streams are pumpedfrom the video server to the receivers using a wireless linkat the last hopThe synthetic MOS of the streaming service atthe receivers decreases significantly as the number of unicaststreams increases as shown in Figure 15 This is because thejitter burstiness and packet loss rate increase as the numberof unicast streams in the network grows yielding the QoEdegradation at the receivers Note that if themulticast scheme

Table 5 Information of test video 2

Parameter ValueVideo title Call Me Maybe (MV)Length 199 secResolution 720 times 480 pixelsEncoding scheme MPEG-2 TSBit rate 1936 kbpsFrame rate 30 fpsGOP size 15Content size 46MB

is used only one video stream is required to be pumped intothe network no matter how many receivers exist That is theQoE of video streaming services is irrelevant to the numberof receivers if the multicast scheme is employed Obviouslyunder the scenario that multiple users watch the same videostream the multicast scheme performs much better than theunicast one in terms of QoE

Finally to investigate the impact of transmission distanceand obstacles on the QoE of wireless streaming services thewireless network scenario 4 plotted in Figure 16 is consideredand the second test video given in Table 5 is used The videostream is multicast to several wireless receivers at differentpositions in our labs No cross-traffic is injected into thenetwork No obstacles exist between the wireless AP andthe receivers at Locations 1 and 2 However the receiver atLocation 1 is much closer to the AP than the receiver atLocation 2 isThere are cupboards and a wall between the APand the receiver at Location 3 The receiver at Location 4 isoutside the door of our lab and is isolated from the AP by awallThe other receiver at Location 5 is also isolated from theAP by a wall and cupboards All receivers receive the samemulticast stream simultaneously and their individual MOSsare estimated using the developed QoEmonitoring programThe results are shown in Figure 17 where the MOSs of thereceivers located at Locations 1 and 2 are the best becauseno obstacle blocks the wireless signals for them while theMOSs of the receivers located at Locations 4 and 5 are theworst because there exist cupboards and a wall between themand the AP The MOS of the receiver located at Location 3is better than those located at Locations 4 and 5 because thereceiver at Location 3 is much closer to the AP than thoseat Locations 4 and 5 are Accordingly the distance and thenumber of obstacles between the WiFi AP and the receiversstrongly affect the QoE of wireless streaming services

5 Conclusions

In this work anAMP and a real-timeQoEmonitoring systemfor video streaming services arerealized The AMP design is

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 6: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

6 International Journal of Digital Multimedia Broadcasting

Figure 8 Monitoring tools for video packets at the client

Figure 9 Experimental network scenario 1

Table 2 Information of test video 1

Parameter ValueVideo title The Big BangTheoryLength 593 secResolution 352 times 288 pixelsEncoding scheme MPEG-2 TSBit rate 1406 kbpsFrame rate 25 fpsGOP size 15Content size 995MB

such as the interarrival time of video frames normalizedplayout rate number of packet losses buffer fullness andthe jitter observed at the client can be displayed as shownin Figure 8 By clicking the interested QoS parameter suchas the ldquoMoveAvg Interarrivalrdquo the curve of the QoS versustime is immediately displayed on the monitoring window ofFigure 8

42 Performance Evaluation of AMP In order to demon-strate our AMP design and QoE monitoring system the firstexperimental network scenario is constructed as shown in

Figure 9 The server pumps the video stream into the firstrouter followed by a bottleneck link and the second routerFinally the disturbed video stream arrives at the receiverAll link capacities in Figure 9 are set to be 10Mbps ThePareto ON-OFF cross-traffic is injected into the bottlenecklink using theDistributed Internet TrafficGenerator (D-ITG)tool [26] Therefore the video stream is disturbed by thecross-traffic in the bottleneck link and thus packet losses andjitters may occur The video is encoded into the MPEG-2 TSat the streaming server and the RTPUDP streaming protocolis employed Table 2 lists the detailed information of the firsttest video and Table 3 lists the parameters of AMP The otherparameters not mentioned here are set the same as those in[22]

First the QoEs of two media playout systems with andwithout AMP are compared The video server pumps amulticast stream into the network and two independentclients simultaneously receive the same multicast stream atthe remote side One client uses the media player with AMPwhile the other one adopts the media player without AMPThe observed QoEs of these two systems with and withoutAMP under different cross-traffic rates 0 4 and 8Mbps aregiven in Figure 10 When the cross-traffic rate is not largerthan 4Mbps the overall QoE is almost over 4 no matterwhether the AMP is enabled or not This is because thejitter burstiness and losses of video packets are very smallunder a low cross-traffic rate Additionally under a low cross-traffic rate the observed QoE on the media player withoutAMP remains very stable while the QoE on the media playerwith AMP slightly oscillates in the first minute The reasonis explained as follows The media player without AMP canstart playback only when the buffer fullness exceeds 119871 = 100yielding the initial playout delay about 4 s and always playsthe video with the normal playout rate However the mediaplayer with AMP can start playback earlier but with a lowerinitial playout rate Since the initial playout rate of the mediaplayer with AMP is smaller than the normal playout ratethe observed QoE on the media player with AMP is slightlydegraded in the first minute as shown in Figure 10 Table 4shows the measured initial playout delays QoSd of the mediaplayers with and without AMP Because the initial playoutdelay in the media player with AMP is much smaller thanthat without AMP the corresponding QoE of QoSd of themedia player with AMP becomes better When the cross-traffic rate becomes 8Mbps the video stream is seriouslydisturbed by the cross-traffic so that the observed QoE at theclient becomes unacceptable (MOS lt 3) The serious QoEdegradation is mainly due to the increase of the number ofunderflow events and underflow timeHowever the observedQoE on the media player with AMP is still much better thanthat without AMP as shown in Figure 10

The corresponding buffer fullness and the playout rateat the client under the cross-traffic rate 6Mbps are shownin Figure 11 According to Figure 11 the buffer fullness ofthe system with AMP is more stable than that of the systemwithout AMP For the media player with AMP the underflowevents never happen because the buffer fullness never dropsto an unacceptable low level after the playback Althoughthe playout rate of the media player with AMP varies the

International Journal of Digital Multimedia Broadcasting 7

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5M

OS

Cross-traffic rate = 0 MbpsPlayer with AMPPlayer without AMP

Cross-traffic rate = 4 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Cross-traffic rate = 8 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Figure 10 Comparison of synthetic MOSsQoEs between the systems with and without AMP under various cross-traffic rates

Player with AMPPlayer without AMP

0

25

50

75

100

125

150

175

200

Buffe

r ful

lnes

s (fr

ames

)

20 40 60 80 1000Time (sec)

Player with AMPPlayer without AMP

20 40 60 80 1000Time (sec)

0

02

04

06

08

1

12

Nor

mal

ized

pla

yout

rate

Figure 11 Comparison of buffer fullness and normalized playout rates between systems with and without AMP (cross-traffic rate = 6Mbps)

8 International Journal of Digital Multimedia Broadcasting

Table 3 Parameters setting in AMP

Parameter 1198710 119871 119867 119873 119888 119903 1199031 1199032Value 30 100 200 500 04 015 03 03

Table 4 Initial playout delay QoSd and corresponding QoE

Cross-traffic rate Player with AMP Player without AMP(Mbps) QoSdQoE QoSdQoE0 1124 sec4776 4001 sec42334 1656 sec4667 4250 sec41898 3876 sec4255 4756 sec4102

deviation of the playout rate is never over 25 so that theQoEdegradation resulting from it is still acceptable However forthe media player without AMP the buffer fullness duringthe time interval (10 s 66 s) decreases almost linearly to zeroand leads to an underflow event yielding a significant QoEdegradation All results in Figures 10 and 11 demonstratethat the AMP significantly improves the QoE of streamingservices

43 Performance Monitoring for Wireless Streaming Videostreaming over wireless networks has become very popularrecently However it is still challenging to offer stream-ing services over wireless networks To clearly identify thechallenging issues of wireless streaming services in thissubsection the AMP function is not enabled in the mediaplayers of all receivers To evaluate the QoE of wirelessstreaming services the second network scenario shown inFigure 12 is considered and the first test video given in Table 2is used The video stream is multicast to two independentreceivers at the last hop via the IEEE 80211n wireless link toReceiver 1 and via the wired link to Receiver 2 Receiver 1 isaway from theWiFi AP by 2m and no obstacle between themFigure 13 shows the observed synthetic MOSs at Receivers1 and 2 under different cross-traffic rates using the QoEmonitor program developed in this paper By Figure 13 theQoE at Receiver 2 using a wired link is better than that atReceiver 1 using a wireless link Even though no cross-trafficexists the MOS of the video streaming service at Receiver1 via the wireless link is less than 4 According to the QoSparameters measured at Receivers 1 and 2 the difference ofQoE between them is mainly due to a higher packet loss ratein the wireless link

Subsequently to compare the performance of multicastand unicast the network scenario 3 shown in Figure 14 isconsidered The second video clip with the resolution 720times 480 p and bit rate 1936 kbps is used as shown in Table 5Different numbers of similar unicast streams are pumpedfrom the video server to the receivers using a wireless linkat the last hopThe synthetic MOS of the streaming service atthe receivers decreases significantly as the number of unicaststreams increases as shown in Figure 15 This is because thejitter burstiness and packet loss rate increase as the numberof unicast streams in the network grows yielding the QoEdegradation at the receivers Note that if themulticast scheme

Table 5 Information of test video 2

Parameter ValueVideo title Call Me Maybe (MV)Length 199 secResolution 720 times 480 pixelsEncoding scheme MPEG-2 TSBit rate 1936 kbpsFrame rate 30 fpsGOP size 15Content size 46MB

is used only one video stream is required to be pumped intothe network no matter how many receivers exist That is theQoE of video streaming services is irrelevant to the numberof receivers if the multicast scheme is employed Obviouslyunder the scenario that multiple users watch the same videostream the multicast scheme performs much better than theunicast one in terms of QoE

Finally to investigate the impact of transmission distanceand obstacles on the QoE of wireless streaming services thewireless network scenario 4 plotted in Figure 16 is consideredand the second test video given in Table 5 is used The videostream is multicast to several wireless receivers at differentpositions in our labs No cross-traffic is injected into thenetwork No obstacles exist between the wireless AP andthe receivers at Locations 1 and 2 However the receiver atLocation 1 is much closer to the AP than the receiver atLocation 2 isThere are cupboards and a wall between the APand the receiver at Location 3 The receiver at Location 4 isoutside the door of our lab and is isolated from the AP by awallThe other receiver at Location 5 is also isolated from theAP by a wall and cupboards All receivers receive the samemulticast stream simultaneously and their individual MOSsare estimated using the developed QoEmonitoring programThe results are shown in Figure 17 where the MOSs of thereceivers located at Locations 1 and 2 are the best becauseno obstacle blocks the wireless signals for them while theMOSs of the receivers located at Locations 4 and 5 are theworst because there exist cupboards and a wall between themand the AP The MOS of the receiver located at Location 3is better than those located at Locations 4 and 5 because thereceiver at Location 3 is much closer to the AP than thoseat Locations 4 and 5 are Accordingly the distance and thenumber of obstacles between the WiFi AP and the receiversstrongly affect the QoE of wireless streaming services

5 Conclusions

In this work anAMP and a real-timeQoEmonitoring systemfor video streaming services arerealized The AMP design is

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 7: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

International Journal of Digital Multimedia Broadcasting 7

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5M

OS

Cross-traffic rate = 0 MbpsPlayer with AMPPlayer without AMP

Cross-traffic rate = 4 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Cross-traffic rate = 8 MbpsPlayer with AMPPlayer without AMP

0

1

2

3

4

5

MO

S

30 60 90 120 150 1800Time (sec)

Figure 10 Comparison of synthetic MOSsQoEs between the systems with and without AMP under various cross-traffic rates

Player with AMPPlayer without AMP

0

25

50

75

100

125

150

175

200

Buffe

r ful

lnes

s (fr

ames

)

20 40 60 80 1000Time (sec)

Player with AMPPlayer without AMP

20 40 60 80 1000Time (sec)

0

02

04

06

08

1

12

Nor

mal

ized

pla

yout

rate

Figure 11 Comparison of buffer fullness and normalized playout rates between systems with and without AMP (cross-traffic rate = 6Mbps)

8 International Journal of Digital Multimedia Broadcasting

Table 3 Parameters setting in AMP

Parameter 1198710 119871 119867 119873 119888 119903 1199031 1199032Value 30 100 200 500 04 015 03 03

Table 4 Initial playout delay QoSd and corresponding QoE

Cross-traffic rate Player with AMP Player without AMP(Mbps) QoSdQoE QoSdQoE0 1124 sec4776 4001 sec42334 1656 sec4667 4250 sec41898 3876 sec4255 4756 sec4102

deviation of the playout rate is never over 25 so that theQoEdegradation resulting from it is still acceptable However forthe media player without AMP the buffer fullness duringthe time interval (10 s 66 s) decreases almost linearly to zeroand leads to an underflow event yielding a significant QoEdegradation All results in Figures 10 and 11 demonstratethat the AMP significantly improves the QoE of streamingservices

43 Performance Monitoring for Wireless Streaming Videostreaming over wireless networks has become very popularrecently However it is still challenging to offer stream-ing services over wireless networks To clearly identify thechallenging issues of wireless streaming services in thissubsection the AMP function is not enabled in the mediaplayers of all receivers To evaluate the QoE of wirelessstreaming services the second network scenario shown inFigure 12 is considered and the first test video given in Table 2is used The video stream is multicast to two independentreceivers at the last hop via the IEEE 80211n wireless link toReceiver 1 and via the wired link to Receiver 2 Receiver 1 isaway from theWiFi AP by 2m and no obstacle between themFigure 13 shows the observed synthetic MOSs at Receivers1 and 2 under different cross-traffic rates using the QoEmonitor program developed in this paper By Figure 13 theQoE at Receiver 2 using a wired link is better than that atReceiver 1 using a wireless link Even though no cross-trafficexists the MOS of the video streaming service at Receiver1 via the wireless link is less than 4 According to the QoSparameters measured at Receivers 1 and 2 the difference ofQoE between them is mainly due to a higher packet loss ratein the wireless link

Subsequently to compare the performance of multicastand unicast the network scenario 3 shown in Figure 14 isconsidered The second video clip with the resolution 720times 480 p and bit rate 1936 kbps is used as shown in Table 5Different numbers of similar unicast streams are pumpedfrom the video server to the receivers using a wireless linkat the last hopThe synthetic MOS of the streaming service atthe receivers decreases significantly as the number of unicaststreams increases as shown in Figure 15 This is because thejitter burstiness and packet loss rate increase as the numberof unicast streams in the network grows yielding the QoEdegradation at the receivers Note that if themulticast scheme

Table 5 Information of test video 2

Parameter ValueVideo title Call Me Maybe (MV)Length 199 secResolution 720 times 480 pixelsEncoding scheme MPEG-2 TSBit rate 1936 kbpsFrame rate 30 fpsGOP size 15Content size 46MB

is used only one video stream is required to be pumped intothe network no matter how many receivers exist That is theQoE of video streaming services is irrelevant to the numberof receivers if the multicast scheme is employed Obviouslyunder the scenario that multiple users watch the same videostream the multicast scheme performs much better than theunicast one in terms of QoE

Finally to investigate the impact of transmission distanceand obstacles on the QoE of wireless streaming services thewireless network scenario 4 plotted in Figure 16 is consideredand the second test video given in Table 5 is used The videostream is multicast to several wireless receivers at differentpositions in our labs No cross-traffic is injected into thenetwork No obstacles exist between the wireless AP andthe receivers at Locations 1 and 2 However the receiver atLocation 1 is much closer to the AP than the receiver atLocation 2 isThere are cupboards and a wall between the APand the receiver at Location 3 The receiver at Location 4 isoutside the door of our lab and is isolated from the AP by awallThe other receiver at Location 5 is also isolated from theAP by a wall and cupboards All receivers receive the samemulticast stream simultaneously and their individual MOSsare estimated using the developed QoEmonitoring programThe results are shown in Figure 17 where the MOSs of thereceivers located at Locations 1 and 2 are the best becauseno obstacle blocks the wireless signals for them while theMOSs of the receivers located at Locations 4 and 5 are theworst because there exist cupboards and a wall between themand the AP The MOS of the receiver located at Location 3is better than those located at Locations 4 and 5 because thereceiver at Location 3 is much closer to the AP than thoseat Locations 4 and 5 are Accordingly the distance and thenumber of obstacles between the WiFi AP and the receiversstrongly affect the QoE of wireless streaming services

5 Conclusions

In this work anAMP and a real-timeQoEmonitoring systemfor video streaming services arerealized The AMP design is

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 8: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

8 International Journal of Digital Multimedia Broadcasting

Table 3 Parameters setting in AMP

Parameter 1198710 119871 119867 119873 119888 119903 1199031 1199032Value 30 100 200 500 04 015 03 03

Table 4 Initial playout delay QoSd and corresponding QoE

Cross-traffic rate Player with AMP Player without AMP(Mbps) QoSdQoE QoSdQoE0 1124 sec4776 4001 sec42334 1656 sec4667 4250 sec41898 3876 sec4255 4756 sec4102

deviation of the playout rate is never over 25 so that theQoEdegradation resulting from it is still acceptable However forthe media player without AMP the buffer fullness duringthe time interval (10 s 66 s) decreases almost linearly to zeroand leads to an underflow event yielding a significant QoEdegradation All results in Figures 10 and 11 demonstratethat the AMP significantly improves the QoE of streamingservices

43 Performance Monitoring for Wireless Streaming Videostreaming over wireless networks has become very popularrecently However it is still challenging to offer stream-ing services over wireless networks To clearly identify thechallenging issues of wireless streaming services in thissubsection the AMP function is not enabled in the mediaplayers of all receivers To evaluate the QoE of wirelessstreaming services the second network scenario shown inFigure 12 is considered and the first test video given in Table 2is used The video stream is multicast to two independentreceivers at the last hop via the IEEE 80211n wireless link toReceiver 1 and via the wired link to Receiver 2 Receiver 1 isaway from theWiFi AP by 2m and no obstacle between themFigure 13 shows the observed synthetic MOSs at Receivers1 and 2 under different cross-traffic rates using the QoEmonitor program developed in this paper By Figure 13 theQoE at Receiver 2 using a wired link is better than that atReceiver 1 using a wireless link Even though no cross-trafficexists the MOS of the video streaming service at Receiver1 via the wireless link is less than 4 According to the QoSparameters measured at Receivers 1 and 2 the difference ofQoE between them is mainly due to a higher packet loss ratein the wireless link

Subsequently to compare the performance of multicastand unicast the network scenario 3 shown in Figure 14 isconsidered The second video clip with the resolution 720times 480 p and bit rate 1936 kbps is used as shown in Table 5Different numbers of similar unicast streams are pumpedfrom the video server to the receivers using a wireless linkat the last hopThe synthetic MOS of the streaming service atthe receivers decreases significantly as the number of unicaststreams increases as shown in Figure 15 This is because thejitter burstiness and packet loss rate increase as the numberof unicast streams in the network grows yielding the QoEdegradation at the receivers Note that if themulticast scheme

Table 5 Information of test video 2

Parameter ValueVideo title Call Me Maybe (MV)Length 199 secResolution 720 times 480 pixelsEncoding scheme MPEG-2 TSBit rate 1936 kbpsFrame rate 30 fpsGOP size 15Content size 46MB

is used only one video stream is required to be pumped intothe network no matter how many receivers exist That is theQoE of video streaming services is irrelevant to the numberof receivers if the multicast scheme is employed Obviouslyunder the scenario that multiple users watch the same videostream the multicast scheme performs much better than theunicast one in terms of QoE

Finally to investigate the impact of transmission distanceand obstacles on the QoE of wireless streaming services thewireless network scenario 4 plotted in Figure 16 is consideredand the second test video given in Table 5 is used The videostream is multicast to several wireless receivers at differentpositions in our labs No cross-traffic is injected into thenetwork No obstacles exist between the wireless AP andthe receivers at Locations 1 and 2 However the receiver atLocation 1 is much closer to the AP than the receiver atLocation 2 isThere are cupboards and a wall between the APand the receiver at Location 3 The receiver at Location 4 isoutside the door of our lab and is isolated from the AP by awallThe other receiver at Location 5 is also isolated from theAP by a wall and cupboards All receivers receive the samemulticast stream simultaneously and their individual MOSsare estimated using the developed QoEmonitoring programThe results are shown in Figure 17 where the MOSs of thereceivers located at Locations 1 and 2 are the best becauseno obstacle blocks the wireless signals for them while theMOSs of the receivers located at Locations 4 and 5 are theworst because there exist cupboards and a wall between themand the AP The MOS of the receiver located at Location 3is better than those located at Locations 4 and 5 because thereceiver at Location 3 is much closer to the AP than thoseat Locations 4 and 5 are Accordingly the distance and thenumber of obstacles between the WiFi AP and the receiversstrongly affect the QoE of wireless streaming services

5 Conclusions

In this work anAMP and a real-timeQoEmonitoring systemfor video streaming services arerealized The AMP design is

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 9: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

International Journal of Digital Multimedia Broadcasting 9

Server

10 Mbps 10 Mbps 10 Mbps

100 Mbps

Receiver 1

Receiver 2Cross-traffic

Figure 12 Experimental network scenario 2

Cross-traffic rate = 6 MbpsReceiver 2 (wired)Receiver 1 (wireless)

0

1

2

3

4

5

MO

S

30 60 90 120 150 180 0Time (sec)

Cross-traffic rate = 0 MbpsReceiver 2 (wired)Receiver 1 (wireless)

30 60 90 120 150 1800Time (sec)

0

1

2

3

4

5

MO

S

Figure 13 Comparison of QoEs of streaming services between wired and wireless receivers under different cross-traffic rates

Server

10 Mbps 10 Mbps 10 MbpsReceiver

Unicast streams

Figure 14 Experimental network scenario 3

implemented into the freeware VLC media player The QoEmonitoring system canmeasure several keyQoSmetrics suchas the packet loss rate underflow time ratio initial playoutdelay and the playout rates of a video streaming serviceand derive the overall QoE in real time Several experimentsare conducted in various scenarios including wired andwireless networks All numerical results demonstrate thatthe QoE monitoring system works well and the proposedAMP significantly improves the QoE of video streamingservices Using the real-time QoE monitoring system net-workcontent providers can find the network troubles andresolve them timely to maintain good QoEs for users In thefuture more advanced video codecs such as H264AVC andH265HEVCwill be implemented into the developed system

1 unicast stream3 unicast stream

5 unicast streams

0

1

2

3

4

5

MO

S

25 50 75 1000Time (sec)

Figure 15 Impact of number of unicast streams onMOS of wirelessreceivers

Conflicts of Interest

The authors declare that there are no conflicts of interestregarding the publication of this paper

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 10: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

10 International Journal of Digital Multimedia Broadcasting

8 m

10 m

Table

Door

Cupboard

Access point

Receiver

Figure 16 Experimental network scenario 4

Location 1Location 2Location 3

Location 4Location 5

25 50 75 1000Time (sec)

0

1

2

3

4

5

MO

S

Figure 17 Impact of transmission distances and obstacles on QoEof wireless streaming services

Acknowledgments

This work was supported by Ministry of Science and Tech-nology of Taiwan under Grant NSC101-2221-E-182-004 andGrant MOST106-2221-E-182-012-MY2

References

[1] Y Xiao X Du J Zhang F Hu and S Guizani ldquoInter-net protocol television (IPTV) the killer application for the

next-generation internetrdquo IEEECommunicationsMagazine vol45 no 11 pp 126ndash134 2007

[2] M Li and C-H Wu ldquoA cost-effective resource allocation andmanagement scheme for content networks supporting IPTVservicesrdquo Computer Communications vol 33 no 1 pp 83ndash912010

[3] S Park and S-H Jeong ldquoMobile IPTV Approaches challengesstandards and QoS supportrdquo IEEE Internet Computing vol 13no 3 pp 23ndash31 2009

[4] G Gardikis G Xilouris E Pallis and A Kourtis ldquoJointassessment of Network- and Perceived-QoS in video deliverynetworksrdquo Telecommunication Systems vol 49 no 1 pp 75ndash842012

[5] M Li and C-Y Lee ldquoA cost-effective and real-time QoE evalu-ation method for multimedia streaming servicesrdquo Telecommu-nication Systems vol 59 no 3 pp 317ndash327 2015

[6] M Li ldquoQoE-based performance evaluation for adaptive mediaplayout systemsrdquo Advances in Multimedia vol 2013 Article ID152359 2013

[7] M Mu J Ishmael W Knowles et al ldquoP2P-based IPTVservices Design deployment and QoE measurementrdquo IEEETransactions on Multimedia vol 14 no 6 pp 1515ndash1527 2012

[8] W-H Hsu and C-H Lo ldquoQoSQoE mapping and adjustmentmodel in the cloud-based multimedia infrastructurerdquo IEEESystems Journal vol 8 no 1 pp 247ndash255 2014

[9] Y Ju Z Lu D Ling XWenW Zheng andWMa ldquoQoE-basedcross-layer design for video applications over LTErdquoMultimediaTools and Applications vol 72 no 2 pp 1093ndash1113 2014

[10] J Lloret A Canovas J Tomas and M Atenas ldquoA networkmanagement algorithm and protocol for improving QoE inmobile IPTVrdquo Computer Communications vol 35 no 15 pp1855ndash1870 2012

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 11: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

International Journal of Digital Multimedia Broadcasting 11

[11] M A Hoque M Siekkinen J K Nurminen M Aalto and STarkoma ldquoMobile multimedia streaming techniques QoE andenergy saving perspectiverdquo Pervasive and Mobile Computingvol 16 pp 96ndash114 2015

[12] L Qian H Chen and L Xie ldquoSVM-based QoE estimationmodel for video streaming service over wireless networksrdquo inProceedings of the IEEE International Conference on WirelessCommunications Signal Processing (WCSP) Nanjing China2015

[13] Y Chen K Wu and Q Zhang ldquoFrom QoS to QoE A tutorialon video quality assessmentrdquo IEEE Communications Surveys ampTutorials vol 17 no 2 pp 1126ndash1165 2015

[14] ITU-TRec P910 ldquoSubjective video quality assessmentmethodsfor multimedia applicationsrdquo 2008

[15] D Saladino A Paganelli andM Casoni ldquoA tool formultimediaquality assessment in NS3 QoEMonitorrdquo SimulationModellingPractice andTheory vol 32 pp 30ndash41 2013

[16] M Schmitt S Gunkel P Cesar and P Hughes ldquoA QoE testbedfor socially-aware video-mediated group communicationrdquo inProceedings of the 2nd InternationalWorkshop on Socially-AwareMultimedia SAM2013 - Co-located with ACMMultimedia 2013pp 37ndash41 Spain October 2013

[17] S Jeong and H Ahn ldquoMobile IPTV QoSQoE monitoringsystem based on OMADM protocolrdquo in Proceedings of the 2010International Conference on Information and CommunicationTechnology Convergence ICTC 2010 pp 99-100 Republic ofKorea November 2010

[18] M Leszczuk M Hanusiak M C Q Farias E Wyckens andG Heston ldquoRecent developments in visual quality monitoringby key performance indicatorsrdquoMultimedia Tools and Applica-tions vol 75 no 17 pp 10745ndash10767 2016

[19] S Ickin M Fiedler K Wac P Arlos C Temiz and K MkochaldquoVLQoE Video QoE instrumentation on the smartphonerdquoMultimedia Tools and Applications vol 74 no 2 pp 381ndash4112014

[20] VideoLAN- VLC medial player official site httpwwwvideolanorg

[21] J Yang H Hu H Xi and L Hanzo ldquoOnline buffer fullnessestimation aided adaptive media playout for video streamingrdquoIEEE Transactions on Multimedia vol 13 no 5 pp 1141ndash11532011

[22] M Li T-W Lin and S-H Cheng ldquoArrival process-controlledadaptive media playout with multiple thresholds for videostreamingrdquoMultimedia Systems vol 18 no 5 pp 391ndash407 2012

[23] M Li C-L Yeh and S-Y Lu ldquoRealization of intelligent mediaplayer and QoE monitoring system for multimedia streamingservicesrdquo IEEE ICCP amp HSIC pp 1ndash4 2013

[24] P Seeling and M Reisslein ldquoVideo traffic characteristics ofmodern encoding standards H264AVC with SVC and MVCextensions and H265HEVCrdquoThe Scientific World Journal vol2014 Article ID 189481 16 pages 2014

[25] ITU-T Rec P800 ldquoMean opinion score (MOS) terminologyrdquo2003

[26] A Botta A Dainotti and A Pescape ldquoA tool for the generationof realistic network workload for emerging networking scenar-iosrdquo Computer Networks vol 56 no 15 pp 3531ndash3547 2012

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom

Page 12: Real-Time QoE Monitoring System for Video Streaming Services with Adaptive Media … · 2019. 7. 30. · (Transport Stream) packet, as shown in Figure , one can easilycalculatethenumber

International Journal of

AerospaceEngineeringHindawiwwwhindawicom Volume 2018

RoboticsJournal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Active and Passive Electronic Components

VLSI Design

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Shock and Vibration

Hindawiwwwhindawicom Volume 2018

Civil EngineeringAdvances in

Acoustics and VibrationAdvances in

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawiwwwhindawicom

Volume 2018

Hindawi Publishing Corporation httpwwwhindawicom Volume 2013Hindawiwwwhindawicom

The Scientific World Journal

Volume 2018

Control Scienceand Engineering

Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom

Journal ofEngineeringVolume 2018

SensorsJournal of

Hindawiwwwhindawicom Volume 2018

International Journal of

RotatingMachinery

Hindawiwwwhindawicom Volume 2018

Modelling ampSimulationin EngineeringHindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Chemical EngineeringInternational Journal of Antennas and

Propagation

International Journal of

Hindawiwwwhindawicom Volume 2018

Hindawiwwwhindawicom Volume 2018

Navigation and Observation

International Journal of

Hindawi

wwwhindawicom Volume 2018

Advances in

Multimedia

Submit your manuscripts atwwwhindawicom