View
2
Download
0
Category
Preview:
Citation preview
POI360Panoramic Mobile Video Telephony over LTE Cellular Networks
Xiufeng Xie
University of Michigan-Ann Arbor
Xinyu Zhang
University of California San Diego
CoNEXT 2017
Background: 360° Video for VR
Sphere view Panoramic frame360° camera
time
360° video for VR
30FPS
360° Video + Video Telephony = Interactive VR!
Mobility Coverage
Challenges & Solution Spaces
Huge VR Traffic Load Calls for Compression
• 360° frame High VR stream bitrate: ▪ 10~20Mbps for 4K MP4 format
▪ Exceed LTE UL (5Mbps)/DL (12Mbps) bandwidth
• Compression based on region of interest (ROI)Human eye can only see part of 360°
Quality
Spatial position
Region-of-Interest (ROI)
Compress unseen parts
Challenge 1: Compression Fails over LTE
• Does not matter if RTT < VR frame interval (e.g., 33ms for 30fps)▪ Typical wireline network✓
• LTE has unstable RTT (5~500ms) depending on traffic & channel
High quality
Lowquality
Lowquality
Compressed frame
High quality
Lowquality
Lowquality
t
ROI
Update ROI knowledge
ROI change ROI quality recover
Lower ROI quality for one RTT
VR stream compressed with new ROI
User-perceived VR quality always fluctuates over LTE
ROI Prediction?
• Predict the ROI by reviewer’s motion?▪ Oculus measurements [1]:
• Avg. head angular speed: 60 Τ° 𝑠
• Avg. head angular acceleration: 500 Τ° 𝑠2
• Head can stop rotation within 120ms
▪ Typical end-to-end LTE video latency can be more than 500ms
Prediction: 120msNeed: 500ms
[1] S.M.LaValle, A.Yershova, M.Katsev, and M.Antonov, “Head Tracking for The Oculus Rift,” in Robotics and Automation (ICRA), 2014 IEEE International Conference on, 2014.
ROI prediction does not work on LTE networks!
Solution: Adaptive Compression
• Responsive ROI update Aggressive▪ Maximize the user-perceived quality
• Irresponsive ROI update Conservative▪ Guarantee the stability of VR quality
Conservative
Aggressive
Smooth quality drop
Sharp quality dropV
ideo
qu
alit
y
Spatial position
ROI center
…
Adaptive compression
Many ways to redistribute the quality
Challenge 2: Irresponsive Rate Control
• Insufficient VR rate control responsiveness
Sluggish loop: large RTT over LTE
Request suitable rate
Measure network-layer statistics
Network
Conventional video rate control
VR users: sensitive to video freezes in immersive environment
LTE network: highly dynamic bandwidth
VR stream LTE uplink
Solution: Cellular Link-Informed Adaptation• Cellular link info as congestion indicator
▪ LTE uplink: typical bottleneck for mobile VR telephony
▪ Diagnostic interface: status of UL firmware buffer
Uplink congestion control based on UL buffer status
Network
End-to-end congestion control
Shortcut: shorter adaptation path better responsiveness
Challenge 3: UL Bandwidth Underutilization
• Existing rate control: unaware of this unique feature▪ Buffer left empty (0 throughput) for 40% of
time! ▪ UL throughput << available bandwidth
Video data UL throughput
LTE uplink resource scheduling:UL throughput depends on its own buffer level
LTE UL firmware buffer
Before UL congestion, higher buffer level higher uplink rate
Solution: Adapt to UL Buffer Level
• Learn relation between UL throughput & buffer level
• Push firmware buffer level to the “sweet” region▪ Sweet region: maximize throughput without congestion
• Buffer level too high: slow down traffic to avoid congestion
• Buffer level too low: speed up traffic to exploit bandwidth
POI360 System Design
Design Overview
360° Cam
Firmware Buffer
Buffer level
Buffer Aware Rate Control
RTP traffic
ROIAdaptive Spatial Compression
Compressed VR stream
Viewer
Cellular uplinkSender
Adaptive Spatial Compression
• Adapt compression mode ▪ Balance ROI quality and stability of ROI
quality
• Design:• Switch mode following ROI update responsiveness
• Responsiveness metric: T3-T1 (duration of lower ROI quality)
T2: sender updates ROI knowledgeT3: ROI quality recovered
Conservative
AggressiveVideo quality
Spatial position
T1: ROI change
Buffer Aware Rate Control
▪ Cross-layer design• Learn buffer’s “sweet” region
• PHY buffer level too high reduce RTP & video bitrate
• PHY buffer level too low increase RTP & video bitrate
Video bitrate Application layer
H.264 Encoding
Packet Pacer
RTP bitrate Transport layer
Compressed frame
PHY bitrate
UL Firmware Buffer
Physical layer
Rate Control
PHY buffer level
Implementation
Live stream 360° video
VR player
LTE phone
Client’s ROI
QXDM
Diag. interface
Evaluation
Micro-benchmark Evaluation
• Validate VR compression design
• Benchmark algorithm: ▪ CMU--Conduit (crop & send ROI)
▪ Facebook--Pyramid encoding
ROI quality (PSNR)
Video-frame-level delay
ROI quality stability
11~13dB of improvement
Reduce delay by 15%
Reduce variation by 5X
Micro-benchmark Evaluation
• Validate our UL buffer-based rate control design▪ Compare with Google Congestion Control (GCC, default rate control of
Google Hangouts & Facebook Messenger)
▪ Our rate control FBCC keeps UL buffer level in the “sweet” region (green) for most of the time
Low usage High usage Overuse (saturation)
System-Level Test
• Test POI360 system under various network conditions▪ Different LTE background traffic load
▪ Different physical channel quality
▪ Different mobility level
• Performance metrics▪ Smoothness
• Video freezing ratio
▪ Quality• Frame-level PSNR
• Mean Opinion Score(MOS)
Different Background Traffic Load• Light LTE background traffic load (early morning)
▪ 1% video freeze
• Heavy LTE background traffic load (noon)▪ 4% video freeze & 2dB PSNR drop
▪ Majority of the frames have either excellent or good quality
PSNR & Video freezing ratio MOS
Different Physical Channel Quality
• Test at places with different channel quality▪ Weak (-115dB RSS); Moderate (-82dB RSS); Strong (-73dB RSS)
▪ Better channel: higher PSNR & MOS, less video freezes
▪ Even the weak channel achieves <3% video freezes
PSNR & Video freezing ratio MOS
Different Mobility Level
• Test under 3 different mobility levels▪ Slow (15mph); urban driving (30mph); highway (50mph)
▪ PSNR & MOS drop with higher mobility. But still have good or excellent quality even under 50mph mobility
▪ More freezes with high mobility: 1% for slow driving, 7% for urban driving. Comparable to legacy non-360 LTE video chat
PSNR & Video freezing ratio MOS
POI360 Summary
• Unique challenges when 360° VR video meets LTE▪ Long RTT of LTE breaks spatial VR compression
▪ Heavy VR traffic load
▪ Low cellular bandwidth utilization
• POI360: the first adaptive 360° VR compression ▪ Adapt compression strategy based on network condition
▪ Achieve balance between traffic load & smoothness
▪ Leverage cellular statistics to enable responsive rate control
• Other works in cellular network-informed mobile applications
* “Accelerating Mobile Web Loading Using Cellular Link Information”,
Xiufeng Xie, Xinyu Zhang, Shilin Zhu, ACM MobiSys’17
* “piStream: Physical Layer Informed Adaptive Video Streaming Over LTE”,
Xiufeng Xie, Xinyu Zhang, Swarun Kumar, Li Erran Li, ACM MobiCom’15
Thank you!
Q & A
Recommended