View
147
Download
0
Category
Preview:
Citation preview
F.LIVEShannon Chen (cchen116@Illinois.edu),
Zhenhuan Gao, and Klara NahrstedtUniversity of Illinois at Urbana-Champaign
Towards Interactive Live Broadcast Free-viewpoint TV Experience
This material is based in part upon work supported by the National Science Foundation (NeTS-0520182)
FREE-VIEWPOINT TV (FTV)
Multi-Lens Capturing
Free-Viewpoint Viewing
DESIRED FTV FEATURES• Interactive• Live• Broadcasting
DESIRED FTV FEATURES• Interactive
• Live• Broadcasting
*beep*
DESIRED FTV FEATURES• Interactive
• Live• Broadcasting
DESIRED FTV FEATURES• Interactive• Live
• Broadcasting
DESIRED FTV FEATURES• Interactive• Live
• Broadcasting
DESIRED FTV FEATURES• Interactive• Live• Broadcasting
1000-scale audience group
EXISTING FTV DELIVERY
FRAMEWORKS• Type-1: View chosen by content provider• EyeVision System [Kanade ‘01] used in Super Bowl
XXXV
Editing DecodeAggregateCapture
Capture
Capture
Encode
Director
Viewpoint decision
Audience
• Interactive• Live
EXISTING FTV DELIVERY
FRAMEWORKS• Type-2: Aggregated stream bundle• Nagoya System [Tanimoto ‘12]
• Interactive• Live
DecodeAggregateCapture
Audience
CaptureViewpoint decision
Capture
Encode
OUR SOLUTION: F.LIVE• View-based delivery framework
Audience 1
Viewpoint decisions
Renderer
Audience 2
Renderer
Decode
Decode
Capture
Capture
Capture
Encode Transmit
Transmit
TransmitEncode
Encode
Session manager
Transmission assignment
• Interactive• Live • Broadcasting ?
OUR SOLUTION: F.LIVE• No aggregation → distributed entities• P2P sharing among audience• Pub/Sub Model
Session manager
OUR SOLUTION: F.LIVE• No aggregation → distributed entities• P2P sharing among audience• Pub/Sub Model
Session manager
Publishers
OUR SOLUTION: F.LIVE• No aggregation → distributed entities• P2P sharing among audience• Pub/Sub Model
Session manager
Broker
OUR SOLUTION: F.LIVE• No aggregation → distributed entities• P2P sharing among audience• Pub/Sub Model
Session manager
Subscribers
OUR SOLUTION: F.LIVE
Session manager
View decisions
OUR SOLUTION: F.LIVE
Session manager
Transmission assignments
OUR SOLUTION: F.LIVE
Session manager
OUR SOLUTION: F.LIVE• No aggregation → distributed entities• P2P sharing among audience• Pub/Sub Model
S
SS
Physical proximity
• Broadcasting
CHALLENGES• Interactive ← Synchronization delay• Live ← Content freshness• Broadcast ← Producer bandwidth
CHALLENGES
R
CHALLENGES• Synchronization delay• Content freshness
Buffer for stream A
Buffer for stream BCam
B
Cam A
Audience site
1
1
CHALLENGES• Synchronization delay• Content freshness
Buffer for stream A
Buffer for stream BCam
B
Cam A
Audience site
1
2
2
1
CHALLENGES• Synchronization delay• Content freshness
Buffer for stream A
Buffer for stream BCam
B
Cam A
Audience site
1
12
3
3
2
Wait
CHALLENGES• Synchronization delay• Content freshness
Buffer for stream A
Buffer for stream BCam
B
Cam A
Audience site
1
12
2
3
4
4
3
CHALLENGES• Synchronization delay• Content freshness
Buffer for stream A
Buffer for stream BCam
B
Cam A
Audience site
1
12
2
3
4
4
3
Rendering
CHALLENGES• Synchronization delay• Content freshness
Buffer for stream A
Buffer for stream BCam
B
Cam A
Audience site
1
12
2
3
4
4
3
Rendering
Propagation delay (i.e., frame elapse = freshness)
Synchronization delay
CHALLENGES• Synchronization delay• Content freshness• Producer Bandwidth
CHALLENGES• Synchronization delay• Content freshness• Producer Bandwidth
CHALLENGES• Synchronization delay• Content freshness• Producer Bandwidth
Gbps
FOREST PLANNING CHALLENGES
• If we have all the subscription information at the beginning of forest planning , an optimal planning is not a hard problem
• However, subscriptions are dynamic• P2P churn• View change “Forest
adaptation”Initial forest construction
Cam B
Cam A
Cam C
FOREST PLANNING CHALLENGES
• If we have all the subscription information at the beginning of forest planning , an optimal planning is not a hard problem
• However, subscriptions are dynamic• P2P churn• View change “Forest
adaptation”Initial forest construction
Cam B
Cam A
Cam C
Audience join
Bandwidth?Freshness?
FOREST PLANNING CHALLENGES
• If we have all the subscription information at the beginning of forest planning , an optimal planning is not a hard problem
• However, subscriptions are dynamic• P2P churn• View change “Forest
adaptation”Initial forest construction
Cam B
Cam A
Cam C
Audience leave
FOREST PLANNING CHALLENGES
• If we have all the subscription information at the beginning of forest planning , an optimal planning is not a hard problem
• However, subscriptions are dynamic• P2P churn• View change “Forest
adaptation”Initial forest construction
Cam B
Cam A
Cam C
Audience leave
3 4
FOREST PLANNING CHALLENGES
• If we have all the subscription information at the beginning of forest planning , an optimal planning is not a hard problem
• However, subscriptions are dynamic• P2P churn• View change “Forest
adaptation”Initial forest construction
Cam B
Cam A
Cam C
View change
FOREST PLANNING CHALLENGES
• If we have all the subscription information at the beginning of forest planning , an optimal planning is not a hard problem
• However, subscriptions are dynamic• P2P churn• View change “Forest
adaptation”Initial forest construction
Cam B
Cam A
Cam C
View change 3 3
FOREST PLANNING CHALLENGES
• If we have all the subscription information at the beginning of forest planning , an optimal planning is not a hard problem
• However, subscriptions are dynamic• P2P churn• View change
We do not aim for optimization (no tearing down trees) Heuristics that deal with new batch of subscription
requests (audience join/leave/view change) Details on initial construction and adaptation
algorithms (w/ pseudocodes) can be found in the paper [Shannon Chen et al. INFOCOM’16]
“Forest adaptation”
Initial forest construction
EVALUATION• Simulation settings
• Single TV-studio performer site
• Network capability of audience sites: [Netmap]• In/out-bound bandwidth, site-to-site propagation delay
• Simulate new subscription requests when there are 0 to 100,000 audiences in the session
High-resolution cameras
Moderate 100-camera array
# of cameras 16 30 100Camera framerate 30 FPS 30 FPS 30 FPSCamera bitrate 12 Mbps (HDTV) 6 Mbps 2 Mbps (SDTV)
EVALUATION METRICS• Synchronization delay (Interactive)• Content freshness (Live)• Producer bandwidth (Broadcast)
SYNCHRONIZATION DELAY
• Unstable at first: not many candidates for newly joined/rejoined audience to find a group of sources with similar propagation delays
• Delay for handling new coming subscriptions during application session is in 100ms-scale in stable state
020
040
060
080
010
000
200400600800
1000High-Res Setting
Audience group size0
200
400
600
800
1000
0200400600800
1000Moderate Setting
Audience group size0 5000 100000
200400600800
1000100-Camera Setting
Audience group size
Sync
del
ay
(ms)
Sync
del
ay
(ms)
Sync
del
ay
(ms)
PRODUCER BANDWIDTH
CONSUMPTION
• P2P sharing restricts the growth of bandwidth consumption
• Outbound bandwidth requirement is well-manageable by Gbps infrastructure
0 200 400 600 80010000
250
500High Bitrate Setting
Audience group size
Tota
l pro
duce
r bi
trate
(Mbp
s)
0 200 400 600 80010000
250
500Moderate Setting
Audience group size
Tota
l pro
duce
r bi
trate
(Mbp
s)0 2500 50000
250
500100-Camera Setting
Audience group size
Tota
l pro
duce
r bi
trate
(Mbp
s)
CONTENT FRESHNESS
• > 50% audience have higher-than-average elapses
• But the tree structure makes the elapse grows sub-linearly• Max elapse < 4.5 seconds
(compare: CBS TV network’s time elapse is 5 sec)
COMPARE TO OTHER FTV FRAMEWORKS
Editing DecodeAggregateCap
Cap
Cap
Encode
Audience
DecodeAggregateCap
Audience
CapViewpoint decision
Cap
Encode
Type-1: customized content
Type-2: aggregated content
Viewpoint decision
MVC
COMPARE TO OTHER FTV FRAMEWORKS
020406080
100120140160
Band
widt
h co
nsum
ptio
n of
aud
ienc
e sit
e (M
pbs)
High-Res Moderate 100-CamerasType-1 (dash)View-based (solid)
High-Res Moderate 100-CamerasType-2 (stripe); View-based (solid)
Producer site bandwidth consumption
Audience site bandwidth consumption
CONCLUSION• We propose a new FTV content delivery framework
which aims at co-existence of three desired features• Interactive• Live• Broadcasting
• Result of large-scale simulation shows the proposed F.Live framework with view-based delivery chain achieves• Interactive response time in 100ms-scale• Acceptable content freshness by TV industry standard• Feasible bandwidth consumption while sustaining
1000-scale audience group
THANK YOU
Recommended