Embed Size (px)
Citation preview
Copyright © 2017 Brightcove, Inc. All Rights Reserved.1
Content and Context-Aware EncodingYuriy Reznik, Brightcove, Inc.
DASH-IF workshop, Comcast media center, Centennial, CO
August 10, 2017
Copyright © 2017 Brightcove, Inc. All Rights Reserved.2
Single-rate encoding:
Encoding for ABR:
Key operations:
– ABR ladder design
– video processing
– encoding
Encoding for ABR
DecodeVideo
processingEncode
Mezzanine
video ES
Delivery
video ES
Targets
DecodeVideo
processingEncode
Mezzanine
video ES
Delivery
video ES 1
Video
processingEncode
Delivery
video ES 2
Video
processingEncode
Delivery
video ES 3
Video
processingEncode
Delivery
video ES n
Ladder of
ABR targets
Encode
request
… … …
Copyright © 2017 Brightcove, Inc. All Rights Reserved.3
VideoCloud architecture:
MUXPublisher Transcoder (Zencoder)
Analytics
Video players
Encoders
packager
packager
packager
…
p
RateTablet
TVTablet PC
MezzanineMezzanineMezzanine
Dashboard
Ingest
API
Back-end
engine
Job
request API
Cloud
Transcoder
Pipeline
bitrate=375,
res=480x270, …
bitrate=700,
res=768x432, …
bitrate=5800,
res=1920x1080,
…
Online Video Platform
Network & Usage
statistics, etc.
Analytics
engine
Copyright © 2017 Brightcove, Inc. All Rights Reserved.4
VideoCloud architecture:
NB: in OVP profile generator is aware of delivery context (anticipated network and usage statistics)!
MUXPublisher Transcoder (Zencoder)
Analytics
Video players
Encoders
packager
packager
packager
…
p
RateTablet
TVTablet PC
MezzanineMezzanineMezzanine
Dashboard
Ingest
API
Back-end
engine
Job
request APIbitrate=375,
res=480x270, …
bitrate=700,
res=768x432, …
bitrate=5800,
res=1920x1080,
…
ABR profile generation in the context of OVP
Analytics
engine
Profile
generator
(PROGEN)
Transcoder
Pipeline
Network & Usage statistics
Profile statistics
Copyright © 2017 Brightcove, Inc. All Rights Reserved.5
Apple HLS authoring specification: VideoCloud “High-Resolution” profile
VideoCloud “Default” profile:
Examples of “static” ABR encoding ladders
Copyright © 2017 Brightcove, Inc. All Rights Reserved.6
All previously shown examples are static encoding ladders
– they provide lists of resolutions and rates that are used for all content, sent to all networks
However, such approach fails to account for differences in characteristics of video content as well as network properties
– differences in video RD performance: – differences in PDFs of networks connected to different devices
A better approach is to design encoding ladders dynamically, accounting for characteristics of
– content content-aware encoding (aka per-title encoding)
– delivery context/model context-aware encoding
Static vs Dynamic encoding ladders
Source: Netflix, 2015 Source: Brightcove VideoCloud analytics, 2017
Copyright © 2017 Brightcove, Inc. All Rights Reserved.7
Quality-rate function 𝑄(𝑅): Quality at each encoding point: Probabilities of using each encoding:
Given a ladder of rates 𝑅1, … , 𝑅𝑛, quality-rate function 𝑄(𝑅), and network PDF 𝑝(𝑅), we can define:
– buffering probability: (probability that playback is not possible, even at lowest rate)
– average quality:
A quality-optimal ladder is set of rates 𝑅1∗, … , 𝑅𝑛
∗ , such that:
𝑄 𝑅1∗, … , 𝑅𝑛
∗ , 𝑝 = max𝑅min<𝑅1≤⋯≤𝑅𝑛 <𝑅max
𝑅1≤𝑅1,max
𝑄 𝑅1, … , 𝑅𝑛, 𝑝 .
Context-aware ladder design – the most basic look at the problem
𝑝 𝑅 < 𝑅1 = න0
𝑅1
𝑝(𝑅)𝑑𝑅
𝑄 𝑅1, … , 𝑅𝑛, 𝑝 =1
1 − 𝑝 𝑅 < 𝑅1𝑄 𝑅1 න
𝑅1
𝑅2
𝑝(𝑅)𝑑𝑅 +⋯+ 𝑄 𝑅𝑛 න𝑅𝑛
𝑅𝑚𝑎𝑥
𝑝(𝑅)𝑑𝑅
Copyright © 2017 Brightcove, Inc. All Rights Reserved.8
Optimal ladders: Content: 5 sources, 720p, H.264, SSIM Network: assumed model:
Optimization process:
Notes:
– optimal ladders are different for different sequences
– the number of ladder points may also be different, as easier to encode content reaches saturation point faster
– optimal encoding ladders may also be different for different target devices, networks, etc.
Examples of optimal ladders
Content n Ladder rates [kbps]Average quality
[SSIM]
Easy
1 120 0.8000
2 120, 530 0.9148
3 120, 361, 917 0.9319
4 120, 297, 619, 1565 0.9392
5 120, 250, 459, 801,1851 0.9430
Easy-
medium
1 338 0.8000
2 338, 893 0.8805
3 338, 675, 1682 0.9018
4 338, 563, 926, 2010 0.9106
5 338, 514, 759, 1193, 2287 0.9152
Medium
1 428 0.8000
2 428, 1076 0.8676
3 428, 776, 1832 0.8877
4 428, 664, 1055, 2152 0.8955
5 428, 616, 882, 1404, 2484 0.8999
Medium-
difficult
1 508 0.8000
2 508, 1315 0.8648
3 508, 869, 1968 0.8847
4 508, 756, 1194, 2293 0.8924
5 508, 703, 987, 1575, 2633 0.8969
Difficult
1 815 0.8000
2 815, 1898 0.8549
3 815, 1301, 2399 0.8687
4 815, 1127, 1763, 2791 0.8754
5 815, 1049, 1446, 2101, 3058 0.8792
𝑄 𝑅1∗, … , 𝑅𝑛
∗ = max𝑅min<𝑅1≤⋯≤𝑅𝑛 <𝑅max
𝑅1≤𝑅1,max
𝑄 𝑅1, … , 𝑅𝑛
𝑄 𝑅1, … , 𝑅𝑛 =𝑝 𝑅1≤𝑅<𝑅2 𝑄 𝑅1 +⋯+ 𝑝(𝑅𝑛≤𝑅<𝑅max)𝑄 𝑅𝑛
1−𝑝 𝑅<𝑅1, where 𝑝 𝛼 ≤ 𝑅 < 𝛽 = 𝛼
𝛽𝑝(𝑅)𝑑𝑅
where: 𝑄 𝑅 - quality rate function, and 𝑝(𝑅) – network PDF.
Content 𝑛 Rates [Kbps]Quality at
rate 1
Quality at
rate n
Average
quality
Easy
1 120 0.8000 0.8000 0.8000
2 120, 530 0.8000 0.9363 0.9148
3 120, 361, 917 0.8000 0.9549 0.9319
4 120, 297, 619, 1565 0.8000 0.9656 0.9392
Easy-
medium
1 338 0.8000 0.8000 0.8000
2 338, 893 0.8000 0.9144 0.8805
3 338, 675, 1682 0.8000 0.9501 0.9018
4 338, 563, 926, 2010 0.8000 0.9568 0.9106
5 338, 514, 759, 1193, 2287 0.8000 0.9610 0.9152
Medium
1 428 0.8000 0.8000 0.8000
2 428, 1076 0.8000 0.9047 0.8676
3 428, 776, 1832 0.8000 0.9356 0.8877
4 428, 664, 1055, 2152 0.8000 0.9422 0.8955
5 428, 616, 882, 1404, 2484 0.8000 0.9474 0.8999
6 428, 580 778, 1075, 1691, 2728 0.8000 0.9504 0.9028
Medium-
difficult
1 508 0.8000 0.8000 0.8000
2 508, 1315 0.8000 0.9105 0.8648
3 508, 869, 1968 0.8000 0.9355 0.8847
4 508, 756, 1194, 2293 0.8000 0.9428 0.8924
5 508, 703, 987, 1575, 2633 0.8000 0.9485 0.8969
6 508, 665, 872, 1196, 1836, 2847 0.8000 0.9514 0.8998
Difficult
1 815 0.8000 0.8000 0.8000
2 815, 1898 0.8000 0.8977 0.8549
3 815, 1301, 2399 0.8000 0.9142 0.8687
4 815, 1127, 1763, 2791 0.8000 0.9231 0.8754
5 815, 1049, 1446, 2101, 3058 0.8000 0.9278 0.8792
6 815, 1003, 1280, 1736, 2378, 3272 0.8000 0.9311 0.8817
Copyright © 2017 Brightcove, Inc. All Rights Reserved.9
There are 2 natural limits:
1. quality at highest rate becomes sufficiently good:𝑄 𝑅𝑛 ≥ 𝑄max
2. average quality is within 𝑿 percent of what is achievable:
𝜉 =𝑄∗ 𝑅1, 𝑝 − 𝑄 𝑅1, … , 𝑅𝑛, 𝑝
𝑄∗ 𝑅1, 𝑝⋅ 100 ≤ X,
where 𝑄∗ 𝑅1, 𝑝 = lim𝑛→∞
𝑄 𝑅1, … , 𝑅𝑛, 𝑝 =1
1−𝑝 𝑅<𝑅1𝑅1𝑅𝑚𝑎𝑥 𝑝(𝑅)𝑄 𝑅 𝑑𝑅
is the average quality achievable for a given content and network
given infinite number of ladder points.
In practice, the threshold X can be 1..5%.
Both limits are complementary:
Sufficient number of ladder points (𝑛) Content 𝑛 Rate ladderQuality at
rate 1
Quality at
rate n
Average
quality𝑸∗ 𝑹𝟏, 𝒑 𝝃
Easy
1 120 0.8000 0.8000 0.8000 0.9542 16.168
2 120, 530 0.8000 0.9363 0.9148 0.9542 4.128
3 120, 361, 917 0.8000 0.9549 0.9319 0.9542 2.342
4 120, 297, 619, 1565 0.8000 0.9656 0.9392 0.9542 1.573
5 120, 250, 459, 801,1851 0.8000 0.9679 0.9430 0.9542 1.173
Easy-
medium
1 338 0.8000 0.8000 0.8000 0.9316 14.126
2 338, 893 0.8000 0.9144 0.8805 0.9316 5.485
3 338, 675, 1682 0.8000 0.9501 0.9018 0.9316 3.189
4 338, 563, 926, 2010 0.8000 0.9568 0.9106 0.9316 2.245
5 338, 514, 759, 1193, 2287 0.8000 0.9610 0.9152 0.9316 1.752
6 338, 484, 674, 950, 1514, 2579 0.8000 0.9645 0.9182 0.9316 1.430
Medium
1 428 0.8000 0.8000 0.8000 0.9156 12.634
2 428, 1076 0.8000 0.9047 0.8676 0.9156 5.249
3 428, 776, 1832 0.8000 0.9356 0.8877 0.9156 3.052
4 428, 664, 1055, 2152 0.8000 0.9422 0.8955 0.9156 2.197
5 428, 616, 882, 1404, 2484 0.8000 0.9474 0.8999 0.9156 1.720
6 428, 580 778, 1075, 1691, 2728 0.8000 0.9504 0.9028 0.9156 1.408
7 428, 557, 715, 927, 1268, 1912, 2907 0.8000 0.9523 0.9047 0.9156 1.194
Medium-
difficult
1 508 0.8000 0.8000 0.8000 0.9133 12.405
2 508, 1315 0.8000 0.9105 0.8648 0.9133 5.295
3 508, 869, 1968 0.8000 0.9355 0.8847 0.9133 3.128
4 508, 756, 1194, 2293 0.8000 0.9428 0.8924 0.9133 2.282
5 508, 703, 987, 1575, 2633 0.8000 0.9485 0.8969 0.9133 1.785
6 508, 665, 872, 1196, 1836, 2847 0.8000 0.9514 0.8998 0.9133 1.468
7 508, 642, 808, 1037, 1423, 2075, 3035 0.8000 0.9536 0.9018 0.9133 1.248
8 508, 624, 763, 941, 1199, 1636, 2282, 3196 0.8000 0.9553 0.9033 0.9133 1.085
Difficult
1 815 0.8000 0.8000 0.8000 0.8938 10.502
2 815, 1898 0.8000 0.8977 0.8549 0.8938 4.355
3 815, 1301, 2399 0.8000 0.9142 0.8687 0.8938 2.816
4 815, 1127, 1763, 2791 0.8000 0.9231 0.8754 0.8938 2.061
5 815, 1049, 1446, 2101, 3058 0.8000 0.9278 0.8792 0.8938 1.635
6 815, 1003, 1280, 1736, 2378, 3272 0.8000 0.9311 0.8817 0.8938 1.355
7 815, 972, 1186, 1512, 1982, 2603, 3446 0.8000 0.9335 0.8835 0.8938 1.158
8 815, 950, 1124, 1370, 1727, 2195, 2790, 3591 0.8000 0.9353 0.8848 0.8938 1.011
9 815, 934, 1081, 1276, 1553, 1920, 2379, 2949, 3716 0.8000 0.9367 0.8858 0.8938 0.897
10 815, 921,1048, 1210, 1430, 1724, 2092, 2539, 3087, 3825 0.8000 0.9379 0.8866 0.8938 0.807
Ladders generated in
previous experiment
Sub-optimality
decays as ~1/n
For easy content 𝑄 𝑅𝑛limit can reduce n
For complex
content and slow
network this limit
is not effective
Copyright © 2017 Brightcove, Inc. All Rights Reserved.10
Thus far we shown that:
– ladder design can be formulated as a math problem
• the problem is well known, as are methods for solving it
– proposed approach allows finding optimal ladder parameters
based on RD models of the source as well as network PDFs
– proposed approach also allows us to understand performance
limits of ABR systems (performance with infinite number of rate
points, buffering ratio, etc.).
However, to design a practical system, we must also:
– consider multiple devices
– consider multiple video resolutions
– consider multiple networks
– consider usage statistics (devices, content length)
– consider publisher preferences in delivery to different devices
– account for differences in quality on different devices
– factor in buffering probability, start time, etc.
– ...
Long list, but it is all formalizable and reducible to practice
– moreover, there an operational practical solution, doing all this!
Towards practical system
𝑄 𝑅1∗, … , 𝑅𝑛
∗ , 𝑝 = max𝑅1, … , 𝑅𝑛
𝑄 𝑅1, … , 𝑅𝑛, 𝑝
Copyright © 2017 Brightcove, Inc. All Rights Reserved.11
Available as CAE ingest profiles in VideoCloud. Example VC job request for CAE encoding
There are 3 standard CAE profiles, plus advanced users can define
their own, by creating custom profile with “dynamic_profile_options”
object. This way, they can control:
– number of renditions
– video framerates & resolutions being used
– video codecs, profiles, GOP length
– minimum quality level, etc.
Brightcove Context Aware Encoding
{
"id": "1234567890",
"version": 1,
"name": "custom-cae-profile",
"dynamic_origin": {
"dynamic_profile_options": {
"min_renditions": 3,
"max_renditions": 6,
"max_resolution": {
"width": 1920,
"height": 1080
},
"max_bitrate": 4200,
"max_first_rendition_bitrate": 400,
"max_frame_rate": 30,
"keyframe_rate": 0.5,
"select_baseline_profile_configuration": true,
"min_ssim": 0.95,
"video_configurations": [
{"width": 1280, "height": 720},
{"width": 960, "height": 540},
{"width": 640, "height": 360}
]
},
}
}
Copyright © 2017 Brightcove, Inc. All Rights Reserved.12
Example of static vs CAE-generated profile
In this example:
– CEA saves bandwidth: 2366kbps vs 3800kbps at 1080p => 36.77% saving!
– CAE saves number of renditions & transcoding costs: 4 vs 8
– CAE delivers more pixels / better quality at low rates: 432p @ 777kbps vs 360p @ 900kpbs
Brightcove Context Aware Encoding
https://www.brightcove.com/en/blog/2017/05/context-aware-encoding-improves-video-quality-while-cutting-costs
Copyright © 2017 Brightcove, Inc. All Rights Reserved.13
Contact:
Yuriy Reznik
Brightcove, Inc
To request trial of Brightcove ABR profile generator, email:
Thank you!
