Upload
chris-adamson
View
188
Download
0
Embed Size (px)
Citation preview
Fall Premieres: Media for Apple Platforms in
Late 2017Chris Adamson • @invalidname
CocoaConf Next Door • June, 2017
Agenda
• HEVC / H.265
• Apple Music API
• Changes to Existing Frameworks
• HTTP Live Streaming
• Developments outside of Apple
HEVC / H.265
H.265• "High Efficiency Video Codec"
• Approximately double the data compression (or much better quality) of H.264 at same bitrate
• Expected use-case: 4K video streaming
• Work began in 2010, standard approved in 2013
• Versions 2, 3, and 4 approved since then
Standardized Mac / iOS support Major Uses
MPEG-1 1993 QuickTime 2.5 Video CD
MPEG-2 1996 QuickTime 6 (optional) DVD, Digital TV
H.264 2003QuickTime 7
(MacOS X 10.4), iPhone OS 1
Blu-Ray, Digital TV, Streaming
H.265 2013 macOS High Sierra, iOS 11 ???
Patent Encumbrances• HEVC contains technology covered by hundreds
(thousands?) of patents
• Patent Pools: MPEG-LA, HEVC Advance, Velos Media
• Don't even cover everything (Technicolor SA)
• Much costlier than H.264 licenses (max $25M/yr [MPEG-LA] $40M/yr [HEVC Advance])
VP9• Google-supported competitor to HEVC
• Royalty-free
• Only patents (🤞) are held by Google
• Supported by 75% of browsers (desktop + mobile)
• Used extensively by YouTube for delivery to non-Apple platforms
AV1
• Royalty-free codec from Alliance for Open Media (AV1 means "AOMedia Video 1")
• Founding members: Amazon, Cisco, Intel, Google, Microsoft, Mozilla, Netflix
• Successor to VP9
• Planned for release in 2017
http://www.streamingmediaglobal.com/Articles/ReadArticle.aspx?ArticleID=118062
Then I asked where HEVC fit in. Aaron explained that Netflix currently deployed HEVC primarily on Smart TVs, and saw the codec as integral to its HDR strategy. For computers and mobile, however, H.264 and VP9 are Netflix's primary codecs, and the focus of most of its current research, which will soon include AV1.
I asked about the future of HEVC given the imminent release of AV1, and the fact that Netflix was a founding member of the Alliance for Open Media. Ronca laughed, explaining that Netflix was still encoding in VC1 format for some older platforms, and that HEVC is important for their UHD/HDR experience, though he noted that the HEVC license uncertainty is an ongoing concern.
NAB 17 Codec Roundup
Jan Ozer HEVC will do well in broadcast, no doubt. Still not available in any browser, iOS, and Netflix prefers VP9/AV1 over HEVC for Android. VP9 gets you most browsers and many smart TVs and OTT boxes (like Roku 4), so it's the smart money UHD codec if you don't need HDR.
http://www.streamingmediaglobal.com/Articles/ReadArticle.aspx?ArticleID=118062
High Dynamic Range (HDR)
http://www.streamingmedia.com/Articles/Editorial/Featured-Articles/HDR-The-More-Bits-the-Better-118643.aspx
HDR: The More Bits, the Better Apple's new Macs feature 10-bit HEVC decoding for HDR video viewing. To understand why that's important, it's worth a look at an old technology—film cameras
By Tim Siglin
Posted on June 6, 2017
The aperture itself determines how deep or shallow the focus will be, which is where we get the terminology for depth of field that's now being tossed about for consumer smartphones. The Apple iPhone 7 "depth effect" feature makes use of two lenses to create a digital version of depth of field.
HEVC in code• AVVideoCodecType.hevc
• AVAssetExportPresetHEVC1920x1080, …3840x2160
• AVOutputSettingsPreset.hevc1920x1080, hevc3840x2160
• kCMVideoCodecType_HEVC
• kVTCompressionPropertyKey_ProfileLevel values: Profile0_Level10, Profile0_Level45, Profile3_Level45
Apple Music API
AppleMusic TODOs
• Web API (MusicKit) for getting Apple Music artists/albums/playlists, getting user's preferences
• Working with Store Kit to actually access those items
• Playing with Media Player
MusicKit web service
• APIs: artists, albums, songs, playlists, search, charts, recommendations
• Must have a developer token to call
• Must get a user token to access personalized features
https://api.music.apple.com/{version}/catalog/{storefront}/{api}?[params]
Subscription UI
• If user isn't an Apple Music subscriber, you can show the subscription view, optionally showing specific content
• Can embed your affil token
Playback
• MPMediaPlayer
• Can create MPMediaItem (local songs), MPMusicPlayerStoreQueueDescriptor (Apple Music songs), MPMusicPlayerPlayParametersQueueDescriptor (personalized collections)
Changes to Existing Frameworks
AVAudioEngine• Offline Rendering
• Auto shutdown
• Completion callbacks (.dataConsumed, .dataRendered, .dataPlayedBack)
• Important: AUGraph (Audio Toolbox) to be deprecated in 2018
Also
• watchOS recording
• AUAudioUnit "preferred view configuration" API
• Support for FLAC and Opus audio codecs
• Spatial audio formats
HTTP Live Streaming
HLS updates• HLS approved for an actual RFC
• Format war with MPEG-DASH continues
• HEVC supported (of course), fMP4 only (not .ts)
• New subtitle standard ISMC1
• Nice but esoteric features (variables in playlists)
• FairPlay is still very much a thing. 🤐
Beyond Apple Park
ReplayKit
• Screen Capture + Livestreaming API
RTMP HLS
https://github.com/shogo4405/GPUHaishinKit.swift
https://github.com/shogo4405/GPUHaishinKit.swift
Fall Premieres: Media for Apple Platforms in
Late 2017Chris Adamson • @invalidname
CocoaConf Next Door • June, 2017