ExoPlayer for Application Developers

Google Developers Summit - April 28

Hassan Abid - GDE

Contents

● Media Playback in Android

● ExoPlayer Overview

● Adaptive Media Playback Protocols and Demo

● Content Protection

2

Terms

DASH : Dynamic Adaptive Streaming over HTTP

HLS : Http Live Streaming

DRM : Digital Rights Management

MPD : Media Presentation Description

ABR : Adaptive Bitrate Streaming

3

Media Playbacks in Android

● VideoView [link]

● MediaPlayer & SurfaceView [link]

● ExoPlayer [link]

4

It supports features not currently provided by MediaPlayer, including DASH, SmoothStreaming and Common Encryption.

5

ExoPlayer is an open source, application level media player built on top of Android’s low level media APIs (Mediacodec and MediaExtractor). Google is already using it for Youtube and Movies app

Pros

● Supports DASH,HLS, Smooth-Streaming …

● DRM protected content

● Customizable

● Easily Updatable

● Few device specific issues

6

Cons

● ExoPlayer’s standard audio and video components rely on Android’s MediaCodec API, which was released in Android 4.1 Jelly Bean (API level 16). Hence they do not work on earlier versions of Android.

7

Component Diagram

FrameworkSampleSource

ExoPlayer (Application Level)

Video/Audio/Text TrackRenderers

Media CodecMedia Extractor

Android Framework

BufferingNetworking Extraction Decoding

Audio Track

8

Rendering

Interfaces● TrackRenderer

○ MediaCodecVideoTrackRenderer

○ MediaCodecAudioTrackRenderer

○ TextTrackRenderer

● SampleSource○ ExtractorSampleSource (Mp4)

○ ChunkSampleSource (Dash, Smooth Streaming)

○ HlsSampleSource (HLS)

● DataSource○ DefaultUriDataSource

○ AssetDataSource9

Usage -MP4

DefaultUriDataSource ExtractorSampleSource

Mp4Extractor

MediaCodecVideoTrackRenderer

MediaCodecAudioTrackRenderer

ExoPlayer - MP4

Mediacodec AudioTrack

Decoding RenderingAndroid Framework

10

Execution Flow

ExoPlayerImplInternal

MediaCodecAudioTrackRenderer

MediaCodecVideoTrackRenderer

SampleSource for Audio Track

SampleSource for Video Track

MediaCodec

AudioTrack

Time Source

1:doSomeWork()

11

2: readData() 3: queueInputBuffer()

4: dequeOutputBuffer()5: handleBuffer()

6: doSomeWork()

7: getCurrentPositionUS()

8: readData()

9: decode & render()

DEMOTraditional MediaPlayback (ExtractorRendererBuilder)

12

13

Code (ExtractorRendererBuilder)

Adaptive Media Playbacks or ABR

● Adaptive bitrate streaming over HTTP

● Accommodate changing network conditions

● No special firewalls, proxies, or cache is required

● Efficient and cost-effective

14

Adaptive Media Playbacks (Example)

15

Adaptive Media Playbacks (Protocols)

● Dynamic Adaptive Streaming over HTTP (DASH) ○ Mpeg

● Smooth Streaming○ Microsoft

● HTTP Live Streaming (HLS)○ Apple

16

DASH● First major trial started in 2012 London Olympics ● How it works? [ref]

HTTP Server

segment

segment

segment

segment

segment

segment

segment

segment

segment

segment

segment

segment

segment

segment

segment

segment

MPD

Dash Client

HTTP ClientHTTP/1.1

Media Player

Control Heuristics

MPD Delivery MPD Parser

Segment Parser

17

1

2

MPD [ref]

18

DASH Object Model

DefaultUriDataSource DashChunkSource

FixedEvaluator

MediaCodecVideoTrackRenderer

ExoPlayer - Dash

Mediacodec AudioTrack

Decoding Rendering

DefaultUriDataSource DashChunkSource

AdaptiveEvaluator

ChunkSampleSource MediaCodecAudioTrackRenderer

DefaultLoadControl

ChunkSampleSource

Android Framework

19

DEMODASH MediaPlayback (DashRendererBuilder)

20

HLS● Developed by Apple● How it works? [ref]

Distribution Client

HTTP/1.1

Index file (.m3u8)

.ts

.ts .ts

.ts

21

HLS Index File .M3U8#EXT-X-VERSION:3

#EXTM3U

#EXT-X-TARGETDURATION:10

#EXT-X-MEDIA-SEQUENCE:1

# Old-style integer duration; avoid for newer clients.

#EXTINF:10,

http://media.example.com/segment0.ts

# New-style floating-point duration; use for modern clients.

#EXTINF:10.0,

http://media.example.com/segment1.ts

#EXTINF:9.5,

http://media.example.com/segment2.ts

#EXT-X-ENDLIST22

HLS Stream Alternates

Index file

Alternate -A

Index file

Alternate - B

Index file

Alternate - C

Index file

.ts

.ts .ts

.ts

.ts

.ts .ts

.ts

.ts

.ts .ts

.ts

23

24

HLS advance index file

HLS Object Model

DefaultUriDataSource HLSChunkSource MediaCodecVideoTrackRenderer

MediaCodecAudioTrackRenderer

ExoPlayer - HLS

Mediacodec AudioTrack

Decoding Rendering

HLSSampleSource

Android Framework

25

FormatEvaluator LoadControl

DEMOHLS MediaPlayback (HlsRendererBuilder)

26

Customization

● TrackRenderer○ Example : TextTrackRenderer

● Extractor

● SampleSource

● FormatEvaluator

● DataSource27

Customization Example

CacheDataSource DashChunkSource

FixedEvaluator

MediaCodecVideoTrackRenderer

ExoPlayer - Dash

Mediacodec AudioTrack

Decoding Rendering

CacheDataSource DashChunkSource

AdaptiveEvaluator

ChunkSampleSource MediaCodecAudioTrackRenderer

DefaultLoadControl

ChunkSampleSource

Android Framework

DefaultUriDataSource

DefaultUriDataSource

28

Digital Rights Management

Microsoft PlayReady Google Widevine

Fairplay (Apple) Adobe PrimeTime

29

DEMOWideVine DRM

30

Digital Rights Management (ExoPlayer)

● DrmSessionManager○ StreamingDrmSessionManager

○ On Android 4.3 (18) and higher

● WideVine DRM○ DASH

31

Conclusions

● ExoPlayer is an application level media player which can

be easily updated and extended by developers

● It is Open Source

● Start with demo code here

32

Applications using ExoPlayer

33

Helpful Links

● ExoPlayer Tutorial Video● DASH, HLS encoder - Bento4● Apple’s HLS Tools [link]● Shaka Player for Web [link]● dash.js [link]

34

Thank You!Sample Code : https://github.com/hassanabidpk/ExoPlayerSample

Contact : hassanabidpk89@gmail.com

Web : http://hassanabid.com35