ORTC Walk Through IETF 88 Robin Raymond, Hookflash Bernard Aboba, Microsoft Adalberto Foresti, Microsoft Open Technologies, Inc. ORCA – Object-RTC API

ORTC Walk Through IETF 88

Robin Raymond, HookflashBernard Aboba, MicrosoftAdalberto Foresti, Microsoft Open Technologies, Inc.

ORCA – Object-RTC API Community Group

Meeting Purpose

• The purpose of this meeting is explain:– Why a community has come together to work on an alternative API– High level features of the ORTC API– How to use the API to implement a simple video chat demo– How you can get involved with developing the spec and code– Introduce open source effort / call to action

• There will be time for refreshments and Q&A the last 30 minutes

• Session is recorded and streamed to the web(sponsored by Hookflash)

Introduction to ORTC

ORTC – Object Realtime Communications – A next-generation JavaScript API for browser,

mobile, and cloud realtime audio / video scenarios– ORTC is an alternative API to the W3C WebRTC 1.0

draft API– It is being developed in the ORCA W3C

Community Group

The ORCA W3C Community Group

W3C Community Groups are forums open to all without fee– Simple IPR policy – royalty-free commitments to

one’s own contributions– Not on standards track– Mature CG specs can get broader IPR

commitments, transition to standards track– Don’t have to be W3C member!

The ORCA W3C Community Group

Object RTC API Community Group, – Formed in August 2013

Chair and editor: Robin Raymond– Goal to develop WebRTC API that exposes an

object API rather than blobs of SDP– Community group founded by JS developers

coming from both SIP/other protocol signaling backgrounds, node.js and WebRTC enthusiasts.

This is not a community group meeting

• Goal here is to explain mission and progress, recruit new members

• To suggest API changes, join the Community Group – we can’t accept suggestions from those who have not signed the Contributor License Agreement

Special Thanks to…

ORCA Community Group asked Microsoft Open Technologies if they would implement ORTC as demo / validation of concept.

Special thanks for Microsoft Open Technologies, Inc. for implementing demo / validation of concept.

(and the refreshments!)

WebRTC Needs• Mobile: SDKs enabling development of mobile apps protocol-

compatible with WebRTC.• Analytics. Diagnostics and monitoring for carrier scale operation. • Cloud. Easy setup and management of WebRTC systems.– Tools. Enabling connection of WebRTC clients to legacy

infrastructure (e.g. node.js)– APIs. Enabling management of STUN/TURN credentials,

audio/video mixers, SIP proxies, XMPP connection managers, etc.

• Extensibility: Enabling addition of functionality beyond the core (e.g. mobile-friendly video codecs such as H.264/SVC)).

• A next generation JavaScript API compatible with the above.

What are ORTC potential uses?

• Browser code (simplified RTC control)• Mobile development via “ortclib” SDK

under development https://github.com/openpeer/ortclib

• Server code via Node.jsnew services or legacy inter-op solutions – call to contributeunder development https://github.com/openpeer/ortc-node

Open Source projects are under way!

https://github.com/openpeer/ortclib

Write ORTC apps using existing WebRTC 1.0 Browsers

An WebRTC 1.0 “shim” will be made available to maintain backwards compatibility

If you like the WebRTC 1.0 API, you can still use the 1.0 API!

WebRTC 1.0 (SDP)

WebRTC 1.0 (SDP)

SDP / SIP (shim)SDP / SIP (shim)

1.0 Strategy

Node.JSNode.JSBrowsersBrowsers

Chrome / Firefox / etc

Chrome / Firefox / etc

ORTC (shim)ORTC (shim)

Node.JSNode.JS

Web AppWeb App

Browsers(see demo)Browsers

(see demo)

ORTCORTC

2.0 Future ?

ORTC Migration Strategy

Web AppWeb App

Mobile(ortclib)Mobile(ortclib) MobileMobile

Isn’t ORTC just CU-RTCWeb?

NoBoth are object models but at different levels

While CU-RTCWeb inspired ORTC, they are are different.

Issues with CU-RTCWeb:- too powerful/difficult to use- exposing too much socket level control (i.e. security risks)- couldn’t shim CU-RTCWeb

Where will ORTC be supported by vendors?

Support is via the shim over WebRTC 1.0 for now for browsers.

Direct ORTC API source code is being developed.

Microsoft Open Tech developed a shim plugin for IE (but not a product announcement).

openpeer/ortcGitHub Specification Repo

https://github.com/openpeer/ortc

•Original home of the ORTC API discussion•Spec discussion spun off to CG, OSS code for ORTC stays in GitHub•Why do we need the GitHub repo?

– Pull requests – discussed on public-orca mailing listhttp://www.w3.org/community/orca/

openpeer/ortclibGitHub Mobility Repo

https://github.com/openpeer/ortclib

•ORTC mobile implementation•Maintained by Hookflash•Exposes C++ WebRTC library as an ORTC API•To be wrapped in iOS, Java (for Android) and JavaScript hybrid

openpeer/ortc-nodeGitHub Node.JS Repo

https://github.com/openpeer/ortc-node

•ORTC Node.JS implementation•Call to action – seeking contributors

– What should implementation look like?– What are native code integration points?

Will contributing to ortcbreak protocol compatibility with web browser?

NoWire level compatibility is maintained

Maintaining wire compatibility is a tenet of ORTC implementations with WebRTC enabled browsers*

We encourage expansion of features, not removal of basic wire support.

* as much as they are compatible with each other

Motivation for ORTC Design

Traditional Browser Rendering Engine

Description Format(HTML/CSS)

Description Format(HTML/CSS)

ResourceResource

Render EngineRender Engine

OutputOutput

JSJS

JS asks browser “Give me webpage that meets this criteria”

11

Modify browser provided HTML33

Set HTML44

Browser generates some HTML (gives to JavaScript)

22

This is not how we program HTML on the web…

WebRTC 1.0 Capture / Render Engine

Description Format(SDP)


ResourceResource

Capture/Render EngineCapture/Render Engine

Input/OutputInput/Output

JSJS



ResourceResource



JSJS

{ Cloud }{ Cloud }

ORTC Capture / Render Engine





{ Cloud }{ Cloud }

JSJS JSJS

ORTC Simple Use Case

RTCConnectionRTCConnection

{ cloud }{ cloud }

ApplicationApplication

Browser



Browser

RTP / DATA / DTLS / ICERTP / DATA / DTLS / ICE

signa

ling

signaling

Questions?

ORTC API

ORTC Simple Use Case


{ cloud }{ cloud }


Browser



Browser

RTP / DATA / DTLS / ICERTP / DATA / DTLS / ICE

signa

ling

signaling

MediaStreamMediaStream

MediaStreamTrackMediaStreamTrack


RTCTrackRTCTrack

RTCTrackRTCTrack

RTCStreamRTCStream

ORTC - Object Relationships


RTCConnectionOptionsRTCConnectionOptions

RTCConnectionInfoRTCConnectionInfo

RTCIceConnectionInfoRTCIceConnectionInfo

RTCDtlsConnectionInfoRTCDtlsConnectionInfo

RTCIceCandidateRTCIceCandidate

RTCSocketRTCSocket

RTCStreamRTCStream MediaStreamMediaStream

RTCTrackRTCTrack

RTCTrackRTCTrack






loca

lre

mot

e

RTCConnectionInfoRTCConnectionInfo

RTCIceConnectionInfoRTCIceConnectionInfo

RTCDtlsConnectionInfoRTCDtlsConnectionInfo

remote

local

RTCConnection

[Constructor(RTCConnectionOptions options)]interface RTCConnection : EventTarget { readonly attribute RTCSocket socket; readonly attribute RTCConnectionState state; attribute RTCConnectionRole role; readonly attribute RTCConnectionInfo local; attribute RTCConnectionInfo remote;

void addRemoteCandidate(RTCCandidateDescription candidate); void connect(); void gather(); (RTCSTream? or RTCTrack?) send((MediaStream or MediaStreamTrack or RTCTrack or RTCStream) streamOrTrack); (RTCStream? or RTCTrack?) receive((RTCStream or RTCTrack) streamOrTrack); sequence<RTCStream> sendStreams(); sequence<RTCTrack> sendTracks(); sequence<RTCStream> receiveStreams(); sequence<RTCTrack> receiveTracks(); void close ();

attribute EventHandler oncandidate; attribute EventHandler onendofcandidates; attribute EventHandler onactivecandidate; attribute EventHandler onstatechanged; attribute EventHandler onnetworkchange; attribute EventHandler onunknowntrack;};

RTCConnection (p1)

- RTCSocket (attached to one network socket)- States

- “new” – connection not started- “searching” – searching for connectivity- “halted” – no connectivity found- “connected” – found connectivity (ready)- “closed” – connection is finished (one-time-use)

RTCConnection (p2)

- role (“controlling” vs “controlled”)- Local and remote information

- ICE credentials (usernameFrag / password)- DTLS fingerprint (generated or discovered)

- ICE Candidates – local and remote (for connectivity checks)

RTCConnection (p3)

- Gathers local ICE candidates- Performs ICE connectivity checks- Causes sending / receiving of media tracks

RTP data over the “nominated” ICE connection

RTCConnection (p4)

RTCStreams and RTCTracks (sending or receiving)

- Contains mapping of on-the-wire RTC/RTP/RTCP/etc information to media streams / media stream tracks)

- Contains respective “MediaStream” or “MediaStreamTrack” objects.

RTCConnection “connect”

Set remote “RTCIceConnectionInfo” ICE “usernameFrag” and “password”

Call “connect” to cause candidate search

Will causes state changes from “new”, “searching”, “haulted”, “connected”

RTCConnection “gather”

Find new candidates- Restarts network topology ICE candidate gathering

Does not affect connection state by itself

Start gathering candidates before “connect”

RTCConnection “send”

Send “MediaStream”, “MediaStreamTrack”, “RTCStream”, “RTCTrack”

Convenience to auto-map “MediaStream” and “MediaStreamTrack” into RTP layer constructs

Manipulate RTP mapping by first creating “RTPStream” / “RTCTrack” from “MediaStream” and “MediaStreamTrack”

RTCConnection “receive”

Send “RTCStream”, “RTCTrack”

Setup mapping for “MediaStream” and “MediaStreamTrack” for RTP layer constructs

Manipulate mapping by constructing “RTPStream” / “RTCTrack” objects

RTCConnection “close”

Shuts down RTCConnection object

Object goes to “closed” state

RTCConnection eventsoncandidate Fires for local candidate (can be cancelled)

onendofcandidates Fires when all local candidates are discovered

onactivecandidate Fires when local:remote candidate connection changes

onstatechanged Fires as connection state changes

onnetworkchange Fires when network topology change has been detected(auto “gather” called but can be cancelled)

onunknowntrack Fires when track discovered which has no mapping

RTCSocket

interface RTCSocket {};

RTCSocket

- No exposed attributes, members or events

- Represents one physical local socket (as well as virtualized paired firewall/relay socket)

- Re-use the same RTCSocket in another RTCConnection to send over same physical local socket

RTCStream

[Constructor, Constructor(MediaStream)]interface RTCTrack { attribute MediaStream source; attribute DOMString msid;

sequence<RTCTrack> tracks(); void start(); void stop(); void remove();};

RTCStream

- References single MediaStream- Collection of RTCTracks- Change collection then call RTCConnection

“send”/”receive” again- Simplifies MediaStream usage in RTCConnection- Convenience “start” / “stop” methods- “remove” method to remove from RTCConnection

RTCTrack[Constructor, Constructor(MediaStreamTrack)]interface RTCTrack { attribute MediaStreamTrack source; attribute DOMString id; attribute DOMString kind; attribute unsigned long ssrc; attribute sequence<RTCCodec> codecs; attribute sequence<RTCMediaAttributes> mediaAttributes; attribute Object rtpExtHeaders;

void start(); void stop(); void remove();};

RTCTrack

- References single MediaStreamTrack- Mapping of RTP/RTCP/etc wire information such as:

- ssrc- codecs- media attributes (e.g. max width / height)

- Change properties then call RTCConnection “send” / “receive” again

- Convenience “start” / “stop” methods- “remove” method to remove from RTCConnection

RTCConnectionInfodictionary RTCConnectionInfo { attribute RTCIceConnectionInfo ice; readonly attribute RTCDtlsConnectionInfo dtls;};

RTCIceConnectionInfo

RTCDtlsConnectionInfo / CertificateFingerprint

dictionary RTCIceConnectionInfo { DOMString usernameFrag; DOMString password;};

dictionary RTCDtlsConnectionInfo { attribute CertificateFingerprint fingerprint;};

dictionary CertificateFingerprint { getter ArrayBuffer (DOMString hashFunction);};

RTCIceCandidateInfodictionary RTCIceCandidateInfo { DOMString foundation; int component; DOMString transport; int priority; DOMString connectionAddress; int connectionPort; DOMString type; DOMString relAddress; int relPort;};

Object Realtime Communications API (ORTC)

• Next generation WebRTC API, under development in the W3C ORCA community group:– http://www.w3.org/community/orca/– Rationale: http://tools.ietf.org/html/draft-raymond-rtcweb-

webrtc-js-obj-api-rationale• Specification:– http://openpeer.github.io/ortc/

• Github repository:– ORTC API drafts and shim: https://github.com/openpeer/ortc– ortclib: https://github.com/openpeer/ortclib– Demos: https://github.com/openpeer/webrtc-demo

To Learn More

• ORCA Community Group: http://www.w3.org/community/orca/

Demo and discussion tomorrow to follow…

Documents

ORTC Walk Through IETF 88 Robin Raymond, Hookflash Bernard Aboba, Microsoft Adalberto Foresti, Microsoft Open Technologies, Inc. ORCA – Object-RTC API