Embed Size (px)
Citation preview
ENABLING VOICE APPLICATIONS WITH WEBRTC & ORTC IN MICROSOFT EDGE
MARK ROBERTS SOFTWARE ENGINEER
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
PSTN
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
Ready Not Ready
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
Ready Not Ready
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
Ready Not Ready
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
Ready Not Ready
+ =
<script src="twilio.js"></script>
© 2009 - 2014 Twilio, Inc. All rights reserved.
Four Basic Tasks
Four basic tasks for a voice application:
© 2009 - 2014 Twilio, Inc. All rights reserved.
Four Basic Tasks
Four basic tasks for a voice application: 1. Capture audio from a microphone
© 2009 - 2014 Twilio, Inc. All rights reserved.
Four Basic Tasks
Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint
© 2009 - 2014 Twilio, Inc. All rights reserved.
Four Basic Tasks
Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint
© 2009 - 2014 Twilio, Inc. All rights reserved.
Four Basic Tasks
Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint 4.Playback received audio
FIRST, THERE WAS FLASH…
© 2009 - 2014 Twilio, Inc. All rights reserved.
Browser Support
<script src="twilio.js"></script>
ALONG CAME WEBRTC…
© 2009 - 2014 Twilio, Inc. All rights reserved.
WebRT-what⁇
© 2009 - 2014 Twilio, Inc. All rights reserved.
WebRT-what⁇
• Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription
• First-class browser support
• Built on open standards(STUN, TURN, ICE, DTLS, RTP)
© 2009 - 2014 Twilio, Inc. All rights reserved.
Four Basic Tasks
Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint 4.Playback received audio
© 2009 - 2014 Twilio, Inc. All rights reserved.
Four Basic Tasks
Four basic tasks for a voice application: 1. Capture audio from a microphone 2. Send audio to an endpoint 3. Receive audio from an endpoint 4.Playback received audio
WebRTC Endpoint WebRTC Endpoint
WebRTC Endpoint WebRTC Endpoint
establish connectivity
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
WebRTC Endpoint
establish connectivity
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
WebRTC Endpoint
establish connectivity
secure connectivity
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
Datagram Transport Layer Security (DTLS)
WebRTC Endpoint
establish connectivity
secure connectivity
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
Datagram Transport Layer Security (DTLS)
WebRTC Endpoint
establish connectivity
secure connectivity
send and receive media
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
Datagram Transport Layer Security (DTLS)
Secure Real-time Transport Protocol (SRTP)
WebRTC Endpoint
establish connectivity
secure connectivity
send and receive media
© 2009 - 2014 Twilio, Inc. All rights reserved.
WebRT-what⁇
• Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription
• First-class browser support
• Built on open standards(STUN, TURN, ICE, DTLS, RTP)
© 2009 - 2014 Twilio, Inc. All rights reserved.
WebRT-what⁇
• Simple JavaScript APIs • RTCPeerConnection • RTCIceCandidate • RTCSessionDescription
• First-class browser support
• Built on open standards(STUN, TURN, ICE, DTLS, RTP)
• Interoperability
• No plugins required!
• Bring-your-own signaling
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
Datagram Transport Layer Security (DTLS)
Secure Real-time Transport Protocol (SRTP)
WebRTC Endpoint
establish connectivity
secure connectivity
send and receive media
WebRTC Endpoint WebRTC Endpoint
© 2009 - 2014 Twilio, Inc. All rights reserved.
Browser Support
<script src="twilio.js"></script>
© 2009 - 2014 Twilio, Inc. All rights reserved.
Browser Support
<script src="twilio.js"></script>
© 2009 - 2014 Twilio, Inc. All rights reserved.
Browser Support
© 2009 - 2014 Twilio, Inc. All rights reserved.
Browser Support
<script src="twilio.js"></script>
EDGE & ORTC ANNOUNCED…
© 2009 - 2014 Twilio, Inc. All rights reserved.
Object RTC (ORTC)
© 2009 - 2014 Twilio, Inc. All rights reserved.
Object RTC (ORTC)
• Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP)
• Bring-your-own signaling
© 2009 - 2014 Twilio, Inc. All rights reserved.
Object RTC (ORTC)
• Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP)
• Bring-your-own signaling
• Does not use Session Description Protocol (SDP)
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
Datagram Transport Layer Security (DTLS)
Secure Real-time Transport Protocol (SRTP)
WebRTC Endpoint
establish connectivity
secure connectivity
send and receive media
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
Datagram Transport Layer Security (DTLS)
Secure Real-time Transport Protocol (SRTP)
SDP
WebRTC Endpoint
© 2009 - 2014 Twilio, Inc. All rights reserved.
Session Description Protocol (SDP)
v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000 a=rtpmap:13 CN/8000
a=fingerprint:sha-256 F1:A4:DB:A3:AA:CC:1C:B5:2F:91:A0:2F:E8:FD:0B:80:F7:02:1F:9E:23:49:1C:06:1F:4F:AB:96:44:F1:CB:D6 a=setup:actpass m=video 51034 UDP/TLS/RTP/SAVPF 100 101 116 117 96 c=IN IP4 54.244.51.11 a=rtcp:27538 IN IP4 54.244.51.11 a=ice-ufrag:mKHEfIR34arV95jd a=ice-pwd:YdwDVrwNPLwIsFF2sJtHZDSk a=fingerprint:sha-256 F1:A4:DB:A3:AA:CC:1C:B5:2F:91:A0:2F:E8:FD:0B:80:F7:02:1F:9E:23:49:1C:06:1F:4F:AB:96:44:F1:CB:D6 a=setup:actpass a=extmap:2 urn:ietf:params:rtp-hdrext:toffset a=extmap:3 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time a=extmap:4 urn:3gpp:video-orientation a=sendrecv a=rtcp-mux a=rtpmap:100 VP8/90000 a=rtcp-fb:100 ccm fir a=rtcp-fb:100 nack
© 2009 - 2014 Twilio, Inc. All rights reserved.
Session Description Protocol (SDP)
v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000
© 2009 - 2014 Twilio, Inc. All rights reserved.
Session Description Protocol (SDP)
v=0 o=- 7419095041521713450 2 IN IP4 127.0.0.1 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS BcUNfqxMTfmJ2aPDEipCTK4BnvabR7xKeTil m=audio 35898 UDP/TLS/RTP/SAVPF 111 103 104 9 0 8 106 105 13 126 c=IN IP4 54.244.51.11 a=rtcp:39742 IN IP4 54.244.51.11 a=sendrecv a=rtcp-mux a=rtpmap:111 opus/48000/2 a=fmtp:111 minptime=10; useinbandfec=1 a=rtpmap:103 ISAC/16000 a=rtpmap:104 ISAC/32000 a=rtpmap:9 G722/8000 a=rtpmap:0 PCMU/8000 a=rtpmap:8 PCMA/8000 a=rtpmap:106 CN/32000 a=rtpmap:105 CN/16000
© 2009 - 2014 Twilio, Inc. All rights reserved.
Object RTC (ORTC)
• Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP)
• Bring-your-own signaling
• Does not use Session Description Protocol (SDP)
• Lower-level JavaScript APIs
WebRTC Endpoint
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
RTCIceTransport
RTCIceGatherer
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
Datagram Transport Layer Security (DTLS)
RTCIceTransport
RTCIceGatherer
RTCDtlsTransport
WebRTC Endpoint
Interactive Connectivity
Establishment (ICE)
Datagram Transport Layer Security (DTLS)
Secure Real-time Transport Protocol (SRTP)
RTCIceTransport
RTCIceGatherer
RTCDtlsTransport
RTCRtpReceiver
RTCRtpSender
© 2009 - 2014 Twilio, Inc. All rights reserved.
Object RTC (ORTC)
• Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP)
• Bring-your-own signaling
• Does not use Session Description Protocol (SDP)
• Lower-level JavaScript APIs • RTCIceGatherer • RTCIceTransport • RTCDtlsTransport • RTCRtpSender • RTCRtpReceiver
© 2009 - 2014 Twilio, Inc. All rights reserved.
Object RTC (ORTC)
• Built on the same open standards as WebRTC (STUN, TURN, ICE, DTLS, RTP)
• Bring-your-own signaling
• Does not use Session Description Protocol (SDP)
• Lower-level JavaScript APIs • RTCIceGatherer • RTCIceTransport • RTCDtlsTransport • RTCRtpSender • RTCRtpReceiver
• Power and flexibility over “plain” WebRTC
© 2009 - 2014 Twilio, Inc. All rights reserved.
Bridging the Gap
ORTC
• RTCIceGatherer
• RTCIceTransport
• RTCDtlsTransport
• RTCRtpSender
• RTCRtpReceiver
WebRTC
• RTCPeerConnection
• RTCIceCandidate
• RTCSessionDescription
© 2009 - 2014 Twilio, Inc. All rights reserved.
Browser Support
© 2009 - 2014 Twilio, Inc. All rights reserved.
Browser Support
<script src="twilio.js"></script>
© 2009 - 2014 Twilio, Inc. All rights reserved.
Browser Support
ORTC
<script src="twilio.js"></script>
© 2009 - 2014 Twilio, Inc. All rights reserved.
Review
© 2009 - 2014 Twilio, Inc. All rights reserved.
Review
• Flash is no longer the only real-time communications solution for browsers.
© 2009 - 2014 Twilio, Inc. All rights reserved.
Review
• Flash is no longer the only real-time communications solution for browsers.
• WebRTC and ORTC enable interoperable, plugin-free communication.
© 2009 - 2014 Twilio, Inc. All rights reserved.
Review
• Flash is no longer the only real-time communications solution for browsers.
• WebRTC and ORTC enable interoperable, plugin-free communication.
• Real-time communications is complex, but developing voice applications does not have to be complicated.
© 2009 - 2014 Twilio, Inc. All rights reserved.
Voice Applications
Ready Not Ready
<script src="twilio.js"></script>
twilio.js
