[ElastixWorld 2016] mediasoup: Powerful WebRTC SFU for Node.js

mediasoupPowerful WebRTC SFU for Node.js

IÑAKI BAZ CASTILLO

SOMETHING ABOUT IT

WHAT IS MEDIASOUP?A WebRTC SFU “Selective Forwarding Unit”

▸ Handles the media layer▸ Doesn’t mix audio/video streams▸ Routes streams to participants

A multi-party video solution for Node.js▸ Not a standalone media server▸ A server-side Node.js module▸ Exposes a JavaScript ES6 API▸ Core written in C++

TOPOLOGIESMULTI-PARTY VIDEO CONFERENCING

TOPOLOGIES

FULL MESH▸ Each participant sends his audio

and video to all other participants

▸ Each participant receives all the streams from all other participants

No media server needed

Low latency

Lot of encodings in each participant

High uplink bandwidth required

TOPOLOGIES

MCU: MULTIPOINT CONTROL UNIT▸ Each participant sends his streams to a

media server

▸ The server mixes all the streams and composes a single one

▸ Example: Asterisk meetme

Simple at client side: single audio/video mixed stream from the server

Interoperability: the server can transcode

Low bandwidth required

CPU expensive decoding/encoding in server side (high latency)

Non flexible client side applications

TOPOLOGIES

SFU: SELECTIVE FORWARDING UNIT▸ Participant sends his streams to a media server

▸ The media server routes the streams to all other participants

▸ …or selects which ones to route

▸ Each participant receives many streams

High throughput, low latency

Low CPU usage at server side (no decoding/encoding)

Good uplink bandwidth usage

The client side application can render each remote video stream as desired

Simulcast/SVC required for real scenarios

TOPOLOGIES

FULL MESH MCU SFU

Client uplink Very high Low Low

Client downlink Very high Low High

Client CPU usage Very high Low Medium

Server CPU usage - Very high Very low

Latency None High Low

Can transcode - Yes No

Requires simulcast/SVC - - Yes

Flexible video layout Yes No Yes

MEDIASOUP IS AN SFU

YES!

MEDIASOUP IS MINIMALIST

MEDIASOUP IS MINIMALIST

WHAT IS *NOT* MEDIASOUP?▸ It is NOT a standalone server▸ It does NOT have “init” scripts for Debian or CentOS▸ It does NOT have a “config” file▸ It does NOT implement the SIP protocol▸ It does NOT implement ANY signaling protocol▸ It does NOT provide an “admin” web interface▸ It does NOT provide a client SDK

MEDIASOUP IS A NODE.JS MODULE

Yours truly

REMEMBER

SO WHAT?

OK…

$ npm install —save mediasoup

MEDIASOUP & NODE.JS

A NODE.JS MODULE▸ A Node.js module is a dependency/library within a

project▸ You create your Node.js based project/application▸ You add mediasoup as a module into it

At the end, mediasoup is yet

another dependency entry in the

package.json of your Node.js

application

MEDIASOUP & NODE.JS

A PACKAGE.JSON EXAMPLE

{ "name": "my-amazing-multiconference-server-app", "version": "1.0.0", "description": "Enterprise conferencing application", "main": "index.js", "author": "My Great Company", "license": "UNLICENSED", "dependencies": { "express": "^4.14.0", "mediasoup": "^1.0.0", "socket.io": "^1.5.0" }}

WHY NODE.JS?

ASK THEM

MEDIASOUP EXPOSES A JAVASCRIPT

ES6 API

// Load mediasoup moduleconst mediasoup = require('mediasoup');

// Create a mediasoup Serverlet server = mediasoup.Server();

// Options for the mediasoup Roomconst roomOptions = {

mediaCodecs: [{

kind : 'audio',name : 'audio/opus',clockRate : 48000

},{

kind : 'video',name : 'video/vp8',clockRate : 90000

}]

};

// Create a mediasoup Roomserver.createRoom(roomOptions)

.then((room) => {// Got the Room instancehandleRoom(room);

});

MEDIASOUP API

LOW LEVEL APImediasoup exposes a low level API similar to ORTC

// Create a Peerlet peer = room.Peer('alice');

// Create a ICE+DTLS Transportpeer.createTransport(options)

.then((transport) => {transport.setRemoteDtlsParameters(data);

});

// Create a RtpReceiver to handle audio from the browserlet audioReceiver = peer.RtpReceiver('audio', transport);

// Create a RtpReceiver to handle video from the browserlet videoReceiver = peer.RtpReceiver('video', transport);

MEDIASOUP API

HIGH LEVEL APImediasoup also exposes a high level API similar to WebRTC 1.0

// Create a PeerConnectionlet peerconnection = new mediasoup.webrtc.RTCPeerConnection(room, 'alice');

// Set the remote SDP offerpeerconnection.setRemoteDescription(desc)

.then(() => {return peerconnection.createAnswer();

}).then((desc) => {

return peerconnection.setLocalDescription(desc);}).then(() => {

// Answer the participant request with the SDP answerrequest.accept({

sdp: peerconnection.localDescription.sdp});

});

MEDIASOUP JUST FORWARDS MEDIA

REMEMBER

Sincerely yours

MEDIASOUP API

JUST MEDIA▸ mediasoup does NOT talk the SIP protocol▸ …nor it talks ANY signaling protocol▸ You can use socket.io (for example) to communicate

with browsers/endpoints via WebSocket▸ …or build your own protocolSimilar to the WebRTC API for

browsers, mediasoup just handles

the media layer

ROADMAP

MEDIASOUP ROADMAP

WORKING ON IT…1.0.0

▸ RTCP: process RTCP reports to allow mediasoup diagnose per peer uplink/downlink issues

▸ WebRTC 1.0: more API needed2.0.0

▸ Simulcast & SVC: handle multiple streams/layers from clients and select which one to route to others

YOUR TURN

THE APPLICATI

ON

THE APPLICATION

ASK YOURSELF▸ Want to build yet another

boring enterprise conferencing app?

▸ May be a funny social app?

▸ Will you build the app on Node.js?

▸ Browser app? mobile app? native Android/iOS SDKs?

▸ Do you need interoperability with PSTN? Really?

LET’S DEMO !!!

DEMO APPLICATION

FIRSTSIGHT“See many people, only talk to one”

Backend▸ Node.js server running mediasoup and websocket modules▸ JS logic to handle media rooms and manage participants

Frontend▸ HTML5 application made with React.js▸ JS logic to handle MediaStream from room participants

MUCHAS GRACIAS

Iñaki Baz Castillo

THE END

http://mediasoup.org