Presentacin de PowerPoint

Introduction to WebRTCLuis Lpezlulop@kurento.orghttp://www.kurento.org

Real-time Communications (RTC)

Internet RTCFragmentationNo universal RTC service

Telco RTCStandardizationPhone system as universal RTC service

2http://www.kurento.orgWebRTC: a new player3http://www.kurento.orgWhy WebRTC4http://www.kurento.orgWhy WebRTC5http://www.kurento.orgWho is who in WebRTC

6http://www.kurento.orgWhats WebRTC: browser architecture*This image has been borrowed from: http://www.html5rocks.com/en/tutorials/webrtc/basics/

7http://www.kurento.orgWebRTC: P2P communications

*This image has been borrowed from: http://www.html5rocks.com/en/tutorials/webrtc/basics/8http://www.kurento.orgWebRTC: NATs

*This image has been borrowed from: http://www.html5rocks.com/en/tutorials/webrtc/basics/http://io13webrtc.appspot.com/#449http://www.kurento.orgDeveloping WebRTC appshttp://io13webrtc.appspot.com/#9

http://www.html5rocks.com/en/tutorials/webrtc/basics/

10http://www.kurento.orgExamplehttps://talky.io/https://www.cubeslam.com

11http://www.kurento.orgWhy Kurento?WWW RTC developer experience12

Developing the client sideDeveloping the infrastructureside

Before WebRTCAfter WebRTCNext natural stepBeginEndBeginEndBeginEnd

Unified APIsStandardsFOSSMultiplatformUnified APIsStandardsFOSSMultiplatformUnified APIsStandardsFOSSMultiplatformhttp://www.kurento.orgKurento media framework components13http://www.kurento.orgKurento Application Server: extending the WWW development modelProcess WWW requestDDBB accessTransactionsSecurity toolsEtc.HTTP request: I want this resourceHTTP response:The resourceProcess media requestMedia APIDDBB accessTransactionsSecurity toolsEtc.Signaling request: I want this mediaSignaling response:The media is hereIntuition behind traditional WWW Applications (Servlets, ASP, PHP, Rails, etc.)Intuition behind Kurento development APIs:Multimedia RTC is just another feature of your application14

http://www.kurento.orgMedia API: media elements and media pipelinesMedia ElementProvides a specific media functionalityBuilding blockSend/receive mediaProcess mediaTransform mediaThe Media API provides a toolbox of media elements ready to be used.New media elements can be addedMedia pipelineChain of media elements implementing the desired media functionality.The Media API provides the capability of creating media pipelines by joining media elements of the toolboxMedia ElementSinkSRC

SinkSRC

SRCSinkWebRtcEndPointFilterPlayerEndPoint

15http://www.kurento.orgApplication architecture

Kurento Media Server InfrastructureDecodeVideoAugmentedRealityEncode VideoComputer VisionVideo Playingand RecordingJava EE compatible containerHTTP ServletSIP ServletWeb servicesDD.BB.JMSOther systems and databasesSpecific application logicMedia APISignaling and WWW trafficMediaMediaSignaling and WWW traffic16

http://www.kurento.orgPossible use cases: just integrate with Java EE and GStreamer VerticalsE-HealthKurento + HAPI (http://hl7api.sourceforge.net/)P2D video conferences as Electronic Health RecordsSmart citiesKurento + NGSI + OpenCV + Google MapsCity crowds movement trackingTraffic density trackingTelco infrastructuresKurento + MobicentsIMS application serverB2B & B2C WWW RTCKurento + CRM APIsEnriched video conferencing with customer personal dataKurento + ESBBilling, video event processing, physical security, etc.

17

http://www.kurento.orgApplication example: requirements

18Kurento Media Server Infrastructurehttp://www.kurento.orgApplication example: code@WebRtcMediaService(name = "MyWebRtcService", path = "/pathToService")public class MyWebRtcService implements WebRtcMediaHandler {

public void onMediaRequest(WebRtcMediaRequest request) {//I can authenticate using any of the Java EE available mechanismsMediaPipelineFactory mpf = request.getMediaPipelineFactory();MediaPipeline mp = mpf.createMediaPipeline();//I could decide the type of processing connecting to a DDBBJackVaderFilter filter = mp.newFilter().withType(JackVaderFilter.class).build();RecorderEndpoint recorder = mp.newRecorderEndpoint().withUri("file:///myFile.webm");filter.connect(recorder); HttpEndpoint httpEndpoint = mp.newHttpEndpoint().build();filter.connect(filter); //I could connect only audio or video separately

request.startMedia(filter, filter);}SinkSRC

Media SourceMedia SinkMediaSinkRecorderEndpointTo fileTo NetworkFrom networkMediaSinkHttpEndpointWebRtcEndpointJackVaderFilterVideotag or CDN

19http://www.kurento.orgApplication example: result

20http://www.kurento.orgMedia element toolbox

Available as part of21http://www.kurento.orgMedia elements: WebRtcEndpointFull implementation of the RTCWeb protocol stackSRTPICEDTLSAllow sending and receiving WebRTC flows at the media server infrastructureMedia SourceMedia SinkWebRtcEndpoint

WebRTC trafficWebRTC traffic22http://www.kurento.orgMedia elements: HttpEndpointMedia downloading compatible with the HTML5 video tagWebM (Chrome, Firefox)MP4 (Chrome, Firefox, IE, Safari)Media uploading compatible with HTML file input tagMultipart supportMediaSinkHttpEndpoint

HTTP WebM/MP423http://www.kurento.orgMedia elements: RtpEndpointFull-duplex RTP multimedia exchangeH.264H.263VP8Many different audio codecs supportedMedia SourceMedia SinkRtpEndpointRTP trafficRTP traffic

24http://www.kurento.orgMedia elements: UriEndpointsPayerEndpointPlay media from file or URLSupport for most popular formatsRecorderEndpointRecord media to file or URLWebMMP4MediaSinkRecorderEndpointPlayerEndpointMedia Source

25http://www.kurento.orgMedia elements: Mixers (in progress)Make possible group communicationsForwardingMixerOne-to-many replication of flowsA source can be assigned to any of the sinksMultiple sources supportedMainMixerMixes mediaA source can be assigned to a combination of sinksMultiple sources supportedMixerMedia SourceMedia SourceMedia SourceMediaSinkMediaSinkMediaSink

26http://www.kurento.orgFiltersSeamless integration into OpenCVFace recognitionAugmented realitySubtitle addingColor manipulationQR detectionPeople counterPlate recognitionEtc.FilterSinkSRC

27http://www.kurento.orgFilter with eventsFilters can provide events to the applicationEvents are generated at the media serverEvents can be propagated to the client appCode example:MediaPipeline mp = mpf.create();PlayerEndPoint playerEndPoint = mp.newPlayerEndPoint("https://ci.kurento.com/video/barcodes.webm").build();ZBarFilter filter = mp.newZBarFilter().build();playerEndPoint.connect(filter);

filter.addCodeFoundDataListener(new MediaEventListener() {@Overridepublic void onEvent(CodeFoundEvent event) {session.publishEvent(new ContentEvent(event.getType(),event.getValue()));...PlayerEndpointMedia Source

ZBarFilterSinkSRC

Events28http://www.kurento.orgThe magic of pipelines: Transparent media adaptationAgnostic media adaptorActs every time a source is connected to a sinkAdapts media formats as required by the involved media elements100% transparent for the application developer

Media ElementSinkSRC

Media ElementSinkSRC

An agnostic media adaptor is hidden behind every connection making compatible element formats transparentlyH.264VP829http://www.kurento.orgMediaSinkComplex examples: Heterogeneous group communicationsMedia SourceMedia SinkWebRtcEndpoint

Media SourceMedia SinkWebRtcEndpoint

Media SourceMedia SinkRtpEndpointRecorderEndpoint

FilterSinkSRC

Mixer

30http://www.kurento.orgMediaSinkMediaSinkComplex examples: WebRTC to HTTPMixerMedia SourceMedia SinkWebRtcEndpoint

RecorderEndpoint

HttpEndpoint

MediaSinkMixerHttpEndpoint

MediaSinkHttpEndpoint

31http://www.kurento.orgMedia SourceMediaSinkMediaSinkComplex examples: WebRTC PlayingMixerMedia SourceMedia SinkWebRtcEndpoint

RecorderEndpoint

HttpEndpoint

PlayerEndpoint

FilterSinkSRC

32http://www.kurento.orgWebRTC CV gamehttp://www.youtube.com/watch?v=CRqT7Q9KkRY

SinkSRC

Sink

SinkSRC

SinkSRC

SinkSRC

SinkSRC

WebRtcEndpointMirrorFilterPointerDetectorFilterChromaFilterFaceOverlayFilterMedia PipelineWebRTC full duplex client video applicationUpload toYoutube33http://www.kurento.orgCollaborations welcomehttp://www.github.com/kurentoThank you very much for your attention

Complains, suggestions and comments can be sent to:Luis Lpezlulop@kurento.org34http://www.kurento.org