Presentacin de PowerPoint
Introduction to WebRTCLuis [email protected]://www.kurento.org
Real-time Communications (RTC)
Internet RTCFragmentationNo universal RTC service
Telco RTCStandardizationPhone system as universal RTC service
2http://www.kurento.orgWebRTC: a new player3http://www.kurento.orgWhy WebRTC4http://www.kurento.orgWhy WebRTC5http://www.kurento.orgWho is who in WebRTC
6http://www.kurento.orgWhats WebRTC: browser architecture*This image has been borrowed from: http://www.html5rocks.com/en/tutorials/webrtc/basics/
7http://www.kurento.orgWebRTC: P2P communications
*This image has been borrowed from: http://www.html5rocks.com/en/tutorials/webrtc/basics/8http://www.kurento.orgWebRTC: NATs
*This image has been borrowed from: http://www.html5rocks.com/en/tutorials/webrtc/basics/http://io13webrtc.appspot.com/#449http://www.kurento.orgDeveloping WebRTC appshttp://io13webrtc.appspot.com/#9
http://www.html5rocks.com/en/tutorials/webrtc/basics/
10http://www.kurento.orgExamplehttps://talky.io/https://www.cubeslam.com
11http://www.kurento.orgWhy Kurento?WWW RTC developer experience12
Developing the client sideDeveloping the infrastructureside
Before WebRTCAfter WebRTCNext natural stepBeginEndBeginEndBeginEnd
Unified APIsStandardsFOSSMultiplatformUnified APIsStandardsFOSSMultiplatformUnified APIsStandardsFOSSMultiplatformhttp://www.kurento.orgKurento media framework components13http://www.kurento.orgKurento Application Server: extending the WWW development modelProcess WWW requestDDBB accessTransactionsSecurity toolsEtc.HTTP request: I want this resourceHTTP response:The resourceProcess media requestMedia APIDDBB accessTransactionsSecurity toolsEtc.Signaling request: I want this mediaSignaling response:The media is hereIntuition behind traditional WWW Applications (Servlets, ASP, PHP, Rails, etc.)Intuition behind Kurento development APIs:Multimedia RTC is just another feature of your application14
http://www.kurento.orgMedia API: media elements and media pipelinesMedia ElementProvides a specific media functionalityBuilding blockSend/receive mediaProcess mediaTransform mediaThe Media API provides a toolbox of media elements ready to be used.New media elements can be addedMedia pipelineChain of media elements implementing the desired media functionality.The Media API provides the capability of creating media pipelines by joining media elements of the toolboxMedia ElementSinkSRC
SinkSRC
SRCSinkWebRtcEndPointFilterPlayerEndPoint
15http://www.kurento.orgApplication architecture
Kurento Media Server InfrastructureDecodeVideoAugmentedRealityEncode VideoComputer VisionVideo Playingand RecordingJava EE compatible containerHTTP ServletSIP ServletWeb servicesDD.BB.JMSOther systems and databasesSpecific application logicMedia APISignaling and WWW trafficMediaMediaSignaling and WWW traffic16
http://www.kurento.orgPossible use cases: just integrate with Java EE and GStreamer VerticalsE-HealthKurento + HAPI (http://hl7api.sourceforge.net/)P2D video conferences as Electronic Health RecordsSmart citiesKurento + NGSI + OpenCV + Google MapsCity crowds movement trackingTraffic density trackingTelco infrastructuresKurento + MobicentsIMS application serverB2B & B2C WWW RTCKurento + CRM APIsEnriched video conferencing with customer personal dataKurento + ESBBilling, video event processing, physical security, etc.
17
http://www.kurento.orgApplication example: requirements
18Kurento Media Server Infrastructurehttp://www.kurento.orgApplication example: code@WebRtcMediaService(name = "MyWebRtcService", path = "/pathToService")public class MyWebRtcService implements WebRtcMediaHandler {
public void onMediaRequest(WebRtcMediaRequest request) {//I can authenticate using any of the Java EE available mechanismsMediaPipelineFactory mpf = request.getMediaPipelineFactory();MediaPipeline mp = mpf.createMediaPipeline();//I could decide the type of processing connecting to a DDBBJackVaderFilter filter = mp.newFilter().withType(JackVaderFilter.class).build();RecorderEndpoint recorder = mp.newRecorderEndpoint().withUri("file:///myFile.webm");filter.connect(recorder); HttpEndpoint httpEndpoint = mp.newHttpEndpoint().build();filter.connect(filter); //I could connect only audio or video separately
request.startMedia(filter, filter);}SinkSRC
Media SourceMedia SinkMediaSinkRecorderEndpointTo fileTo NetworkFrom networkMediaSinkHttpEndpointWebRtcEndpointJackVaderFilterVideotag or CDN
19http://www.kurento.orgApplication example: result
20http://www.kurento.orgMedia element toolbox
Available as part of21http://www.kurento.orgMedia elements: WebRtcEndpointFull implementation of the RTCWeb protocol stackSRTPICEDTLSAllow sending and receiving WebRTC flows at the media server infrastructureMedia SourceMedia SinkWebRtcEndpoint
WebRTC trafficWebRTC traffic22http://www.kurento.orgMedia elements: HttpEndpointMedia downloading compatible with the HTML5 video tagWebM (Chrome, Firefox)MP4 (Chrome, Firefox, IE, Safari)Media uploading compatible with HTML file input tagMultipart supportMediaSinkHttpEndpoint
HTTP WebM/MP423http://www.kurento.orgMedia elements: RtpEndpointFull-duplex RTP multimedia exchangeH.264H.263VP8Many different audio codecs supportedMedia SourceMedia SinkRtpEndpointRTP trafficRTP traffic
24http://www.kurento.orgMedia elements: UriEndpointsPayerEndpointPlay media from file or URLSupport for most popular formatsRecorderEndpointRecord media to file or URLWebMMP4MediaSinkRecorderEndpointPlayerEndpointMedia Source
25http://www.kurento.orgMedia elements: Mixers (in progress)Make possible group communicationsForwardingMixerOne-to-many replication of flowsA source can be assigned to any of the sinksMultiple sources supportedMainMixerMixes mediaA source can be assigned to a combination of sinksMultiple sources supportedMixerMedia SourceMedia SourceMedia SourceMediaSinkMediaSinkMediaSink
26http://www.kurento.orgFiltersSeamless integration into OpenCVFace recognitionAugmented realitySubtitle addingColor manipulationQR detectionPeople counterPlate recognitionEtc.FilterSinkSRC
27http://www.kurento.orgFilter with eventsFilters can provide events to the applicationEvents are generated at the media serverEvents can be propagated to the client appCode example:MediaPipeline mp = mpf.create();PlayerEndPoint playerEndPoint = mp.newPlayerEndPoint("https://ci.kurento.com/video/barcodes.webm").build();ZBarFilter filter = mp.newZBarFilter().build();playerEndPoint.connect(filter);
filter.addCodeFoundDataListener(new MediaEventListener() {@Overridepublic void onEvent(CodeFoundEvent event) {session.publishEvent(new ContentEvent(event.getType(),event.getValue()));...PlayerEndpointMedia Source
ZBarFilterSinkSRC
Events28http://www.kurento.orgThe magic of pipelines: Transparent media adaptationAgnostic media adaptorActs every time a source is connected to a sinkAdapts media formats as required by the involved media elements100% transparent for the application developer
Media ElementSinkSRC
Media ElementSinkSRC
An agnostic media adaptor is hidden behind every connection making compatible element formats transparentlyH.264VP829http://www.kurento.orgMediaSinkComplex examples: Heterogeneous group communicationsMedia SourceMedia SinkWebRtcEndpoint
Media SourceMedia SinkWebRtcEndpoint
Media SourceMedia SinkRtpEndpointRecorderEndpoint
FilterSinkSRC
Mixer
30http://www.kurento.orgMediaSinkMediaSinkComplex examples: WebRTC to HTTPMixerMedia SourceMedia SinkWebRtcEndpoint
RecorderEndpoint
HttpEndpoint
MediaSinkMixerHttpEndpoint
MediaSinkHttpEndpoint
31http://www.kurento.orgMedia SourceMediaSinkMediaSinkComplex examples: WebRTC PlayingMixerMedia SourceMedia SinkWebRtcEndpoint
RecorderEndpoint
HttpEndpoint
PlayerEndpoint
FilterSinkSRC
32http://www.kurento.orgWebRTC CV gamehttp://www.youtube.com/watch?v=CRqT7Q9KkRY
SinkSRC
Sink
SinkSRC
SinkSRC
SinkSRC
SinkSRC
WebRtcEndpointMirrorFilterPointerDetectorFilterChromaFilterFaceOverlayFilterMedia PipelineWebRTC full duplex client video applicationUpload toYoutube33http://www.kurento.orgCollaborations welcomehttp://www.github.com/kurentoThank you very much for your attention
Complains, suggestions and comments can be sent to:Luis [email protected]://www.kurento.org