1. Real Time Media Stream Processing with FI-WAREIvan GraciaKurento.orgigracia@kurento.org, @kurentoms

2. Introducing multimedia infrastructuresPeer-to-Peer Application (without media infrastructure)WebRTC video streamApplication with media infrastructureMedia infrastructure1 3. Multimedia infrastructures for theFuture Internet2Media ishereMedia gotthereMedia gotthereAnalyseTransformStoreTranscodingMCURecordingEnrichAugmentAdaptSensorsContextEventsMedia ishere 4. Enrich, augment, adapt, analyse, transform, store: whatsthe problem?Complexity3 5. The Stream Oriented Generic Enabler SO GE Interoperable media exchange (multiplatform/multiprotocol)WebRTC, RTP, HTTP (video tag), etc. Standard capabilities Transcoding, MCU, recording Advanced capabilities Computer vision, augmented reality, mixing, blending, etc.Is distributed through a flexible FOSS license4Multimedia infrastructureAPIs REST API JavaScript API Java API LGPL 2.1 6. Kurento: the equation5FutureInternetMultimediaInfrastructureSimpleDevelopmentAPIs5 7. General Architecture6 8. Key concepts: media elements and pipelines7Media Element Provides a specific media functionality Send/receive media. These are the Endpoints Process media Transform media Ready to be used New media elements can be addedMedia ElementSinkSRCMedia pipeline Chain of media elements implementing thedesired media logic The Media Server provides the capability ofcreating media pipelines by joining mediaelements of the toolboxMedia Pipeline 9. 8 10. 9 11. Developers create applications just connecting elementsApplication 2Application 3Application 1Toolbox of media elements10 12. Media Server (KMS): The nucleus of KurentoKMS is a middleware for media streams:Receives the streamProcess the streamIssues the stream11SendReceiveAnalyzeAugmentEnrichTransformTranscodeRecordProcessReplicateMediaSourceMediaSinkKMS 13. How to control the media server?REST APIJSON RPCWebsocket interface in KMS12 14. API ImplementationsFor now13 15. Kurento ProtocolJSON-RPC over WSMedia TrafficClient Kurento14Application codeMedia TrafficMedia ServerApplicationServerJava KurentoClientJavaScriptKurento ClientApplication codeApplication code KurentoProtocolJSON-RPCover WSSignalingprotocolKurentoMedia ServerClientKurento Applications 16. Application execution flow15ClientCodeKurento API MediaServer (KMS)I want this media in this typeCommands requestingthe creation of a pipeline(JSON-RPC request)What you want is herePipelinecreationphaseMediaexchangephase12Specific logic(e.g. AAA)MediapipelinecreationMedia exchange between client and server 17. Campus Party Brazil - 20141616 18. Campus Party Brazil - 201417 19. Lets get started!Tutorialshttp://www.kurento.org/docs/current/tutorials.html18 20. What do we need?An instance of the Kurento Media ServerFI-LABLocal installationMavenNPMBowerYour favorite IDE19 21. Tutorial 1 - Hello world20 22. Tutorial 1 - Hello world21 23. Tutorial 1 - Hello world, server22 24. Tutorial 1 - Hello world, client23 25. Tutorial 1 - Hello world (Node.js)24 26. Tutorial 1 - Hello world (Node.js)25 27. Tutorial 2 - WebRTC magic mirror26 28. Tutorial 2 - WebRTC magic mirror27 29. Tutorial 2 - WebRTC magic mirror28 30. Tutorial 3 - WebRTC one-to-one video call with recordingand filtering29 31. Tutorial 3 - WebRTC one-to-one video call with recordingand filtering30 32. Expanding Kurento31 33. Media Server structureKurento is a pluggable framework.Kurento-media-server uses:Kms-coreKms-elementsKms-filtersOther available modules:Kms-crowddetectorKms-pointerdetectorKms-chromaKms-platedetector32KurentoMedia ServerKMSCOREKMSELEMENTSKMSFILTERSKMSCROWDDETECTORKMSCHROMAKMSPLATEDETECTORKMSFILTERS 34. How to develop a module?Two flavours:Gstreamer developerOpenCV developerDefine the module using IDLThe tool kurento-module-creator will do themagic:Generate code for server-sideGenerate code for clientes.33 35. How to develop a module?1. Create module structure using kurento module scaffold2. Modify kmd file to add module properties3. Generate code4. Add filter functionality5. Compile6. Install7. Generate client-side code34 36. Example 1: For GStreamer developershttps://github.com/Kurento/kms-plugin-sample35 37. Example 2: For OpenCV developershttps://github.com/Kurento/kms-opencv-plugin-sample36 38. Kurento and the FI-LAB FI-LAB Working instance of FI-WARE enabling freeexperimentation with technology http://lab.fi-ware.org Creating a Kurento instance from an image Use latest version of Kurento images. Creating a Kurento instance using recipes Use Ubuntu 14.04 LTS clear image Use latest version of Kurento recipes37 39. Kurento and the FI-LAB38 40. To learn more FI-WARE web page http://www.fi-ware.org Kurento Web page http://www.kurento.org Kurento source code https://github.com/kurento39 41. Thanks!40