Embed Size (px)
DESCRIPTION
By Víctor Jáquez. SysLink is a multi-core Inter-Processor Communication (IPC) software stack, pushed by Texas Instrument, where the developers could take advantage of asymmetric multiprocessing with several heterogeneous processors running different instances of operating system, whether Linux or any other flavor of RT OS. In the realm of the multimedia processing, this mechanism offloads the CPU-intensive (de)codification tasks from the host processor, enhancing the user experience, particularly in embedded devices, where specialized processing units could be stored on SoCs. This talk will describe the main SysLink components (Linux kernel API, RCM modules, etc.) and how we could map those concepts into GStreamer elements, providing hardware accelerated multimedia, but without the burden of OpenMAX. The target SoC is the OMAP4 using a panda-board.
Citation preview
GStreamer & SysLink
Víctor Manuel Jáquez Leal
2011-10-22 Sat
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 1 / 37
Outline
1 Introduction
2 Objective
3 SysLink
4 Hardware
5 GStreamer
6 Future
7 References
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 2 / 37
Parallel computing
Serial computing is now dead.
Parallel computing is a revolution that is now upon us.
Although it started more than 40 years ago.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 3 / 37
Why Parallel computing?
To achieve performance guarantee.
It generally leads to lower power consumption.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 4 / 37
Parallel computing challenges
Programming for serial computing is already di�cult.
Programming for parallel computing will only exacerbate thisdi�culty.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 5 / 37
Parallel computing approaches
Symmetric Multiprocessing
Identical processors.
A single shared mainmemory.
A single OS instance.
Asymmetric Multiprocessing
Might be di�erent type ofprocessors.
Might have private memory.
Might run di�erent OS.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 6 / 37
Parallel computing approaches
Symmetric Multiprocessing
Identical processors.
A single shared mainmemory.
A single OS instance.
Asymmetric Multiprocessing
Might be di�erent type ofprocessors.
Might have private memory.
Might run di�erent OS.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 6 / 37
Symmetric MultiprocessingSMP
It is the most commoncon�guration.
It has bandwidth issues.
It lacks of fault-tolerantfeatures.
The cache coherencymechanism consumes toomuch power.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 7 / 37
Asymmetric MultiprocessingAMP
A problem in one kernel/CPUwill not a�ect the wholesystem.
Applications must use adi�erent API.
Requires more hardwareresource management.
Wider kernel memoryconsumption.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 8 / 37
Another AMP Con�guration
One master processor.
Multiple remote processors.
slave processors are treated asdevices (under /dev).
One shared memory butregistering private regions.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 9 / 37
Inter-Processors CommunicationIPC
An IPC mechanism is key for the AMP architecture.
Sharing memory regions between processors.Data interchange.
Passing messages between processors.Task synchronization.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 10 / 37
Examples of such multi-core SoCs
Texas Instruments's OMAP processors
Qualcomm's SnapDragon processors
Broadcom's SiByte family of processors
Freescale's 8641D dual-core PowerPC-based SOC.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 11 / 37
What is SysLink?
SysLink is the latest generation AMP/IPC driver developed byTexas Instrument.
It provides mechanisms to control and communicate with remoteprocessors.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 12 / 37
Objective
This talk will describe the main SysLink components and how wecould map those concepts into GStreamer elements.
No OpenMAX
No extra libraries (syslink-userspace / libmemmgr / etc.)
Warning!!!
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 13 / 37
Objective
This talk will describe the main SysLink components and how wecould map those concepts into GStreamer elements.
No OpenMAX
No extra libraries (syslink-userspace / libmemmgr / etc.)
Warning!!!
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 13 / 37
Project
Gitorious repository page
https://gitorious.org/vj-pandaboard/syslink/
These slides are also here
https://gitorious.org/vjaquez-misc/syslink-talk/
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 14 / 37
History of TI's AMP/IPC implementations (1/6)
OMAP2: Reference implementation.
OMAP3: DSP/Bridge
OMAP2/OMAP3/DaVinci: DSPLink
OMAP4: SysLink
OMAP4/OMAP5: rpmsg / remoteproc
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 15 / 37
History of TI's AMP/IPC implementations (2/6)
OMAP2: Reference implementation.
dsp-gateway by Nokia.Full open source.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 16 / 37
History of TI's AMP/IPC implementations (3/6)
OMAP3: DSP/Bridge
Full featured implementation.Only supports DSP (C64x+).Linux side into stagering repository.DSP side is restricted.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 17 / 37
History of TI's AMP/IPC implementations (4/6)
OMAP2/OMAP3/DaVinci: DSPLink
A stripped implementation of DSP/Bridge.It was not submitted to upstream.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 18 / 37
History of TI's AMP/IPC implementations (5/6)
OMAP4: SysLink
Supports other types of cores (ARM Cortex-M3 and C64x)Handled in downstream (Ubuntu)
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 19 / 37
History of TI's AMP/IPC implementations (6/6)
OMAP4/OMAP5: rpmsg / remoteproc
Developed completely in upstream.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 20 / 37
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Zero-copy shared memory: . . .
2D memory bu�ers allocation: . . .
Remote function calls: . . .
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 21 / 37
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Zero-copy shared memory: . . .
2D memory bu�ers allocation: . . .
Remote function calls: . . .
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 21 / 37
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Zero-copy shared memory: . . .
2D memory bu�ers allocation: . . .
Remote function calls: . . .
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 21 / 37
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Zero-copy shared memory: . . .
2D memory bu�ers allocation: . . .
Remote function calls: . . .
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 21 / 37
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Zero-copy shared memory: . . .
2D memory bu�ers allocation: . . .
Remote function calls: . . .
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 21 / 37
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Zero-copy shared memory: . . .
2D memory bu�ers allocation: . . .
Remote function calls: . . .
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 21 / 37
Key features of SysLink
Messaging: . . .
Dynamic memory management: . . .
Dynamic loading: . . .
Power Management: . . .
Zero-copy shared memory: . . .
2D memory bu�ers allocation: . . .
Remote function calls: . . .
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 21 / 37
SysLink v2 architecture
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 22 / 37
SysLink v2 components
MultiProc: Identi�es each available processor.
SharedRegion: Handles shared memory areas.
Gate: Protects memory regions from remote processors.
MessageQ: Variable length message passing across processors.
Notify: Registers callbacks execute when a remote event occur.
HeapBuf: Manages �xed size bu�ers within the shared memory.
HeapMem: Manages variable size bu�ers within the sharedmemory.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 23 / 37
SysLink v2 components
MultiProc: Identi�es each available processor.
SharedRegion: Handles shared memory areas.
Gate: Protects memory regions from remote processors.
MessageQ: Variable length message passing across processors.
Notify: Registers callbacks execute when a remote event occur.
HeapBuf: Manages �xed size bu�ers within the shared memory.
HeapMem: Manages variable size bu�ers within the sharedmemory.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 23 / 37
SysLink v2 components
MultiProc: Identi�es each available processor.
SharedRegion: Handles shared memory areas.
Gate: Protects memory regions from remote processors.
MessageQ: Variable length message passing across processors.
Notify: Registers callbacks execute when a remote event occur.
HeapBuf: Manages �xed size bu�ers within the shared memory.
HeapMem: Manages variable size bu�ers within the sharedmemory.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 23 / 37
SysLink v2 components
MultiProc: Identi�es each available processor.
SharedRegion: Handles shared memory areas.
Gate: Protects memory regions from remote processors.
MessageQ: Variable length message passing across processors.
Notify: Registers callbacks execute when a remote event occur.
HeapBuf: Manages �xed size bu�ers within the shared memory.
HeapMem: Manages variable size bu�ers within the sharedmemory.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 23 / 37
SysLink v2 components
MultiProc: Identi�es each available processor.
SharedRegion: Handles shared memory areas.
Gate: Protects memory regions from remote processors.
MessageQ: Variable length message passing across processors.
Notify: Registers callbacks execute when a remote event occur.
HeapBuf: Manages �xed size bu�ers within the shared memory.
HeapMem: Manages variable size bu�ers within the sharedmemory.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 23 / 37
SysLink v2 components
MultiProc: Identi�es each available processor.
SharedRegion: Handles shared memory areas.
Gate: Protects memory regions from remote processors.
MessageQ: Variable length message passing across processors.
Notify: Registers callbacks execute when a remote event occur.
HeapBuf: Manages �xed size bu�ers within the shared memory.
HeapMem: Manages variable size bu�ers within the sharedmemory.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 23 / 37
SysLink v2 components
MultiProc: Identi�es each available processor.
SharedRegion: Handles shared memory areas.
Gate: Protects memory regions from remote processors.
MessageQ: Variable length message passing across processors.
Notify: Registers callbacks execute when a remote event occur.
HeapBuf: Manages �xed size bu�ers within the shared memory.
HeapMem: Manages variable size bu�ers within the sharedmemory.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 23 / 37
Remote Command MessagingRCM
It handles remote function calls.
RCM client
It manages the allocation, sending, receiving of RCM messagesto/from the RCM server.
RCM Server
It processes inbound messages received from an RCM client.After processing a message, the server will return the message tothe client.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 24 / 37
RCM Usage
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 25 / 37
OMAP4430Pandaboard
Dual-core ARM Cortex-A9 MPCore.
Dual-core ARM Cortex-M3.
DSP C64x (not much info).
3D graphic acceleration PowerVR SGX540.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 26 / 37
Ducati
The two Cortex-M3 processors comprise Ducati.
Sys M3: runs the Notify driver.App M3: executes the entire multimedia software.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 27 / 37
Distributed Codec EngineDCE
libdce
It is library to remotely invoke the hardware accelerated codecson IVA-HD via SysLink/RCM.
DCE AppM3
It is similar in purpose to remote CE for the OMAP4430.CE is a framework that enables applications to instantiate andwork with codecs and algorithms using a common API.It is synchronous (sends a bu�er and waits for the output).It is open source (TI's licence).
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 28 / 37
Distributed OpenMAXDOMX
It is a RPC shim for an IL client to use a component on Ducati
DOMX AppM3
It is asynchronous.It is closed (distributed binary only).Tightly coupled with OpenMAX IL API.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 29 / 37
DMM/TILER
It organizes video/image memory in a 2D fashion.
It facilitates
Allocating.
Freeing.
Mapping.
of 2D blocks in the container(s).
Also
Rotating
Mirroring
the 2D blocks or its rectangularsubsections.
To be replaced by GEM (The Graphics Execution Manager)https://lwn.net/Articles/283798
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 30 / 37
Row Stride
The stride is the number of bytes from one row of pixels inmemory to the next row of pixels in memory.
If padding bytes are present, the stride is wider than the width ofthe image.
DMM/TILER 2D bu�ers require 4KiB row-stride
GstStrideTransform (2009 thread)
http://gstreamer.freedesktop.org/wiki/NewGstVideovideo/x-raw-yuv-strided ???
https://gitorious.org/gstreamer-omap/gst-plugins-base/trees/v0.10.35+ti/gst/stride
Use GstMeta in 0.11 ???
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 31 / 37
2D TILER Bu�er Pool
GstDucatiBu�erPool (1D)
https://gitorious.org/gstreamer-omap/gst-ducati/blobs/master/src/gstducatibu�erpool.c
But for 2D ??
Use GstBu�erPool ???
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 32 / 37
Sync vs Async output bu�ers
DCE is sync
No problem here.
DOMX is async
How to handle it?GstPad tastks and queue in a thread ?
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 33 / 37
rpmsg / remoteproc
rpmsg
A virtio-based messaging bus that allows kernel drivers tocommunicate with remote processors available on the system.
remoteproc
A generic driver that maintains the state of the remoteprocessor(s).Firmware loading.
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 34 / 37
Conclusion
Is it possible to live without OpenMAX?
How to handle GEM/TILER bu�ers?
strides?bu�er pool?Asynchronous output bu�ers?
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 35 / 37
Image credits
ASMP vs SMPhttp://ohlandl.ipv7.net/CPU/ASMP_SMP.html
Who's Afraid of Asymmetric Multiprocessing?
http://rtcmagazine.com/articles/view/100404
Ducati for dummieshttp://www.omappedia.org/wiki/Ducati_For_Dummies
SysLink project
http://omappedia.org/wiki/Syslink_Project
Work in Progress (CC BY-NC 2.0) blumpy
https://secure.�ickr.com/photos/10537966@N00/325853852/
Pandaboard (C) Tiago Maluta
https://secure.�ickr.com/photos/22248939@N04/5877274222/
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 36 / 37
Thanks!
Questions?
Víctor Manuel Jáquez Leal () GStreamer & SysLink 2011-10-22 Sat 37 / 37
