Hyunjun Kozzoon@igalia.com

VA-API bindingto Rust

Who is Hyunjun Ko (zzoon)

● Working on GStreamer since 2013.

● Regular contributor to Gstreamer VA-API

● Joined Igalia Apr, 2016.

● IRC : zzoon

GStreamer VA-API, what is it?

● A set of Gstreamer elements● vaapisink, vaapipostproc, vaapiXXXdec,

vaapiXXXenc.

● Uses VA-API software stack(libva) for hardware accelerated video processing

Rust, why?

● Safety, safety and Safety● Prevent from memory corruption and race

condition.● Doesn’t lose performance.

VA-API binding to Rust

● Name: libva-rust● https://github.com/zzoon/libva-rust● Produce FFI(Foreign Function Interface) to

libva.● Provide Rust structures and APIs for VADisplay,

VAContext, VASurface, VAImage…● Define Traits for Decoder, Encoder, Renderer..

What exists?● FFI generated by rust-bindgen 0.20

● Need to be re-generated by the latest one.

● Structures and APIs

pub struct VADisplay { disp: ffi::VADisplay, native_disp: *const VANativeDisplay, ...}

impl VADisplay { pub fn initialize(native_disp: *mut VANativeDisplay) -> Result<VADisplay, ()> pub fn get_profiles(&self) -> Box<Vec<VAProfile>> pub fn get_entrypoints(&self, profile: VAProfile) -> Vec<VAEntrypoint> …}

What exists?● Structures and APIs

pub struct VAImage { image: ffi::VAImage, } impl VAImage { pub fn new(...) pub fn put_image(&self, va_disp: &VADisplay) pub fn get_buffer(&self, va_disp: &VADisplay) pub fn get_num_planes(&self) pub fn get_offset(&self, idx: usize) pub fn get_stride(&self, idx: usize) }

pub struct VASurface { id: ffi::VASurfaceID, format: c_uint, width: c_uint, height: c_uint, num_surfaces: c_uint,}

Impl VASurface { pub fn new() pub fn put_surface() pub fn derive_image() pub fn sync() ...}

What exists?● Traits

● VARenderer

● VARendererX11

● Needs more traits for Decoder, Encoder, PostProc..

pub trait VARenderer: Send + 'static { fn open(&mut self) -> Option<u8>; fn close(&mut self) -> Option<u8>; fn set_resolution(&mut self, width: u32, height: u32) -> Option<u8>; fn render(&self, data: &[u8], len: usize) -> Option<u8>;}

impl VARenderer for VARendererX11 { fn open fn close fn set_resolution fn render}

Implement Gstreamer plugin

● Fork gst-plugin-rs (https://github.com/sdroege/gst-plugin-rs)

● Create new branch ‘vaapi’

● Implements rsvaapisink based on RsBaseSink.

Implement Gstreamer plugin● Code

extern crate libva_rust;

use libva_rust::va::*;use libva_rust::renderer::*;use libva_rust::renderer_x11::*;

Implement Gstreamer plugin● Code

impl VAapiSink { pub fn new(_sink: &RsBaseSink) -> VAapiSink {

…. let va_disp = VADisplay::initialize(native_display as *mut VANativeDisplay).unwrap(); let va_renderer = VARendererX11::new(va_disp, 0, 0).unwrap();

VAapiSink { va_renderer: va_renderer,

…. } }}

Implement Gstreamer plugin

● Code

● Example ● gst-inspect-1.0 rsvaapisink● gst-launch-1.0 videotestsrc ! video/x-

raw,format=NV12 ! rsvaapisink

fn render(&mut self, sink: &RsBaseSink, buffer: &gst::BufferRef) -> Result<(), FlowError> {

let map = match buffer.map_readable() { …

}; let data = map.as_slice();

self.va_renderer.render(&data, data.len());…

}

Plan

● Make it rusty more!● Implements Decoder/Encoder/VPP● Handle VASurface Memory by Caps● Support to share VADisplay from each element.

Q & A