High Level Application Scripting With EFL and LuaJIT

High level application scripting

with Enlightenment Foundation Libraries and LuaJIT

Daniel KolesaSamsung Open Source Group

[email protected]@octaforge

Lua Workshop 2015 Stockholm

The ProblemWhat are we dealing with?

What is the EFL?

I A suite of graphics (and other) libraries

I A widget toolkit

I Originally written for the Enlightenment window manager

I Scalable to both embedded and desktop systems

I Lightweight, written in C

I Significant usage includes Samsung’s Tizen platform

What is the EFL?


I A widget toolkit





What is the EFL?


I A widget toolkit





What is the EFL?


I A widget toolkit





What is the EFL?


I A widget toolkit





What is the EFL?


I A widget toolkit





What is the EFL?


I A widget toolkit





EFL components

I Historically, several projects with individual build systems

I Merged in version 1.8

I EFL - the core library set

I ELementary - the widget toolkit built on the EFL

I Some others still separate - Evas generic loaders etc.

EFL components






EFL components






EFL components






EFL components






EFL components






Evas

I The oldest EFL component

I A scene graph canvas

I Represents the canvas as state with Evas objects

I Provides several core primitives (rectangles, images etc.)

I Widgets provided by other libraries are Evas objects

Evas






Evas






Evas






Evas






Evas






Ecore

I Provides the interface between the OS and the rest of the EFL

I Interaction with X11, Wayland, SDL, Cocoa, DirectX etc.

I Provides a main loop for applications

I Also several other unrelated things

Ecore





Ecore





Ecore





Ecore





Edje

I Edje is a layout/UI description language

I It is also a way to compile said language together with data

I The result can be loaded from the application and used as UI

I Separation of UI/layout and application logic

I Widgets provided by libs can be embedded in Edje

Edje






Edje






Edje






Edje






Edje






Elementary

I A widget toolkit

I Provides a large set of widgets as Evas objects

I Can be used standalone (boxed style layouting defined from C)

I Can be used with Edje (Elementary widgets, Edje layout)

Elementary

I A widget toolkit




Elementary

I A widget toolkit




Elementary

I A widget toolkit




Elementary

I A widget toolkit




Eo

I An object system for the EFL

I The base for everything EFL 2.0

I Allows for backwards compatible legacy APIs

I Provides everything one would expect from an object system

I Much of EFL 2.0 API is interfaces

I We aim for API and ABI stability

Eo







Eo







Eo







Eo







Eo







Eo







Overview

I Sick and tired of writing apps in C

I There has to be a better way

I Scripting languages?

I They kind of suck

Overview




I They kind of suck

Overview




I They kind of suck

Overview




I They kind of suck

Overview




I They kind of suck

Why is that?

I Slow

I Lack of a proper type system

I Tons of bloat we don’t need

I Inconvenient binding

I Resource heavy

I Battery eating

I Could there be a solution?

Why is that?

I Slow




I Resource heavy

I Battery eating


Why is that?

I Slow




I Resource heavy

I Battery eating


Why is that?

I Slow




I Resource heavy

I Battery eating


Why is that?

I Slow




I Resource heavy

I Battery eating


Why is that?

I Slow




I Resource heavy

I Battery eating


Why is that?

I Slow




I Resource heavy

I Battery eating


Why is that?

I Slow




I Resource heavy

I Battery eating


Scripting language overview

Python

I An obvious choice at first

I Tons of problems

I Slooooow

I Bloated

I No decent unified API

I Hard to embed

I Language design issues

Python


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Python


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Python


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Python


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Python


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Python


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Python


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Ruby

I A not so obvious choice

I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Ruby


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Ruby


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Ruby


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Ruby


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Ruby


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Ruby


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Ruby


I Tons of problems

I Slooooow

I Bloated


I Hard to embed


Perl

I You serious?

I Tons of problems

I Slooooow

I Bloated


I Hard to... ok that’s enough

Perl

I You serious?

I Tons of problems

I Slooooow

I Bloated



Perl

I You serious?

I Tons of problems

I Slooooow

I Bloated



Perl

I You serious?

I Tons of problems

I Slooooow

I Bloated



Perl

I You serious?

I Tons of problems

I Slooooow

I Bloated



Perl

I You serious?

I Tons of problems

I Slooooow

I Bloated



Perl

I You serious?

I Tons of problems

I Slooooow

I Bloated



JavaScript

I Popular these days

I Fairly decent quality JIT implementations

I Each implementation has its own weird API

I Language is pretty bad

I But probably a better choice than the above

JavaScript






JavaScript






JavaScript






JavaScript






JavaScript






Io, Self, Squirrel etc.

I Many different choices

I Mostly slow interpreters

I Some are decent languages

I Mostly fairly light on resources

I Not good enough

I People would look at us weird






I Not good enough







I Not good enough







I Not good enough







I Not good enough







I Not good enough







I Not good enough


Lua

I A popular scripting language

I Good language with a few quirks

I Light on resources

I Great embedding API

I Interpreted... :(

I LuaJIT saves the day

Lua





I Interpreted... :(


Lua





I Interpreted... :(


Lua





I Interpreted... :(


Lua





I Interpreted... :(


Lua





I Interpreted... :(


Lua





I Interpreted... :(


Why is LuaJIT awesome?

I One of the fastest JIT compilers around

I Very light on resources

I Performance close to C for real world scenarios

I The exact same API as Lua!

I API and ABI compatible with Lua 5.1

I Portable bytecode

I We have a winner







I Portable bytecode

I We have a winner







I Portable bytecode

I We have a winner







I Portable bytecode

I We have a winner







I Portable bytecode

I We have a winner







I Portable bytecode

I We have a winner







I Portable bytecode

I We have a winner







I Portable bytecode

I We have a winner

Why LuaJIT sucks

I VM written in assembly - only a handful of architectures

I OTOH, it’s good enough

I Classic Lua C API is not JITed

I A handful of other things not JITed

I No multithreading support

I LuaJIT 2.1 significantly improves the situation

Why LuaJIT sucks







Why LuaJIT sucks







Why LuaJIT sucks







Why LuaJIT sucks







Why LuaJIT sucks







Why LuaJIT sucks







Elua

What is it?

I An application runtime

I Ships with the EFL

I Lightweight

I Reusable

What is it?



I Lightweight

I Reusable

What is it?



I Lightweight

I Reusable

What is it?



I Lightweight

I Reusable

What is it?



I Lightweight

I Reusable

What is it supposed to achieve?

I Rapid application development

I Clean, high level code

I Lightweight, good performance

I Easier app distribution (app store model)

I More safety (memory etc.)































How does it work?

I Elua library and binary

I Library provides Elua-related C utilities for the Lua state

I Binary is a minimal application launcher

I Together they set up Lua state + ecore main loop

I Most of Elua is written in Lua itself

How does it work?






How does it work?






How does it work?






How does it work?






How does it work?






Bindings

I Any decent application runtime needs APIs

I Writing bindings is a pain in the ass

I Maintaining bindings is a bigger pain in the ass

I What do?

Bindings




I What do?

Bindings




I What do?

Bindings




I What do?

Bindings




I What do?

Eolian

I Two things

I An API description language we invented

I A library to parse this language and expose the informationvia API

I We describe our EFL APIs in it

I We can reuse it to generate bindings

I No maintenance burden

Eolian

I Two things






Eolian

I Two things






Eolian

I Two things






Eolian

I Two things






Eolian

I Two things






Eolian

I Two things






Lualian

I Lualian is the Elua binding generator

I Itself a Lua application powered by Elua

I Easy to maintain and high level

I Bindings generated at EFL build time

Lualian





Lualian





Lualian





Lualian





How do our bindings work?

I LuaJIT provides a foreign function interface (FFI)

I This allows us to directly access C APIs

I Lualian generates FFI-powered bindings













Eo and Elua

I We have a Lua Eo layer

I This provides a native-feeling Eo Lua core

I Handles callbacks, method resolution and other things

I Generated bindings properly register bound classes with this

I Native-feeling APIs!

Eo and Elua






Eo and Elua






Eo and Elua






Eo and Elua






Eo and Elua






Other Elua APIs

I Elua also provides a collection of Lua utilities

I This includes a Lua-side object system

I Also bindings to most relevant Eina components

I Used by bindings

I Eo interacts with our Lua-side object system

I Completely transparent

Other Elua APIs




I Used by bindings



Other Elua APIs




I Used by bindings



Other Elua APIs




I Used by bindings



Other Elua APIs




I Used by bindings



Other Elua APIs




I Used by bindings



Other Elua APIs




I Used by bindings



Distribution

I Bundled with EFL → always available

I App store to promote development and distribution of apps

I Well supported - encouraging people to write apps

I Closely matching EFL APIs → free documentation

I Needs self-contained package format (use eet?)

I Make it possible to run Elua apps on Tizen?

Distribution







Distribution







Distribution







Distribution







Distribution







Distribution







Status

Where are we?

I Working, solid base

I Binding generator needs some work

I Elua library needs a lot of work

I EFL interfaces

Where are we?




I EFL interfaces

Where are we?




I EFL interfaces

Where are we?




I EFL interfaces

Where are we?




I EFL interfaces

EFL interfaces

I A linked project

I Design and implement EFL 2.0 APIs

I Eo based

I Why is this relevant to bindings?

EFL interfaces

I A linked project


I Eo based


EFL interfaces

I A linked project


I Eo based


EFL interfaces

I A linked project


I Eo based


EFL interfaces

I A linked project


I Eo based


Eo files

I Our current Eo files are not complete

I Missing enum definitions, structs etc.

I Complete Eo files → validation is possible

I Guaranteed definitions for types etc.

I Allows for proper binding generation

Eo files






Eo files






Eo files






Eo files






Eo files






What does this all mean?

I Our bindings are currently not fully functional

I Many will fail to load because of missing type definitions

I Dependencies are not correctly handled

I Elementary is not handled at all

I Our demos work with what we have + manually adjustedbindings































Other issues

I Internationalization and localization

I Seamless VFS integrated with Lua IO streams

I Sandboxing

I Startup time minimization

I Portable bytecode loading

Other issues



I Sandboxing



Other issues



I Sandboxing



Other issues



I Sandboxing



Other issues



I Sandboxing



Other issues



I Sandboxing



Code breakdown

Entry point

I Everything starts in the Elua launcher

I Lua state gets initialized

I Elua supports application lookup path

I Apps get executed

I Ecore main loop is started

Entry point




I Apps get executed


Entry point




I Apps get executed


Entry point




I Apps get executed


Entry point




I Apps get executed


Entry point




I Apps get executed


Module system

I Elua provides a custom module system that works outside ofLua’s

I Seamless (replaces require)

I Modules support init/shutdown callbacks

I Those take care of opening and closing the appropriate libs

Module system





Module system





Module system





Module system





Library system

I C libraries are opened and initialized as necessary

I Opened C libraries are cached and reference counted

I Eolian-generated bindings use this

I Symbols are retrieved and bound using C FFI

I This is wrapped in high level API (no FFI exposed to apps)

Library system






Library system






Library system






Library system






Library system






Eina bindings

I Elua provides hand-bound APIs from Eina

I Containers are read only

I Provides APIs to convert them to Lua values

I Various modules like accessor, iterator, list, log, file etc.

I Will be further expanded as necessary

Eina bindings






Eina bindings






Eina bindings






Eina bindings






Eina bindings






Other modules

I Elua includes a benchmarking module in core

I Also features a powerful command line argument parser

I Lualian is included

I A custom module system implementation

I Util module provides an object system, library system, stringbuffer implementation and other utils

Other modules






Other modules






Other modules






Other modules






Other modules






Conclusion

I Still largely incomplete

I Getting there

I Eo files need work

I Eolian validator needs to be fully enabled

I Lua APIs need plenty of work

Conclusion


I Getting there




Conclusion


I Getting there




Conclusion


I Getting there




Conclusion


I Getting there




Conclusion


I Getting there




Thank you.

Daniel KolesaSamsung Open Source [email protected]

@octaforgeLua Workshop 2015 Stockholm

Software

High Level Application Scripting With EFL and LuaJIT