Re-Implementing Apache Thrift usingModel-Driven Engineering Technologies:

An Experience Report

Sina Madani and Dimitris Kolovos

Department of Computer ScienceUniversity of York

{sm1748, dimitris.kolovos}@york.ac.uk

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 1/9

Motivation

Several experience reports on applications of MDE in industryMostly based on interviewsLimited availability of “hard data”

e.g. measurements extracted by implementing the samesoftware with/without MDE technologies

Such data can provide useful insights and/or act as a goodmarketing tool for MDE

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 2/9

In this work . . .

We re-implemented a subset of Apache Thrift using EMF,Xtext and Epsilon

Thrift is a web-services framework originally developed atFacebook (http://thrift.apache.org)Re-implementaton available inhttps://github.com/SMadani/ThriftMDE/

We performed quantitative and qualitative comparison of thetwo implementations

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 3/9

Apache Thrift

Interface description language for modelling services,input/output data types/structures (similar to CORBA’s IDL)Several transport protocols (e.g. JSON, binary, compressed)“Compiler” that produces client/server stubs from interfacedescriptions in Java, Ruby, JavaScript etc.Compiler implemented using flex, Bison and C++

Code generation through string concatenation (!!)

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 4/9

MDE re-implementation of Thrift’s Compiler

Hand-crafted abstract syntax → EMFflex, Bison → XtextModel validation in C++ → OCL-like constraints

Implemented using the Epsilon Validation Language (EVL)Concatenation-based code generators → Template-based codegenerators

Implemented using the Epsilon Generation Language (EGL)Only re-implemented code generators for Ruby and JavaTo enable automated equivalence testing we produce identicalcode (including bugs)

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 5/9

Quantitative Evaluation: Lines of Code

Implementation Original MDE DifferenceLanguage definition(parsing & validation)

3,419 447 -87%

Language-neutralcode

712 1,036 +68%

Java generator 5,129 2,224 -57%Ruby generator 1,231 422 -66%Total 10,491 4,149 -60%

RemarksWe only counted hand-written code for both implementations(i.e. excluded code generated by flex, Bison, EMF and Xtext)

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 6/9

Quantitative Evaluation: KBs (characters)

Implementation Original MDE DifferenceLanguage definition(parsing & validation)

105 14 -87%

Language-neutralcode

22 26 +15%

Java generator 187 73 -61%Ruby generator 40 14 -65%Total 395 128 -68%

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 7/9

Qualitative Evaluation

The MDE implementation is more modularCode generation is split over a number of templates for eachlanguage vs. all in one class in the current implementationValidation logic is contained in a set of EVL constraints vs.being spread across the codebase

Xtext is significantly more concise than flex and BisonUsing a template-based code generation language

Reduced the accidental complexity of string concatenationMade little difference in terms of the complexity of the outputcomputation logic

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 8/9

Conclusions and Further Work

The MDE re-implementation is significantly more concise(60-70%)

To argue about maintainability we need to conductexperiments involving developers

Xtext is a significant improvement over flex and BisonA template-based code generation language is more concisebut does not reduce the complexity of the code generator

Irreducable essential complexity?Our conclusions are based on one data point

Need to repeat this exercise with additional software to buildconfidence (e.g. Google Protocol Buffers?)

S. Madani et. al. – OCL Workshop 2016 September 14, 2016 – Slide 9/9