DICE Horizon 2020 Project Grant Agreement no. 644869 http://www.dice-h2020.eu Funded by the Horizon 2020

Framework Programme of the European Union

Quality-Aware Development of Big Data Applications with DICE

Damian A. Tamburri on behalf of G. Casale, E. Di Nitto and P. Jamshidi

The Rise of Big Data

o Software market rapidly shifting to Big data Expected 32% compound

annual growth rate in EU through 2016 Just 35% of Big data

projects are successful [CapGemini’15]

ESOCC - EU Projects Track

Data

Location

Velocity Volume

Transfer Privacy

Technologies

NoSQL

Hadoop DBaaS

Spark

Storm

Methods & Tools

UML

Prototyping

DevOps

Cloud

Integration

Quality

Value

ESOCC - EU Projects Track

The Rise of Big Data: Data-Intensive Applications (DIA)

Coordinator (Kafka)

Orchestrator (Hadoop Cluster)

Data Store

Batch Layer

Speed Layer

Serving Layer Serving Layer Serving Layer

Data Source

Data Source

• Continuous DIA architecting; • Infrastructure-as-code (e.g., TOSCA*) to allow track/update of “deployment”

architectures;

*Topology and Orchestration Specification for Cloud Applications (TOSCA) OASIS industry standard.

Source nodes

Messaging queue node

Storage node

…

TOSCA Blueprint

Live infrastr.

ESOCC - EU Projects Track

DIAs live in DevOps contexts!

ESOCC - EU Projects Track

MDE for Big Data, the DICE vision and approach

ESOCC - EU Projects Track

MDE for Big Data, the DICE vision and approach DevOps!

ESOCC - EU Projects Track

MDE for Big Data, the DICE vision and approach MDE!

MDE and DevOps: a possible synergy in DICE

ESOCC - EU Projects Track

Big Data Technologies

Cloud Testbed `

DICE IDE

Profile

Plugins

Sim Ver Opt

DPIM

DTSM

DDSM TOSCA Methodology

Deploy Config Test

Mon

Anomaly

Trace

IQE

Data Intensive Application (DIA)

CI FI

Design Time (Dev…)

Run-time (…Ops)

*http://dice-h2020.eu/ && https://github.com/dice-project/

DICE, a Complete MDE Toolkit for DIAs*

DICE Platform Independent Model (DPIM)

DICE Technology Specific Model (DTSM)

DICE Deployment Specific Model (DDSM)

is implemented by

is deployed onto

TOSCA blueprint

Analysis

Analysis

Analysis & Optimization

M2M transformation

M2M transformation

M2T transformation

o To provide sound abstractions and UML Profile for DIAs

o To provide a sound method for: o DICE-based

Continuous DIA Architecting

o DICE-based Continuous DIA Deployment

ESOCC - EU Projects Track

DICE incremental modeling and analysis[6]

Thanks!

www.dice-h2020.eu

ESOCC - EU Projects Track

Bibliography o [1] A. Rajbhoj, V. Kulkarni, and N. Bellarykar. Early experience with model-driven development of mapreduce based big data

application. In Software Engineering Conference (APSEC), 2014 21st Asia-Pacific, volume 1, pages 94–97, Dec 2014.

o [2] S. Santurkar, A. Arora, and K. Chandrasekaran. Stormgen - a domain specific language to create ad-hoc storm topologies. In Computer Science and Information Systems (FedCSIS), 2014 Federated Conference on, pages 1621–1628, Sept 2014.

o [3] Abel Gómez, José Merseguer, Elisabetta Di Nitto, Damian A. Tamburri “Towards a UML Profile for Data Intensive Applications”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS) workshop, to appear

o [4] Elisabetta Di Nitto, Pooyan Jamshidi, Michele Guerriero, Ilias Spais, Damian A. Tamburri “A Software Architecture Framework for Quality-Aware DevOps”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS) workshop, to appear

o [5] Matej Artacˇ, Tadej Borovsak, Elisabetta Di Nitto, Michele Guerriero, Damian A. Tamburri, “Model-Driven Continuous Deployment for Quality DevOps”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS) workshop, to appear

o [6] Guerriero, Michele; Tajfar, Saeed; Tamburri, Damian A. & Di Nitto, Elisabetta (2016), “Towards a model-driven design tool for big data architectures.”, in 'BIGDSE@ICSE' , ACM, , pp. 37-43 .

o [7] S. Bernardi et Al. “A Dependability Profile within MARTE”, JOURNAL OF SOFTWARE AND SYSTEMS MODELING.

ESOCC - EU Projects Track