View
5
Download
0
Category
Preview:
Citation preview
Toward A Model-Driven Design Tool for Big Data Architectures
Michele Guerriero
Saeed Tajfar
Damian Andrew Tamburri
Elisabetta Di Nitto
Introduction
Coordinator (Kafka)
Orchestrator (Hadoop Cluster)
Data Store
Batch LayerSpeed Layer
Serving LayerServing LayerServing Layer
Data Source
Data Source
Lambda architectureDistributedcomputation
Data streaming
HDFS
Distributed storage
Cloud infrastructure
1
How many Big Data technologies do I need to
know and combine?
Which resources, how many do I need and how
do we configure the deployed technologies?
What if I want to know properties andperformance of my application?
Simplify software design and reduce costs
Simplify Deployment
2
Questions Goals
Support Analysis
Platform Independent Model (DPIM)
Technology Specific Model (DTSM)
Deployment Specific Model (DDSM)
is implemented by
is deployed onto
TOSCA blueprint
Analysis
Analysis
Analysis & Optimization
M2M transformation
M2M transformation
M2T transformation
Model-Driven Big Data Design Architecture
4
Core DTSM Package
Storm Package
HadoopMRPackage
SparkPackage
<<Uses>>
<<Uses>>
Oryx 2Package
<<Uses>>
M2M Transformation
DPIM Model
Core DTSM Package
<<Uses>>
...Extensibility
DTSM Meta-Model
6
● Build Deployment Topology
● Standards adoption (TOSCA)
● Deployment Technological Packages
DDSM Meta-Model
8
● Preliminary steps toward the model-driven engineering of Big Data applications.
● Great potential behind MDE for data-intensive applications!
● Future steps:
○ increase models expressiveness (data quality, privacy concerns)
○ validation against industrial case studies
○ increase automation mechanisms
○ technological support
Conclusion and Future Works
10
Recommended