View
1
Download
0
Category
Preview:
Citation preview
EUBra-BIGSEA:cloud serviceswith QoS guarantees for Big
DataAnalytics
IgnacioBlanqueron Behalf oftheEUBRA-BIGSEAConsortium
www.eubra-bigsea.eu|@bigsea_eubr
EUBra-BIGSEA
www.eubra-bigsea.eu |@bigsea_eubr
• AEuropean-BrazilianConsortiumaimingat― ThedevelopmentofQoS andsecurecloudservicestosupportBig
Data.― ThedevelopmentofBigDataservicesforcapturing,federatingand
annotatinglargevolumesofdata.― Theuseofefficient technologies forguaranteeingthefulfilmentof
thesecurity andprivacy policies.― Thetransfer ofthistechnologytoareal user scenario withhigh
socialandbusinessimpact,andofhighinterestforbothEUandBR.
BIGSEAinaNutShell
www.eubra-bigsea.eu |@bigsea_eubr
• BIGSEAis structured along 7(+Projectcoordination)activities defined asWPs.
• User Scenario will make useofthe DataAnalytics APIandthe Programming modelinterface.
• The Programming model interfacewill usethedervices deployed using TOSCAstandard
― Service specifications will beinstantiatedon acloud IaaS,automatically managingQuality ofService.
― The BigData Analytics services will runonthe cloud infrastructure.
• Asecurity framework will bedefined toprovide security andprivacy.
• Globally,dissemination andexploitationactivities will pursue the outreach andtransferoftechnology.
QoS IaaS
BigDataAnalytics
Program.model
interface
Security
UserScenario
Dissemin.&Outreach
InnovationManag.
Accesstokens
Assurance
BigDataPrivacy
Assurance
Accesstokens
TOSCAspecs
Requests
DevelopmentIDE
BIGSEAArchitecture
www.eubra-bigsea.eu |@bigsea_eubr
Mesos
Scheduler
Resources
CloudManag.Framew.
EC3/IM
CLOUDVAMP
Monitor
ProactivePolicies
Program.Model
(COMPSs)
IDE(Lemonad
e)
Creates & executesruntime
Triggers Vertical elasticity
Request and configures VMs
ProvidesAgents
PhysicalResources
LogicalComponents
ManagementofPhysicalResources
Physical Resources
www.eubra-bigsea.eu |@bigsea_eubr
• Resources areprovisioned by aMesos Cluster totheexecution frameworks- COMPSs,Marathon,Spark,Chronos.
• Mesos Cluster is composed ofasetofVMs provisionedfrom aCMF- Automated Deployment- Platform-agnosticism- Vanilla VMIs
• HorizontalElasticity atthe level ofthe resources- The registration ofnewframeworks will check for the availability ofthe needed resources.
- Newrequests will trigger booting upnewresources if needed.- Idlepure computing resources will bepowered off.
• VerticalElasticity atthe level ofthe VMs- Higher flexibility inthe allocation ofphysical resources.
VMRC
InfrastructureManager
CloudSelectorVMRC
CloudConnector
Conf.Manager
FicherosConf&CTX
XML-RPCAPI RESTAPI
WebInterface CLIInterface
MV
MV
MV
...
RADL
VMMasterCntxt.Agent
Ansible
OpenNebula OpenStack EC2 ...
Ansible
VMRC
...
Infrastructure Manager(IM)www.grycap.upv.es/im
www.eubra-bigsea.eu |@bigsea_eubr
• TwoapproachestoVMImanagement- Deployexistingvanillaimages(plainOS)andconfigureinstancesusingtoolstore-createthedesiredconf.• Example:InstantiateaplainUbuntu12.04AMIon
AmazonEC2anduseAnsible toautomaticallyinstallafullLAMP(Apache,MySQL,PHP)stack.
- CreatespecificVMIsfordifferentCloudsfromtemplates.• Deploymentsbasedonrecipes,configurationand
contextualizationservices.
- Enablesplatform-agnosticorhybriddeployments.
VMI catalogue
Platform-agonstic Deployment
www.eubra-bigsea.eu |@bigsea_eubr
LocalComputer
IMserver
EC3client
RADL
credentials
$ docker pull eubrabigsea/ec3client$ docker pull grycap/im
Mesosmaster
HadoopDN
HadoopDN
HadoopDN
Mesosslave
Mesosslave
Mesosslave
Mesosslave
$ ./ec3 launch myMesosHadClus ubuntu-one mesos docker spark nfs hadoop-a /root/auth.dat -u http://localhost:8899
Bound to ONE, but easy to extend by:• Changing base image• Providing credentials
Address of the IM server
Credentials
Platform-agnostic Deployment
www.eubra-bigsea.eu|@bigsea_eubr
• Threeparts:• Cloud-backend(ubuntu-one.radl)
• Threeobjects:front,wn &wnmesos• Fourattributesperobject:name,imageurl,usernameand
password.• ResourceManagementFramework(mesos.radl)
• Definesnetwork:ports,DNSnames,interfaces.• Definessystemvirtualhardware:Memory,CPUs.• Configuration:MasterandSlaves.
• Otherdependencies• docker.radl:On every node.• spark.radl:On every node.• nfs.radl:On every node.• hadoop.radl:on master&wn instances.
Elastic ComputeClusters on the Cloud(EC3)
• EC3(Elastic CloudComputingCluster)leverages CLUEStocreateelastic virtualclusters inthe Cloud.
- Noupfront investment.- Customizable.- Usage-dependent cost.- Automaticelasticity.
Resource-level elasticity
• When anewMesos framework is requested,EC3will check ifthere areenough resources tofulfill the request
- Additionally,Marathon andChronos plugins will capturethe details oftherequest.
- EC3will request the CloudManagementFrameworktodeploy newVMs toallocate the necessary resources for the newframework• EC3usesInfrastructure Manager
toinstall andre-configurethe software.
- When resources become idlefor awhile,they areundeployed.
- Dataresources arenotundeployed.
Mesosmaster
Mesos MasterDocker
ChronosMarathon
SparkHadoop NN
Hadoop2ndNN
NFS ServerEC3
Vnode1Hadoop DN
Spark
Vnode2Hadoop DN
Spark
Vnode3Hadoop DN
Spark
Vnode4Mesos Slave
DockerSpark
mounted home
Vnode7Mesos Slave
DockerSpark
mounted home
158.42.104.229
10.0.0.6710.0.0.70 10.0.0.74 10.0.0.81
10.0.0.XX10.0.0.XX
Verticalelasticity attheresource level
•VMs areallocated from aCMFthat manages ageneral-purpose on-premise cloud• Physical limitations arenormally atthe memory sizerather than atthe CPUshare.
• Aphysical machinecanbeshared among differentdeployments• Both BIGSEAstack or other one.• Or due toincreased isolation.
• Oversubscribing memory canenable reaching highernumber ofVMs perphysical node,not bounding toaspecific partitioning.• AVMfrom one deployment can”release”the freememory toother VMs inthe node.
Infrastructure management
• Interaction with the CLUESclient toget information about theresources powered on /offandtoforce them tobepowered on /off.
• Mainly intended for administration,it canbeused toanticipatepowering on aresource .
/v1/resource/slaves GET method, lists all the resources registered in the master orcluster of masters.
/v1/resource/slaves/slaveid GET method, provides the status information of a specific slave.
/v1/resource/up PUT method, boots up a new resource.
/v1/resource/slaves/slaveid DELETE method, powers down a specific resource.
/v1/resource/slaves/slaveid/enable POST method, enables a disabled resource.
/v1/resource/slaves/slaveid/disable POST method, disables an active resource – it is not considered anylonger to be powered off by the system.
Automated Oversubscription forCloudPlatforms (CLOUDVAMP)http://www.grycap.upv.es/cloudvamp
•Asystem todynamically adjust the physical memoryallocated toVMs tothe realusage.
• It canbeused toallocatemoreVMs than inafixedmemory allocation model.
• It resizes memory inthe VMasneed andeven migratesVMs tonewhosts.
Logical Components
User-level interaction
User’sComputer
github
Docker Hub BIGSEA end-point
BIGSEA LemonadeIDE
Resources
app
Spark / COMPSsGenerated
code
Docker commit& push (3)
Dockerfilecommit & push (3) Automated
build (3)Docker pull
(5) Framework (5)
Submit(incl QoS)
(4)
BIGSEA QoSproactive
Resourcerequest (4)
Docker pull (1)
Characterization (4)
User-level interactionUser’sComputer
github
Docker Hub
BIGSEA end-point
BIGSEA LemonadeIDE
Resourcesap
p
COMPSs / Spark UserCode (2)
GUI (2)
Spark / COMPSs Generatedcode (3)
Docker commit& push (3)
Dockerfile commit & push (3)
Automatedbuild (3)
Docker run (5)
Framework (5)
Submit(incl QoS)
(4)
BIGSEA QoSproactive
Resourcerequest (4)
Docker pull (1)
Characterization (4)
ProgrammWriting
QoS characterization
CustomizingDepend.
ProactiveResourceCharacter.
Submission
• Combining Spark and general code in an IDE.• Using COMPSs for DAG parallelilsation.• Use of pre-characterized building-blocks and algorithms.
• Container images with BIGSEA dependencies (preferred), alsosupporting direct execution (e.g. Spark).
• Automated build or container push.
• Run-time policies optimize resource allocation at initial deployment. • Performance models recalculate QoS metrics (execution time) and
trigger optimization module (if necessary).• API provides system parameter reconfiguration.
• Submission from a single end-point.• Frameworks interacting with the Mesos cluster.• Adding QoS constraints (deadlines, execution time,…) and
previous proactive resources demand characterization.
SingleSubmission end-point
• JSONdocument with the information on theexecution container,framework resources andQoS.
/v1/scheduler POST method, submits a jobrequest in JSON.
/v1/scheduler/jobs GET method, lists all the jobs inthe scheduler (a JSON with all thejob). It can accept a JSON with adeadline that defines the datelimit.
/v1/scheduler/job/jobid GET method, gets all theinformation from a specific job;DELETE method, kills the specificjob; POST method, reallocatesresources.
/v1/scheduler/job/jobid/scale PUT method, changes theresource allocation of aframework.
{ "type": "CMD","name": "my_job_name","deadline": "2016-06-10T17:22:00Z+2","periodic": "R24P60M","expectedduration": "10M""container" : [
"type": "DOCKER","image": "eubrabigsea/ubuntu","forcePullImage": true"volumes": [{ "containerPath": "/var/log/","hostPath": "/logs/","mode": "RW"}],
"portMappings": [{ "containerPort": 8080,"hostPort": 0,"protocol": "tcp" }]],
"environmentVariables": [{ "name": "value" } ],"cpu" : "1.5", "mem" : "512M", "disk" : "1G","command" : "python -m SimpleHTTPServer 8000"
}
QoS Proactive Policies• Policies arebased on historical data(lookuptable)and
current system status(QoS monitoring)• Analytical performancemodels provide newestimation of
quality metrics tothe optimization module(atinitialdeployment andexecution times)
• Optimization modulerecalculates resource allocation,ifnecessary (newapplicationarrival,changeson the systemconfiguration).
Monitoring•UseOpenStackMONASCAasmonitoringframework
•Definemetrics atthecontainer level.
• Triggers newresource allocationrequest ifnecessary.
VerticalElasticityattheleveloftheframework
•Directly managed by the submission service• It hides the different framework features,interactingwith the scheduler and/or Mesos.
•Asexample• InChronos it gets the current configuration ofthe job andresubmits the job with the newresource allocation andthesame configuration.
• InMarathon it updates the json andusesthe specific APIcall$ curl -i -L -H 'Content-Type: application/json' -X PUT -d@”temporary.json" marathonserver:port/v2/apps/appname?force=true
• InSpark andCOMPSs directly interacting with the MesosFramework.
Elasticity scenario
• Two multithreaded processing benchmarksrunningconcurrently inthe same node• Change the allocation from 50%-50%ofCPUto25%-
75%asthe deadline was approaching.
Conclusions
•EUBra-BIGSEAaims atproviding asetofcloudservices to• Facilitate the deployment ofcomplex multi-frameworkBigDatainfrastructures.
• Provide elasticity atthe level ofthe physical resourcesandframeworks.
• Include QoS Constraints.• Facilitate the useofcustomized environments.
•Those cloud services aredirectly used by higher-level programming models hiding the platformparticularities.
Recommended