View
24
Download
0
Category
Preview:
Citation preview
ROBINSYSTEMS
DatabaseContainerizationPlatformChecklist
AdeeshFulayDirectorofProducts
@AdeeshFadeesh@robinsystems.com
Agenda
› DataManagementChallenges› WhyContainers?› TheFeatureChecklist› RobinSolutionandDemos› Summary
DataManagementLandscapeandChallenges
Traditional ModernDataPlatforms
§ CostContainmento Highsoftwarelicense&supportcosts
§ DevOpsAgilityo Quickdatabaseclone,snapshotand
time-travel
§ PerformancePredictabilityo Unpredictableperformancein
consolidatedenvironments
§ OperationalComplexityo Painfuldistributedapplication
deployment,scalingandongoingmanagement
§ HighHardwareCAPEXo Clustersprawl,DataDuplication
§ EnterpriseGradeRobustnesso Backup,restore,failover,security
ExistingInfrastructureChoicesareInadequate 4
Bare-Metal VMs Purpose-BuiltAppliances
Containers
Inefficient
Complex
PerformanceOverhead
VM,DBsoftwareSprawl
VeryExpensive
Applicationspecific
Containers– WHAT?
› NextGenerationofVirtualizationtechnology› OS-based,anddeliversbaremetalperformance› OperationallyverysimilartoVirtualMachines› DedicatedIPaddress,filesystems,separateuserspace,etc.
› Agile› Containerscanbelaunchedinseconds
› Betterhardwareconsolidation› 10xormorehigherconsolidationbenefits
› EliminateOSduplicationandapplicationsoftwaresprawl
0
20000
40000
60000
80000
100000
120000
RandomReads RandomWrites RandomMixed
IOPS
BareMetal Container VM
Source:IBMResearchhttp://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf
Containers– 2xFasterthanVMs
PoorDockerAdoptionforDatabases
Dockerprimarilydesignedforstatelessapplications› Ignoresmostimportantcomponent– storage
OrchestrationtoolstreatDatabasesasstatelessaswell
https://goto.docker.com/Docker-Survey-2016.html
CattlevsPets!
› Petsaregivennameslikepussinboots.acme.com› Theyareunique,lovinglyhandraisedandcaredfor› Whentheygetill,younursethembacktohealth› Examples:Databases,bigdata,andotherstateful apps
› Cattlearegivennumberslikeinst0042.acme.com› Theyarealmostidenticaltoothercattle› Whentheygetill,yougetanotherone› Examples:Allstatelessapplications
FutureapplicationarchitecturesshoulduseCattle,butPetswithstrongconfigurationmanagementareviableandstillneeded
*OriginalslidefromCern regardingdifferentservicemodels
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
OriginalBlog: http://www.robinsystems.com/blog/stateful-database-containerization-platform-checklist/
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
1.MultipleContainerTypes(andmaybeevenVMs)
• TheNgnix• Node.js• Postgres
• Redis• MariaDB• Node.js
• ElasticSearch• Wordpress• Node.js
SYSTEMCONTAINERS
• Eachcontainerrunsanentireservicestack(multipleapplicationspercontainer)
• MeanttobeusedaslightweightVM
• Filesystemneutral
• Examples– LXC,OpenVZ,SolarisZones
• Supportsin-placepatch/upgrade&SSHaccess
GreatforTraditionalApplications
Ngnix
Node.js
Postgres
Redis
MariaDB
Node.js
ElasticSearch
Wordpress
Ngnix
APPLICATIONCONTAINERS
• Eachcontainerrunsasingleapplication(singleconcernpercontainerphilosophy)
• Meanttobeusedasapplicationpackage
• Containersuselayeredfilesystems(Overlayfs,AUFS,etc)
• Examples– Docker,Rocket
• Patch/Upgradeentailsreplacingcontainerimage
GreatforModernApplications
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
2.Application-awareInfrastructureCURRENT
DBAburdenedwithmanydecisions
DESIREDSimplifiedApplicationManagement,
InvisibleInfrastructure
Fluid,Elastic,HighPerformanceInfrastructurePlatform
Deploy
Scale
SnapshotClone
Restore
Capacityplanning
Storage
NetworkMemory CloudPerformance
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
3.DataPersistenceManagement
› Application=multiplecontainers
› Storageallocatedpercontainer,andnotperhostorVM
› MultipleVolumes/container› Blockstorage:formattoxfs,ext4,orraw› Eachcontainercanbindtomultiplevolumes
– HDD,SSD,NVMe
› Rapidcreate/destroy› Millisecondsresponsetimetocreate/delete
tokeepupw/containers
› Highervolume› 10-100xmorecontainersbeingcreatedthan
VMs
› Diskrepair,replication,compression,…
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
4.CloningProdtoTest/Dev
PROD Test Master
Downstream environments Dev & Test
Storage Usage for one application
› FullClonesofyourproductiondatabase
› Timeconsuming
› ResultsinDataSprawl
› CreateTestmastertoavoidperformancepenalty
› Needawaytocreateasmanyenvironmentsbutnotpaythestoragepenalty?
4.Snapshot,Clone,andRestore 17
ProdCluster
FullCluster,DistributedApplicationSnapshotsOS+DBsoftware+DBconfiguration+data
InstantClusterClone
ProdScaleTestEnv inMinutes
Need to test a change? Corruption or data loss?
2xstorageLongdatacopyManualre-configuration
ErasecommitlogsErasedatafilesRestorebackupRestartcluster
Lotmorethanjuststoragesnapshots
ProdCluster ProdCluster ProdCluster ProdCluster ProdCluster
VirtualTestCluster
InstantRestore
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
5.DatabaseWorkloadTypes
DataWarehousing(OLAP,Reporting,PredictiveAnalytics)
OLTP(Highlytransactionalsystem)
› LowLatency› Impactsuserexperience› SSDssuperiortoHDDs
› HighBandwidthi.e.Throughput› (Throughput=IOPSxBlocksize)› Dealwithlargevolumesofdata
› Multipledatabaseshittingthesamedisks› SustainedhighIOPS&PredictablelowlatencyConsolidated
=
5.NeedforQualityofService(QoS)
› NoisyNeighborproblem› Containersgiveprocessisolationbutnotperformanceisolation
› Predictableperformance› GuaranteedIOPSforapplication/container
› Scaleup/down› Traditionaldatabasesscaleup;moderndistributeddatabasesscaleout› Buthowdowecatertotransientspikes?› Containersonlyallowforcpu andmemory,butwhataboutIOPS?
Compute
Disk1 Disk2 DiskN
Consolidationw/containers
Compute
Disk1 Disk2 DiskN
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
6.As-IsFailover
ContainerFailover,Migration
APPLICATIONCLUSTERStatelessfailoverpolicieswillnotNOTworkfordatabases✔Containerrescheduledtoadifferenthostx ContainergetsanewIPx Losesdataasthevolumeisnotremountedonthenewhost
KeyUseCases:› Automaticfailover› Manualnodeevacuationforplannedmaintenance(e.g.OSorhardwareupgrades)› Clusterrebalance
22
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
7.NetworkAutomation
› Dockernetworkingisearlystageanddiverse(bridge,host,overlay,n/wdriver)› Applicationshavetochoosethebestoption› Docker,bydefault,usesLinuxEthernetBridgesforbasicrouting.Thishasalotofperformanceissues.SamewithOverlaynetwork.› Usethehostnetworkstackinstead(-net=host)
Databasesnetworkrequirements:› Spanmultiplehostsi.e.clustered› Eachcontainershouldhaveuniquename&IPaddresses› PurposefulDBnames(appdb.us.acme.com)registeredinDNS› Exposewellknownports(MySQL:3306,Oracle:1521)› Minimalnetworkperformanceimpact
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
8.Multi-Tenancy&Isolation
Tenant1 Tenant2
Tenant2
Tenant1
HardIsolation
SoftIsolation
› Sharednothing;eachtenantgetsdedicatedcompute
› DedicateddiskIOcapacity
› Addhardwareforextracapacity
› Poorconsolidationduetosilos
› Sharedcompute,multi-tenancyprovidedbycontainers
› Shareddisk,butdedicatedIOcapacity
› Containersresizedforextracapacity
› Higherconsolidation
TheDatabaseContainerizationPlatformChecklist
MultipleContainerFormats
Application-aware
infrastructure
Data Persistence(volume)
management
Snapshot,clone,restore
QualityofService
As-isfailover Networkautomation
Multi-tenancy&Isolation
Easeofsetupand
management
GuidingPrinciple:Dowhat’srightfortheapplication/database!
9.EaseofSetupandManagement
7CLUSTERMANAGER(mesos,kubernates)
8SCHEDULER
(marathon,cronos)
2STORAGEcapacitytier
(ceph,gluster,..)
4NETWORKING
(openvswitch,socketplane)
10MONITORING(graphite,sensu)
5CONFIGURATION(zookeeper,etcd)
9UI
(custom)
1SERVERHARDWARE(supermicro,dell,…)
3STORAGE
performancetier(SSD,Flash,NVMe)
11TROUBLESHOOTING
(splunk,logstash)
6CONTAINERS(docker,LXC)
1. Whensomethingbreaks,hardtoidentifytherootcause
2. Ifperformancedegrades,hardtoidentifyhotspots
3. Extendingtheplatform requiresconfiguring multiplecomponents
4. Keepingitoperationalrequiresknowledgeofallcomponents
11ComponentAssembly
WhatisRobin? 29
Industry’sFirst
APPLICATION-DEFINEDDATACENTERSoftware
Container-based,Software-definedInfrastructureforData-CentricApplicationsContainers+Storage+Networking+ApplicationOrchestration
SimplifyApplicationManagement
GuaranteePredictablePerformance
MaximizeCapacityUtilization
HowRobinWorks 30
Application-definedCompute
Application-definedStorage
EXISTINGCOMMODITYHARDWARE
ROBINCONTAINER-AWARE,STORAGEANDDATAPLANE
ROBINCONTAINER-BASEDCOMPUTEPLANE
Cloud-extendCompute&Storage
ROBINFABRIC
CONTROLLER
Fluid,Elastic,HighPerformanceInfrastructurePlatform
APPLICATIONORCHESTRATIONNETWORKING DATALIFECYCLEMANAGEMENT
HowRobinWorks 31
Application-definedCompute
Application-definedStorage
EXISTINGCOMMODITYHARDWARE
ROBINCONTAINER-AWARE,STORAGEANDDATAPLANE
ROBINCONTAINER-BASEDCOMPUTEPLANE
Cloud-extendCompute&Storage
ROBINFABRIC
CONTROLLER
Fluid,Elastic,HighPerformanceInfrastructurePlatform
DramaticallyAgile,SimpleApplicationManagement
Elastic,GuaranteedQoS
ConsolidatewithBareMetalPerformance
Deploy
Scale
SnapshotClone
Restore
APPLICATIONORCHESTRATIONQoS GUARANTEEDATALIFECYCLEMANAGEMENT
PlatformforALLDatabase&BigDataApplications
Stateless Stateful DataApps EnterpriseApps
Demos
› ClusterLifecycleManagement› Initialdeploy&scaleout
› QualityofService› Dynamic&real-timecontrolofCPU,Memory,&IOPS
› DataLifecycleManagement› Snapshot,clone,andrestore
› Reducehardwarefootprintby50%ormore
› Baremetalperformance
› GuaranteedApplication-to-SpindleQoS
Business
Benefits
ReduceCAPEX
› Simplifiedapplicationdeployment
› Self-servicedatalifecyclemanagement
ImproveProductivity
› 10xfasterapplicationdeployment
› Improvedapplicationuptime
ReduceOPEX
www.robinsystems.com info@robinsystems.com
THANKYOU
Recommended