Orchestra)ngCloudNa)veApplica)onsatScaleintheEnterprise

CameronBrunnerArchitect–CloudNa)veProjects

CloudNa)veCompu)ngFounda)on

Mission:Tocreateanddrivetheadop)onofanewcompu)ngparadigmthatisop)mizedformoderndistributedsystemsenvironments.Thepar)cipantsbelievethatsystemsarchitectedwillbe:•  Containerpackaged•  Dynamicallymanaged•  Micro-servicesoriented

•  Highperformancecompu)ng/technicalcompu)ng•  Extremelylargeclustersperformingcomplexcomputeworkloads

•  GenomeSequencing,FluidDynamics,RiskModeling,OilandGasExplora)on,SemiconductorDesignandTes)ng,VoiceAnalysis,BigData

Foundingmember

OurCustomers

Data Services Energy Government Financial Life Sciences Manufacturing / Technology

BrownFieldversusGreenField

•  Complex•  Poli)cal•  Highrisk

•  Cleanslate•  Perfectformicro-services

TheEnterpriseisbrownfield!

7

UseCase•  Scien)ficdataanaly)csforgenomesequencediscovery

-  Massivedataanalysisàlargecluster-  Life-scienceanalysisstandardizedonGridEngine

•  Clusterisasharedresource-  Manyusers-  Advancedpolicies,e.g.,fair-sharing,back-filling,anddependableresourcecontrols

-  Advancedjobtypes,e.g.,arrayjobs-  Detailedaccoun)ngandbillingforresourceconsump)on

Challenges•  Sandboxing–maintainmanyproduc)onenvironmentsforalong)me•  Atminimalornoperformanceimpact:

-  Fromrunningapplica)onsinacontainer-  Fromnetworkandsharedfilesystemaccessfromwithinacontainer-  Fromstar)ngthesamecontainersoverandoveronnodes-  Avoidtoreloadimages

ChallengesandSolu)on

8

Solu5on•  CRGNexXlowworkflowmanagement•  IntegratedwithUnivaGridEngine•  IntegratedwithDocker•  MakeUnivaGridEngineDocker-aware

-  EnableDockerjobs-  Containerimagecache-awarescheduling

•  4%increaseofDockerapplica)onrun)mevsna)verun)mewithcachedimages

•  12.5%increasewithcontainerbootstraping,i.e.,downloadingfromimagerepository-  Imagecacheawareschedulinghassolidbenefitonu)liza)on

andthroughput•  CostisconsideredlowversusbenefitbyCRG

-  Usecaserequirementsreallycan'tbesa)sfiedwithoutcontainers

Results

9

EnterpriseContainerAdop)on–Challenges

•  Storage–  Containersaregenerallyephemeral...andsoisthestorage–  Enterprisesolu)onsnotalwaysperformantinthecloud(i.e.,NFS)

•  Networking–  Earlydaysforcontainernetworking.S)llevolving.

•  Security–  Isola)onnotatrobustasvirtualmachines–  Poten)alforthreatsinpubliccontainerrepositories–  SecurityissuesnotaswellunderstoodasothermorematureareasofIT...

nothardenedthroughwidespreaduse–  ContainerBreakouts,DoSacacks,KernelExploits,CompromisedSecrets

•  BusinessProcesses–  Complexbusinessprocess,policies,mul)pleusergroups...allcreate

complexityandchallengeswhenarchitec)ngandorchestra)ngcontainerizedworkloads

–  Runningcontainerwithnon-containerworkloadscanbetricky

Naviga)ngtheEnterprise

•  Abridgefromthebrownfields–  Orchestra)on

•  UnivaGridEngine–ContainerEdi)on,ApacheMesos–  Storage

•  NFS,Gluster,Ceph,Flocker–  Networking

•  Virtualswitches•  Op)onsincludeFlannel,Weave,DockerNetworking,Calico,OpenVswitch,OpenContrail

–  Configura)onManagement•  Ansible,Puppet,Salt,•  Orsimplifywithanimmutableopera)ngsystems(CoreOS,Atomic...)

Kubernetes

•  Aclusteringsolu)onformanagingcontainersinamul)-nodeenvironment– Masters,Nodes,Pods,Replica)onControllers,Services,Labels,Namespaces

•  Providescontainergrouping,healing,scaling,andloadbalancing

•  NowpartofCNCFwithmul)plecontributorsincludingRedHat,Google,Meteor,Huawei,CoreOS...

DemoOverview

1.Willdemonstratescalingofworkloadonanon-premisecluster2.Whentheclusterissaturated,newnodeswillbeautoma)callyprovisionedinthecloud.3.ThosenodeswillthenbeaddedtotheKubernetesclusterandwillstarttotakeonworkload.4.Asoverallworkloadisthenreduced,thecloudmachinewillberemovedfromtheK8sclusteranddeprovisioned.

Demo

LocalNodes GCPNodes

NavopsInstaller

KubernetesMaster

KubernetesNode

VPNServerInstance

KubernetsNodeInstance

NavopsContainers VPN

Kubernetes

Navops

HTTPClients WebApp

Pods

NavopsRuleEngine

LocalKubernetes

Node

GCEKubernetes

Nodes

Monitor Scale

ThedemoconsistsofahybridinstallwithalocalKubernetesMaster,alocalKubernetesNode,and0toNGCEKubernetesNodes.

Navopsmonitorstheloadofthewebapplica)onandscalestheKubernetesclustertomeetdemandbyburs)ngintoGCE.

DemoRecap

Solving the Set Up Problem – In Minutes

“Implementing Docker and Kubernetes is

complicated.

It’s even more complicated for enterprises that have diverse workloads, various app stacks, heterogeneous infrastructures, and limited resources.”

InfoWorld May 2015

17

18

Cloud On Premises Servers / VMs

After

“The next step is large scale orchestration and scale.” 451 Research

Before

Containers and other workloads need resources

•  Run containers at scale

•  Blend containers with other workloads

•  Maximize resources / use of cloud

Navops orchestration solution

Thankyou!

CameronBrunnerbrunner@univa.com