EDGE_2016-SCL-2484-A Software Defined Scalable and Flexible

#ibmedge© 2016 IBM Corporation

A Software Defined Scalable and Flexible Container Management SolutionBin Xu

Yong Feng

Li Cao

#ibmedge2

Please Note• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without

notice and at IBM’s sole discretion.

• Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

• The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

• The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

• Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

#ibmedge3

Agenda

• Container is Leading the IT Transformation• Container Introduction• Container Ecosystem

• IBM Spectrum Computing Offerings for Container Management• IBM Spectrum Conductor Family Capabilities• Reference Cases

• Demo

#ibmedge4

Container is Leading the IT Transformation• Cloud is widely adopted by most of the companies.

• 95% companies are using cloud, and the adoption of private/hybrid cloud is steadily increasing.

• The interesting in Container(Docker) is expanding rapidly.

• Container: A group of processes run in isolation• Uses cgroups to limit resource consumption• Uses namespaces for isolation

• Container History• Started on Unix with chroot in 1979. • Variants in the evolution:

– FreeBSD jails in 1998, – SWsoft's (later Parallels') Virtuozzo in 2001 (and the associated

OpenVZ in 2005), – Solaris Zone in 2005, – workload partitions (WPARs) on AIX and HP-UX Containers in 2007.– LXC (2008): Modern Linux container, OS container

• Docker(2013): – Build, Ship, run. Focusing on application container

Source: RightScale

#ibmedge5

Containers and VMs Solve Different ProblemsContainers is more light and better performance:

• Portability: VM (Gigabytes) vs. Container (Megabytes), VMs are constraint to Hypervisor and hardware-emulation

• Performance: Containers can boot and restart in seconds, compared to minutes for virtual machines. And no extra overhead of a hypervisor and guest OS makes containers consume less CPU and memory.

• Management cost: Each VM requires a full functional operating system, and then extra management for them.

Great advantage to use containers in:

• DevOps

• Batch computing

• Lightweight PaaS

• Microservices

Host Server

Host Operating System

Hypervisor

OS

Bins/Libs

App1

OS

Bins/Libs

App2

OS

Bins/Libs

App3

Host Server

Host Operating System

Container Engine

Bins/Libs

App1

Bins/Libs

App2

Bins/Libs

App3

Virtual Machines Containers

#ibmedge6

Next-Generation Application Development Requires Efficient Container Management

Microservices Architecture:• Decomposed into small pieces

• Loosely coupled

• Easier to scale development

• Improved fault isolation

• Each service can be developed and deployed independently

• Eliminates any long-term commitment to a technology stack

#ibmedge7

Container Management Platform is Emerging• Container Management Platforms are also known as container managers,

container orchestration engines (COEs), container orchestration platforms and containers-as-a-service (CaaS) platforms.

• It helps enterprises address the challenges of deploying containers. These platforms let administrators monitor, manage, secure, and scale containers similar to how they treat existing servers and virtual machines.

Bare Metal / Virtualization EnvironmentOn-prem / off-prem

Auto-scaling / Upgrade

Image Management(Appstore, registry)

ServiceRegistration & Discovery

Security / Compliance

Routing & Load Balancing

Monitoring & Logging

Scheduling

HA / Backup & Restore

#ibmedge

Layer 1Infrastructure

Layer 2 Operating System

Layer 3 ContainerEngine

Layer 4 Orchestration

Layer 5 CMP

Tool Chain

8

Container Ecosystem

© 2016 IBM Corporation #ibmedge

IBM Spectrum Computing Offerings for Container Management

#ibmedge10

Software Defined Infrastructure Portfolio

IBM Spectrum Cluster Foundation(OpenStack/xCAT)

IBM Spectrum ScaleInfrastructureManagement

Data & StorageManagement

IBM Spectrum Computing

IBM Spectrum Symphony

IBM Spectrum

LSF

High Performance Analytics

(Low Latency Parallel)

High Performance Computing

(Batch, Serial, MPI, Workflow)

Next GenerationApplication

(Spark, Container)

ResourceManagement

WorkloadEngines

Linux on zx86PowerDiskTape

On-premise, On-cloud, Hybrid Infrastructure(heterogeneous distributed computing and storage environment)

Flash VMdocker

Support and Services

IBM Spectrum Conductor

#ibmedge11

Key Components for Container Management

• IBM Spectrum Conductor• IBM Spectrum Conductor is an on premises platform for managing containerized applications

that is based on the container orchestrator Kubernetes. Through the implementation of a robust resource manager, Apache Mesos, IBM Spectrum Conductor is able to manage resources for hybrid environments that contain both containerized and non-containerized applications as well as mixed hardware architecture.

• IBM Spectrum Scale• IBM Spectrum Scale is software-defined storage for high performance, large scale workloads

on-premises or in the cloud. Built on the award winning IBM General Parallel File System (GPFS), this scale-out storage solution provides file, object and integrated data analytics for all kinds of workloads.

• IBM Spectrum Cluster Foundation• IBM Spectrum Cluster Foundation automates the creation of multiple scale-out environments

on a shared infrastructure used by multiple teams. The software creates an agile environment for running both high-performance computing (HPC), analytics and containers.

#ibmedge

IBM Container Service

IBM Container Management Platform Options

On-prem, customer managed

Bluemix Local


On-prem, IBM

managedOff-prem, IBM managed

Bluemix Public or Dedicated

Ecosystem: Docker Data Center, Mesosphere DC/OS, Redhat OpenShift

#ibmedge

Community Value IBM Value-add Customer Value

Docker Hub Registry holds a repository of 75000+ Docker images

Lots of application integrated with Mesos

Kubernetes enable micro-service architecture

• Client unique registry available on premises• Security readiness guidance via the Vulnerability Advisor• Build-in applications of popular open source projects and

IBM enterprise products in App Store

Access to the images and application you require to deploy containers that meet your business needs and strategy

Open-source, standardized, lightweight, self sufficient container technology

• Balance workload between on-prem and off-prem• Deployment choice with openPOWER and x86_64

Flexibility to choose on-prem and off-prem or mix for your business

Build, ship, and run standardized containers

• Integrated monitoring & logging• Elasticity to grow storage & container needs• Integrated CI/CD flow• Life-cycle management of containers and data volumes

Docker ease of use combined with enterprise-level integrity and confidence

Embrace Open Source

13

#ibmedge

Simplify Container Management with Integrated Controls

Container Orchestration

Resource Management

Security

On-Premise

App Catalog

Unified UI

Development

InfrastructureDelivery

14

• Integrated

• Open

• Secure

• Highly Available

• Customer-managed


Resource scheduling

Auto-scaling

Power & X86

#ibmedge15

Full Lifecycle Management for Container

Hypervisor


Unified Web-based Interface

Resource Management

Pattern based cluster template

Private Registry

Monitoring and Reporting

Cloud Native Spark

Pattern based cluster templateComputing

Framework

x86

15

• Container orchestration• Resource management• Application life-cycle

management/schedule/deployment• Scaling, Rolling upgrade• Service Registry/Discovery

• Container infrastructure• Load Balance• Multi-host Networking • Distribute storage management• Image/Software repository management• Configure management• Logs/Meters/Alert• User/Account management (Multiple tenancy & RBAC)

• Ops management• Installation/upgrade• Health check

#ibmedge16

Spark and Big Data Analytics

Cognitive Containerized Applications

Session Scheduler

Workflow

Installer(Deploy,

Reconfigure, HA, Scale,

Rolling update)Mesos Agent

K8s executor

pod pod pod container container

containercontainer

Mesos Master

Kubernetes

GUI

Service Discovery

AuthenticationAuthorization

Distributed Key-value

Store

Image registry

Monitor

IBM Spectrum Conductor Overview Architecture

16

HPC

App Store

Persistent Volume

Service Load Balance

Trouble-shooting

Network Topology

#ibmedge17

IBM Spectrum Conductor Architecture

km ctrlmanager

km apiserver

km scheduler

Agent Node

Master Node

Boot Node

Ansible based installer and ops

manager

LDAP Server

Mesosmaster MySQL

haproxy etcd GUI cfc-auth Keystone

VIP

Mesos Agent

km proxy

Agent Node

Pod Pod Pod

Docker

Agent Node

VIP VIP

17

cfc-routerImage-mgrappstorenetwork mgr

Heapter

km agent

Kube-DNS

Flanneld

Mesos Agent

km proxy

Pod Pod Pod

Docker

km agent

Flanneld

Mesos Agent

km proxy

Pod Pod Pod

Docker

km agent

Flanneld

master mgr

#ibmedge | 18

IBM Spectrum Conductor GUI

Create a Container Cloud for developers supporting DevOps practices and cloud-native apps. Pre-built app catalog for fast deployment of OSS tools. Reduce developer friction, creating faster time to results

1

Improve Developer Productivity

Fine grain, dynamic allocation of resources maximizes efficiency of Spark instances sharing a common resource pool.

2Increase Resource Utilization

Proven architecture at extreme scale, with enterprise class workload management, monitoring, reporting, and security capabilities.

3Reduce Administration Costs

18

#ibmedge

Response to the Lesson Learned

19

• Gaps in current state of Kubernetes• Single installer and operation manager to manage disparate parts together • Unified GUI as management console for various services• Single API end-point• Single user service end-point and load balancer• Central authentication and authorization manager• Resource manager to support various workload manager and fine-grain

resource sharing

• Enterprise Requirement• HA topology• System services live rolling upgrade and live reconfiguration• Heterogonous environment (Power, X86, GPU and so on) • Trouble-shooting, audit, alarm and event• Multiple site

#ibmedge20

IBM Spectrum Conductor Community Edition

• Community Edition v0.1 (Tech Preview) is releasing soon!

• Free to use as you wish.

• We are looking for feedback for our roadmap.

• Register on our community page: http://ibm.biz/ConductorForContainers

http://ibm.biz/ConductorForContainers

http://ibm.biz/ConductorForContainers

#ibmedge

Release Timeline

21

3Q16 4Q16 1Q17

Community Edition 0.1

Initial versionKubernetes&Mesos API/CLIGUIInstaller and HAAuthentication LDAPApp storePrivate image registry

Sample Apps in App StoreNginx, SockShop

Build-in Network Flannel

Build-in Persistent StorageNFS, Glusterfs

HW Support Power, x86


SparkSpark Session Scheduler

Open SourceRebase on fr8r

KubernetesBuild-in ingress service

HW Support Z


CI/CD flowJenkins Jenkins git/cvsJenkins private docker registry

BatchKubernetes batch

Build-in App in App StoreMarathon, Tomcat, React & Django, blockchain, tensorflow, R-studio, OpenCV,

Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

#ibmedge

More challenges when setting up the environment on-premise…

22

• Cluster Ops (Day-0, …)• Repeatable automatic deployment with great customized flexibility • Build on bare metal directly• Complex hardware environment: (x86/Power, GPU, etc)• Continually monitoring, maintenance and upgrade

• Hybrid services from Developer• Container service is major, but VM service (even bare-metal) is still

required for special case.

• Multiple clusters• Multiple regions• Isolated clusters to address strong security• Resource balancing

#ibmedge

OverviewPowerful lifecycle management for scale-out cluster environments

Key Capabilities• Simplified management with cluster template designer• Scales from single clusters to complex multi-team environments • Robust, scalable alerting and reporting• Automated infrastructure management – one-click cluster

deployment• Enhanced Cluster management: cluster maintenance, health

check and cluster upgrade (Bulk and Rolling)

Benefits• Faster time to cluster readiness• Unified interface for management and monitoring• Increased administrator productivity• Single infrastructure supporting multiple business needs

Software Define the Infrastructure with Templates

Hypervisor

IBM Spectrum Cluster Foundation

Unified Web-based Interface

Infrastructure Management

Pattern based cluster template

Cluster template

designer

Monitoring and Reporting

IBM Spectrum LSF IBM Spectrum Conductor

Pattern based cluster templateWorkload based

cluster template

x86

23

#ibmedge

Infrastructure Resource Aggregation

xCATBare-Metal

Generic Public Cloudadapter

Cluster Deployment

PaaS BD & A

Infrastructure discovery

Image Registry(OS, VM, container)

SW Repository

Logging/Metric

Alert & Policy

Authentication

Load Balance

DevOps

IBM Spectrum Conductor with Infrastructure Management

Discover bare metals and quickly deploy the environment on-demand (bare metal, virtualization or hybrid)

1

Simplify IT operations

Fine grain, dynamic allocation of resources maximizes efficiency of servers (Bare metals and VMs) sharing a common resource pool.

2Increase Resource Utilization

Proven architecture at extreme scale, with enterprise class infrastructure management, monitoring, reporting, and security capabilities.

3Reduce Administration Costs

24

#ibmedge

Deliver an Agile Containerization Infrastructure in Enterprise

25

ServerStorage Network ServerServer

IBM Spectrum Cluster Foundation

OrchestrationCluster Template

xCAT

IBM Spectrum Conductor Cluster#1

Operating System

Bare Metal

IBM Spectrum Scale

Docker Engine

Elastic scale in/out

Design

DeployMonitor & Health

upgrade

scale

Automation

OpenStack

Virtualizations Pools

Bare Metal

Operating System

IBM Spectrum Scale

OpenStack (KVM)

VM VM VM VM

Provisioning

IBM Spectrum Conductor Cluster#2

POD

Benefits• Auto deploy customized OpenStack to offer the virtualization pools• Auto deploy two container management environments on both bare metals and virtual machines.• Easy to adjust the size of container management environments to balance the workload， and full • Building up Multi-tenant management based on LDAP

POD POD POD

POD POD


Reference Cases

#ibmedge

Requirements from a China Telecom Customer

| 27

• Resource Management (Mesos)• Support Non-Container Service (Marathon)

• Support Container Service (Kubernetes)

• Support Big Data Applications (Myriad vs. YARN on Kubernetes)

• Support Service Categories and Routers• Unified DNS server for Service registration and discovery

• Support Software Defined Network

• Based on HA-proxy

27

• Support Application Level Resource Sharing and preemption• Provide resource sharing and preemption:

• When resources for Big Data are free, other application can borrow idle resource;

• and intelligent recovered when needed

• Support Infrastructure Level dynamic balancing• Applications could be scheduled to bare-metal and VMs according to application type

• Bare-metal pool node and virtualization pool could be balanced

#ibmedge

Container Management Solution Architecture for Telecom

| 28

MesosAgent

MesosAgent

Mesos Agent

Marathon

Resource Scheduler

…...

k8sm-scheduler

k8sm-apiserver

k8sm-controll-manager

Mesos Agent

Mesos Agent

Software Defined Network (Flannel, DNS) Software Defined Storage (Ceph, IBM Spectrum Scale)

Heterogeneous Nodes

Container Service Big Data Others

Kubernetes

Myriad

Application Catalog

IBM Value-adds

Customer Solution

IBM SolutionSchedule

rPortal

Auth

APP Store

28


Demo


Q & A

#ibmedge

Notices and Disclaimers

31

Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

IBM products are manufactured from new parts or new and used parts. In some cases, a product may not be new and may have been previously installed. Regardless, our warranty terms apply.”

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

#ibmedge

Notices and Disclaimers Con’t.

32

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

http://www.ibm.com/legal/copytrade.shtml


Thank You

Documents

EDGE_2016-SCL-2484-A Software Defined Scalable and Flexible