View
236
Download
1
Category
Preview:
Citation preview
Package once / Run anywhereBig Data and HPC workloads
HPC Knowledge Portal 2015
Tryggvi LarussonCTO & Co-founder GreenQloud
Common pain points today
Regarding any software deployment on Cloud, HPC Clusters or local
environment
• Every environment is slightly different
• OS, Networking, Libraries, Compilers etc.
Docker Basics
• Container runtime on Linux
• “Standard” container image format
• Centralized repository of images:
• https://hub.docker.com/
• (like github except for runtimes)
CoreOS Basics
• Very minimal Linux distribution
• Containing basically the kernel and a few unix utilities
• Main tools:
• systemd for running services
• etcd for clustering hosts together
• fleet for distribution across cluster
• docker for running applications
“Everything CoreOS is building has been inspired by the way Google runs its
data center infrastructure”
- Alex Polvi, CEO CoreOS
Source: http://www.datacenterknowledge.com/archives/2014/07/16/etcd-secret-sauce-googles-kubernetes-pivotals-cloud-foundry/
Docker Advantages
• Simple Packaging
• Thousands of ready made images
• Low overhead (compared to VMs)
• Portability (Build once run anywhere)
• Composable
• (One step closer to the Lego block dream)
Docker Build: Dockerfile
FROM ubuntu:14.04
MAINTAINER Tryggvi Larusson <tryggvi@greenqloud.com>
RUN apt-get update
RUN apt-get -y install mysql-server
EXPOSE 3306
CMD ["/usr/bin/mysqld_safe"]
docker build -t=“mysql” .
Development of Computing
1960s 1970s 1980s 1990s 2000s 2010s 2020s
Mainframes Minicomputers PCs PCs PCs + ASP Cloud Distributed -Federated Cloud
Centralized
Closed Source
Centralized
Closed Source
Decentralized
Closed Source
Decentralized
Closed Source
Hybrid
Closed + Open Source
Centralized
Closed + Open Source
Decentralized
Open Source
Present
Development of the cloud
PHASE 1Virtualization
PHASE 2Cloud
PHASE 3Hybrid Cloud
PHASE 4Cloud Federation
Local Datacenters+ Virtualization Private Cloud
Public Cloud
Private Cloud
Public Cloud
Present
Hybrid Cloud
Source: http://www.infoq.com/news/2014/08/vm-containers-performance
STREAM benchmark 16 SocketsLinpack
STREAM benchmark 1 Socket
Convergence
Local Datacenters+ Virtualization
Private Cloud
Public Cloud
HPC Clusters
On-Premise/Big-Data Clusters
Next-Gen Distributed Cloud
Run anywhere
KVM VM VMWare VM
Debian Linux
Baremetal Machine
CoreOS Linux
myapp1
myapp2
myapp1
myapp2
myapp1
myapp2
Cloud Instance
myapp1
myapp2
CentOS Linux
Implications for HPC developers/admins
• Freedom of choice
• Have more options in selecting where your workloads actually run - easier to experiment
• Speed of deployment
• Standard containers make it very easy to automate
• Collaboration
• Easily share your HPC application to others and get feedback
• Build on an HPC app/container somebody already built and improve it
Recommended