Embed Size (px)
Citation preview
Agenda
• Docker
• Kubernetes
• Kubernetes on AWS
• Demo
Docker
Automate the deployment of any application as a lightweight, portable, self-sufficient container
that will run virtually anywherehttps://www.docker.com/what-docker#Why-Should-I-Care-(For-Developers)
Docker For Developers
Docker automates the repetitive tasks of setting up and configuring development environments so that developers
can focus on what matters: building great software.
Docker for Ops
• Develop and deploy bug fixes and new features without roadblocks and scale applications in real time.
Diff VM• Containers have similar resource isolation and
allocation benefits as virtual machines but a different architectural approach allows them to be much more portable and efficient.
Docker Engine
• It uses a shared kernel and independent user mode sessions to provide isolated run-times for multiple applications on the same host.
Docker Container
• It is a packaged application running in an isolated mode on a Docker host.
Docker Client
• It provides a command line interface to control the Docker engine.
Docker Image
• It is an application template ready to be deployed multiple times.
Docker image
http://blog.octo.com/en/docker-registry-first-steps/
Docker Architecture
http://southworks.com/blog/tag/docker/
Docker Compose
• Docker Compose allows you to define your multi-container application with all of its dependencies in a single file, then spin your application up in a single command
Docker Orchestration
https://www.linkedin.com/pulse/docker-redefining-cloud-computing-landscape-akash-goswami
Kubernetes• Open Source Automation Framework for
deploying, managing, and scaling applications.
Why Kubernetes
• Kubernetes provides a common API and self-healing framework which automatically handles machine failures and application deployments, logging, and monitoring.
Kubernetes Concept• Clusters - set of compute, storage, network
resource
• Pods - colocated group of application containers that share volumes and a networking stack
• Replication Controllers - ensure a specific number of pods, manage pods, status updates
• Services - cluster wide service discovery
Architecture
Components
etcd flannel
kubernetes master
kubernetes node
kubernetes node
kubectl• Pod
• Replication controller
• Deployment
• Service
• Volumes
https://kubernetes.io/docs/user-guide/deployments/
Kubernetes on AWS
https://kubernetes.io/docs/getting-started-guides/aws/
Kubernetes on AWSexport AWS_DEFAULT_PROFILE=default export KUBERNETES_PROVIDER=aws; wget -q -O - https://get.k8s.io | bash export AWS_ACCESS_KEY_ID=XXXX export AWS_SECRET_ACCESS_KEY=XXX export KUBERNETES_PROVIDER=aws export KUBE_AWS_ZONE=us-east-1c export NUM_NODES=3 export MASTER_SIZE=t2.medium export NODE_SIZE=m3.large export AWS_S3_REGION=us-east-1 export AWS_S3_BUCKET=XXX export INSTANCE_PREFIX=k8s export NODE_ROOT_DISK_SIZE=10 export MASTER_ROOT_DISK_SIZE=10 export AWS_SSH_KEY=/Users/dummy/.ssh/metatron
kube-up.sh
Amazon Container Service
• Amazon ECS makes it easy to deploy, manage, and scale Docker containers running applications, services, and batch processes.
Amazon Container Servie
http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html
Amazon Container Servie
Screenshot
Screenshot
Episode
• Pod restart
• Failed Containers that are restarted by the kubelet are restarted with an exponential back-off delay (10s, 20s, 40s …) capped at five minutes, and is reset after ten minutes of successful execution
PersistentVolume• awsElasticBlockStore
kind: PersistentVolume apiVersion: v1 metadata: name: metatron-pv labels: type: amazonEBS spec: capacity: storage: 5Gi accessModes: - ReadWriteMany awsElasticBlockStore: volumeID: vol-XXXXXXXXXXXXXXXX fsType: ext4
PersistentVolumeClaimkind: PersistentVolumeClaim apiVersion: v1 metadata: name: metatron-pvc labels: type: amazonEBS spec: accessModes: - ReadWriteOnce resources: requests: storage: 5Gi
Deployment spec: volumes: - name: ebs-volume awsElasticBlockStore: volumeID: vol-XXXXXXXXXXXXXXXX fsType: ext4 containers: - name: jupyter image: jerryjung/jupyter-notebook-gfs:latest volumeMounts: - mountPath: "/home/jupyter/work" name: ebs-volume
Kubernetes meets Hadoop• Persistent Volume Issue by permission
• Shared Nothing Architecture - hadoop
• DaemonSet or StatefulSet
for i in 0 1; do kubectl exec yarn-nm-$i -- sh -c 'hostname'; done yarn-nm-0 yarn-nm-1
DEMO
Q&A
Thank you
