Microservices & Containerscc.ee.ntu.edu.tw/~rbwu/rapid_content/course/IoT_Intro/... · 2019. 9. 29. · National Taiwan University D epartment of E lectrical E ngineering *Source:

National Taiwan UniversityDepartment of Electrical Engineering

Alex Lai, Ph.D.

Adjunct Professor

[email protected]

Cyber III:

IoT, Microservices, &

Software Containers


Microservices & Software Containers

Introduction of Docker and Kubernetes

Hands-on

Agenda

2


Virtualization is one of the most well-known

mechanisms to implement Cloud Computing.

Recap: A Note on Virtualization

3


Formally OS-level virtualization (“Containerization”)■ = The OS kernel allows the existence of multiple isolated user-space

contexts.

Good at■ Fast deployment of microservices. (快進快出)

Compared to VM Lower overhead

Faster deployment

No heterogeneity

Lower protection

相輔相成而非取代

Recap: A Note on Container (vs. VM)

*Source: Adapted from Wikipedia and InwinStack presentation. 4


5*Source: Adapted from Microservices: From Design to Deployment by Chris Richardson with Floyd Smith, an NGINX eBook, 2016. .

“Control” “Coordination”

Microservices – A variant of the service-oriented architecture (SOA)

organizing an application as a collection of fine-grain, loosely coupled,

autonomous primitives, with clear scope.

Compared to the old-school “monolithic” architecture:

IoT Realization via Microservices*


A Software Container Premier

6

Containerization, also known as OS-level virtualization, refers to an

OS feature in which the kernel allows the existence of multiple isolated

user-space instances (software containers).

Why Containerization: Good at implementing Microservices.

Example of Container Technologies:

LXC (Linux Containers)

*Source: Adapted from Wikipedia


Linux Containers (LXC) An operating system–level virtualization method for running multiple

isolated Linux systems (containers) on a single control host.

7


Linux Kernel Features used by LXC

Namespaces

(mnt, pid, net, ipc, uts/hostname, user ids)

cgroups

(cpu, memory, disk, i/o - resource management)

AppArmor, SELinux (security/access control)

seccomp (computation isolation)

chroot

(file system isolation)

8


More about Linux Containers

9*Source: Adapted from linuxcontainers.org.


Docker, started as an open source

project in 2013, is an open-

source project that automates the

deployment

of applications inside software

containers

Docker uses the resource isolation

features of the Linux kernel such

as cgroups and kernel namespaces,

and a union-capable file

system such as OverlayFS to allow

independent "containers" to run

within a single Linux instance.

Container Example (2): Docker

10

https://en.wikipedia.org/wiki/Open-sourcehttps://en.wikipedia.org/wiki/Application_softwarehttps://en.wikipedia.org/wiki/Software_containerhttps://en.wikipedia.org/wiki/Linux_kernelhttps://en.wikipedia.org/wiki/Cgroupshttps://en.wikipedia.org/wiki/Linux_namespaceshttps://en.wikipedia.org/wiki/Union_mounthttps://en.wikipedia.org/wiki/OverlayFS


Docker Container

Docker Image

Docker Registry

Dockerfile

A Simplified Docker Overview

11


Docker is built upon

cgroup and namespacing capabilities of the

Linux kernel

Go programming language (written in Go)

Docker Image Specification (for container image

management)

LibcontainerSpecification (namespaces,

filesystem, resources, security, etc)

12


Virtual Machines Vs Docker Containers

13


Docker Architecture

14

Docker Architecture


Docker Image Structure

15


Limitations with standalone Docker

16


Introduction to Kubernetes (k8s)

17


Kubernetes (known as k8s) is an open-source system for automating deployment, scaling, and management of containerized applications.

It groups containers that make up an application into logical units for easy

management and discovery. Kubernetes builds upon 15 years of

experience of running production workloads at Google, combined with

best-of-breed ideas and practices from the community.

With Kubernetes, users are able to create and manage microservices:

■ Deploy applications quickly and predictably.

■ Scale applications on the fly.

■ Roll out new features seamlessly.

■ Limit hardware usage to required resources only.

■ Supports GCE, CoreOS, Azure

■ Can manage Docker containers as a default implementation

Kubernetes – What it is

18


⧫ Automated arrangement, coordination, and management of containers

Why Kubernetes:Container Orchestration

Kubernetes Worker 3

Running Environment 1

is the best place for you

Create a copy to make a HA

Kubernetes Worker 2

Kubernetes Worker 1

Kubernetes Master

19


What can k8s do?

Kubernetes provides a container-centric management

environment. It orchestrates computing, networking, and

storage infrastructure on behalf of user workloads.

This provides much of the simplicity of Platform as a

Service (PaaS) with the flexibility of Infrastructure as a

Service (IaaS), and enables portability across

infrastructure providers.

20


High-level k8s Architecture

21


High-level k8s Overview

22

22

Master

Node 1 Node 2 Node 3

Service

Service A Service B Service C Service D

MasterMaster

Pod A-1

Container

Container

Container

Pod A-1

Container

Container

Container

Pod A-1

Container

Container

Container

Pod A-2

Container

Container

Pod A-2

Container

Container

Pod B-1

Container

Container

Container

Pod B-1

Container

Container

Container

Pod B-2

Container

Pod C-1

Container

Container

Container

Pod C-1

Container

Container

Container

Pod D-1

Container

Container

Container

Pod D-2

Container

Container

Container

Container

Pod D-2

Container

Container

Container

Container


High Level Components of k8sMaster

API Server

Controller Manager

Etcd

Service and Labels

Node (VM/PM)

Kubelet

Kube-proxy

Pod

Container

23


Master Componentskube-apiserver

Component on the master that exposes the Kubernetes API. It is the front-

end for the Kubernetes control plane.

etcd

Consistent and highly-available key value store used as Kubernetes’

backing store for all cluster data.

kube-scheduler

Component on the master that watches newly created pods that have no

node assigned, and selects a node for them to run on.

kube-controller-manager

Node Controller, Replication Controller, Endpoints Controller, Service

Account & Token Controllerscontroller-manager.

https://kubernetes.io/docs/concepts/overview/components/

24


Minion Node (“Node”) Overview

25


kubelet

An agent that runs on each node in the cluster. It makes sure that

containers are running in a pod.

kube-proxy

Enables the Kubernetes service abstraction by maintaining network rules on

the host and performing connection forwarding.

Container Runtime

The software that is responsible for running containers. Kubernetes

supports several runtimes: Docker, rkt, runc and any OCI runtime-spec

implementation.

*Source: https://kubernetes.io/docs/concepts/overview/components/ 26

Minion Node (“Node”) Components


Cluster NetworkingNeed to achieve:

Highly-coupled container-to-container communications

Pod-to-Pod communications

Pod-to-Service communications

External-to-Service communications

Additionally required ingredients

Flannel/Calico, OpenVSwitch, NSX-T…

*Source: https://kubernetes.io/docs/concepts/cluster-administration/networking/ 27


LoadBalancer - Haproxy Server(Docker on a VM)

node1 node2 node3

Master 1, 2

Link up by

Networking

PodService: Dashboard, SkyDNS, Monitoring, EFK-Stack

Architecture – Revisited & Revised

28


Namespace, Services, Pod, Expose

Namespace BNamespace A

Service X

Select “X”

e.g.,

http://service-X

Service Y

Select “Y”

e.g.,

http://service-Y

Service Z

Select “Z”

e.g.,

http://service-Z

Expose: NodePort, LoadBalancer, Ingress

29

Kubernetes Runtime


Link up by Flannel Networking

30*Source: Adapted from https://github.com/coreos/flannel


Link up by Calico Networking

31

*Source: Adapted from Project Calico Web page (https://www.projectcalico.org/)


Pods ControllersReplicationController

ReplicaSet

Deployments

DaemonSet

Jobs - Run to Completion

*Source: https://kubernetes.io/docs/concepts/workloads/controllers/replicaset/#how-to-use-a-replicaset 32


Working with kubectl

33


Scripting the Pod Controller ActionsAn Example of kubectl YAML file

Deployment

Replica Set

Pod Spec

Container

Spec

Container

Spec

How are updates handled?

Rolling/recreation

How many Pods should run?

Node selector Service labels

Docker image Environment

variables Storage Claims

34


Exposing Services (1) - NodePort

35


Exposing Services (2) - LoadBalancer

36


Exposing Services (3) - Ingress

37


Additional Services in k8s PodsDashboard

SkyDNS

Monitoring

InfluxDB

Heapster

Grafana

EFK-Stack(log)

Elasticsearch

Fluent

Kibana

38


k8s Environment in NTUEE

NTUEE Kubernetes Development Environment for IoT

Master : 140.112.18.229

• SSH Login(ID: student1 / Password: student1)

Node1 : 140.112.18.230

Node2 : 140.112.18.231

A “Hello World” Example

嘗試建立與刪除一K8s service, 此service包含由Apache Pod與其他所組成

39


DashBoard

https://140.112.18.229:6443/api/v1/namespaces/kube-

system/services/https:kubernetes-dashboard:/proxy/#!/login

Token Login

• https://hackmd.io/s/H1yzV-sjz

Login

40

40

https://140.112.18.229:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/loginhttps://hackmd.io/s/H1yzV-sjz


首次登入畫面

41

• 修改URL中的namespacedefault改成各組名稱(例：team1)

• 提示訊息：權限不足

41


42

正確進入所屬Namespace


建立一個Service,一個Pod包含apache, grafana 兩個container

YAML Example：https://hackmd.io/s/ByU1rZioz

①

②建立方式• 上傳已編輯的.yaml檔案• 透過dashboard UI

43

新建一個Service

https://hackmd.io/s/ByU1rZioz


使用空白字符（不能使用Tab）分層，同層元素左側對齊

用“-”來表示一些序列的項（Sequence）

用“：”來表示一對項目（Map）里的欄目（Key）和其相應的值（Value）

單行註解由“#”字號開始，可以出現在行中任何位置

字符串一般不使用引號，但必要的時候可以用引號框住

YAML也有用來描述好幾行相同結構的數據縮寫語法，數組用“[]”包括起來，hash用“{}”來包括

44

A Note on YAML Fundamentals


• Apache container 內部port 80, 外部 32391• Grafana container 內部 port 3000, 外部 32260

45

新建完成


Master, Node1, Node2 IP:外部PORT

Master IP Node1 IP Node2 IP

46

Apache 測試


①

②

47

刪除-Pod


①

②

48

刪除-Service


Kubeadm (官方推出)

Minikube (官方推出)

Ansible

Rancher(Web UI) https://rancher.com/kubernetes/

Kubernetes快速部署工具

49

49


Hands-on

50


Before we start: Preparation

1. Set up & Test-drive Docker Environment

2. Set up & Test-drive k8s by employing Rancher admin console

3. Link a Physical Connected Thing (your RPi) to the k8s service

51

Objectives


Download Lab Container VM Images w/ Ubuntu 16.04 64-bitEither VMWare Workstation & VirtualBox version

Use Bridge mode (instead of NAT) in Networking configuration

52

Before Starting…

Bridge Mode in VMWare Workstation

Bridge Mode in VirtualBox


Configuring the Remaining Essential SettingsSSH (w/ Root Login permission)

Set Proper Firewalling Rules

53

Before Starting (cont.)

#sudosudo -i

#updateapt-get update

#SSH Serviceapt-get -y install openssh-server

#如出現「E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it」，執行:rm -rf /var/lib/dpkg/lock

#install vimapt-get -y install vim

#install curlapt-get -y install curl

#修改root密碼passwd root

#讓Root登入vi /etc/ssh/sshd_configPermitRootLogin prohibit-password 改 PermitRootLogin Yes

#restart sshd servicesystemctl restart sshd

#停用防火牆 BAD Practice but just works for testing nowsystemctl stop ufwsudo ufw disable


安裝指令: docker version

官網安裝說明: https://docs.docker.com/install/

GitHub : https://github.com/rancher/install-docker

54

Set up Docker Service in your VM

• apt-get install -y apt-transport-https ca-certificates curl software-properties-common

• curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

• add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"

• apt-get update

• apt-cache madison docker-ce

• apt-get install -f -y docker-ce=17.03.2~ce-0~ubuntu-xenial

#安裝完後確認及服務是否正常# 版本確認:• docker version

服務確認• systemctl status docker


https://hub.docker.com/

55

The Docker Hub



docker pull [image name]

docker images

此處以 Ubuntu Image 為例

56

Pull a Docker Image from Repository


docker run [OPTIONS] IMAGE[:TAG|@DIGEST]

[COMMAND] [ARG...]

docker exec -it [name or container id] bash

apt-get update

apt-get install [service name]

docker stop [name or container id]

docker start [name or container id]

docker restart [name or container id]

57

Run, Operate & Configure Containers


docker run -d -p 5000:5000 -v

/mnt/data/registry:/var/lib/registry --name

registry registry

docker run -d -p 8080:8080 --name registry-web

--link registry -e

REGISTRY_URL=http://10.1.1.196:5000/v2

hyper/docker-registry-web

Commit 改過的Imagedocker commit [OPTIONS] CONTAINER

[REPOSITORY[:TAG]]

58

Set up & Commit into a

Private Docker Repository


A ‘Hello World’ Example of Docker

Get one base Docker image


List Docker images

Run “hello world” and List Container

59


‘Hello World’ (cont.): Detached mode

Run hello world in detached mode(-d)

Exec to the container

stop/ start /restart the container

#docker stop

60


Rancher 2.0 已經包裝好為 container 形式使用putty ssh 登入VM，並輸入下列指令docker run -d --restart=unless-stopped -p

80:80 -p 443:443 rancher/rancher

61

k8s: Install Rancher Admin Console


開啟瀏覽器，輸入VM的IP，即會出現登入畫面，第一次登入請先設定admin密碼:

62

k8s: Start Rancher


設定Server URL 並儲存

63

k8s: Configure Rancher Server URL


Add Cluster->點選「CUSTOM」，並「Cluster Nmae」中輸入自定名稱，完成後點選「NEXT」

64

k8s: Add a Cluster from Rancher


勾選「etcd, Control Plane, Worker」將docker指令複製，完後點選「Done」

65

k8s: Configure Nodes from Rancher


回到putty畫面，貼上剛剛複製的docker指令即開始安裝, 可用Web browser看到各services安裝過程:

66

k8s: Configure Nodes (cont.)


點選「Default」Namespace，再點選「Import YAML」

67

k8s: Testing your 1st Cluster


點選「Default」Namespace，再點選「Import YAML」What is the Service described in the YAML file?

68

k8s: Import YAML Test File


The YAML File to be used hereapiVersion: v1kind: Servicemetadata:

namespace: defaultname: nginx-svclabels:

service: nginx-servicespec:

selector:service: nginx-pod

type: NodePortports:- name: http

port: 80 ---apiVersion: extensions/v1beta1kind: Deploymentmetadata:

name: nginx-podnamespace: default

spec:replicas: 1template:

metadata:labels:

service: nginx-podspec:

containers:- name: nginx-pod

image: nginximagePullPolicy: IfNotPresentports:- containerPort: 80

69

k8s: Import YAML Test File


Service

Pod

70

k8s: Your 1st Service & Pod


71

Session Takeaways

Microservice (微服務) 是一種軟體架構，以協調多數小型功能區塊為基礎，利用模組化的方式組合出複雜的大型應用程式。相較於注重控制流程的單體式(Monolithic) 應用，Microservice輕量化與快進快出的特性，適用於現代小型但量多之服務，包括IoT在內。

Software Container (軟體容器)是藉由OS層虛擬化(OS Level Virtualization) 防止各執行程式之資料與命名空間互相干擾，因此特別適合實作微服務。

由當代標準 Kubernetes (k8s) 與 Docker 實務演練中，瞭解如何使用Container實作Micro Cloud Service for IoT.


Docker 官網: https://docs.docker.com/

互動操作學習: https://www.katacoda.com/courses/docker

Docker-從入門到實踐:https://philipzheng.gitbooks.io/docker_practice/content/introduction/what.html

K8S部署: https://kairen.github.io/

官網: https://kubernetes.io/

互動操作學習: https://www.katacoda.com/courses/kubernetes

72

參考資料與推薦閱讀

https:https:https:https:https:


Q&A

73

Documents

Microservices & Containerscc.ee.ntu.edu.tw/~rbwu/rapid_content/course/IoT_Intro/... · 2019. 9. 29. · National Taiwan University D epartment of E lectrical E ngineering *Source: