Building a Kubernetes Powered Central Modules Repository

CNCF Webinar, August 22nd 2019

@jfrog @rimusz

Who Am I?

• Rimantas Mocevicius

• Senior Developer

• Big supporter of Open Source Software, Co-founder of Helm, author of CoreOS Essentials book

• Love coffee, green tea, halva and cheesecake

@jfrog @rimusz

We Golang and Kubernetes

@jfrog @rimusz

GO Central Modules Repository

https://gocenter.io

@jfrog @rimusz

GO, GO, Kubernetes

Lessons learned and best practices

@jfrog @rimusz

Running on Google Kubernetes Engine

• Mature Kubernetes offering

• Choice of managed services

• Choice of troubleshooting tools like Stackdriver, BigQuery and etc

• Regional GKE clusters for Staging and Production

@jfrog @rimusz

Number of clusters currently running 5

This is what our cluster looks like

https://console.cloud.google.com/kubernetes/deployment/us-east1/caddyshack-stg1/caddyshack-stg/caddyshack-webui?project=cpd-caddyshack-stg&authuser=1&organizationId=201145417873&tab=overview&deployment_overview_active_revisions_tablesize=50&duration=PT6H&pod_summary_list_tablesize=20&service_list_datatablesize=20

https://console.cloud.google.com/kubernetes/deployment/us-east1/caddyshack-stg1/caddyshack-stg/caddyshack-server?project=cpd-caddyshack-stg&authuser=1&organizationId=201145417873&tab=overview&deployment_overview_active_revisions_tablesize=50&duration=PT6H&pod_summary_list_tablesize=20&service_list_datatablesize=20

https://console.cloud.google.com/kubernetes/deployment/us-east1/caddyshack-stg1/caddyshack-stg/caddyshack-discovery?project=cpd-caddyshack-stg&authuser=1&organizationId=201145417873&tab=overview&deployment_overview_active_revisions_tablesize=50&duration=PT6H&pod_summary_list_tablesize=20&service_list_datatablesize=20

https://console.cloud.google.com/kubernetes/deployment/us-east1/caddyshack-stg1/caddyshack-stg/caddyshack-notifier?project=cpd-caddyshack-stg&authuser=1&organizationId=201145417873&tab=overview&deployment_overview_active_revisions_tablesize=50&duration=PT6H&pod_summary_list_tablesize=20&service_list_datatablesize=20

https://console.cloud.google.com/kubernetes/deployment/us-east1/caddyshack-stg1/caddyshack-stg/caddyshack-processor?project=cpd-caddyshack-stg&authuser=1&organizationId=201145417873&tab=overview&deployment_overview_active_revisions_tablesize=50&duration=PT6H&pod_summary_list_tablesize=20&service_list_datatablesize=20

https://console.cloud.google.com/kubernetes/deployment/us-east1/caddyshack-stg1/caddyshack-stg/caddyshack-validator?project=cpd-caddyshack-stg&authuser=1&organizationId=201145417873&tab=overview&deployment_overview_active_revisions_tablesize=50&duration=PT6H&pod_summary_list_tablesize=20&service_list_datatablesize=20

@jfrog @rimusz

We’re big on those pipelines and promotions concepts

Feature branch --->

@jfrog @rimusz

Implementing with best practices

• Tillerless Helm v2

• Secrets

• Namespaces

• Persistence

• Probes

• Resource Limits

• Node/pode affinity

• Node pools

@jfrog @rimusz

Deployments using Helm

● Tillerless Helm v2 is used● Helm Charts are versioned● Separate values files per

environment● Even secrets are Helm Charts● Namespaces

Deployments using Helm

Scaling on many levels

Run more microservicesCompute

Scaling based on queue depthMessaging

What goes up must come

down… usually

Up and Down

Provision nodes when neededNode

@jfrog @rimusz

stackdriver-dashboard

@jfrog @rimusz

Scaling on many levels

15 pods ---> 3 Kubernetes nodes

25 pods ---> 5 Kubernetes nodes

@jfrog @rimusz

Node-pools

@jfrog @rimusz

Node-pools

@jfrog @rimusz

Node-pools

Keeping a close eye on cluster and container health

● Stackdriver● Prometheus● Grafana

@jfrog @rimusz

grafana

Chaos engineering and load testing

We keep in touch with our nodes

Kube-Slack Jenkins

@jfrog @rimusz

Questions?

@jfrog @rimusz

Thank you!

@jfrog @rimusz

Building a Kubernetes Powered Central Modules Repository · 2019-12-20 · Building a Kubernetes...

Documents

Kubernetes Immersion

Azure Architecture Center (aka.ms/mspnp) Application... · Kubernetes Marathon Swarm Service Fabric Request Repository Validation Cluster state store Etcd Consul Zookeeper Service

Kubernetes Operators

Kubernetes for the Ansible users - OpenInfraDays 2019 · Why? What? 2. Ansiblefor Kubernetes oImplementingKubernetes oMigratingexistingservicesto Kubernetes oManagingthe lifecycleof

Kubernetes Ingress Services by VMware NSX® Advanced Load ... · Kubernetes/OpenShift Cluster Integration using Avi Kubernetes Operator Avi Kubernetes Ingress Services can be used

Kubernetes Security

Kubernetes HA @ AppDirect - Montreal Kubernetes Meetup

Docker & Kubernetes

VMware vSphere 7 with Kubernetes...VMware Cloud Foundation Services Powered by innovations in vSphere 7, VMware Cloud Foundation Services is a new, integrated Kubernetes and RESTful

Kubernetes Workshop

Infrastructure Design for Kubernetes · Kubernetes 1.9 Kubernetes 1.10 Kubernetes 1.11 Kubernetes 1.12 December 2017 March 2018 June 2018 September 2018 Kubernetes 1.13 December 2018

Kubernetes Administration Certification€¦ · 2. Basics of Kubernetes • Define Kubernetes • Meaning of Kubernetes • Adoption • Project Governance • Labs 3. Installation

Kubernetes: What's it do? - Red Hateparis/kubernetes/kube.pdf · Kubernetes: What's it do? Presenter Eric Paris Red Hat. 2 Eric Paris Agenda What is Kubernetes Kubernetes Primitives

Kubernetes Concepts

Docker, Kubernetes, CCP...•UI –Kubernetes, API •Security (policies, encryption) •Add / remove Kubernetes nodes •Lifecycle management (OS updates, Kubernetes upgrades) •Monitoring

Kubernetes Patterns Kubernetes Patterns

Kubernetes Introduction

Troubleshooting Kubernetes

Kubernetes Networking

Kubernetes Boston — Custom High Availability of Kubernetes