View
619
Download
0
Category
Preview:
Citation preview
Containerizing MongoDB with Kubernetes
Dan Worth (fuboTV)
Brian McNamara (CloudyOps)
Who are we?Dan Worth
● Software Engineer
● Currently at fuboTV
● @djworth on the Internet
Who are we?Brian McNamara
● Ops Engineer
● @mcnamarabrian / @cloudyops
● Enjoys learning new things and quoting corny 80s comedies whenever possible
What are we going to talk about?● In the beginning of fuboTV
● Fundamentals of Kubernetes (Dan)
● Fundamentals of MongoDB high availability (Brian)
● Challenges of running stateful services on Kubernetes (Brian)
Let’s Start with a Story...
fubo.tv Business Overview● Sports first virtual multichannel video programming distributor (MVPD)
● Partnership with 21st Century Fox and Sky
● Create the best live events platform
● Subscription based
● Build communities around sports (teams, matches, players)
fubo.tv Started Taking Over Streaming World● Needed scalable offering
● Bursty demand patterns around soccer / futbol matches
● Lots of in-house development experience
● Not a lot of in-house operational experience
● Didn’t want to maintain hardware
fubo.tv v1● Node.js hosted with well-known PaaS provider
● MongoDB hosted with a separate provider
● Focus was on introducing application features, iterating quickly
Things Changed in Philly
Enter Kubernetes● Presentation by Kelsey Hightower
● Sysadmin who can code
● 2015 PhillyETE: Managing
Containers at Scale with CoreOS
and Kubernetes
● Container cluster manager
● Don’t sweat the scheduling of
containers in your cluster
● Live demo of rolling application
updates
Minds = Blown
How Can We Do That?● Google’s infrastructure for everyone else
● Loved demo
● Saw the possibilities of Kubernetes
● But….still didn’t want to maintain hardware
Enter Google Container Engine
● No need to run a Kubernetes cluster in-house
● Google provided service
● Fully managed
Fundamentals of Kubernetes● Open Source container cluster manager by Google
● Run Anywhere (GKE, GCE, AWS, Bare metal)
● Self-healing when using the right primitives
● Service discovery and load balancing
● Secret and configuration management
● Key high level domain objects
○ Pods
○ Replication Controllers
○ Services
Fundamentals of Kubernetes (Pods)● Pods
○ Unit of Scheduling
○ One or more containers
○ Define environment
○ Pods get their own IP addresses
spec:containers: - name: mongo image: mongo:3.2 ports: - containerPort: 27017 resources: limits: cpu: 4
Fundamentals of Kubernetes (RC)● Replication Controller
● All the goodness of the Pod
● Additional benefit of defining count of pods
Replication Controller ExampleapiVersion: v1kind: ReplicationControllermetadata: name: mongo1
spec: replicas: 1 selector: name: mongo1 template: metadata: labels: name: mongo1 spec: containers: - name: mongo1 image: mongo:3.2
volumeMounts: - name: mongo1-data mountPath: /data/db resources: Limits: cpu: 4 memory: 4Gi ports: - name: "mongo" containerPort: 27017 protocol: TCP command: - ...
Fundamentals of Kubernetes (Services)● Provides stable endpoint to pods / replication controllers
● Uses metadata like ports and selectors to identify how to map endpoint to pod
Service ExampleapiVersion: v1kind: Servicemetadata: name: mongo1-service labels: name: mongo1-servicespec: ports: - port: 27017 targetPort: 27017 protocol: TCP selector: name: mongo1 type: LoadBalancer
Fundamentals of KubernetesService
Replication Controller
Pod Pod Pod
Fundamentals of MongoDB High Availability● Possible to scale reads and writes
○ Scaling reads: use replica sets
○ Scaling writes: use shards
● Clients can do things to take advantage of availability primitives
● We’ll focus on scaling reads using replica sets
Fundamentals of MongoDB High Availability (cont)
MongoDB Replica Set
Fundamentals of MongoDB High Availability (cont)
Heartbeat among replica set members
Fundamentals of MongoDB High Availability (cont)
Automated election of Primary in the event of failure
Fundamentals of MongoDB High Availability (cont)> rs.config(){ "_id" : "replica_set_name", "version" : 105978, "protocolVersion" : NumberLong(1), "members" : [ { "_id" : 0,
"host" : "ip_or_hostname:port_number", "arbiterOnly" : false, "buildIndexes" : true, "hidden" : false, "priority" : 1, "tags" : {}, "slaveDelay" : NumberLong(0), "votes" : 1 }, { … }}
Challenges of Running Stateful Services on Kubernetes● Kubernetes is amazing at running and rescheduling containers with Pods,
Replication Controllers, and Services
● Stateless services are easiest to manage but...
● Sometimes we need things to maintain state
Challenges of Running Stateful Services MongoDB on Kubernetes● Kubernetes Replication Controller
○ Ensure requisite number of Pods are scheduled
○ Don’t guarantee consistent hostname or IP address
● MongoDB replica set configuration uses well defined endpoints
○ Remember that rs.conf() output?
○ Updating replica set configuration by hand feels dirty and you’re a bad person if you want to do
that.
● MongoDB data should persist
○ If not, when a new replica set member comes up there will be a full sync
○ Kubernetes manages the scheduling but who needs the full sync?
Built-in Primitives to the Rescue● Kubernetes
○ Replication Controller + Service
○ Label selector allows for consistent association between pod and service hostname / IP
○ Persistent volume can be defined
■ Allows GCE volumes to move with Pod
■ Result: No need to do expensive resync of data
● MongoDB
○ Take advantage of service endpoint when defining replica set
■ Service IP or DNS
Kubernetes - Code SamplesReplication Controller Code
Service Code
Parting Thoughts● MongoDB has good resilience in the face of failure but be sure to test different
failure scenarios.
● Docker is great to work with, but make sure your development workflows, tools
and harnesses are adapted to build and run apps with it.
● Kubernetes is still relatively young, but maturing quickly. You need to carefully
evaluate whether you want to roll your own platform with it, or instead rely on a
hosted service like Google Container Engine
Questions?… and thanks!
Dan Worth (@djworth)
Brian McNamara (@mcnamarabrian / @cloudyops)
ReferencesGoogle Container Engine (GKE)
Kubernetes Docs
MongoDB Read Preference
MongoDB Replication Introduction
MongoDB Replica Set Tag Sets
MongoDB Leaf in the Wild Post
CloudyOps Deploying a 3-node MongoDB Replica Set with Kubernetes
2015 PhillyETE: Managing Containers at Scale with CoreOS and Kubernetes
Recommended