Upload
benjamin-gandon
View
203
Download
0
Embed Size (px)
Citation preview
BOSH&
DiegoBenjamin Gandon
@BenjGa
2016-04-04
@BenjGa
Founder & CEO @ Gstack.io
Formerly: CTO @ Alvarum.com
Benjamin Gandon
BOSH
At:
We do:
And:
BOSHDeploys Distributed Systems
Argh! It’s again those Americans! Don’t worry, Herr General, we will find out soon!
What is BOSH
BOSH is a flexible and resilient technology that ………………………
complete ecosystems of distributed software.
1. Package2. Deploy3. Monitor4. Manage
(start/stop, scale, resurrect, update, …)
5. Upgrade
Instance in a BOSH Deployment
Container (or VM)
Filesystem + BOSH Agent
NginxHAproxy Pcre
HAproxy Nginx
Stemcell
Packages
Jobs
Instance Stemcell
+ Jobs
+ Packages
= Instance
Equation
/var/vcap/jobs/*
/var/vcap/packages/*
Availability Zone #3
AZ2AZ1
What we do with those “instances”
HAproxy
Stemcell
Pkg1 Pkg2
Nginx
Stemcell
Pkg3 Pkg2
HAproxy
Stemcell
Pkg1 Pkg2
Nginx
Stemcell
Pkg3 Pkg2
Nginx
Stemcell
Pkg3 Pkg2
● Garden containers (in a VitrualBox VM)● Docker (experimental)● AWS● OpenStack● CloudStack● Azure● vSphere
BOSH secret? → Use abstract Cloud Provider Interfaces (CPIs)
Deploy where ?
DiegoFlexible Container Engine
From
Try at home! Because…
● BOSH
● Cloud Foundry
● Diego
Open Source
Why “diego”?
● Since 2019, intuition for stateless “not-yet-containers” → the Droplets
● Their engine: the Droplet Execution Agent
→ DEA
● When rewritten in Go: DEA-GO
→ Diego!
Diego architecture
(logs)
(controller)
(http routes)
Source: https://github.com/cloudfoundry-incubator/diego-design-notes
Not just about containers
● Long Running Processes
→ LRPs
● Tasks
→ Servers
→ Cron jobs
Two ways to use Diego
● Multi-tenant Diego
→ as Cloud Foundry central container engine
● Single-tenant Diego
→ as Lattice, a single-tenant Diego cluster
Dude, go to http://lattice.cf
Lattice is definitely cool
● Containerized workloads○ Long running, or temporary tasks○ Dynamically scaled○ Dynamically balanced across cells (but no live re-balancing)
● Cluster scheduler● HTTP load balancing● Log aggregation● Health management
Some code!
Diego Auctions Scoring
// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L15-L24type CellState struct {
RootFSProviders RootFSProvidersAvailableResources ResourcesTotalResources ResourcesLRPs []LRPTasks []TaskStartingContainerCount intZone stringEvacuating boolVolumeDrivers []string
}// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L74-L79func (c CellState) ComputeScore(res *Resource, startingContainerWeight float64) float64 {
remainingResources := c.AvailableResources.Copy()remainingResources.Subtract(res)startingContainerScore := float64(c.StartingContainerCount) * startingContainerWeightreturn remainingResources.ComputeScore(&c.TotalResources) + startingContainerScore
}// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L90-L93type Resources struct {
MemoryMB int32DiskMB int32Containers int
}// See: github.com/cloudfoundry-incubator/rep/…/resources.go#L110-L115func (r *Resources) ComputeScore(total *Resources) float64 {
fractionUsedMemory := 1.0 - float64(r.MemoryMB)/float64(total.MemoryMB)fractionUsedDisk := 1.0 - float64(r.DiskMB)/float64(total.DiskMB)fractionUsedContainers := 1.0 - float64(r.Containers)/float64(total.Containers)return (fractionUsedMemory + fractionUsedDisk + fractionUsedContainers) / 3.0
}
// See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {var winnerCell *CellwinnerScore := 1e20
// Lists all zones, and count how many times the LRP is already in each of themzones := accumulateZonesByInstances(s.zones, lrpAuction.ProcessGuid)
// Retain only zones with compatible filesystem (e.g. Linux or Windows)filteredZones := filterZones(zones, lrpAuction)
if len(filteredZones) == 0 {return nil, auctiontypes.ErrorCellMismatch
}
sortedZones := sortZonesByInstances(filteredZones)
// …
}
Diego Auctions Evaluation (1/3)
// See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {// …for zoneIndex, lrpByZone := range sortedZones {
for _, cell := range lrpByZone.zone {score, err := cell.ScoreForLRP(&lrpAuction.LRP, s.startingContainerWeight)if err != nil {
continue}
// Look for the Diego cell with minimal overall utilizationif score < winnerScore {
winnerScore = scorewinnerCell = cell
}}
// if (not last zone) && (this zone has the same # of instances as the next sorted zone)// acts as a tie breakerif zoneIndex+1 < len(sortedZones) &&
lrpByZone.instances == sortedZones[zoneIndex+1].instances {continue
}
if winnerCell != nil {break
}}// …
}
Diego Auctions Evaluation (2/3)
// See: github.com/cloudfoundry-incubator/auction/…/auctionrunner/scheduler.go#L206-L258
func (s *Scheduler) scheduleLRPAuction(lrpAuction *auctiontypes.LRPAuction) (*auctiontypes.LRPAuction, error) {// …
if winnerCell == nil {return nil, rep.ErrorInsufficientResources
}
err := winnerCell.ReserveLRP(&lrpAuction.LRP)if err != nil {
s.logger.Error("lrp-failed-to-reserve-cell", err, lager.Data{"cell-guid": winnerCell.Guid, "lrp-guid": lrpAuction.Identifier()})return nil, err
}
winningAuction := lrpAuction.Copy()
// Return a copy of the auction containing the winning cellwinningAuction.Winner = winnerCell.Guidreturn &winningAuction, nil
}
Diego Auctions Evaluation (3/3)
→ Simple as that!
Thank you!
www.gstack.io (not yet in Google)
www.bosh.iowww.lattice.cfwww.cloudfoundry.org
Questions?↘ ↙
→ @BenjGa ←↗ ↖