Cloud Foundry Open Tour - London

cloud foundry update

mark lucovskyvp of engineering, cloud foundry

agenda• cloud foundry anniversary recap

• linux of the cloud

2© 2012 VMware, Inc. All rights reserved

cloudfoundry.org

© 2012 VMware, Inc. All rights reserved 3

the new cloudfoundry.org

reviewers +2/-2

test verification score ✓ ✗ git push triggers test execution

Gerrit Code Reviewreviews.cloudfoundry.org

Jenkins CIci.cloudfoundry.org

githubgithub.com/cloudfoundry

+1/-1 committers

on +2 and ✓ change pushed to github

cloudfoundry.com

production updated ~2x/week


80% of work is below the water line• Developer Visible Features

• services• frameworks, runtimes• tools and ide’s• micro cloud foundry

• Contributor Visible Features• re-visiting architectural issues• extensive hardening• algorithm tuning• scalability enhancements• operational enhancements


app lifecycle management: at launch• awesome conceptual architecture,

logical flows

• simplistic design provided a solid conceptual foundation

• scalability, robustness, operational efficiency, and extensibility was inadequate

• deconstruction/refactoring• enable multiple independently

scalable components

• eliminate “boot storms” at scale

• tune the hot data paths

• apply learning from the beta• across the board and to all major

system components

cc - db

cloud controller

NFS

healthmanager

fetch droplets

start/stop instances

get expected state

actual state

resources, droplets, packages

dea


app lifecycle management: +1yr

stagingjobs

redis

cc - db

cloud controller

stager

package cache

blobstore

uaa - dbdea

healthmanager

staging logs

uaa - AuthN

more new code than old code

New!

New!

New!

New!

New!

New!

New!

v2

v2

v2*


app execution (DEA): launch and +1yr• Major internal rewrite

• Validating isolation model in Jenkins

• Live in CF.com ~30d

apps

dea.rb

warden container*

app

private fs

private network

nat and firewall

dea.rbcontainer api

* also running in our Jenkins workers


router: at launch and +1yr

nginx

router.rb

nats

dea

<app>.cf.com

node:port

proxied request

proxied request

http request

nginx(+lua extension)

upstreamlocator svc

nats

dea

<app>.cf.com

node:port

proxied request

lookupnode:port

http request


© 2012 VMware, Inc. All rights reserved

production cloud foundry clusters• 500 – 5,000 VMs

• 40+ unique node types

• 75+ unique software packages

• 75+ unique environments

• 2x/week cf.com updates

• 24x7x365 non-stop operation

• No-downtime deployments

• Reliable, robust, repeatable deployments, updates, capacity adjustments

• Small teams manage many instances production, staging, stress, qa,

dev

Google style problem Google style solution

cloudfoundry.com

10


cloud foundry BOSH• cloud foundry BOSH is an open source tool-chain for

release engineering, deployment, and lifecycle management of large scale distributed services• prescriptive way of creating releases, managing systems and

services• it is not a collection of shell scripts, not a pile of Perl

• built to deploy/manage production-class, large scale clusters• production-class cloud foundry clusters: 500+ VMs, 40+ jobs, 75+

packages • multi-node, multi-tier, complex clusters: e.g., our Gerrit/Jenkins

Cluster

• built for devops usage and scale by a crack team of veterans• a project, not a product: command line interface, YAML, etc. • continuous improvement, iterative development, rough edgesgithub.com/cloudfoundry/bosh

11


BOSH: under the hood

active jobs

diskIaaS CPI

workers

bosh cli

redis nats

stemcellagent

db

blobs

director healthmon

“BOSH is deployed by BOSH”

cloudfoundry.com

BOSH User

12


IaaS neutral by design

CPI: code completefunctional status: “work in progress”

vSphere: battle tested implementation, thousands of deployments

vCloud Director: “work in progress”, 2H 2012

contribute: github.com/cloudfoundry/bosh

Cloud Provider Interface (CPI)

Cloud Foundry BOSH

13


linux of the cloud

14

© 2012 VMware, Inc. All rights reserved 15© 2012 VMware, Inc. All rights reserved

linux of the cloudmost widely used OS generation in computing

the primary os of an era

value proposition of a “primary os”• developers can target the broadest possible market

with highly leveraged portable code

• applications have a place to reliably run

• customers can use the os knowing they have the broadest choice of applications

• there is enough of a market to support a healthy eco-system where money is changing hands

• OSS, portable OS’s magnify all of the positive effects listed above, commercial friendly licensing is a further magnifier (Apache2/MIT)



linux of the cloud

17

recent generations in computing


micro’s pc’s

CP/M

workgroup, servers

MS/DOS WINDOWS NT8080, z-80 x86 family

sun

vax, sun, convex, sequent

super-mini’s

workstationsmini’s pdp-11 datacenter server x86 vm

Version 6,7 SunOS, 4.x BSD Linux

VMS, Solaris, 4.x BSD

OS/400, Novell, NT

sun


linux du jour summary• CP/M == linux of the micro-computer era

• MS/DOS == linux of the pc era

• Windows NT == linux of the 32/64bit pc era

• Unix v6/v7, 4.2BSD == linux of the mini-computer era

• SunOS == linux of the workstation era

• Linux == linux of the datacenter server era

??? ?? == linux of the cloud



linux of the cloud

20

what the os provides• ability to create and launch applications

• with an appropriate amount of isolation• with an appropriate level of resource management

• with a high degree of application portability• app will run on any instance of the os• apps are largely isolated from hardware

• allow application to interact with environment• devices• other computers and services

• modern and relevant to current problems and scale• foundation capable of supporting an ecosystem


traditional os layering


physical hardware

hardware abstraction layer

core operating system

shell libraries & APIs user apps

system call interface

OS largely defined by its syscall layer


OSsyscall

File, Device, I/O System:- open(2), readv(2), etc.- mov ah,021h; int 21h; write

- NtOpenFile

Networking:- socket(2), select(2), bind(2)- mov ax,5E00h; int 21h; hostname

- NtDeviceIoControlFile

System Info, Misc Mgmt:- swapon(2), gettimeofday (2)- mov ah,01Bh; int 21h; drvinfo

- NtEnumerateBootEntries

Process & Memory Mgmt:- fork(2), exec(2), sbrk(2),- getpagesize(2), sigpause(2)

- mov ah,04Bh; int 21h; exec

- mov ah,031h; int 21h; tsr

- NtCreateThread, NtTerminate- NtCreateSection, NtLockVM

Hardware Abstraction Layer

CP/M


- shell

- core os - hal

- hardware

8080, z-80

bios

bdos

pcpconsole app to launch apps and built in commands (pip, etc.)

file, file system, device accessability to launch applications

raw device access apisdisk/block access

bdos function0 - 112

CP/M• operating system was portable across a range of

micro-computers

• bios abstracted away uniqueness for the OS layer

• large market of 8080/z-80 machines had an OS

• apps like Wordstar, Multiplan, dBase II, made real $$

• bdos/bios architecture allowed decent hardware innovation• bank-switched multi-user systems• better serial ports, devices, etc.


MS-DOS


- shell

- core os - hal

- hardware

8086+

ibm pc-bios

pcdos

command.comconsole app to launch apps and built in commands (dir, type, etc.)

file, file system, device access (prn, aux, com)ability to launch applications, mild memory mgmt

raw device accesspc io, disk, graphics

apps coded to hardware as needed

int 21h

MS-DOS – PC-DOS• started diverse and portable (MS-DOS), morphed to

IBM compatible and PC-DOS

• pc rom-bios was extremely complete and accessible, grew well with each advancement in hw

• virtually all x86 (including 8088) ran PC-DOS with a cloned rom-bios

• massive market for ISV software, PC-DOS was the DOS powering Windows 95…

• IBM PC compatibility was all that mattered • huge market of machines to run PC-DOS/Windows16• in exchange for highly constrained pc innovation


WINDOWS NT


- shell

- core os - hal

- hardware

i386+

hal

NT Executive/Kernel

command.exe32-bit shell, based on win32 base api

file, file system, device access, networkingsmp, process and threading, rich memory mgmt

raw device access like busses, interrupt controllers, timers, etc.

hardware access by kernel and hal only

ntdll/zwapi,ntapi

WINDOWS NT• 1988 – smp, pre-emptive kernel, highly portable

• high degree of portability across all modern laptops, desktops, x86 based systems. portable across other cpu architectures (but all of those proved niche…)

• NT HAL allowed a wide degree of freedom for hardware guys (at high and low end)

• huge market for apps from Office to games and everything in between

“Windows runs best in a VM. One that you can throwaway when windows gets sick and

infected with registry cancer”29© 2012 VMware, Inc. All rights reserved

Unix, Linux


- shell

- core os - hal

- hardware

68000 – x86++

arch modules

kernel

shadvanced shell with great set of commands

file, file system, device access, networkingsmp, process and threading, rich memory mgmt

raw device access like busses, interrupt controllers, timers, etc. (asm/*, arch/*)

hardware access by kernel and arch only

section(2)

Unix/Linux• Version 6 – 1975, Version 7 – 1979: pdp-11 focused,

variations for lsi-11.

• V7 – first highly portable unix: (68010, Z8000, 8086, Vax, Eclipse, Sun, etc.) Networking, demand paged mm, non-pre-emptive kernel.

• 4.2BSD – 1983 networking, demand paging, perf

• hal layer implemented by convention, as needed, in custom arch/* and asm/* source code. solid portable core so design is extremely well exercised.

• Linux – 1991 == the endgame for Unix. open source, unix-like core, portable, licensing compatible with large webfarms



cloud era

32

os layering review


physical hardware

hardware abstraction layer

core operating system

shell libraries & APIs user apps


the role of IaaS• programmable, on-demand, custom configured hw

• old-school BIOS API: create vga device• mov ah,0; mov al,12h; int 10h

• cloud-era BIOS API: create quad-core, 16GB server• Bosh::Cloud.create_stemcell(path, props)

IaaS == the BIOS of the cloud


physical hardware

HAL

cloudstack

vSphere

OpenStack

Hyper-V

Azure

eucalyptus

aws

google

IaaS du jour

multi-blade, multi-core, large memory x86, unlimited net, storage, dc


aws == AMI of the cloud?

35

cloud os layering



shell

core os


physical hardware

HAL IaaS: aws, vSphere, OpenStack, eucalyptus, etc.

IaaS Equalizer Interface - CPI

core os:- system call layer implemented as REST endpoint- ability to create and launch apps- portable across all IaaS hardware abstractions- OSS with liberal licensing (MIT/Apache2)

shell:- modern, scriptable, cli, thin layer above REST API- webui, control panel and IDE integrations

cloud foundry os layering


eucalyptus

OpenStack

cloudstack

vSphere

Hyper-V

Azureaws

google


vcap core:- REST control api, uniform syscall API across all cloud foundry instances- highly portable application model, URL addressable apps- service layer: uniform access to files, databases, kv stores, etc. uniform- IaaS/HAL abstraction orchestrated by BOSH- OSS with liberal licensing Apache2

vmc shell:- scriptable CLI- thin UI layer above REST API- STS plugin for integration into Eclipse, Cloud9 IDE, etc.

cloud foundry syscall


OS syscall

service management:- next-gen “I/O” system- unified discovery/binding- REST api - vmc services - vmc create-service - vmc delete-service - vmc bind-service - vmc unbind-service

REST control API:app managementservice managementuser/system management

app management:- http url addressable apps- simple horizontal scale out - REST api - vmc push; vmc update - vmc stop; vmc start - vmc mem; vmc map - vmc instances - vmc {files, logs, crashlogs}

Hardware Abstraction Layer

cloud foundry is a modern cloud os• cloud-centric application launch API

• horizontally scalable instances, full memory control• http addressable and background apps

• highly portable, end-to-end OS• BOSH – tool-chain for release engineering, deployment,

and lifecycle management of large scale distributed systems. CPI based infrastructure abstraction

• VCAP – Cloud Foundry core (the green box). OSS, infrastructure neutral

• next generation service-based “I/O” system API• service-independent: discovery, binding, lifecycle API• connects apps to services: ~open(‘/dev/mongoDB’)

• OSS with liberal Apache2/MIT licensing


generations in computing


micro’s pc’s

CP/M

workgroup, servers

MS/DOS WINDOWS NT8080, z-80 x86 family

OS/400, Novell, NT

sun


super-mini’s

workstationsmini’s pdp-11 datacenter server x86 vm

Version 6,7 SunOS, 4.x BSD Linux

VMS, Solaris, 4.x BSD

sun


cloud vSphere, aws, OpenStack, eucalyptus, Azure, Hyper-V, cloudstack, google

Cloud Foundry ???


www.cloudfoundry.com/jobs

41

Technology

Cloud Foundry Open Tour - London