Upload
others
View
18
Download
0
Embed Size (px)
Citation preview
Running Cloud Foundry at Swisscom
CF Summit Europe 2015, November 3th
Boban GlisovicCloud Engineer, PaaS Core
Diego ZamboniCloud Architect, Health Management
CF Summit Berlin 2015 (c) Swisscom 2015
About Swisscom
Everythingalways on
Internet-based
Globalcompetition
• Telecommunication, IT Services and more within Switzerland
• Fixed network: By the end of 2015, Swisscom will supply 2.3 million homes and businesses with ultra-fast broadband.
• Mobile network: By 2016, Swisscomwill have extended 4G/LTEcoverage to 99% of theSwiss population.
• > 20’000 Employees
Swisscom’sunified approach to cloud infrastructure & services
IaaS SaaS PaaS
IT Architect EndUser Application Developer
Enterprise Cloud Service Cloud Application Cloud
same
infra-structure
identical
tools
same
services
Building a 360°CloudUsed by everybody: For consuming services, building services, building platforms
Open Standards à no Lock-in
Scalable – based on requirements, time and budget
Simple: a standardized platform
CF Summit Berlin 2015 (c) Swisscom 2015
CF Summit Berlin 2015 (c) Swisscom 2015
ProductApplication Cloud
Virtual PrivatePublic
Product
Cloud Infrastructure
Cloud Foundry (PaaS)
Services (3rd Party)
Self Service Portal
Support
Business Model
Support
Professional Services Optional
Pay-per-use
Contract base
(Minimum Consumption)
Legacy-Integrations (IdP etc.)
Shared
Shared
Dedicated Network (VXLAN)
Shared
Dedicated
Standard, 24/7
High Availability (96.0-99.9% )
• Base fee• At least 3 months• Setup fee included
Services (Database etc.) Shared and Dedicated Dedicated
Optional
Community & Std
Secure Datacenter Interconnection (VPN/LAN-I) Optional
• No base fee• At least 1 day
Launch: 2nd of Oct 15
Cloud Foundry at Swisscom
IaaS
BOSH
Messaging (NATS)
DEA
Healthmanager
Cloud Controller
Dynamic Router
Service Broker
Logging & Metrics
UAA/Login
User Provided Services
Apps
Buildpacks
Services3rd party integration
Business Integration
Stateful container-based services
SDSOpenstack SDN
External servicesIAM federation ELB/WAF
PortalBilling Extensions
Lifecycle?
Operations?
CF Summit Berlin 2015 (c) Swisscom 2015
Lifecycle
IaaS
BOSH
Messaging (NATS)
DEA
Healthmanager
Cloud Controller
Dynamic Router
Service Broker
Logging & Metrics
UAA/Login
User Provided Services
Apps
Buildpacks
Services3rd party integration
Business Integration
Stateful container-based services
SDSOpenstack SDN
External servicesIAM federation ELB/WAF
PortalBilling Extensions
Lifecycle?
Operations?
CF Summit Berlin 2015 (c) Swisscom 2015
Lifecycle
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Lifecycle?
Operations?
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
Ia a S
BOS HM e ssa g ing (NAT S )
DE A
He a lthm a na g e rClo ud Co ntro llerDyna m ic R o ute r
S e rvice Bro ke r
Lo g g ing & M e trics
UAA/Lo g in
Use r P ro vid e d S e rvice s
AppsBuild pa cks
S e rvice s3 r d pa rty inte g ra tio n
Busine ss Inte g ra tio n
S ta te ful co nta iner-ba se d se rvice s
S DSOpe nsta ck S DN
E x te rna l se rvice sIAM fe d e ra tio n E LB/WAF
P o rta lBilling E x te nsio ns
CF Summit Berlin 2015 (c) Swisscom 2015
Lifecycle
main challenges
> automated provisioning of PaaS stacks
> keep things “up-to-date”
> integrations
CF Summit Berlin 2015 (c) Swisscom 2015
Lifecycle
main challenges
> automated provisioning of PaaS stacks
> keep things “up-to-date”
> integrations
> # cat cf-stub.yml | wc –l
> ~ 2200 LOC> # cat cf-bs.yml | wc –l
> ~ 300 LOC
> # cat appcloud.yml | wc –l
> ~ 15 LOC
> spiff / bosh-workspace
> SC appcloud bootstrap
CF Summit Berlin 2015 (c) Swisscom 2015
Lifecycle
main challenges
> automated provisioning of PaaS stacks
> keep things “up-to-date”
> integrations
Addressed by:
> standardization
> automation
> testing
> layers of abstraction
> strong CI/CD process
> an awesome team JCF Summit Berlin 2015 (c) Swisscom 2015
Operations
> Challenges:
> Tremendous SLA requirements
> Proper instrumentation and constant monitoring
> Fixing problems ASAP
> Reducing the workload of ops teams
> Addressed by:
> Deep instrumentation
> Loosely-coupled systems (add/remove freely)
> Automation
> Aggregation
> Clear separation of automation vs documentation
CF Summit Berlin 2015 (c) Swisscom 2015
Any
infrastructure
component
The OODA loop
CF Summit Berlin 2015 (c) Swisscom 2015
CF Summit Berlin 2015 (c) Swisscom 2015
OODA loops are all over the place(disjoint and incomplete,AKA existing monitoring and management tools)
IaaS
BOSH
Messaging (NATS)
DEA
Healthmanager
Cloud Controller
Dynamic Router
Service Broker
Logging & Metrics
UAA/Login
User Provided Services
Apps
Buildpacks
Services3rd party integration
Business Integration
Stateful container-based services
SDSOpenstack SDN
External servicesIAM federation ELB/WAF
PortalBilling Extensions
CF Summit Berlin 2015 (c) Swisscom 2015
OODA loops are all over the place(disjoint and incomplete,AKA existing monitoring and management tools)
IaaS
BOSH
Messaging (NATS)
DEA
Healthmanager
Cloud Controller
Dynamic Router
Service Broker
Logging & Metrics
UAA/Login
User Provided Services
Apps
Buildpacks
Services3rd party integration
Business Integration
Stateful container-based services
SDSOpenstack SDN
External servicesIAM federation ELB/WAF
PortalBilling Extensions
CF Summit Berlin 2015 (c) Swisscom 2015
Design principle: don’t reinvent the wheel
Use existing mechanisms as much as possible
Bosh healthmonitor
Consul checks CloudFoundryhealth manager
Puppet policies OpenStack health monitors
PlumGrid health management
Linux/Unix cronjobs
App-specific internal monitoring
InfluxDB data
CF Summit Berlin 2015 (c) Swisscom 2015
Use existing mechanisms as much as possible
Build on top to coordinate and add missing information
Design principle: aggregate and unify
CF Summit Berlin 2015 (c) Swisscom 2015
Delegate the hard decisions to humans, but use them to improve analysis
Use existing mechanisms as much as possible
Build on top to coordinate and add missing information
Design principle: iterative improvement
Image based on a work at hyperboleandahalf.blogspot.com. CC BY-NC-ND 3.0 US
CF Summit Berlin 2015 (c) Swisscom 2015Inspiration: Netflix’s Chaos Monkey (https://github.com/Netflix/SimianArmy/)
Automated disruption to test automated responses
Self testing: Chaos Heidi
Consul checks
Consul checks
Consul checks
Checkresults
Health manager
Status and other Consul events
Message bus
Responder
Responder
Responder
Responder = Puppet, mco, scripts, etc.
Rules
Events
Resu
lts a
nd
logg
ing
even
tsAnalysis engine
Logical architecture
Heidi
agents
CF Summit Berlin 2015 (c) Swisscom 2015
Console
CF Summit Berlin 2015 (c) Swisscom 2015
Console
CF Summit Berlin 2015 (c) Swisscom 2015
Conclusions
> You cannot run CF on a vacuum
> You can’t always have a green field! Integrations are hard
> Network integration
> Security
> Other legacy systems
> Health management is indispensable
> Automate as much as possible
> Delegate hard decisions to humans
> Don’t try to solve everything at once
CF Summit Berlin 2015 (c) Swisscom 2015
developer.swisscom.comCloudFoundry From Switzerland: As Tasty as our Chocolate!
@Swisscom_Dev
CF Summit Berlin 2015 (c) Swisscom 2015
Thank you!Boban GlisovicDiego Zamboni