J. Christopher WagnerRobert Bosch
LHC3174BE
#VMworld #LHC3174BE
VMware Cloud on AWS: An Architectural and Operational Deep Dive
VMworld 2017 Content: Not fo
r publication or distri
bution
• This presentation may contain product features that are currently under development.
• This overview of new technology represents no commitment from VMware to deliver these features in any generally available product.
• Features are subject to change, and must not be included in contracts, purchase orders, or sales agreements of any kind.
• Technical feasibility and market demand will affect final delivery.
• Pricing and packaging for any new technologies or features discussed or presented have not been determined.
Disclaimer
#LHC3174BU CONFIDENTIAL 2
VMworld 2017 Content: Not fo
r publication or distri
bution
VMware Cloud on AWS – Architectural Overview
• Level setting – The big picture
• The Store Window
– The console
– The vCenter permissions model
• The Raw Materials
• The Factory Floor – Day in the (death of) a host
#LHC3174BU CONFIDENTIAL 3
VMworld 2017 Content: Not fo
r publication or distri
bution
VMC: SDDC as a Service on AWSKey Concepts
1
2
3Customer
datacenter
Managed SDDC stack hosted on public clouds
– Converged compute (ESXi), storage (VSAN), network (NSX)
– SDDC clusters, not commodity VMs
– Installation, patching, and upgrades managed by VMware
Consistent operational model enables hybrid cloud
– Managed via vCenter including full API and CLI support
– Seamless workload mobility on-prem/cloud and cloud/cloud
– Hybrid and Cloud-only deployment options
Leverage cloud economics aligning capacity & demand
– Elastic cloud capacity lets customers scale on demand
– Dedicated, single tenant, secure
– Single bill for VMware software + Cloud capacity
VMware SDDC as a Service
AWS cloud servers, storage, networking
AWS Cloud
#LHC3174BU CONFIDENTIAL 4
VMworld 2017 Content: Not fo
r publication or distri
bution
The Store WindowConsole and vCenter
VMworld 2017 Content: Not fo
r publication or distri
bution
The VMC Console
#LHC3174BU CONFIDENTIAL 6
VMworld 2017 Content: Not fo
r publication or distri
bution
Orgs (Organizations)
• Flexible Container for:
– Authentication (including federation)
– Authorization (OrgOwners control RBAC for OrgMembers)
– Service access and subscriptions
• Simply put:
– A user can be a member of one or more Orgs
– A user can have different roles in different Orgs
– An Org can be associated with one or more services
– A user can have different roles for different services within an Org
• UX
– Users work in the context of a single Org – has switcher built in
Currently not federated with SSO
#LHC3174BU CONFIDENTIAL 7
VMworld 2017 Content: Not fo
r publication or distri
bution
#LHC3174BU CONFIDENTIAL 8
VMworld 2017 Content: Not fo
r publication or distri
bution
vCenter – Permissions
• Guiding principles
– Retain control of all management and infrastructure components
– All else consistent with standard vCenter
YES
• VMOps, vApp
• Resource
• Customer Datastore*
• Network* (logical networks)
• SPBM
• Content Library
• Tagging
• Folder, System, Alarm*
NO
• Host
• Datacenter
• Mgmt Datastore
• Network (physical)
• Cluster
NO (in design)
• SMP-FT
• vmCrypt
• VIBS
• H5 Plugins
#LHC3174BU CONFIDENTIAL 9
VMworld 2017 Content: Not fo
r publication or distri
bution
The Raw Materials
VMworld 2017 Content: Not fo
r publication or distri
bution
VMware Cloud on AWS: Architecture
On-Premises
vCenter
VPN
Hybrid Networking
HybridLinked Mode
Single Pane of Glass UI,Hybrid VM Provisioning
Provisioning, Lifecycle,
Operations
Metrics, Logs,Events, Billing
SaaS
CSP
Identity
Billing
Subscription
VMC
Console
AWS Driver
SRE/OI
Metrics
Logs
Alerts
Fleet
Mgmt
Customer-
Owned
VPCVMware-Owned VPC
ELB
RDS
S3
VM VMVM VMVM VM
vCenter
vSAN
VMC
PoP
ESX
I3NSX
HA/DRS
ESX
I3
ESX
I3
ESX
I3
ESX
I3
ESX
I3
#LHC3174BU CONFIDENTIAL 11
VMworld 2017 Content: Not fo
r publication or distri
bution
#LHC3174BU CONFIDENTIAL 12
VMworld 2017 Content: Not fo
r publication or distri
bution
On-Premises
vCenter
VPN
Hybrid Networking
HybridLinked Mode
Single Pane of Glass UI,Hybrid VM Provisioning
Customer-
Owned
VPCVMware-Owned VPC
ELB
RDS
S3
ESX
VM VM
ESX
VM VM
ESX
VM VM
HA/DRS
vCenter
vSAN
NSXI3 I3 I3
Provisioning, Lifecycle,
Operations
Metrics, Logs,Events, Billing
VMC
PoP
SaaS
CSP
Identity
Billing
Subscription
VMC
Console
AWS Driver
SRE/OI
Metrics
Logs
Alerts
Fleet
Mgmt
VMware Cloud on AWS: The Cloud Data Center
#LHC3174BU CONFIDENTIAL 13
VMworld 2017 Content: Not fo
r publication or distri
bution
SDDC Deployment Architecture
• Single SDDC software stack per Cloud Data Center
• Management appliances run on the customer’s cluster
– Protected using hard resource reservations and permissions
• Hosts organized into a HA/DRS/vSAN cluster
– Minimum of 4 hosts per cluster
• vCenter Server Appliance with Embedded Platform Services Controller
• Agent/PoP: Native EC2 VM deployed alongside the SDDC
– Service functionality that needs to run in the SDDC environment: deploy, log/metric filtering
– Jumpbox for accessing the management network for troubleshooting
#LHC3174BU CONFIDENTIAL 14
VMworld 2017 Content: Not fo
r publication or distri
bution
New vCenter Features for VMC
• Hybrid Linked Mode
– Single pane of glass spanning on-premises and cloud
– Works across different administrative domains
– Works across different SDDC versions
• Pod Service
– Orchestration of operations that span vSphere/vSAN/NSX
– Add/remove host
– Patch/update/upgrade SDDC
– Add/remove cluster (future)
• Backup/Restore
– Mgmt appliances – prior to patch
– Mgmt appliances – periodic
#LHC3174BU CONFIDENTIAL 15
VMworld 2017 Content: Not fo
r publication or distri
bution
DLR
Default 10.1.1.0/24
Default Compute GW
(NAT, FW, VPN, DHCP, DNS)
AWS Network
Internet GW
VMware Cloud on AWS – Default Networking Components
N-S Internet Traffic
VMware Cloud on AWS
Networking (NSX)Workloads on
logical networks
Management Infrastructure
Management GW
(NAT, FW, VPN, DNS)
10.1.3.0/2410.1.2.0/24 10.1.240/24
#LHC3174BU CONFIDENTIAL 16
VMworld 2017 Content: Not fo
r publication or distri
bution
The Factory FloorDay in the (death) of a host
VMworld 2017 Content: Not fo
r publication or distri
bution
The Players
• VPC/ESX – vCenter + ESX
• VPC/PoP – fm-monitoring-agent
• SaaS – MQ
• SaaS – Alert Processing Engine
• SaaS – Service Desk (ticketing)
• SaaS – Autoscaler
• SaaS – Provisioning Engine
• VPC/ESX – Pod
• VPC/ESX – vCenter
#LHC3174BU CONFIDENTIAL 18
VMworld 2017 Content: Not fo
r publication or distri
bution
Failed Host Remediation
19
Service Desk
ESHistorical
APEStream + historical + debounce
AutoscalerChecks:• Verify• Standby/maint
MQ
ESX -> VC (/minute)
(misses 2 -> alert)
FMA -> ESX (/30s)
(critical services)
FMA -> VC (/30s)
(alerts)
Console/provision
POD
VC
AWS
Install
Remediate:• Reboot• Add host• Re-verify• VSAN rebuild• Garage old host for triage
#LHC3174BU CONFIDENTIAL
VMworld 2017 Content: Not fo
r publication or distri
bution
Autoscaler Injecting vCenter Events
#LHC3174BU CONFIDENTIAL 20
VMworld 2017 Content: Not fo
r publication or distri
bution
SDDC-SRE Team
• Dedicated team handling SDDC mgmt and infrastructure alerts
• Fully integrated with alerting, service desk
• RTS service:
– Automated runbooks
– Manual runbook accelerators
– Provides auditable access to customer infrastructure
#LHC3174BU CONFIDENTIAL 21
VMworld 2017 Content: Not fo
r publication or distri
bution
VMworld 2017 Content: Not fo
r publication or distri
bution
VMworld 2017 Content: Not fo
r publication or distri
bution