Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
PAC485Managing Datacenter Resources
Using the VirtualCenter Distributed Resource Scheduler
Carl WaldspurgerPrincipal Engineer, R&D
This presentation may contain VMware confidential information.
Copyright © 2005 VMware, Inc. All rights reserved. All other marks and names mentioned herein may
be trademarks of their respective companies.
Talk Overview
Context and featuresManaging resourcesVirtual machine placementSystem architectureSummary
What Is DRS?DRS = Distributed Resource SchedulerAutomatic virtual machine placement
Optimize load balance across hostsDecide if, when, and where to migrateReact to dynamic load changes
Cluster-wide resource managementScalable resource controlsConfigurable automation levelsIntegrated UI for all controls
DRS Can Help You…
Manage variable loadsWorkloads often dynamic, time-dependentQuickly shift loads in response to demand
Administer many virtual machinesHierarchical organizationDelegated administration
Move towards utility computingThink more about aggregate resource poolsThink less about individual hosts
Where Does DRS Fit In?
New productRequires VirtualCenter 2 and ESX Server 3Modular plug-in for VirtualCenter
DRS moduleImplements algorithms, enforces policiesManaged using VirtualCenter UI
Leverages core technologiesVMotion for migrating live VMs across hostsSophisticated resource management
Key FeaturesVirtual machine placement
Choose initial host when VM powers onDynamic rebalancing using VMotion
Configurable automation levelsManual – recommend initial host and migrationsPartial – automatic initial host, recommend migrationsFull – automatic initial host and migrations
Resource poolsFlexible grouping, sharing, and isolationHierarchical organization and delegation
Talk Overview
Context and featuresManaging resourcesVirtual machine placementSystem architectureSummary
Managing ResourcesBasic controls
Same as in current products Shares – specify relative importanceMin – guaranteed resource availabilityMax – limit resource consumption
Resource poolsNew feature leveraged by DRS Hierarchical management
Basic Control: SharesImportance
Entitlement directly proportional to sharesAnalogy: shares of stock in corporation
Relative unitsAbstract number, only ratios matterEntitlement depends on total shares issued
Named valuesPredefine high, normal, low with 4 : 2 : 1 ratioDefaults to normal
Shares Examples
Change shares forvirtual machineDynamic reallocation
Add virtual machine, overcommit resourcesGraceful degradation
Remove virtual machineExploit extra resources
Basic Control: MinGuaranteed resources
Minimum service level reservationEven when system overcommitted
Absolute unitsMHz for cpu, MB for memoryDefaults to zero for cpu, memory
Virtual machine admission controlReserve resources for minsSum of all VM mins ≤ capacityPrevent power-on if check fails
Min ExampleTotal capacity
600 MHz reserved400 MHz available
Admission control2 VMs try to power-on300 MHz min eachUnable to admit both
VM1 powers onVM2 not admitted
VM1VM1 VM2VM2
Basic Control: MaxResource limit
Upper bound on consumptionEven when system undercommitted
Absolute unitsMHz for CPU, MB for memoryDefaults to “unlimited” for cpuDefaults to guest RAM size for memory
Max ExampleCurrent utilization
600 MHz active400 MHz idle
Start CPU-bound VM200 MHz maxExecution throttled
New utilization800 MHz active200 MHz idleVM prevented fromusing idle resources
VMVM
Resource EntitlementsResources that each VM “deserves”
Combining shares, min, and maxAllocation primarily based on sharesConstrained by min and max
What if VM idles?Don’t give VM more than it demandsResources redistributed to active VMsUnused mins not wasted
Resource PoolsMotivation
Allocate aggregate resources for sets of VMsIsolation between pools, sharing within poolsFlexible hierarchical organizationAccess control and delegation
What is a resource pool?Named object in VirtualCenter inventoryAccess control permissionsMin, max, and shares for both CPU and memoryParent pool, child pools and VMs
Resource Pools ExampleAdmin manages usersPolicy: Alice’s share50% more than Bob’sUsers manage own virtual machines Not shown: min, maxVM allocations:
Bob
200 Admin
VM3
400 Bob
Alice300 Admin
75 Alice 75 Alice
Admin
30%
30%
40%
VM2VM1
Example: Bob Adds Virtual MachineSame policyPools isolate usersAlice still gets 50%more than BobVM allocations:
Bob
200 Admin
400 Bob
Alice300 Admin
75 Alice 75 Alice
Admin
800 Bob
30%
30% 13%
27%VM3VM2VM1 VM4
Resource Pool Admission ControlPool admission control
Same check as before, at pool levelSum of mins for pool children ≤ pool capacityWhen create pool, power-on VM, change settings
Growable Min optionDynamically request more capacity from parentSimplifies policies where hard partitions too rigid
Resource Pools UI
Delegated AdministrationCluster administrator
Default pool contains all cluster resourcesAggregate cpu and memory capacity of all hostsCarves up cluster resources into poolsProvides bulk allocations to pool administrators
Pool administratorPool may reflect department, project, client, etc.Carves up pool resources into smaller pools for users
End userAllocates resources from personal pool to virtual machinesView restricted to personal pool hierarchy
Best PracticesUse Mins and Shares appropriately
Shares generally more flexible policy toolUse shares to isolate without hard partitioningUse mins to guarantee acceptable service
Maintain some spare capacityDon’t use mins that commit entire clusterSlack for maintenance, rebalancingNeeded to tolerate host failures
Talk Overview
Context and featuresManaging resourcesVirtual machine placementSystem architectureSummary
Virtual Machine PlacementGoals
Balance virtual machine load across hosts in clusterEnforce resource policies accuratelyRespect placement constraints
Dynamic balancingMonitor key virtual machine, pool, and host metricsDeliver entitled resources to pools and VMsRecommend migrations (prioritized list)
Initial placementPower on virtual machine in resource poolRecommend host (prioritized list)
Placement ConstraintsVMotion compatibility
Processor typeSAN and LAN connectivity
Anti-affinity rulesRun virtual machines on different hostsMotivation: high-availability, clustering
Affinity rulesRun virtual machines on same hostMotivation: locality, performance benefits
Dynamic BalancingWhat to balance?
Load, adjusted for resource entitlementLoad = utilization, if all VMs equally important
When to balance?Re-evaluate every few minutesChanges to pool or VM settingsAdd or remove host
AggressivenessMigration rate, recommendation strengthDepends on severity of imbalance
Balancing DetailsCompute virtual machine entitlements
Based on pool and virtual machine resource allocationsDon’t give virtual machine more than it demandsReallocate extra resources fairly
Compute host loadsSum entitlements for virtual machines on hostNormalize by host capacity
Consider possible VMotionsEvaluate effect on cluster balanceEvaluate migration cost for involved hosts
Recommend best moves (if any)
Dynamic Balancing UI
Initial Placement UI
Best PracticesFollow strong recommendations
Otherwise balance and fairness may deteriorateSome VMotion is necessary
Enable automationChoose default based on environment, comfort levelUse per-VM automation level overridesLet DRS autonomously manage most VMsCan keep human in loop for critical VMs
Talk Overview
Context and featuresManaging resourcesVirtual machine placementSystem architectureSummary
System Architecture Overview
VirtualCenter
clients
DB
cluster n
•••cluster 1
••• •••
stats + actions
SDKUI
DRS
DRS1
n
•••
System Architecture ConstraintsCluster size
LAN, not WANUp to 32 hosts per clusterHost capacities may differ significantly
Time scaleMinutes, not millisecondsVMotion VM downtime ≈ milliseconds,but end-to-end latency ≈ tens of seconds Migrate VM infrequently ≈ minutes to hours
Algorithm performanceMilliseconds, not minutesOperations occur at human time scale
SummaryAutomatic virtual machine placement
Recommendations or full automationInitial placement at virtual machine power-onDynamic load balancing
Powerful resource controlsFlexible cluster-wide policiesHierarchical resource poolsVirtual machine affinity rules
Future directionsIntegrated I/O bandwidth managementDetect longer-term trends, proactive migration