Web Server Load Balancing/Scheduling

Web Server Load Web Server Load Balancing/SchedulingBalancing/Scheduling

Asima SilvaAsima Silva

Tim SutherlandTim Sutherland

OutlineOutline

Web Server IntroductionWeb Server IntroductionInformation Management BasicsInformation Management BasicsLoad Sharing PoliciesLoad Sharing Policies– FLEXFLEX– WARDWARD– EquiLoadEquiLoad– AdaptLoadAdaptLoad

SummarySummaryConclusionsConclusionsFuture WorkFuture Work

Introduction to Web Server Load Introduction to Web Server Load BalancingBalancing

Request enters a Request enters a router router

Load balancing server Load balancing server determines which determines which web server should web server should serve the requestserve the request

Sends the request to Sends the request to the appropriate web the appropriate web serverserver

InternetInternet

Router

Load-Balancing Server

Web ServersRequestResponse

Traditional Web Cluster

How do we split up information?How do we split up information?

ContentContent Server FarmServer Farm

?

Information StrategiesInformation Strategies

ReplicationReplication PartitionPartition

Load Balancing ApproachesLoad Balancing Approaches

File DistributionFile Distribution RoutingRouting

Content/Locality Content/Locality AwareAware

DNS ServerDNS Server

Size AwareSize Aware Centralized Centralized RouterRouter

Workload AwareWorkload Aware Distributed Distributed DispatcherDispatcher

IssuesIssues

Efficiently processing requests with Efficiently processing requests with optimizations for load balancingoptimizations for load balancing– Send and process requests to a web server Send and process requests to a web server

that has files in cachethat has files in cache– Send and process requests to a web server Send and process requests to a web server

with the least amount of requestswith the least amount of requests– Send and process requests to a web server Send and process requests to a web server

determined by the size of the requestdetermined by the size of the request

FLEXFLEX

Locality awareLocality aware load-balancing strategy load-balancing strategy based on two factors:based on two factors:– Accessed files, memory requirementsAccessed files, memory requirements– Access rates (working set), load requirementsAccess rates (working set), load requirements

PartitionsPartitions all servers into equally all servers into equally balanced groupsbalanced groups

Each server Each server transferstransfers the response to the the response to the browser to reduce bottleneck through the browser to reduce bottleneck through the router (TCP Handoff)router (TCP Handoff)




Size AwareSize Aware Centralized RouterCentralized Router


Flex DiagramFlex Diagram

DNS DNS ServerServer

Requests

To Client Browser

ForwardsRequest

S1 S2

S3 S4

S5 S6

W(S1) ≈ W(S2) ≈ W(S3) ≈ … ≈ W(S6)

Ar(S1) ≈ Ar(S2) ≈ Ar(S3) ≈ … ≈ Ar(S6)

FLEX Cont.FLEX Cont.Advantages:Advantages:– Highly scalableHighly scalable– Reduces bottleneck by the load balancerReduces bottleneck by the load balancer– No software is requiredNo software is required– Reduces number of cache missesReduces number of cache misses

FLEX Cont. IIFLEX Cont. II

Disadvantages:Disadvantages:– Not dynamic, routing tale must be recreated Not dynamic, routing tale must be recreated – Only compared to RROnly compared to RR– Number of access logs required on each Number of access logs required on each

server could be tremendousserver could be tremendous– Responsibility of load-balancing and Responsibility of load-balancing and

transferring response is given to web servers transferring response is given to web servers – unorganized responsibility– unorganized responsibility

– How often to update access rates and working How often to update access rates and working sets? Monitor?sets? Monitor?

WARDWARD

Workload-Aware Request Distribution StrategyWorkload-Aware Request Distribution StrategyServer coreServer core are essential files that represent are essential files that represent majority of expected requests majority of expected requests Server core is Server core is replicatedreplicated at every server at every serverWard-analysis computes the nearly optimal core Ward-analysis computes the nearly optimal core size determined by size determined by workloadworkload access patterns access patterns– Number of nodesNumber of nodes– Node RAMNode RAM– TCP handoff overheadTCP handoff overhead– Disk access overheadDisk access overhead



DNSDNS Server Server



WARD Cont.WARD Cont.Three components: dispatcher (load Three components: dispatcher (load balancer), distributor (router), web serverbalancer), distributor (router), web server

Three progressive architectures:Three progressive architectures:

Dispatcher

Distributor

Front End

LAN

Server

Server

Server

DispatcherDistributor

Front End

LAN

Server

ServerDistributor

SwitchDispatcher

Distributor

Front End

LAN

Server

ServerDistributor

Switch

Dispatcher

Single front-end distributor, centralized dispatcher

Co-located distributor and server

Co-located distributor, server, and dispatcher

LARD CARD WARD

WARD DiagramWARD Diagram

Requests

S1S2

S3

S4

S5S6

Queue:Queue:

Queue:

Queue:

Queue:Queue:

SwitchSwitch

•Each computer is a distributor and a dispatcher

WARD Cont. IIWARD Cont. II

Similar to FLEX, sends response directly to Similar to FLEX, sends response directly to clientclient

Minimizes forwarding overhead from handoffs Minimizes forwarding overhead from handoffs for the most frequent filesfor the most frequent files

Optimizes the overall cluster RAM usageOptimizes the overall cluster RAM usage

““by mapping a small set of most frequent files to by mapping a small set of most frequent files to be served by multiple number of nodes, we can be served by multiple number of nodes, we can improve both locality of accesses and the cluster improve both locality of accesses and the cluster performance significantly”performance significantly”

WARD Cont. IIIWARD Cont. III

Advantages:Advantages:– No decision making, core files are replicated No decision making, core files are replicated

on every serveron every server– Minimizes transfer of requests and disk reads, Minimizes transfer of requests and disk reads,

both are “equally bad”both are “equally bad”– Outperforms Round RobinOutperforms Round Robin– Efficient use of RAMEfficient use of RAM– Performance gain with increased number of Performance gain with increased number of

nodesnodes

WARD Cont. IVWARD Cont. IV

Disadvantages:Disadvantages:– Core files are created on past day’s data, Core files are created on past day’s data,

could decrease performance up to 15%could decrease performance up to 15%– Distributed dispatcher increases the number Distributed dispatcher increases the number

of TCP requests transfersof TCP requests transfers– If core files not selected correctly, higher If core files not selected correctly, higher

cache miss rate and increased disk accessescache miss rate and increased disk accesses

WARD ResultsWARD Results

EquiLoadEquiLoad

Determines which server will process a Determines which server will process a request determined by the size of the request determined by the size of the requested filerequested file

Splits the content on each server by file Splits the content on each server by file size, forcing the queues sizes to be size, forcing the queues sizes to be consistent.consistent.



DNSDNS Server Server



EquiLoad Solves Queue Length EquiLoad Solves Queue Length ProblemsProblems

This is badThis is bad

QueueQueue

QueueQueue

This is This is betterbetter

QueueQueue

QueueQueue

1k1k 1k1k 1000k1000k

1k1k 1k1k 1k1k 2k2k 2k2k 1k1k 1k1k 2k2k 1k1k

100k100k 100k100k

1k1k 1k1k 1k1k 2k2k 100k100k

EquiLoad DiagramEquiLoad Diagram

DistributorDistributor

Requests

To Client Browser

ForwardsRequest

S1 S2

S3 S4

S5 S6

1k-2k 2k-3k

3k-10k 10k-20k

20k-100k >100k

DispatcherDispatcher(periodically calculates (periodically calculates partitions)partitions)

EquiLoadEquiLoad

AdvantagesAdvantages– Dynamic repartitioningDynamic repartitioning– Can be implemented at various levelsCan be implemented at various levels

DNSDNS

DispatcherDispatcher

ServerServer

– Minimum queue buildupMinimum queue buildup– Performs well under variable workload and Performs well under variable workload and

high system loadhigh system load

EquiLoadEquiLoad

DisadvantagesDisadvantages– Cache affinity is neglectedCache affinity is neglected– Requires a front end dispatcherRequires a front end dispatcher– Distributor must communicate with serversDistributor must communicate with servers– Thresholds of parameter adjustmentThresholds of parameter adjustment

EquiLoad EquiLoad AdaptLoad AdaptLoad

AdaptLoad improves upon EquiLoad using AdaptLoad improves upon EquiLoad using “fuzzy boundaries”“fuzzy boundaries”– Allows for multiple servers to process a Allows for multiple servers to process a

requestrequest– Behaves better in situations where server Behaves better in situations where server

partitions are very close in sizepartitions are very close in size

AdaptLoad DiagramAdaptLoad Diagram

DispatcherDispatcher

Requests

To Client Browser

ForwardsRequest

S1 S2

S3 S4

S5 S6

1k-3k 2k-4k

3k-10k 8k-20k

15k-100k >80k

DistributorDistributor(periodically calculates (periodically calculates partitions)partitions)

AdaptLoad ResultsAdaptLoad Results

SummarySummary




Size AwareSize Aware Centralized Centralized RouterRouter


FLEX

EquiLoad, AdaptLoad

WARD

ConclusionsConclusions

There is no “best” way to distribute content There is no “best” way to distribute content among servers.among servers.

There is no optimal policy for all website There is no optimal policy for all website applications.applications.

Certain strategies are geared towards a Certain strategies are geared towards a particular website application.particular website application.

Future WorkFuture Work

Compare and contrast the three policiesCompare and contrast the three policies

Figure out how often nodes should be Figure out how often nodes should be repartitionedrepartitioned

Compare each policy to a standard Compare each policy to a standard benchmarkbenchmark

Figure out which policy works in a Figure out which policy works in a particular environmentparticular environment

Questions?Questions?

Anyone have one?Anyone have one?

Documents

Web Server Load Balancing/Scheduling