Load balancing (computing)

In computing, load balancing distributes workloadsacross multiple computing resources, such as comput-ers, a computer cluster, network links, central process-ing units or disk drives. Load balancing aims to opti-mize resource use, maximize throughput, minimize re-sponse time, and avoid overload of any single resource.Using multiple components with load balancing insteadof a single component may increase reliability throughredundancy. Load balancing usually involves dedicatedsoftware or hardware, such as a multilayer switch or aDomain Name System server process.Load balancing differs from channel bonding in that loadbalancing divides traffic between network interfaces ona network socket (OSI model layer 4) basis, while chan-nel bonding implies a division of traffic between physicalinterfaces at a lower level, either per packet (OSI modelLayer 3) or on a data link (OSI model Layer 2) basis.

1 Internet-based services

One of the most commonly used applications of load bal-ancing is to provide a single Internet service from multi-ple servers, sometimes known as a server farm. Com-monly load-balanced systems include popular web sites,large Internet Relay Chat networks, high-bandwidth FileTransfer Protocol sites, Network News Transfer Protocol(NNTP) servers, Domain Name System (DNS) servers,and databases.For Internet services, the load balancer is usually a soft-ware program that is listening on the port where exter-nal clients connect to access services. The load bal-ancer forwards requests to one of the “backend” servers,which usually replies to the load balancer. This allowsthe load balancer to reply to the client without the clientever knowing about the internal separation of functions.It also prevents clients from contacting back-end serversdirectly, which may have security benefits by hiding thestructure of the internal network and preventing attackson the kernel’s network stack or unrelated services run-ning on other ports.Some load balancers provide a mechanism for doingsomething special in the event that all backend servers areunavailable. This might include forwarding to a backupload balancer, or displaying a message regarding the out-age.It is also important that the load balancer itself does notbecome a single point of failure. Usually load balancers

are implemented in high-availability pairs whichmay alsoreplicate session persistence data if required by the spe-cific application.[1]

1.1 Round-robin DNS

Main article: Round-robin DNS

An alternate method of load balancing, which does notnecessarily require a dedicated software or hardwarenode, is called round robin DNS. In this technique, mul-tiple IP addresses are associated with a single domainname; clients are expected to choose which server to con-nect to. Unlike the use of a dedicated load balancer,this technique exposes to clients the existence of multi-ple backend servers. The technique has other advantagesand disadvantages, depending on the degree of controlover the DNS server and the granularity of load balanc-ing desired.Another more effective technique for load-balancing us-ing DNS is to delegate www.example.org as a sub-domain whose zone is served by each of the same serversthat are serving the web site. This technique works partic-ularly well where individual servers are spread geograph-ically on the Internet. For example,one.example.org A 192.0.2.1 two.example.org A203.0.113.2 www.example.org NS one.example.orgwww.example.org NS two.example.orgHowever, the zone file for www.example.org on eachserver is different such that each server resolves its ownIP Address as the A-record.[2] On server one the zone filefor www.example.org reports:@ in a 192.0.2.1On server two the same zone file contains:@ in a 203.0.113.2This way, when a server is down, its DNSwill not respondand the web service does not receive any traffic. If the lineto one server is congested, the unreliability of DNS en-sures less HTTP traffic reaches that server. Furthermore,the quickest DNS response to the resolver is nearly alwaysthe one from the network’s closest server, ensuring geo-sensitive load-balancing . A short TTL on the A-recordhelps to ensure traffic is quickly diverted when a servergoes down. Consideration must be given the possibilitythat this technique may cause individual clients to switchbetween individual servers in mid-session.

1

2 1 INTERNET-BASED SERVICES

1.2 Scheduling algorithms

Numerous scheduling algorithms are used by load bal-ancers to determine which back-end server to send a re-quest to. Simple algorithms include random choice orround robin. More sophisticated load balancers may takeadditional factors into account, such as a server’s reportedload, least response times, up/down status (determined bya monitoring poll of some kind), number of active con-nections, geographic location, capabilities, or how muchtraffic it has recently been assigned.

1.3 Persistence

An important issue when operating a load-balanced ser-vice is how to handle information that must be kept acrossthe multiple requests in a user’s session. If this informa-tion is stored locally on one backend server, then subse-quent requests going to different backend servers wouldnot be able to find it. This might be cached informationthat can be recomputed, in which case load-balancing arequest to a different backend server just introduces aperformance issue.Ideally the cluster of servers behind the load balancershould be session-aware, so that if a client connectsto any backend server at any time the user experienceis unaffected. This is usually achieved with a shareddatabase or an in-memory session database, for exampleMemcached.One basic solution to the session data issue is to send allrequests in a user session consistently to the same backendserver. This is known as persistence or stickiness. A sig-nificant downside to this technique is its lack of automaticfailover: if a backend server goes down, its per-sessioninformation becomes inaccessible, and any sessions de-pending on it are lost. The same problem is usually rel-evant to central database servers; even if web servers are“stateless” and not “sticky”, the central database is (seebelow).Assignment to a particular server might be based on ausername, client IP address, or be random. Because ofchanges of the client’s perceived address resulting fromDHCP, network address translation, and web proxies thismethod may be unreliable. Random assignments must beremembered by the load balancer, which creates a burdenon storage. If the load balancer is replaced or fails, thisinformation may be lost, and assignments may need to bedeleted after a timeout period or during periods of highload to avoid exceeding the space available for the assign-ment table. The random assignment method also requiresthat clients maintain some state, which can be a problem,for example when a web browser has disabled storage ofcookies. Sophisticated load balancers use multiple per-sistence techniques to avoid some of the shortcomings ofany one method.Another solution is to keep the per-session data in a

database. Generally this is bad for performance becauseit increases the load on the database: the database is bestused to store information less transient than per-sessiondata. To prevent a database from becoming a single pointof failure, and to improve scalability, the database is of-ten replicated across multiple machines, and load balanc-ing is used to spread the query load across those replicas.Microsoft's ASP.net State Server technology is an exam-ple of a session database. All servers in a web farm storetheir session data on State Server and any server in thefarm can retrieve the data.Efficient but simple approaches are a very common casewhere the client is a web browser, per-session data canbe stored in the browser itself. One way to achieve thisis to use a browser cookie, suitably time-stamped and en-crypted. Another is URL rewriting. Storing session dataon the client is generally the preferred solution: then theload balancer is free to pick any backend server to handlea request. However, this method of state-data handling ispoorly suited to some complex business logic scenarios,where session state payload is big and recomputing it withevery request on a server is not feasible. URL rewritinghas major security issues, because the end-user can easilyalter the submitted URL and thus change session streams.

1.4 Load balancer features

Hardware and software load balancers may have a varietyof special features. The fundamental feature of a loadbalancer is to be able to distribute incoming requests overa number of backend servers in the cluster according to ascheduling algorithm. Most of the following features arevendor specific:

• Asymmetric load: A ratio can be manually assignedto cause some backend servers to get a greater shareof the workload than others. This is sometimes usedas a crude way to account for some servers havingmore capacity than others and may not always workas desired.

• Priority activation: When the number of availableservers drops below a certain number, or load getstoo high, standby servers can be brought online.

• SSL Offload and Acceleration: Depending on theworkload, processing the encryption and authenti-cation requirements of an SSL request can becomea major part of the demand on the Web Server’sCPU; as the demand increases, users will see slowerresponse times, as the SSL overhead is distributedamong Web servers. To remove this demand onWeb servers, a balancer can terminate SSL connec-tions, passing HTTPS requests as HTTP requests tothe Web servers. If the balancer itself is not over-loaded, this does not noticeably degrade the perfor-mance perceived by end users. The downside of this

3

approach is that all of the SSL processing is con-centrated on a single device (the balancer) whichcan become a new bottleneck. Some load balancerappliances include specialized hardware to processSSL. Instead of upgrading the load balancer, whichis quite expensive dedicated hardware, it may becheaper to forgo SSL offload and add a few Webservers. Also, some server vendors such as Ora-cle/Sun now incorporate cryptographic accelerationhardware into their CPUs such as the T2000. F5Networks incorporates a dedicated SSL accelerationhardware card in their local traffic manager (LTM)which is used for encrypting and decrypting SSLtraffic. One clear benefit to SSL offloading in thebalancer is that it enables it to do balancing or con-tent switching based on data in the HTTPS request.

• Distributed Denial of Service (DDoS) attack protec-tion: load balancers can provide features such asSYN cookies and delayed-binding (the back-endservers don't see the client until it finishes its TCPhandshake) to mitigate SYN flood attacks and gen-erally offload work from the servers to a more effi-cient platform.

• HTTP compression: reduces amount of data to betransferred for HTTP objects by utilizing gzip com-pression available in all modern web browsers. Thelarger the response and the further away the clientis, the more this feature can improve response times.The tradeoff is that this feature puts additional CPUdemand on the Load Balancer and could be done byWeb servers instead.

• TCP offload: different vendors use different termsfor this, but the idea is that normally each HTTPrequest from each client is a different TCP connec-tion. This feature utilizes HTTP/1.1 to consolidatemultiple HTTP requests from multiple clients into asingle TCP socket to the back-end servers.

• TCP buffering: the load balancer can buffer re-sponses from the server and spoon-feed the data outto slow clients, allowing the web server to free athread for other tasks faster than it would if it hadto send the entire request to the client directly.

• Direct Server Return: an option for asymmetricalload distribution, where request and reply have dif-ferent network paths.

• Health checking: the balancer polls servers for appli-cation layer health and removes failed servers fromthe pool.

• HTTP caching: the balancer stores static content sothat some requests can be handled without contact-ing the servers.

• Content filtering: some balancers can arbitrarilymodify traffic on the way through.

• HTTP security: some balancers can hide HTTP er-ror pages, remove server identification headers fromHTTP responses, and encrypt cookies so that endusers cannot manipulate them.

• Priority queuing: also known as rate shaping, theability to give different priority to different traffic.

• Content-aware switching: most load balancers cansend requests to different servers based on the URLbeing requested, assuming the request is not en-crypted (HTTP) or if it is encrypted (via HTTPS)that the HTTPS request is terminated (decrypted)at the load balancer.

• Client authentication: authenticate users against avariety of authentication sources before allowingthem access to a website.

• Programmatic traffic manipulation: at least one bal-ancer allows the use of a scripting language to allowcustom balancing methods, arbitrary traffic manip-ulations, and more.

• Firewall: direct connections to backend servers areprevented, for network security reasons Firewall is aset of rules that decide whether the traffic may passthrough an interface or not.

• Intrusion prevention system: offer application layersecurity in addition to network/transport layer of-fered by firewall security.

2 Use in telecommunications

Load balancing can be useful in applications with redun-dant communications links. For example, a companymay have multiple Internet connections ensuring networkaccess if one of the connections fails. A failover arrange-ment would mean that one link is designated for normaluse, while the second link is used only if the primary linkfails.Using load balancing, both links can be in use all thetime. A device or program monitors the availability ofall links and selects the path for sending packets. The useof multiple links simultaneously increases the availablebandwidth.

2.1 Shortest Path Bridging

Main article: Shortest Path Bridging

The IEEE approved the IEEE 802.1aq standard May2012,[3] also known and documented in most books asShortest Path Bridging (SPB). SPB allows all links to beactive through multiple equal cost paths, provides fasterconvergence times to reduce down time, and simplifies

4 6 EXTERNAL LINKS

the use of load balancing in mesh network topologies(partially connected and/or fully connected) by allowingtraffic to load share across all paths of a network.[4][5]SPB is designed to virtually eliminate human error dur-ing configuration and preserves the plug-and-play naturethat established Ethernet as the de facto protocol at Layer2.[6]

2.2 Routing

Further information: Routing

Many telecommunications companies have multipleroutes through their networks or to external networks.They use sophisticated load balancing to shift traffic fromone path to another to avoid network congestion on anyparticular link, and sometimes to minimize the cost oftransit across external networks or improve network reli-ability.Another way of using load balancing is in network moni-toring activities. Load balancers can be used to split hugedata flows into several sub-flows and use several networkanalyzers, each reading a part of the original data. Thisis very useful for monitoring fast networks like 10GbE orSTM64, where complex processing of the data may notbe possible at wire speed.

3 Relationship to failovers

Load balancing is often used to implement failover—thecontinuation of a service after the failure of one or moreof its components. The components are monitored con-tinually (e.g., web servers may be monitored by fetchingknown pages), and when one becomes non-responsive,the load balancer is informed and no longer sends trafficto it. When a component comes back on line, the load bal-ancer begins to route traffic to it again. For this to work,there must be at least one component in excess of theservice’s capacity. This can be much less expensive andmore flexible than failover approaches where each singlelive component is paired with a single backup compo-nent that takes over in the event of a failure. Some typesof RAID systems can also utilize hot spare for a similareffect.

4 See also

• Affinity mask

• Application Delivery Controller

• Autoscaling

• Cloud computing

• Common Address Redundancy Protocol

• Edge computing

• Fog computing

• Network Load Balancing

• Network Load Balancing Services

• Processor affinity

• SRV record

5 References[1] “High Availability”. linuxvirtualserver.org. Retrieved

2013-11-20.

[2] IPv4 Address Record (A)

[3] Shuang Yu (8 May 2012). “IEEE APPROVESNEW IEEE 802.1aq™ SHORTEST PATH BRIDGINGSTANDARD”. IEEE. Retrieved 2 June 2012.

[4] Peter Ashwood-Smith (24 Feb 2011). “Shortest PathBridging IEEE 802.1aq Overview”. Huawei. Retrieved11 May 2012.

[5] Jim Duffy (11 May 2012). “Largest Illinois healthcaresystem uproots Cisco to build $40M private cloud”. PCAdvisor. Retrieved 11 May 2012. Shortest Path Bridgingwill replace Spanning Tree in the Ethernet fabric.

[6] “IEEE Approves New IEEE 802.1aq Shortest Path Bridg-ing Standard”. Tech Power Up. 7 May 2012. Retrieved11 May 2012.

6 External links• Dynamic Load Balancing and Scheduling

• A sample implementation with source codes byserver routing for load balancing with full auto fail-ure recovery and session-sticking/affinity supported

• Open source based Zen Load Balancer

• ProxyOS.org Linux distribution focus on load bal-ancing

5

7 Text and image sources, contributors, and licenses

7.1 Text• Load balancing (computing) Source: http://en.wikipedia.org/wiki/Load%20balancing%20(computing)?oldid=656547838 Contributors:The Anome, Sharuzzaman, Patrick, TakuyaMurata, Karada, Glenn, Whkoh, Conti, Joy, Jamesday, Jni, Hankwang, Greudin, Mikepel-ley, Isopropyl, Giftlite, DavidCary, Varlaam, JimD, Remy B, Gracefool, Wmahan, Chowbok, Pgan002, Beland, Olivier Debre, Green-Reaper, Ferrans, Gazpacho, Discospinster, Rich Farmbrough, JoeSmack, Phrozenfire, Kb, Mwanner, Jbyers, Vipul, Jpgordon, Reiny-day, YoLoL, Mikemsd, A-Day, Googol plex, Liao, Walter Görlitz, Wensong, Yamla, Cburnett, MrVibrating, Smaira24, Marasmusine,Alvis, Plicease, Pchov, Topgun9, Jwoodger, Rjwilmsi, Barklund, UriBudnik, Toby Douglass, FlaBot, SchuminWeb, Ghen, Oweng, KFP,Simishag, Chobot, Bgwhite, Daverocks, Splash, ChrisBrown, Kimchi.sg, Mipadi, Porao, Spike Wilbury, Slarson, Amwebb, Dawhitfield,Emijrp, Zzuuzz, LeonardoRob0t, SmackBot, PaulWay, Dparvin, Kargoneth, Gilliam, Oli Filth, JonHarder, Radagast83, Datashark, Iz-toku, The undertow, Kuru, Ckatz, JHunterJ, Beetstra, Davydog, Kirkbauer, Kvng, Hu12, Msyme, Tawkerbot2, Rodym, Ale jrb, Particle-bry, Dfrench75, Phatom87, NHRinger, Cydebot, Gogo Dodo, Etienne.navarro, B, Eneville, Tomcat66 g500, Thijs!bot, Pjvpjv, Towope-dia, Donspaulding, Kr2006, AntiVandalBot, Alphachimpbot, Matfz, Vernetto, TenguTech, JAnDbot, Barek, MER-C, CosineKitty, Mar-tinkunev, PhilKnight, Yaneshtyagi, Shigdon, Ariel@compucall, Jscriba, Mouchoir le Souris, Thetobi, Kovyrin, Jim.henderson, R'n'B, Pub-lic Menace, Barts1a, Pctrettp, Phani96, Lowbassman, Buckyball77, Siteobserver, Technopat, Cheskis, Ghowett, Jamelan, Billinghurst,Lova Falk, Ealtman, Pmelerud, Nagy, Kbrose, SieBot, Gerakibot, Tataryn, Cfleisch, Mbenna, C'est moi, Trustable, CultureDrone,Chillum, Kwatanabe, C0nanPayne, GoldenBear92, MrRadic, ClueBot, The Thing That Should Not Be, Aintneo, EoGuy, Incapearl,Aua, Vin Kaleu, Eeekster, HenryFoster, Ash71p, Jphebert, NeuroDrone, DumZiBoT, Bmatschke, Stevedotdavis, Creekwayne, Tim-Tay, XLinkBot, Stickee, Dejaphoenix, Dsimic, Addbot, Mortense, Some jerk on the Internet, Rleanos, Favonian, Jasper Deng, M2mcf,Zhong.xie, Yobot, Coyotekish, Jborman, AnomieBOT, AmritasyaPutra, ThaddeusB, Emilyhawkins, Kyleflaherty, Materialscientist, Mart-inhill, Merlissimo, Brian2wood, GrouchoBot, Mmoayyed, Emilyjetnexus, Bobchrist, Thehelpfulbot, FrescoBot, Wooblox, Meltem par-maksiz, Zhub, HJ Mitchell, Geek2003, Ivoras, Xattila, Tiepatel, RalphBarron, Petergjansen, Jandalhandler, Charles.mjhm, Funkysh,RockSolidCosmo, Linahwang, Emilyychew, Ariyako, EmausBot, Eight40, LindseyEKerr, Wikipelli, ZacBowlingAlt, Exschwimmer, In-foElfiq, Rotpunkt, Nstronics, ChuispastonBot, NTox, Guoxiao281, Anita5192, ClueBot NG, Cyberajith, CocuBot, Personalmountains,Sleddog116, Mirekez, Saarp, Liran.zelkha, Theopolisme, Helpful Pixie Bot, MsFionnuala, Eriksquires, Bedis9, Castleinfo6, Zujua, Kyu-den, BattyBot, Emily.hawkins72, EdWitt, Manorsoft, Cpinckney32, Avanuopsw, Jamesmcmahon0, François Robere, Carschno, Werdde-mer, Meteor sandwich yum, Wwatinski, Jvkiet, Loadbalancer2, Jeffstaten, Dguerri, Biblioworm, Adaviswiki and Anonymous: 337

7.2 Images• File:Ambox_important.svg Source: http://upload.wikimedia.org/wikipedia/commons/b/b4/Ambox_important.svg License: Public do-main Contributors: Own work, based off of Image:Ambox scales.svg Original artist: Dsmurat (talk · contribs)

• File:Bus_icon.svg Source: http://upload.wikimedia.org/wikipedia/commons/c/ca/Bus_icon.svg License: Public domain Contributors: ?Original artist: ?

• File:Edit-clear.svg Source: http://upload.wikimedia.org/wikipedia/en/f/f2/Edit-clear.svg License: Public domain Contributors: TheTango! Desktop Project. Original artist:The people from the Tango! project. And according to the meta-data in the file, specifically: “Andreas Nilsson, and Jakub Steiner (althoughminimally).”

• File:Question_book-new.svg Source: http://upload.wikimedia.org/wikipedia/en/9/99/Question_book-new.svg License: Cc-by-sa-3.0Contributors:Created from scratch in Adobe Illustrator. Based on Image:Question book.png created by User:Equazcion Original artist:Tkgd2007

7.3 Content license• Creative Commons Attribution-Share Alike 3.0