Least load dispatching algorithm server nodes

for parallel Web

K.H. Yeung, K.W. Suen and K.Y. Wong

Abstract: A least load dispatching algorithm for distributlng requests to parallel Web server nodes is described. In this algorithm, the load offered to a node by a request is estimated based on the expected transfer time of the corresponding reply through the Intemet. This loading information is then used by thc algorithm to identify the least load node of the Web site. By using this algorithm, each request will always be sent for service at the earliest possible time. Performance comparison using NASA and ClarkNet access logs between the proposed algorithm and commonly used dispatching algorithms is performed. The results show that the proposed algorithm gives 10% higher throughput than that of the commonly used random and round-robin dispatching algorithms.

1 Introduction

The use of World Wide Web (Web or WWW) has grown exponentially in recent years. Popular Web sites need to serve millions of client requests per day. For example, there are over 120 millioii HTTP hits per day at Netscape's site [I]. To support this growing request rate, scalable Web server architectures on a cluster of server nodes have been studied extensively [IM]. In these systems, each node serves the incoming requests independently as a single server site and returns the requested documents to the clients directly. The document set can be replicated on each node's local disk_ partitioned according to the document tree and then stored on different nodes, or stored in a central storage and shared by the nodes through a LAN. To evenly spread the requests among the nodes, systems with different dispatch- ing algorithms have been proposed. A good sumidly of distributed Web-server architectures can be found in [2, 31. Existing systems include Netscape Navigator [I], Smart Client 171, SWEB [4], Round Robin DNS [5], ONE-IP 181, TCP router [9], and cNetwork Dispatcher [IO]. The dispatching algorithm used by these techniques can be classified into either a fcedback type or a non-feedback type algorithm.

Fcedback-type algorithms require the current loading information of the nodes for node selection. Smart Client [7] uses a client interface applet and a director applet to provide the user interface, and masks the details of how to contact each individual node. In the Network of Workstations (NOW) Project at the University of California, Berkeley, Smart Client selects a least loaded node among 100 nodes to open a Telnet window. To do this, browser loads the Telnet Web page with the two applets. The director applet periodically polls the NOW'S operating system to retrieve the load averages of the nodes in the site. When a user clicks on a button provided by the client interface applet, the clieni will select the node based on the current load information. This technique works best when all clients

0 IEE. 2002 IEE Proceedings online no. 20020169 DO/: IU.IM9/ip~com:?0020169 Paper fint m i & I Ilh October Zoo0 and in revised fnm 18th Desemher 2001 The authors are with LhC Deprimeni olEktronic Engiaeenng, City Unirenity of Hong Kong. Tat Chee Avenue. Kowloon Tong, Kowloon. Hong Kong

IEE Proc Comm~i., Vol. 149. I\'". 4, August 2W2

locate inside an intranet. When the client locates outside the intranet, this technique potentially increases the Web traffic through the Internet. This technique basically uses a least load dispatching algorithm.

SWEB [4] employs both the Round Robin DNS and H l T P redirection [ I I] to provide load balancing. The user requests are first routed to SWEB nodes via the DNS rotation. Each node contains a scheduler and collaborates with other nodes for dynamic exchange in the system CPU, network and disk load information. After a node receives a request. the scheduler decides to process it locally or assign it to another node using HTTP redirection. To avoid the ping-pong effect, any HTTP request is only allowed to be redirected once. The major drawback of the technique is that the latency to the client is doubled for small requests. SWEB also uses a least load dispatclung algorithm.

The TCP routcr [9] employs a single 1P image to avoid requests reaching nodes directly. All requests are routed by a router, which then distributes them to a node based on load CharaCtenStiCS. The router changes the 1P destina- tion address when sending requests to the selected node. The node will directly return the document to the client without passing the dispatcher, and change the source address to the router address instead of its own address. However, because the address is changed at the TCPjlP layer, the kernel code of every node has to be modified. This problem i s solved by a later system called eNetwork Dispatcher [IO], which can configure all nodes to have a secondary IP address alias to the publicised site address. The prototype of eNetwork Dispatcher reported in [I21 handled over 500 HTTP hits per second with under 5% CPU utilisation and negligible additional latency overhead during the Atlanta 1996 Olympic Games. Both the TCP router and eNetwork Dispatcher use a least load dispatch- ing algorithm.

Non-feedback type algorithms distribute the requests to the nodes by only using the information given by the client requests. Netscape Navigator [I] uses a randomisation scheme inside itself to access the site home.netscape.com. When a Navigator accesses the site, it periodically queries the DNS for a hostnome of the form homeX.netscape.com, where X is a random number between 1 and 32. This technique is clearly not available to sites other than home.netscape.com, and it simply uses a random dispatch- ing algorithm.

223

Round Robin DNS [SI implements load balancing in the domain name to IP resolution proccss. When a client resolves a domain name, the DNS server selects the IP address of one of the nodes in a round-robin fashion. However; the client and the inmediatc name servers will cache the resolved IP and thus will weaken the round-robin effect. The dispatching algorithm used in Round Robin DNS is closed to a round robin one.

any incoming requests will not bypass the dispatcher to reach nodes directly. All requests are received by a router and routed to a single dispatcher. which redirects all requests to the different nodes based on the results of a hashing function on the client IP addresses. The effect of the hash function implies that a random dispatching algorithm is uscd in ONE-IP.

As can be seen from the above discussion. although non- feedback type algorithms arc casicr to implement, they will not give optimal dispatching perfonnance due to their lack of loading information of the nodcs. On the other hand fecdhack type algorithms can give much better pcrfor- mance: but have the drawbacks that loading is difficult to quantify and the transfer of the loading information introduces additional network latency. In this paper, we attempt to bring the best of both worlds by proposing a method to estimate thc load offered by a 1-equest by only using the request information. Based on this estimation. least load dispatching of requests can be performed without the need for the loading of the nodes to he known in advance.

2 Proposed dispatching algorithm

2. 7 Model for dispatching algorithm

ONE-IP [8] uses a single IP image to the clients so that ~

Consider a scalable Web site which consists of K nodes and one central dispatcher as shown in Fig. I . All requests from clients are qucued in the dispatcher before reaching the nodes. Each node will serve the Web client requests and transfer documents directly to thc clients. Assume that each of the K nodes can serve 111 requests simultaneously, or there are 111 service processes running in each node. Whcn the n~ processes are all busy in senicing requests, additional requests dispatched to the node will be queued in a FCFS queue. Our goal is that the nodc with the lightest load should be found for serving the incoming request. This can be done by dispatching a new requcst to node i, having a server process ; which will become idle first among all the ( K x P I I ) server processes of the system. In doing this, each

web site

@<>/ (node 2)

node K 0

request is guaranteed to he served at the earliest possible time by the system. In the following we will show how to find which process will becoinc idle first.

2.2 Load estimation The response time of a Web server consists of CPU time. disc 110 time and network delay. Since the response time mainly depends on the network delay for transferring a document through the Internet [13], we can use the transfer time of the document to estimate the load offered to a node.

To estimate the transfer time of a document d. denoted as T(d). we need to know its transfer size S(r/), and the transmission speed of the path betwecn the user and the site, “(d):

However; we cannot know the transfer size in advance because the user may stop the transmission at any moment. To estimate it, we can use the average transfer size of the document, A’(</). S(d) can bc easily obtained by scanning the access logs stored in the nodes. This operation can be perfonned regularly during non-busy working hours, say at midnight. Therefore, the overhead of this operation has minimal impact on system performance. For those docu- ments not logged in access log, we can use the file size of the document for this cstimation. The estimated transfer time of a document is therefore

--

-

On the other hand, the transmission speed can he obtained by using the command ping to calculate the transfer rate as mentioned in [14]. As can be seen from the simulation results given in this paper, using a fixed transfer rate will also give a very good load estimation. Therefore, the service time is only proportional to the average transfer size. That is.

T ( d ) cx S(d) ( 3 )

Lctf;, be the time when server processj in node i finishes serving all its jobs and become idle. where IsisK and 1 <,j<ii?. Note that Ji includes the time needed to senie some of the requests which are still in the queue of node i.J, can be obtained by

hi = T ( d ) (4) d t P , j

where P,/ is the set of documcnts which will be served by processj in node i. Note that each time a new request is dispatched to a node i. a particular process ; with the smallest,f;, will certainly serve this request after a time period off,. Thus. the f , of this particular process should be increased by an amount of time needed to scrvc the new request. The set Pg for this particular p,rocess should now also include one more clement, which is the new document being requested. The new .fij and P, can then be used for evaluating the next request arrived at the system. llieorei?i I : By dispatching every new request to node i, where min(fiI-fir, .._,jib ...-. f&) =.A the minimal average rcspond time for all served requests can he obtained. Prooj Let R = {r,, r2 , . . . , rN} be the set of requests which thc system serves in sequence. For each r,, E R, we denote its arrival time by r,z> its waiting time in the queue by W,, and

its sewice time by T,r Then, the average respond time, ART, can bc obtained by

Since C7;, is the same for any dispatching algorithms,

ART can he minimisid if tlie term C K, can he minimised

Suppose r,, is dispatched to pr0cess.j in node i, then

Kr = (.hi - t,) (6) Since the Icast-load dispatching algorithm always chooses i and j with the smallest/;,, W,, is smallest for r,, by (6). Since the tenn It$ is minimised. the average respond time

obtained is also minirnised. Fig. 2 shows that the 5th arrival, rs, arrives at the system

at 15. Since niin(/;i,h2,/;3rfi4) =,fil, rs will he dispatched to node I . After dispatching the request. fi will be adv;inced to a later time with an offset equal to the transfer time of rS. that is. cfi I),,,, = (1; + Ts.

5th arrival rs arrives at time ts

and will be dispatched to node 1

I

I

I

1 I * I 1. f12

node 1

3 Performance of dispatching algorithm

To study the performance of the proposed dispatching algorithm, a qucueing model similar to that in [6] is used. Fig. 3 shows the simulation model being used. As shown. each node is modelled to have a FCFS queue and 111 servers. The arrivals to the system are generated by the ~cccss logs from the NASA Kennedy Space Center Web server and the ClarkNet Web server. (Access lays are downloaded from [ 151). The statistics of the logs can be found in Table I . As discussed in Section 2, tlie service time of’ each request is modelled by the average transfer sue of the request over the transmission speed in serving the request. For the transmission speed, two differcnt simulations are run. In the first simulation, we assume a constant traiisinissioii speed (therefore the service time is only proportional to the average transfer size). In the second simulation, the transmission speed of a request is assumed to be either 2400 bit/s, 9600 bit/s. 14400 hit/s. or 28800bit/s. The prob- ability of choosing a particular transmission speed is based on the modem speed distribution of the access points of CompuServe in Japan.

IEE Pro<. Lbnerani.. Vo/. M Y , No 4, .4u!1,11 2002

I I I

Table 1:

Web server NASA ClarkNet

Statistics of access logs

Date

Total requests

Total HTML requests

Total image document requests

Total video and audio document requests

Total byte transferred

Total unique documents

6 July 1995

91282

29181 132%)

56983 462%)

2147 1<0.5%)

2.2GB 2148

30 August 1995

241256

46749 119%)

188246 178%) 511 1<0.5%)

2.4GB

12673

Fig. 4 shows the pel-fomiance of the least-load algorithm when a constant transmission s p e d is assumed. The maximum throughput of the least-load algorithm using the NASA log is about 8 requests/s and that using ClarkNet log is about 23 requests/s. They are 10% and 5% higher than those of the random and round-robin algorithms, respectively. Fig. 5 shows the performalice of the least-load algorithm when tlic transmission speed is varied. The maximum throughput of the least-load algorithm using the NASA log is about 3.4 request@ and that using ClarkNet log is about 8.5 requests/s. They are 10% and 9% higher than those of the random and round-robin algorithms, respectively. When we compare the results for constant and varied transniission speeds, the throughput for both logs is either the Same or increased. This is because the least-load algorithm also takes the transmission spced into account to calculate the load on each server node. However, the random and round-robin algorithms distribute the requests without considering the load information. We believe that if the range of the transmission speed varies to a larger extent, the perfomiance gain will be more significant.

4 Conclusions

We have proposed in this paper a least load dispatching algorithm for parallel Web server nodes. The algorithm achieves even request dispatching by using a novel method in load estimation. Simulation based on real access logs shows that the proposed algorithm gives 10% higher throughput than those of the round-robin and random

225

12

11

10

'99

' 6

- 8

7

6

5

+ + 4 t o + i

o round robin + least-load

6 -

h I 6 +

a t

V + + + * e c s f * +

I 3 4 5 6 7 6

request arrival rate. requestis a

6 I

' l

2 1 ' " ' I 4 6 12 16 20 24

request arrival rate. iequeslb

b

Fig. 4 Peforiiuince mnpori.sm wirh conslant truntsvtissiorr speed a NASA Web scrver b ClarkNet Web sewer

algorithms. Since the proposed algorithm does not need to know the loading information of each node, it does not have the drawbacks of feedback-type algorithms.

5 Acknowledgments

The work described in this paper was partially supported by a grant from the Research Grants Council of the Hong Kong Special Administrative Region. China (Project No. CityU 1098/99E). This work was also partially supported by a grant nos (7100069) from City University.

6 References

I MOSEDALE, D.. FOSS. W.. and MCCOOL. R.: 'Lessons leamed administering Netscape's Internet site', IEEE lnirrnei C<mpui., 1997. 1. (2). pp. 28-35 CARDELLINI, V., COLAJANNI. M.. and YU. P.S.: 'Dynamic load balancing on web-rener systems', IEEE Inrcrriei Cmtpei.. 1999.3, (3, pp. 28-39 SCHROEDER. T.. GODDARD, S.. and RAMAMURTHY, 8.: 'Scalable web SBT\TT clustering technolagjes', IEEE N<mr, 2000. 14. (3). pp. 3845

4 ANDRESEN. D.. YANG. T.. HOLMEDAHL, V.. and IBARRA, O.H.: 'SWEB towards a icalablc World Wide Wcb server on multicomputers'. Proceedings of 10th lntemational conference on Parallcl processing systems.. 1996. pp. 85&856

2

3

226

28

26

24

01 22 g

2c

18

16

14

c round rob," + least-load

+ O +

+O + +o c *

+ + + + C

+ * C + -+

* & A *+* e * , + Q $ * + + ? + '

1.5 2.0 2.5 3.0 request arrival rate, requesth

a

16 I

0 round robin + least-load *+ 1 8

h

+ I 6 h

h + & *

" 1 2 3 4 6 6 7 8 9

request arrival rate, requestis

b

Fig. 5 n NASA Web sewer h ClarkNet Web sewer

Peifiwnunce conzpari.wn with caried tronsnission speed

5 KWAN. T.T.. McCRATH. R.E.. and REED. D.A.: 'NCSAs World Wide Wch rerver: design and pcrfomance', Compum. 1995. 28, ( I 1). nn 6%-ld ry. "" , ,

6 YEUNG. K.H.. and W E N . K.W.: 'Probability based rcplacement almrilhm for WWW sewer amilvs'. Proceedinas of international

'A scalable and highly available web ser\'cc'. Proceedings of COMPCON96. 1996. pp. 8>99? GOLDSZMIDT. G.S.: 'Load manaeement for scaling up Internet servims'. Proceedings OC IEEE nm&k operations and management symposium, 1998. pp. 828-835 BERNERS-LEE. T., FIELDING, R.. FRYSTKY. H.: 'Hypenext transfer protocol HTTP/I.O'. http://~\?u.ics.un.edu/pub/ietf/http/ &I945

I2 IBM eNetwork DisDatcher. h t t ~ : i l w ~ ~ . s o f t w a r e . i b m . ~ ~ m l ~ ~ ~ t w ~ ~ k l

I O

11

. . dispatcher/ DILLEY, J., FRIEDRICH. R.. JIN. T., and ROLIA, J . : 'Measure ment tools and modeling techniques for evaluating web server oerformance'. Procmlincs of 9th inlrmtional conference on

13

Computer performance e&luev;lhation: modeling techniques and tools. SI. Malo. France. 1997. pp. 15S168

14 STEVENS. W.R.: 'TCP/IP illustraled volume 3' (Addison-Wesley.