Group Ratio Round-Robin: O(1) Proportional Share ... · Bogdan Caprita, Wong Chun Chan, Jason Nieh, Clifford Stein∗, and Haoqiang Zheng Department of Computer Science Columbia University

Group Ratio Round-Robin: O(1) Proportional Share Schedulingfor Uniprocessor and Multiprocessor Systems

Bogdan Caprita, Wong Chun Chan, Jason Nieh, Clifford Stein∗, and Haoqiang ZhengDepartment of Computer Science

Columbia UniversityEmail: {bc2008, wc164, nieh, cliff, hzheng}@cs.columbia.edu

Abstract

We present Group Ratio Round-Robin (GR3), the first pro-portional share scheduler that combines accurate propor-tional fairness scheduling behavior with O(1) schedulingoverhead on both uniprocessor and multiprocessor systems.GR3 uses a simple grouping strategy to organize clientsinto groups of similar processor allocations which can bemore easily scheduled. Using this strategy, GR3 combinesthe benefits of low overhead round-robin execution with anovel ratio-based scheduling algorithm. GR3 introduces anovel frontlog mechanism and weight readjustment algo-rithm to operate effectively on multiprocessors. GR3 pro-vides fairness within a constant factor of the ideal general-ized processor sharing model for client weights with a fixedupper bound and preserves its fairness properties on multi-processor systems. We have implemented GR3 in Linuxand measured its performance. Our experimental resultsshow that GR3 provides much lower scheduling overheadand much better scheduling accuracy than other schedulerscommonly used in research and practice.

1 IntroductionProportional share resource management provides a flexibleand useful abstraction for multiplexing processor resourcesamong a set of clients. Proportional share scheduling has aclear colloquial meaning: given a set of clients with asso-ciated weights, a proportional share scheduler should allo-cate resources to each client in proportion to its respectiveweight. However, developing processor scheduling mech-anisms that combine good proportional fairness schedulingbehavior with low scheduling overhead has been difficult toachieve in practice. For many proportional share schedulingmechanisms, the time to select a client for execution growswith the number of clients. For server systems which mayservice large numbers of clients, the scheduling overhead ofalgorithms whose complexity grows linearly with the num-ber of clients can waste more than 20 percent of system re-sources [3] for large numbers of clients. Furthermore, little

∗also in Department of IEOR

work has been done to provide proportional share schedul-ing on multiprocessor systems, which are increasingly com-mon especially in small-scale configurations with two orfour processors. Over the years, a number of schedulingmechanisms have been proposed, and much progress hasbeen made. However, previous mechanisms have either su-perconstant overhead or less-than-ideal fairness properties.

We introduce Group Ratio Round-Robin (GR3), thefirst proportional share scheduler that provides constantfairness bounds on proportional sharing accuracy with O(1)scheduling overhead for both uniprocessor and small-scalemultiprocessor systems. In designing GR3, we observedthat accurate, low-overhead proportional sharing is easy toachieve when scheduling a set of clients with equal pro-cessor allocations, but is harder to do when clients requirevery different allocations. Based on this observation, GR3

uses a simple client grouping strategy to organize clientsinto groups of similar processor allocations which can bemore easily scheduled. Using this grouping strategy, GR3

combines the benefits of low overhead round-robin execu-tion with a novel ratio-based scheduling algorithm.

GR3 uses the same basic uniprocessor scheduling al-gorithm for multiprocessor scheduling by introducing thenotion of a frontlog. On a multiprocessor system, a clientmay not be able to be scheduled to run on a processor be-cause it is currently running on another processor. To pre-serve its fairness properties, GR3 keeps track of a frontlogper client to indicate when the client was already runningbut could have been scheduled to run on another processor.It then assigns the client a time quantum that is added to itsallocation on the processor it is running on. The frontlogensures that a client receives its proportional share alloca-tion while also taking advantage of any cache affinity bycontinuing to run the client on the same processor.

GR3 provides a simple weight readjustment algo-rithm that takes advantage of its grouping strategy. Ona multiprocessor system, proportional sharing is not fea-sible for some client weight assignments, such as havingone client with weight 1 and another with weight 2 on a

2005 USENIX Annual Technical Conference USENIX Association 337

two-processor system. By organizing clients with similarweights into groups, GR3 adjusts for infeasible weight as-signments without the need to order clients, resulting inlower scheduling complexity than previous approaches [7].

We have analyzed GR3 and show that with only O(1)overhead, GR3 provides fairness within O(g2) of the idealGeneralized Processor Sharing (GPS) model [16], where g,the number of groups, grows at worst logarithmically withthe largest client weight. Since g is in practice a small con-stant, GR3 effectively provides constant fairness boundswith only O(1) overhead. Moreover, we show that GR3

uniquely preserves its worst-case time complexity and fair-ness properties for multiprocessor systems.

We have implemented a prototype GR3 processorscheduler in Linux, and compared it against uniproces-sor and multiprocessor schedulers commonly used in prac-tice and research, including the standard Linux sched-uler [2], Weighted Fair Queueing (WFQ) [11], Virtual-TimeRound-Robin (VTRR) [17], and Smoothed Round-Robin(SRR) [9]. We have conducted both simulation studies andkernel measurements on micro-benchmarks and real ap-plications. Our results show that GR3 can provide morethan an order of magnitude better proportional sharing ac-curacy than these other schedulers, in some cases with morethan an order of magnitude less overhead. These resultsdemonstrate that GR3 can in practice deliver better propor-tional share control with lower scheduling overhead thanthese other approaches. Furthermore, GR3 is simple to im-plement and easy to incorporate into existing schedulingframeworks in commodity operating systems.

This paper presents the design, analysis, and evalua-tion of GR3. Section 2 describes the uniprocessor schedul-ing algorithm. Section 3 describes extensions for multipro-cessor scheduling, which we refer to as GR3MP . Section 4analyzes the fairness and complexity of GR3. Section 5presents experimental results. Section 6 discusses relatedwork. Finally, we present some concluding remarks anddirections for future work.

2 GR3 Uniprocessor SchedulingUniprocessor scheduling, the process of scheduling a time-multiplexed resource among a set of clients, has two basicsteps: 1) order the clients in a queue, 2) run the first client inthe queue for its time quantum, which is the maximum timeinterval the client is allowed to run before another schedul-ing decision is made. We refer to the units of time quantaas time units (tu) rather than an absolute time measure suchas seconds. A scheduler can therefore achieve proportionalsharing in one of two ways. One way, often called fairqueueing [11, 18, 28, 13, 24, 10] is to adjust the frequencythat a client is selected to run by adjusting the position ofthe client in the queue so that it ends up at the front of thequeue more or less often. However, adjusting the client’sposition in the queue typically requires sorting clients based

on some metric of fairness, and has a time complexity thatgrows with the number of clients. The other way is to ad-just the size of a client’s time quantum so that it runs longerfor a given allocation, as is done in weighted round-robin(WRR). This is fast, providing constant time complexityscheduling overhead. However, allowing a client to mo-nopolize the resource for a long period of time results in ex-tended periods of unfairness to other clients which receiveno service during those times. The unfairness is worse withskewed weight distributions.

GR3 is a proportional share scheduler that matcheswith O(1) time complexity of round-robin scheduling butprovides much better proportional fairness guarantees inpractice. At a high-level, the GR3 scheduling algorithmcan be briefly described in three parts:

1. Client grouping strategy: Clients are separated intogroups of clients with similar weight values. Thegroup of order k is assigned all clients with weightsbetween 2k to 2k+1 − 1, where k ≥ 0.

2. Intergroup scheduling: Groups are ordered in a listfrom largest to smallest group weight, where the groupweight of a group is the sum of the weights of allclients in the group. Groups are selected in a round-robin manner based on the ratio of their group weights.If a group has already been selected more than its pro-portional share of the time, move on to the next groupin the list. Otherwise, skip the remaining groups inthe group list and start selecting groups from the be-ginning of the group list again. Since the groups withlarger weights are placed first in the list, this allowsthem to get more service than the lower-weight groupsat the end of the list.

3. Intragroup scheduling: From the selected group, aclient is selected to run in a round-robin manner thataccounts for its weight and previous execution history.

Using this client grouping strategy, GR3 separatesscheduling in a way that reduces the need to schedule enti-ties with skewed weight distributions. The client groupingstrategy limits the number of groups that need to be sched-uled since the number of groups grows at worst logarithmi-cally with the largest client weight. Even a very large 32-bitclient weight would limit the number of groups to no morethan 32. The client grouping strategy also ensures that allclients within a group have weight within a factor of two.As a result, the intragroup scheduler never needs to sched-ule clients with skewed weight distributions. GR3 groupsare simple lists that do not need to be balanced; they do notrequire any use of more complex balanced tree structures.

2.1 GR3 Definitions

We now define the state GR3 associates with eachclient and group, and describe in detail how GR3 uses

2005 USENIX Annual Technical Conference USENIX Association338

Cj Client j. (also called ’task’ j)φC The weight assigned to client C.φj Shorthand notation for φCj

.DC The deficit of C.N The number of runnable clients.g The number of groups.Gi i’th group in the list ordered by weight.|G| The number of clients in group G.G(C) The group to which C belongs.ΦG The group weight of G:

∑C∈G φC .

Φi Shorthand notation for ΦGi.

σG The order of group G.φG

min Lower bound for client weights in G: 2σG .wC The work of client C.wj Shorthand notation for wCj

.WG The group work of group G.Wi Shorthand notation for WGi

.ΦT Total weight:

∑Nj=1 φj =

∑gi=1 Φi.

WT Total work:∑N

j=1 wj =∑g

i=1 Wi.

eC Service error of client C: wC −WTφC

ΦT

EG Group service error of G: WG −WTΦG

ΦT

eC,G Group-relative service error of client C withrespect to group G: wC −WG

φC

ΦG

Table 1: GR3 terminology

that state to schedule clients. Table 1 lists terminologywe use. For each client, GR3 maintains the followingthree values: weight, deficit, and run state. Each client re-ceives a resource allocation that is directly proportional toits weight. A client’s deficit tracks the number of remainingtime quanta the client has not received from previous allo-cations. A client’s run state indicates whether or not it canbe executed. A client is runnable if it can be executed.

For each group, GR3 maintains the following fourvalues: group weight, group order, group work, and currentclient. The group weight is the sum of the correspondingweights of the clients in the group run queue. A group withgroup order k contains the clients with weights between 2k

to 2k+1 − 1. The group work is the total execution timeclients in the group have received. The current client is themost recently scheduled client in the group’s run queue.

GR3 also maintains the following scheduler state:time quantum, group list, total weight, and current group.The group list is a sorted list of all groups containingrunnable clients ordered from largest to smallest groupweight, with ties broken by group order. The total weight isthe sum of the weights of all runnable clients. The currentgroup is the most recently selected group in the group list.

2.2 Basic GR3 Algorithm

We initially only consider runnable clients in our dis-cussion of the basic GR3 scheduling algorithm. We dis-

cuss dynamic changes in a client’s run state in Section 2.3.We first focus on the GR3 intergroup scheduling algorithm,then discuss the GR3 intragroup scheduling algorithm.

The GR3 intergroup scheduling algorithm uses theratio of the group weights of successive groups to deter-mine which group to select. The next group to scheduleis selected using only the state of successive groups in thegroup list. Given a group Gi whose weight is x times largerthan the group weight of the next group Gi+1 in the grouplist, GR3 will select group Gi x times for every time that itselects Gi+1 in the group list to provide proportional shareallocation among groups.

To implement the algorithm, GR3 maintains the to-tal work done by group Gi in a variable Wi. An index i totracks the current group and is initialized to 1. The schedul-ing algorithm then executes the following simple routine:

INTERGROUP-SCHEDULE()

1 C ← INTRAGROUP-SCHEDULE(Gi)2 Wi ←Wi + 1

3 if i < g and Wi+1Wi+1+1 > Φi

Φi+1(1)

4 then i← i + 15 else i← 16 return C

Let us negate (1) under the form:

Wi + 1

Φi≤

Wi+1 + 1

Φi+1(2)

We will call this relation the well-ordering condition of twoconsecutive groups. GR3 works to maintain this conditiontrue at all times. The intuition behind (2) is that we wouldlike the ratio of the work of Gi and Gi+1 to match the ratioof their respective group weights after GR3 has finished se-lecting both groups. Recall, Φi ≥ Φi+1. Each time a clientfrom Gi+1 is run, GR3 would like to have run Φi/Φi+1

worth of clients from Gi. (1) says that GR3 should not runa client from Gi and increment Gi’s group work if it willmake it impossible for Gi+1 to catch up to its proportionalshare allocation by running one of its clients once.

To illustrate how intergroup scheduling works, Fig-ure 1 shows an example with three clients C1, C2, andC3, which have weights of 5, 2, and 1, respectively. TheGR3 grouping strategy would place each Ci in group Gi,ordering the groups by weight: G1, G2, and G3 have or-ders 2, 1 and 0 and weights of 5, 2, and 1 respectively.In this example, each group has only one client so thereis no intragroup scheduling. GR3 would start by selectinggroup G1, running client C1, and incrementing W1. Basedon (1), W1+1

W2+1 = 2 < Φ1

Φ2= 2.5, so GR3 would select

G1 again and run client C1. After running C1, G1’s workwould be 2 so that the inequality in (1) would hold and GR3

would then move on to the next group G2 and run client C2.Based on (1), W2+1

W3+1 = 2 ≤ Φ2

Φ3= 2, so GR3 would reset


1C 2C1C 1C 1C 1C 2C 3C

G1G

G2

3

2 | 1

1 | 1

5 | 1 5 | 2

2 | 1

1 | 1

5 | 3

1 | 1 1 | 1 1 | 1 1 | 1 1 | 1

2 | 2 2 | 2 2 | 22 | 1 2 | 2 2 | 3

5 | 55 | 45 | 3 5 | 6 5 | 6

1 | 1

t

Figure 1: GR3 intergroup scheduling. At each time step,the shaded box contains the pair ΦG |WG +1 for the groupG before it is selected.

the current group to the largest weight group G1 and runclient C1. Based on (1), C1 would be run for three timequanta before selecting G2 again to run client C2. Afterrunning C2 the second time, W2 would increase such thatW2+1W3+1 = 3 > Φ2

Φ3= 2, so GR3 would then move on to

the last group G3 and run client C3. The resulting schedulewould then be: G1, G1, G2, G1, G1, G1, G2, G3. Eachgroup therefore receives its proportional allocation in ac-cordance with its respective group weight.

The GR3 intragroup scheduling algorithm selects aclient from the selected group. All clients within a grouphave weights within a factor of two, and all client weightsin a group G are normalized with respect to the minimumpossible weight, φG

min = 2σG , for any client in the group.GR3 then effectively traverses through a group’s queuein round-robin order, allocating each client its normalizedweight worth of time quanta. GR3 keeps track of subuni-tary fractional time quanta that cannot be used and accumu-lates them in a deficit value for each client. Hence, eachclient is assigned either one or two time quanta, based onthe client’s normalized weight and its previous allocation.

More specifically, the GR3 intragroup scheduler con-siders the scheduling of clients in rounds. A round is onepass through a group G’s run queue of clients from begin-ning to end. The group run queue does not need to be sortedin any manner. During each round, the GR3 intragroup al-gorithm considers the clients in round-robin order and exe-cutes the following simple routine:

INTRAGROUP-SCHEDULE(G)

1 C ← G[k] � k is the current position in the round2 if DC < 13 then k ← (k + 1) mod |G|4 C ← G[k]5 DC ← DC + φC/φG

min

6 DC ← DC − 17 return C

For each runnable client C, the scheduler determinesthe maximum number of time quanta that the client can beselected to run in this round as b φC

φGmin

+DC(r−1)c. DC(r),

the deficit of client C after round r, is the time quantumfraction left over after round r: DC(r) = φC

φGmin

+ DC(r −

1)−b φC

φGmin

+DC(r−1)c, with DC(0) = φC

φGmin

. Thus, in each

round, C is allotted one time quantum plus any additionalleftover from the previous round, and DC(r) keeps track ofthe amount of service that C missed because of roundingdown its allocation to whole time quanta. We observe that0 ≤ DC(r) < 1 after any round r so that any client Cwill be allotted one or two time quanta. Note that if a clientis allotted two time quanta, it first executes for one timequantum and then executes for the second time quantumthe next time the intergroup scheduler selects its respectivegroup again (in general, following a timespan when clientsbelonging to other groups get to run).

To illustrate how GR3 works with intragroupscheduling, Figure 2 shows an example with six clientsC1 through C6 with weights 12, 3, 3, 2, 2, and 2, re-spectively. The six clients will be put in two groups G1

and G2 with respective group order 1 and 3 as follows:G1 = {C2, C3, C4, C5, C6} and G2 = {C1}. The weightof the groups are Φ1 = Φ2 = 12. GR3 intergroup schedul-ing will consider the groups in this order: G1, G2, G1,G2, G1, G2, G1, G2, G1, G2, G1, G2. G2 will sched-ule client C1 every time G2 is considered for service sinceit has only one client. Since φG1

min = 2, the normalizedweights of clients C2, C3, C4, C5, and C6 are 1.5, 1.5, 1,1, and 1, respectively. In the beginning of round 1 in G1,each client starts with 0 deficit. As a result, the intragroupscheduler will run each client in G1 for one time quantumduring round 1. After the first round, the deficit for C2, C3,C4, C5, and C6 are 0.5, 0.5, 0, 0, and 0. In the beginning ofround 2, each client gets another φi/φG1

min allocation, plusany deficit from the first round. As a result, the intragroupscheduler will select clients C2, C3, C4, C5, and C6 to runin order for 2, 2, 1, 1, and 1 time quanta, respectively, dur-ing round 2. The resulting schedule would then be: C2, C1,C3, C1, C4, C1, C5, C1, C6, C1, C2, C1, C2, C1, C3, C1,C3, C1, C4, C1, C5, C1, C6, C1.

2.3 GR3 Dynamic Considerations

We now discuss how GR3 allows clients to bedynamically created, terminated, or change run state.Runnable clients can be selected for execution by the sched-uler, while clients that are not runnable cannot. With no lossof generality, we assume that a client is created before it canbecome runnable, and a client becomes not runnable beforeit is terminated. As a result, client creation and terminationhave no effect on the GR3 run queues.

When a client C with weight φC becomes runnable,it is inserted into group G = G(C) such that φC is between2σG and 2σG+1 − 1. If the group was previously empty, anew group is created, the client becomes the current clientof the group, and g, the number of groups, is incremented.If the group was not previously empty, GR3 inserts theclient into the respective group’s run queue right before thecurrent client; it will be serviced after all of the other clients


G1

000

00

G1

0

0

0

G1

0

0.50.50

G1

0

0.50.500 0

G1

0.50.50

000

00.50.5

2C 3C1C 1C 4C 1C 5C 1C 6C 1C

0

G1

0

00.51

2C 1C 2C 1C

0

G1

0

0

0

0

G1

0

010

0.5

3C 1C 3C 1C 4C 1C 5C 1C 6C 1C

0

G1

0

G1

00

G1

0

00

00

00

0

00

0

G20

G1G20

G20

G20

G20

G20

G20

G20

G20

G20

G20

G200

0000

0.5

t

0 0 0 0 0 0 0 0 0 0 0 00

00

0

00

0

Figure 2: GR3 intragroup scheduling. At each time step, the shaded box contains the deficit of the client before it is run.

in the group have first been considered for scheduling. Theinitial deficit DC will be initialized to 0.

When a newly runnable client C is inserted into itsrespective group G, the group needs to be moved to its newposition on the ordered group list based on its new groupweight. Let this new position be k. The correspondinggroup work and group weight of G need to be updated andthe client’s deficit needs to be initialized. The group weightis simply incremented by the client’s weight. We also wantto scale the group work of G such that the work ratio ofconsecutive groups will continue to be proportional to theirweight ratio:

WG =

8

>

<

>

:

j

(Wk+1 + 1) ΦG

Φk+1

k

− 1 if k < g

l

(Wg−1 + 1) ΦG

Φg−1

m

− 1 if k = g

We will motivate these equations when analyzing the fair-ness of the algorithm in Section 4, but intuitively, we wantto preserve the invariants that result from (2).

When a client C with weight φC becomes notrunnable, we need to remove it from the group’s run queue.This requires updating the group’s weight, which poten-tially includes moving the group in the ordered group list, aswell as adjusting the measure of work received according tothe new processor share of the group. This can be achievedin several ways. GR3 is optimized to efficiently deal withthe common situation when a blocked client may rapidlyswitch back to the runnable state again. This approach isbased on “lazy” removal, which minimizes overhead asso-ciated with adding and removing a client, while at the sametime preserving the service rights and service order of therunnable clients. Since a client blocks when it is running,we know that it will take another full intragroup round be-fore the client will be considered again. The only actionwhen a client blocks is to set a flag on the client, marking itfor removal. If the client becomes runnable by the next timeit is selected, we reset the flag and run the client as usual.Otherwise, we remove the client from G(C). In the lat-ter situation, as in the case of client arrivals, the group mayneed to be moved to a new position on the ordered group listbased on its new group weight. The corresponding groupweight is updated by subtracting the client’s weight fromthe group weight. The corresponding group work is scaled

by the same rules as for client insertion, depending on thenew position of the group and its next neighbor. After per-forming these removal operations, GR3 resumes schedul-ing from the largest weight group in the system.

Whenever a client C blocks during round r, we setDC(r) = min(DC(r−1)+φC/φ

G(C)min −dwe, 1), where w

is the service that the client received during round r until itblocked. This preserves the client’s credit in case it returnsby the next round, while also limiting the deficit to 1 so thata client cannot gain credit by blocking. However, the groupconsumes 1 tu (its work is incremented) no matter how longthe client runs. Therefore, the client forfeits its extra creditwhenever it is unable to consume its allocation.

If the client fails to return by the next round, we mayremove it. Having kept the weight of the group to the oldvalue for an extra round has no adverse effects on fairness,despite the slight increase in service seen by the group dur-ing the last round. By scaling the work of the group androunding up, we determine its future allocation and thusmake sure the group will not have received undue service.We also immediately resume the scheduler from the first(largest) group in the readjusted group list, so that any mi-nor discrepancies caused by rounding may be smoothed outby a first pass through the group list.

3 GR3 Multiprocessor Extensions (GR3MP )We now present extensions to GR3 for scheduling a P -waymultiprocessor system from a single, centralized queue.This simple scheme, which we refer to as GR3MP , pre-serves the good fairness and time complexity properties ofGR3 in small-scale multiprocessor systems, which are in-creasingly common today, even in the form of multi-coreprocessors. We first describe the basic GR3MP schedul-ing algorithm, then discuss dynamic considerations. Table2 lists terminology we use. To deal with the problem of in-feasible client weights, we then show how GR3MP uses itsgrouping strategy in a novel weight readjustment algorithm.

3.1 Basic GR3MP Algorithm

GR3MP uses the same GR3 data structure, namelyan ordered list of groups, each containing clients whoseweights are within a factor of two from each other. When a


P Number of processors.℘k Processor k.C(℘) Client running on processor ℘.FC Frontlog for client C.

Table 2: GR3MP terminology

processor needs to be scheduled, GR3MP selects the clientthat would run next under GR3, essentially scheduling mul-tiple processors from its central run queue as GR3 sched-ules a single processor. However, there is one obstacle tosimply applying a uniprocessor algorithm on a multipro-cessor system. Each client can only run on one processorat any given time. As a result, GR3MP cannot select aclient to run that is already running on another processoreven if GR3 would schedule that client in the uniproces-sor case. For example, if GR3 would schedule the sameclient consecutively, GR3MP cannot schedule that clientconsecutively on another processor if it is still running.

To handle this situation while maintaining fairness,GR3MP introduces the notion of a frontlog. The front-log FC for some client C running on a processor ℘k (C =C(℘k)) is defined as the number of time quanta for C accu-mulated as C gets selected by GR3 and cannot run becauseit is already running on ℘k. The frontlog FC is then queuedup on ℘k.

Given a client that would be scheduled by GR3 butis already running on another processor, GR3MP uses thefrontlog to assign the client a time quantum now but de-fer the client’s use of it until later. Whenever a proces-sor finishes running a client for a time quantum, GR3MPchecks whether the client has a non-zero frontlog, and, ifso, continues running the client for another time quantumand decrements its frontlog by one, without consulting thecentral queue. The frontlog mechanism not only ensuresthat a client receives its proportional share allocation, it alsotakes advantage of any cache affinity by continuing to runthe client on the same processor.

When a processor finishes running a client for a timequantum and its frontlog is zero, we call the processor idle.GR3MP schedules a client to run on the idle processorby performing a GR3 scheduling decision on the centralqueue. If the selected client is already running on someother processor, we increase its frontlog and repeat the GR3

scheduling, each time incrementing the frontlog of the se-lected client, until we find a client that is not currently run-ning. We assign this client to the idle processor for one timequantum. This description assumes that there are least P+1clients in the system. Otherwise, scheduling is easy: an idleprocessor will either run the client it just ran, or idles un-til more clients arrive. In effect, each client will simply beassigned its own processor. Whenever a processor needs toperform a scheduling decision, it thus executes the follow-ing routine:

1C 1C 1C 1C 1C 1C

FC1=0 FC1

=0

t

2C 2C 3C 2C 2C 3C t

1C 2C 1C 1C C 32C1C 2C 1C 1C C 32C

FC1=1 FC1

=1p

1

p2

Figure 3: GR3 multiprocessor scheduling. The two pro-cessors schedule either from the central queue, or use thefrontlog mechanism when the task is already running.

MP-SCHEDULE(℘k)

1 C ← C(℘k) � Client just run2 if C = NIL

3 then if N < P4 then return NIL � Idle5 else if FC > 06 then FC ← FC − 17 return C8 C ← INTERGROUP-SCHEDULE()9 while ∃℘ s.t. C = C(℘)

10 do FC ← FC + 111 C ← INTERGROUP-SCHEDULE()12 return C

To illustrate GR3MP scheduling, Figure 3 shows anexample on a dual-processor system with three clients C1,C2, and C3 of weights 3, 2, and 1, respectively. C1 andC2 will then be part of the order 1 group (assume C2 isbefore C1 in the round-robin queue of this group), whereasC3 is part of the order 0 group. The GR3 schedule is C2,C1, C2, C1, C1, C3. ℘1 will then select C2 to run, and ℘2

selects C1. When ℘1 finishes, according to GR3, it willselect C2 once more, whereas ℘2 selects C1 again. When℘1 again selects the next GR3 client, which is C1, it findsthat it is already running on ℘2 and thus we set FC1

= 1and select the next client, which is C3, to run on ℘1. When℘2 finishes running C1 for its second time quantum, it findsFC1

= 1, sets FC1= 0 and continues running C1 without

any scheduling decision on the GR3 queue.

3.2 GR3MP Dynamic Considerations

GR3MP basically does the same thing as the GR3

algorithm under dynamic considerations. However, thefrontlogs used in GR3MP need to be accounted for ap-propriately. If some processors have long frontlogs for theircurrently running clients, newly arriving clients may not berun by those processors until their frontlogs are processed,resulting in bad responsiveness for the new clients. Al-though in between any two client arrivals or departures,some processors must have no frontlog, the set of such


processors can be as small as a single processor. In thiscase, newly arrived clients will end up competing with otherclients already in the run queue only for those few proces-sors, until the frontlog on the other processors is exhausted.

GR3MP provides fair and responsive allocations bycreating frontlogs for newly arriving clients. Each newclient is assigned a frontlog equal to a fraction of the to-tal current frontlog in the system based on its proportionalshare. Each processor now maintains a queue of frontlogclients and a new client with a frontlog is immediately as-signed to one of the processor frontlog queues. Rather thanrunning its currently running client until it completes itsfrontlog, each processor now round robins among clientsin its frontlog queue. Given that frontlogs are small in prac-tice, round-robin scheduling is used for frontlog clients forits simplicity and fairness. GR3MP balances the frontlogload on the processors by placing new frontlog clients onthe processor with the smallest frontlog summed across allits frontlog clients.

More precisely, whenever a client C arrives, andit belongs in group G(C), GR3MP performs the samegroup operations as in the single processor GR3 algorithm.GR3MP finds the processor ℘k with the smallest front-log, then creates a frontlog for client C on ℘k of lengthFC = FT

φC

ΦT, where FT is the total frontlog on all the

processors. Let C′ = C(℘k). Then, assuming no furtherclients arrive, ℘k will round-robin between C and C′ andrun C for FC and C′ for FC′ time quanta.

When a client becomes not runnable, GR3MP usesthe same lazy removal mechanism used in GR3. If it isremoved from the run queue and has a frontlog, GR3MPsimply discards it since each client is assigned a frontlogbased on the current state of the system when it becomesrunnable again.

3.3 GR3MP Weight Readjustment

Since no client can run on more than one processorat a time, no client can consume more than a 1/P fractionof the processing in a multiprocessor system. A client Cwith weight φC greater than ΦT /P is considered infeasi-ble since it cannot receive its proportional share allocationφC/ΦT without using more than one processor simultane-ously. GR3MP should then give the client its maximumpossible service, and simply assign such a client its ownprocessor to run on. However, since the scheduler usesclient weights to determine which client to run, an infea-sible client’s weight must be adjusted so that it is feasi-ble to ensure that the scheduling algorithm runs correctlyto preserve fairness (assuming there are at least P clients).GR3MP potentially needs to perform weight readjustmentwhenever a client is inserted or removed from the run queueto make sure that all weights are feasible.

To understand the problem of weight readjustment,consider the sequence of all clients, ordered by weight:

S1,N = C1, C2, . . . , CN with φ1 ≥ φ2 ≥ . . . ≥ φN .We call the subsequence Sk,N = Ck, Ck+1, . . . , .CN Q-feasible, if φk ≤

1Q

∑Nj=k φj .

Lemma 1. The client mix in the system is feasible if andonly if S1,N is P -feasible.

Proof. If φ1 > ΦT

P , C1 is infeasible, so the mix is infea-sible. Conversely, if φ1 ≤

ΦT

P , then for any client Cl,φl ≤ φ1 ≤

ΦT

P , implying all clients are feasible. The mix

is then feasible ⇐⇒ φ1 ≤ΦT

P = 1P

∑Nj=1 φj , or, equiva-

lently, S1,N is P -feasible.

Lemma 2. Sk,N is Q-feasible ⇒ Sk+1,N is (Q − 1)-feasible.

Proof. φk ≤ 1Q

∑Nj=k φj ⇐⇒ Qφk ≤ φk +

∑Nj=k+1 φj ⇐⇒ φk ≤

1Q−1

∑Nj=k+1 φj . Since φk+1 ≤

φk, the lemma follows.

The feasibility problem is then to identify the leastk (denoted the feasibility threshold, f ) such that Sk,N is(P − k + 1)-feasible. If f = 1, then the client mix is feasi-ble. Otherwise, the infeasible set S1,f−1 = C1, . . . , Cf−1

contains the infeasible clients, whose weight needs to bescaled down to 1/P of the resulting total weight. The car-dinality f − 1 of the infeasible set is less than P . However,the sorted sequence S1,N is expensive to maintain, such thattraversing it and identifying the feasibility threshold is notan efficient solution.

GR3MP leverages its grouping strategy to performfast weight readjustment. GR3MP starts with the unmod-ified client weights, finds the set I of infeasible clients, andadjust their weights to be feasible. To construct I , the al-gorithm traverses the list of groups in decreasing order oftheir group order σG, until it finds a group not all of whoseclients are infeasible. We denote by |I| the cardinality of Iand by ΦI the sum of weights of the clients in I ,

∑C∈I φC .

The GR3MP weight readjustment algorithm is as follows:

WEIGHT-READJUSTMENT()

1 RESTORE-ORIGINAL-WEIGHTS

2 I ← ∅3 G← greatest order group4 while |G| < P − |I| and 2σG > ΦT −ΦI−ΦG

P−|I|−|G|

5 do I ← I ∪G6 G← next(G) � by group order7 if |G| < 2(P − |I|)8 then I ← I ∪ INFEASIBLE(G, P − |I|, ΦT − ΦI)

9 ΦfT ← ΦT − ΦI

10 ΦT ←P

P−|I|ΦfT

11 for each C ∈ I

12 do φC ←ΦT

P


The correctness of the algorithm is based on Lemma2. Let some group G span the subsequence Si,j of the se-quence of ordered clients S1,N . Then 2σG+1 − 1 ≥ φi ≥. . . ≥ φj ≥ 2σG and it is easy to show:

• 2σG > ΦT −ΦI−ΦG

P−|I|−|G| ⇒ j < f (all clients in S1,j areinfeasible).

• 2σG ≤ ΦT −ΦI−ΦG

P−|I|−|G| ⇒ j+1 ≥ f (all clients in Sj+1,N

are feasible).

Once we reach line 7, we know Sj+1,N is (P − j)-feasible,and i ≤ f ≤ j + 1. If |G| ≥ 2(P − |I|), GR3MP canstop searching for infeasible clients since all clients C ∈ Gare feasible, and f = i (equivalently, Si,N is (P − |I|)-feasible): φC < 2σG+1 ≤ 2 1

|G|ΦG ≤ 1P−|I|ΦG ≤

1P−|I|(ΦT − ΦI). Otherwise, if |G| < 2(P − |I|), then

i < f ≤ j + 1 and GR3MP needs to search through Gto determine which clients are infeasible (equivalently, findf ). Since the number of clients in G is small, we can sortall clients in G by weight. Then, starting from the largestweight client in G, find the first feasible client. A simplealgorithm is then the following:

INFEASIBLE(G, Q, Φ)

1 I ← ∅2 for each C ∈ G in sorted order3 do if φC > 1

Q−|I|(Φ− ΦI)

4 then I ← I ∪ {C}5 else return I6 return I

GR3MP can alternatively use a more complicatedbut lower time complexity divide-and-conquer algorithm tofind the infeasible clients in G. In this case, GR3MP par-titions G around its median C̄ into GS , the set of G clientsthat have weight less than φC̄ and GB , the set of G clientsthat have weight larger than φC̄ . By Lemma 2, if C̄ is fea-sible, GS ∪ {C̄} is feasible, and we recurse on GB . Other-wise, all clients in GB ∪{C̄} are infeasible, and we recurseon GS to find all infeasible clients. The algorithm finisheswhen the set we need to recurse on is empty:

INFEASIBLE(G, Q, Φ)

1 if G = ∅2 then return ∅3 C̄ ← MEDIAN(G)4 (GS , GB)← PARTITION(G, φC̄)

5 if φC̄ >Φ−ΦGB

Q−|GB |

6 then return GB ∪ {C̄}∪INFEASIBLE(GS , Q− |GB| − 1, Φ− ΦGB

− φC̄)7 else return INFEASIBLE(GB , Q, Φ)

Once all infeasible clients have been identified,WEIGHT-READJUSTMENT() determines the sum of the

weights of all feasible clients, ΦfT = ΦT − ΦI . We can

now compute the new total weight in the system as ΦT =P

P−|I|ΦfT , namely the solution to the equation Φf

T + |I| xP =

x. Once we have the adjusted ΦT , we change all the weightsfor the infeasible clients in I to ΦT

P . Lemma 6 in Section 4.2shows the readjustment algorithm runs in time O(P ) and isthus asymptotically optimal, since there can be Θ(P ) infea-sible clients.

4 GR3 Fairness and ComplexityWe analyze the fairness and complexity of GR3 andGR3MP . To analyze fairness, we use a more formal notionof proportional fairness defined as service error, a measurewidely used [1, 7, 9, 17, 18, 19, 25, 27] in the analysis ofscheduling algorithms. To simplify the analysis, we willassume that clients are always runnable and derive fairnessbounds for such a case. Subsequently, we address the im-pact of arrivals and departures.

We use a strict measure of service error (equivalentin this context to the Normalized Worst-case Fair Index [1])relative to Generalized Processor Sharing (GPS) [16], anidealized model that achieves perfect fairness: wC =WT

φC

ΦT, an ideal state in which each client C always re-

ceives service exactly proportional to its weight. Althoughall real-world schedulers must time-multiplex resources intime units of finite size and thus cannot maintain perfectfairness, some algorithms stay closer to perfect fairness thanothers and therefore have less service error. We quantifyhow close an algorithm gets to perfect fairness using theclient service time error, which is the difference betweenthe service received by client C and its share of the totalwork done by the processor: eC = wC −WT

φC

ΦT. A pos-

itive service time error indicates that a client has receivedmore than its ideal share over a time interval; a negativeerror indicates that it has received less. To be precise, theerror eC measures how much time a client C has receivedbeyond its ideal allocation. A proportional share schedulershould minimize the absolute value of the allocation errorof all clients with minimal scheduling overhead.

We provide bounds on the service error of GR3 andGR3MP . To do this, we define two other measures of ser-vice error. The group service time error is a similar mea-sure for groups that quantifies the fairness of allocating theprocessor among groups: EG = WG−WT

ΦG

ΦT. The group-

relative service time error represents the service time errorof client C if there were only a single group G = G(C)in the scheduler and is a measure of the service error of aclient with respect to the work done on behalf of its group:eC,G = wC −WG

φC

ΦG. We first show bounds on the group

service error of the intergroup scheduling algorithm. Wethen show bounds on the group-relative service error of theintragroup scheduling algorithm. We combine these resultsto obtain the overall client service error bounds. We alsodiscuss the scheduling overhead of GR3 and GR3MP in


terms of their time complexity. We show that both algo-rithms can make scheduling decisions in O(1) time withO(1) service error given a constant number of groups. Dueto space constraints, most of the proofs are omitted. Furtherproof details are available in [5].

4.1 Analysis of GR3

Intergroup Fairness For the case when the weight ratiosof consecutive groups in the group list are integers, we getthe following:

Lemma 3. If Φj

Φj+1∈ N, 1 ≤ j < g, then −1 < EGk

≤

(g − k) Φk

ΦTfor any group Gk.

Proof sketch: If the group currently scheduled is Gk,then the work to weight ratio of all groups Gj , j < k, isthe same. For j > k, Wj+1

Φj+1≤ Wj

Φj≤ Wj+1+1

Φj+1− 1

Φjas a

consequence of the well-ordering condition (2). After somerearrangements, we can sum over all j and bound Wk, andthus EGk

above and below. The additive 1Φj

will cause theg − 1 upper bound.

In the general case, we get similar, but slightly weakerbounds.

Lemma 4. For any group Gk, − (g−k)(g−k−1)2

Φk

ΦT− 1 <

EGk< g − 1.

The proof for this case (omitted) follows reasoning similarto that of the previous lemma, but with several additionalcomplications.

It is clear that the lower bound is minimized whensetting k = 1. Thus, we have

Corollary 1. − (g−1)(g−2)2

ΦG

ΦT− 1 < EG < g − 1 for any

group G.

Intragroup Fairness Within a group, all weights arewithin a factor of two and the group-relative error is boundby a small constant. The only slightly subtle point is to dealwith fractional rounds.

Lemma 5. −3 < eC,G < 4 for any client C ∈ G.

Overall Fairness of GR3 Based on the identity eC =eC,G + φC

ΦGEG which holds for any group G and any client

C ∈ G, we can combine the inter- and intragroup analysesto bound the overall fairness of GR3.

Theorem 1. − (g−1)(g−2)2

φC

ΦT− 4 < eC < g + 3 for any

client C.

The negative error of GR3 is thus bounded by O(g2) andthe positive error by O(g). Recall, g, the number of groups,does not depend on the number of clients in the system.

Dynamic Fairness of GR3 We can consider a client ar-rival or removal as an operation where a group is first re-moved from the group list and added in a different placewith a different weight. We argue that fairness is pre-served by these operations: when group Gk is removed,then Gk−1, Gk, and Gk+1 were well-ordered as definedin (2), so after the removal, Gk−1 and Gk+1, now neigh-bors, will be well-ordered by transitivity. When a group,call it Gi+(1/2), is inserted between Gi and Gi+1, it can beproven that the work readjustment formula in Section 2.3ensures Gi+(1/2) and Gi+1 are well-ordered. In the case ofGi and Gi+(1/2), we can show that we can achieve well-ordering by running Gi+(1/2) at most one extra time. Thus,modulo this readjustment, the intragroup algorithm’s fair-ness bounds are preserved. An important property of ouralgorithm that follows is that the pairwise ratios of workof clients not part of the readjusted group will be unaf-fected. Since the intragroup algorithm has constant fairnessbounds, the disruption for the work received by clients in-side the adjusted group is only O(1).

Time Complexity GR3 manages to bound its service er-ror by O(g2) while maintaining a strict O(1) schedulingoverhead. The intergroup scheduler either selects the nextgroup in the list, or reverts to the first one, which takes con-stant time. The intragroup scheduler is even simpler, as itjust picks the next client to run from the unordered roundrobin list of the group. Adding and removing a client isworst-case O(g) when a group needs to be relocated in theordered list of groups. This could of course be done inO(log g) time (using binary search, for example), but thesmall value of g in practice does not justify a more compli-cated algorithm.

The space complexity of GR3 is O(g) + O(N) =O(N). The only additional data structure beyond the un-ordered lists of clients is an ordered list of length g to orga-nize the groups.

4.2 Analysis of GR3MP

Overall Fairness of GR3MP Given feasible clientweights after weight readjustment, the service error forGR3MP is bounded below by the GR3 error, and aboveby a bound which improves with more processors.

Theorem 2. − (g−1)(g−2)2

φC

ΦT− 4 < eC < 2g + 10 +

(g−1)(g−2)2P for any client C.

Time Complexity of GR3MP The frontlogs create anadditional complication when analyzing the time complex-ity of GR3MP . When an idle processor looks for its nextclient, it runs the simple O(1) GR3 algorithm to find aclient C. If C is not running on any other processor, weare done, but otherwise we place it on the frontlog and thenwe must rerun the GR3 algorithm until we find a client


that is not running on any other processor. Since for eachsuch client, we increase its allocation on the processor itruns, the amortized time complexity remains O(1). The up-per bound on the time that any single scheduling decisiontakes is given by the maximum length of any schedulingsequence of GR3 consisting of only some fixed subset ofP − 1 clients.

Theorem 3. The time complexity per scheduling decisionin GR3MP is bounded above by (g−k)(g−k+1)

2 + (k +1)(g − k + 1)P + 1 where 1 ≤ k ≤ g.

Thus, the length of any schedule consisting of at mostP − 1 clients is O(g2P ). Even when a processor has front-logs for several clients queued up on it, it will schedule inO(1) time, since it performs round-robin among the front-logged clients. Client arrivals and departures take O(g)time because of the need to readjust group weights in thesaved list of groups. Moreover, if we also need to usethe weight readjustment algorithm, we incur an additionalO(P ) overhead on client arrivals and departures.

Lemma 6. The complexity of the weight readjustment al-gorithm is O(P ).

Proof. Restoring the original weights will worst case toucha number of groups equal to the number of previously in-feasible clients, which is O(P ). Identifying the infeasibleclients involves iterating over at most P groups in decreas-ing sequence based on group order, as described in Sec-tion 3.3. For the last group considered, we only attemptto partition it into feasible and infeasible clients of its sizeis less than 2P . Since partitioning of a set can be done inlinear time, and we recurse on a subset half the size, thisoperation is O(P ) as well.

For small P , the O(P log(P )) sorting approach todetermine infeasible clients in the last group considered issimpler and in practice performs better than the O(P ) re-cursive partitioning. Finally, altering the active group struc-ture to reflect the new weights is a O(P + g) operation, astwo groups may need to be re-inserted in the ordered list ofgroups.

5 Measurements and ResultsWe have implemented GR3 uniprocessor and multiproces-sor schedulers in the Linux operating system and measuredtheir performance. We present some experimental dataquantitatively comparing GR3 performance against otherpopular scheduling approaches from both industrial prac-tice and research. We have conducted both extensive sim-ulation studies and detailed measurements of real kernelscheduler performance on real applications.

Section 5.1 presents simulation results comparingthe proportional sharing accuracy of GR3 and GR3MPagainst WRR, WFQ [18], SFQ [13], VTRR [17], and

SRR [9]. The simulator enabled us to isolate the impactof the scheduling algorithms themselves and examine thescheduling behavior of these different algorithms acrosshundreds of thousands of different combinations of clientswith different weight values.

Section 5.2 presents detailed measurements of realkernel scheduler performance by comparing our prototypeGR3 Linux implementation against the standard Linuxscheduler, a WFQ scheduler, and a VTRR scheduler. Theexperiments we have done quantify the scheduling over-head and proportional share allocation accuracy of theseschedulers in a real operating system environment under anumber of different workloads.

All our kernel scheduler measurements were per-formed on an IBM Netfinity 4500 system with one or two933 MHz Intel Pentium III CPUs, 512 MB RAM, and 9GB hard drive. The system was installed with the DebianGNU/Linux distribution version 3.0 and all schedulers wereimplemented using Linux kernel version 2.4.19. The mea-surements were done by using a minimally intrusive trac-ing facility that writes timestamped event identifiers intoa memory log and takes advantage of the high-resolutionclock cycle counter available with the Intel CPU, providingmeasurement resolution at the granularity of a few nanosec-onds. Getting a timestamp simply involved reading thehardware cycle counter register. We measured the times-tamp overhead to be roughly 35 ns per event.

The kernel scheduler measurements were performedon a fully functional system. All experiments were per-formed with all system functions running and the systemconnected to the network. At the same time, an effort wasmade to eliminate variations in the test environment to makethe experiments repeatable.

5.1 Simulation Studies

We built a scheduling simulator that measures theservice time error, described in Section 4, of a scheduleron a set of clients. The simulator takes four inputs, thescheduling algorithm, the number of clients N , the totalsum of weights ΦT , and the number of client-weight combi-nations. The simulator randomly assigns weights to clientsand scales the weights to ensure that they add up to ΦT .It then schedules the clients using the specified algorithmas a real scheduler would, assuming no client blocks, andtracks the resulting service time error. The simulator runsthe scheduler until the resulting schedule repeats, then com-putes the maximum (most positive) and minimum (mostnegative) service time error across the nonrepeating portionof the schedule for the given set of clients and weight as-signments. This process is repeated for the specified num-ber of client-weight combinations. We then compute themaximum service time error and minimum service time er-ror for the specified number of client-weight combinationsto obtain a “worst-case” error range.


2000 4000 6000 8000Number of clients

65000 130000

195000 260000

Sum of weights

-10000 0

10000 20000

Service Error

Figure 4: WRR error


65000 130000

195000 260000

Sum of weights

0 200 400 600 800

Service Error

Figure 5: WFQ error


65000 130000

195000 260000

Sum of weights

-800-600-400-200

0

Service Error

Figure 6: SFQ error


65000 130000

195000 260000

Sum of weights

-3000 0

3000 6000 9000

Service Error

Figure 7: VTRR error


65000 130000

195000 260000

Sum of weights

-400-200

0 200 400

Service Error

Figure 8: SRR error


65000 130000

195000 260000

Sum of weights

-2

0

2

Service Error

Figure 9: GR3 error


65000 130000

195000 260000

Sum of weights

-2

0

2

Service Error

Figure 10: GR3MP error


65000 130000

195000 260000

Sum of weights

0 2 4 6 8

10

Scheduling Decisions per Task Selection

Figure 11: GR3MP over-head

To measure proportional fairness accuracy, we ransimulations for each scheduling algorithm on 45 differentcombinations of N and ΦT (32 up to 8192 clients and16384 up to 262144 total weight, respectively). Since theproportional sharing accuracy of a scheduler is often mostclearly illustrated with skewed weight distributions, oneof the clients was given a weight equal to 10 percent ofΦT . All of the other clients were then randomly assignedweights to sum to the remaining 90 percent of ΦT . For eachpair (N, ΦT ), we ran 2500 client-weight combinations anddetermined the resulting worst-case error range.

The worst-case service time error ranges for WRR,WFQ, SFQ, VTRR, SRR, and GR3 with these skewedweight distributions are in Figures 4 to 9. Due to spaceconstraints, WF2Q error is not shown since the results sim-ply verify its known mathematical error bounds of −1 and1 tu. Each figure consists of a graph of the error range forthe respective scheduling algorithm. Each graph shows twosurfaces representing the maximum and minimum servicetime error as a function of N and ΦT for the same rangeof values of N and ΦT . Figure 4 shows WRR’s servicetime error is between −12067 tu and 23593 tu. Figure 5shows WFQ’s service time error is between −1 tu and 819tu, which is much less than WRR. Figure 6 shows SFQ’sservice time error is between −819 tu and 1 tu, which isalmost a mirror image of WFQ. Figure 7 shows VTRR’sservice error is between −2129 tu and 10079 tu. Figure 8shows SRR’s service error is between −369 tu and 369 tu.

In comparison, Figure 9 shows the service time er-ror for GR3 only ranges from −2.5 to 3.0 tu. GR3 hasa smaller error range than all of the other schedulers mea-sured except WF2Q. GR3 has both a smaller negative andsmaller positive service time error than WRR, VTRR, andSRR. While GR3 has a much smaller positive service er-ror than WFQ, WFQ does have a smaller negative service

time error since it is bounded below at −1. Similarly, GR3

has a much smaller negative service error than SFQ, thoughSFQ’s positive error is less since it is bounded above at 1.Considering the total service error range of each scheduler,GR3 provides well over two orders of magnitude better pro-portional sharing accuracy than WRR, WFQ, SFQ, VTRR,and SRR. Unlike the other schedulers, these results showthat GR3 combines the benefits of low service time errorswith its ability to schedule in O(1) time.

Note that as the weight skew becomes more accentu-ated, the service error can grow dramatically. Thus, increas-ing the skew from 10 to 50 percent results in more thana fivefold increase in the error magnitude for SRR, WFQ,and SFQ, and also significantly worse errors for WRR andVTRR. In contrast, the error of GR3 is still bounded bysmall constants: −2.3 and 4.6.

We also measured the service error of GR3MP usingthis simulator configured for an 8 processor system, wherethe weight distribution was the same as for the uniprocessorsimulations above. Note that the client given 0.1 of the totalweight was feasible, since 0.1 < 1

8 = 0.125. Figure 10shows GR3MP ’s service error is between −2.5 tu and 2.8tu, slightly better than for the uniprocessor case, a benefitof being able to run multiple clients in parallel. Figure 11shows the maximum number of scheduling decisions thatan idle processor needs to perform until it finds a client thatis not running. This did not exceed seven, indicating thatthe number of decisions needed in practice is well belowthe worst-case bounds shown in Theorem 3.

5.2 Linux Kernel Measurements

To evaluate the scheduling overhead of GR3, wecompare it against the standard Linux 2.4 scheduler, a WFQscheduler, and a VTRR scheduler. Since WF2Q has the-


0.1

1

10

100

0 50 100 150 200 250 300 350 400

Ave

rage

sch

edul

ing

cost

(us

)

Number of clients

GR3VTRRWFQ [O(log N)]WFQ [O(N)]LinuxGR3 MPLinux MP

Figure 12: Average scheduling overhead

oretically the same time complexity as WFQ (but withlarger constants, because of the complexity of its steps), wepresent WFQ as a lower bound for the overhead of WF2Q.We present results from several experiments that quantifyhow scheduling overhead varies as the number of clientsincreases. For the first experiment, we measure schedulingoverhead for running a set of clients, each of which exe-cuted a simple micro-benchmark which performed a fewoperations in a while loop. A control program was used tofork a specified number of clients. Once all clients wererunnable, we measured the execution time of each schedul-ing operation that occurred during a fixed time duration of30 seconds. The measurements required two timestampsfor each scheduling decision, so measurement error of 70 nsare possible due to measurement overhead. We performedthese experiments on the standard Linux scheduler, WFQ,VTRR, and GR3 for 1 to 400 clients.

Figure 12 shows the average execution time requiredby each scheduler to select a client to execute. Results forGR3, VTRR, WFQ, and Linux were obtained on uniproces-sor system, and results for GR3MP and LinuxMP were ob-tained running on a dual-processor system. Dual-processorresults for WFQ and VTRR are not shown since MP-readyimplementations of them were not available.

For this experiment, the particular implementationdetails of the WFQ scheduler affect the overhead, sowe include results from two different implementations ofWFQ. In the first, labeled “WFQ [O(N)]”, the run queueis implemented as a simple linked list which must besearched on every scheduling decision. The second, labeled“WFQ [O(log N)]”, uses a heap-based priority queue withO(log N) insertion time. To maintain the heap-based pri-ority queue, we used a fixed-length array. If the numberof clients ever exceeds the length of the array, a costly ar-ray reallocation must be performed. Our initial array sizewas large enough to contain more than 400 clients, so thisadditional cost is not reflected in our measurements.

As shown in Figure 12, the increase in schedulingoverhead as the number of clients increases varies a greatdeal between different schedulers. GR3 has the smallestscheduling overhead. It requires roughly 300 ns to select aclient to execute and the scheduling overhead is essentiallyconstant for all numbers of clients. While VTRR schedul-ing overhead is also constant, GR3 has less overhead be-cause its computations are simpler to perform than the vir-tual time calculations required by VTRR. In contrast, theoverhead for Linux and for O(N) WFQ scheduling growslinearly with the number of clients. Both of these sched-ulers impose more than 200 times more overhead than GR3

when scheduling a mix of 400 clients. O(log N) WFQ hasmuch smaller overhead than Linux or O(N) WFQ, but itstill imposes significantly more overhead than GR3, with 8times more overhead than GR3 when scheduling a mix of400 clients. Figure 12 also shows that GR3MP providesthe same O(1) scheduling overhead on a multiprocessor,although the absolute time to schedule is somewhat higherdue to additional costs associated with scheduling in mul-tiprocessor systems. The results show that GR3MP pro-vides substantially lower overhead than the standard Linuxscheduler, which suffers from complexity that grows lin-early with the number of clients. Because of the impor-tance of constant scheduling overhead in server systems,Linux has switched to Ingo Molnar’s O(1) scheduler in theLinux 2.6 kernel. As a comparison, we also repeated thismicrobenchmark experiment with that scheduler and foundthat GR3 still runs over 30 percent faster.

As another experiment, we measured the schedulingoverhead of the various schedulers for hackbench [21],a Linux benchmark used for measuring scheduler perfor-mance with large numbers of processes entering and leav-ing the run queue at all times. It creates groups of readersand writers, each group having 20 reader tasks and 20 writertasks, and each writer writes 100 small messages to each ofthe other 20 readers. This is a total of 2000 messages sentper writer, per group, or 40000 messages per group. Weran a modified version of hackbench to give each readerand each writer a random weight between 1 and 40. Weperformed these tests on the same set of schedulers for 1group up to 100 groups. Using 100 groups results in up to8000 processes running. Because hackbench frequently in-serts and removes clients from the run queue, the cost ofclient insertion and removal is a more significant factor forthis benchmark. The results show that the simple dynamicgroup adjustments described in Section 2.3 have low over-head, since O(g) can be considered constant in practice.

Figure 13 shows the average scheduling overhead foreach scheduler. The average overhead is the sum of thetimes spent on all scheduling events, selecting clients to runand inserting and removing clients from the run queue, di-vided by the number of times the scheduler selected a clientto run. The overhead in Figure 13 is higher than the av-


0.1

1

10

100

1000

0 10 20 30 40 50 60 70 80 90 100

Tim

e (u

s)

Number of Groups

GR3VTRRWFQ [O(log N)]WFQ [O(N)]LINUXGR3 MPLinux MP

Figure 13: Hackbench weighted scheduling overhead

erage cost per schedule in Figure 12 for all the schedulersmeasured since Figure 13 includes a significant componentof time due to client insertion and removal from the runqueue. GR3 still has by far the smallest scheduling over-head among all the schedulers measured. The overheadfor GR3 remains constant while the overhead for O(log N)WFQ, O(N) WFQ, VTRR, and Linux grows with the num-ber of clients. Client insertion, removal, and selection to runin GR3 are independent of the number of clients. The costfor GR3 is 3 times higher than before, with client selectionto run, insertion, and removal each taking approximately300 to 400 ns. For VTRR, although selecting a client to runis also independent of the number of clients, insertion over-head grows with the number of clients, resulting in muchhigher VTRR overhead for this benchmark.

To demonstrate GR3’s efficient proportional sharingof resources on real applications, we briefly describe threesimple experiments running web server workloads usingthe same set of schedulers: GR3 and GR3MP , Linux2.4 uniprocessor and multiprocessor schedulers, WFQ, andVTRR. The web server workload emulates a number ofvirtual web servers running on a single system. Eachvirtual server runs the guitar music search engine usedat guitarnotes.com, a popular musician resource web sitewith over 800,000 monthly users. The search engine is aperl script executed from an Apache mod-perl module thatsearches for guitar music by title or author and returns alist of results. The web server workload configured eachserver to pre-fork 100 processes, each running consecutivesearches simultaneously.

We ran multiple virtual servers with each one hav-ing different weights for its processes. In the first experi-ment, we used six virtual servers, with one server havingall its processes assigned weight 10 while all other servershad processes assigned weight 1. In the second experiment,we used five virtual servers and processes assigned to eachserver had respective weights of 1, 2, 3, 4, and 5. In the

third experiment, we ran five virtual servers which assigneda random weight between 1 and 10 to each process. For theLinux scheduler, weights were assigned by selecting nicevalues appropriately. Figures 14 to 19 present the resultsfrom the first experiment with one server with weight 10processes and all other servers with weight 1 processes. Thetotal load on the system for this experiment consisted of600 processes running simultaneously. For illustration pur-poses, only one process from each server is shown in thefigures. Conclusions drawn from the other experiments arethe same; those results are omitted due to space constraints.

GR3 and GR3MP provided the best overall propor-tional fairness for these experiments while Linux providedthe worst overall proportional fairness. Figures 14 to 17show the amount of processor time allocated to each clientover time for the Linux scheduler, WFQ, VTRR, and GR3.All of the schedulers except GR3 and GR3MP have apronounced “staircase” effect for the search engine processwith weight 10, indicating that CPU resources are providedin irregular bursts over a short time interval. For the ap-plications which need to provide interactive responsivenessto web users, this can result in extra delays in system re-sponse time. It can be inferred from the smoother curvesof Figure 17 that GR3 and GR3MP provide fair resourceallocation at a finer granularity than the other schedulers.

6 Related WorkRound robin is one of the oldest, simplest and most widelyused proportional share scheduling algorithms. Weightedround-robin (WRR) supports non-uniform client weights byrunning all clients with the same frequency but adjusting thesize of their time quanta in proportion to their respectiveweights. Deficit round-robin (DRR) [22] was developed tosupport non-uniform service allocations in packet schedul-ing. These algorithms have low O(1) complexity but poorshort-term fairness, with service errors that can be on theorder of the largest client weight in the system. GR3 uses anovel variant of DRR for intragroup scheduling with O(1)complexity, but also provides O(1) service error by usingits grouping mechanism to limit the effective range of clientweights considered by the intragroup scheduler.

Fair-share schedulers [12, 14, 15] provide propor-tional sharing among users in a way compatible with aUNIX-style time-sharing framework based on multi-levelfeedback with a set of priority queues. These schedulerstypically had low O(1) complexity, but were often ad-hocand could not provide any proportional fairness guaran-tees. Empirical measurements show that these approachesonly provide reasonable proportional fairness over rela-tively large time intervals [12].

Lottery scheduling [26] gives each client a number oftickets proportional to its weight, then randomly selects aticket. Lottery scheduling takes O(log N) time and relieson the law of large numbers for providing proportional fair-


0

0.5

1

1.5

2

0 50 100 150 200 250 300

CP

U A

lloca

tion

(s)

Time (s)

Weight 10Weight 1Weight 1Weight 1Weight 1Weight 1

Figure 14: Linux uniprocessor

0

0.5

1

1.5

2

0 50 100 150 200 250 300

CP

U A

lloca

tion

(s)

Time (s)


Figure 15: WFQ uniprocessor

0

0.5

1

1.5

2

0 50 100 150 200 250 300

CP

U A

lloca

tion

(s)

Time (s)


Figure 16: VTRR uniprocessor

0

0.5

1

1.5

2

0 50 100 150 200 250 300

CP

U A

lloca

tion

(s)

Time (s)


Figure 17: GR3 uniprocessor

0

0.5

1

1.5

2

2.5

3

3.5

4

0 50 100 150 200 250 300

CP

U A

lloca

tion

(s)

Time (s)


Figure 18: Linux multiprocessor

0

0.5

1

1.5

2

2.5

3

3.5

4

0 50 100 150 200 250 300

CP

U A

lloca

tion

(s)

Time (s)


Figure 19: GR3MP multiprocessor

ness. Thus, its allocation errors can be very large, typicallymuch worse than WRR for clients with smaller weights.

Weighted Fair Queueing (WFQ) [11, 18], was first de-veloped for network packet scheduling, and later applied touniprocessor scheduling [26]. It assigns each client a virtualtime and schedules the client with the earliest virtual time.Other fair queueing variants such as Virtual-clock [28],SFQ [13], SPFQ [24], and Time-shift FQ [10] have alsobeen proposed. These approaches all have O(log N) timecomplexity, where N is the number of clients, because theclients must be ordered by virtual time. It has been shownthat WFQ guarantees that the service time error for anyclient never falls below −1 [18]. However, WFQ can allowa client to get far ahead of its ideal allocation and accumu-late a large positive service time error of O(N), especiallywith skewed weight distributions.

Several fair queueing approaches have been proposedfor reducing this O(N) service time error. A hierar-chical scheduling approach [26] reduces service time er-ror to O(log N). Worst-Case Weighted Fair Queueing(WF2Q) [1] introduced eligible virtual times and can guar-antee both a lower and upper bound on error of −1 and+1, respectively for network packet scheduling. It has alsobeen applied to uniprocessor scheduling as Eligible Vir-tual Deadline First (EEVDF) [25]. These algorithms pro-vide stronger proportional fairness guarantees than otherapproaches, but are more difficult to implement and still re-quire at least O(log N) time.

Motivated by the need for faster schedulers with goodfairness guarantees, one of the authors developed Virtual-Time Round-Robin (VTRR) [17]. VTRR first introducedthe simple idea of going round-robin through clients but

skipping some of them at different frequencies withouthaving to reorder clients on each schedule. This is doneby combining round-robin scheduling with a virtual timemechanism. GR3’s intergroup scheduler builds on VTRRbut uses weight ratios instead of virtual times to provide bet-ter fairness. Smoothed Round Robin (SRR) [9] uses a dif-ferent mechanism for skipping clients using a Weight Ma-trix and Weight Spread Sequence (WSS) to run clients bysimulating a binary counter. VTRR and SRR provide pro-portional sharing with O(1) time complexity for selecting aclient to run, though inserting and removing clients from therun queue incur higher overhead: O(log N) for VTRR andO(k) for SRR , where k = log φmax and φmax is the max-imum client weight allowed. However, unlike GR3, bothalgorithms can suffer from large service time errors espe-cially for skewed weight distributions. For example, we canshow that the service error of SRR is worst-case O(kN).

Grouping clients to reduce scheduling complexity hasbeen used by [20], [8] and [23]. These fair queueing ap-proaches group clients into buckets based on client virtualtimestamps. With the exception of [23], which uses expo-nential grouping, the fairness of these virtual time bin sort-ing schemes depends on the granularity of the buckets andis adversely affected by skewed client weight distributions.On the other hand, GR3 groups based on client weights,which are relatively static, and uses groups as schedulableentities in a two-level scheduling hierarchy.

The grouping strategy used in GR3 was first intro-duced by two of the authors for uniprocessor scheduling [6]and generalized by three of the authors to network packetscheduling [4]. A similar grouping strategy was indepen-dently developed in Stratified Round Robin (StRR) [19] for


network packet scheduling. StRR distributes all clients withweights between 2−k and 2−(k−1) into class Fk (F herenot to be confused with our frontlog). StRR splits time intoscheduling slots and then makes sure to assign all the clientsin class Fk one slot every scheduling interval, using a creditand deficit scheme within a class. This is also similar toGR3, with the key difference that a client can run for upto two consecutive time units, while in GR3, a client isallowed to run only once every time its group is selectedregardless of its deficit.

StRR has weaker fairness guarantees and higherscheduling complexity than GR3. StRR assigns each clientweight as a fraction of the total processing capacity of thesystem. This results in weaker fairness guarantees when thesum of these fractions is not close to the limit of 1. For ex-ample, if we have N = 2k + 1 clients, one of weight 0.5and the rest of weight 2−(k+2) (total weight = 0.75), StRRwill run the clients in such a way that after 2k+1 slots, theerror of the large client is −N

3 , such that this client will thenrun uniterruptedly for N tu to regain its due service. Clientweights could be scaled to reduce this error, but with addi-tional O(N) complexity. StRR requires O(g) worst-casetime to determine the next class that should be selected,where g is the number of groups. Hardware support canhide this complexity assuming a small, predefined maxi-mum number of groups [19], but running an StRR processorscheduler in software still requires O(g) complexity.

GR3 also differs from StRR and other deficit round-robin variants in its distribution of deficit. In DRR, SRR,and StRR, the variation in the deficit of all the clients af-fects the fairness in the system. To illustrate this, considerN + 1 clients, all having the same weight except the firstone, whose weight is N times larger. If the deficit of allthe clients except the first one is close to 1, the error of thefirst client will be about N

2 = O(N). Therefore, the deficitmechanism as employed in round-robin schemes doesn’t al-low for better than O(N) error. In contrast, GR3 ensuresthat a group consumes all the work assigned to it, so thatthe deficit is a tool used only in distributing work within acertain group, and not within the system. Thus, groups ef-fectively isolate the impact of unfortunate distributions ofdeficit in the scheduler. This allows for the error bounds inGR3 to depend only on the number of groups instead of themuch larger number of clients.

A rigorous analysis on network packet scheduling[27] suggests that O(N) delay bounds are unavoidable withpacket scheduling algorithms of less than O(log N) timecomplexity. GR3’s O(g2) error bound and O(1) time com-plexity are consistent with this analysis, since delay and ser-vice error are not equivalent concepts. Thus, if adapted topacket scheduling, GR3 would worst-case incur O(N) de-lay while preserving an O(g2) service error.

Previous work in proportional share scheduling hasfocused on scheduling a single resource and little work has

been done in proportional share multiprocessor scheduling.WRR and fair-share multiprocessor schedulers have beendeveloped, but have the fairness problems inherent in thoseapproaches. The only multiprocessor fair queueing algo-rithm that has been proposed is Surplus Fair Scheduling(SFS) [7]. SFS also adapts a uniprocessor algorithm, SFQ,to multiple processors using a centralized run queue. Notheoretical fairness bounds are provided. If a selected clientis already running on another processor, it is removed fromthe run queue. This operation may introduce unfairness ifused in low overhead, round-robin variant algorithms. Incontrast, GR3MP provides strong fairness bounds withlower scheduling overhead.

SFS introduced the notion of feasible clients alongwith a O(P )-time weight readjustment algorithm, whichrequires however that the clients be sorted by their originalweight. By using its grouping strategy, GR3MP performsthe same weight readjustment in O(P ) time without theneed to order clients, thus avoiding SFS’s O(log N) over-head per maintenance operation. The optimality of SFS’sand our weight readjustment algorithms rests in preserva-tion of ordering of clients by weight and of weight pro-portions among feasible clients, and not in minimal overallweight change, as [7] claims.

7 Conclusions and Future WorkWe have designed, implemented, and evaluated Group Ra-tio Round-Robin scheduling in the Linux operating system.We prove that GR3 is the first and only O(1) uniprocessorand multiprocessor scheduling algorithm that guarantees aservice error bound of less than O(N) compared to an ide-alized processor sharing model, where N is the number ofrunnable clients. In spite of its low complexity, GR3 offersbetter fairness than the O(N) service error bounds of mostfair queuing algorithms that need O(log N) time for theiroperation. GR3 achieves these benefits due to its group-ing strategy, ratio-based intergroup scheduling, and highlyefficient intragroup round robin scheme with good fairnessbounds. GR3 introduces a novel frontlog mechanism andand weight readjustment algorithm to schedule small-scalemultiprocessor systems while preserving its good boundson fairness and time complexity.

Our experiences with GR3 show that it is simple toimplement and easy to integrate into existing commodityoperating systems. We have measured the performanceof GR3 using both simulations and kernel measurementsof real system performance using a prototype Linux im-plementation. Our simulation results show that GR3 canprovide more than two orders of magnitude better pro-portional fairness behavior than other popular proportionalshare scheduling algorithms, including WRR, WFQ, SFQ,VTRR, and SRR. Our experimental results using our GR3

Linux implementation further demonstrate that GR3 pro-vides accurate proportional fairness behavior on real ap-


plications with much lower scheduling overhead than otherLinux schedulers, especially for larger workloads.

While small-scale multiprocessors are the mostwidely available multiprocessor configurations today, theuse of large-scale multiprocessor systems is growing giventhe benefits of server consolidation. Developing accurate,low-overhead proportional share schedulers that scale effec-tively to manage these large-scale multiprocessor systemsremains an important area of future work.

AcknowledgementsChris Vaill developed the scheduling simulator and helpedimplement the kernel instrumentation used for our experi-ments. This work was supported in part by NSF grants EIA-0071954 and DMI-9970063, an NSF CAREER Award, andan IBM SUR Award.

References

[1] J. C. R. Bennett and H. Zhang. WF2Q: Worst-Case FairWeighted Fair Queueing. In Proceedings of IEEE INFO-COM ’96, pages 120–128, San Francisco, CA, Mar. 1996.

[2] D. Bovet and M. Cesati. Understanding the Linux Kernel.O’Reilly, Sebastopol, CA, second edition, 2002.

[3] R. Bryant and B. Hartner. Java Technology, Threads, andScheduling in Linux. IBM developerWorks Library Paper.IBM Linux Technology Center, Jan. 2000.

[4] B. Caprita, W. C. Chan, and J. Nieh. Group Round-Robin:Improving the Fairness and Complexity of Packet Schedul-ing. Technical Report CUCS-018-03, Columbia University,June 2003.

[5] W. C. Chan. Group Ratio Round-Robin: An O(1) Propor-tional Share Scheduler. Master’s thesis, Columbia Univer-sity, June 2004.

[6] W. C. Chan and J. Nieh. Group Ratio Round-Robin:An O(1) Proportional Share Scheduler. Technical ReportCUCS-012-03, Columbia University, Apr. 2003.

[7] A. Chandra, M. Adler, P. Goyal, and P. J. Shenoy. SurplusFair Scheduling: A Proportional-Share CPU Scheduling Al-gorithm for Symmetric Multiprocessors. In Proceedings ofthe 4th Symposium on Operating System Design & Imple-mentation, pages 45–58, San Diego, CA, Oct. 2000.

[8] S. Y. Cheung and C. S. Pencea. BSFQ: Bin-Sort FairQueueing. In Proceedings of IEEE INFOCOM ’02, pages1640–1649, New York, NY, June 2002.

[9] G. Chuanxiong. SRR: An O(1) Time Complexity PacketScheduler for Flows in Multi-Service Packet Networks. InProceedings of ACM SIGCOMM ’01, pages 211–222, SanDiego, CA, Aug. 2001.

[10] J. Cobb, M. Gouda, and A. El-Nahas. Time-Shift Schedul-ing – Fair Scheduling of Flows in High-Speed Networks.IEEE/ACM Transactions on Networking, pages 274–285,June 1998.

[11] A. Demers, S. Keshav, and S. Shenker. Analysis and Sim-ulation of a Fair Queueing Algorithm. In Proceedings ofACM SIGCOMM ’89, pages 1–12, Austin, TX, Sept. 1989.

[12] R. Essick. An Event-Based Fair Share Scheduler. In Pro-ceedings of the Winter 1990 USENIX Conference, pages147–162, Berkeley, CA, Jan. 1990. USENIX.

[13] P. Goyal, H. Vin, and H. Cheng. Start-Time Fair Queue-ing: A Scheduling Algorithm for Integrated Services PacketSwitching Networks. IEEE/ACM Transactions on Network-ing, pages 690–704, Oct. 1997.

[14] G. Henry. The Fair Share Scheduler. AT&T Bell Laborato-ries Technical Journal, 63(8):1845–1857, Oct. 1984.

[15] J. Kay and P. Lauder. A Fair Share Scheduler. Commun.ACM, 31(1):44–55, 1988.

[16] L. Kleinrock. Computer Applications, volume II of Queue-ing Systems. John Wiley & Sons, New York, NY, 1976.

[17] J. Nieh, C. Vaill, and H. Zhong. Virtual-Time Round-Robin:An O(1) Proportional Share Scheduler. In Proceedings ofthe 2001 USENIX Annual Technical Conference, pages 245–259, Berkeley, CA, June 2001. USENIX.

[18] A. Parekh and R. Gallager. A Generalized Processor Shar-ing Approach to Flow Control in Integrated Services Net-works: The Single-Node Case. IEEE/ACM Transactions onNetworking, 1(3):344–357, June 1993.

[19] S. Ramabhadran and J. Pasquale. Stratified Round Robin: aLow Complexity Packet Scheduler with Bandwidth Fairnessand Bounded Delay. In Proceedings of ACM SIGCOMM’03, pages 239–250, Karlsruhe, Germany, Aug. 2003.

[20] J. Rexford, A. G. Greenberg, and F. Bonomi. Hardware-Efficient Fair Queueing Architectures for High-Speed Net-works. In Proceedings of IEEE INFOCOM ’96, pages 638–646, Mar. 1996.

[21] R. Russell. Hackbench: A New Multiqueue Sched-uler Benchmark. http://www.lkml.org/archive/2001/12/11/19/index.html, Dec. 2001. Message toLinux Kernel Mailing List.

[22] M. Shreedhar and G. Varghese. Efficient Fair Queueingusing Deficit Round Robin. In Proceedings of ACM SIG-COMM ’95, pages 231–242, Sept. 1995.

[23] D. Stephens, J. Bennett, and H. Zhang. ImplementingScheduling Algorithms in High Speed Networks. IEEEJSAC Special Issue on High Performance Switches/Routers,Sept. 1999.

[24] D. Stiliadis and A. Varma. Efficient Fair Queueing Algo-rithms for Packet-Switched Network. IEEE/ACM Transac-tions on Networking, pages 175–185, Apr. 1998.

[25] I. Stoica, H. Abdel-Wahab, K. Jeffay, S. Baruah, J. Gehrke,and G. Plaxton. A Proportional Share Resource AllocationAlgorithm for Real-Time, Time-Shared Systems. In Pro-ceedings of the 17th IEEE Real-Time Systems Symposium,pages 288 – 289, Dec. 1996.

[26] C. Waldspurger. Lottery and Stride Scheduling: Flexi-ble Proportional-Share Resource Management. PhD thesis,Dept. of EECS, MIT, Sept. 1995.

[27] J. Xu and R. J. Lipton. On Fundamental Tradeoffs betweenDelay Bounds and Computational Complexity in PacketScheduling Algorithms. In Proceedings of ACM SIGCOMM’02, pages 279–292, Aug. 2002.

[28] L. Zhang. Virtualclock: a New Traffic Control Algorithmfor Packet-Switched Networks. ACM Transactions on Com-puter Systems, 9(2):101–124, May 1991.


Documents

Group Ratio Round-Robin: O(1) Proportional Share ... · Bogdan Caprita, Wong Chun Chan, Jason Nieh, Clifford Stein∗, and Haoqiang Zheng Department of Computer Science Columbia University