Zeta: Scheduling Interactive Services with Partial Execution Yuxiong He, Sameh Elnikety, James Larus,…

Zeta: Scheduling Interactive Services with Partial Execution

Yuxiong He, Sameh Elnikety, James Larus, Chenyu YanMicrosoft Research and Microsoft Bing

2

MotivationFast ResponseGood quality

High utilizationFewer servers $$

Improve system throughput while meeting response time and quality target for interactive services.

Contribution• Develop a new scheduling algorithm Zeta– Exploit partial execution and request deadline– Improve response quality and system throughput

3

400 450 500 550 600 650 7000.960000000000001

0.965000000000001

0.970000000000001

0.975000000000001

0.980000000000001

0.985000000000001

0.990000000000001

0.995000000000001

1

Queries Per Second (QPS)

QualityZeta

Original

Increase capacity 29%

Potentially save 22% servers

4

Outline

• Workload characteristics

• Zeta scheduling algorithm

• Evaluation– System implementation– Simulation

5

Workload Characteristics(1) Deadline(2) Partial execution(3) Concave quality profile

6


7


time

Qua

lity

timeQ

ualit

y

(a) without partial execution (b) with partial execution

request completion time


8


time

Qua

lity

timeQ

ualit

y

(a) without partial execution (b) with partial execution



9

Concave Quality Profile of Bing Search

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Normalized Processing Time

Quality

10

Scheduling Problem• Inputs• Queue of requests• Request deadline• Average service demand of requests• Unknown exact service demand of a request

• Output• Assign processing time to request R1

• Objective : maximize total quality of all requests

R1 R2 R3

11

Zeta Scheduling• Key idea– Prevent long requests from starving short ones– Partially execute long requests if needed

R1 R2 R3

R1

R1 R2 R3

R2 R3

12

Two Techniques of ZetaKey Idea: Prevent long requests from starving short ones

(1) Equi-partitioning (EQ) at heavy load:• Allocate resource equally among requests• PEQ = T / Q

T: available time for all requests Q: queue length for waiting requests

R1 R2 R3

R1

R2 R3Reserved

13

Problem of EQ at Light LoadKey Idea: Prevent long requests from starving short ones


T: available time for all requests Q: queue length for waiting requests

R1 R2

R1 ReservedR2

14


(2) Reservation (RESV) at light load: • Reserve time for later requests in the queue based on

average service demand• PRESV = T – (Q – 1) x D

T: available time for all requests Q: queue length for waiting requests D: average service demand

R1 R2

R1 ReservedR2

Assigned to R1

15




mean service demand• PRESV = T – (Q – 1) x D

How to select between RESV and EQ? T: available time for all requests Q: queue length for waiting requests D: average service demand

16




mean service demand• PRESV = T – (Q – 1) x D

Request processing time P = max(PRESV,PEQ) T: available time for all requests Q: queue length for waiting requests D: average service demand

17

Zeta Scheduling Algorithm• Get request R at the front of the job queue

• Assign processing time to request R– Compute PEQ , PRESV

– Assign P = max(PEQ,PRESV) to Request R

• Execute request R until it completes or used up P processing time

18

Background: Microsoft

Aggregators

ISIS IS IS

Webpages

Index Server

Aggregator

. . . . . .

19

Implementation and Evaluation Setup

• Implementation at index server– Original, FIFO/EDF

• Terminate requests upon deadline or completion– Zeta

• Use Zeta scheduling to assign request processing time

• ~100 lines of code changes

• Standard Bing testing cluster, ~800 servers• Query trace from production

20

1) Improve Average Response Quality

400 450 500 550 600 650 7000.960000000000001

0.965000000000001

0.970000000000001

0.975000000000001

0.980000000000001

0.985000000000001

0.990000000000001

0.995000000000001

1

QPS

Quality

Original

Zeta

475 610

Increase system capacity 29%

Potentially save 22% servers

21

2) Reduce Response Quality Variance

400 450 500 550 600 650 7000

0.002

0.004

0.006

0.008

0.01

0.012

0.014

0.016

QPS

Quality Variance

Original

Zeta

22

3) Improve 95% Response Quality

400 450 500 550 600 650 7000.700000000000001

0.750000000000001

0.800000000000001

0.850000000000001

0.900000000000001

0.950000000000001

1

QPS

95% Quality

Original

Zeta

23

Simulation• Sensitivity to quality profiles– Effective on various coarse-grain concave-like profiles.

Normalized processing time

Qua

lity

0 5 10 15 20 25 30 350.86

0.88

0.9

0.92

0.94

0.96

0.98

1

ZetaFIFO

Arrival Rate

Qua

lity

Simulation• Sensitivity to quality profiles– Effective on various coarse-grain concave-like profiles.


Qua

lity

0 5 10 15 20 25 30 350.86

0.88

0.9

0.92

0.94

0.96

0.98

1

ZetaFIFO

Arrival Rate

Qua

lity


Qua

lity

0 5 10 15 20 25 30 350.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1

ZetaFIFO

Arrival Rate

Qua

lity

25

Conclusion• Develop Zeta scheduling algorithm– Exploit workload characteristics

• Deadline, partial execution, concave quality profile– Combine EQ and RESV techniques – Improve response quality and throughput

• Evaluation– System implementation• Bing search: increase throughput by 29%

– Simulation• Effective on various concave-like quality profiles

26

Thank you!

Questions?

Documents

Zeta: Scheduling Interactive Services with Partial Execution Yuxiong He, Sameh Elnikety, James Larus,…