42
1 Mor Harchol-Balter Carnegie Mellon University School of Computer Science

Mor Harchol-Balter Carnegie Mellon University School of Computer Science

  • Upload
    homer

  • View
    37

  • Download
    0

Embed Size (px)

DESCRIPTION

Scheduling Your Network Connections. Mor Harchol-Balter Carnegie Mellon University School of Computer Science. FCFS. jobs. jobs. PS. SRPT. jobs. Q: Which minimizes mean response time?. “size” = service requirement. load r < 1. Q: Which best represents - PowerPoint PPT Presentation

Citation preview

Page 1: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

1

Mor Harchol-BalterCarnegie Mellon UniversitySchool of Computer Science

Page 2: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

2

“size” = service requirement

load < 1

jobs SRPT

jobs

jobs PS

FCFS

Q: Which minimizes mean response time?

Page 3: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

3

“size” = service requirement

jobs SRPT

jobs

load < 1

jobs PS

FCFS

Q: Which best represents scheduling in web servers ?

Page 4: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

4

IDEA

How about using SRPT instead of PS in web servers?

Linux 0.S.

WEBSERVER(Apache)

client 1

client 2

client 3

“Get File 1”

“Get File 2”

“Get File 3”

Internet

Page 5: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

5

Many servers receive mostly static web requests.

“GET FILE”

For static web requests, know file size

Approx. know service requirement of request.

Immediate Objections

1) Can’t assume known job size

2) But the big jobs will starve ...

Page 6: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

6

Outline of Talk

[BH – Sigmetrics 01] “Analysis of SRPT: Investigating Unfairness”[HSW-Performance 02] “Asymptotic Convergence of Scheduling Policies…”[WH – Sigmetrics 03*] “Classifying Scheduling Policies wrt Unfairness …”

THEORY

IMPLEMENT

www.cs.cmu.edu/~harchol/

[HSBA – TOCS 03] “Size-based Scheduling to Improve Web Performance”[SH – ITC 03*] “Web servers under overload: How scheduling can help”[MSAH – ICDE03] “Priority Mechanisms for OLTP and Web Applications”

(M/G/1)

Schroeder

Wierman

Page 7: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

7

THEORY SRPT has a long history ...

1966 Schrage & Miller derive M/G/1/SRPT response time:

1968 Schrage proves optimality

1979 Pechinkin & Solovyev & Yashkov generalize

1990 Schassberger derives distribution on queue length

BUT WHAT DOES IT ALL MEAN?

Page 8: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

8

THEORYSRPT has a long history (cont.)1990 - 97 7-year long study at Univ. of Aachen under Schreiber SRPT WINS BIG ON MEAN!

1998, 1999 Slowdown for SRPT under adversary: Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen, Bender, Chakrabarti, etc. SRPT STARVES BIG JOBS!

Various o.s. books: Silberschatz, Stallings, Tannenbaum: Warn about starvation of big jobs ...

Kleinrock’s Conservation Law: “Preferential treatment given to one class of customers is afforded at the expense of other customers.”

Page 9: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

9

Unfairness Question

SRPT

PS?

?

Let =0.9. Let G: Bounded Pareto(= 1.1, max=1010)

Question: Which queue does biggest job prefer?

M/G/1

M/G/1

Page 10: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

10

Results on UnfairnessLet =0.9. Let G: Bounded Pareto(= 1.1, max=1010)

SRPT

PS

I SRPT

Page 11: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

11

Unfairness – General Distribution

All-can-win-theorem:

For all distributions, if ½,

E[T(x)]SRPT E[T(x)]PS for all x.

Page 12: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

12

All-can-win-theorem:

For all distributions, if ½,

E[T(x)]SRPT E[T(x)]PS for all x.

Proof idea:

x

t

dt

0 )1 1x

)

x

xFx

2

2

))1(2

(

0x

dttft 2 )(

Waiting time (SRPT) Residence (SRPT) Total (PS)

Page 13: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

13

Classification of Scheduling Policies

ALWAYS FAIR For all loads, for all service distributions,

ALWAYS UNFAIR For all loads, for all service distributions,

SOMETIMES UNFAIR For some loads:

For other loads :

PSP xTExTEx )]([)]([ ,

PSP xxTExTE ,)]([)]([

PSP xTExTEx )]([)]([ ,

PSP xxTExTE ,)]([)]([

Page 14: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

14

Classification of Scheduling Policies

AlwaysFAIR

AlwaysUnfair

Sometimes Unfair

Age-BasedPolicies

Preemptive Size-basedPolicies

Remaining Size-basedPolicies

Non-preemptive

PS PLCFS

FB

PSJF

LRPT

FCFS

LJF

SJF

SRPT Lots of open problems…

FSP

Page 15: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

15

What does SRPT mean within a Web server?

• Many devices: Where to do the scheduling?

• No longer one job at a time.

IMPLEMENT From theory to practice:

Page 16: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

16

Server’s Performance BottleneckIMPLEMENT

5

Linux 0.S.

WEBSERVER(Apache)

client 1

client 2

client 3

“Get File 1”

“Get File 2”

“Get File 3”

Rest ofInternet ISP

Site buyslimited fractionof ISP’s bandwidth

We model bottleneck by limiting bandwidth on server’s uplink.

Page 17: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

17

Network/O.S. insides of traditional Web server

Sockets take turnsdraining --- FAIR = PS.

WebServer

Socket 1

Socket 3

Socket 2Network Card

Client1

Client3

Client2BOTTLENECK

IMPLEMENT

Page 18: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

18

Network/O.S. insides of our improved Web server

Socket corresponding to filewith smallest remaining datagets to feed first.

WebServer

Socket 1

Socket 3

Socket 2Network Card

Client1

Client3

Client2

priorityqueues.

1st

2nd

3rd

S

M

L

BOTTLENECK

IMPLEMENT

Page 19: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

19

Experimental Setup

Implementation SRPT-based scheduling: 1) Modifications to Linux O.S.: 6 priority Levels 2) Modifications to Apache Web server 3) Priority algorithm design.

Linux 0.S.

1

2

3

APACHEWEB

SERVER

Linux

12

3

200

Linux

123

200

Linux

12

3

200

switch

WA

N E

MU

WA

N E

MU

WA

N E

MU

Page 20: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

20

Experimental Setup

APACHEWEB

SERVER

Linux 0.S.

123

Linux

123

200

Linux

123

200

Linux

123

200

switch

WA

N E

MU

WA

N E

MU

WA

N E

MU

Trace-based workload: Number requests made: 1,000,000Size of file requested: 41B -- 2 MBDistribution of file sizes requested has HT property.

Flash

Apache

WAN EMU

Geographically-dispersed clients

10Mbps uplink

100Mbps uplink

Surge

Trace-based

Open system

Partly-open

Load < 1

Transient overload

+ Other effects: initial RTO; user abort/reload; persistent connections, etc.

Page 21: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

21

Preliminary Comments

• Job throughput, byte throughput, and bandwidth utilization were same under SRPT and FAIR scheduling.

• Same set of requests complete.

• No additional CPU overhead under SRPT scheduling. Network was bottleneck in all experiments.

APACHEWEB

SERVER

Linux 0.S.

123

Linux

123

200

Linux

123

200

Linux

123

200

switch

WA

N E

MU

WA

N E

MU

WA

N E

MU

Page 22: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

22

Load

FAIR

SRPTMea

n R

esp

onse

Tim

e (s

ec)

Results: Mean Response Time (LAN)

.

.

.

.

.

.

Page 23: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

23

Percentile of Request Size

Mea

n R

esp

onse

tim

e (

s)

FAIR

SRPT

Load =0.8

Mean Response Time vs. Size Percentile (LAN)

Page 24: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

24

Transient Overload

Page 25: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

25

Transient Overload - Baseline

Mean response time

SRPTFAIR

Page 26: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

26

Transient overloadResponse time as function of job

size

small jobswin big!

big jobsaren’t hurt!

FAIR

SRPT

WHY?

Page 27: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

27

Baseline Case

WAN propagation delays

WAN loss

Persistent Connections

Initial RTO value

SYN Cookies

User Abort/Reload

Packet Length

Realistic Scenario

WAN loss + delay

RTT: 0 – 150 ms

Loss: 0 – 15%

Loss: 0 – 15%RTT: 0 – 150 ms,

0 – 10 requests/conn.

RTO = 0.5 sec – 3 sec

ON/OFF

Abort after 3 – 15 sec, with 2,4,6,8 retries.

Packet length = 536 – 1500 Bytes

RTT = 100 ms; Loss = 5%; 5 requests/conn.,RTO = 3 sec; pkt len = 1500B; User abortsAfter 7 sec and retries up to 3 times.

FACTORS

Page 28: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

28

Transient Overload - Realistic

Mean response time

FAIR SRPT

Page 29: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

29

SRPT scheduling is a promising solution for reducing

mean response time seen by clients, particularly when the load at server bottleneck is high, or under transient overload conditions.

SRPT results in negligible or zero unfairness to large requests.

SRPT is easy to implement and efficient. No CPU overhead. No drop in throughput.

Results corroborated via implementation and analysis.

Conclusion so far …

Page 30: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

30

More questions …

STATIC web

requests

Everything so far in talk …

DYNAMICweb

requests

Current work…

SchroederMcWherterSchroeder

Wierman

Page 31: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

31

Online Shopping

Internet

client 1

client 2

client 3

“buy”

“buy”

“buy”

Web Server(eg: Apache/Linux)

Database(eg: DB2, Oracle, PostgreSQL)

• Dynamic responses take much longer – 10sec• Database is bottleneck.

Page 32: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

32

Online Shopping

Internet

client 1

client 2

client 3

“$$$buy$$$”

“buy”

“buy”

Web Server(eg: Apache/Linux)

Database(eg: DB2, Oracle, PostgreSQL)

Goal: Prioritize requests

Page 33: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

33

Isn’t “prioritizing requests” problem already solved?

Internet

“$$$buy$$$”

“buy”

“buy”

Web Server(eg: Apache/Linux)

Database(eg: DB2, Oracle, PostgreSQL)

No.Prior work mostly simulation or RTDBMS.

Page 34: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

34

Which resource to prioritize?“$$$buy$$$”

“buy”

“buy”

Web Server(eg: Apache/Linux)

Internet

Internet

Database

Disks LocksCPU(s)

High-Priority client Low-Priority client

Page 35: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

35

Q: Which resource to prioritize?“$$$buy$$$”

“buy”

“buy”

Web Server(eg: Apache/Linux)

Internet

Internet

Database

Disks LocksCPU(s)

High-Priority client Low-Priority client

A: 2PL Lock Queues

Page 36: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

36

What is bottleneck resource?

• IBM DB2 -- Lock waiting time (yellow) is bottleneck.• Therefore, need to schedule lock queues to have impact.

Fix at 10 warehouses #clients = 10 x #warehouses

Page 37: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

37

Why lock scheduling is hardLock resource 1

HHLL L

HHLL L

Lock resource 2

NP H may wait long time

NPinherit Speeding up L may hurt H in long run

Pabort Rollback cost + wasted work + really hurt L’s.

Page 38: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

38

Results:

Implementationstudy of

NP, NPinherit, Pabort

under TPC-C workload,Shore DBMS

Developnew policy

POW (Preempt on Wait)

Page 39: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

39

Results:

Think time

Responsetime

Pabort

NPinherit

Pabort

NPinherit

Page 40: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

40

Results:

Think time

Responsetime

Pabort

NPinherit

Pabort

NPinherit POW:

Best of both

Page 41: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

41

More work in SYNC project…

DBMS(eg: DB2, Oracle)QoSInternet

“$$$buy$$$”

“buy”

“buy”

Web Server

QoS from outside the box

Scheduling the TeraGrid

Time-varying load in systems

Impact of closed versus open system models

PSCSDSC

NCSA

Page 42: Mor Harchol-Balter Carnegie Mellon University School of Computer Science

42

Scheduling is a very cheap solution… No need to buy new hardware No need to buy more memory Small software modifications

…with a potentially very big win in some situations.

Conclusion

Thank you!