Upload
sandeep-joshi
View
188
Download
5
Embed Size (px)
Citation preview
Permit RajControl systems background. Traffic shaping in networking.
Two strategies:1. Leaky bucket : A queue with constant service time. Drops packets on overflow.
2. Token bucket : Bucket holds tokens permits are acquired before transmission. Allows burstiness in traffic.
PID controller
Backpressure
2
Rate Limiter features 1.Enforce long term steady state rate
2.Allow short bursts to exceed limit
3.Warm-up : Gradually increase the rate after idle period
4.Allow changing the rate at run-time
5.Handle requests based on priority (fairness).
4
Rate-limiter implementations in software1.RocksDB
2.Facebook WDT
3.Apache Kafka
4.Apache Spark (Google Guava toolkit)
5.Akka toolkit(not covering)
6.Node.js(not covering)
7.Conclusion5
RocksDBKey value store which uses Log-Structured Merge (LSM) trees.
It has a rate limiter to throttle disk writes done during two different workflows.
1. Flush threads which write in-memory tree to disk.
2. Compaction thread which merges trees on disk.
Throttler requires 3 parameters
3. Refill period
4. Refill bytes per period
5. Fairness which decides which of two queues to serve first
6
RocksDBState kept by Throttler
1. Available bytes [i.e. tokens]
2. Next refill time
3. Queue [low] and Queue[high] into which new requests inserted
Workflow
4. Inserts requests into a queue
5. Leader of the queue awakes at “next refill time” and increments available bytes.
6. Sets the next refill time = now + refill period
7. Services all requests inside the queue until available bytes is exhausted.7
RocksDB1.Enforce long term steady state rate
2.Allow short bursts to exceed rate
3.Warm-up : Gradually increase the rate after idle period
4.Allow changing the rate at run-time
5.Service requests based on priority (fairness).
10
Facebook WDT (Warp-speed Data transfer) http://www.github.com/facebook/wdt
Open-source library which is used for file transfer between data centers. (e.g. transmit MySQL backups to another data center)
Both sender and receiver spawn multiple threads
All threads on sender or receiver share the same “Throttler”
Throttler limits average rate as well as peak bursts.
11
Facebook WDT1.Enforce long term steady state rate
2.Allow short bursts to exceed rate
3.Warm-up : Gradually increase the rate after idle period
4.Allow changing the rate at run-time
5.Service requests based on priority (fairness).
13
Apache KafkaClient-based quotas to limit publisher and consumer processing.
Enforces Fixed-rate in every window
Every request is inserted into into a DelayQueue from which elements can be retrieved only after expiry (DelayQueue is part of java concurrent library)
Delay_time = (window_size) * (observed_rate - desired_rate) / observed_rate
Allows changing quota at run-time.
15
Apache SparkDynamic rate limiter
Two components : Driver (master) and Receiver (accepts ingest)
1.Driver (master) uses PID-based Rate estimator to recompute the desired rate at end of every batch
2.Driver sends new rate to Receiver
3.Receiver uses Google Guava Rate limiter to throttle block generation
16
Apache SparkPID Controller used in the Spark Driver to estimate best rate
1.Proportional (current): correction based on current error
2.Integral gain (past): correction based on steady-state error.
3.Derivative gain (future) : prediction based on rate of change of error
18
Google Guava rate limiterhttps://github.com/google/guava
Used by Receiver in Apache Spark to limit the rate
Stores expected time of next request, instead of time of previous request.
Under-utilization : It stores unused permits up to a max threshold.
Warm-up period : Stored permits are given out gradually by increasing the sleep time.
20
Google Guava rate limiter
21
Allows peak bursts while maintaining average limit.Allows warmup period
Rate-limiting techniquesAll transmitters call some “Throttle()” function before transmission.
Some implementations push requests into a queue, others just calculate sleep time and decrement permits.
Retain time of next estimated wakeup instead of time of previous call.
Permits are added if new epoch is detected.
Save unused permits upto some maximum limit - this handles underutilization.
While using unused permits, increase the sleep time. This increases the warmup period.
22
Comparison of implementationsrocksdb wdt guava/spark kafka
Type Leaky bucket Token Token bucket Leaky
Enforce Average rate Y Y Y Y
Allow short bursts exceeding average
Y Y
Warm-up after idle period
Y
Alter rate at runtime Y Y Y Y
Priority Y 23
References1.RocksDB : util/rate_limiter.cc
2.WDT : Throttler.cpp
3.Apache Kafka : ClientQuotaManager.scala
4.Apache Spark : PIDRateEstimator.scala, RateLimiter.scala
5.Adding Back-pressure to Spark Streaming by Dean Wampler, Typesafe (http://files.meetup.com/1634302/Backpressure%2020160112.pdf)
6.Google Guava : RateLimiter.java and SmoothRateLimiter.java 24
Facebook WDTIf (long term rate > average)
Sleep for (ideal - elapsed) time
Else If (short term rate > peak)
Add tokens based on time difference since last call
Sleep until tokens available are positive
25
Node.jsRate limiter packages available
https://github.com/jhurliman/node-rate-limiter
express-rate
26