Reorganizing Website Architecture for HTTP/2 and Beyond

Copyright(C)2016DeNACo.,Ltd.AllRightsReserved.

Reorganizing Website Architecture for HTTP/2 and Beyond

DeNA Co., Ltd.Kazuho Oku


Who am I

n  lead developer of H2O HTTP/2 server⁃  one of the most sophisticated HTTP/2 impl.⁃  initial public release: 2014/10 (license: MITL)⁃  used by Fastly, etc.

n  author of HTTP performance-related I-Ds⁃  Cache Digests for HTTP/2⁃  103 Early Hints

n  lives in Japan, works for DeNA Co., Ltd.



Current State of HTTP



Why use HTTP/2?

n  latency has become the bottleneck of the Webn  HTTP/2 to conceal latency by raising concurrency

⁃  6 concurrent requests in HTTP/1⁃  ~100 in HTTP/2



Current state of HTTP

n  HTTP/2 (RFC 7540) released on May 2015

1: https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/http2-review-data.pdf


45

28

37

41

18

31

0% 20% 40% 60% 80% 100%

2015/7

2016/7

#oftransac+onsbyFirefox1

HTTP HTTPS(H1) HTTPS(H2)


Achievements

n  header compression (HPACK)⁃  compression ratio: 90% (at median) according to

Mozillan  multiplexing

⁃  the tiled benchmark



So is everything optimal now? Not quite.



Issues

n  errors in prioritizationn  TCP head-of-line blockingn  inefficiency of HTTP/2 pushn  compression ratio of small responses



Prioritization



イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。

イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを

イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。

The prioritization tree


Root

LeaderG

FollowerGweight:1

HTMLweight:32

Imageweight:22

Imageweight:22

Imageweight:22

CSSweight:32

CSSweight:32

n  hybrid approach using weights and chaining⁃  servers are expected to obey to the priority

specified by the clientsn  Firefoxʼs prioritization tree is shown below

JSweight:32

JSweight:32

イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破損している可能性があります。コンピューターを再起動して再度ファイルを開いてください。それでも赤い x が表示される場合は、イメージを削除して挿入してください。



The prioritization tree

n  some web browsers fail to specify priority⁃  Safari, Blink⁃  older versions of Chrome also had issues⁃  server-side countermeasures required





イメージを表示できません。メモリ不足のためにイメージを開くことが

イメージを表示できません。メモリ不足のためにイメージを開くことができないか、イメージが破



Root

HTMLweight:16

CSSweight:16

JSweight:16

Imageweight:16

Imageweight:16

Imageweight:16


HTTP/2 prioritization: the solution

n  bandwidth distribution on server-side:⁃  use Weighted Fair Queuing (WFQ) or Deficit

Round Robin (DRR)⁃  some servers do it right:•  nghttp2 (and Apache) implements WFQ in O(log N)•  H2O approximates WFQ in O(1)

n  detect dumb clients and fallback to server-driven prioritization⁃  H2O reprioritizes CSS, JS for clients that do not

use priority chains



HTTP/2 prioritization: benchmark

n  differences between the times spent until first-paint (red bar)



TCP head-of-line blocking



Typical sequence of HTTP/2


HTTP/2 200 OK

<!DOCTYPE HTML>...<LINK HREF="style.css"...

client server

GET /

GET /style.css

needtoswitchfromsendingHTMLtosendingCSSatthisverymoment,orfromsendingimagestosendingCSSincasetheCSSindiscoveredlate(i.e.hiddenassets)

1RTT

*



n  head-of-line blocking:⁃  high-priority data blocked by preceding data in

flightn  TCP head-of-line blocking:

⁃  data in TCP send buffer blocks following data of higher priority





n  typical H2 server writes much more than that can be sent immediately⁃  unsent data in TCP send buffer (and TLS buffer)

head-of-line blocks following data

TCPsendbuffer

CWNDunacked pollthreshold

TLSbuf.

TLSRecords

sentimmediately notimmediatelysent

HTTP/2frames


TCP head-of-line blocking: the solution

n  write only what can be sent immediately⁃  obtain CWND and unacked size using TCP_INFO

n  adjust poll threshold to delay write notification until TCP becomes ready to send some data immediately


CWNDunacked pollthreshold

TLSRecords

sentimmediately notimmediatelysent

HTTP/2frames

TCPsendbuffer


TCP head-of-line blocking: benchmark 1


n  conditions:⁃  server in Ireland, client in Tokyo (RTT 250ms)⁃  load tiny js at the top of a large HTML

n  result: delay decreased from 511ms to 250ms⁃  i.e. JS fetch latency was 2 RTT, became 1 RTT•  similar results in other environments


TCP head-of-line blocking: benchmark 2

n  using same data as previousn  server and client both in Japan (25ms RTT)


0

50

100

150

200

250

300

HTML JS

millisecon

ds�

downloadingHTML(andJSwithin)RTT~25ms�

master latopt


TCP buffers exist everywhere

n  HTTP/2 serverʼs TCP send buffer⁃  the negative impact can be avoided by a clever

HTTP/2 server (as shown by H2O)n  TLS terminator

⁃  better to avoid using, if performance mattersn  mobile networks

⁃  cannot be controlled by a web-site admin⁃  QUIC would be the final solution



Push



Push

n  positive reports:⁃  “20-30% speedup on page render time”2

n  negative comments:⁃  many unnecessary pushes (47% are reset2)⁃  increased render time in anti-patterns3

⁃  “consider preload instead of push”3

n  what is the cause of these negative views?

2: https://github.com/HTTPWorkshop/workshop2016/blob/master/talks/server-push.pdf3: https://docs.google.com/document/d/1K0NykTXBbbbTlv60t5MyJvXjqKGsCVNYHyLEXIxYMv0/edit



Itʼs the cached resources

n  server shouldnʼt push resources already cached⁃  but HTTP/2 doesnʼt provide a way to determine⁃  pushing same resources multiple times is waste

of bandwidthn  solution: cache-digests

⁃  now an IETF HTTP WG draft⁃  https://tools.ietf.org/html/draft-ietf-httpbis-

cache-digest-01



Cache-digests

n  client sends a digest of cached responses along with an HTTP request

n  server examines the digest, pushes only the resources that arenʼt cached

n  digest: list of SHA(URL)s stored in Golomb-coded sets (a space-efficient variant of bloom filter)

n  size of digest: ~ N*log2(1/P) bits⁃  N: # of resources⁃  P: false-positive-rate



Example

n  set false positive rate to: P=1/27

n  calculate SHA256 of two URLs (N=2)⁃  https://example.com/style.css (SHA256: baf9…)⁃  https://example.com/script.js (SHA256: 1174…)

n  truncate to log2(N*1/P) bits and sort⁃  0x11, 0xba

n  emit log2(N), log2(1/P) in 5 bits, then the deltas of sorted list using Golomb coding (subtracted by 1)⁃  00001 00111 1 0010001 01 010 1000 N 1/P 0x11 0xa8 (0xba-0x11-1)



Overhead of sending a Cache Digest

n  # of fresh responses in browser cache per domain is expected to be <~1K⁃  looking at about:cache•  *.facebook.com: 790•  *.google.com: 373

n  HTTP request + cache-digest can be packed into a single TCP packet⁃  when setting P to 1/128 or 1/256•  or, P can be set to a bigger value to conserve space,

or cache-digest could be split into multiple packets following the HTTP request



Implementation Status

n  three ways to send a cache-digest⁃  I-D proposes the use of HTTP/2 frame⁃  header-based scheme is TBD•  so that websites can implement cache-digest using

ServiceWorker (e.g. cache-digest.js)

1: the listed softwares may have interoperability issues due to implementing different versions of the draft

2: approximating the cache state using cookies has accuracy issues


Conveyer ServerImplementa+on ClientImplementa+on

HTTP/2frame nghYp2 N/A

HTTPheader1 Apache,H2O cache-digest.js

HTTPcookie2 H2O notneeded


but even after solving the never-push-cached-resources issue, push doesnʼt necessarily improve

performance



Benchmark scores

n  first-paint does not change⁃  since the browser needs both HTML and CSS⁃  push only changes the order between the two

n  less time spent showing a blank page⁃  since unload happens when HTML arrives



so is push an useless technology? no



Three use-cases of H2 push

n  prioritizationn  push while processing requestn  push from edge



Pushing for prioritization


client server

GET /

GET/style.css

HTTP/2 200 OK<html><link rel=style.css...

HTTP/2 200 OKbody: ...#title: ...

1.  send CSS, JS first2.  then send HTML

(can be rendered progressively)

withoutpush

client server

GET /GET /style.css HTTP/2HTTP/2 200 OKbody: ...#title: ...

withpush

HTTP/2 200 OK<html> <link rel=style.css ...


Push while processing request

n  web applications involving DB access, etc.


req.�

processrequest�push-asset�

HTML�

push-asset�

push-asset�

push-asset�

req.�

processrequest�

asset�

HTML�

asset�

asset�

asset�

req.�

450ms(5RTT+processing=m

e)�


e)�

withoutpush� withpush�


Push from edge

n  CDNsʼ use-case⁃  utilize the conn. while waiting for app. response


req.�

push-asset�

HTML�

push-asset�

push-asset�

push-asset�

client� edgeserver(CDN)� app.server�

req.�

HTML�


Impact of push

n  prioritization doesnʼt have much benefit⁃  as discussed

n  push-while-processing, and push-from-edge are the real use-cases⁃  but how to use?



Push while processing


req.�

processrequest�

push-asset�

HTML�

push-asset�

push-asset�

push-asset�

req.�

processrequest�

asset�

HTML�

asset�

asset�

asset�

req.�


e)�


e)�

withoutpush� withpush�n  One way is to configure the H2 server

mruby.handler: | Proc.new do |env| push_paths = [] if /(\/|\.html)$/.match(env["PATH_INFO"]) push_paths << "/style.css” ... end [399, push_paths.empty? ? {} : { "link" => push_paths.map{|p| "<#{p}>; rel=preload” }.join("\n") }, []] endfastcgi.connect: ...


Push while processing


n  or use 103 Early Hints⁃  application sends 103 then processes the request

GET / HTTP/1.1Host: example.com

HTTP/1.1 103 Early HintsLink: </style.css>; rel=preload

HTTP/1.1 200 OKContent-Type: text/htmlLink: </style.css>; rel=preload

<!DOCTYPE HTML>...

HTTP/2server app.server

GET/

103EarlyHintsLink:…

200OK

processrequest

WebBrowser

GET/

200OK

push


103 Early Hints

n  a new I-D (published last week)⁃  https://tools.ietf.org/html/draft-kazuho-early-

hints-status-code-00n  the status code indicates that the server is likely to

send a final request with the headers included in the informational response⁃  can be used to hint preloading 3rd party assets

(e.g. ads) as welln  already supported by: nghttp2, H2O

⁃  other server / web-application framework developers have shown interest



Push from edge

n  at least some CDNs are likely to provide configuration directives at the edge-server level


req.�

push-asset�

HTML�

push-asset�

push-asset�

push-asset�

client� edgeserver(CDN)� app.server�

req.�

HTML�


compression ratio of small responses



File size vs. compression ratio

n  claim: no need to concatenate, since H2 improves request concurrency

n  reality: small files suffer from low compression ratio⁃  problem for API as well

note: first N bytes of jquery-3.1.1.js compressed with best quality settings


0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

256 1,024 4,096 16,384 65,536 86,709filesize�

Compressionra/oofaJavaScriptfile�

gzip

brotli


Two proposals

n  Shared Dictionary Compression over HTTP (SDCH)⁃  uses a pre-shared compression dictionary⁃  https://tools.ietf.org/html/draft-lee-sdch-spec-00

n  Compression Dictionaries for HTTP/2⁃  re-uses the compression context of a previously

transmitted response⁃  https://tools.ietf.org/html/draft-vkrasnov-h2-

compression-dictionaries-01



Differences between the approaches

n  pre-shared dictionary:⁃  achieves high compression ratio when the

dictionary is cached by the client (i.e. when revisiting)

⁃  cannot be used at first visitn  re-using compression context of another response:

⁃  works effectively at first visit⁃  easier to use, since no configuration needed⁃  may have security issues•  i.e. CRIME attack



Compression ratio of small responses

n  beneficial to concatenate small files (to ~32KB or larger) for the time being



Summary



HTTP/2 is not perfect (yet)

n  choice of servers / CDNs causes difference⁃  be careful when looking at benchmarks

n  avoid using:⁃  TLS terminators⁃  hidden assets (or un-hide them w. preload links)

n  use push for either of:⁃  push-while-processing (103 Early Hints)⁃  push-from-edge (configure your CDN)

n  continue to concatenate files, at least the tiny ones


Technology

Reorganizing Website Architecture for HTTP/2 and Beyond