46
static file cache Static file caching using realurl, mod_rewrite and mod_expires. . . . It slows down the warming of the earth. Michiel Roos Netcreators

Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

  • Upload
    others

  • View
    19

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

static file cache

Static file caching using realurl, mod_rewrite and mod_expires.

. . . It slows down the warming of the earth.

Michiel RoosNetcreators

Page 2: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

What does it do?

Ehrm . . . it caches static files?

I mean . . . it statically caches static pages?

Eh . . . it generates static html files from static pages. If a static html file exists, mod_rewrite will redirect the visitor to the static page.

This means that TYPO3 will not be loaded at all.

Your server will have less work to do and will use less power. This helps to keep our earth cool ;-)

Page 3: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

What's in it for me?

23000 %speed improvement for static pages

Page 4: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Key features

it's fscking fast

transparent to the user

works in the existing domain and port

sends cache headers

Page 5: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Requirements

apache

mod_rewrite

realurl

mod_expires

required recommended

Page 6: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

in the beginning

Tim Lochmüller: fl_staticfilecache (cheers Tim! ;-)

the 'not so good':xclasses

not my kind of code

backend module? But why?

need to actively clear cache

cache headers don't work with static html

Page 7: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

The GOOD!

Using mod_rewrite is a great idea!

TYPO3 is very good at serving dynamic content

Apache is very good at serving static html files

Impressive speed improvement possible

Page 8: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Questions

What can be cached?

How can we invalidate / clear cache?

How can we make cache headers work?

What hooks exist?

Where are they?

When are they called?

Does it work?

Does it really work?

Page 9: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Shadow TYPO3

create hook:insertPageIncache

clear hooks:clearCachePostProc

clearPageCacheEval

tslib_fe-PostProc

catch Ctrl+Shift+Reload:tslib_fe-PostProc

Page 10: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

How does it work?

cache pages to static html file

use mod_rewrite to:check if a static html file exists, if it does . . .

redirect the request to the static html file, otherwise . . .

fall through to the next rewrite rule.

Page 11: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

What does it look like?

RewriteCond -> RewriteRule [L]

RewriteCond %{DOCUMENT_ROOT}/typo3temp/tx_ncstaticfilecache/%{HTTP_HOST}/%{REQUEST_URI}index.html -f

RewriteRule .* typo3temp/tx_ncstaticfilecache/%{HTTP_HOST}/%{REQUEST_URI} [L]

Page 12: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Other conditions

RewriteCond %{HTTP:Pragma} !no-cache

RewriteCond %{HTTP:Cache-Control} !no-cache

RewriteCond %{HTTPS} off

RewriteCond %{HTTP_COOKIE} !be_typo_user [NC]

RewriteCond %{REQUEST_METHOD} GET

RewriteCond %{QUERY_STRING} ^$

Page 13: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Enabling cache headers?

Static files don't send cache headers.

If you are using mod_rewrite, chances are . . .

. . . you also have access to mod_expires.

Page 14: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Mod Expires

If realurl is used we have a cache directory structure resembling the URI.

Every cached file lives in it's own directory.

Use the timeout value for the page . . .

. . . to write a .htaccess file for every index.html.<IfModule mod_expires.c>

ExpiresActive on

ExpiresByType text/html M60

</IfModule>

Page 15: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Cleaning up . . .

Once the static file exists, it will stay.

Eventually it will expire.

When the browser requests a fresh page, it will still get the old one.

Unless we manually remove the static file after it has expired.

Page 16: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

The numbers

ab

apache bench

try this on your moms server!

Page 17: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Server torture

requests / s time / request ms transfer rate Kbps test time sdynamic 6.3 158.68 83.82 158.68TYPO3 cache 23.28 42.96 283.13 42.96static 5346.62 0.19 71805.12 0.19

ab -c 100 -n 1000 http://www.fictive.org/gpl-short/

dynamic

TYPO3 cache

static

Page 18: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Serve more requests per second

dynamic TYPO3 cache static

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

5500

requests / s

6 23

Page 19: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Serve requests faster

dynamic TYPO3 cache static

0102030405060708090

100110120130140150160

mean time per request in ms

0.2

Page 20: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Performance increase factor

230times as quick

Page 21: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Let's see that again!

23000 %speed improvement

Page 22: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

I have a dream . . .

That one day . . .

. . . all static file cache extensions . . .

. . . will become one!

called: 'staticfilecache'

me scratch!

Page 23: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

Tools

eclipse

svn

firefox + live http headers

cc_devlog

grep, sed et al

Page 24: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

1

static file cache

Static file caching using realurl, mod_rewrite and mod_expires.

. . . It slows down the warming of the earth.

Michiel RoosNetcreators

Recently Tim Lochmuller tranferred the key to netcreators.

Kaspar recently released staticpub

Benjamin Mack released bestfilecache

Page 25: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

2

What does it do?

Ehrm . . . it caches static files?

I mean . . . it statically caches static pages?

Eh . . . it generates static html files from static pages. If a static html file exists, mod_rewrite will redirect the visitor to the static page.

This means that TYPO3 will not be loaded at all.

Your server will have less work to do and will use less power. This helps to keep our earth cool ;-)

Page 26: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

3

What's in it for me?

23000 %speed improvement for static pages

Page 27: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

4

Key features

it's fscking fast

transparent to the user

works in the existing domain and port

sends cache headers

Page 28: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

5

Requirements

apache

mod_rewrite

realurl

mod_expires

required recommended

Page 29: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

6

in the beginning

Tim Lochmüller: fl_staticfilecache (cheers Tim! ;-)

the 'not so good':xclasses

not my kind of code

backend module? But why?

need to actively clear cache

cache headers don't work with static html

Page 30: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

7

The GOOD!

Using mod_rewrite is a great idea!

TYPO3 is very good at serving dynamic content

Apache is very good at serving static html files

Impressive speed improvement possible

Page 31: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

8

Questions

What can be cached?

How can we invalidate / clear cache?

How can we make cache headers work?

What hooks exist?

Where are they?

When are they called?

Does it work?

Does it really work?

Page 32: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

9

Shadow TYPO3

create hook:insertPageIncache

clear hooks:clearCachePostProc

clearPageCacheEval

tslib_fe-PostProc

catch Ctrl+Shift+Reload:tslib_fe-PostProc

What can we cache? When do we clear cache? We can follow TYPO3's behaviour.

If TYPO3 caches a page, we will create a static version, if TYPO3 clears the page cache, we will delete the static version.

Page 33: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

10

How does it work?

cache pages to static html file

use mod_rewrite to:check if a static html file exists, if it does . . .

redirect the request to the static html file, otherwise . . .

fall through to the next rewrite rule.

Page 34: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

11

What does it look like?

RewriteCond -> RewriteRule [L]

RewriteCond %{DOCUMENT_ROOT}/typo3temp/tx_ncstaticfilecache/%{HTTP_HOST}/%{REQUEST_URI}index.html -f

RewriteRule .* typo3temp/tx_ncstaticfilecache/%{HTTP_HOST}/%{REQUEST_URI} [L]

Mod rewrite works with rulesets consising of zero or more rewrite conditions followed by a rewrite rule.

If all conditions are met, the rule is executed.If one of the conditions is not met, the ruleset is

terminated. mod rewrite moves on to the next ruleset.

The Rewrite condition checks (amongst other things) the staticfilecache path to see if a static html file exists.

Other things it checks for:RewriteCond %{HTTP:Pragma} !no-cacheRewriteCond %{HTTP:Cache-Control} !no-cacheRewriteCond %{HTTPS} offRewriteCond %{HTTP_COOKIE} !be_typo_user [NC]RewriteCond %{REQUEST_METHOD} GETRewriteCond %{QUERY_STRING} ^$If it does, it rewrites the request to the static file.

Page 35: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

12

Other conditions

RewriteCond %{HTTP:Pragma} !no-cache

RewriteCond %{HTTP:Cache-Control} !no-cache

RewriteCond %{HTTPS} off

RewriteCond %{HTTP_COOKIE} !be_typo_user [NC]

RewriteCond %{REQUEST_METHOD} GET

RewriteCond %{QUERY_STRING} ^$

Check for Ctrl+Shift+ReloadDon't cache HTTPS traffic by default. You may

choose to comment out this option if your site runs fully on https. If your site runs mixed, you will not want https traffic to be cached in the same typo3temp folder where it can be requested over http.

NO backend user is logged in. Please not that the be_typo_user expires at the end of the browser session. So, although you have already logged out of the backend, you will still have to either restart your browser or remove the cookie manually for this rule to work.

We only redirect GET requestsWe only redirect URI's without query stringsNC: non case sensitive

Page 36: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

13

Enabling cache headers?

Static files don't send cache headers.

If you are using mod_rewrite, chances are . . .

. . . you also have access to mod_expires.

Static files do not send cache headers. Of course there is meta: expires. But it's a hassle. May be implemented at a later stage if tests show it works as it should.

Use mod_expires to send the cache headers.This is possible because of realurl.

Page 37: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

14

Mod Expires

If realurl is used we have a cache directory structure resembling the URI.

Every cached file lives in it's own directory.

Use the timeout value for the page . . .

. . . to write a .htaccess file for every index.html.<IfModule mod_expires.c>

ExpiresActive on

ExpiresByType text/html M60

</IfModule>

We use the page timeout value to write a htaccess file.

We use the format 60M for example. This means that cache headers will be sent containing an expiry time of 'file modification time' + 60 seconds.

So as time progresses the cache headers sent for the file will send shorter expiry times.

In combination with the cleaner script we can build an up to date cache.

We select the expired pages by querying the pages table with: 'expires <='.$GLOBALS['EXEC_TIME']);

You need to run your cache cleaner script in a cron job with your lowest cache expiry time.

Page 38: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

15

Cleaning up . . .

Once the static file exists, it will stay.

Eventually it will expire.

When the browser requests a fresh page, it will still get the old one.

Unless we manually remove the static file after it has expired.

Page 39: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

16

The numbers

ab

apache bench

try this on your moms server!

Page 40: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

17

Server torture

requests / s time / request ms transfer rate Kbps test time sdynamic 6.3 158.68 83.82 158.68TYPO3 cache 23.28 42.96 283.13 42.96static 5346.62 0.19 71805.12 0.19

ab -c 100 -n 1000 http://www.fictive.org/gpl-short/

dynamic

TYPO3 cache

static

ab is a tool for benchmarking your Apache Hypertext Transfer Protocol (HTTP) server. It is designed to give you an impression of how your current Apache installation performs. This especially shows you how many requests per second your Apache installation is capable of serving.

-c concurrency Number of multiple requests to perform at a time.

Default is one request at a time.

Page 41: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

18

Serve more requests per second

dynamic TYPO3 cache static

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

5500

requests / s

6 23

Page 42: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

19

Serve requests faster

dynamic TYPO3 cache static

0102030405060708090

100110120130140150160

mean time per request in ms

0.2

Page 43: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

20

Performance increase factor

230times as quick

Page 44: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

21

Let's see that again!

23000 %speed improvement

Page 45: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

22

I have a dream . . .

That one day . . .

. . . all static file cache extensions . . .

. . . will become one!

called: 'staticfilecache'

me scratch!

Page 46: Static file caching using realurl, mod rewrite and mod expires. · Static files don't send cache headers. If you are using mod_rewrite, chances are . . .. . . you also have access

23

Tools

eclipse

svn

firefox + live http headers

cc_devlog

grep, sed et al