Illustrated buffer cache

An illustrated guide to the buffer cache

Selena DeckelmannBased on Greg Smith’s “Inside the Buffer Cache”

http://www.westnet.com/∼gsmith/content/postgresql

Databases

Table

Table

Table

Databases are made of tables...

Table$PGDATA/base/

A table is just a directory...

Table$PGDATA/base/

Table

Table

You can have more than one table.

Table$PGDATA/base/

Table

TableFile

File

File

File

... Up to 1 GB

8K8K8K8K8K8K8K...

Made of Made of

Files are made up of 8K blocks.*

* you can change that at compile time.

shared_buffers

shared_buffers

ptr ptr ptr ptr ptr ...

An array of pointers to 8K blocks(and some other stuff)

shared_buffers is configured at database start.You have to restart Postgres to change it.

shared_buffers:the other stuff

FREE FREE FREE FREE FREE FREE

In the beginning...

shared_buffers:the other stuff

FREE FREE FREE FREE FREE FREE456 12 100 612 32 409

shared_buffers:the other stuff

X X X X X X456 12 100 612 32 409

shared_buffers:the other stuff

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409Tag

shared_buffers:the other stuff

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409FLAGS FLAGS FLAGS FLAGS FLAGS FLAGS

Tag

Status

shared_buffers:the other stuff

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED flags flags flags flags flags

Tag

Status

UPDATE mytable (status) VALUES(‘my cat is the best ever’)where user = ‘selenamarie’;

{ }

shared_buffers:the other stuff

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY flags flags flags flags

Tag

Status

COMMIT;{ }

shared_buffers:the other stuff

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY OTHER OTHER OTHER OTHER

Tag

Status

BLAH BLAH BLAH;{ }

shared_buffers:the other stuff

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY OTHER OTHER OTHER OTHER

1 5 5 4 3 0

Tag

StatusUsage

shared_buffers:the lifecycle

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY OTHER OTHER OTHER OTHER

1 5 5 4 3 0

myPgProcess calls:BufferAlloc(‘base/file1’, 100);{ }

shared_buffers:the lifecycle

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY PINNED OTHER OTHER OTHER

1 5 5 4 3 0

myPgProcess calls:BufferAlloc(‘base/file1’, 100);{ }

shared_buffers:the lifecycle

myPgProcess calls:BufferAlloc(‘base/file1’, 101);{ }

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY OTHER OTHER OTHER OTHER

1 5 5 4 3 0

shared_buffers:the lifecycle

myPgProcess calls:BufferAlloc(‘base/file1’, 101);{ }

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY OTHER OTHER OTHER OTHER

1 5 5 4 3 0

Eviction!

Who’s it gonna be?

shared_buffers:the lifecycle

myPgProcess calls:BufferAlloc(‘base/file1’, 101);{ }

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY OTHER OTHER OTHER OTHER

0 5 5 4 3 0

Who’s it gonna be?

Scan the cache!Usage != 0,

so decrement.

shared_buffers:the lifecycle

myPgProcess calls:BufferAlloc(‘base/file1’, 101);{ }

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 409PINNED DIRTY OTHER OTHER OTHER OTHER

0 4 4 3 2 0

Who’s it gonna be?

Usage == 0,Goodbye!

shared_buffers:the lifecycle

myPgProcess calls:BufferAlloc(‘base/file1’, 101);{ }

X X X X X Xbase/file1 base/file2 base/file1 base/file3 base/file1 base/file1

456 12 100 612 32 101PINNED DIRTY OTHER OTHER OTHER PINNED

0 4 4 3 2 1

Who’s it gonna be?

shared_buffers:optimizations*

SELECT pg_relation_size(‘mytable’);VACUUM mytable;{ }

X X X X X X X ...base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ...

456 12 100 612 32 101 891 ...PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ...

0 4 4 3 2 0 1 ...

Result is greater than:

shared_buffers/4

* Version 8.3 and later

shared_buffers:optimizations

SELECT pg_relation_size(‘mytable’);VACUUM mytable;{ }

X X X X X X X ...base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ...

456 12 100 612 32 101 891 ...PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ...

0 4 4 3 2 0 1 ...

256K ringbuffer allocated!

* Version 8.3 and later

shared_buffers:optimizations

SELECT pg_relation_size(‘mytable’);VACUUM mytable;{ }

X X X X X X X ...base/file1 base/file2 base/file1 base/file3 base/file1 base/file1 base/file2 ...

456 12 100 612 32 101 891 ...PINNED DIRTY OTHER OTHER OTHER PINNED OTHER ...

0 4 4 3 2 0 1 ...

256K ringbuffer allocated!

OVERSIMPLIFICATION WARNING!

Safe duringVACUUM

* Version 8.3 and later