Commandments of scale

Baltimore, MD

Thank you! Thank you!Organizers:

Jason DixonKatherine Jeschke

Brand/Design & A/VMiguel MontanezJess Brown

PhotographyClinton WolfeCiprian Tutu

StaffDeAnna GibsonPat NortonSherry Schlossnagle

A Brief History of Computing

1877 switchboard... lessons? boys are childish jerks, hire women.

A Brief History of Computing

1877 switchboard... lessons? boys are childish jerks, hire women.

sundial likely of ancient greek origindigital clock invented in 1972 by the watchmaking company called HamiltonMD5 hashing by Ronald RivestAmazon dynamo (consistent hashing of data across distributed nodes)amongst others: Werner Vogels

sundial likely of ancient greek origindigital clock invented in 1972 by the watchmaking company called HamiltonMD5 hashing by Ronald RivestAmazon dynamo (consistent hashing of data across distributed nodes)amongst others: Werner Vogels

sundial likely of ancient greek origindigital clock invented in 1972 by the watchmaking company called HamiltonMD5 hashing by Ronald RivestAmazon dynamo (consistent hashing of data across distributed nodes)amongst others: Werner Vogels

/* Process each 16-word block. */ For i = 0 to N/16-1 do

/* Copy block i into X. */ For j = 0 to 15 do Set X[j] to M[i*16+j]. end /* of loop on j */

/* Save A as AA, B as BB, C as CC, and D as DD. */ AA = A BB = B

CC = C DD = D

/* Round 1. */ /* Let [abcd k s i] denote the operation a = b + ((a + F(b,c,d) + X[k] + T[i]) <<< s). */ /* Do the following 16 operations. */ [ABCD 0 7 1] [DABC 1 12 2] [CDAB 2 17 3] [BCDA 3 22 4] [ABCD 4 7 5] [DABC 5 12 6] [CDAB 6 17 7] [BCDA 7 22 8] [ABCD 8 7 9] [DABC 9 12 10] [CDAB 10 17 11] [BCDA 11 22 12] [ABCD 12 7 13] [DABC 13 12 14] [CDAB 14 17 15] [BCDA 15 22 16]

/* Round 2. */ /* Let [abcd k s i] denote the operation a = b + ((a + G(b,c,d) + X[k] + T[i]) <<< s). */ /* Do the following 16 operations. */ [ABCD 1 5 17] [DABC 6 9 18] [CDAB 11 14 19] [BCDA 0 20 20] [ABCD 5 5 21] [DABC 10 9 22] [CDAB 15 14 23] [BCDA 4 20 24] [ABCD 9 5 25] [DABC 14 9 26] [CDAB 3 14 27] [BCDA 8 20 28] [ABCD 13 5 29] [DABC 2 9 30] [CDAB 7 14 31] [BCDA 12 20 32]

/* Round 3. */ /* Let [abcd k s t] denote the operation a = b + ((a + H(b,c,d) + X[k] + T[i]) <<< s). */ /* Do the following 16 operations. */ [ABCD 5 4 33] [DABC 8 11 34] [CDAB 11 16 35] [BCDA 14 23 36] [ABCD 1 4 37] [DABC 4 11 38] [CDAB 7 16 39] [BCDA 10 23 40] [ABCD 13 4 41] [DABC 0 11 42] [CDAB 3 16 43] [BCDA 6 23 44] [ABCD 9 4 45] [DABC 12 11 46] [CDAB 15 16 47] [BCDA 2 23 48]

/* Round 4. */ /* Let [abcd k s t] denote the operation a = b + ((a + I(b,c,d) + X[k] + T[i]) <<< s). */ /* Do the following 16 operations. */ [ABCD 0 6 49] [DABC 7 10 50] [CDAB 14 15 51] [BCDA 5 21 52] [ABCD 12 6 53] [DABC 3 10 54] [CDAB 10 15 55] [BCDA 1 21 56] [ABCD 8 6 57] [DABC 15 10 58] [CDAB 6 15 59] [BCDA 13 21 60] [ABCD 4 6 61] [DABC 11 10 62] [CDAB 2 15 63] [BCDA 9 21 64]

/* Then perform the following additions. (That is increment each of the four registers by the value it had before this block was started.) */ A = A + AA B = B + BB C = C + CC D = D + DD

end /* of loop on i */

sundial likely of ancient greek origindigital clock invented in 1972 by the watchmaking company called HamiltonMD5 hashing by Ronald RivestAmazon dynamo (consistent hashing of data across distributed nodes)amongst others: Werner Vogels

/* Process each 16-word block. */ For i = 0 to N/16-1 do

/* Copy block i into X. */ For j = 0 to 15 do Set X[j] to M[i*16+j]. end /* of loop on j */

/* Save A as AA, B as BB, C as CC, and D as DD. */ AA = A BB = B

CC = C DD = D

/* Round 1. */ /* Let [abcd k s i] denote the operation a = b + ((a + F(b,c,d) + X[k] + T[i]) <<< s). */ /* Do the following 16 operations. */ [ABCD 0 7 1] [DABC 1 12 2] [CDAB 2 17 3] [BCDA 3 22 4] [ABCD 4 7 5] [DABC 5 12 6] [CDAB 6 17 7] [BCDA 7 22 8] [ABCD 8 7 9] [DABC 9 12 10] [CDAB 10 17 11] [BCDA 11 22 12] [ABCD 12 7 13] [DABC 13 12 14] [CDAB 14 17 15] [BCDA 15 22 16]

/* Round 2. */ /* Let [abcd k s i] denote the operation a = b + ((a + G(b,c,d) + X[k] + T[i]) <<< s). */ /* Do the following 16 operations. */ [ABCD 1 5 17] [DABC 6 9 18] [CDAB 11 14 19] [BCDA 0 20 20] [ABCD 5 5 21] [DABC 10 9 22] [CDAB 15 14 23] [BCDA 4 20 24] [ABCD 9 5 25] [DABC 14 9 26] [CDAB 3 14 27] [BCDA 8 20 28] [ABCD 13 5 29] [DABC 2 9 30] [CDAB 7 14 31] [BCDA 12 20 32]

/* Round 3. */ /* Let [abcd k s t] denote the operation a = b + ((a + H(b,c,d) + X[k] + T[i]) <<< s). */ /* Do the following 16 operations. */ [ABCD 5 4 33] [DABC 8 11 34] [CDAB 11 16 35] [BCDA 14 23 36] [ABCD 1 4 37] [DABC 4 11 38] [CDAB 7 16 39] [BCDA 10 23 40] [ABCD 13 4 41] [DABC 0 11 42] [CDAB 3 16 43] [BCDA 6 23 44] [ABCD 9 4 45] [DABC 12 11 46] [CDAB 15 16 47] [BCDA 2 23 48]

/* Round 4. */ /* Let [abcd k s t] denote the operation a = b + ((a + I(b,c,d) + X[k] + T[i]) <<< s). */ /* Do the following 16 operations. */ [ABCD 0 6 49] [DABC 7 10 50] [CDAB 14 15 51] [BCDA 5 21 52] [ABCD 12 6 53] [DABC 3 10 54] [CDAB 10 15 55] [BCDA 1 21 56] [ABCD 8 6 57] [DABC 15 10 58] [CDAB 6 15 59] [BCDA 13 21 60] [ABCD 4 6 61] [DABC 11 10 62] [CDAB 2 15 63] [BCDA 9 21 64]

/* Then perform the following additions. (That is increment each of the four registers by the value it had before this block was started.) */ A = A + AA B = B + BB C = C + CC D = D + DD

end /* of loop on i */

sundial likely of ancient greek origindigital clock invented in 1972 by the watchmaking company called HamiltonMD5 hashing by Ronald RivestAmazon dynamo (consistent hashing of data across distributed nodes)amongst others: Werner Vogels

1664 Robert Hooke: sound transmission through a distended wire1969 first known FIFO impl. Peter Alfke at Fairchild SemiconductorEavesdropping was developed by “bad children”1987 “Exploiting virtual synchrony in distributed systems” by Birman and Joseph

1664 Robert Hooke: sound transmission through a distended wire1969 first known FIFO impl. Peter Alfke at Fairchild SemiconductorEavesdropping was developed by “bad children”1987 “Exploiting virtual synchrony in distributed systems” by Birman and Joseph

1664 Robert Hooke: sound transmission through a distended wire1969 first known FIFO impl. Peter Alfke at Fairchild SemiconductorEavesdropping was developed by “bad children”1987 “Exploiting virtual synchrony in distributed systems” by Birman and Joseph

1664 Robert Hooke: sound transmission through a distended wire1969 first known FIFO impl. Peter Alfke at Fairchild SemiconductorEavesdropping was developed by “bad children”1987 “Exploiting virtual synchrony in distributed systems” by Birman and Joseph

1664 Robert Hooke: sound transmission through a distended wire1969 first known FIFO impl. Peter Alfke at Fairchild SemiconductorEavesdropping was developed by “bad children”1987 “Exploiting virtual synchrony in distributed systems” by Birman and Joseph

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

RFC1901

1854 Ignazio Porro patented this binocular designmailboxes Paris in the 18th century, Britian in the 19th century1987 the International Organization for Standardization set for the OSI model16th century, popularized in the Elizabethan era for hangings1956 IBM introduced the harddriveGraphs? perhaps cavemen.Pressure measurement: Evangelista Torricelli in 1643 and te U-Tube by Christian Huygens in 1661SMS concept was developed in the Franco-German GSM cooperation in 1984Harold Black made the first negative feedback amplifier in the 1930s... inspiring at least me.

the construction of monitoring systems

Keep on keepin’ on.

Eventually we’ll get it (more) right

We need to keep improving.

Internet engineering is still very young.

Ten Commandments

ofWeb Scale

@postwait: the “Webscale! Wow!” campaign

I. Thou shalt not claim hugeness falsely

1TB is not a lot of data100 million tuples is not a large table1000 write transactions/second is not a lotA distributed system must contain 3 nodesA geographically distributed system must exceed one of:

1+ node in each of 3+ locations3+ nodes in each of 2+ locations

I. Thou shalt not claim hugeness falsely

1TB is not a lot of data

100 million tuples is not a large table

1000 write transactions/second is not a lot

A distributed system must contain 3 nodes

A geographically distributed system must exceed one of:

1+ node in each of 3+ locations

3+ nodes in each of 2+ locations

II. Thou shalt not worship shininess

shiny does not mean goodgood means goodcope

II. Thou shalt not worship shininess

shiny does not mean good

good means good

cope

III. Thou shalt always remain recoverable.

data is preciousmap acceptable business liability to technical tolerances in systemsnever trust your systemstest failures; force failuresthis is particularly true in large scale systemsrecoverability of petabytes is “non-trivial”

III. Thou shalt always remain recoverable.

data is preciousmap acceptable business liability to technical tolerances in systemsnever trust your systems

test failures; force failuresthis is particularly true in large scale systems

recoverability of petabytes is “non-trivial”

IV. Respect that different problems may need different solutions

a set P of problems is being solved by technology T.Pnew is added to P and T is found insufficient.a new technology S solves Pnew wellso now P must be solved by S.This only proves: flawed logic

Results in: poorly fitted solutions

IV. Respect that different problems may need different solutions

a set P of problems is being solved by technology T.Pnew is added to P and T is found insufficient.a new technology S solves Pnew wellso now P must be solved by S.

This only proves: flawed logicResults in: poorly fitted solutions

V. Thou shalt not covet thy neighbor’s technology.

unless one wish to covet their problems and failures to.do not learn by copying technologies and implementations.learn by adapting designs and sharing experiences.

V. Thou shalt not covet thy neighbor’s technology.

unless one wish to covet their problems and failures to.do not learn by copying technologies and implementations.learn by adapting designs and sharing experiences.

VI. That which cannot be measured cannot be scaled.

in a world where nothing is implemented, only designed,empirical measurement is the closest thing to realitywithout measurement you are blindmeasure everything

forget nothingmeasurement is a prerequisite of success

VI. That which cannot be measured cannot be scaled.

in a world where nothing is implemented, only designed,empirical measurement is the closest thing to realitywithout measurement you are blindmeasure everything

forget nothingmeasurement is a prerequisite of success

VII. Operate even in the greatest of storms

overall robustness is achieved through individual fragility

the architecture should be highly componentized

each component should perform well or not at all

the overall system should operate despite removal of as many components as possible

VII. Operate even in the greatest of storms

overall robustness is achieved through individual fragility

the architecture should be highly componentized

each component should perform well or not at all

the overall system should operate despite removal of as many components as possible

VIII. Embrace the math of engineers

89 + 23 = 11089 + 23 != 11.2validate all your models, calculations and extrapolations with back-of-the-envelope maththou shalt have adequate quantity of envelopes.

VIII. Embrace the math of engineers

89 + 23 = 11089 + 23 != 11.2validate all your models, calculations and extrapolations with back-of-the-envelope maththou shalt have adequate quantity of envelopes.

IX. Thou shalt perform most favourably

scaling out is required in some problem spacespoor performance is not required in anyscaling out (or the ability to do so) isno excuse for poor performance

IX. Thou shalt perform most favourably

scaling out is required in some problem spacespoor performance is not required in anyscaling out (or the ability to do so) isno excuse for poor performance

X. Thou shalt be consistent... eventually

eventual consistency only works when it is eventually consistent.building these systems is hard.thou shalt not use those yet to be proved correct.

X. Thou shalt be consistent... eventually

eventual consistency only works when it is eventually consistent.building these systems is hard.thou shalt not use those yet to be proved correct.

XI. Thou shalt share with your brothers and sisters.

(one for the road)we can learn by failureswe can learn from the experiences of our peersI suggest a healthy mix

XI. Thou shalt share with your brothers and sisters.

we can learn by failureswe can learn from the experiences of our peersI suggest a healthy mix

Baltimore, MD