dataops BCN 2019 - Size does matter · 2019-06-27 · PARQUET Athena PRESTO dt=2018-08-21-01-51 dt=2018-08-21-01-50 dt=2018-08-21-01-49 dt=2018-08-21-01-48 dt=2018-08-21-01-47 Fridge

Lucas Ceballos Size does matter @ Dataops BCN 2019

�1


�2

Lucas Ceballoslinkedin.com/in/lucas-ceballos

CTO @ Headway Born @ ArgentinaCTO @ Smadex


�3

SIZEDOES MATTER


�3

SIZEDOES MATTERHOW TO BUILD A SERVERLESS PETA-BYTE SCALE DATABASE

Lucas Ceballos Size does matter @ Dataops 2019

SIZEDOES MATTER �4

CHAPTER 1CHAPTER 2

CHAPTER 3 - 5CHAPTER 6

CONTEXTPROBLEMATTEMPTSQ&A



CONTEXTCHAPTER 1



D I G I T A L A N D A P P G R O W T H M A R K E T I N G C O M P A N Y



“We help our clients to grow through digital advertising”



DeliverYou



NEW USERSUSER ACQUISITION

NEW ORDERSUSER RE-ENGAGEMENT



IMPRESSION+50K / SEG

CLICK+5K / SEG

CONVERSION+15K / HR

AD L IFE CYCLE


SIZEDOES MATTER

Click IDImpression ID

�10


CLICK+5K / SEG

CONVERSION+15K / HR

AD L IFE CYCLE




CLICK+5K / SEG

CONVERSION+15K / HR

AD L IFE CYCLE

△t△t




CLICK+5K / SEG

CONVERSION+15K / HR

AD L IFE CYCLE

△t△t<1 min 1 - ∞ min




CLICK+5K / SEG

CONVERSION+15K / HR

AD L IFE CYCLE

△t△t<1 min 1 - ∞ min




CLICK+5K / SEG

CONVERSION+15K / HR

AD L IFE CYCLE

△t△t<1 min Attribution Window



In the world of digital attribution, an attribution window is a time period during which eligible conversion events can be claimed. The conversion events can be an app install, added to cart, or purchase.

ATTRIBUTION WINDOW:



UNDER THE HOOD

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks

Incoming Conversions

W R

attribution window



UNDER THE HOOD

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks


W R

attribution window



UNDER THE HOOD

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks


W R

attribution window



UNDER THE HOOD

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks


W R

attribution window



UNDER THE HOOD

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks


W R

attribution window



1808210149a523932a4ceaed

CLICK ID

Date & Time (up to minute) Random Hexadecimal String



{“_id”:"1808210149a523932a4ceaed","version":5,"earn":0,"spent":0,"converted":false,"campaignId":280392,"categoryGroup":"mainstream","publisherId":"10466_581_34_863796162656735232","adSlot":null,"siteId":null,"sellerId":null,"formatId":null,"sourceUid":"59af4b01e9cbfd600085ca9df0e709ed","appId":"","bundleId":null,"deviceId":null,"ip":"170.79.119.28","referer":null,"userAgent":"Mozilla/5.0(Linux;Android8.0;N820Build/KOT49H)AppleWebKit/537.36(KHTML,likeGecko)Chrome/60.0.3112MobileSafari/537.36","countryCode":"BR","countryName":"Brazil","region":"RioGrandedoNorte","city":"Natal","isp":"JDSMedeirosProvedorDeInternet-Me","connectionType":"wifi","latitude":-5.8002,"longitude":-35.2107,"device":"N820","deviceType":"mobile","browser":"Chrome","browserVersion":"60.0.3112","os":"Android","osVersion":"8.0","payout":3.23,"cost":1.87,"share":0,"impressions":0,"clicks":1,"conversions":0,"createdAt":"20180821014947","updatedAt":"20180821014947","date":"2018-08-21T04:49:47.205Z","datetime":"2018-08-2101:49:47","dateUpdated":"2018-08-21T04:49:47.205Z","callbackUrl":"http://click.foo-bar.com/fass/?atoken=f8ad6505fa1b53fc948c8543dc0afc7a05296200&version=1&c=tracker.postback&click_id={sid}"}

CLICK TAXONOMY

JSONFORMAT

1KBNO COMP.

650BGZIP



BIGGERATTRIBUTION WINDOW

LONGERCLICK RETENTION PERIOD

=



THE PROBLEMCHAPTER 2



UNDER THE HOOD

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks


W R

attribution window



500.000.000 x 650

x 30

Daily clicks Avg. click size Days/month

~ 9 TB / month



EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks

Incoming Events

W R





COST

2

1 3LATENCYSIZE



FIRST ATTEMPTCHAPTER 3



FIRST ATTEMPT

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks

Incoming Events

W R



FIRST ATTEMPT

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks

Incoming Events

W R



FIRST ATTEMPT

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks

Incoming Events

W R







2ND ATTEMPTCHAPTER 4



2ND ATTEMPT

EC2

EC2

EC2

EC2

EC2

EC2

EC2

EC2

Incoming Clicks

Incoming Events

W R



FASTSCALABLE FIXED SIZE COSTLY

PROS CONS



10TB x 3 x 1

x 102,4

500M clicks/month Mongo replicas Months Retention GP2 cost/tb/month

~ 6500 USD / month



A NEW HOPECHAPTER 4



REQUIREMENTS

KEY VALUE CHEAP TB SCALE FAST



ALTERNATIVES

REDIS MEMCACHED DYNAMODB CASSANDRA



ALTERNATIVES


In Memory



ALTERNATIVES


In Memory Costly







THINK OUT OF THE BOXCHAPTER 5



t1 t2

attribution window



t1 t2

attribution window

24H



t1 t2

attribution window

24H

90%<24H

10%>24H



t1 t2

attribution window

1H



t1 t2

attribution window

1H

70%<1H

30%>1H



“DIVIDE AND CONQUER”FINAL CHAPTER



H TLast hour clicks

C LD> 1 hour clicks

FREQUENTLY ACCESSED

SMALL NUMBER

INFREQUENTLY ACCESSED

HUGE NUMBER

+ +

= =



H TLast hour clicks

C LD> 1 hour clicks

+ +

= =

Fridge



UNDER THE HOOD

EC2

EC2

EC2

EC2

Incoming Clicks EC2

EC2

EC2

EC2


H T H T

C LDC LDFridge


SIZEDOES MATTER �36H TEC2

EC2

EC2

EC2

Incoming Clicks

CSV

EC2

EC2

EC2

EC2


(450b)

1 hour ~ 9GB

ElasticacheHADES



EC2

EC2

EC2

EC2

Incoming Clicks

C LD

JSON

1m BUFFERING

PARQUET

Fridge

ALFREDDO



EC2

EC2

EC2

EC2


C LDSQS Athena Clicks

H TSQS

Retry

Fridge



EC2

EC2

EC2

EC2



H TSQS

Retry

Fridge



EC2

EC2

EC2

EC2



H TSQS

Retry

Fridge



JSON

PARQUET

AthenaPRESTO dt=2018-08-21-01-51

dt=2018-08-21-01-50

dt=2018-08-21-01-49

dt=2018-08-21-01-48

dt=2018-08-21-01-47

Fridge



JSON

PARQUET


dt=2018-08-21-01-50

dt=2018-08-21-01-49

dt=2018-08-21-01-48

dt=2018-08-21-01-47

Fridge

1808210149a523932a4ceaed



JSON

PARQUET


dt=2018-08-21-01-50

dt=2018-08-21-01-49

dt=2018-08-21-01-48

dt=2018-08-21-01-47

Fridge

1808210149a523932a4ceaed



EC2

EC2

EC2

EC2



H TSQS

Retry

Fridge



443 + 300 + 290 + 140

S3 Storage Elasticache Athena Lambdas

~ 1173 USD / month


�42

[email protected]

Moltes Gracies!Q&A

lucas-ceballos

We’re hiring!

Documents

dataops BCN 2019 - Size does matter · 2019-06-27 · PARQUET Athena PRESTO dt=2018-08-21-01-51 dt=2018-08-21-01-50 dt=2018-08-21-01-49 dt=2018-08-21-01-48 dt=2018-08-21-01-47 Fridge