Talking About Twitter s Architecture

Talking About Twitter'sArchitecture

Grupo 9:54807 João Reis55455 Jaime Bernardo57743 António Sousa

Talking About Twitter'sArchitecture

The importance of scalable architectures!

What is twitter?

The Need

The Need

The Need

The Need

How do we keep everyone updated about our life?

How do we keep updated about everyone's life?

The Solution

Microblogging

A Great Idea

Hammer View

What Happened?

Twitter got popular

As it often happens...

twitter's Bottlenecks

twitter's Bottlenecks• no monitoring, no graphs, no statistics

twitter's Bottlenecks• no monitoring, no graphs, no statistics

• memcached

• single mySQL database

twitter's Bottlenecks• no monitoring, no graphs, no statistics

• memcached

• single mySQL database

• Abuses

twitter's Bottlenecks• no monitoring, no graphs, no statistics

• memcached

• single mySQL database

• Abuses

• Plan to partition in the future

how can they solve this?

Solution

completely re-write the application

how are they doing it?

Databases

• more machines • index everything

• denormalize

Starling

• Distributed Queuing • Transactional Playback

• Fast

• Simple

• Speaks Memcache's Language

• 100% Pure Ruby

API

• Representational State Tranfer (REST) • HTTP Requests

• XML

• RSS

• ATOM

Use The Community

Open Source

StarlingAPI

Feedback

Abuses

• Build tools to detect

• Be ruthless. Delete them as users.

Performance Monitoring

Has it worked?

New services

Hundreds of updates per minuteBesides normal usage

Is it finished?

It's a work in progress

Still has some downtime, from time to time.

Regular updates.

Lessons to learn

Lessons to learn• Scale early.

Lessons to learn• Scale early.• Create performance monitors.

Lessons to learn• Scale early.• Create performance monitors.• Involve the community.

Lessons to learn• Scale early.• Create performance monitors.• Involve the community. • Foresee abusive uses.

Lessons to learn• Scale early.• Create performance monitors.• Involve the community. • Foresee abusive uses.• Don't use inadequate

architectures.

QUESTIONS?