Upload
matthias-wahl
View
104
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Talk about building real shared nothing web backends using crate data on Web Backend Meetup May 2014
Citation preview
Shared Nothing Web Backends
Matthias Wahl - developer @ crate.io
Agenda
Shared Nothing
Crate
Shared Nothing Architectures using Crate
Cluster Workout
Prepare for Workout
1.) bash -c "$(curl -L try.crate.io)"
https://cdn.crate.io/downloads/releases/crate-0.38.2.tar.gz
2.) sh$ bin/crate
0.) share our “craty” WLAN pw: select*fromcrate;
Prepare for Workout
3.) start up the twitter tutorial
Shared Nothing
Shared Nothing
Wikipedia: !
a distributed computing architecture in which each
node is independent and self-sufficient, and there is no
single point of contention across the system. More
specifically, none of the nodes share memory or disk
storage.
Shared Nothing
Origin:
!
Micheal Stonebraker
“The Case for Shared Nothing” (1986)
!
http://pdf.aminer.org/000/255/770/
the_case_for_shared_nothing.pdf
Shared Nothing
Principle of most functional languages
Shared state/resources (memory) is what makes threading hard
No SPOF
Easy concurrency
DB
Cache
App App App App
DB - Replica
Cache Cache
Bad Idea
DB
Cache
App App App App
DB - Replica
Cache Cache
Bad Idea
Crate
Austria and Berlin based Startup
founded 2013
open source since april https://github.com/crate/crate
Crate
shared nothing massively scalable datastore
standing on the shoulders of giants
Crate
Crate
automatic sharding and replication
(semi-) structured models
blob support
SQL query language
Crate
powerful fulltext search capabilities
complex aggregations in near real-time
- no joins
Crate
crate-java
crate-jdbc
crate-python (+SQLAlchemy)
crate-ruby
…
CRATE Clients
Shared Nothing with Crate
data and query execution is distributed among all nodes
no master/slave - all nodes are equal
automatic sharding & replication
Shared Nothing with Crate
example architecture
crate
app
crate
app
crate
app
…
Load Balancer
Shared Nothing with Crate
horizontally scalable as hell
still flexible enough for complex applications
high availability for your whole stack
Shared Nothing with Crate
create table blogpost ( id string primary key, created timestamp, text string, creator string )
Shared Nothing with Crate
from sqlalchemy import Column, String, DateTime from microblog.model import Base import uuid !!def genuuid(): return str(uuid.uuid4()) !!class BlogPost(Base): ! __tablename__ = 'blogpost' ! id = Column(String, default=genuuid, primary_key=True) text = Column('text', String, nullable=False) creator = Column('creator', String, nullable=False) created = Column('created', DateTime, nullable=False)
Shared Nothing with Crate
@rpcmethod_route(request_method="GET") def list(self): """ Return all blogposts """ query = DBSession.query(BlogPost).order_by(BlogPost.created.desc()) blogposts = query.all() result = [] for post in blogposts: result.append({'id': post.id, 'created': post.created.isoformat(), 'text': post.text, 'creator': post.creator}) return {"data": {"blogposts": result}}
Shared Nothing with Crate
get into detail: !
http://lovelysystems.github.io/lovely.microblog/
Cluster Workout!!!
CRATE
Thank you
web: https://crate.io/
github: https://github.com/crate
twitter: @cratedata
IRC: #crate
stackoverflow tag: cratedata