Upload
nosqlmatters
View
457
Download
4
Embed Size (px)
Citation preview
© 2014 IBM Corporation
Building Applications for SuccessGlynn Bird, Developer Advocate
Agenda
• Application Development
• Scaling
• Real-world examples & how can Cloudant help?
Introduction
@glynn_bird [email protected]
Glynn BirdDeveloper AdvocateCloudant
http://glynnbird.com
Development - Go-to tools
Basic
Pascal
C
C++
Forth
PHP4Java
PHP5
ObjC
Node.js
1980s 1990s 2000s 2010s
File System
NoSQL
SQL
Scalability?
• Pick your tools of choice• PHP + MySQL, Ruby + Mongo
• Build on your machine
• Deploy!
• Scalability is an ops problem?
• Hint: “no”
Scaling tip #1 - Keep things separate
• Separate • load balancing• app• database• cache
• Separation of layers allows independent scaling
http://bit.ly/10xGe2YGoogle’s first server rack
Scaling tip #2 - Keep app servers stateless
• Store all persistent data in a database
• Don’t be tempted to store sessions on local disk
• Stateless app servers can be created and destroyed at will
http://bit.ly/10xGe2Y
IBM 305 RAMAC 5Mb Disk - 1957
Scaling tip #3 - queue-driven workflow
• Store tasks in queues
• Workers servers consume the queue
• Queues allow workers to be scaled at will
http://bit.ly/1Aycjse
Scaling tip #4 - Automate everything
• No need to manually configure servers anymore
• Workers servers consume the queue
• It should be as easy to deploy N servers as 1 server
http://bit.ly/1Aycjse
What does failure to scale look like?
http://bit.ly/1Aycjse
What does failure to scale look like?
• increased latency
• downtime
• high maintenance costs
• data loss
• disgruntled customers
http://bit.ly/1Aycjse
Cloudant Highlights• Founded in 2009 by data scientists:
Hoffman, Miller & Kocoloski
• 20,000+ freemium and 225+ paying customers
• Key collaboration & investment agreement with Samsung Electronics
• HQ’d in Boston with offices/presence in Seattle, San Francisco, Bristol, UK & Seoul, SK
• 34 global locations on 4 hosting providers
• Acquired by IBM in February 2014 into the Information Management software group
Cloudant NoSQL DBaaS
Indexed JSON
Storage
Geo-Load Balancing
App Layer / Browser / Mobile App
Physical Storage @ IBM SoftLayer, Rackspace, Microsoft Azure, AWS
Cloudant JSON API
JSON CRUD
Multi- Master
Replication
Device Sync
Lucene Search
2D, 3D Geospatial
Incremental MapReduce Monitoring Security
IO Control
Managed for you 24x7 by Cloudant developer-operators
REST (HTTPS)
How can Cloudant help you to scale?
• Scales massively & elastically• Handle millions of daily active users
• Guaranteed performance & up time• Removes risk of project & SLA delivery failure
• More agile development for web & mobile• No rigid schemas to slow development
• Managed for you 24x7• Stay focused on new development, no DB administration
Cloudant empowers clients to get their applications to market faster in a cost-effective, hassle-free delivery model with guaranteed performance and support.
Example 1 - IoT Data Logging
• Temperature sensor £1.50
• Raspberry Pi £25
• Network attachedthermometer
DS18B20
DS18B20
Software
• Bash script
• Where should we put the data ?
sensorid=`find /sys/bus/w1/devices/* -type l | grep -o '[0-9]*-.*$' | head -n 1`filename="/sys/bus/w1/devices/$sensorid/w1_slave"T=`tail -n 1 "$filename" | grep -o "[0-9]*$"`t=${T:0:2}.${T:2:3}unix=`date "+%s"`d=`date "+%Y-%m-%d %H:%M:%S %Z"`json="{ \"ts\": $unix, \"date\": \"$d\", \"temperature\": $t, \"sensor_id\": \"$sensorid\" }"
DS18B20
Storage
• Local CouchDB
• Installed with one line
sensorid=`find /sys/bus/w1/devices/* -type l | grep -o '[0-9]*-.*$' | head -n 1`filename="/sys/bus/w1/devices/$sensorid/w1_slave"T=`tail -n 1 "$filename" | grep -o "[0-9]*$"`t=${T:0:2}.${T:2:3}unix=`date "+%s"`d=`date "+%Y-%m-%d %H:%M:%S %Z"`json="{ \"ts\": $unix, \"date\": \"$d\", \"temperature\": $t, \"sensor_id\": \"$sensorid\" }"curl -X POST -H "Content-Type: application/json" -d "$json" "http://localhost:5984/logger/"
Where does Cloudant fit in?
• Replicate to Cloudant
• Scale by adding more sensors
• Data collection can be offline
• Sync when connected
Demo
• http://celsius.mybluemix.net
• Replication• collect data at the edge, replicate when online
• Cloudant can store data at massive scale with high availability
• Time-series data is a great fit for Cloudant
Why is Cloudant a good fit?
• MapReduce indexes on date field
• Allows range querying with grouping e.g. find min/max/avg between two dates, grouped by year/month/day/hour etc.
Time-series data in Cloudant
{ "_id": "00339bc7", "_rev": "1-0ef01b53", "ts": 1411723490, "date": "2014-09-26 09:24:50 UTC", "temperature": 20.937, "sensor_id": "28-000006746cde" }
[2014,9,26,9,24,50] ---> 20.937
• Mobile game
• Stores games state, usage stats, achievements etc. locally
• Needs to be cross-platform
Example #2 - Mobile applications
• In-browser, cross-platform datastore
• CouchDB compatible
• Syncs with Cloudant
Option 1 - PouchDB + HTML5
http://pouchdb.com/
• iOS and Android native libraries
• Local storage + query
• Syncs to and from Cloudant
Option 2 - CloudantSync
https://cloudant.com/product/cloudant-features/sync/
One database per user
• Each user’s data has it’s own access control
• Each database is relatively small; small enough to fit on phone
• Scales massively; hundreds of thousands of concurrent users
One database per user
https://cloudant.com/product/cloudant-features/sync/
• Business Card collection app
• Native Android App
• Companion Web App
Demo
https://cloudant.com/product/cloudant-features/sync/
How did that work?
pouchdb
Android Phone App
IBM BlueMix
Web App
var db = new PouchDB("myfirstdatabase");
db.post( { a:1, b: "2", c: true } );
db.sync(URL, { live: true });
What about the code?
https://cloudant.com/product/cloudant-features/sync/
• Replication is a hard problem. PouchDB + Cloudant solve it
• One database per-user is a great way to scale mobile apps
Why is Cloudant a good fit?
• Building with Cloudant as your data layer has become a “go to” development solution
• Building with Cloudant builds in scale from day one• large data sets• lots of smaller data sets
• Cloudant syncs!
Conclusion
https://cloudant.com/product/cloudant-features/sync/