Upload
jerjou
View
113
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Talk given at Seattle Tech Forum on Dev 15, 2010 at Bellevue City Hall.
Citation preview
Computing at Scale:Data Exploration
Jerjou Cheng, Barry Brumitt
HELLO
/jur'jō/ɯoɔ˙ǝlƃooƃ@noɾɹǝɾ
MY NAME IS
Developer Programs Engineer(Developer Relations)
Google Storage
Michael PIERROThttp://www.freephotobank.org/v/sky-stars/cloud/Cloud-19.jpg.html
Computing at Scale:Data Exploration
Jerjou Cheng, Barry Brumitt
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
Introductions
Who are theseservices for?
A World without Clouds
• Build a web applicationoStartup costsoMaintenance / reliabilityoScaling
Michael Scheltgenflickr.com/mscheltgen/
Google App Engine
Easy to startEasy to maintainEasy to scale
Users
gigy Socialize - traffic
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
A World without Clouds
• Store data• Reliability• Sharing• Large objects
Michael Scheltgenflickr.com/mscheltgen/
Google Storage for Developers
• Google infrastructure• You control access to your data• Store massive data in Google's cloud• Easy interface
Example
Internal use cases
• Content hosting
• Sharing
• Data Import Google BigQuery
Google Prediction API
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
Prediction API
• Cloud-hosted machine learning as service• Simple interface over complex analysis• Predict results in real-time
How does it work?
"english" The quick brown fox jumped over the lazy dog.
"english" To err is human, but to really foul things up you need a computer.
"spanish" No hay mal que por bien no venga.
"spanish" La tercera es la vencida.
"english" To be or not to be, that is the question.
"spanish" La fe mueve montañas.
The Prediction APIfinds relevantfeatures in the sample data during training.
The Prediction APIlater searches forthose featuresduring prediction.
Prediction API
1. Upload
2. Train
Upload your training data toGoogle Storage
Build a model from your data
Make new predictions
prediction/v1.1/training?data={}POST : a training request
prediction/v1.1/training/{}/predictGET : model infoPOST : a prediction request
Use the API, gsutil or any compatible utility to upload your data to Google Storage
3. Predict
Example
Prediction API
• Google's machine learning algorithms• Available as RESTful HTTP service• Predict results in real-time
Overview
• Google App Engine• Google Storage for Developers• Prediction API• BigQuery
GET /information HTTP/1.0
• To request access and get more information, go to:o http://code.google.com/appengineo http://code.google.com/apis/bigqueryo http://code.google.com/apis/predicto http://code.google.com/apis/storage