Upload
soshi-nemoto
View
194
Download
0
Embed Size (px)
Citation preview
Google BigQuery - The first step -
2016/05/28 Mulodo Vietnam Co., Ltd.
What’s BigQueryOfficial site : https://cloud.google.com/bigquery/docs/
BigQuery is Google's fully managed, petabyte scale, low cost analytics data warehouse.
BigQuery is NoOps—there is no infrastructure to manage and you don't need a database administrator—so you can focus on analyzing data to find meaningful insights, use familiar SQL, and take advantage of our pay-as-you-go model.
→ WDH: SQL like (easy to use), Petabyte scale(for Huge data)
Features of BigQueryColumn oriented Database
Full Scan instead of using Index
Charge for scanned data size
http://static.googleusercontent.com/media/research.google.com/ja//pubs/archive/36632.pdf https://www.flickr.com/photos/mairin/galleries/72157624181197793/https://cloud.google.com/blog/big-data/2016/03/google-bigquery-cuts-historical-data-storage-cost-in-half-and-accelerates-many-queries-by-10x
The first step - overview -1. Preparation (create account)
2. Try BigQuery with prepared data
3. Upload your data to your BigQuery storage and try it.
Preparation steps1. Preparation (create account)
1. Create your Google account
2. Move to Google Cloud platform (has no account)
3. “Try IT Free”
4. “Sign up for free trial”
5. Account registration
6. Move to Google Cloud Platform (login)
7. Select ‘BigQuery’ from left menu (top-left icon)
Create your google account1. Preparation (create account)
1. Create your Google account
https://accounts.google.com/signUp
Move to Google Cloud Platform1. Preparation (create account)
2. Go to Google Cloud platform (has no account) 3. “Try IT Free”
https://cloud.google.com
Move to Google Cloud Platform1. Preparation (create account)
2. Go to Google Cloud platform (has no account) 3. “Try IT Free”
https://cloud.google.com
Move to Google Cloud Platform1. Preparation (create account)
2. Go to Google Cloud platform (has no account) 3. “Try IT Free”
https://cloud.google.com
Account registration1. Preparation (create account)
4. “Sign up for free trial” 5. Account registration
Account registration1. Preparation (create account)
4. “Sign up for free trial” 5. Account registration
<- Credit Card required
Move to Google Cloud Platform1. Preparation (create account)
6. Move to Google Cloud Platform (login) 7. Select ‘BigQuery’ from left menu (top-left icon)
https://cloud.google.com
Move to Google Cloud Platform1. Preparation (create account)
6. Move to Google Cloud Platform (login) 7. Select ‘BigQuery’ from left menu (top-left icon)
https://cloud.google.com
Move to Google Cloud Platform1. Preparation (create account)
6. Move to Google Cloud Platform (login) 7. Select ‘BigQuery’ from left menu (top-left icon)
https://cloud.google.com
Now you are ready to use BigQuery
Credit card ....
I can hardly check how much I have as free credit.
Credit card ....
I can hardly check how much I have as free credit.
Okay. Anytime you can check your free creit and trial days easily.
... I wanna try BigQuery
Try BigQuery with public data
Try BigQuery with public data
Query
Query
SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000
Query
SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000
Query
SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000
Query
SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000
Query
SELECT repository_name FROM [bigquery-public-data:samples.github_timeline] LIMIT 1000
EX) Query complete (2.1s elapsed, cached) Query complete (3.8s elapsed, 61.4 MB processed)
More query
SELECT actor, actor_attributes_name , count( actor_attributes_name ) as amount FROM [bigquery-public-data:samples.github_timeline]
group by actor, actor_attributes_name order by amount desc LIMIT 1000
Much more query
You can check query syntax by yourself.
https://cloud.google.com/bigquery/query-reference
Upload your data
1. Create database
2. Create Table (with CSV)
3. check “Table ID”
4. QUERY!
Create new dataset
Create new dataset
Create new dataset
Create new dataset
Create newTable (with CSV file)
Create newTable (with CSV file)
Create newTable (with CSV file)
Create newTable (with CSV file)
nemo,47,[email protected] foo,30,[email protected] bar,25,[email protected]
sample.csv
Create newTable (with CSV file)
nemo,47,[email protected] foo,30,[email protected] bar,25,[email protected]
sample.csv
Check “Table ID”
Check “Table ID”
Check “Table ID”
QUERY for your own table!!
Now you are ready to use
BigQuery.
NextInstall GCP SDK
Use command line tools
Characteristic of Google BigQuery.
Especially, Tips for business use.
Thank you!