Upload
javier-ramirez
View
72
Download
2
Tags:
Embed Size (px)
Citation preview
javier ramirez@supercoco9
Get more fromAnalytics with
Google BigQuery
datawaki
about me
19 years working on software: banking, e-commerce, government, CMS, start-ups...
founder of https://datawaki.comhttps://teowaki.com
Google Developer Expert on the Cloud Platform
mail: [email protected] twitter: @supercoco9
datawaki
BigQueryis awes..
I use GoogleAnalytics
javier ramirez @supercoco9 https://datawaki.com
Isn't Google Analytics good enough?
javier ramirez @supercoco9 https://datawaki.com
Google Analytics is great but...
It lets you access aggregated data and reports, not individual sessions/visits data.
Even premium accounts get sampled reports when there are too many data (and not all the reports can be unsampled).
javier ramirez @supercoco9 https://datawaki.com
Google Analytics is great but...
If you need to manage many different segments, and if you want to combine segments, it can get tricky.
Moreover, you can only segment or create reports using the pre-defined filters, which might or not be enough for you*.
*even if segments have experienced a huge improvement with Universal Analytics
javier ramirez @supercoco9 https://datawaki.com
Google Analytics is great but...
It's not easy to cross data in Analytics with data from other sources (CRM, invoicing system...)
Now you can use Import Data from UniversalAnalytics, but there are many constraints to what you can do
javier ramirez @supercoco9 https://datawaki.com
Google Analytics is great but...
Good for knowing what's happening in your application, but difficult for:
* business intelligence/big data (data mining, find patterns...)
* machine learning (classify information, predict future trends...)
javier ramirez @supercoco9 https://datawaki.com
big data -analysed and organised into information- has big value
BIG DATA
HIPSTER
javier ramirez @supercoco9 https://datawaki.com
data that exceeds the processing capacity of conventional database systems. The data is too big, moves too fast, or doesn’t fit the structures of your database architectures.
Ed Dumbill program chair for the O’Reilly Strata Conference
javier ramirez @supercoco9 https://datawaki.com
bigdata is cool but...
expensive cluster
hard to set up and monitor
not interactive enough
What if I could...
..query billions of rows in seconds..
..using a SQL-like web interface..
..on a fully managed cloud..
..paying only when I use it?
javier ramirez @supercoco9 https://datawaki.com
Designed to run analytics over huge volumes of raw data, and to integrate with other data sources
javier ramirez @supercoco9 https://datawaki.com
Google BigQuery
Google BigQuery
Data analysis as a service
https://cloud.google.com/products/bigquery/
javier ramirez @supercoco9 https://datawaki.com
onemorething
Google Analytics Premium users get free daily exports from GA to BigQuery.
javier ramirez @supercoco9 https://datawaki.com
Google BigQuery + GA Premium
All your raw data.Unsampled.Use it however you want.
BOOM!javier ramirez @supercoco9 https://datawaki.com
Google BigQuery + GA Premium
o'reilly
khan academy
US Cellular Case Study5th largest US telecommunications company over 10.6 million customers
They didn't know how many offline (in-store +telesales) sales were originated by online media.
After combining GA data with other internal data,they can more accurately attribute sales to digital channel (website, social, search and display)
It helps them optimize their campaign and forecastsales
data schema
javier ramirez @supercoco9
it's just SQL
javier ramirez @supercoco9 https://datawaki.com
SELECT trafficSource.source, SUM( totals.transactions ) AS total_transactionsFROM playground.ga_sessions_20140621GROUP BY trafficSource.sourceORDER BY total_transactions;
basic queries (metric/dimension)
SELECT device.isMobile, SUM ( totals.pageviews ) AS total_pageviewsFROM playground.ga_sessions_20140621GROUP BY device.isMobileORDER BY total_pageviews;
SELECT IF(DOMAIN(trafficSource.source) is null,
trafficSource.source,DOMAIN(trafficSource.source))
AS normalized_source, SUM ( totals.transactions ) AS total_transactions
FROM playground.ga_sessions_20140621GROUP BY normalized_sourceORDER BY total_transactions;
basic queries with a twist
SELECT ( SUM(total_transactionrevenue_per_user) / SUM(total_visits_per_user) ) AS avg_revenue_by_user_per_visitFROM ( SELECT SUM(totals.visits) AS total_visits_per_user, SUM( totals.transactionRevenue ) AS total_transactionrevenue_per_user, visitorId FROM playground.ga_sessions_20140621 WHERE totals.visits>0 AND totals.transactions>=1 AND totals.transactionRevenue IS NOT NULL GROUP BY visitorId ) ;
Average amount spent per visit
2 segments, combined
SELECT hits.item.productName AS other_purchased_products, COUNT(hits.item.productName) AS quantityFROM playground.ga_sessions_20140621WHERE fullVisitorId IN ( SELECT fullVisitorId FROM playground.ga_sessions_20140621 WHERE hits.item.productName CONTAINS 'Light Helmet' AND totals.transactions>=1 GROUP BY fullVisitorId ) AND hits.item.productName IS NOT NULL AND hits.item.productName !='Light Helmet'GROUP BY other_purchased_productsORDER BY quantity DESC;
Users who bought product A,also bought product B
SELECT prod_name, count(*) as transactionsFROM(SELECT fullVisitorId, min(date) AS date, visitId, hits.item.productName as prod_nameFROM (SELECT fullVisitorId, date, visitId, totals.transactions, hits.item.productName FROM (TABLE_DATE_RANGE([dataset.ga_sessions_], TIMESTAMP('2014-06-01'), TIMESTAMP('2014-06-14'))))WHERE fullVisitorId IN(SELECT fullVisitorIdFROM (TABLE_DATE_RANGE([dataset.ga_sessions_], TIMESTAMP('2014-06-01'), TIMESTAMP('2014-06-14'))) GROUP BY fullVisitorId HAVING SUM(totals.transactions) > 1)AND hits.item.productName IS NOT NULLGROUP BY fullVisitorId, visitId, prod_name ORDER BY fullVisitorId DESC)GROUP BY prod_name ORDER BY transactions DESC;
* example query from the lunametrics blog. Check them out for more awesomeness
Products that are purchasedand lead to other products being purchased
SELECT fullvisitorID, visitID, visitNumber, hits.page.pagePathFROM playground.ga_sessions_20140621where hits.type='PAGE'order by fullvisitorID, visitID, hits.hitnumber asc
Identify user path/user actions
individual users data is awesome
Cross CRM data with individual users actions to seehow your response to incidents affect your users.
Use the “frequently bought together” query and findusers who didn't buy the related products. Send ane-mail campaign with an offer for those products.
integrating with external data sources
* Connectors/REST API* Export into GCS* Import into BigQuery
javier ramirez @supercoco9 https://datawaki.com
Working with external data
BigQuery pricing
$20 per stored TB
$5 per processed TB*the 1st TB every month is free of charge
** GA premium get $500 free credit monthly
javier ramirez @supercoco9 https://datawaki.com
for GA premium users BigQuery is effectively
for free
*unless you upload huge external data or make huge queries
javier ramirez @supercoco9 https://datawaki.com
What if I don't havea GA Premium
Account?
just send your own data
javier ramirez @supercoco9 https://datawaki.com
define a data structure that fits your needs (or replicate the one GA provides)
use a JS snippet to send data to your server, thento BigQuery
or use
javier ramirez @supercoco9 https://datawaki.com
Just add an extra snippet to your GA
datawaki
javier ramirez @supercoco9 https://datawaki.com
send data from any other source (CRM, back-end,sensors, mobile apps, log system, external tools...)
datawaki
PostgreSQL
Log
NGINX
Log
NGINX
Logstash
Redis
BigQuery
Neo4j
RubyWorker
RubyWorker
RailsApp
Alertsystem
datawaki in a nutshell
Reportsystem
userinteraction
data input
javier ramirez @supercoco9 https://datawaki.com
* Get full access to your data
* Receive reports by e-mail
* Get individual or group alerts* if there is a purchase over $1000 * if a user has visited a product page over
20 times in one week and didn't buy* if a product is seen over 200 times one hour
* every time a product reaches 5000 views
datawaki
Want to know more?https://cloud.google.com/products/bigquery/
Need help?https://teowaki.com/services
Thanks!Gracias
Javier Ramírez@supercoco9
datawaki