Upload
yalisassoon
View
627
Download
0
Embed Size (px)
Citation preview
carbonite.com
November 2014 — State of Carbonite Analytics
4
• SQL Server Warehouse (Death Star) • Numerous Pipelines • Replication from Production Systems
• Reporting Systems • Cognos • Tableau
• Digital Analytics • IBM Coremetrics • Tealium • C3 Metrics
carbonite.com
We needed more…
5
• Which of our marketing campaigns are effective?
• Where should we be allocating our marketing spend?
• What are the weak points in the customer acquisition lifecycle?
• What features are customers using in our products?
• How do we optimize web, product, and mobile workflows? What works?
• What are our customers doing in our products?
• How do we connect the digital analytics world with our traditional BI Warehouse?
• We need a scalable, cost effective solution • We wanted a lambda architecture (batch and
stream processing) • We wanted to use AWS • We didn’t want a proprietary internal system • We wanted to use and create as much open
source technology as possible
• Do we build or buy?
carbonite.com
Project Holocron — Build a modern Data Warehouse solution
6
• Our research • Hortonworks / Hadoop • AWS EMR, Kinesis, Redshift • re:Invent, Hadoop Summit, etc.
• We found the winning infrastructure • AWS • Looker • Snowplow • Redshift
carbonite.com
2015, Q1 - Onboarding
8
• Goals • Get Snowplow Up and Running • Get Web data flowing (tracking beacons) • Setup Looker • Create proof of concept, sample, reports
• How we achieved our quick wins • Managed Snowplow Hosting (1 week!) • Deployed Beacon to all of our sites • Worked with Looker and Snowplow to
create a new Event model in Looker • Created basic page view and session
reports
carbonite.com
2015, Q2 - Digging In
9
• Goals • Replace Coremetrics • Create a web marketing attribution
framework • Create a stable operations platform
• How we achieved our quick wins • Created dozens of reports in Looker
based on customer requirements • Took advantage of Snowplows built in
Web Events (Page views, link clicks, etc.) • Worked with our CMO to create a best in
class marketing measurement framework (PCT)
• Starting managing links in Excel (yuck) • Migrate Redshift to separate AWS prod
account (protecting corporate-side data) • Cluster management with Ansible and
CloudFormation • Ansible management of IAM • Implemented a Blacklist
carbonite.com
2015, Q3 - Adoption Hurdles
10
• Goals • Solve our Looker adoption issues • Get Link attribution info into Redshift • Standardize Event and Tag management
across projects • Support Cart, Form Tracking, and
Custom Events • Ensure Operational Integrity of Platform
• How we achieved our quick wins • Created a Django API (Viceroy) for
managing and storing PCT attributed links • Updated API to support Blacklist
management • Set aside a strike team to sit with
Marketing to help communication and adoption
• Use Viper to standardize all of the analytics libraries we use (Google, HotJar, Optimizely, Tealium, etc.)
• Use Viper to provide standardized API for Custom IGLU events
• Created Operational processes to watchdog our data (with Looker Reports)
carbonite.com
2015, Q4 - Adoption Hurdles
11
• Goals • Provide Link Management capabilities
as a self-service utility to Marketing • Find a Cost Management solution for
AWS • Create a means of increasing
confidence in our platform data • Find a utility that will help us with more
complex ETL tasks such as Click Streaming and Data Ingestion
• How we achieved our quick wins • We released Alpha and Beta of Project
Viceroy Link Builder using ReactJS/Redux
• We chose CloudHealth for cost management, it’s awesome
• We created a prototype weekly “Ion Cannon” email to help us determine what we want to automate later
• We implemented Databricks so that we could perform advanced analytics using Spark
carbonite.com
2016 and beyond…
12
• Goals • Implement a system of Monitoring our
Marketing Tags for performance issues and auditing
• Instrument our Products with Event Tracking
• Convert many of our Looker PDTs to Spark / EMR
• How we achieved our quick wins • We’ve implementing TagInspector
Realtime and Scanner • Viper 2.0 for Endpoint and all sites • Android Tracker for Mobile • Custom Events and Contexts for all! • We’re implementing Informatica Cloud • Working with the Snowplow team to
customize the Enrichment process to use Spark
• Databricks for Dev and Test, EMR for Prod
carbonite.com
Project Viper
13
• The Analytics Team “Tag” • Decouple our efforts from the Web Teams • Single place to manage all of the various Analytics Tags • Single line of JavaScript for all of our needs
• An Open Source Dev Side “Tag Manager” • To be Open Sourced in 2016 • Event Driven Framework • Built in JS ES6/2015 • Custom NPM Modules for Viper Plugins • Snowplow Support • Tealium Support • TagInspector Support • … and more
carbonite.com
Project Viceroy
14
• Web Attribution Management • Built on our PCT Framework • Marketing Manager friendly, easy to use • Manage Marketing Attribution • REST API • ReactJS Frontend
• Create Placements • Ad Server Templates (Marin, DCM, etc.) • Email Systems (Responsys, Exact Target, etc) • Direct Links • bit.ly links
• Standardized Attribution • No more typos • No more missing parameters • No more malformed links
THANK YOU! QUESTIONS?
Robert Johnson Director, Analytics Platform [email protected]