7 Practical Tips on Lean Analytics

  • Published on
    09-Aug-2015

  • View
    119

  • Download
    1

Embed Size (px)

Transcript

  1. 1. 7 Practical Tips on Lean Analytics Janne Aukia 17 Feb 2015
  2. 2. 1. Lean Analytics requires a Lean project Illustration from commons.wikimedia.org/wiki/File:Kepler-solar-system-1.png
  3. 3. Challenging for a consultancy? Lean Analytics works well with Lean Startup, Lean UX and Agile projects Question designs, question authorities, ask difficult questions, be prepared to kill features. Requires freedom to experiment
  4. 4. 2. Write down a success metric for every feature
  5. 5. Was this feature used more than Google+ login? If feature isnt successful, just kill it. Measure only actionable metrics. Is the feature for acquisition, retention or revenue?
  6. 6. 3. You need lots of users for an A/B test
  7. 7. Typically at least 10k events, such as visits or people using a feature. Doesnt work for small startups.
  8. 8. 4. Make big changes Illustration from http://quod.lib.umich.edu/m/moa/agb8710.0001.001/26
  9. 9. If you cannot see the changes from 2 meters away, most likely your AB test wont either. Google A/B tested 41 shades of blue. Dont do that leads to local optimum. Illustration from http://www.doxsey.net/blog/too-practically-minded-to-be-of-any-theological-good/
  10. 10. 5. Measure your tests correctly
  11. 11. Never terminate early a running test Use a split test calculator, for example: http://getdatadriven.com/ab-significance-test
  12. 12. 6. Push through radical changes with A/B
  13. 13. 7. Use the right tools
  14. 14. Google Analytics can be used for tracking events but it is painful. Mixpanel more suited for the job. Consider Keen.io, Heap as well.
  15. 15. 1.Lean Analytics requires a Lean project 2.Specify success metric for every feature 3.You need lots of users for an A/B test 4.Make big changes 5.Measure your tests correctly 6.Push through radical changes with A/B 7.Use the right tools Summary