Big data, Analytics and Beyond

Big Data, Analytics and Beyond Sri Krishnamurthy, CFA, CAP

Founder and CEO www.QuantUniversity.com

Lecturer, Babson College sri@quantuniversity.com

BABSON COLLEGE — ALL RIGHTS RESERVED

Agenda

• Big Data and Analytics : A brief introduction

• Leveraging technology to gain competitive advantage

• The Five technological trends of the last decade

• Technology tips to implement a successful analytics strategy

BIG DATA: A BRIEF INTRODUCTION

BABSON COLLEGE — ALL RIGHTS RESERVED

BABSON COLLEGE — ALL RIGHTS RESERVED

http://tinyurl.com/bzbxf44

BABSON COLLEGE — ALL RIGHTS RESERVED

BABSON COLLEGE — ALL RIGHTS RESERVED

BABSON COLLEGE — ALL RIGHTS RESERVED

BABSON COLLEGE — ALL RIGHTS RESERVED

Traditional Analytics versus Big-Data Analytics– The paradigm shift

• Exploration versus canned reports

• Information from multiple sources

• Trading off accuracy for speed

• What versus Why?

LEVERAGING TECHNOLOGY TO GAIN COMPETITIVE ADVANTAGE

BABSON COLLEGE — ALL RIGHTS RESERVED

Netflix

• Close to 30 million customers

• Beat brick-and-mortar giant Blockbuster.

• Uses Amazon’s cloud services

• Recommendation engines, A/B Testing infrastructure, data stores etc.

BABSON COLLEGE — ALL RIGHTS RESERVED

Zynga - Farmville

• Expected 200,000 daily users in first 2 months

• Got 1 million users every month

• Scaled seamlessly since the application was built on the cloud

BABSON COLLEGE — ALL RIGHTS RESERVED

New York Times

• Used Amazon EC2

• Process 4TB of images to 11 million PDF files in 24 hrs

• Cost: $240

BABSON COLLEGE — ALL RIGHTS RESERVED

5 TECHNOLOGICAL TRENDS IN THE LAST DECADE

BABSON COLLEGE — ALL RIGHTS RESERVED

Trend 1: Machine learning comes to age

• Parallel and distributed algorithms have made mining massive data sets simpler

– Example: Map-Reduce (Hadoop)

• Large-scale recommendation systems possible with newer algorithms

BABSON COLLEGE — ALL RIGHTS RESERVED

Trend 2: Memory is cheap

• Cost to store all the music in the world - $600!

BABSON COLLEGE — ALL RIGHTS RESERVED

Trend 3: Good bye datacenters. Hello Cloud!

• Rent versus Buy

• Super computing: Cycle computing set up a 30,000 core cluster at the cost of $1,279/hr for 8 hrs on Amazon

• 1000 CPUs for 1 hr = 1 CPU for 1000 hrs

Microsoft’s datacenter in Ireland to support the Azure cloud

BABSON COLLEGE — ALL RIGHTS RESERVED

Trend 4: Open source drives innovation

• Hadoop: Software framework that supports data-intensive distributed applications

• Mahout : Java-based machine learning algorithms

• Mongo-DB: A No SQL document database

BABSON COLLEGE — ALL RIGHTS RESERVED

Trend 5: Rich Visualization

• From reporting to data exploration

• Visualizing live data streams

TECHNOLOGY TIPS TO IMPLEMENT A SUCCESSFUL ANALYTICS STRATEGY

BABSON COLLEGE — ALL RIGHTS RESERVED

Source: http://tdwi.org/research/2011/09/best-practices-report-q4-big-data-analytics.aspx

BABSON COLLEGE — ALL RIGHTS RESERVED

Tips

1. Assess: How analytical is your organization now?

2. Ask: Do you really have a big-data problem?

3. Need: Is there a compelling case to address the problem now? Can you articulate the business value of solving this problem ?

4. Change: Does your organization have the right kind of skills , aptitude and culture to implement a solution?

5. Foresee: Does the technology roadmap you plan to adopt meet the demands you see in the future?

BABSON COLLEGE — ALL RIGHTS RESERVED

Summary

• Data has significantly increased in the last decade • Huge opportunities for businesses; tech has been a key-enabler. • Technology trends made Big Data and Analytics possible:

Trend 1: Machine learning comes to age Trend 2: Memory is cheap Trend 3: Good bye datacenters. Hello Cloud! Trend 4: Open source drives innovation Trend 5: Rich Visualization

• Before embarking on implementing a big-data solution, companies must analyze whether the solution would help them achieve the business value they aim for.

Big Data, Analytics and Beyond Sri Krishnamurthy, CFA, CAP

Founder and CEO www.QuantUniversity.com

Lecturer, Babson College sri@quantuniversity.com