32
© 2016 MapR Technologies 1 © 2016 MapR Technologies 1 Today’s Presenters Rafael Godinho Technical Evangelist Tim Morgan Managing Director

MapR on Azure: Getting Value from Big Data in the Cloud -

Embed Size (px)

Citation preview

Page 1: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 1© 2016 MapR Technologies 1

Today’s Presenters

Rafael GodinhoTechnical Evangelist

Tim MorganManaging Director

Page 2: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 2© 2016 MapR Technologies 2

Agenda

• Big Data & the Cloud

• Customer Use Case

• Azure Overview & Demo of MapR on Azure

Page 3: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 3© 2016 MapR Technologies 3

Data Gravity

Data tends to stay where it is generated

Applications and services are attracted to the data

Page 4: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 4© 2016 MapR Technologies 4

Flexible processing where change is the norm

Distributed processing across clusters, data centers and public and private cloud environments

Supports global apps that can scale arbitrarily

Key to Real-time at Scale: Global Cloud Processing

Page 5: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 5© 2016 MapR Technologies 5

Open Source Engines & Tools Commercial Engines & Applications

Enterprise-Grade Platform Services

Dat

aPr

oces

sing

Web-Scale StorageMapR-FS MapR-DB

Search and

Others

Real Time Unified Security Multi-tenancy Disaster Recovery Global NamespaceHigh Availability

MapR Streams

Cloud and

Managed Services

Search and Others

Unified M

anagement and M

onitoring

Search and

Others

Event StreamingDatabase

Custom Apps

MapR Converged Data Platform

HDFS API POSIX, NFS Kakfa APIHBase API OJAI API

Page 6: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 6© 2016 MapR Technologies 6

MapR on Microsoft Azure Marketplace

MapR and Microsoft enable enterprise grade big data applications in the Azure cloud

Simplified Deployment

Azure Marketplace’s automated deployment capabilities make big data easy

Azure’s infrastructure can scale up to match any requirement and scale down for value

MapR integrates with other Azure services to enable customers to analyze any type of data to unlock the biggest insights

Unlimited Scale Seamless Interoperability

Product Alignment

Page 7: MapR on Azure: Getting Value from Big Data in the Cloud -

About Sullexis

• Sullexis is a professional services firm that specializes in helping its clients to create, manage, and enhance data to accelerate and improve decision making across the enterprise. We bring data and technology together to make our clients measurably more effective

• With industry experience ranging from energy and manufacturing to finance and high tech, Sullexis brings the technology, processes, and strategies together to make you more effective in what you do

• Founded in 2006, Sullexis is headquartered in Houston, TX and has a delivery center in Monterrey, MX.

• Our consultants have implemented solutions across the US, Caribbean, Europe and Latin America.

Presentation Title 7

Page 8: MapR on Azure: Getting Value from Big Data in the Cloud -

Client Background

• Our client is one of North America’s largest Oilfield Services companies providing well construction, completion and operating services to exploration and production companies.

• A significant number of acquisitions over the last 10 years resulted in 18 different ERP applications running on 5 different platforms. To enable future, scale-able growth, they embarked on an ERP standardization project. The goal to put the entire company on one technology stack with a common process.

• Having decided to consolidate on a single ERP, the client still needed to determine how best to handle compliance, regulatory and operational needs associated with the legacy systems.

• Migrating transaction data to the new ERP would be cost prohibitive and risky; and market ready data archiving solutions were costly and unable to meet the defined business needs.

• This left retaining the legacy systems themselves, which would be very costly, or finding a new approach that was cost effective, reliable and could meet the business needs.

8

18 to 1

Page 9: MapR on Azure: Getting Value from Big Data in the Cloud -

Key Requirements

Preserve and provide easy access to ALL data• Preserve all structured and unstructured data (approx 12 TBs)• Ability to run legacy reports to meet compliance, regulatory and ongoing business needs• Easy for a business person to use directly to minimize IT resource dependency• Ability to provide consolidated views across disparate data sets

Be cost effective• Flexible and scalable compute/data storage options (ex. Use of cold storage)• Provide access through existing BI and reporting tools (ex. Hyperion, MS Power BI, SAP Lumira)

to eliminate new purchases and training• Enable 100% decommissioning of legacy systems

Enable the future• Establish processes and tools that support future company acquisitions• Provide platform to enable new and innovate data applications and solutions

9

Page 10: MapR on Azure: Getting Value from Big Data in the Cloud -

Solution Selection Process

Initial Analysis• Market Research• Vendor presentations

Two week POC ‘bake-off’ to demonstrate:• Rapid integration of different data sources both structured and unstructured • Connectivity to SAP ECC and Oracle EBS• Reporting capabilities re-using SAP Lumira

Winning POC Solution• A MapR Converged Data Platform cluster installed in MapR’s private cloud• Predefined adapters for Oracle used to extract and load structured data to MapR (<100GB)• Unstructured data of CSV, PDFs and TXT loaded and made viewable through Elastic Search• Apache Drill and a local install of SAP Lumira connected to the MapR cluster to demonstrate

reporting capabilities

10

Page 11: MapR on Azure: Getting Value from Big Data in the Cloud -

Solution Architecture

Page 12: MapR on Azure: Getting Value from Big Data in the Cloud -

Project Considerations

Technology Factors• Reliability and speed of connection to cloud• Count and category of machines in cloud

(CPU, RAM, Storage)• Volume of data (row size and count)• Ongoing transaction use of source system• Variable needs for data (frequency,

response, volume)

Project Factors• Timeliness of and accessibility to various

parties• Cataloging of all data• Evaluation of transactional status of existing

data sets, and how to address moving targets (blackout periods, iterative loads, journaling)

• Sample extracts from every table• Ability to validate data loads (row counts

samples)

Page 13: MapR on Azure: Getting Value from Big Data in the Cloud -

Solution Architecture

NFS

PDF, CSV, XLS Oracle Navision SysPro MS Excel Great Plains

Dat

a Web-Scale StorageMapR-FS MapR-DB

Real Time Unified Security Multi-tenancy Disaster Recovery Global NamespaceHigh Availability

MapR StreamsEvent StreamingDatabase

Enterprise Grade Platform

13

PDF TIFF CSV

Page 14: MapR on Azure: Getting Value from Big Data in the Cloud -

Why Azure

• Sullexis and client both experienced with Azure and MSFT

• MapR Quick Start on Azure made it easy and fast to get started

• MapR already successfully running well on Azure (see blog)

• Client’s enterprise MSFT account made it simple to procure and administer

• Connectivity to Azure via ExpressRoute mitigated some of the reliability and latency of

connection

14

Page 15: MapR on Azure: Getting Value from Big Data in the Cloud -

Apache Drill - Flexible & FastAccess to any data type, any data source

• Relational• Nested data• Schema-less

Rapid time to insights• Query data in-situ• No Schemas required• Easy to get started

Integration with existing tools• ANSI SQL• BI tool integration

Scale in all dimensions• TB-PB of scale• 1000’s of users• 1000’s of nodes

Granular security• Authentication• Row/column level controls• De-centralized

15

Page 16: MapR on Azure: Getting Value from Big Data in the Cloud -

Sqoop – Easy & Efficient

Leveraging a Sullexis developed direct connect extract tool based on Sqoop was seen as meeting all the technology and project factors:

• Addresses all source data• Support for both Oracle and SQL Server• Import direct to Parquet• Supports type mapping• Supports incremental imports and merges• Enables validation via row count matches• Provides for parallel imports for enhance speed (but also allows for throttling)

16

Page 17: MapR on Azure: Getting Value from Big Data in the Cloud -

Elastic Search – Simple & Transparent

17

Reporting Client Browser

Web UI

edgenode 1node 0 node 2

POSIX Client

PDF TIFF CSV PDF TIFF CSV PDF TIFF CSV

MapR-FS

ODBC or JDBC HTTP(S)

Page 18: MapR on Azure: Getting Value from Big Data in the Cloud -

Highlights

• Quick and easy startup

• Primary technical concerns around latency to the cloud can be successfully mitigated (e.g. client’s cluster enabled transfer rates of 100-140 million records per hour)

• While early, the base business case will result in a payback within a few months and business users have suggested that data access is easier now than originally available in the legacy system

• This ERP legacy system decommissioning approach can be executed in as little 2 months for a complete data archive to 6 months with robust operational reporting

• Provides repeatable tools and process available for future system decommissioning needs

• The client is already experimenting with the platform for use as an IoT sensor data historian. So far the results have been encouraging

18

Page 19: MapR on Azure: Getting Value from Big Data in the Cloud -

About Us

We are attuned to the challenges facing organizations in a variety of industries and understand the constant pressure to improve business processes and make better decisions. But beyond that, we have a passion for technology. Using that passion, we help our clients use proven technology coupled with our real-world knowledge to accelerate and improve the flow of data and information and

improve productivity. The technical improvements we provide equip our customers to make the best business decisions possible. Helping our clients unleash the power of their data is our focus.

MapR on Azure: Getting Value from Big Data in  the Cloud 19

Darrell Petty
I would delete this slide or just move it to the end; fluffy and repetitive of the next slide
Page 20: MapR on Azure: Getting Value from Big Data in the Cloud -

Nearly 50 million Office Online users

48 Million Subscribers in 41 countries

Outlook.com has over 400 Million active users and is the world’s fastest growing email service

1 Billion mobile notifications a month

Bing holds 20.2 percent of US market share

- comScore

Yammer has over 8 million registered users

Over 250 million people use OneDrive

On average, Skype users use the service 50 Billion

minutes/mo

Xbox delivered over 740 million hours of entertainment

Office for iOS has been downloaded over 80M times

Page 21: MapR on Azure: Getting Value from Big Data in the Cloud -

Analyst reports

Page 22: MapR on Azure: Getting Value from Big Data in the Cloud -

Achieve global scale, in local regions

34 regions

Page 23: MapR on Azure: Getting Value from Big Data in the Cloud -

Platform Services

Infrastructure ServicesCompute Storage

Datacenter Infrastructure

Application Platform

WebApps

MobileApps

API Apps

Notification Hubs

HybridCloud

Backup

StorSimple

Azure SiteRecovery

Import/Export

Networking

DataSQL Database DocumentDB

Redis Cache

AzureSearch

StorageTables

SQL DataWarehouse

Azure AD Health Monitoring

Virtual Network

ExpressRouteBlob Files DisksVirtual

Machines

AD PrivilegedIdentity Management

Traffic Manager

AppGateway

OperationalAnalytics

Compute Services

Cloud Services

Batch RemoteApp

ServiceFabric

Developer Services

Visual Studio

ApplicationInsights

VS Team Services

Containers

DNS VPN Gateway

Load Balancer

Domain Services

Analytics & IoTHDInsight Machine

Learning Stream Analytics

Data Factory

EventHubs

Data LakeAnalytics Service

IoT Hub

Data Catalog

Security & Manageme

nt

Azure ActiveDirectory

Multi-FactorAuthentication

Automation

Portal

Key Vault

Store/Marketplace

VM Image Gallery& VM Depot

Azure ADB2C

Scheduler

Xamarin

HockeyAppPower BI Embedded

SQL Server Stretch Database

MobileEngagement

Functions

IntelligenceCognitive Services Bot Framework Cortana

Security Center

Container Service

Queues

VM Scale Sets

Data Lake Store

Dev/Test Lab

IntegrationBizTalkServices

Service BusLogic Apps

API Management

Media & CDNContent DeliveryNetwork

Media Services

Media Analytics

Page 24: MapR on Azure: Getting Value from Big Data in the Cloud -

Architecture of MapR on Azure

… …

MapR Converged Data Platform

Page 25: MapR on Azure: Getting Value from Big Data in the Cloud -
Page 26: MapR on Azure: Getting Value from Big Data in the Cloud -

Demo

Running MapR from Azure Marketplace

Page 27: MapR on Azure: Getting Value from Big Data in the Cloud -

Windows NFS Map/

Reduce Hive Drill Excel/PowerBI

Demo

Page 28: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 28© 2016 MapR Technologies 28

Digital transformation for better customer experienceDeliver self-service insights across the business

• MapR platform on the Azure cloud to modernize their infrastructure and sunset legacy systems.

• Faster exploration of data with Apache Drill mitigating need for schema development.

• Support for use cases such as customer 360, supply chain & image analysis

OBJECTIVES

CHALLENGES

SOLUTION

• Modernize analytics & improve speed of marketing campaigns• Reduce cost of existing systems•

• Existing technologies prohibiting effective & timely reporting and analysis• Very long time to extract value from the data leading to lots of Excel

Leading optical retail chain

Page 29: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 29© 2016 MapR Technologies 29

New Analytical Insights to Real Estate TenantsOptimize tenants experience and drive additional revenue

• MapR on the Azure cloud helps analyze more data types for faster insights• Analysts query and search work orders to identify

maintenance and utilization trends, enabling cost savings. • Optimization of tenants’ experience to capture additional rental revenue.

OBJECTIVES

CHALLENGES

SOLUTION

• Identify maintenance and utilization trends and enable cost savings via predictive maintenance

• Modernize data infrastructure with a new analytics platform• • M&A activity resulting in hundreds of siloed databases• Inability to handle new data types such as IOT sensor data and provide new insights

LARGE COMMERCIAL REAL ESTATE MANAGEMENT COMPANY

Page 30: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 30© 2016 MapR Technologies 30

MapR Customers in All Major IndustriesFINANCIAL SERVICES RETAIL & CPG SECURITY ONLINE SERVICES &

SOFTWAREMEDIA &

ENTERTAINMENT

MANUFACTURING, UTILITIES, OIL &

GASADVERTISING HEALTH COMMUNICATIONS GOVERNMENT

United Healthcare

Page 31: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 31© 2016 MapR Technologies 31

Azure and MapR Resources – 3 steps to get started

• Azure Overviewhttps://www.mapr.com/partners/partner/microsoft-azure-microsofts-cloud-computing-platform-moving-faster-achieving-more

• 7 Steps to Deploy the MapR Sandbox on Azurehttps://www.mapr.com/blog/7-steps-deploy-mapr-sandbox-microsoft-azure

• Azure Test Drivehttp://mapr.testdrivelabs.com/ (subject to change)

Page 32: MapR on Azure: Getting Value from Big Data in the Cloud -

© 2016 MapR Technologies 32© 2016 MapR Technologies 32

Q & A

@mapr

@mapr.com

Engage with us!

maprtech

mapr-technologies

https://www.mapr.com/get-started-with-mapr

https://www.mapr.com/training

https://www.mapr.com/ebooks/big-data-all-stars/