The Executive View on Big Data Platform Hosting - Evaluating Hosting Services for Big Data Computing Cloud Expo 2013

Hitachi Consulting The Executive View on Big Data Platform Hosting E v a l u a t i n g H o s t i n g S e r v i c e s f o r B i g D a t a C o m p u t i n g

Authors: Chad M. Lawler, Ph.D. Director of Consulting, Cloud Computing Practice - Hitachi Consulting

Art Vancil, CCP, CCSK Senior Manager, Big Data Architect - Hitachi Consulting

Better

| © Copyright 2013 Hitachi Consulting 2

• Evaluate Big Data Hadoop Hosting Services

• Considerations for Big Data POCs

Presentation Overview • The Solution Requirements & The Solution Stacks

• The Application Development Process & The Platform Environment

• The Evaluation and Decision Process & Hitachi Big Data Solutions & POC Approach

Today’s Presentation Available at: • http://www.slideshare.net/chadmlawler

Todays Purpose:


http://www.slideshare.net/chadmlawler



www.linkedin.com/in/chadmlawler

www.cardcloud.com/chadlawler

Chad M. Lawler, Ph.D. Director of Consulting Services

Cloud Computing

14643 Dallas Parkway, Suite 800, Dallas, Texas 75254

Office: 469.221.2894

Email: [email protected]

www.hitachiconsulting.com/cloud

Chad M. Lawler, Ph.D. Director of Consulting, Cloud Computing Practice - Hitachi Consulting

• Advising Cloud Service Providers – Working with leading CSPs providing consulting services for

CSP strategy, planning, marketing analysis & research, product & service development, go-to-market

planning/ launch and internal process assessment optimization and automation

• Enabling Cloud Adoption - Providing an extensive portfolio of cloud adoption consulting services

to help businesses and organizations adopt, develop and implement cloud services and enable cloud

transformation.

• Cloud Service Brokerage & Multi-Cloud Governance – Delivery consulting and solutions to

positions CIOs to become Cloud Service Brokers for their organizations to centralize and streamline

management of multiple cloud services use, providing a single point of transparency, accountability,

sourcing and governance control

• Cloud Consulting, Executive Strategy & Planning

• Cloud Product & Service Development

• Cloud Competitive Market Analysis

• Cloud Go-to-Market Strategy & Product Launch

• Cloud Marketing & Sales Messaging, IP & Collateral

• Cloud Service Provider Advisory & Service Development

• Cloud Business Case Development & Financial Modeling

• Public, Private & Hybrid Cloud Computing

• Cloud Service Governance Frameworks Consulting

• Cloud Service Broker Solution Consulting

http://www.linkedin.com/in/chadmlawler

http://www.cardcloud.com/chadlawler


Big Data Solution Requirements

Classes of Big Data Challenges

• Data that is naturally unstructured and/or rich in structure that require extensions of traditional data management tools

• Ability to create flexible analyses

• Ability to evolve analyses to meet future needs

• High velocity data that is increasingly being used in agile decision making

• Data set sizes that pose significant challenges to traditional data management infrastructure

• Peta Bytes, Zetta Bytes

Complexity

Velocity

Variety

Volume


Big Data Management

Fundamental Improvements

• Structural constraints to performing new analyses

Unconventional Analytics – Prediction & Sentiment Analysis

• Data volumes and velocity exceed processing capacity

Larger Data Volumes (petabytes)

•Relational data infrastructure is constrained in handling Unstructured & Rich Structured data

Analysis of Unstructured Data

• BI Capabilities optimized for traditional BI, not Big Data

Integrate and Leverage Existing BI

• Low cost Data Storage & Management Capacity

• Framework/platform driven high performance allowing creation of on-

demand analytic structures

• Software Managed Recovery & Graceful degradation

• Minimized data movement within the environment

• On-line Component Recovery

• Capture data in raw form and process to meet evolving integration &

analytic needs

• Ability to overcome relational database limitations to handle rich structure

• Data Science discipline helps meet business demands

• Extension of BICC framework to manage Big Data class of problems


The Solution Stacks

Hitachi Innovation Platforms Aligned to Each Big Data Problem Class

Hadoop and related

tools

Stream Data

Monitoring and

Analysis tools

Text Parsing /

Taxonomy / Video

analysis / Indexing

tools

Statistical algorithms

Volume Velocity Variety Complexity

Choose the tools that align to your particular class of Big Data problems

Ex: High Data Volumes,

Mechanical Sensors,

Machine to machine

Ex: Real Time Traffic,

Stock/Financial Ex: Social Media &

Marketing

Ex: Predictive

Analytics, Process

Optimization

• High Data Volume Solutions (Volume)

• Real-time Analytics Solutions (Velocity)

• Unified Information Access Solutions (Variety)

• Advanced Analytics (Complexity)


Depending upon the enterprise and its requirements, the IT department will need to

develop capabilities and services in one or more of the Solution Stacks. The new

architectures are particularly demanding in a variety of ways:

• Skills Development

• Time to Market

• Disaster Recovery

• Operations and Maintenance

The Big Data Application Development Process

Assessing the Capabilities to Deliver New Analytics Applications


Some applications are easily migrated to the cloud and others strain under remote network

conditions, multi-tenancy security risks, virtualization performance overhead, or other cloud

model characteristics. Some of these characteristics are particularly troublesome for big

data solutions, while others are particularly advantageous:

• Performance in the Cloud

• Security in the Cloud

• Data Growth in the Cloud

• Business Continuity in the Cloud

The Big Data Platform Environment

How Does Cloud Delivery Impact Big Data Applications?

• Governance in the Cloud

• Termination of Services in the

Cloud

• Costs & TCO


Depending upon the rigor and maturity of their IT processes, an enterprise may or may not

have the opportunity to formally present their requirements and to evaluate various

platform options. For those teams who are able to do so, they may follow a simple

evaluation process outline:

• State the Prioritized Requirements

• Measure the Options

• Weigh the Options in terms of time, cost, and organization impact

The ultimate choice of a Big Data architecture and its components will be dependent upon

the solution class, the infrastructure site, and the infrastructure platform . These choices

will impact the feasibility, cost, and the success of the Big Data application.

The Big Data Evaluation and Decision Process

Weighing the Delivery Options


Big Data Solution Class & Infrastructure Platforms

Infrastructure Platform Suitability Varies by Solution Class

Infrastructure Platform Single tenant

dedicated

Multi-tenant

shared

Virtualized

Multi-tenant private

cloud

Virtualized

Multi-tenant

public cloud

Supported sites • Customer site

• Managed Hosting

• Managed Hosting • Managed Hosting

• Public Cloud

High Volume

High Velocity

Unstructured Data

Advanced Analytics

Managed Services

Lower Performance Platforms

Production Performance

Platforms

Big

Da

ta S

olu

tio

n C

lass


Big Data Solution Infrastructure Platform Considerations

Infrastructure Platform Suitability Varies by Success Criteria

Single tenant

dedicated

Multi-tenant shared Virtualized

Multi-tenant

private cloud

Virtualized

Multi-tenant

public cloud

Security / Privacy

Internal Infrastructure Data

Movement

Performance impact

dependent upon

configurable bandwidth

Performance impact

dependent upon job mix from

all tenants

Performance impact dependent

upon job mix from all tenants

Performance impact dependent

upon job mix from all tenants

Data location control Close proximity data is

practical

Some impact to data access

due to job mix

Some impact to data access

because of job mix

Least control over data location

User access network

configuration cost/effort

Network connectivity must

be configured

Network connectivity must be

configured

Network connectivity must be

configured

Connectivity is available

Performance Control

Performance throughput Attainable Impacted Degraded Degraded

End-user demand pattern Consistent, steady demand

and loads

Inconsistent with idle and/or

high demand periods

High variability, extreme peaks

and lulls

High variability, extreme peaks

and lulls

High availability Highest cost and effort Shared cost and effort

Shared cost and effort

Readily available at lowest cost

and effort

Cost

Lower Performance Platforms Production Performance Platforms

Success

Criteria

Infrastructure

Platform


Different Big Data Problem Classes

A single Big Data Platforms will likely not meet all the different requirements

Different Big Data Solutions

For Volume, Velocity, Variety & Complexity

Different Big Data Platform Tools

Different stacks of tools and applications for each class of problem

Different Big Data Infrastructure Options

Multiple infrastructure choices with different advantages and drawbacks

Different Big Data Delivery Options

Choices in how you manage the infrastructure, platform and tools

Big Data Solution Platforms…. So What…?

Prepackaged Big Data Platforms are Not One Size Fits All!

Better

Hitachi Consulting Big Data Analytics Solutions G l o b a l C e n t e r f o r I n n o v a t i o n A n a l y t i c s


Big Data Analytics Solutions - The Solution Stacks

Hitachi Analytics Platforms Aligned to Each Problem Class

Hadoop and related

tools

Stream Data

Monitoring and

Analysis tools

Text Parsing /

Taxonomy / Video

analysis / Indexing

tools

Statistical algorithms

Volume Velocity Variety Complexity

Hitachi Innovation Platform for

Unified Information Access

Solutions (Variety)

Document Processing Solutions

Document storage solution – optional

cloud management system

Document index and search solution

Taxonomy

Unified Information Access tool for

integration with structured analytics

Audio/ Video Processing Solutions

Data object storage solution – optional


Speech recognition and translation tool

Video recognition and identification tool

BI Analytics tools

Hitachi Innovation Platform for Real-

time Analytics Solutions (Velocity)

In-memory Solutions

Custom server appliance – optional


Near-line storage

HANA or equivalent software

Etc.

Data Streaming Real-time Solutions

Hitachi’s HSDP (Hitachi Stream Data

Platform) and CQL (Continuous Query

Language)

Storage platform – optional cloud

management system

BI Analytics tools

Hitachi Innovation Platform for

Advanced Analytics (Complexity)

Pattern-matching Solutions

Data storage solution – optional cloud

management system

Statistical algorithm software

BI Analytics tools

Predictive Modeling Solutions

Data storage solution – optional cloud

management system

Statistical algorithm software

BI Analytics tools

Hitachi Innovation Platform for High

Data Volume Solutions (Volume)

High Volume Data storage/processing

Solutions

Hadoop HW and SW – optional cloud

management system

Hadoop Management tools

Ingestion tools

Search tools

Calculation tools


Big Data Analytics Solutions

Hitachi Business Solution Focus

ADVANCED

ANALYTICS

Prescriptive

Analytics “What Should I Do”

Predictive

Analytics “What Will Happen”

Descriptive

Analytics “What Happened

and Why”

Human

Capital

Advanced analytics to

help organizations

understand the causal

relationships that

influence productivity,

safety and employee

performance to

improve hiring,

training /

development,

promotion and

retention practices

Customer,

Sales &

Marketing Advanced analytics

to help businesses

and agencies to

build deeper

understanding of

customer

preferences &

behaviors to expand

their customer base,

improve customer

satisfaction, and

drive customer

profitability

Financial

Performance &

Risk Mgmt. Advanced analytics to

enable businesses and

agencies to gain more

visibility, control, and

understanding over

financial processes and

performance and to

more effectively

manage risk, fraud, and

compliance across an

organization

Operations

Advanced analytics to

provide companies the

deep insight and

analysis needed to

become more

responsive

operationally by

anticipating and

responding to changes

aligning organizational

decision making with

operational and

strategic objectives

Business Solution Focus



Hitachi Consulting & Hitachi Data Systems Trusted by Leading Organizations

FORTUNE® GLOBAL 500 COMPANIES

9/10 OF TECHNOLOGY FIRMS

>1/2

OF ENERGY, RETAIL, MANUFACTURING, INSURANCE, TRANSPORTATION, FINANCIAL SERVICES

3/4 OF HEALTHCARE AND TELCO

82% OF FORTUNE GLOBAL 100



Hitachi Breadth of SAP HANA Services

THE HITACHI DIFFERENCE

Hitachi is equipped to support every aspect of a HANA solution. We

are one of the very few SAP Partners who can provide this level of

support.

Hitachi HANA Breadth of Services

Strategy

Roadmap

Value Analysis

Proof-of-Value

Investment Strategy

Converged

Platform®

Hardware

Software

Storage

Services

HANA

Appliance

Sizing

Configuration

Scaling

Implement

Accelerators

Rapid Deployment Solutions

SAP and Hitachi Accelerators

Solutions

Demand Signal Management

Integrated Planning

Custom Development

Services

Managed Services

Hosting

Cloud Services

Equipping


Big Data Analytics Solutions - SAP HANA Live in Five

Value for Enterprise and BW on SAP HANA

Rapid use case(s) solution development

Defined project metrics and ROI

Rapid time to value for phase 1

1-2 day Workshop to define use case(s) / scope

5 weeks from start to finish!

6. Develop plan to move into production

1. Workshop

Use Case development

Metrics, goals for project

Scope, data sources

Develop Project timelines

2. Blueprint to execution

Standard Content and Modeling

Data integration development

Reporting and analytics development

Knowledge transfer

3. Tuning and testing

User testing

Technology testing & validation

4. Validation Business and IT Metrics

Goals

5. Acceptance Solution signoff


1. Enterprise Data Warehouse

2. Agile Marts

3. Near Real-Time SAP Operational Reporting

4. Near Real-Time Data Hub

5. Big Data Analytics platform

6. Demand Signal Repository

7. Custom Web and Mobile Application platform

8. Advanced Analytics and Predictive Modeling platform

SAP HANA Analytics Customer Use Cases and Value

How are our Customers using SAP HANA?


The way you get started in Big Data will impact what you end up doing in your enterprise

production deployment down the road.

• Data Model Complexities

• Agility, Flexibility & Configurability of the Environment

• Data Consistency

• Reliability, Availability, Performance, Security

• Operations & Management

So, it’s important to be proactive and make the right decisions and choices for your Big

Data solution stack, infrastructure and hosting environment. We recommend starting with

a Proof of Concept (POC) to explore and validate your business requirements.

There is a Great Deal to Consider to get Started with Big Data

So how do you Choose?


Big Data Analytics Solution POCs

Big Data POC & Governance Process

Scoping Workshop

Qualification &

Approval - Solution Stack

- HDS

- HCC

- Customer Signs

for POC

Delivery Execution

- POC Platform

- POC Services

Results Acceptance

by Customer Signs

- License

- Implementation Platform

- Implementation Services

HDS Big Data

Platform Delivery

Licensing Services Delivery

1 2 3 4 6 5 7


Big Data Analytics Solution POCs

Big Data POC & Governance Process

Big Data

Platform

Roles

Introduce

opportunity

Further qualify

and approve

POC

Provide

Licensing

Provide

License

HDS Roles Propose

platform solution

Further qualify

and approve

POC

Deliver platform,

install, and

config

Demonstrate

Technical

Platform results

Plan,

Coordinate,

Deliver

Implementation

Platform, install

and config

HCC Roles Propose

application

solution

Further qualify

and approve

POC

Deliver design,

develop, and

demonstrate

results

Demonstrate

Application

Solution results

Plan, Coordinate,

Deliver

Implementation

Services

Customer

Roles

Offer use case

and

requirements

Execute POC

Agreement

Provide facility

(optional),

coordinate,

support

Evaluate results

Execute

Implementation

Agreement

Receive and

accept

Maintain

support,

integrate, and

accept

Change

management

support, integrate,

and accept

Change

management

1 2 3 4 6 5 7

Scoping

Workshop

Qualify

Approve

Delivery

Execution

Acceptance License Implementation

Platform Implementation

Services

Implementation



Advanced Analytics Customer Centers, Client Service Teams, and Big Data Labs

Santa

Clara

Dallas

NYC London

Manchester

- Advanced Analytics Customer Center - Client Teams - Big Data Lab

Denver


Explore Big Data Problem Classes

For Volume, Velocity, Variety & Complexity

Discover Hitachi Big Data Solutions

Determine how Hitachi Big Data Analytics Solutions Can Meet your Business Needs

Leverage Big Data Platform Tools

Implement the Right Stack of Tools & Applications for your Specific Big Data Needs

Implement on the Right Big Data Infrastructure

Leverage the Best-fit Infrastructure to meet your Business Analytics Requirements

Select the Big Data Delivery Option

Make the Right Choices for Manage Dig Data Infrastructure, Platform & Tools

Get Started with a Hitachi Big Data Analytics POC Today!

Hitachi Can Help You Explore Big Data Analytics with a POC


Hitachi, Ltd. – Global Technology Leader

Hitachi, Ltd. ranks 40th on the 2011 FORTUNE Global 500®

Hitachi delivers innovations that answer society’s

challenges. With our talented team and proven

experience in global markets, we can inspire the world.

• Invested $4.8 Billion in R&D (2011)

• Strategic focus on Social Innovation Business

• More than 100 years of product and service

innovation, engineering and quality

Society Changes, Hitachi Transforms It.

• Founded in 1910

• $117.8B FY11 Revenue

• 939 Companies

• 323,540 Employees


Hitachi Consulting – Quick Overview

Delivering Business Transformation Services

Management Consulting

Technology Solutions

Managed Services

• Operational Strategy & Implementation

• Business & Operational Improvement

• Organizational Transformation

• Enterprise Performance Management / Analytics

• IT Advisory

• Environmental Sustainability Solutions

• Enterprise Applications

• Enterprise Technology

• Application Development and Integration

• Cloud & Big Data Solutions

• Technology Infrastructure – Cloud, Premise, Mobile

• Application Maintenance Outsourcing

• Outsourced Product Development

• Business Process Outsourcing

• IT Outsourcing


Hitachi Consulting

Japan / China

Hitachi Consulting

Europe

Hitachi Consulting

North America

Hitachi Global

Delivery Centres

MANCHESTER

HYDERABAD

MADRID

LISBON

BOSTON EDISON,NJ

BETHESDA

PHILADELPHIA

ATLANTA

CHICAGO

HOUSTON DALLAS

DALLAS

SEATTLE PORTLAND

SAN FRANCISCO LOS ANGELES

IRVINE

Hitachi Consulting - Global Capabilities and Backing

Assurance and backing of a global brand:

23rd largest Global Corporation

943 companies; 400,000+ employees

Integrated business with “one team”

approach:

Consistent competencies and

structure across three continents

Design-build-run through a global

delivery model

Ability to deliver global solutions

through global teams

Hitachi Consulting Global capabilities and reach:

Revenues of over $600M

4,400+ employees

30 Offices Worldwide

Global Delivery Centers in India and China

GUANGZHOU

BEIJING

SINGAPORE

SHANGHAI

TOKYO

MIDDLE EAST

LONDON

BARCELONA

PUNE

BANGALORE


Connect with Me:

www.linkedin.com/in/chadmlawler

Today’s Presentation Available at: http://www.slideshare.net/chadmlawler

Chad M. Lawler, Ph.D. Director of Consulting Services

Cloud Computing

14643 Dallas Parkway, Suite 800, Dallas, Texas 75254

Office: 469.221.2894

Email: [email protected]

www.hitachiconsulting.com/cloud/


http://www.linkedin.com/in/chadmlawler



Authors: Chad M. Lawler, Ph.D. Director of Consulting, Cloud Computing Practice - Hitachi Consulting

Art Vancil, CCP, CCSK Senior Manager, Big Data Architect - Hitachi Consulting

Better

Education

The Executive View on Big Data Platform Hosting - Evaluating Hosting Services for Big Data Computing Cloud Expo 2013