Big Data Hands-On Labs:

Date Time LocationTuesday 3:45pm – 4:45pm Hotel Nikko - PeninsulaWednesday 1:15pm – 2:15pm Hotel Nikko - PeninsulaThursday 11:30am – 12:30pm Hotel Nikko - Peninsula

Or download: Big Data Lite Virtual Machine

Oracle Big Data Appliance for Customers and Partners

Jean-Pierre DijcksOracleBig Data Product Management

Paul KentSASVP Big Data

Oracle Big Data Appliance for Customers and Partners

Big Data Appliance Recap

Why You Should Consider Big Data Appliance

Driving Business Value with SAS on Big Data Appliance

Oracle Big Data Management System

Oracle Database

Oracle IndustryModels

Oracle Advanced Analytics

Oracle Spatial & Graph

Big Data Appliance

Cloudera Hadoop

Oracle NoSQL Database

Oracle R Advanced Analytics for Hadoop

Oracle R Distribution

Oracle Database

Oracle Advanced Security

Oracle Advanced Analytics

Oracle Spatial & Graph

Oracle Exadata

Oracle Big DataConnectors

Oracle DataIntegrator

Oracle Big Data SQL

Recap: Big Data Appliance OverviewBig Data Appliance X4-2

Sun Oracle X4-2L Servers with per server:• 2 * 8 Core Intel Xeon E5 Processors• 64 GB Memory• 48TB Disk space

Integrated Software:• Oracle Linux, Oracle Java VM• Oracle Big Data SQL*• Cloudera Distribution of Apache Hadoop – EDH Edition• Cloudera Manager• Oracle R Distribution• Oracle NoSQL Database

* Oracle Big Data SQL is separately licensed

Recap: Standard and Modular

Starter Rack is a fully cabled and configured for growth with 6 servers

In-Rack Expansion delivers 6 server modular expansion block

Full Rack delivers optimal blend of capacity and expansion options

Grow by adding rack – up to 18 racks without additional switches

Recap: Harness Rapid Evolution

BDA 4.0

BDA 4.0 – Sept 2014• Big Data SQL• Node Migration

BDA 3.x – April 2014• CDH 5.0 (MR2 & YARN)• AAA Security• Encryption

BDA 2.x – April 2013• Starter Rack• In-Rack Expansion• EM Integration

BDA 1.0 – Jan 2012• Initial BDA• Mammoth Install

Operational Simplicity Simplify Access to ALL Data

Core Design Principles for Big Data Appliance

• Oracle Big Data SQL – Oracle SQL on ALL your data– All Native Oracle SQL Operators– Smart Scan for Optimized Performance

• Oracle Security – Govern all Data through a Single Set of

Security Policies

Oracle Big Data SQL – A New Architecture

• Powerful, high-performance SQL on Hadoop– Full Oracle SQL capabilities on Hadoop– SQL query processing local to Hadoop nodes

• Simple data integration of Hadoop and Oracle Database– Single SQL point-of-entry to access all data– Scalable joins between Hadoop and RDBMS data

• Optimized hardware– Balanced Configurations– No bottlenecks

Big Data SQL

SELECT w.sess_id, c.nameFROM web_logs w, customers cWHERE w.source_country = ‘Brazil’AND w.cust_id = c.customer_id;

Relevant SQL runs on BDA nodes

10’s of Gigabytes of Data

Only columns and rows needed to answer query are returned

Hadoop Cluster

Big Data SQL

Oracle Database

CUSTOMERSWEB_LOGS

Big Data SQL

SELECT w.sess_id, c.nameFROM web_logs w, customers cWHERE w.source_country = ‘Brazil’AND w.cust_id = c.customer_id;

Relevant SQL runs on BDA nodes

10’s of Gigabytes of Data

Only columns and rows needed to answer query are returned

Hadoop Cluster

Big Data SQL

Oracle Database

CUSTOMERSWEB_LOGS

SQL Push Down in Big Data SQL

• Hadoop Scans on Unstructured Data• WHERE Clause Evaluation• Column Projection• Bloom Filters for Better Join Performance• JSON Parsing, Data Mining Model Evaluation

Feedback Loop

Data Management

Big Data Platform

(Hadoop/NoSQL)

Relational Data Warehouse

(OCDM)

Analytic Apps

Customer Experience

Operations

Monetization

Adapters

ETL/ELT Adapters

Real-Time Adapters

ThirdParty

DataSources

Oracle Comms Apps (BSS/OSS)

Oracle Comms Ntwk Products (Tekelec

& Acme)

Other Oracle Apps (CRM, ERP, etc.)

Third Party Sources

Oracle Communications Data ModelReference Architecture

To Other Apps

• No Bottlenecks• Full Stack Install and Upgrades• Simplified Management

– Cluster Growth– Critical Node Migration

• Always Highly Available• Always Secure• Very Competitive Price Point

Successful Big Data Systems GrowFrom Cluster Install with HA to Large Clusters to Dealing with Operational Issues

Day 1• 12 node BDA for Production• Hadoop HA and Security Set-up • Ready to Load Data

Full install with a single command:

./mammoth –i rck_1

NN Example Service: Hadoop Name Nodes

RCK_1 RCK_2

Day 90Add 12 New Nodes across two Racks

Cluster expansion with a single command:

mammoth –e newhost1,…,newhostn

RCK_1 RCK_2This expansion automatically optimizes HA setup across multiple racks

Cluster Expansion with a single command:

mammoth –e newhost1,…,newhostn

Because of uniform nodes and IB networking,no data is moved

RCK_1 RCK_2

Day nCritical Node Failure => Primary Name Node

RCK_1 RCK_2

• Automatic Failover to other NameNode

• Automatic Service Request to Oracle for HW Failure

RCK_1 RCK_2

• Restore HA with a Single commandbdacli admin_cluster migrate N1

• Reinstate the Repaired Node with a Single Command:

bdacli admin_cluster reprovision N1

Operational Simplicity

Quicker to Deploy

Cheaper to Buy

“Oracle Big Data Appliance is an excellent choice for customers looking to work with the full suite of Cloudera’s leading Hadoop-based technology. It’s more cost-effective and quicker to deploy than a DIY cluster.”

⁻Mike Olson, Cloudera founder, Chief Strategy Officer, and Chairman of the Board

Real-time access to better data means better insights, which means better decisions and better business results

Integrate data associated with customer telemetry, configurations, service history, diagnostics, knowledge & support information

Big Data Initiative @ Oracle Global Support Services

Anticipate Detect Predict Automate Delight

Operational Simplicity

Simplify Access to ALL Data

Core Design Principles Enable Success

There is one more thing…

Business Value = Applications

Big Data Appliance powers instant Business Value

Customer Experience Management

Cyber SecuritySolutions

CommunicationsData Model

Introducing

Paul Kent - SAS

Big Data and Big Analytics – So Much more Gunpowder!Paul KentVP BigData, SAS Research and Development

1. Change 2. Safari Pics

[CON8279] Oracle Big Data Appliance: Deep Dive and Roadmap for Customers and PartnersOracle Big Data Appliance is the premier Hadoop appliance in the market. This session describes the roadmap for customers in the areas of high-performance SQL on Hadoop and securing big data, plus overall performance improvements for Hadoop.

A special focus in the session is the roadmap and benefits Oracle Big Data Appliance brings to Oracle partners.

To illustrate the benefits of running on a standardized and optimized Hadoop platform, SAS presents the findings of its tests of SAS In-Memory Analytics on Oracle Big Data Appliance.

Agenda1. SAS & Oracle Partnership

2. Family Stories1. Hadoop

2. Oracle Engineered Systems Family

3. SAS Software Family

3. Deployment Patterns

Reflection on a stronger partnership than ever

Both leaders in Big Data –

Jointly solving the most difficult and demanding Big Data Problems

Providing simplicity and agility to create flexible configurations

Extensive engineering collaboration

Can we answer:

How Does it Work?

How Does it Perform?

THE TAMOXIFEN DILEMMA

SOURCE: http://commons.wikimedia.org/wiki/File:Tamoxifen-3D-vdW.png

Elephant :: 3 Good Ideas !!1. Never forgets

2. Is a good (hard) worker

3. Is a Social Animal (teamwork)

MPP (Massively Parallel) hardware running database-like software

“data” is stored in parts, across multiple worker nodes

“work” operates in parallel ,on the different parts of the table

Controller Worker Nodes

Hadoop – Simplified View

Head Node Data 1 Data 2 Data 3 Data 4…

MYFILE.TXT

..block1 -> block1

..block2 -> block2

..block3 -> block3

Idea #1 - HDFS. Never forgets!

MYFILE.TXT

..block1 -> block1 block1 copy2

..block3 -> block3 copy2 block3

MYFILE.TXT

..block1 -> block1 block1copy2

..block3 -> block3 copy2 block3X

Redundancy Wins!

Idea #2 – MapReduce – Send the work to the Data

We Want the Youngest Person in the Room

Each Row in the audience is a data node

I’ll be the coordinator

• From outside to center, accumulate MIN• Sweep from back to front. • Youngest Advances

Recap: Standard and Modular

Starter Rack is a fully cabled and configured for growth with 6 servers

In-Rack Expansion delivers 6 server modular expansion block

Full Rack delivers optimal blend of capacity and expansion options

Grow by adding rack – up to 18 racks without additional switches

Oracle Big Data SQL – A New Architecture

• Powerful, high-performance SQL on Hadoop– Full Oracle SQL capabilities on Hadoop– SQL query processing local to Hadoop nodes

• Simple data integration of Hadoop and Oracle Database– Single SQL point-of-entry to access all data– Scalable joins between Hadoop and RDBMS data

• Optimized hardware– Balanced Configurations– No bottlenecks

Oracle Confidential – Internal/Restricted/Highly Restricted 46

Diversity. It’s a good thing!

Impala Nyala

4 Important Things

#1 Join the Family

HADOOP

Hive QLSAS

SERVER

SAS ACCESS to Hadoop

#2 Be Familiar

SAS / High Performance Analytics

HADOOP

SAS HPA

Procedures

SERVER

#3 Use the Cluster!

Prepare Explore / Transform Model

• HPDS2

• HPDMDB

• HPSAMPLE

• HPSUMMARY

• HPCORR

• HPREDUCE

• HPIMPUTE

• HPBIN

• HPLOGISTIC

• HPREG

• HPNEURAL

• HPNLIN

• HPCOUNTREG

• HPMIXED

• HPSEVERITY

• HPFOREST

• HPSVM

• HPDECIDE

• HPQLIM

•HPLSO

•HPSPLIT

•HPTMINE

•HPTMSCORE

Controller

Client

#1 Join the Family

#2 Be Familiar

#3 Use the cluster

#4 Have a pretty face!

4 Important Things (for cluster friendly software)

1.Join the Family

2.Be Familiar

3.Performance

4.Have a pretty face

SAS BIG DATA ON BIG DATA APPLIANCE

• Flexible Architectural options for SAS deployments• Can run on Starter, Half and Full configurations

• Optionally select nodes “N, N-1, N-2, …” for additional SAS

Services such as SAS Compute Tier, SAS MidTier

• Optionally select node subset “N, N-1, N-2, N-3, …) for more

dedicated resources for SAS Analytic Compute Environment by

shifting Big Data Appliance roles

• Option to selectively add more memory on a per node basis

depending on specific workload distribution

SAS Midtie

STARTER BDA

SAS Visual Analytics

Metadata ServerSAS Compute

SAS HPA Root Node

SAS VISUAL ANALYTICS, HIGH-PERFORMANCE ANALYTIC COMPUTE ENVIRONMENT CO-LOCATED WITH HADOOP

SAS Midtie

STARTER BDA

SAS HPA Root Node

Consider:

Extra Memory for 5,6?

SAS Midtie

FULL RACK BDA

LASR Worker

HDFS Data 17

SAS HPA Root Node

LASR Worker

HDFS Data 18

FULL RACK BDA ASSEMBLED IN OSC, SYDNEY AUSTRALIA

Basic Smoke Tests Confirmed:Interoperate with Hadoop and Map Reduce

Read and Write text files to/from HDFS

Read and Write Tabular files to/from Hive (will confirm Oracle BIGSQL in OSC-SC)

Read and Write SAS binary format files to/from HDFS

High Degree Of Parallelism (DOP) reads via Map-Only jobs

SAS LASR server co-exists on/with datanodes

SAS HPA tasks scheduled on datanodes

Table 1: Summation of 5/20/100/200 columns; Baseline: DOP=1 (no parallelism)120M rows, 400 columns, reg_simtbl_400

SAS High-Performance Analytics PerformanceSAS Format Data (SASHDAT)

1107 var11.795 Mobs97GB5.7GB/node

1107 var73.744 Mobs608GB35.7GB/node 6x

Create 208.79 sec 2284.29 sec 11

Scan/Count 24.60 sec 259.38 sec 10.5

HPCORR 295.20 1410.40 4.7

HPCNTREG 336.79 1547.59 4.6

HPREDUCE (u) 236.55 2467.76 10.4

HPREDUCE (s) 219.50 2037.74 9.3

OSC-AU FullRack BDA

• 408 Threads

• 600 GB dataset

• 17 servers

Your Problem solved ASAP

EXADATA INTEGRATION

SAS EMBEDDED PROCESSING (EP) TO EXADATALEVERAGING BIG DATA SQL

SAS Midtie

LASR Worker

…HDFS Data 18

SAS HPA Root Node

SAS EP

Big Data SQL

SAS High-Performance Analytics PerformanceSAS EP Parallel Data Feeders

DOP=1 DOP=24 DOP=24(flash cache)

Add(5) 1.25min 1.5min .5min

Add(20) 2.5min 1.5min .5min

Add(100) 13min 1.5min .6min

Add(200) 16min ~2min 1.25min (10x)

Table 2: Scan times for 2 tables (200 columns, 400 columns, 120M rows); Baseline: SAS/ACCESS vs. HPA EP feeder

SAS High-Performance Analytics PerformanceSAS EP Parallel Data Feeders

Access Access /DBSlice

SAS HPAUsing EP

Reg_sim_200 1:01:12 0:28:37 0:08:00

Reg_sim_400 1:49:11 0:55:33 0:16:05 (7x!)

SAS High-Performance Analytics PerformanceSAS Format Data (SASHDAT) and Oracle EXADATA

1107 var11.795 Mobs97GB5.7GB/nodeSASHDAT

907 var11.795 Mobs79.7GB4.7GB/nodeEXADATA

1107 var73.744 Mobs608GB35.7GB/nodeSASHDAT

Create 208.79 sec 931.22 sec 2284.29 sec

Scan/Count 24.60 sec 956.16 sec 259.38 sec

HPCORR 295.20 833.24 1410.40

HPCNTREG 336.79 756.97 1547.59

HPREDUCE (u) 236.55 1055.11 2467.76

HPREDUCE (s) 219.50 1051.93 2037.74

ORACLE ENGINEERED SYSTEMS FOR

SuperClusterExaData ExaLogic Virtual

Compute

Appliance

Big Data

Appliance

Database

Backup, Recovery,

Logging Appliance

Storage

Appliance

SAS AND ORACLE WORKING TOGETHER TO CREATE CUSTOMER VALUE

• Joint R & D development and Product Management teams in Cary and Redwood Shores

• Focus on driving SAS technology components to run natively in Oracle database

• Joint performance engineering optimizations

• Template physical architectures developed based on use-cases

• Physically tested and benchmarked together

• Reduction in physical effort• Overall reduction in lifecycle

• Best Practice papers• SAS and Oracle Engineers

provide joint "Sizing and Architecture Analysis and Design"

SAS AND ORACLEBETTER TOGETHER

Paul.Kent @ sas.com

@hornpolish

paulmkent

Big Data Hands-On Labs:

Documents

Virtual Training Labs: Delivering Hands-On Classrooms … · 5 Virtual Training Labs: Delivering Hands-On Classrooms in Cloud Environments ©Skytap,Inc. Allrightsreserved. 4. puting

Big Huge Labs

Hands-on Labs: Raspberry Pi 2 + Windows 10 IoT Core

WiMAX in the Classroom: Teaching Cellular Networking with Hands-on Labs

ATS zManager Workshop Hands-on Labs - IBM WWW Page · ATS zManager Workshop Hands-on Labs ... Run the workload and observe z/OS enclave creation ... Open Firefox and point your browser

Hands on Labs - Windows Forms

Saturday, May 2, 2015 Hands-On Labs and Workshops

Accion Labs - Big Data Services

Effectiveness of Simulation versus Hands-on Labs: A Case ... · Paper ID #13152 Effectiveness of Simulation versus Hands-on Labs: A Case Study for Teach-ing an Electronics Course

Partner Exchange 2015 Hands-on Labs Guide - VMware · 2016-05-05 · Partner Exchange 2015 Hands-on Labs Guide Listed below are the Hands-on Labs that will be available at Partner

Enterprise Library Hands on Labs

Big hands lifting the capesize curve

Little Seeds in Big Hands

MultiStore Hands On Advanced Labs Data ONTAP 8.1 7 … · MultiStore Hands-On Advanced Labs Data ONTAP 8.1 7-Mode Scott Gelb, Insight Integrated Systems

Hands -on Labs - ICCS · 2011. 1. 24. · hands-on labs. The Parboil framew of Illinois, and is meant to provide an easier interface to manipulate benchmarks and measure performance

Partner Exchange 2015 Hands-on Labs Guide - VMware · Partner Exchange 2015 Hands-on Labs Guide ... 1 Review Infrastructure in the vSphere Web Client 15 2 ... as well as external

VMworld 2014 Hands-on Labs Guide - blogs. · PDF fileVMworld 2014 Hands-on Labs Guide ... This lab will focus on taking the traditional benefits of vSphere and extending it further

Serverquarium 2013 Powering the Hands-On-Labs at MMS

Hands-On Big Data

Partner Exchange 2015 Hands-on Labs Guidedocs.hol.vmware.com/announcements/PEX2015.pdfPartner Exchange 2015 Hands-on Labs Guide Listed below are the Hands-on Labs that will be available