IM Forum 2013 PureData v2.0 FINAL.ppt

8/11/2019 IM Forum 2013 PureData v2.0 FINAL.ppt

1/24

IBM Information Management Forum, 11. September 2013, Wien

Pure Data System for HaoopBig Data !pp"ian#e

Dipl.Ing. Wolfgang Nimfhr

Business Development ExecutiveIBM Software GroupBig Data


2/24

!"#$ International Business Machines %orporation

! IBM Information Management &orum' ##. Septem(er !"#$' Wien

$%ery #ompany &as a Big Data ' !na"yti#s (pportunity

)&e po*er of Data #oming toget&er

Wit& t&e po*er of )e#&no"ogy

)o e"i%er Impro%e (ut#omes

$nri#& your information basewith Big Data Exploration

Pre%ent #rimewith Security and Intelligence Extension

(ptimi+e operationswith Operations Analysis

ain I) effi#ien#y an s#a"ewith Data Warehouse Augmentation

Impro%e #ustomer intera#tionwith Enhanced 360 !iew o" the #ustomer

)o generate Business -a"ue


3/24


$ IBM Information Management &orum' ##. Septem(er !"#$' Wien 2013 IBM /orporation

Ho* o you &arness t&e e"ep&ant in t&e room


4/24


5/24


4 IBM Information Management &orum' ##. Septem(er !"#$' Wien

ive

5ig

Map+e/uceD&S

HCatalog

Visualization

Development

6ools

et5s simp"ify Big Data 6

Designed to

Simplif- the (uil/ing'

/eplo-ing an/ management

of a a/oop cluster

Spee/ the time2to2value for

a/oop an/ unstructure/

/ata

Maximi7e the overall

anal-tic ecos-stem 5rovi/e enterprise securit-

an/ platform management

From #ustom an #omp"e7 6)o organi+e simp"i#ity

$Based on IB% internal testing and customer "eed&ac'( )#ustom &uilt clusters) re"er to clusters that are not

pro"essionally pre*&uilt+ pre*tested and optimi,ed( Indi-idual results may -ary(


6/24



!nnoun#ing t&e ne* PureData System for Haoop

0ccelerate time to value

0ccelerate time to insight

Simplif- (ig /ata a/option an/ consumption

Exten/ the value of the /ata warehouse

Implement enterprise class (ig /ata

Minimi7e s-stem setup an/ a/ministration

System for Hadoop


7/24



IBM PureData System for HaoopAccelerate .adoop analytics with appliance simplicity

Spee

Spee/ to insight with (uilt2in anal-tics Spee/ to value with accelerate/ /eplo-ment

Simp"i#ity

+ea/- to loa/ /ata in hours

Integrate/ s-stem management

0ppliance approach re/uces complexit-

Single point of support

Smart

Esta(lish a cost efficient online /ata archive

Easil- leverage /ata across the (ig /ataplatform

Enterprise securit-' governance an/ high

availa(ilit-

System for Haoop


8/24


: IBM Information Management &orum' ##. Septem(er !"#$' Wien

Benefits of IBM 5ureData S-stem for a/oop Dep"oy 87 faster

than custom*&uilt solutions$

Bui"t9in %isua"i+ation

to accelerate insight

Bui"t9in ana"yti# a##e"erators2unli'e &ig data appliances on the mar'et

Sing"e system #onso"e

"or "ull system administration

:api maintenan#e upates

with automation

;o assemb"y re


9/24


A IBM Information Management &orum' ##. Septem(er !"#$' Wien

InfoSp&ere BigInsig&tsEntry 1oint "or .adoop


10/24


#" IBM Information Management &orum' ##. Septem(er !"#$' Wien

!na"yti# !##e"erators Social Me/ia 0ccelerator Machine Data 0ccelerator BigSheets sprea/sheet an/ visuali7ation 0/vance/ 6ext 0nal-tics 0ccelerator C0? ,uer- language

Performan#e an (ptimi+ation 0/aptive Map +e/uce

0/vance/ Sche/uler BigIn/ex for large scale in/exing &ast' splitta(le compression

Se#urity +ole (ase/ authori7ation

(ptim De%e"opment Stuio Eclipse (ase/ IDE for Cava

Big Data Integration Information Server' InfoSphere

Streams' Nete77a' DB!

$nterprise $nab"ement Big S?

G5&S2&%>

IB%2s distri&ution is &ased on Apache .adoop and utili,es many o" the capa&ilities

includes in that distri&ution+ &ut IB% is "ocused on ma'ing its distri&ution more o" an

enterprise class o""ering(

BigInsig&ts -a"ue !bo%e an Beyon Haoop


11/24


## IBM Information Management &orum' ##. Septem(er !"#$' Wien

IBM2certifie/ 0pache a/oop

!ministration ' Se#urity

Wor4"oa (ptimi+ation

Integrate De%e"opment $n%ironment

/onne#tors =Data, !na"yti#s, Integration>

!%an#e )e7t !na"yti#s $ngine

-isua"i+ation ' $7p"oration

(pensour#e#omponents

!itiona"

enterprise#apabi"ities

?ey enterprise #apabi"ities on top of an unmoifie opensour#e founation


12/24


#! IBM Information Management &orum' ##. Septem(er !"#$' Wien

1

2

BigInsig&ts 2.1 features a %ariety of en&an#ements t&ate"i%er 4ey $nterprise Haoop #apabi"ities

* >ut of the (ox igh

0vaila(ilit-

* Seamless' automatic an/

transparent failover for D&S

NameNo/e* Eliminates a/min intervention

* +e/uces /owntime for

recover- of the cluster

* ar/ware fencing to

guarantee /ata integrit-

* No single point of failure

* Built2in igh 0vaila(ilit-

* 5>SI compliance

* Enhance/ Securit- with 0%?

support

* Support for Storage 5ools

* SnapShot capa(ilit-

* %omprehensive Stan/ar/ 0NSI

S? support to access /ata

store/ in BigInsights

* Stan/ar/s compliant CDB% F

>DB% /rivers* ?everages Map+e/uce

parallelism in complex /ata sets

* Direct access for low2latenc- in

small ,ueries' e.g. su(2secon/

response to Base ,ueries

Hig& !%ai"abi"ityPFS9FP( supportBig S@ /ognos %10, 6


13/24


#$ IBM Information Management &orum' ##. Septem(er !"#$' Wien

BigInsig&ts $nterprise $ition 2.1

%onnectivit- an/ Integration Streams

Nete77a

6extprocessingengine an/li(rar-

CDB%

&lume

Infrastructure Ca,l

ive

5ig

Base

Map+e/uce

D&S

ooHeeperIn/exing ?ucene

0/aptiveMap+e/uce

>o7ie

6ext compression

Enhance/securit-

&lexi(lesche/uler

>ptional

IBM an/

partner

offerings

0nal-tics an/ /iscover- 0ppsJ

DB!

BigSheets

We( %rawler

Distri( file cop-

DB export

Boar/rea/er

DB import

0/ hoc ,uer-

Machinelearning

Dataprocessing

. . .

0/ministrative an/

/evelopment tools

We( console

* Monitor cluster health' Ko(s' etc.

* 0// L remove no/es

* Start L stop services

* Inspect Ko( status

* Inspect wor;flow status* Deplo- applications

* ?aunch apps L Ko(s* Wor; with /istri( file s-stem

*Wor; with sprea/sheet interface

*Support +ES62(ase/ 05I* . . .

+

Eclipse tools

* 6ext anal-tics* Map+e/uce programming

* Ca,l' ive' 5ig /evelopment

* BigSheets plug2in /evelopment* >o7ie wor;flow generation

Integrate/installer

>pen Source IBMIBM

%ognos BI

Big S?

0ccelerator formachine /ataanal-sis

0ccelerator forsocial /ataanal-sis

Guar/ium DataStageData Explorer

S,oop

%atalogG5&S &5>G5&S &5>


14/24


#) IBM Information Management &orum' ##. Septem(er !"#$' Wien

$7p"oration,Integrate

Ware&ouse, anMart Aones

Discover-

Deep +eflection >perational

5re/ictive

All Data Sources

Information

Ingestion

an

(perationa"

Information

/aseManagement

!na"yti#s

!pp"i#ations

!"erts

aning !rea,

!na"yti#s Aone

an !r#&i%e

+aw Data Structure/ Data 6ext 0nal-tics

Data Mining Entit- 0nal-tics

Machine?earning

:ea"9time

!na"yti# Aone i/eoL0u/io

Networ;LSensor

Entit- 0nal-tics

5re/ictive

Stream5rocessing

DataIntegration

Master Data

Streams

Information o%ernan#e, Se#urity an Business /ontinuityInformation o%ernan#e, Se#urity an Business /ontinuity

Big Data Ecosystem Analytic Applications

/ogniti%e4earn Dynamically5

Pres#ripti%e

Best Outcomes5

Prei#ti%eWhat #ould .appen5

Des#ripti%eWhat .as .appened5

$7p"oration anDis#o%ery

What Do ou .a-e5

/"ou

Ser%i#es

IBM Watson

IBM Big Data ' !na"yti#s :eferen#e !r#&ite#ture

PureData

System

for

Haoop

!#ti%e

!r#&i%e

Big Data$7p"oration

Pre

Pro#essingHub


15/24


#4 IBM Information Management &orum' ##. Septem(er !"#$' Wien

7se #ase8 Big Data $7p"oration

se /ases

Explore new /ata an/ previousl-

untappe/ sources

isuali7e an/ gain new insight with eas-

to use sprea/sheet2st-le anal-sis

I/entif- useful information that woul/

a// value when integrate/

=se/ for /ata profiling to un/erstan//ata (efore moving to other s-stems


16/24



0/2hoc anal-tics for /ata scientists

0nal-7e a variet- of /ata 2

unstructure/ an/ structure/

Browser2(ase/

Sprea/sheet metaphor for exploringL

visuali7ing /ata

Gather Extract Explore Iterate

%rawl gather statisticall-

0/aptergather /-namicall-

Document2level info

%leanse' normali7e

0nal-7e' annotate' filter

isuali7e resultsIterate through an- prior

step

Sprea/sheet2st-le anal-sis process with BigSheets

7se #ase8 Big Data $7p"oration


17/24



7se #ase8 !#ti%e !r#&i%e

se /ases

Imme/iate storage alternative of col/

/ata

%ost savings for col/ /ata

%ompliance re,uirements

Simple anal-tics L exploration

PureDataSystem for Analytics

PureDataSystem for Hadoop


18/24


#: IBM Information Management &orum' ##. Septem(er !"#$' Wien

7se #ase8 Pre9Pro#essing Hub

se /ases

0ggregation of /ata

5re2process cleansing

%ompliance re,uirements

Simple anal-tics L exploration


19/24


#A IBM Information Management &orum' ##. Septem(er !"#$' Wien

PureData System for HaoopBringing Big Data to the enterprise

Simplif- the /eliver- of unstructure/

/ata to the enterprise

Integrate a/oop with the /ata

warehouse

?everage a/oop for /ata archive

5rovi/e (est in class securit-

5rovi/e /ata exploration across

structure/ an/ unstructure/ /ata

0ccelerate insight with machine /ata

0ccelerate insight with social /ata

Simp"ify

BigDa

ta

fort&e

enter

prise1


20/24


!" IBM Information Management &orum' ##. Septem(er !"#$' Wien

For apps "i4e $9#ommer#e6

Database cluster services optimized for

transactional throughput and scalability

For apps "i4e /ustomer !na"ysis6

Data warehouse services optimized for

high-speed, peta-scale analytics and simplicity

For apps "i4e :ea"9time Frau Dete#tion6

perational data warehouse services optimized to

balance high performance analytics and real-time

operational throughput

!eeting "ig Data #hallenges $ %ast and &asy'

IBM PureData System Fami"y

System for Transactions

System for Analytics

System for Operational Analytics

System for Hadoop

For $7p"oratory !na"ysis ' @ueryab"e !r#&i%e

Hadoop data services optimized for big data analyticsand online archive with appliance simplicity

2013 IBM /orporation20


21/24


22/24


!! IBM Information Management &orum' ##. Septem(er !"#$' Wien

$7p"oration,Integrate

Ware&ouse, anMart Aones

Discover-

Deep +eflection >perational

5re/ictive

All Data Sources

Information

Ingestion

an

(perationa"

Information/ase

Management

!na"yti#s

!pp"i#ations

!"erts

aning !rea,

!na"yti#s Aone

an !r#&i%e

+aw Data Structure/ Data 6ext 0nal-tics

Data Mining Entit- 0nal-tics

Machine?earning

:ea"9time

!na"yti# Aone

i/eoL0u/io

Networ;LSensor

Entit- 0nal-tics

5re/ictive

Stream5rocessing

DataIntegration

Master Data

Streams

Information o%ernan#e, Se#urity an Business /ontinuityInformation o%ernan#e, Se#urity an Business /ontinuity

Big Data Ecosystem Analytic Applications

/ogniti%e4earn Dynamically5

Pres#ripti%eBest Outcomes5

Prei#ti%eWhat #ould .appen5

Des#ripti%eWhat .as .appened5

$7p"oration anDis#o%ery

What Do ou .a-e5

/"ou

Ser%i#es

IBM Watson

IBM Big Data ' !na"yti#s :eferen#e !r#&ite#ture

7%ise

Data

$7p"orer


23/24


!$ IBM Information Management &orum' ##. Septem(er !"#$' Wien


24/24


!) IBM Information Management &orum' ##. Septem(er !"#$' Wien

Part of t&e IBM Big Data P"atform(or)load ptimized Solutions for all your analytic needs

!na"yti#s ' De#ision Management

So"utions

Big Data Infrastru#ture

IBM Big Data P"atform

!##e"erators

Information Integration ' o%ernan#e

-isua"i+ation' Dis#o%ery

!pp"i#ationDe%e"opment

SystemsManagement

Stream/omputing

HaoopSystem

DataWare&ouse

PureDataSystem for Analytics

PureDataSystem for Hadoop

2013 IBM /orporation2E

Documents

IM Forum 2013 PureData v2.0 FINAL.ppt