Upload
hoanganh
View
215
Download
0
Embed Size (px)
Citation preview
© 2016 IBM Corporation
Cognitive Solutions in the Context of IBM SystemsCognitive Analytics / Integration Scenarios / Use Cases
Mano SrinivasanOpen Source Solutions Architect [email protected]
© 2016 IBM Corporation2
Topics and Questions to be addressed
What is Cognitive Analytics
What does IBM has to offer and What are the key Use Cases?
Integration Scenarios and the role of Open Source, incl. SystemML and Apache Spark
Summary
© 2016 IBM Corporation4
99%60%10%
Understands natural language and human speech
Adapts and Learns from user selections and responses
Generates and evaluates
hypothesis for better outcomes
3
2
1
Cognitive Analytics in the Context of Big Data IBM Watson drives optimized Outcomes
© 2016 IBM Corporation6
When Would we use Cognitive Analytics?
When patterns exists in our data− Even if we don’t know what they are
We can not pin down the functional relationships mathematically − Else we would just code up the algorithm
When we have lots of (unlabeled) data− Data is of high-dimension
• High dimension “features” (For example, sensor data)
© 2016 IBM Corporation7
The need for cognitive analytics is driven by the confluence of SoLoMo (Social, Local, Mobile), Big Data, and Cloud
VeracityVeracity VarietyVariety
VelocityVelocity VolumeVolume
Cognitive Systems
Cognitive Analytics in the Context of Big Data – Key Drivers
© 2016 IBM Corporation8
Topics and Questions to be addressed
What is Cognitive Analytics ??
What does IBM has to offer?
Key Use Cases and Integration Scenarios
Summary and Takeaway
© 2016 IBM Corporation9
The Evolution of Analytics
CognitiveAnalytics
PredictiveAnalytics
PrescriptiveAnalytics
DescriptiveAnalytics
Descriptive“After-the-facts” analytics by analyzing historical data Provides clarity as to where an enterprise or an organization stands related to defined business measures Applied to all LoB for fact finding, visualization of success and failure
CognitivePertaining to the mental processes of perception, memory, judgment, learning, and reasoningRange of different analytical strategies that are used to learn about certain types of business related functionsNatural language processing
PredictiveLeverages data mining, statistics and ML algorithms, etc. to analyze current and historical data to predict future events and business outcome. Discovers patterns derived from historical and transactional data to optimize business measures
PrescriptiveSynthesizes big data, mathematical and computational sciences, and business rules to suggest decision optionsTakes advantage of a future opportunity or mitigate a future risk and shows the implication of each decision option
© 2016 IBM Corporation10
Scope of Advanced Analytics – leading towards Cognitive Business
IBM Analytics breadth covers the full spectrum of decisions IBM z Analytics contributes and enables this breadth of analytics
Descriptive
Prescriptive
Predictive
Cognitive
What has happened?
What could happen?
How can we achieve the best outcome?
How can we learn dynamically?
IBM BrandedBig Data and
AnalyticsPlatform
IBM BrandedBig Data and
AnalyticsPlatform B
usin
ess
Valu
e
Information Layer How is data managed and stored?
How can everyone be more right….more often?
Source: IBM and IDC Business Analytics, Business Rules Management Systems 2012 WW market estimates
IOP
© 2016 IBM Corporation11
Cognitive Business and its Analytics Foundation in IBMA Watson-centric View
Offerings:
Applications:
Watson ExplorerWatson AnalyticsWatson Curator
Watson Services on BlueMixWatson Developer Cloud (Bluemix)Watson ToolingWatson Health
Solutions:
IBM Analytics
Source: http://www.ibm.com/analytics/us/en/ and http://www.ibm.biz/cognitivera
Watson Engagement AdvisorWatson Discovery AdvisorWatson Policy AdvisorWatson Decision AdvisorWatson Company Analyzer
Watson for Wealth ManagementWatson for OncologyChef Watson
Products:
Platform:
Behavior Based Customer Insight Regulatory & Compliance AnalyticsMulti-Channel Fraud Analytics
IBM Analytics Platform:
DB2 Analytics AcceleratorQMF 11.2.1Spark on z/OS
DataWorksDataWorks ForgeData Science Experience (DSX)
InfoSphere Information ServerInformation Governance Catalog. . .
© 2016 IBM Corporation12
The key is open standards
XML-based industry standard
Defines statistical and data mining models to share across applications
Eliminates the need for custom code
Compatibility | Enterprise Grade | Simple Management
© 2016 IBM Corporation13
Big Data and Analytics HPC Cloud
IBM Blue Stack – primary BD&A
(Note: DB2 BLU focus remains for AIX & Linux)
ISV Stack– Data focus
ISV Stack– Application focus Cluster focus MSP focus
• BigInsights w/ IOP – IBM Data Engine for Hadoop & Spark
• BigInsights + Analytics - IBM Data Engine for Analytics
• Cognos/SPSS – IBM Solution for Analytics
•WebSphere
• Relational DBs;MariaDB, PostgreSQL, EnterpriseDB
• NoSQL DBs;MongoDB, Redis, Cassandra, Neo4J
• In-Memory DBs;Hana, DB2 Blu
• SAP applications with Hana + S4Hana
• Infor and PegaSystems with EnterpriseDB
• Magento, SugarCRM, WordPress with MariaDB
•NFV for Telco
• Elastic Storage Server
• Life Sciences / Genomics
• Research, Oil and Gas, Seismic, CAE
•Climate Modeling, Weather Prediction
• EasyScale
• Hybrid Cloud opportunities (e.g. SoftLayer, ScaleMatrix, etc)
13
vRealize
Open Source Products your way..
© 2016 IBM Corporation14
Linux on z Systems
Power
Spark
DB2
Spark
Spark Spark
x86
Spark Spark
Leverage non-z data
Leverage Linux on z virtualization benefits
Leverage z/OS data and transactions
CICS WAS
DB2 VSAM
z/OS
IMS
IMS
Spark
Leverage all your data without moving itApache Spark – A unified analytics platform
Spark
Spark
Spark
© 2016 IBM Corporation17
Topics and Questions to be addressed
What is Cognitive Analytics ??
What does IBM has to offer?
What are the key Use Cases and Integration Scenarios ?
Summary and Takeaway
© 2016 IBM Corporation18
Behavior Based Customer Insight (BBCI) for BankingCustomer Details View in Branch Office Application
The view includes, for instance:− Customer current products − Products that could be offered − Levels related to the possibility for the customer to be in overdraft, to churn etc.− . . .
© 2016 IBM Corporation19
Behavior Based Customer Insight (BBCI) for BankingOverview
Predictive customer insight− Predictive analytics− Data models− Analytical models− Scoring components
Sentiment analytics− E-Mail tone analyzer
Rest APIs− Insight consumption
E-Mail Tone Analyzer
Cashflow Analysis
Product Propensity
Upsell Propensity
Churn Propensity Analysis
Behavior-based Segmentation
Financial Event Prediction
Peer Segmentation
Life State Prediction
CustomerProfile
TransactionData
AccountData
InteractionData
(structured)
InteractionData
(e-Mails)
CensusData
Data Sources
Stru
ctur
edN
on-S
truct
ured
Internal External
. . .
. . .
Business Use Cases
© 2016 IBM Corporation20
Behavior Based Customer Insight (BBCI) for BankingLeveraging IDAA for BBCI
DB2 for z/OS&
DB2 AnalyticsAccelerator
IBM z Systems
Application(s)
In-DBTransformation
BBCI
BBCI Rest API
BBCI DB
SQL Queries
Application Layer
Wrapper Service
SPSS
Ana
lytic
al M
odel
s
SPSS
Col
labo
rativ
e &
Dep
loym
ent S
ervi
ces
ETL
© 2016 IBM Corporation21
Target Solution ArchitectureUse Case: Web and Mobile Bank Application to increase User Experience
DB2 for z/OS&
DB2 AnalyticsAccelerator
IOP & BigInsightsHDFS / (GPFS)
IBM z Systems
Big SQL
Hive / HCatalog
Application(s) SQL
Split_Query_1Split_Query_2
Application Layer
SOAP EnvelopsMetaInformation
(Apache Flume)
BigIntegrate
Spark / R
IBM StreamsAnalytical model
Scoring deployment
Scala / Python / Java / R / SQL
Big
Inte
grat
e(o
ptio
nal)
Agg
rega
tion
and
trans
form
atio
nof
new
with
his
toric
al d
ata
Spark Streaming
© 2016 IBM Corporation23
Analytics and Machine Learning
Watson – Cognitive Solutions
Interactions with varied Data stores
Legacy Integration
Open source Analytics Integration
Parallelization and GPUs
IO Bandwidth
Data Compression
Memory and Cache Sizes
Storage – SAS / SSD
© 2016 IBM Corporation24
General Purpose CPU - Multicore GPU – Thousands of
Cores
GPUs are well suited for parallel processing tasks. They have thousands of core that can work in parallel.
Significant Analytics Acceleration can be achieved with concurrent execution of Analytics workloads.
Common Programming Languages for offloading.
Parallelization and GPUsParallelization and GPUs
Processor Caching
Data and function calls are placed in the Caches.
Effiency and Latency improvement, when data addresses are kept in caches.
Good Cache hierarchy improves overall Performance.
© 2016 IBM Corporation25
IBM FlashSystem
IBM Flashsystems are optimized for high volumes of unstructured data for Analytics.
Supplement your existing Analytic’s infrastructure.
Decrease overall response times.
Increase efficiency/utilization across the IT stack.
Completely eliminate storage performance issues
Resilient Memory Bandwidth
SMT – Thread Per Core
Cache Latency
Virtualization and On-Demand creation of Clusters
Systems Hardware
Storage Configuration
© 2016 IBM Corporation26
CAPI Attached Flash Optimization
Issues Read/Write Commands from applications to eliminate 97% of instruction path length CAPI Flash controller Operates in User Space
Pin buffers, Translate, Map DMA, Start I/O
Application
LVM
Disk & Adapter DD
Read/WriteSyscall
strategy() iodone()
FileSystemstrategy() iodone()
Interrupt, unmap, unpin,Iodone scheduling
< 500 Instructions
ApplicationPosix AsyncI/O Style API
User LibraryShared Memory Work Queue
aio_read()aio_write()20K Instructions
Attach flash memory to POWER8 via CAPI coherent Attach
CAPI ( Coherence Accelerator Processor Interface)
© 2016 IBM Corporation27
Increased parallelism to enable analytics processing
A3 B3 C3
A2 B2 C2
ScalarSINGLE INSTRUCTION, SINGLE DATA
SIMDSINGLE INSTRUCTION, MULTIPLE DATA
Instruction is performed for every data element
Perform instructions on every element at once
Sum and Store
C1
C2
C3
A1 B1
A2 B2
A3 B3
INSTRUCTION
A1 B1 C1
Sum and Store
ValueEnable new applicationsOffload CPUSimplify coding
Smaller amount of code helps improve execution efficiencyProcess elements in parallel enabling more iterationsSupports analytics, compression, cryptography, video/imaging processing
SIMD (Single Instruction Multiple Data) processing
© 2016 IBM Corporation29
Close the gap
Integrated Hardware | Data | Analytics Software | Business Process
IBM Systems and Storage
© 2016 IBM Corporation30
Summary and Takeaway
Integration of various offerings is key to enable Cognitive Business− IOP and BigInsights− Big SQL− Spark Integration
IBM Systems contributes to Cognitive Business by making z/OS and other data stores easily accessible and consumable for Cognitive Analytics tasks− DB2 Analytics Accelerator− DataWorks with Data Science Experience (DSX) − Spark on z/OS
Industry specific opportunities for z Analytics to enable Cognitive Business, e.g.− FinTech