WEX Overview (Part II) - IBM...WEX Overview (Part II) Eric Smith – Watson Explorer Solution...

WEX Overview(Part II)Eric Smith – Watson Explorer Solution Architect

Topics

• New Insights from Unstructured Data

• Tailoring WEX to your Environment

• Connecting WEX to other Analytic Tools

2

What is Unstructured Data?

3

News Articles Email Social Media

4

WEX

5

6

Watson Knowledge Studio• Cloud based, machine learning

solution for developing new domain knowledge for Watson tools

• Information stored in knowledge

graphs

• Runtime environment for fine tuning

and refining annotations

• Leverage in Watson Explorer or

Alchemy Language applications

IBM Confidential

Watson Knowledge Studio Clip

8

NHTSA Demo with WKS Annotator

9

Ontolection Trainer

• Provides a true machine learning

capability for creating an ontology

• ML algorithms are executed against a text corpus of data

• Output is leveraged within a

search collection to enable query expansion.

• Enhances Natural Language

Querying

10

Tailoring WEX to Your Environment

11

Analytic Components

Content Miner

Content Analytics

Admin Console

Analytics Infrastructure

Control Monitor

Configuration

Security Scheduler Logging

Websphere(Embedded or Enterprise)

360

Admin

360 Info

App

Foundational Infrastructure

Control Monitor ConfigurationSecurity Scheduler Logging

Crawlers

Content

Preparation(Text Analytics)

1

Indexer

2

Advanced

Analytics

& Search(runtime)

CrawlersConversion

Pipeline

1

Indexer

2

Search(runtime)

3

360 Info App User

Business Analysts &

Data Scientists

Data Sources

Foundational Components

4

4

3

UIMA

Domain Expert

Content

Analytics

Studio

Integrations with

REST

API

Annotator Admin

Console for

Foundational

Components

IBM Master Data

Mgmt

IBM

Counter

Fraud

BigInsights

BI Reporting

IBM Product

Integrations

Websphere(Embedded or Enterprise)

WEX Advanced Edition

WEX Functional Architecture

On- Premise

Connector

Admin UI

Document Text

ExtractionIndexing

Application

Builder

On Premise

Annotation

Watson CloudWatson

Services

WEX Conversion Pipeline

Watson

Services

13

14

Development EnvironmentLink to System Requirements: http://www-

01.ibm.com/support/docview.wss?uid=swg27045727

DE

V U

sers

Number of Server(s) CPUs Cores For Memory (GB) Storage

Each Server Each Server Each server

WEX FC Development

-18 64 3 TB

• Up to 3 to 6 TB of data

• No High-availability -failover

• RHEL Linux• On-Premise

• Application Builder

• WAS Liberty Profile

• Result aggregation

• Display rendering

• *Can be a VM

WEX AC

(Content

Analytics)

Developme

nt Server

WEX FC

Developme

nt Server

• NLP Annotation• Content Mining• *Can be a VM

64-bit x86, IBM POWER7, IBM POWER8, or IBM Z System

64-bit (AMD64 or Intel 64) x86 system

Normal flow (Primary)

Data replication (DI)

Fail-over flow

Normal flow (Secondary)

Annotator Flow

Normal flow (Primary)

Data replication

Load B

ala

ncer

• 15TB of data (10% structured)

• Projected index size:• Structured (1.5TB)• Unstructured (2 TB)

• High-availability - failover

• 8 Queries/second

• Indexing• Query service

Engine

Layer

• Crawling• Connectors

• Indexing failover

Crawl/Index Layer

• Clustering• Federated Search

• Result aggregation• Display rendering• App Builder

Application

Layer

Type of Server CPUs Cores For Memory (GB) Storage

Each Server Each Server Each server

Application 8 32 200 GB

Engine 16 64 2 TB

Crawler 16 32 1TB

16

WEX EE Production Environment – 3 Tier Architecture

HW

Load B

ala

ncer

Number of Server(s) CPUs Cores For Memory (GB) Storage

Each Server Each Server Each server

Application -6

Engine- 6

16

32

128

128

500 GB

3 TB

Data – 6 32 64 3 TB

Normal flow (Primary)

Data replication (DI)

Fail-over flow

Normal flow (Secondary)

• Up to 14 TB of data

• High-availability -failover

• 7 Queries/second• RHEL Linux

• Indexing• Query

Routing• Search

Results

Engine

Layer

• Crawling and Indexing

• Data Refreshing

Data Layer

• User Interface• Integration

Layer

Application

Layer

64-bit (AMD64 or Intel 64) x86

system

64-bit (AMD64 or Intel 64) x86 system

64-bit (AMD64 or Intel 64) x86 system (Can be

VMs)

Questions to Ask

• What is the use case?

– Search, Analytics, Both?

• How much data?

• What kind of data?

• Data Growth?

• Usage?

• Interface Options?

17

Connecting WEX to Other Analytic Tools

18

Streams

Product Integrations

19

MDM* InfoSphere

BigInsights*StreamsFileNet P8

WebSphere Portal

I2 Analyst Notebook

Cognos

SPSS

© 2017 International Business Machines Corporation 20

Natural Language Understanding – Augmented Indexes

Extract metadata automatically to improve exploration without building any annotators!

© 2017 International Business Machines Corporation 21

Watson Discovery Service – Runtime Integration

Bring curated news and blog content that has been augmented by Alchemy in context into a Watson Explorer application

Questions?

22