© 2006 IBM Corporation IBM Information Server Virtualizing Access to Information

© 2006 IBM Corporation

IBM Information Server

Virtualizing Access to Information

© 2006 IBM Corporation2


What is Data Virtualization?

Virtual Access using Federation

Federation as a Information Service

Summary


What is Data Virtualization?

Providing information in the form that applications and users need, while hiding the complexity of the underlying sources. Data virtualization allows information to be accessed through a common interface that centralizes the control data access.


Complexity and Cost

Heterogeneous, distributed data

Inconsistent islands of information underlie applications

Complications from M&A and departmental purchases

Complex & costly synchronization

Inconsistent and poor quality data

No feedback on quality of service

Impossible to support business transformation

CRMOrder Proc

SupplyChain

Procure-ment


But what if…

BI toolsBusinessAnalysis

MgmtReports

Accessing data anywhere in your enterprise

No matter where it resides

Regardless of what format it is in

Regardless of vendor

Without creating new databases and without disruptive changes to existing ones…

And all your data appeared to be in a single relational database


What is Federation?

Federation is an integration pattern that allows a collection of resources to be viewed and manipulated as if they were a single resource while retaining their autonomy and integrity. It is the technology on which EII is based.

7 © 2006 IBM Corporation

How does Federation compare to…

EAI

Application

Application

Application

InterpretTransform

Route

DataSource

… DataSource

extract

transform

load

Target /Data

Warehouse

ETL

Database

Database

Database

capture,apply

Replication


The IBM Solution: IBM Information ServerDelivering information you can trust

Understand

Cleanse Transform Deliver

Parallel ProcessingRich Connectivity to Applications, Data, and

Content

IBM Information Server

Discover, model, and govern information

structure and content

Standardize, merge,and correct information

Combine and restructure

information for new uses

Synchronize, virtualize and move information for in-

line delivery

Unified Deployment

Unified Metadata Management



Why Virtual Access


Making Federation into a Service

Summary and References


Federation

Transparent Appears to be one source Independent of how and where data is

stored Applications continue to work despite of

any change in how data is storedHeterogeneous Accesses data from diverse sources Relational, Structured, XML, messages,

Web, …Extensible Bring together almost any data source. Wrapper Development ToolkitHigh Function Full query support against all data Capabilities of sources as wellAutonomous Non-disruptive to data sources, existing

applications, systems.High Performance Optimization of distributed queries


VSAMSequential

IMS AdabasCA-DatacomCA-IDMS

Federated Sources

DB2InformixOracleSybase Teradata Microsoft SQL Server

ODBC

OLE DBExcelFlat filesLife sciencesCustom-built

DB2 CM FamilyDomino.doc DocumentumFileNetOpen TextStellentInterwovenHummingbird

WebSphereFileNet

Lotus NotesMicrosoft Index Server

IBM Lotus Extended Search

SametimeQuickPlaceMicrosoft Exchange

WebSphere BI Adaptors

SAPPeopleSoftSiebel

Plus partner tools and custom-built connectors extend access to more sources

Content& Imaging

Workflow systems

Relationaldatabases

WebOther

CollaborationSystems

XMLWeb services

Packagedapplications

Mainframefiles

Mainframedatabases

SQL

ContentSQLFederation Server

Classic Federation Server for z/OS

II Content Edition


Federation Basic Capabilities

Nicknames look just like tables to the application

Federated execution plans chosen by cost-based optimizer

Optimizer decides how to distribute query work between II and remote sources. Cost-based pushdown of operations.

Query fragments executed remotely sent via the native client library in the source’s own dialect

Support for remote SPs

Nicknam

e

Nicknam

e

Table

Cost-based optimizer

Rel. Wrapper

Client libraryNR. Wrapper

Client library

Local + Remote Execution Plans

Remote sources

WebSphere Federation Server

Non-SQL

SP

SP


500 rows of ‘1’

'Pushdown' of query operations

The federated server decides whether some or all parts of a query can be "pushed-down", i.e. processed at the remote data source(s). Pushdown-ability depends on

– availability of needed functionality at remote source with same semantics

– server options (example: is collating sequence at Federated server and remote source the same?)

Example: A remote source can handle an equality predicate, but not count(*).

Fed ServerPushdown

CompensateCOUNT(*)

Select count(*)from customers_NKWhere id=123

Select ‘1’from customersWhere id=123

500

Datasource


Local caching of remote data

Sending remote requests can be expensive. Why not use previously evaluated query result to answer new queries?

Applications remain intact

Materialized query table (MQT): local table defined by the result of a query

– Can include single table, joins, aggregations

– Can be indexed, replicated in partitioned environment

– Optimizer “routes to” them transparently as appropriate

– Can include local tables/views, relational and nonrelational nicknames

– Cache tables wizard

Remote

Join

Remote Local

Select

vs



Why Virtual Access


Federation as a Information Service

Summary and References


What if you could…

Package your information so it could be reused for many purposes by many users

Insulate your users from having to know how your information was produced

Make changes to how information created without breaking the applications that use them

Make it accessible to virtually all your users regardless of their location while maintaining control over who is accessing what data


… a service?

A repeatable business task – e.g., check

customer credit; open new account

What is Information Service?

… service oriented architecture (SOA)?

An IT architectural style that supports

service orientation

… Information Service?

Publishing consistent, reusable services that make it easier for applications, people and processes to access reliable information without knowing how or

where it is stored.


WebSphere Information Services Director

Packages information integration logic as services that insulate developers from underlying sources

Allows these services to be invoked as EJB, JMS, or Web services

Provides load balancing & fault tolerance for requests across multiple Information Servers

Provides foundation infrastructure for Information Services


WebSphere Information Services Director

Web Services EJB JMS

WISD Services

Shared Services

Logging service

Load balancer

Design service

Reporting service

Security service

Administration service

DesignRepository

Operational Repository

Repository

Metadata Services

WISD Server

DataStage QualityStageDB2 Federation

SOA Handlers

WISD Agent

DataStage QualityStageDB2 Federation

SOA Handlers

WISD Agent


Data Federation Pattern

Data Virtualization ThroughData Federation Server

Business Process

Activity 1 retrieve Activity n... ...

Portal

portlet

portlet

portlet

portlet

StructuredData Source

unstructured

MainframeData Source

...

Consumer

Provider

Application

Interface: SQL, XQuery

Data Federation – Traditional Context


Business Process

Activity 1 retrieve Activity n... ... Portal

portlet

portlet

portlet

portlet

Data Federation Pattern

Data Virtualization ThroughData Federation Server

Information Service Enablement

StructuredData Source

unstructured

MainframeData Source

...Consumer

Provider

Application

ESB

Interface: SCA (SDO), WSDL/SOAP, JMS/XML, RMI, …

Data Federation – SOA Context


Info

rmat

ion

S

ervi

ce D

irec

tor

Service-enabling information tasksAll the client knows is the operation they want to performs and how to invoke it

CurrentInventory

Request Arrives • Check authorization• Start monitoring & logging• Match service request to

information task to be done

Complex heterogeneous query

• Route to federation server

• Execute query against information sources

Request in

Query result

Response out

Response Returned

SOAP/http

Portal


Info

rmat

ion

S

ervi

ce D

irec

tor

Same Information Service – Multiple client bindings

CurrentInventory

Request in

Response out

Request Arrives

Complex heterogeneous query

Query result

Response Returned

SOAP/http EJB

Business ProcessPortal


Info

rmat

ion

S

ervi

ce D

irec

tor

Multiple Information Services

CurrentInventory

Request in

Response out

Request Arrives

Complex heterogeneous

query Query result

Return Response

SOAP/http EJBUpdateInventory

Current InventoryProcessing

ComplexDataStage Job Job result

Update InventoryProcessing

• User of Information Services shielded from complexity of the information systems

• Information system can evolve without impacting users

• Single control point for access to information systems

Business ProcessPortal


Value of Federation for Virtual Data Access

Speed time to market for new applications– Simplify and enrich portal development

– Reduce hand-coding by half

– Reduce skills requirements

– Use familiar SQL programming model and existing tools

– Build on a standards-based, strategic integration platform

Enhance value and insight from existing assets and applications– Work within your existing infrastructure

– Extend existing warehouses

– Combine existing data and content assets in new ways

– Facilitate cross-divisional reporting

Increase control over IT costs– Reduce need to rip and replace

– Reduce need to manage redundant data


The premier information management event for business and IT executives, managers, professionals, DBA's and developers.

Select from over 800 sessions: a 2 1/2 day business leadership track with 180 sessions and a 5 day technical track with 650 sessions.

Latest strategy and product announcements Large Expo Center Hands on labs One on ones with executives and specialists Birds of a Feather roundtables

IBM Information On Demand 2006 ibm.com/events/InformationOnDemand

IBM Information On Demand 2006IBM Information On Demand 2006 October 15-20, 2006 Anaheim, California

Participate in the PREMIER discussion on the future of Information Management

Learn how the transformation to Information as a Service will help you unlock business value and drive competitive advantage

Hear how your peers are realizing ROI

Understand the roadmap to long term strategic advantage

Learn best practices in your industry

Receive the best in technical education and free certification

Extensive opportunities for networking with both your peers and industry experts

Why attend:


Documents

© 2006 IBM Corporation IBM Information Server Virtualizing Access to Information