© 2002 IBM Corporation IBM Israel Software Symposium 2003 accelerate your business DB2 Information Integrator Software Jaffa Sztejnbok IT Specialist, Information

© 2002 IBM Corporation

IBM Israel Software Symposium 2003

accelerate your business

DB2 InformationIntegrator Software

Jaffa SztejnbokIT Specialist, Information ManagementGlobal Technology [email protected]


IBM Israel Software Symposium 2003 2

Agenda

What is Enterprise Information Integration

Without Information Integrator

Data Challenges

Complementary Information Integration Approaches

IBM DB2 Information Integration Products and Value

IBM’s Information Integrator 8.1

Demo



What is Enterprise Information Integration ?

Provides access to diverse, distributed, and real-time data as if it were a single source, no matter where it resides.

Helps businesses : –Shorten application development time

–Improve productivity and application efficiency

–Leverage existing data assets for the benefit of the business



Without Data Federation

DB2

Oracle

SQLServer

Excelspreadsheet

XMLdocument

InternetSSL

clientapplications

Browsers

Data managementlayer

Applicationlayer

Webservers



New composite applications need data from multiple sourcesƒ Consumers expect holistic, personalized, and value-added contentƒ Relational, XML, packaged applications, content repositories, file

systems all contain critical business information

Increasing emphasis on current dataƒ Real-time analyticsƒ Business activity monitoring

Petabytes will be the measure of available online data ƒ All client interactions are important ( e.g. instant messages, audio

records, Web traffic,…)

ƒ Internet and intranet content

Variety, Velocity, and Volume

Data Challenges



Complementary Information Integration Approaches Consolidate data for local access

– Data warehouses

– Operational data stores

– Production applications

– Creating additional reference copies

– Typically managed by ETL (Extract, Transform, Load) or replication technologies

Integrated access to distributed sources, Distributed Access

– Real time data, e.g., stock quotes ,Extending a data warehouse with real-time data

– Data changes rapidly

– Wide heterogeneity in data to be accessed ,relational and non-relational format

– Data which is not practical or possible to copy and when movement of data is small



BiologicalBiologicalData and Data and

AlgorithmsAlgorithms

relational databases

data warehouses

DataSources

Extended Search

Sources

spreadsheets

contentrepositories

…@office e-mail reports fax

ContentSources

IBM DB2 Information Integration Products

DB2 Information Integrator

SQL programming model Leverage SQL skills and

tools Federated data server

and replication server

DB2 Information Integrator for Content Content programming model Leverage CM skills and tools Federated data server, text

mining, and workflow engine



DB2 Information Integrator 8.1A Federated Data Server – Query distributed data as if it where a single source

Define integrated view across diverse and distributed data– Wide range of data and content sources

– Extensible to virtually any data source

Query as if a single source– Use standard SQL query and SQL expressions

– Include text semantics in the search

– Surface specialized functions into SQL

– Leverage query optimization and caching

Compose XML documents– Combine diverse sources

– Validate against DTDs or schema

Publish results to a message queue– Familiar DB programming model

Single source, relational updates

Integrated SQLView

DB2, Oracle, SQL Server, Sybase, Teradata, OLE DB, ODBC, Excel, XML, message queues, Web services, flat files, document repositories, content repositories, LDAP directories, WWW,

email databases, and more.



DB2 Information Integrator 8.1

A Replication Server – Manage consolidation for performance and availability

Distribute data among relational databases

– DB2, Informix, Microsoft, Oracle, Sybase, Teradata

Support flexible topologies– Distribution: One to many

– Consolidation: Many to one

Match data movement modes to usage requirements– Table-at-a-time for warehouse loading during

batch window

– Transaction-consistent for online data Choose latency characteristics

– Scheduled, interval-based, continuous

Apply transformations in-line – Standard SQL expressions or stored procedure execution.

DB2

Microsoft Microsoft



Define integrated views across diverse and distributed data

ƒ IBM Content Manager portfolio and other content repositories e.g. FileNET, Lotus

databases, ODBC and JDBC compliant relational databases, and IBM Lotus

Extended Search sources (LDAP directories, WWW, email databases,…)

Search federated data

ƒ Search application uses the IBM Content

Manager API

Mine additional metadata from text

documents

ƒ Identify document language

ƒ Extract entities like names or technical terms

ƒ Categorize documents based on a taxonomy

ƒ Group documents based on related content

ƒ Create a document synopsis

Define workflows

DB2 Information Integrator for Content



DB2 Information Integrator Value Extend current investments

– Work within your existing infrastructure

– Consolidate data or access distributed data as if it were a single data source

– Combine existing data and content assets in new ways

– Use familiar SQL programming model and existing tools

– Build on a standards-based, strategic integration platform

Speed time to value for composite applications– Reduce hand-coding 40%-65%

– Reduce skill requirements

– Reduce development time by half

Control costs– Reduce payroll costs

– Reduce need to rip and replace

– Reduce need to manage redundant data



ApplicationDeveloper

Speeding Application Development

RDBMS

Non-relational data

Non-traditional data

Development effort to handle:

Unique interfaces for each data type

Joining data from varied sources

Aggregation and grouping

Correlating dataSpecial features:Set processingIn-built db

transformation functions

OptimisationAutomatic local

cachingData driven triggers

OtherSQL is on OPEN StandardSQL is easily testable,

independent of the application

JDBC, XML, WebServices

II handles:Interfaces for each

data typeJoining data from

varied sourcesTransformationCorrelating data



Crystal Decisions

Vision As a world-leading information infrastructure company, Crystal Decisions helps businesses make better decisions by bringing together their people and their information.

Challenge Improve response time for complex queries over distributed

heterogeneous data sourcesSolution

Provides transparent, globally optimized access to heterogeneous, distributed data. Crystal Reports accesses the distributed data as if it were a single database. Response time improvement of up to 98% seen in house.

Business Value"Users of Crystal Reports and Crystal Enterprise, with DB2 Information Integrator, can … discover new ways to meet the information needs of their organization."

• Janet Wood, Vice President of Business Development, Crystal Decisions.Competitive Value

“DB2 Information Integrator provides Crystal Reports with exceptionally fast and efficient federated querying capability.“

• Trevor Smith, Program Manager, Business Development Group, Crystal Decisions



Without Data Federation

DB2

Oracle

SQLServer

Excelspreadsheet

XMLdocument

InternetSSL

clientapplications

Browsers

Data managementlayer

Applicationlayer

Webservers



Federated Access to Diverse Data



Information Integration

IBM DB2 Information Integrator Software

Data federation–Extensible read/write access across diverse data and content sources

–Database programming model (SQL)

–Content programming model (OO API)

Data placement –Caching and replication over heterogeneous information

Data transformation –SQL, XML, Web services

–Advanced search and mining

–Metadata management

Part of a complete integration solution –XML publishing, consumption, and interchange

–WebSphere business integration

–Open platform based on industry standards

Integrating diverse business information

across and beyond the enterprise



Functions

Data Federation Transparency: hides differences among sources

ƒ Appears to be one source

ƒ Supports a high level query language

ƒ Functional compensation and passthru

Heterogeneity: integrates data from diverse sourcesƒ Relational, XML, flat files, spreadsheet,

messages, content repositories, Web, …

High Functionƒ One query integrates data from multiple sources

ƒ Capabilities of sources as well

Extensibilityƒ Access wide range of data sources

ƒ Development wrapper toolkit

Autonomyƒ Non-disruptive to data sources, existing applications, systems.



Federation leverages a full database engineƒ Query Processor, Execution Engine, Catalog, Client Access, Security,

Transactions

Query processing extended for Federated Dataƒ Pushdown Analysisƒ Analyze how to decompose a user queryƒ Generate an optimal query execution plan using cost estimates including

data source knowledge: database statistics, indexes, source functions, server and network capacities

ƒ Allows function compensation

Performance , Optimization of distributed queries



Replication Architecture



Improve query performance and availability Administrator defines Materialized Query Table

ƒ Precomputed or frequently used valuesƒ Any data from the federated systemƒ Application indicates ability to use cacheƒ Implicit or explicit use

Developer enables cache useƒ If enabled, reads are handled from the

cache, writes passed through to the sourceƒ If not, reads and writes passed through to

sourceCache refresh managed:

ƒ Manuallyƒ By replication

Flexible caching topologies supported

Heterogeneous Caching Feature



Four important tasks:ƒ Data modeling

– Map data model to relational data model (tables with rows and columns)

– Map functions into SQL operationsƒ Query Planning

– Represent data source capabilities– Push down as much work to data source as sensible– Detect missing function at source (so engine can compensate)– Supply cost and cardinality information

ƒ Connection and transaction management ƒ Query Execution and data retrieval

– Execute parts of a user’s query for a specific data source

Wrappers



Configuration steps:

ƒ Wrapper: the wrapper code module itself

ƒ Server: a specific data source, with associated attributes

ƒ User mapping: information needed to connect to a specific

server

ƒ Nickname: a specific data set managed by a server,

mapped to rows and columns in the federated server

Defined to system via DDL commands

ƒ GUI administration generates DDL

Stored in the system catalog

Configuration



Administration Tools



SybaseOracleSQL ServerDB2InformixODBCTeradata

Wrappers which support discoveryHMMEREntrezXMLFlat FileExcelExtended Search

"Create Nicknames" window

Launches customized GUI

Returns Nickname defintions

Customized "Discover" GUI

Discovery for Nicknames



Definitionsƒ Manage control definitions

for replicationƒ Customize names and sizes

of objectsOperations

ƒ Start Capture, Apply, Monitor, Analyzer, and Trace

ƒ Issue commands such as STOP or STATUS

Monitoringƒ Perform static and dynamic

monitoring

Replication Administration



Application Development : Access DB2 catalogs and DB2 II federated sources

DB2 Development CenterWebsphere Studio

Microsoft Visual Studio .NET



Demo…

Query Result

ExcelSQL Server

DB2



For more information

http://db2ii.dfw.ibm.com/demo/

http://db2ii.dfw.ibm.com/demo/



Summary Information integration is a foundation for companies to build an On Demand Operating

Environment enabling them to align their IT infrastructure to business priorities

DB2 Information Integrator provides access to diverse, distributed, and real-time data as if it were a single source, no matter where it resides.

DB2 Information Integrator will help businesses – Shorten application development time

– Improve productivity and application efficiency

Rely on IBM’s proven technology and support for open standards



The whole is worth more then its constituent parts

DB2 Information Integrator

helps businesses to leverage existing data assets into knowledge

for the benefit of the business



Don’t forget to give us feedback

Presentation Code: A4

Documents

© 2002 IBM Corporation IBM Israel Software Symposium 2003 accelerate your business DB2 Information Integrator Software Jaffa Sztejnbok IT Specialist, Information