Early Delta Init

1

mySAP BISAP BW 3.0

Openess and Connectivity

Rainer UhleBW Product Management

SAP AG

SAP AG 2002, SAP Skills 2002 I1_Uhle 2

Topics

BW Extraction from R/3

SAP BW & Ascential DataStage ETL

BW Connectivity - Openess

BW Dealing with XML Formats

BW DataBase Connect

BW Open Hub Service

2


The Organizational Challenge

e.g.at business unit level provide a:� local/ subsidary view

� regional view

� global view

Provide each organizational unit or better each role with the needed reliable, consolidated, integrated, actual and historical information

at headquarter levelacross business unitsprovide � regional view� global view


Business Roles and Related Content are Key

BW

3


Open Data Warehouse Architecture


Open Data Warehousing with SAP BW

� Transparency and manageability of all loading processes

� Flexible set of ETL capabilies

� Open to ETL-tools

� Close integration of Ascential DataStage

� Application level ETL from SAP systems

� XML feeds (open standards) and flat files

� DB connect for legacy data

4


BW Architecture: Layers & Accessiblity

Data Warehouse� Non volatile�Granular� Historical foundation� Integrated� Built with ODS

Objects

Multidimensional Model

�Multidimensional analysis

� Aggregated view

� Integrated

� Built with InfoCubes

Operational Data Store�Operational Reporting � Near Real-Time / Volatile�Granular � Built with ODS Objects


Accessiblity

XML/A� XML for Analysis� Based on HTTP/SOAP� Any platform

ODBO�OLE DB for OLAP � Based on COM�Windows platform only

BAPI�OLAP BAPIs� Based on RFC� Access library on any SAP

platform

5


� Controlled distribution of consistent data

� Target: file or DB table

� Central monitoring

� Select filter criteria and columns

� Scheduling

� Full or delta mode

InfoCubes

ODS Objects

BW Server

ExternalDataMart

. . .

. . .

RelationalTable

FlatFile

Open Hub Service - Details

Master Data


Topics





BW DataBase Connect

BW Open Hub Service

6


Types of Extractors

Application Specific Extractors

BW Extractors

BW ContentDataSources

Generic Extractor

Database View

SAP Query

FI CO

HR

GeneratedDataSources

CO-PA

LIS

FI-SLLO Cockpit

... ...

BW ContentDataSources

GeneratedDataSources

Creation of own DataSources� Possibility to generate own

DataSources in Application

� Generic extractor tool for customer specific tables or applications

Creation of own DataSources� Possibility to generate own

DataSources in Application

� Generic extractor tool for customer specific tables or applications

Business Content DataSources� Build in Extractors

� High coverage (Applications, Industries)

� Transaction - and Master Data

� Ready to run

� Reduce implementation efforts and costs

Business Content DataSources� Build in Extractors

� High coverage (Applications, Industries)

� Transaction - and Master Data

� Ready to run

� Reduce implementation efforts and costs


Logistic Extraction for BW – Delta Mode

R/3 OLTP-System

DocumentData

∆-MANAGEMENT-------------------------------------------------------

Delta-Extraction

Process 2Store Session Data (Document + Context)for BW needs

Process 1Store Document in DB

Dialoguesession

Update Queue

Save Document

Backgroundprocessing

Delta Queue

7


Logistic Extraction – Batch Set Up Scenario

R/3 OLTP-System

DocumentData

LISCommunication

Structure

Setup-Tables

Extract Structures

MC11VA0HDR

MC11VA0ITM

MC11VA0SCL

MC11V_0ITM

MC11V_0SCL

MC11V_0ITM_SETUP

MC11V_0SCL_SETUP

.....


Extracting Transactional Data

R/3 OLTP-System Business Information Warehouse

Data-/InfoSource

∆-MANAGEMENT----------------------------------------------------

Delta-Extraction

Cubes

ODS/PSA

Full Upload withDelta Initialization

Delta Upload

Setup-Tables

8


Service API 3.0B – Zero Downtime Service

Zero-downtime during initial load / Early-delta initialization

In logistics, you can achieve zero-downtime during the initial load: data can already be written to the delta queue at the same time as data is transferred from the setup tables. A flag in the scheduler (as of 3.0A SP7) controls this feature.

Zero-downtime prior to and during initial load using a mirror source system

The Service API zero downtime service allows you to fill the setup tables in a mirror system, while posting continues in the original source system and and fills the Delta queue. After the setup tables have been filled and the initial load from the mirror system into the BW has been completed, the status information is synchronized with the original source system. Deltatransfer takes place from the original source system now into BW.


Topics





BW DataBase Connect

BW Open Hub Service

9


Scenario for Integrating XML Data Sources

‘Push’ of document-type data in delta queue of corresponding DataSource within the BW server

External application

CRM

document-typedata

document-typedata BW

DataSource

BW

Delta queue

FileDataSource

InfoSource

mass data

File system

Delta InitFull

pushXML data

FTP (non XML format)

SOAPrfc

Service

SOAPrfc

Servicepull


SOAP Body for HTTP Post Request

Example for HTTP/SOAP body (DataSource ‘Test’)

<?xml version="1.0" ?><SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/"><SOAP:Body><rfc:_-BIC_-QITEST_RFC xmlns:rfc="urn:sap-com:document:sap:rfc:functions"><DATASOURCE>TEST</DATASOURCE><DATA>

<item><VENDOR>Smith</VENDOR><MATERIAL>soap</MATERIAL><DATUM>20010815</DATUM><UNIT>KG</UNIT><AMOUNT>1.25</AMOUNT>

</item><item>

<VENDOR>Fritz</VENDOR><MATERIAL>body lotion</MATERIAL><DATUM>20010816</DATUM><UNIT>ml</UNIT><AMOUNT>150</AMOUNT>

</item></DATA></rfc:_-BIC_-QITEST_RFC></SOAP:Body></SOAP:Envelope>

10


Data Flow: Generic Staging

MarketplacesMarketplaces

Java servicesJava services

wwwwww

Commercial layer, xCBL

XML parsing

ODS, InfoCubes,Master Data

PSATransformation

(XSL)<Country>DE</Country>

<City>Hamburg</City>

CountryDE

CityHamburg

Delta queue

SOAP compliantHTTP Service

BW Server

Ext. Application

Not available so far


File DataSource as Entry Point for SOAP Processing

Create BW-DataSourcewith SOAP-Interface

Field names given fromXML data format have to be added manually

11


Pushing Data into the Queue via HTTP

<?xml version="1.0" ?><SOAP:Envelope xmlns:SOAP="http://schemas.xmlsoap.org/soap/envelope/"><SOAP:Body><rfc:_-BIC_-QIURGETXML_RFC xmlns:rfc="urn:sap-com:document:sap:rfc:functions"><DATASOURCE>URGETXML</DATASOURCE><DATA>

<item><VENDOR>Otto</VENDOR><MATERIAL>Seife</MATERIAL><DATUM>20010104</DATUM><UNIT>KG</UNIT><AMOUNT>1234567</AMOUNT>

</item><item>

<VENDOR>Fritz</VENDOR><MATERIAL>Rasierwasser</MATERIAL><DATUM>20010213</DATUM><UNIT>ml</UNIT><AMOUNT>100</AMOUNT>

</item></DATA></rfc:_-BIC_-QIURGETXML_RFC></SOAP:Body></SOAP:Envelope>


DeltaQueue Entries after http Request

12


Topics





BW DataBase Connect

BW Open Hub Service


DB Connect (BW3.0B) - Scenario

Application

ORACLEDataBase

Purchasing

Unix

SAP BW

MS SQLDataBase

NTDB-Connect

A purchasing application runs on a legacy system based on an ORACLE database. To analyze the data from the purchasing application, the data needs to be loaded into the BW System (possibly installed on a different database, e.g. MS SQL-Server). DB-Connect can be used to connect the DBMS of of the purchasing application and extractdata from the tables or views.

13


DBConnect as part of the Open Data Warehouse Architecture


DB Connect - Architecture

DatabaseCatalog

DatabaseCatalog

ExternalDatabase Server

StagingEngine

DataSource

InfoSource

List of Tables + Views

Field Details

Data

Direct access to external RDBMS

Read metadata and data

Access to DBMSs supported by mySAP Technology

SAP BW

14


DB Connect Architecture

DB-Client forMSSQL

SAP DBSL forMSSQL

SAP DBMultiConnect

SAP DB-Interface

MSSQLDatabase Management System

SAP BW 3.0Application Server

SA

P B

asis

6.1

0 / 6

.20

BW

DB-Client forORA

SAP DBSL forORA

OracleDatabase Management System

DB ConnectImplementation


DB Connect – Supported Source DBMS

DB Connect enables data to be extracted from the following databasesystems supported by mySAP technology

� DB2 UDB

� DB2/400

� DB6

� Informix

� MS SQL Server 7.0 & MS SQL Server 2000

� Oracle 8.1.7.3+

� SAP DB

15


DB Connect - Prerequisites

� SAP BW 3.0B

� Table names and field names have to be in capital letters

� Usage of views on top of tables is recommended

� The SAP specific part of the database interface, the DBSL (database shared library) for the source DBMS, has to be installedon the application server. It can be downloaded from the SAP Service Marketplace. It is licensed and delivered to customers solely for use as part of the SAP BW.

� In some cases you need a Windows based AppServer, e.g. for source MS SQL

� The database specific DB-Client for the source DBMS has to be installed on the application server. It has to be licensed by the customer from the DB vendor and is delivered as part of the database access license .


Topics





BW DataBase Connect

BW Open Hub Service

16


Open interface to thestaging engine

� For 3rd party extraction tools and

� For customers (for own extraction programs)

� Exchange meta data

� Import of transcationdata

� Import of master data

Complementary Software Partners (CSP)


SAP - Ascential Partnership

DataStage is SAP’s preferred data integration technology.

SAP signed Reseller Agreement with Ascential

SAP resells DataStage and DataStage Load PACK for SAP BW.

SAP and Ascential working together in Sales & Marketing, Engineering & Support

Ascential Engineer at SAP in Germany

….

17


BW Load Pack

XML Pack

Appl. Spec.Extr.Pack

FileInterfaceODBC

BW Load Pack

PlugIns

XML Pack

FileInterfaceODBC

DataStage ETL Components


E-Business Applications

^̂

Legacy Systems

VSAMRDBMS(DB2)

ADABAS

R/2

File

Non SAP Non SAP Business

API

SAPBW

Service

API

Dev

elop

men

t

Test

ing

Pro

duct

ive

BW Service API for SAPBW Service API for SAP

S I E B E LS I E B E L



SAP‘s Offering

PeopleSoft

Oracle


SAP BW Load PACK Capabilities

DataStage Load PACK for SAP BW

� Meta Data Browser allows you to browse and select SAP BW Source System, InfoSource and Transfer Structure meta data

� RFC Server listens and responds to SAP BW request for data

� Uses standard SAP Business API’s

� Extracts meta data from SAP BW

� Validates meta data match

� Sequence, launch and monitor all job steps from SAP BW

� SAP Certified Solution

� National Language Support (multi-byte)

18


SAP BW Load PACK Release 3.0 – Enhancements

Enhanced Meta Data Integration - Browse, Select, Create & Change SAP BW Meta Data Objects from DataStage

� InfoCatalogs

� Source Systems, including generate Source System and assignmentof program ID for launching RFC servers

� InfoSources, including master and transaction data, and Source System assignment

� InfoObjects, including characteristics, hierarchies, key figures and units

� InfoPackages, including enhanced DataStage parameter & value selection from SAP BW, and invoking InfoPackage from DataStage

AvailableJune 2002

certified


BW Certification Levels for CSP Partners

BW 1.2 BW 3.0BW 2.0 BW 4.0

BAPI‘s BAPI‘sBAPI‘s BAPI‘s

Certification Level

1.2 2.0 4.0

StagingBAPI‘s

Staging&Metadata

BAPI‘s

Staging&Metadata&

.....BAPI‘s

no significantBAPI

development

19


BW

LoadP

ack^

RDBMS

DataStage Packs – General Requirements

SAP BW

BAPI

Extract P

ack‘s

RDBMS

Foreign Appl. Foreign Appl.

Operational Data

MetaData

Business Objects

Delta Capabilities

Business Content

Direct Access

Access viaApplication DataStage Job


Example for Application Pack‘s

Siebel PlugIn (Siebel Releases 6.3/2000; 7.0/2001)

� EIM capabilities are used to access the underlying datamodel via the Business Object level (220 Business Objects?)� Step 1: Set up a EIM configuration file� Step 2: Start EIM to slect the data needed� Step 3: ODBC Extraction of that EIM Table

� Direct A ccess capabilities give access the underlying datamodel directly viaDDIC information from Siebel (list of tablenames)� Browsing Tables, selecting Columns and Fields generates SQL Statements that can be

enhanced easily to views, joins, projections on the relevant subset of DB tables

� Delta Capabilities are givenlast date of jobrun can be passed to EIM as selection criteria for step 2 (last update ge last date of jobrun, last update field very konsistent and reliablewithin Siebel Applicatipons)

20


EIM Plug-in Operations

Siebel OLTP Tables

Siebel EIM Tables

EIM Configuration File

EIM Component

Operating system

DATABASE

1. Config file created.

2. Run EIM

3. Extract data


EIM Tab

21


EIM Configuration file

[Siebel Interface Manager]

TABLEOWNER = SIEBELPROCESS = ds_auto_process

[ds_auto_process]TYPE = ExportBATCH = 12TABLE = EIM_ORDER_ITEMUSE INDEX HINTS = FalseCLEAR INTERFACE TABLE = TrueEXPORT ALL ROWS = FalseEXPORT MATCHES = (LAST_UPD > '1970-01-01 00:00:00' and LAST_UPD <= '2001-

10-24 09:29:29')

EIM Configuration FileEIM Component


Direct Access to Siebel: Selecting a Table

22


Direct Access to Siebel: Selected Columns


Direct Access to Siebel: Delta Tab

23


Direct Access to Siebel: SQL Tab


Example for Application Pack‘s

PeopleSoft PlugIn (PS Releases 7.5; 8.0; 8.3)

� ‚Add Business View ‘ functionality accesses the underlying datamodelvia the Business Object level (using Panels and Panel Groups)

� ‚Add Table View‘ functionality accesses the underlying datamodel directly via DDIC information from PS (list of tablenames)

� Browsing Tables, selecting Columns and Fields generates SQL Statements that can be enhanced easily to views, joins, projections on the relevant subset of DB tables

� Delta Capabilities are givenautomatic search for ‚effective Date‘ fields within PS (Type: Timestamp)automatic creation of a delta-file with the date of the last jobrun

24


FI, MM,APO, . .runon mySAP.com

Local officesuse Excel forbudget planning

Purchasing runs

on a legacy sys.Siebel is in use for CRM

PeopleSoftis in use for HR

Ready für Success

Web applicationused for your B2B

procurement


Topics





BW DataBase Connect

BW Open Hub Service

25


Open Data Warehouse Architecture


Information Distribution - Scenarios

SAP BWSAP BW

SAP BW

Analytical Application Sales

Data MartRegionalData Mart

Legacy Application

Open Hub Service BW Open Hub ServiceBW Open Hub Service

BW as an Information HubBW as an Information Hub

•• Controlled Distribution Controlled Distribution of consolidated andof consolidated andintegrated data integrated data

26


ODS Objects

Open Hub Service - Details

z Controlled distribution of consistent data

z Target: file or DB table

z Central monitoring

z Select filter criteria and columns

z Scheduling

z Full or delta mode

(Data + Metadata)


Functional Details

� Asynchronous data transfer for mass data

� New meta object InfoSpoke describes

� Data sources: IC, ODS-Object, attributes, texts, hierarchies

� Full and Delta (Delta for InfoCubes and ODS objects )

� Logical Target system, e.g. ‚Data Mart US‘

� Distribution targets: File, BW table

� Format: CSV

� Selection criteria

� Selected columns

� Transformation via BAdI (optional)

� Scheduling

� Integrated in process chains (process type InfoSpoke)

� Monitoring by application log

27


BW

Extern

Model: Design-Time

Data Source

Metadata,Selectabilty,Data

Data Target

Metadata,Data

Transfomation

Process

Reference to Data SourceReference to TransformationReference to Data

VarianteScheduler

Periodicity,Starting condition

Selections for data source


BW

Extern

Model: Run-Time

Data Source

Data

Data Target

Data

Transformation

Request

Selections for data source

Datapackage(in) Datapackage(out)

Log/Monitor

Reference to Data SourceReference to TransformationReference to Data

Request Broker

28


Definition of InfoSpoke (1)

A

B

InfoSpoke Header

� Source

� Destination

� Description

Destination

� Destination Type

� Description

A

B



C

Field List

� Take over DataSource fields

C

29



D

Selection criteria

� Selections for all characteristics of Data Source

D


Open Hub Monitor - Overview

30


Open Hub Monitor – Single Request


Summary

� Openness to extract data from BW

� All BW data objects are supported for extraction

� Delta tracking for each target system

� Scheduling and central monitoring

SAP BW can serve as an enterprise information hub and feed data to downstream systems

31


Outlook

� Support XML as export format

� Enable validation of the extract, e.g. check records or generate audit/control report

� Push the data to a program (RFC-Server)

� Extract based on a query


Open Hub Pricing Issues

What the price list says about Open Hub:

„Open Hub has to be licensed whenever data is being extracted and transferred from mySAP BI into 3 rd party target systems. The communication between multiple SAP BW’s does not require Open Hub.“

Note: The term ‘Open Hub’ here has a broader scope, than the software functionality ‘Open Hub Service’

32


Warehouse Management with SAP BW

� DBMS independent (Oracle, MS, IBM, Informix)

� Scalable architecture

� Mass volume capable

� Consistent meta data (includingbusiness definitions)

� Extensive monitoring and error tracking

� Data Quality checks

� Multi-lingual Master Data

� Open Hub Services

� Openess to multiple sources


� © Copyright 2002 SAP AG. Alle Rechte vorbehalten.

� Weitergabe und Vervielfältigung dieser Publikation oder von Teilen daraus sind, zu welchem Zweck und in welcher Form auch immer, ohne die aus-drückliche schriftliche Genehmigung durch SAP AG nicht gestattet. In dieser Publikation enthaltene Informationen können ohne vorherige Ankün-digung geändert werden.

� Die von SAP AG oder deren Vertriebsfirmen angebotenen Softwareprodukte können Softwarekomponenten auch anderer Softwarehersteller enthalten.

� Microsoft®, WINDOWS®, NT®, EXCEL®, Word®, PowerPoint® und SQL Server® sind eingetragene Marken der Microsoft Corporation.

� IBM®, DB2®, DB2 Universal Database, OS/2®, Parallel Sysplex®, MVS/ESA, AIX®, S/390®, AS/400®, OS/390®, OS/400®, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere®, Netfinity®, Tivoli®, Informixund Informix® Dynamic ServerTM sind Marken der IBM Corporation in den USA und/oder anderen Ländern.

� ORACLE® ist eine eingetragene Marke der ORACLE Corporation.

� UNIX®, X/Open®, OSF/1® und Motif® sind eingetragene Marken der Open Group.

� Citrix®, das Citrix-Logo, ICA®, Program Neighborhood®, MetaFrame®, WinFrame®, VideoFrame®, MultiWin® und andere hier erwähnte Namen von Citrix-Produkten sind Marken von Citrix Systems, Inc.

� HTML, DHTML, XML, XHTML sind Marken oder eingetragene Marken des W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

� JAVA® ist eine eingetragene Marke der Sun Microsystems, Inc.

� JAVASCRIPT® ist eine eingetragene Marke der Sun Microsystems, Inc., verwendet unter der Lizenz der von Netscape entwickelten und implementierten Technologie.

� SAP, SAP Logo, R/2, RIVA, R/3, SAP ArchiveLink, SAP Business Workflow, WebFlow, SAP EarlyWatch, BAPI, SAPPHIRE, Management Cockpit, mySAP, mySAP.com und weitere im Text erwähnte SAP-Produkte und -Dienstleistungen sowie die entsprechenden Logos sind Marken oder eingetragene Marken der SAP AG in Deutschland und anderen Ländern weltweit. MarketSet und Enterprise Buyer sind gemeinsame Marken von SAP Markets und Commerce One. Alle anderen Namen von Produkten und Dienstleistungen sind Marken der jeweiligen Firmen.

Copyright 2002 SAP AG. Alle Rechte vorbehalten

Documents

Early Delta Init