Welcome to the Learning Objekt Operational Data Provisioning&Extraction.After the explanations of the entire ODP Architecture and typical scenarios forOperational Analytics this unit will focus on ODP based extraction scenariosintroducing new capabilities in SAP Business Objects Data Services.

1

The Objectives of this unit are to show a lot of details and specific benefitsusing Data Services ERP Extractors based on ODPs.You will learn how:

– Extractor Support of Data Services works in general

– Business Suite Extractors are presented from the Data Services ETL Perspective

– Typical Dataflow Examples with Delta Support look alike

Therefore the Agenda looks like it is shown here:

3

With Extractors, BW customers can use one tool to define all extraction, validation, andcleansing rules to load SAP and non-SAP data into BW. Non-BW customers can getnative access to the Business Suite without going through BW.

What are Extractors?Basically nothing else than ABAP Function Modules the ERP team has to writeWhenever BW needs access to data, a new Extractor is written by SAP (or thecustomer)Provide some user exits, but in general are considered not modifiableInclude the majority of the transformation logic for BW – many BW data flows are asimple copy of Extractor to BW objectMany sources have multiple Extractors: some obsolete, some used for different needs;for example, one returning the data aggregated, one for the detailed transactionsAs the ERP team is responsible for them, they know how to get delta or implementedways in the ERP system to log changesExtractors usually do not have much metadata, no foreign key relationships, primarykeys not marked as such – BW and help.sap.com are good sources for information

For us all of that does not matter – Typically a customer will ask “I want the data forxyz, BW gets it from the Extractor called abc”

4

Data Services has many ways to connect to the SAP Business Suiteapplications (ERP).

• Read tables via ABAP - what you do in an ABAP dataflow: an ABAP programis executed on the ERP server, which generates a data file that is transferred tothe Data Services server (ftp, shared directory) and read by the Data Servicesengine for further processing. Today, it is the recommended approach to readlarge volumes of data from SAP via Data Services.

• RFC_Read_Table - used when you use a table in a regular data flow.

-RFC BAPI calls are like stored procedures in databases. They can also beused to write data back to SAP.

• IDOCs - mostly used for real-time messages, but are also used in ourmigration offering to load data back to SAP.

And finally:

• Extractors - These were not supported in Data Services versions prior to 4.0;they were used by BW only, but provide a lot of benefits as you will see in thissession.Exactly this extractor technology will be opened up through the new ODP APIthat will make this technology available for external use in ETL processes.

Based on the new ODP API SAP Datastores in Busineaa Objects DataServices have a new Extractors category that you can browse, import by name,and drag onto a dataflow. The first call returns all data; subsequent calls giveyou the delta, or you can specify a datetime.

The ODP API does not show all Extractors, it only shows the releasedones. The idea is that over the time multiple Extractors have beendeveloped by SAP, some became obsolete, some might not work withthis API. So along with the ODP API a new table is created in thedictionary called ROOSATTR containing all the Extractors the API andhence DataServices 4.0 supports. Initially these will be a very limited listfocusing only on the most important Extractors and will grow over thetime. And for sure it will not include customer written Extractors.So there is a SAP program - transaction code unknown as of today -named RODPS_OS_EXPOSE with which customer written Extractorscan be added to above table. More information on the availability ofstandard extractors are found in SAP Note 1558737.

The graphic shows the main features of the ODP API implementation in DataServices.• In the ERP system all extractors that are available for external use are describedin the table ROOSATTR.• So called Datastores in Data Services describe source system connections suchas databases, applications, SAP BW Systems (OpenHub API) and most recentlySAP Business Suite systems (ODP API). After connecting an SAP ERP system asa new Datastore in Data Services a node Extractors will appear in the Datastoretree that allows to browse into the list of published extractors in the ROOSATTRtable of the particular ERP system.At the current moment only single ODP extractors are supported for from the ODPAPI. In future complex ODPs will be supported as well.• Data Services also supports Metadata Browsing for the SAP ERP world offeringso called External Metadata for extractors from SAP ERP source systems. In thistree meta information from all supported Application Components.

7

ODP Extractors can be used in regular data flows and ABAP data flows, butregular data flows are preferred because they enable data streaming. The usercan set the Initialize flag to Yes to return all data, or No to return only thechanges since the last time.

Using the Extractors is (almost) no difference to using tables. Both arequeried, return data and then used in the ETL context as usually. Thefirst major difference is, other to SAP table reading, it is actuallyrecommended to add the Extractor to a regular dataflow. Remember,when a SAP table is added to a regular dataflow we read it via theRFC_READ_TABLE function module which returns all the data in onefunction call. Good for a few rows, certainly not the way to read millionsof rows. Extractors are different, the API is more like a SQL dataset. TheExtractor is Prepared and Executed, then you fetch one array of recordsafter the other putting the fetched data to the downstream transformimmediately.

So apart from adding the Extractor into a dataflow, there are just twomore settings to make for an initial load, the initialization flag should beset to yes and the Extractor date-time to a global start date

Extractors give you the delta, but still as a Designer you need to implementhow to use the delta information.

The delta behavior of the particular extractor has to be considered in the detailsof the transformation and load part of the data flow, as delta logic can beunderstand different in separate application contexts.

The initialization flag is to tell the Extractor we want to read all its data, notjust the delta. It is kind of the same as the truncate table flag used in thetarget table for an initial load. As in the first run the target table is emptyanyhow, truncating (emptying) the table is not required. But what if theinitial load should be re-executed? The dataflow makes sure the data istruncated first. Extractor behave in the same way. If the read neverhappened, the setting of the initialization flag is of no importance. But if were-run the initial load and the init flag would be set to ‘No’, we would get thedelta only – what would be not acceptable.

Does that mean you need to have two data flows, one with the initial flagset to yes and one with no for the delta?No, you can use a global variable here as well. However in most cases youwill have two data flows anyhow, the initial load will bulk load the data asquickly as possible, the delta load with all its logic to deal with changes,deletes etc.The Extractor date-time is used to associate an Extractor run with a date.Although this is not needed for the Initial load, this date will help us buildingdelta loads later.

10

You should now be able to• Explain Extractor Support of Data Services in general

• Understand the role of Business Suite Extractors seen from the Data ServicesETL Perspective

• Elaborate Typical Dataflow Examples with Delta Support

You can find further information on Additional ODP-specific Administration Tasks inthe SAP Library for SAP NetWeaver 7.0 including Enhancement Package 3 underhttp://help.sap.com.

12

13