PBS Archive Data Extraction From ERP Guidelines-1

Transfer of Archived SAP R/3 and

SAP ERP Data to SAP NetWeaver BI

Using PBS archive add ons

29 August 2007

Transfer of Archived SAP R/3 and SAP ERP Data to SAP NetWeaver BI 2

© 2003 - 2009 PBS Software GmbH Schwanheimer Strasse 144a

64625 Bensheim, Germany

Phone: +49 - 6251 - 174 0

Fax: +49 - 6251 - 174 174

Email: [email protected]

Internet: www.pbs-software.com

PBS archive add ons® is a registered trademark of PBS Software GmbH, Bensheim.

SAP, SAP ERP, R/3, R/2, ABAP/4 are registered trademarks of SAP AG, Walldorf/Baden.

Acrobat Reader is a registered trademark of Adobe Systems Incorporated.


Table of Contents 1 Introduction ............................................................................... 4

2 Standard Extractors from BI Content ...................................... 5

3 Data Transfer through Flat Files .............................................. 9 3.1 Extract Archived Data from SAP R/3 and SAP ERP............................... 10 3.2 Transfer to SAP BW / SAP NetWeaver BI .............................................. 13 3.3 Import into SAP BW 3.X.......................................................................... 14 3.4 Import into SAP NetWeaver BI 7.x.......................................................... 18

4 Creation of a generic DataSource .......................................... 25

5 Module-specific Features........................................................ 28 5.1 PBS archive add on CFI.......................................................................... 28 5.2 PBS archive add on CCO........................................................................ 31 5.3 PBS archive add on CCOPA................................................................... 32


1 Introduction Data Warehouse systems enable efficient access to data from heterogeneous information sources, suitable storage and end user-specific representation of information that has been gathered in this way.

Data from almost any source can be used for data procurement and provision in the SAP BW system. Main sources are SAP systems that are implemented in the company. The data procurement layer (also called the ETL process layer) is used to extract, transfer and load this data into the BW system.

A growing number of customers need to extract archived data from an R/3 or ERP source system and load it into a BW system. This documentation describes two methods to fulfill this task.

In the first method extractors from BI Content are used. Chapter 2 lists all DataSources from SAP BI Content which are able to select archived data, together with DataSources from PBS.

The second method, described in chapter 3, is based on data transfer through flat files. It is a generic method which allows the user to transfer any structured data to SAP BW / SAP NetWeaver BI.


2 Standard Extractors from BI Content

The SAP BI Content provides a set of standardized DataSources which are widely used by customers. However, only a small percentage supports the extraction of archived data. In order to allow our customers the extraction of indexed data from archive, many PBS archive add ons now contain a selected number of archive-enabled DataSources and load reports.

Table 1 lists all DataSources from SAP BI Content which are able to select archived data. In addition, all DataSources that are provided by a specific PBS archive add on are added to the list.

In general, DataSources use two different methods to select archived data: either direct access to archive in the DataSource itself, or indirect access through selecting archived data via initial load. The second method is especially used in all DataSources from Logistics by using communication structures from the Logistics Information System (LIS).

Now, the second method will be described more in detail. In order to extract data from Logistics, all DataSources from BI Content use an indirect access method. First the data is stored in LIS tables as a buffer. Then the extractor reads from these LIS tables. To build initial data loads for SAP BW / SAP NetWeaver BI, first you need to populate the corresponding LIS tables with the related transaction. Hence, for archive access these transactions must be able to select archived data.

Table 2 lists all DataSources from Logistics with their corresponding transactions for an initial load. In addition, the transactions and related programs are listed in Table 3. In case of PBS versions, the supported SAP release is added to the program name together with a release date.

More module-specific details about using PBS extractors and load programs can be found in chapter 5.


Data Source Technical Name

Data Source Description Archive Access possible in

SAP BI Content

PBS archive add on

0CO_OM_OPA_6 Orders: Actual Costs with Delta Extraction

CCO (from 4.6C)

0CO_OM_CCA_9 Cost Centers: Actual Costs with Delta Extraction

CCO

0CO_OM_ABC_7 Processes: Actual Costs with Delta Extraction

CCO

0CO_OM_WBS_6 WBS Elements: Actual Costs with Delta Extraction

CCO

0FI_AP_3 Vendors: Line Items CFI

0FI_AP_4 Vendors: Line Items with Delta Extraction

CFI

0FI_AR_3 Customers: Line Items CFI

0FI_AR_4 Customers: Line Items with Delta Extraction

CFI

0FI_GL_4 General Ledger Line Items with Delta Extraction

CFI (from 4.6C)

0FI_TX_4 Taxes: Line Items with Delta Extraction

CFI

0UC_SALES_STATS_01 IS-U Sales Statistics CISU 0UC_SALES_STATS_02 IS-U Sales Statistics CISU Customer-specific DataSources

Profitability Analysis CCOPA

2LIS_02_xx Purchasing Documents MM 2LIS_03_BF Material movements CFI 2LIS_03_UM Revaluations CFI 2LIS_04_xx Production Orders CPP 2LIS_05_xx Quality Notifications CQM 2LIS_08_xx Shipments and

Shipment Costs CSD

2LIS_11_xx Sales Orders SD 2LIS_12_xx Delivery Notes SD 2LIS_13_xx Billing Documents SD 2LIS_17_xx Plant Maintenance

Orders/Notifications CPM

2LIS_18_xx Customer Service Orders/Notifications

CPM

2LIS_45_xx Agency Documents AB


Table 1: DataSources with archive selection from SAP BI Content and PBS archive add ons

Logistics DataSources

Initial Load Transaction

Description Archive Support

PBS Version

2LIS_02_xx OLI3BW MM: Purchase Y --- 2LIS_03_BF OLI1BW MM: Material movements N Y (CFI) 2LIS_03_UM OLIZBW MM: Invoice Verification N Y (CFI) 2LIS_04_xx OLI4BW PP: Production Orders N Y (CPP) 2LIS_04_xx OLIFBW PP: Rep. Manuf. N Y (CPP) 2LIS_05_xx OLIQBW QM: Notifications/Inspect. N Y (CQM) 2LIS_08_xx VTBW SD: Shipment N Y (CSD) 2LIS_08_xx VIFBW SD: Shipment Costs N Y (CSD) 2LIS_11_xx OLI7BW SD: Orders Y --- 2LIS_12_xx OLI8BW SD: Deliveries Y --- 2LIS_13_xx OLI9BW SD: Invoices Y --- 2LIS_17_xx OLIIBW PM: Maintenance Orders N Y (CPM) 2LIS_18_xx OLISBW CS: Customer Services N Y (CPM) 2LIS_45_xx OLIABW AB: Agency Business Y

Table 2: Initial load of extract data from Logistics with LIS communication structures


Transaction Description Program PBS version OLI3BW Purchase docs. RMCENEUA ---- OLI1BW Material

movements RMCBNEUA /PBS/RMCBNEUA (CFI)

OLIZBW Revaluations RMCBNERP /PBS/RMCBNERP (CFI)(no transaction) Material movemts

per material*) RMCBMMAT /PBS/RMCBMMAT (CFI)

from release 4.7 OLI4BW Production

Orders RMCFNEUA /PBS/RMCFNEUA (CPP)

(on demand) OLIFBW Rep. Manuf. RMCFNEUD /PBS/RMCFNEUD (CPP)

(on demand) OLIQBW Notifications /

Inspec. RMCQNEBW /PBS/RMCQNEBW

(on demand) (CQM)VTBW Shipments VTRBWVTBWNEW /PBS/VTRBWVTBWNEW

(on demand) (CSD) VIFBW Shipment Costs VTRBWVIFBW /PBS/VTRBWVIFBW

(on demand) (CSD) OLI7BW Orders RMCVNEUA ---- OLI8BW Deliveries RMCVNEUL ---- OLI9BW Invoices RMCVNEUF ---- OLIIBW Maintenance

orders RMCINEBW /PBS/RMCINEBW (CPM)

OLISBW Customer

Services RMCSNEBW /PBS/RMCSNEBW

(on demand) (CPM)OLIABW Agency Business RMCENEUB ----

Table 3: Transactions and program names for initial load

*) This function is only used for making corrections to initial stocks and should only be used in exceptional cases. For further details, see SAP Notes 79083 and 111691.


3 Data Transfer through Flat Files The following chapter describes a generic method which is based on data transfer through flat files.

These flat files contain data in AIS (Audit Information System) format. Since DART (Data Access Retention Tool) extractions use the same format, they can also be used as a data source.

The transfer of archived data is performed in three steps. In the first step the archived R/3 or ERP data is extracted in text format and written into flat files. Then all these files are transferred to the destination, whereas in the third step the data in the flat files is imported into the SAP BW system.

Up to now, extraction reports are available for the PBS archive add ons CAB, CCO, CFI, CMM, CPCA, CSD and CWM.

Note: Please keep in mind that the method described in this chapter is only a workaround solution. The BI Content extractors use extended business rules for extraction which cannot be realized by this method. As long as the respective extractor from BI Content is not able to read archived data (see chapter 2), extraction before archiving should be the preferred way.

As an alternative, SAP describes a method which is based on the information structures of the Archiving Information System (SAP AS). For detailed information refer to http://service.sap.com/bw SAP BW InfoIndex Archiving ‘Howto … Extract archived R/3 or ERP data via the Archive Information System’.

In the following pages the extraction of purchasing document data (archiving object MM_EKKO) will be used as an example. Direct access to the archived data is provided by the PBS archive add on CMM. The method described can be transferred to any other archiving object which is supported by a PBS archive add on.


3.1 Extract Archived Data from SAP R/3 and SAP ERP

Call CMM extract report /PBS/CMM_LIS_ITEM_EKKO (via SE38). From ECC 5.0 or higher, you can also call transaction /PBS/CMME_EXPORT. In order to extract only archived data, use ‘3’ for the data source (Archive only). Use ‘Item list’ as list layout, and use ‘Export to file’ as output option. Specify file name and destination for the extract file.

Diagram 1: CMM transaction /PBS/CMME_EXPORT


Before you start the extraction it is recommended to view some records as an example by using the Display/Print option. You will get a list as shown in the following diagram. The layout is defined by the display variant which you have chosen in the selection screen. By changing the display variant or creating a new one it is possible to change the layout structure of the list. Starting from the list the screen menu provides the function to change or define display variants.

Diagram 2: List of purchase orders selected by the CMM extract report


Use ‘Settings’ ‘Layout’ ‘Change’ to change the layout, i.e. the containing fields and the desired order. Save the layout structure as a new display variant, which you can choose in the selection screen of the extract report.

Diagram 3: Change of the list layout


3.2 Transfer to SAP BW / SAP NetWeaver BI

After extraction you will get an extract file in the following format. The first 8 lines describe the field structure (technical name, long/short description, type, length and decimal places). After this the extracted data is listed whereas each row is separated by tabs. Each line is terminated by CR LF.

Diagram 4: Extracted purchase documents in SAP AIS format

In the next step the data has to be transferred from the source system to the BW system or at least to the client workstation (front end).

The FTP transfer should be done in two steps. First, use binary mode from source to front end. Then use text mode from front end to destination to ensure that all lines are correctly terminated (CR LF in Windows servers, LF in UNIX servers).


3.3 Import into SAP BW 3.X

Importing data from flat files requires the definition of a flat file source system in SAP BW.

Make sure you have defined such a source system. If not, please proceed as follows. From the Administrator Workbench, click the Source Systems tabstrip. Select the appearing Source Systems node, right-click, and choose ‘create source system’. From the generated list, select ‘File System’ as shown below.

Diagram 5: Definition of a source system with system type “File”


In the next step you have to assign a flat file source system to an InfoSource. Since SAP BW has no direct interface to get the flat file structure, you need to manually define the communication structure.

It is recommended to use the InfoSource you have already been using for non-archived data.

If there is no adequate InfoSource it has to be created. From the Administrator Workbench, click the InfoSources tabstrip. Select an InfoArea, right-click, and choose ‘Create InfoSource’. In the appearing window a communication structure with appropriate InfoObjects has to be constructed, corresponding to the flat file field definitions.

Diagram 6: InfoSource communication structure


After defining the DataSource structure the transfer rules have to be set. These rules assign the InfoObjects of the InfoSource to the corresponding fields of the flat file structure.

Diagram 7: InfoSource transfer structure / transfer rules


After completing the transfer structure and the transfer rules of the InfoSource, there is only the definition of the InfoPackage left.

From the Administrator Workbench, right-click the flat file source system which has been assigned to the InfoSource, and select ‘Create InfoPackage’. From the appearing window, select the ‘External data’ tab and set the options as described in the diagram below.

File type: CSV file Data separator: 009 Hex Number of header rows: 08

Diagram 8: Specifying the data file format in the InfoPackage definition


It is recommended to check the import procedure with the ‘Preview’ function in the Scheduler before the data import gets started. Possible problems concerning the data layout (e.g. incompatible format) can be detected that way in an early stage.

After scheduling the data loads, you can monitor the progress of each data load job request. Often, differences between the flat file data structure and the transfer structure cause a failure of the load job. Make sure that the file definitions map correctly with the metadata defined in SAP BW.

3.4 Import into SAP NetWeaver BI 7.x

Importing data from flat files requires the definition of a flat file source system in SAP NetWeaver BI.

Make sure you have defined such a source system. If not, please proceed as follows. From the Data Warehousing Workbench, click the Source Systems tabstrip. Select the appearing Source Systems node, right-click, and choose ‘Create’. From the generated list, select ‘File System’ as shown in Diagram 9.

Now a DataSource has to be created. In NetWeaver 7 two types of DataSources are available: NetWeaver 7 DataSources and 3.x DataSources. In the following we will refer to the new type of DataSources.

From the source systems tabstrip in the DW Workbench, double-click on the generated source system. Select or create an application component, right-click, and select ‘Create DataSource’. From the appearing window, choose the data type of the DataSource: transaction data, master data text or master data attributes. In this example we are using the transaction data type. After that, you have to set the options as shown in Diagram 10. The most important options are as follows:

Data Format: Separated with separator (for example, CSV) Data separator: 009 Hex Number of header rows: 08

After setting the options the DataSource must be activated.

It is recommended to check the parameters with the ‘Preview’ function in the DataSource definition. Possible problems concerning the data layout (e.g. incompatible format) can be detected that way in an early stage.


Diagram 9: Definition of a source system with system type “File”

In the next step you have to connect the DataSource to a destination by defining a transformation. In SAP NetWeaver 7 the following BI object types can be used as destination: InfoSources, InfoCubes and DataStore Objects.

In this example we are using an InfoSource as a destination. If there is no adequate InfoSource it has to be created. From the Data Warehousing Workbench, click the InfoSources tabstrip. Select an InfoArea, right-click, and choose ‘Create InfoSource’. In the appearing window an InfoSource structure with appropriate InfoObjects has to be constructed, corresponding to the flat file field definitions (see Diagram 11). If necessary, the related InfoObjects must be created as well.

After defining DataSource and InfoSource the transfer rules have to be set. These rules assign the InfoObjects of the InfoSource to the corresponding fields in the DataSource. In SAP NetWeaver 7 a so-called transformation is used for that purpose.


Diagram 10: Definition of a DataSource in the “File” source system

Diagram 11: Definition of an InfoSource


Diagram 12 shows the definition of a transformation between DataSource and InfoSource. Unlike the modeling of 3.x transfer rules, the mapping is not done by combining the corresponding fields in a list, but uses a graphical design method with arrows. After completing the mappings, the transformation must be activated as well.

Diagram 12: Definition of transformation rules between DataSource and InfoSource


Diagram 13: Complete data flow from the DataSource through the InfoSource to an InfoProvider

After connecting the DataSource to an InfoSource, the InfoSource will be assigned to its data target. In this example we are using an InfoCube to serve that purpose.

Diagram 13 shows the complete data flow from DataSource to InfoCube. When using a NetWeaver 7 DataSource, the load process must be done in two steps: from source system to DataSource, and from DataSource to InfoCube.

In the first step the data is extracted from the source system and stored in the PSA table which is assigned to the DataSource. After creating an InfoPackage to the DataSource the extraction can be scheduled (see Diagram 14). After scheduling the data loads, you can monitor the progress of each data load job request. Often differences between the flat file data structure and the DataSource structure cause a failure of the load job. Make sure that the file definitions map correctly with the metadata defined in SAP NetWeaver BI.

The second step consists of creating a Data Transfer Process (see Diagram 15) and loading the data target with it (see Diagram 16). If the transformations are not properly designed, it will result in a failure of the load process. If the process succeeds without errors, the data import has been successfully completed.


Diagram 14: Scheduling the data load in the InfoPackage definition

Diagram 15: Creation of a Data Transfer Process (DTP) from DataSource to InfoProvider/InfoObject


Diagram 16: Scheduling the Data Transfer Process from DataSource to InfoProvider/InfoObject


4 Creation of a generic DataSource

When defining the DataSource transfer structure in the Administrator Workbench (see chapter 3), an existing DataSource can be very helpful. For that reason, the extract report creates an additional table in the Data Dictionary. This table is overwritten by the report during each extraction. Therefore, it always contains the current extract structure. The structures use the name /PBS/xxx_EXTRACT where xxx represents the respective PBS archive add on. By defining a generic DataSource in the R/3 or ERP system and moving it to the BW system, the transfer structure in this DataSource can be used as a template for the flat file DataSource in chapter 3.

The procedure is as follows: In the source system, call transaction RSO2. Choose a name for the new transaction data DataSource, and click the ‘Create’ button. Specify application component, description and extraction table. In our example, we are using /PBS/CMM_EXTRACT.

Diagram 17: Creating a generic DataSource in transaction RSO2


When saving your DataSource, then you will be asked to specify which fields are used for selection or should be hidden (see diagram below).

In order to complete the DataSource definition, you have to press the ‘Save’ button again.

Diagram 18: Specifying the fields for data selection


Replicate the DataSource in the BW system as shown below. From the Administrators Workbench, use the ‘Source Systems’ tabstrip, select the source system which corresponds to your R/3 or ERP system, and select the application component which you have specified above. Right-click and select ‘Replicate DataSources’. After that the DataSource structure should be available in your BW system.

Diagram 19: Replicating the defined DataSource via transaction RSA1


5 Module-specific Features

5.1 PBS archive add on CFI

The support of selected DataSources for archive access is available for SAP Release 4.6C and above. For SAP Release 4.6C the extractors and programs are delivered in a separate transport.

Activating PBS functionality

Archive data access is activated by exchanging the function module definition in table ROOSOURCE. To do this, PBS provides Customizing program /PBS/CFI_SETUP_BW which can be started via transaction SA38 or SE38.

Each DataSource can be activated individually for PBS archive access. To do this, you merely select the corresponding checkbox and start the program. For productive use, deactivate the button 'Test Run'. It is possible to reset to the SAP standard settings at any time.

The PBS BW extractors for DataSources 0FI_AR_4, 0FI_AP_4, 0FI_GL_4 and 0FI_TX_4 only support the update modes F (full) and C (initialization).

Recommendation:

1. Activation of PBS BW extractors

2. Execution of extraction in the modes F or C

3. Resetting of BW extractors to the SAP standard

4. Execution of delta extraction before the SAP data archiving

The already existing BW extracts which you only executed with SAP extractors need to be set up again.


Diagram 20: Activation/Deactivation of PBS versions for BW extraction


Customer-Specific Enhancements to Extractors

Many DataSources provided by SAP within Business Content can be used for data procurement from SAP source systems. These extract structures, however, often need to be changed, because all customers have their own requirements which data an extractor should provide. SAP supports this customer requirement in the form of append structures. The individual fields of the append structures that have been appended to an extract structure can be filled with data via function enhancements (Customer Exits). You can access the function enhancements via transaction code CMOD. First, a project must be created to which SAP enhancement RSAP0001 has been added. These Customer Exits now contain function modules that must be programmed via ABAP:

EXIT_SAPLRSAP_001 for the enhancement of transaction data

EXIT_SAPLRSAP_002 for the enhancement of attributes and texts

EXIT_SAPLRSAP_004 for the enhancement of hierarchies.

If a function enhancement has been performed in the function module for transaction data EXIT_SAPLRSAP_001, this needs to be checked in order to determine whether it is necessary to adjust it for PBS archive access. This is not done automatically via the activation program described above!

Support for the adjustment can be provided by the PBS Conversion Tool (see the separate manual in the software delivery or in the PBS menu under 'Documentation Guide') or for specific adjustments, the PBS Archive Access Tutorial via transaction /PBS/UTIL_ACCESS_ARC.


5.2 PBS archive add on CCO

The support of selected DataSources for archive access is available in the PBS archive add on CCO from SAP Release 4.6C.


Archive data access is activated by exchanging the function module definition in table ROOSOURCE. For this purpose, PBS provides the transaction /PBS/CCO_SETUP_BW.

Diagram 21: Activation/Deactivation of PBS versions for BW extraction


5.3 PBS archive add on CCOPA

The support of selected data sources for archive access is available in the PBS archive add on CCOPA from SAP Release 4.6C.


Archive data access is activated by exchanging the function module definition in table ROOSOURCE. For this purpose PBS provides the transaction /PBS/CCOPA_SETUP_BW.