Know-How Network: SAP BW Data Load Performance BW Data Load Performance Analysis and Tuning ... ALE ALE IDOC ALE IDOC Update Rules Transfer ... high ABAP Runtime:

  • View
    216

  • Download
    3

Embed Size (px)

Text of Know-How Network: SAP BW Data Load Performance BW Data Load Performance Analysis and Tuning ......

  • Know-How Network: SAP BW Data Load Performance Analysis and Tuning

    Ron SilbersteinPlatinum ConsultantNetweaver RIG US Business Intelligence SAP Labs, LLC

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein2

    Welcome SAP Developer Network!

    http://www.sdn.sap.com

    http://www.sdn.sap.com/http://www.sdn.sap.com/

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein3

    Agenda 3

    Loading InfoCubes and ODS Objects

    Data Load Overview

    PSA, Transfer Rules, Update Rules

    Extraction Performance

    Parallel Data Load

    Aggregate Maintenance

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein4

    Agenda 4

    Loading InfoCubes and ODS Objects

    Data Load Overview

    PSA, Transfer Rules, Update Rules

    Extraction Performance

    Parallel Data Load

    Aggregate Maintenance

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein5

    Overview: Data Load Process 5

    Goals of performance optimization:First tune the individual single execution and then the whole load processes.

    Eliminating unnecessary processesReducing data volume to be processedDeploying parallelism on all available levels

    Parallel processes are fully scalable

    SourceSystem

    Business InformationWarehouse

    BWS-APIBW

    S-API

    PSA

    LoadingProcess

    Extractor

    Extractor

    InfoCube

    ODS

    BWS-APIBW

    S-API

    ALE

    ALE

    IDOC

    ALE

    IDOC

    UpdateRules

    TransferRules

    ALE

    Scheduler

    tRFC

    The typical BW data load process :

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein6

    SAP Service API: Extraction/Load Mechanism

    InfoCube

    Transfer Rules

    Update rules

    Communication StructureCommunication Structure

    transfer structuretransfer structure

    Master Data

    Transfer Rules

    Communication StructureCommunication Structure

    AttributesTexts

    transfer structuretransfer structure

    Extraction Source StructureExtraction Source Structure

    Header

    ItemTransaction Data

    transfer structuretransfer structure

    transfer structuretransfer structure

    Extraction Source StructureExtraction Source Structure

    ATTRTXTMaster Data

    SourceSystem

    BW

    DataSourceDataSource

    Update rules

    BWS-API

    BWS-API

    ODSBW

    S-API NEWBW 3.0

    6

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein7

    Agenda 7

    Loading InfoCubes and ODS Objects

    Data Load Overview

    PSA, Transfer Rules, Update Rules

    Extraction Performance

    Parallel Data Load

    Aggregate Maintenance

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein8

    How to identify high Extraction Time ? 8

    Determine the extraction time:

    Data load Monitor :Transaction RSMO

    SourceSystem

    Business InformationWarehouse

    PSA

    LoadingProcess

    Extractor

    Extractor

    InfoCube

    ODS

    ALE

    ALE

    IDOC

    ALE

    IDOC

    ALEScheduler

    tRFCUpdaterules

    Transferrules

    S-API

    S-API

    Extraction

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein9

    Monitoring Extraction: Resource Utilization 9

    Look for Many Long-Running Processes Further Analysis in case ofResource problems when extractingdata...

    Check SM51 / SM50 in the source system

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein10

    Extraction Time is too high ? 10

    For specific application areas specific notes exist; refer to relevant SAP notes and apply when necessary.

    Further Analysis

    in case of

    PERFORMANCE

    problems

    extracting data...

    Transaction:

    RSA3

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein11

    Extraction Time is too high ? 11

    Analyze high ABAP Runtime:

    Use SE30 Trace option in parallel session. Select corresponding Work process with extraction job

    Particularly Useful forUser Exits

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein12

    Extraction Time is too high ? 12

    Identify expensive SQL Statements

    Use ST05 Trace with Filter on the extraction user (e.g. ALEREMOTE). Make sure that no concurrent extracting jobs run at the same time with this execution.

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein13

    Extraction Tuning: Load Balancing 13

    Parallel processes:distribute to different servers

    avoid bottlenecks on one serverConfig in table ROIDOCPRMS

    RFC destinations (SM59)Example: RFC connection from BW to R/3 and R/3 to BW

    InfoPackages, event chains and Process Chains: all can be processed on specified server groups.XML Data loads: HTTP/HTTPS processes can be allocated to specific server groups

    Expected Results:Avoid CPU/Memory bottlenecks on one server Greater Throughput: Faster time to completion per request

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein14

    Extraction Tuning: Configuring DataPackage Size 14

    Size of DataPackages: Influencing FactorsSpecific to application datasource, the contents and structure of records in the extracted datasets. Package size: impacts frequency of COMMITs in DB .SAP OSS note 417307: Extractor Packet Size Collective Note for SAP ApplicationsConsider both the source system and the BW system (RSADMINC).Package size specified in table ROIDOCPRMS and/or InfoPackages

    Scenario:Set up the parameters according to the recommendations; if upload performance is not improved, try to find other values that fit exactly your requirements.

    Expected results:In a resource constrained systems, reduce DataPackage sizeIn larger systems, increasing the package size to speed collection;

    but take care not to impact communication process and unnecessarily hold work processes in SAP source system.

    Greater throughput = Faster time to completion per request

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein15

    Tuning Extraction: Configuration (Slide 1) 15

    Further Analysis in case of Resourceproblems when extracting data...

    OSS note409641 fordetails

    Double Check ROIDOCPRMS Settings

    If no entry was maintained, the data is transferred with a standard setting of 10000 kbyte and 1 Info IDOC for each data packet.

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein16

    Tuning Extraction: Configuration (Slide 2) 16

    Consider Decreased Data Package Size for an specific InfoPackage

    Further Analysis in case ofResource problems when extractingdata...

    ? ?

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein17

    Tuning Extraction: Configuration (Slide 3) 17

    Use Selection Criteria Further Analysis in case ofResource problems when extractingdata...

    ?

    ?Consider building indices on DataSource Tables based on selection criteria

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein18

    Extractor Tuning: SAP and Generic DataSources 18

    SAP Content extractionConvert old LIS extractors to Logistics Extraction CockpitV3 Collection jobs for different DataSources can be executed in parallelTune customer exit coding

    Generic extractors:Collector jobs can be executed in parallelInfoPackages executed in parallel to extract data

    Not possible for delta extracts from one generic data sourceInvestigate Secondary indexes on fields used for selectionOptimize custom collector ABAP coding

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein19

    Extraction / Load Tuning: Flat Files 19

    Use a predefined record length (ASCII file)

    File should reside on the application servernot on the client PC

    Avoid large loads across a network.

    Avoid reading load files from tape (copy to disk first)

    Avoid placing input load files on high I/O disksExample: same disk drives or controllers as the DB tables being loaded.

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein20

    Agenda 20

    Loading InfoCubes and ODS Objects

    Data Load Overview

    PSA, Transfer Rules, Update Rules

    Extraction Performance

    Parallel Data Load

    Aggregate Maintenance

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein21

    Analyze high PSA Upload Times

    SourceSystem

    Business InformationWarehouse

    PSA

    LoadingProcess

    Extractor

    Extractor

    InfoCube

    ODS

    ALE

    ALE

    IDOC

    ALE

    IDOC

    ALEScheduler

    tRFCUpdaterules

    Transferrules

    S-API

    S-API

    Data load Monitor :Transaction RSMO

    Load Into PSA

    21

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein22

    PSA Partitioning 22PSA

    PSA Table PartitionsData Load

    Size of each PSA partition, value in number of records

    Transaction SPRO or RSCUSTV6 :From SPRO Business Information Warehouse > Links to Other Systems > Maintain Control Parameters for the Data Transfer

    Note: If you start more than one load process at a time expecting to have each request in a separate partition, it probably will not work as expected; the PSA threshold is not yet reached when the second process starts writing into PSA

  • 2003 SAP Labs, LLC,Know-How Netowke , Ron Silberstein23

    Data Processing-Transfer Rules

    SourceSystem

    Business InformationWarehouse

    PSA

    LoadingProcess

    Extractor

    Extractor

    InfoCube

    ODS

    ALE

    ALE

    IDOC

    ALE

    IDOC

    ALEScheduler

    tRFCUpdaterules

    Transferrules

    S-API

    S-API

    Data load Monitor :Transaction