Best-Practice Experiences for Smooth SAP NetWeaver MDM

Best-Practice Experiences for Smooth SAP NetWeaver MDM Implementations

Applies to: SAP NetWeaver MDM 5.5 SP06, Outlook for SAP NetWeaver MDM 7.1 For more information, visit the Master Data Management homepage.

Summary This document provides sizing- and performance-related information for the SAP NetWeaver Master Data Management 5.5 platform. Besides the description of "technical" topics like Multithreading, Locking, and Scalability, there are also "application"-specific topics like Matching, Import Steps, and Syndication Steps discussed. In addition recommendations for data modelling are provided.

Authors: Detlef Thoms, Klaus David

Company: SAP AG

Created on: 01 September 2008

Author Bio

Detlef Thoms has ten years of SAP experience that he has gained in Development Support in the software development and as SCM Senior Solution Consultant in SAP projects and numerous SAP implementations. Since January 2006 he is working for SAP NetWeaver RIG in the area of Master Data Management.

Klaus David works at SAP Labs, LLC, with a focus on SAP NetWeaver Master Data Management (SAP NetWeaver MDM), and is a member of the SAP NetWeaver RIG Americas. Prior to joining the RIG, he worked in Product Management for SAP NetWeaver MDM.

SAP COMMUNITY NETWORK SDN - sdn.sap.com | BPX - bpx.sap.com | BOC - boc.sap.com © 2008 SAP AG 1

https://www.sdn.sap.com/irj/sdn/nw-mdm

Error! No text of specified style in document.

Table of Contents 1. Core capabilities .............................................................................................................................................3

1.1 Repository load time .................................................................................................................................3 1.2 MDS – Runtime/multithreading .................................................................................................................3 1.3 Matching....................................................................................................................................................4 1.4 Matching Strategy: Match attribute values................................................................................................7

2. Import Process..............................................................................................................................................10 2.1 Memory Footprint and Import Performance............................................................................................10 2.2 Importing single source field to multiple display fields ............................................................................14 2.3 Strategies for Attribute/Value Import.......................................................................................................16 2.4 Excel Import ............................................................................................................................................20 2.5 Suppressing Initial Syndication ...............................................................................................................22 2.6 Field triggers for syndication ...................................................................................................................23

3. Landscape – Portal – MDM communication.................................................................................................27 4. Related Content............................................................................................................................................29 Copyright...........................................................................................................................................................30



1. Core capabilities

1.1 Repository load time

Business Case: There is a significant difference between the loading time for loading with or without indices. If the indices are created once, there is normally no need to load the repository with indices again.

When should a repository be loaded with indices?

The indices have to be recreated only if an error occurs. By loading with indices, the load time can increase up to three times compared to a load time for loading without indices.

Recommendation:

o Set the CPU count= {actual number of CPUs} parameter accordingly (hyper threading does not count). Repository loading is a multi-stage process and some of the stages run in parallel to some extend if CPUs are available. As of MDM 5.5 SP6, this setting applies only to building the sort indices during the Load Repository operation in the pre-load stage.

What can be done if the load process crashes?

1. Load without creating indices Nevertheless, the missing indices will be created.

Only if step 1 fails:

1. Restart MDS

2. Verify, repair, and load with rebuild indices

1.2 MDS – Runtime/multithreading

Business Case: For some processes there seems to be a performance issue.

o Does additional hardware help to improve performance?

o Are multiple threads used for requests?

o Are multiple CPUs useful?

o How are read and write requests handled?

All MDM requests to a given repository are placed in that repository's Queue:

o Whereas read requests can be executed in parallel, a write request will require exclusive access to the MDM repository.

o If a write operation is the next task in the request queue, the execution will wait until previous read requests have been worked off.

o Then the write operation will be initiated and subsequent operations have to wait until it is finished.



Remark: As the lock of a repository only takes a very short period of time, the end user might notice this behavior only during parallel import or syndication of high volumes

Recommendation: Plan mass transactions like mass import accordingly for example in nightly batch runs

1.3 Matching

Business Case: A matching run lasts very long. What are the influencing parameters and what matching strategies can be used to improve performance? How is an operation calculated?

Influencing parameters for matching time are:

o Number of matching rules

o Function „Equals“ vs „Token Equals“

o Number of source records

o Number of target records

Example: The number of operations for 3000 x 50000 records is 3000 multiplied by the number of rules in the matching strategy.

In the strategy “MDM_Persons” there are 4 rules, so the number of operations (displayed when the matching is run) is 12000.



Remark: It takes less time to perform an operation if function “Equals” is used due to the number of specific comparisons.



Recommendation:

o Use workflow to plan nightly matching run and persist matching results

o Try to reduce the number of tokens

By using transformations matching rules. Matching runs on the transformed field.

BUT: Transformations are performance intensive if a large number of records is involved.

By creating an additional matching field in the data model and filling it via import using value conversion filters (Replace (, ), AG, Corp, ….). Matching runs on the additional field.

Matching is done asynchronously. This means if the data manager is “killed” by the user, the matching task still runs on the MDM server. A very large matching run might freeze the Data Manager (“UI freeze”) that started it. But it will be possible to log on from another Data Manager. A performance issue associated with executing very large matching strategies in real-time might impact the usability of other logged in users if only one CPU is available. Multiple CPUs do not help improve the performance of a single matching task, but can help support more concurrent matching tasks (1 CPU for each 1 matching run).

Recommendation: Split up huge matching runs in several smaller runs if possible (means if business requirements still can be met) to benefit from multiple CPU‘s



Outlook: Multi-threaded matching is planned to be enabled. This means a matching run will benefit of multiple CPU‘s and improve responsiveness.

1.4 Matching Strategy: Match attribute values

Business Case: The following chapter describes how to create a matching strategy to match attribute values.

To identify duplicates there is the need to compare the taxonomy values. The comparison of the category might be not sufficient.

Therefore the attribute values have to be compared.

Attributes usage and fill rate must be analyzed before implementing any of these methods, to minimize the list of attributes which are matched as part of the strategy.

Listed below are the alternate methods which can be combined together to achieve best matching results and performance.

The general idea is to use the “equal” function instead of “Token equals” to improve performance.

o Import the attribute values into the matching fields (in addition to the taxonomy attributes).

Advantage: Repository loading time is not damaged because of calculated fields.

Disadvantages: The list of matched attributes is not dynamic; it is embedded in the repository schema. Import maps should be modified.

o Calculated fields holding distinct attribute values.

Advantage: The list of matched attributes is dynamic; it is determined by the calculated field expressions.




Disadvantage: Repository loading time is higher because of calculated fields.

o Ca

hed

cause of calculated fields. The concatenated attribute values should not be NULL.

ulated with mandatory taxonomy attribute values.

1. y match field to the main table

rformed

“NULL” value.

3. Attributes can be added to the calculation expression

lculated field holding concatenated mandatory attribute values.

Advantages: Only single field is matched for all attributes. The list of matcattributes is dynamic; it is determined by the calculated field expressions.

Disadvantages: Repository loading time is higher be

Example: Calculated field holding concatenated mandatory attribute values

For a matching on attributes an additional field is pop

Add taxonom

2. Add callable validations for calculation of “Matching Attributes” field

To define the mandatory attributes within a repository following steps can be pe

1. Select the taxonomy branch at the Data Manager search pane.

2. Use the attributes search pane to find attributes with no



Remark: Be aware that an attribute is not mandatory any more if a new record is imported that has a „Null“ alue for a mandatory attribute.

v


2. Import Process

2.1 Memory Footprint and Import Performance

Business Case: An import lasts very long. A further investigation shows that the number of records to be imported compared to the number shown in the preview tab of the Import Manager does not fit.

Why does the Import take so long?

Example:Two Main table records (e.g material master), several segments for each record

Sample XML Structure - and two sample records for your orientation:



Based on the sample the following number of records will be imported:

In a real life example for the number of 2 records for Main Table and 5 nested structures with different occurrences per record this will lead to the import of 1860 records!



The increase of numbers of records is caused by adding Joins & Lookups.

Recommendation:

o Perform several imports with simplified Import Maps. Split a single complex import map into individual maps - one to load the main table, and additional maps for each qualified lookup table.

o Split Import Files into several files with less segments

One for Main table only

One for each Segment in XSD-Structure

o Whenever possible, load all the reference data (flat/ hierarchy/taxonomy tables) before loading the main table.



Recommendation applied to our real life example leads to import of 43 records compared to 1860 records !

For MDM 7.1 the import logic will be adjusted as follows:

Because MDM then imports directly from the XML file, rather than the virtual extended table, some import-related features behave differently in MDM 7.1 than they did in MDM 5.5. Please check the release notes.



2.2 Importing single source field to multiple display fields

Business Case: A Main table-field with reference to Lookup Table containing two Display fields has to be imported.

Lookup table has key-mapping disabled, the initial population of lookup table using both display-fields as matching key

During Import into Main table only “incomplete” information regarding lookup-values is provided (Only ISO-Code, but no country name provided).

This means automapping of values doesn’t work here. How to perform an efficient automated import of the values?

For better understanding the problem in detail:




Solution:

Enable Key-mapping for Lookup-table and use remote-key for matching during initial population.

For import in the main table mapping in Import Manager is much more comfortable.


2.3 Strategies for Attribute/Value Import

Business Case: From an ERP-system main table records including taxonomy and attribute information should be included. How to perform Import Mapping for attributes as convenient as possible?

Solution 1:

o XSD-File Definition has attributes in own segment



Challenges

o Each attributes defined as an own element !!!! => XSD-Defintion could get very large

o New attribute in Taxonomy leads to enhancement in XSD-Definition => Repository needs to be unloaded for updates

o The more attributes you have, the slower the Import works

o In Import Manager each attribute needs to be mapped



Solution 2:

o XSD-File Definition has attributes in own segment

o Attributes are defined abstract

o Attributes are unbounded in definition

Example for a XML-File with values:



What has to be done in Import Manager?

Advantages of solution 2:

XSD structure is very simple structured

o Doesn’t change when additional attributes need to be created

o This means also no repository maintenance required



o Import Mapping very simple

– Only one field to be mapped to handle all attribute/value combinations !!!!

2.4 Excel Import

Business Case: Excel sheets are imported from different regions.

o What is the influence of the regional settings?

o How should the different decimal separators “,” and “.” be handled in the import maps?

o How are the different formats of Excel are handled in Import Manager?

Example: Import “Price” information

o Currency (General, Number)



o Text - Different settings of decimal point

In the current release it is not supported to parse two different settings of the decimal point (/thousands) in source tables with the same map, since the current design is the ability to have only one setting per map.

Recommendation:

o Use only “Numeric” Excel formats for the prices as described in the previous slide

o If “Text” format should be kept, use two import maps if source is always known

o Text - „Text“ interpretated as „Numeric“

Why is the Price formatted as „Text“ in Excel interpreted as NUMERIC in Import Manager?

For the interpretation of Import Manager not the format but the internal Data type Excel is using is relevant

How to check the Excel Data Type Internal Data type:



2.5 Suppressing Initial Syndication

Business Case: Initial Syndication should be suppressed

Important:

o Suppress “Unchanged Records” doesn’t help

o MDM memorizes updated systems by “Remote system”, not by Map or Port

Solution

o Use MDM Syndicator GUI and create a simple map, e.g. containing only one field

o Perform Syndication for remote-system and choose as destination your harddisk

o After syndication the remote-system is considered “syndicated”

o Now start Syndication server => from now on only Delta-changes will get syndicated for the remote-system.



2.6 Field triggers for syndication

Business Case: A record should be syndicated only, when certain fields, e.g. “Description”, “Material Type” have been changed

Motivation

o Perform Syndication only, when important fields are changed

o Avoidance of unnecessary triggering of approval workflows in Client system

A workflow does the trick:



Data Manager Work – Workflow definition

o Workflow called immediately after Record Update, Record add, Record Import

o Validation checks whether specific fields have changed

One branch (called when validation returns true) triggers syndication (marks record for syndication)

One branch stops the workflow (no action in this case)

Console Work – Definition of two additional fields in Main Table

o TimeStampRecord of type Time Stamp to document change of any field in record

o TimeStampChange of type Time Stamp to document changes of fields Description, Material Type or Materials Group



Data Manager Work – Definition of a validation

o Validation checks, whether fields Description, Material Type or Materials Group changed

o Simple comparison of previously defined Time Stamp fields

Syndicator Work – Syndication Map definition

o No special selection criteria used

o Goal: No record should slip through the selection criteria



Console Work – Port Definition

o Port should grab only these records which were triggered by our workflow definition => processing type needs to be set to manual

o If other records should be syndicated (not determined by workflow) => additional port needs to be defined



3. Landscape – Portal – MDM communication Business Case: A portal should be connected to MDM. What has to be done?

The connection between the portal and the MDM Server works with user mapping only. From 7.1: User mapping is not obligatory.

Either the portal user itself or the portal user group is mapped to MDM repository user(s). Then the mapping data is inherited from the group. This is also possible using roles.

Recommendation: Create user groups and map the portal user group to an MDM repository user.

Outlook for MDM 7.1

1. MDM Connectivity:

All MDM Java applications connect to MDS via Java API and MDM Connector, using either Trusted Session or Authenticated Session.

Trusted Session

o The MDS is configured with a list of trusted connections. These connections are specified using their IP address.

o Authentication: only user names are passed to MDM

o Possibility of change tracking based on portal user

o For further information please see the MDM 7.1 Security guide

Authenticated Session

o Authentication: user name and password are passed to MDM (User Mapping only)

2. SSO Support:

The connection to MDM via the SAP logon ticket will be supported – Only for iViews and Web Services. The ticket evaluation is done by the portal and not by the MDS – as such this is not real SSO. The user will be extracted from the ticket and propagated to MDS.

Prerequisites:

Trusted Connection WAS-MDS configured

Same users in Portal and in MDS: LDAP directory or users replication

User Management properties will not be added to MDM System object.



Example:



4. Related Content • SAP MDM 5.5 Documentation Center:

service.sap.com/installMDM

• How to guides: www.sdn.sap.com/irj/sdn/howtoguides → Data Unification

o How to Activate Field Triggers for Syndication

o Good to Know Topics Related to Sizing and Performance

o How to Avoid Problems with Your Data Model in SAP NetWeaver MDM – Do's and Don'ts




Copyright © 2008 SAP AG. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.

Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors.

Microsoft, Windows, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation.

IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries, xSeries, zSeries, System i, System i5, System p, System p5, System x, System z, System z9, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, Informix, i5/OS, POWER, POWER5, POWER5+, OpenPower and PowerPC are trademarks or registered trademarks of IBM Corporation.

Adobe, the Adobe logo, Acrobat, PostScript, and Reader are either trademarks or registered trademarks of Adobe Systems Incorporated in the United States and/or other countries.

Oracle is a registered trademark of Oracle Corporation.

UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group.

Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc.

HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology.

Java is a registered trademark of Sun Microsystems, Inc.

JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape.

MaxDB is a trademark of MySQL AB, Sweden.

SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary.

These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

These materials are provided “as is” without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement.

SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials.

SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party web pages.

Any software coding and/or code lines/strings (“Code”) included in this documentation are only examples and are not intended to be used in a productive system environment. The Code is only intended better explain and visualize the syntax and phrasing rules of certain coding. SAP does not warrant the correctness and completeness of the Code given herein, and SAP shall not be liable for errors or damages caused by the usage of the Code, except if such damages were caused by SAP intentionally or grossly negligent.

Documents

Best-Practice Experiences for Smooth SAP NetWeaver MDM