How-to Guide SAP NetWeaver ‘04

How to… SAP BW Unicode with an MDMP Source System Version 1.00 – September 2004 Applicable Releases: SAP NetWeaver ’04 (SAP BW 3.5)

© Copyright 2004 SAP AG. All rights reserved. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. Microsoft, Windows, Outlook, and PowerPoint are registered trademarks of Microsoft Corporation. IBM, DB2, DB2 Universal Database, OS/2, Parallel Sysplex, MVS/ESA, AIX, S/390, AS/400, OS/390, OS/400, iSeries, pSeries, xSeries, zSeries, z/OS, AFP, Intelligent Miner, WebSphere, Netfinity, Tivoli, and Informix are trademarks or registered trademarks of IBM Corporation in the United States and/or other countries. Oracle is a registered trademark of Oracle Corporation. UNIX, X/Open, OSF/1, and Motif are registered trademarks of the Open Group. Citrix, ICA, Program Neighborhood, MetaFrame, WinFrame, VideoFrame, and MultiWin are trademarks or registered trademarks of Citrix Systems, Inc. HTML, XML, XHTML and W3C are trademarks or registered trademarks of W3C®, World Wide Web Consortium, Massachusetts Institute of Technology. Java is a registered trademark of Sun Microsystems, Inc. JavaScript is a registered trademark of Sun Microsystems, Inc., used under license for technology invented and implemented by Netscape. MaxDB is a trademark of MySQL AB, Sweden.

SAP, R/3, mySAP, mySAP.com, xApps, xApp, SAP NetWeaver, and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and in several other countries all over the world. All other product and service names mentioned are the trademarks of their respective companies. Data contained in this document serves informational purposes only. National product specifications may vary. These materials are subject to change without notice. These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty. These materials are provided “as is” without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP shall not be liable for damages of any kind including without limitation direct, special, indirect, or consequential damages that may result from the use of these materials. SAP does not warrant the accuracy or completeness of the information, text, graphics, links or other items contained within these materials. SAP has no control over the information that you may access through the use of hot links contained in these materials and does not endorse your use of third party web pages nor provide any warranty whatsoever relating to third party web pages. SAP NetWeaver “How-to” Guides are intended to simplify the product implementation. While specific product features and procedures typically are explained in a practical business context, it is not implied that those features and procedures are the only approach in solving a specific business problem using SAP NetWeaver. Should you wish to receive additional information, clarification or support, please refer to SAP Consulting.

- 1 -

1 Business Scenario

MDMP (Multi-Display, Multi-Processing) is a technology developed in the past by SAP, to enable the use of multiple character sets of disparate languages together within a single SAP instance. The current and clear technological solution for handling international languages with SAP systems is Unicode, which integrates all of the character sets of the world’s language into a single code page. Unicode is a relatively new solution (in terms of availiblity for widespread use). Many customer sites working with international languages, in earlier implementation projects, have relied on MDMP in order to support a multi-language SAP system environment.

Within the scope of a BW Unicode project, there may be an MDMP SAP source system (ie, SAP R/3, CRM, etc) for use with the Unicode BW system. In extracting data from an MDMP source system, it’s important to ensure that text data stored using MDMP technology is transferred to, and stored accurately in, the SAP BW system. Special handling is required in order to account for datasources where text characters are stored without a language key.

2 Background Information

2.1 MDMP (Multiple Display – Multiple Processing)

Prior to the availability of Unicode, the set of language characters available for use within a system was limited to a character set contained within a specific code page. For example, all of the characters of the English, German, Spanish, French and Portuguese languages are contained within the Latin-1 code page 1100, but characters from other various languages (ie Japanese Kanji) are not included in the set. There is no one single (non-Unicode) code page that includes the characters of dissimilar languages such as Russian, English, German and Mandarin (Chinese) together in one large set. Unicode is the only technology where text characters from multiple disimillar languages can be truly stored together in one database.

MDMP provides the ability to work with disparate languages together in an SAP system, using application techniques for storing and displaying text data (and thus working around pre-Unicode DB and OS limitations with code pages). When text data is entered into an MDMP system, each character is stored in the database using only its hexadecimal value – the hex value representing that character on the code page of the of logon language of the user who originally enters the data. When the data is displayed, each hex value is mapped back into a natural language character by the application.

MDMP functionality has the ability to dynamically assign a code page, mapping the hex value to a natural language character, based on the code page containing the language of the user’s logon session. A Japanese user working in an R/3 MDMP system (logged on with the Japanese language) can view texts that were originally entered in Kanji (Japanese), but this user cannot correctly view text data originally entered by a user logged on in Russian. This is because the hex values that represent the Russian text data would not map correctly to Kanji characters; the Japanese user would see “garbage” characters if they tried to, for instance, view a customer name that was entered with Russian characters.

An extraction job running against a DataSource in an MDMP system must have some indication of which code page to use, to properly map a record’s hex values into language characters when preparing a data package to send to BW. Some DataSources are based on application tables where a language key is stored originally along with specific text data in those cases the code page assignment is made easily (and automatically) by the extraction job. DataSources in an MDMP source system, where there is no language key available, present challenges for BW extraction and data load. Complexity may arise, in determining

- 2 -

how to best determine the proper language assignment for a record (which would correctly map the hex values representing text into language characters) during extraction processing.

SAP BW has not been released for MDMP and with the evolvement of Unicode, MDMP is not recommended anymore for other SAP systems.

For related information, please check also OSS note 745030.

2.2 Settings in Source System

If you declare a Unicode system in your source system, be sure that the source is defined as a Unicode source in SM59 / special options. If you are connecting a Unicode source system to a BW, you’ll have to set it up respectively.

2.3 Prerequisite for Values of Key Fields in SAP BW

In order to ensure a correct and smooth conversion, we strongly recommend all keys in SAP BW not containing any special characters, i.e. all keys in SAP BW being defined as US-7-bit-ASCII. This is true for InfoObject keys (like customer ID and material ID) as well as for InfoObject attributes (like country, material brand etc.). In case you have to build keys that exceed the US-7-bit ASCII character range, you have to make sure that these keys will be consistently transferred and converted from any source system. Usually, this is simple for single code page source systems, but for MDMP systems, you’ll have to make sure that the conversion is consistently done using the same principle as described below.

The technical prerequisites are SAP BW 3.1 Content SP 10 or higher and Service API 3.0B SP5 or higher in the source system. This Service API is included in PI_BASIS 2002.1 SP3 or higher.

2.4 SAP BW DataSources and Connection to MDMP sources

Unicode conversion relates only to character-like data types, i.e. text information.

Typically, transaction data and master data attributes do not contain any text information (if keys are designed as US-7-bit-ASCII – see above); so there would be no conversion necessary. In the rare cases they can do, proceed as with text data sources.

The DataSources that have to be analyzed in detail are usually master data texts (and hierarchy texts). There are two possibilities: the texts may be implemented language-dependent (i.e., the DataSource contains language information) or language-independent (i.e., the DataSource does not contain language information).

2.4.1 Language-dependent Master Data Texts

Most text DataSources are language-dependent. For example, plant text data or material text data can be entered in different languages. The following picture illustrates a language-dependent table as data source:

- 3 -

Several languages can be assigned to one code page (e.g. DE, FR and CH belong to the Western European Latin-1 code page). But as there is exactly one code page for any language in a SAP system, knowing the language is enough to uniquely determine the corresponding code page.

When loading data from this type of DataSource into a Unicode BW system, the extraction framework automatically recognizes the source language (and hence, the source code page). The characters of text fields are then interpreted properly, when adding a new record into the dataset to be sent to BW.

If the DataSource contains a language field, the conversion is done automatically without any additional set-up. In case there are several language fields in the extract structure, the system takes the first field. As of WebAS version 6.10, you can also set a language flag in order to specifically select the language decisive field.

You can check in transaction RSA6 of your source system, if your DataSource contains a language key. For example, 0MATERIAL_TEXT contains it like the following screenshot shows:

You can check the extract structure by double-clicking:

- 4 -

By double-clicking the language component (SPRAS), you can check if the ‘text lang’ field is set. This flag is only available for source system with WebAS version 6.10 or higher. In case there are more than one language field, you can flag the language field that should determine the conversion. If no flag is set – or in WebAS versions before 6.10 –, the first ‘language’ field (i.e., field where data element is of data type ‘LANG’) defines the source language for the conversion.

In the following illustration, note that the data type “LANG: identifies this as a language value; also note the checkbox for “text lang”, which as of 6.10 can be a special identifier for such text fields.

2.4.2 Language-independent Master Data Texts

Some DataSources do not contain a language key field. The common example is the customer name. You do not translate the name of a customer into different languages; the customer name is usually language-independent.

- 5 -

The following screenshot shows an example of the datasource 0CUSTOMER_TEXT in an SAP R/3 system, which typically does not contain any language designation:

In an MDMP R/3 system, the display of language text is limited to only characters contained within the specific code poge relevant to the user’s logon language.

SAP BW data extraction jobs run with a system user id. Every user id holds a language designation based on logon language, but that language’s assigned code page is likely to be too limited to handle many various international language characters (Japanese Kanji, Russian, Arabic, Hindi, etc). Therefore, additional information is necessary, in order to accurately determine proper dynamic language assignment for a given record’s text data. In most cases, heuristic rules may be implemented to accurately derive a language key for each datasource record. For example, the language of the country where the customer’s headquarters is located may be a good indicator for proper language designation. Another example could be a language key based on organizational assignment, such as sales organization. The communication language set for a sales organization might reasonably be applied to all customers for the sales org.

In order to ensure that language designation is accurately implemented for a datasource, careful analysis is needed of the business processes that drive with data entry into text fields. Questions of centralization vs. localized data entry processes often arise, in order to determine the best technique to utilize in a deriving language value. Understanding these processes typically requires signigicant input from business

- 6 -

people in the organization who have an understanding of data-generating operations. Business representatives to the project may need to enter into discussions with foreign colleagues, to fully understand the business processes related to text data entry in international locations. The project plan should allow time for such important internal discussions and information gathering to occur.

The logon language of the user, who originally performed the data entry, might not be stored along with text data entered. In these cases, techniques should be utilized during extraction to accurately determine a language key, one that best represents the probable logon language of the user who entered the text data originally.

For example, a global organization might have a subsidiary in Shanghai that handles the entry of all master data for all customers located in China, Malaysia, and Singapore. The users in Shanghai enter all customer names in Simplfied Chinese characters, but the data entry language is not stored with the customer text master data. In this example case, an accurate language key designation of “Chinese” might be set for all datasource records during extraction, for all customers with “China”, “Malaysia”, or “Singapore” maintained as their country in central address management.

For some organizations, it might not be accurate to simply utilize the communication language value (for customer text) directly from the ADRC table to designate language. The communication language in ADRC, for a customer number located in Kuala Lumpur, might not be Chinese. These types of questions must be resolved in when there is no language key for a datasource in an MDMP SAP source system.

In the following section, we describe the straightforward approach step-by-step – in the appendix, we will discuss another two approaches.

- 7 -

3 3 The Step-by-Step Solution

3.1 Identification of all relevant language-independent DataSources

First, you may want to indentify all language-independent DataSources, which you might need to consider.

You can use the following program to find most of the relevant DataSources.

report ybw_langindep_datasources . ************************************************************************ *This program let's you know if datasource has language key or not. It's *not 100% accurate as: * a) it can't recognise if langu field is available in the datasource * but is flagged to be available only in the user exit. * b) For complex structures like product hierarchy it can't find * language key even though it's available. *However above is very small sub-set compared to overall datasources that *you are trying to identify. ************************************************************************ tables: roosfield. "DataSource Fields select-options: s_oltps for roosfield-oltpsource. parameters: p_nolang as checkbox default 'X', "Display datasource where no langauge key is available. p_lang as checkbox default 'X'. "Display datasource where langauge key is available. * table for datasources with language field data: begin of itab_langu occurs 0, oltpsource like roosfield-oltpsource, field like roosfield-field, end of itab_langu. * table for datasources without language field data: begin of itab occurs 0, oltpsource like roosfield-oltpsource, field like roosfield-field, end of itab. start-of-selection. * get all datasources select oltpsource field from roosfield into table itab where objvers = 'A' and oltpsource in s_oltps. *get all datasources which has language key loop at itab. if itab-field cp '*LANGU*'

- 8 -

or itab-field cp '*SPRAS*'. move-corresponding itab to itab_langu. append itab_langu. endif. endloop. sort itab_langu. sort itab. * you have record for each field in this table that's why delete *duplicates. delete adjacent duplicates from itab comparing oltpsource. delete adjacent duplicates from itab_langu comparing oltpsource. end-of-selection. sort itab_langu. sort itab. loop at itab. read table itab_langu with key oltpsource = itab-oltpsource. if sy-subrc ne 0. if p_nolang = 'X'. *display datasource which do not have language field write:/ itab-oltpsource, 'No language key ' color col_negative. endif. else. if p_lang = 'X'. *display datasource for which language field is available write:/ itab-oltpsource, 'Nothing to be done ' color col_key. endif. endif. endloop.

3.2 Description of the scenario

As an example, we show how to extract correct customer information from an MDMP R/3 system to a Unicode BW.

You have to find a (heuristic) rule that enables you to derive the language (and, hence, the code page), which the customer name was entered in. In most cases, information about the country, where the customer lives in, or any organizational assignments (such as company code or sales organization).

This is not always true, as, for example, a customer in Japan can have Korean name (because he moved there). In order to be 100% correct and to meet high data quality standards, you will have to add information / code page information manually.

- 9 -

We will discuss the common approach using heuristics within a customer exit. You’ll find two other approaches in the appendix.

3.3 Using a user exit logic to enhance the language information

In this approach, you program your own logic in order to derive the language information. This will be done in the customer exit for the respective DataSources.

1. Enhance the DataSource in transaction RSA6. Search 0CUSTOMER_TEXT. Place the cursor here and press ‘Enhance Extract Structure’.

Enter ‘SPRAS’ as component and as component type.

2. Double click on the component SPRAS and set the flag ‘Text Lang’.

This automatically controls the conversion later – it is only available for source systems based on WebAS 6.10 or higher. Activate the Append Structure.

If the WebAS is lower than 6.10, the flag is not needed; instead the first field of type LANG will control the conversion.

3. Be sure that the language field is not hidden. Save the DataSource.

- 10 -

4. Now create a customer exit (here: ZMDMP) in transaction CMOD.

5. Enter a description and save the project.

Press ‘enhancement assignments’

6. Enter RSAP0001 for the BW DataSource

enhancements and save. Click on ‘Components’.

7. Double-click on EXIT_SAPLRSAP_002 for

enhancements of master data and texts. (EXIT_SAPLRSAP_001 is used for transaction data enhancements.)

8. Double-click on the Include-name

ZXRSAU02.

- 11 -

9. Click yes to create the include.

10. Double-click on the Include-name ZXRSAU02 and implement your own logic. Save and activate your include.

Note: the coding is only an example coding. It might not fulfill important performance requirements. Please let your code review by an experienced consultant!

Consider a modular approach when dealing with several DataSources. You could call a function module for every DataSource; this reduces complexity and maintenance.

11. Go back to transaction CMOD and activate your project.

- 12 -

12. Now you can extract the data. The customer exit is active and enhances your structure with the corresponding language value. This value forces the RFC modules to convert the data correctly.

Note that you do not have to modify the transfer structure in the BW!

- 13 -

4 Appendix

In the appendix, we describe additional two approaches for the MDMP connection. The first one results in a rather 100% correct solution and is achieved by manually adding the language information for EACH record. The second one avoids any changes in the source system and assigns the source language to the SAP BW job which is extracting the data.

4.1 Adding language / code page information manually The most time-consuming, but completely correct solution is to manually add the code page/language information to the table. The table possibly has to be extended by an additional field.

1. Go to ABAP dictionary (transaction SE11) and display table KNA1.

2. If no other field of component tyoe SPRAS is available, create new append structure ZKNA1_SPRAS and enter component SPRAS and component type SPRAS. Activate the append.

If KNA1 is not active, activate this table as well.

- 14 -

3. Now maintain the field SPRAS for all customer values (e.g. SE16 maintenance or custom-built UI). Enter the language, which the customer name was entered in.

4. Enhance the DataSource in transaction

RSA6. Search 0CUSTOMER_TEXT. Place the cursor here and press ‘Enhance Extract Structure’.

Enter ‘SPRAS’ as component and as component type. Activate the structure.

5. Double click on the component SPRAS and set the flag ‘Text Lang’.

This automatically controls the conversion later. Note that this field is not available before WebAS 6.10.

6. Replicate the DataSource into your SAP

BW and load the data. The customer text names are converted correctly according to the entered source language.

- 15 -

4.2 Splitting Processes according to source code pages If you can select customers whose names are written in a specific, known code page, you can define the source language for the specific job in BW. The following pictures illustrate this behaviour.

- 16 -

1. Be sure that in the RFC connection to your source system, no language is defined.

2. Go to the InfoPackage that loads

0CUSTOMER_TEXT data and enter specific customers (e.g. 1000 – 2000).

3. Schedule the InfoPackage for a given time

(possibly periodically) and start the job.

- 17 -

4. Click on ‘Jobs’ and confirm the given job name by pressing ‘Execute’.

5. Now press ‘Step’.

6. Place the cursor on the line and press

‘Change’ (the pencil).

7. Enter the source language here and save

your settings.

- 18 -

8. When the job starts, all texts for customers between 1000 and 2000 will be converted from Japanese.

www.sdn.sap.com/irj/sdn/howtoguides