Unicode in Netweaver

8/4/2019 Unicode in Netweaver

http://slidepdf.com/reader/full/unicode-in-netweaver 1/118

© 2004 Hewlett-Packard Development Company, L.P.The information contained herein is subject to change without notice

Unicode in SAPN etW eaver

Sebastian Buhlinger

SAP Consultant, HP-SAP EMEA CC



3/ 31/ 2004 2

Agenda

1. Introduction to Unicode

2. Unicode & SAP in General

3. Technology in Depth

4. Sizing Information for Unicode-based SAP Systems



3/ 31/ 2004 3

Introduction to Unicode



3/ 31/ 2004 4

1. Introduction to Unicode

• W hat is text?• History of character encoding

• Problem of character encoding

• From ASCII to Unicode• W hat is Unicode exactly?

• The Unicode Standard

• W here is Unicode used?

• The Unicode Consortium

• Unicode Encodings



3/ 31/ 2004 5

W hat is text?

• Code pages & encodings describe the handling of and theway text is stored in

• Computers• Files• Data structures

• Inside a computer program or data file, text is stored as asequence of numbers – just like “everything else”

• A character is a:• Letter,• Digit,• Period,

• Hyphen,• Punctuation or• Math symbol

• Furthermore there are control characters – typically not visible



3/ 31/ 2004 6

History of Character Encoding

• Historically, computers were pretty slow, had fairlylittle memory and were very expensive

• Up to 1960s I/ O meant pushing holes into papertapes

• Most of the character sets date back to punch-cardage and are designed with these cards in mind

• In the early days of computers every hardwaremanufacturer used proprietary technology (andencodings)

• International data interchange was no issue andso nothing needed to fit together



3/ 31/ 2004 7

Problem of character encoding

• W hich number is assigned to which character?• W hen typing an ‘A’ on the keyboard, the

computer uses the character code as a basis forpulling the character shape of ‘A’ from a font filelisting with the same binary number, and displaysor prints it

• The character ‘A’ may also have different integervalues in different programs or data files (‘A’ mightbe ‘ ’ in an Arabic font file)

• In some instances no number available for certaincharacters (f.i. “&auml” à Ä)

• All data encoded in the form of binary numericalcodes



3/ 31/ 2004 8

Character repertoire

• English alphabet: with some digits and littlemore:~ 60 characters

• W estern European Standard: ~ 300 characters

for several languages• Korean: ~12.000 syllables

• Chinese dictionaries: ~ 50.000 letters

• Hundreds of other characters in common use,such as math and currency symbols



3/ 31/ 2004 9

From ASCII to Unicode

• Most character sets and encodings in 70s/ 80swere modifications or extensions of ASCII

• Many of them used 8-bit with a subset of the94 used ASCII characters

• Most common encodings nowadays use singlebyte per character (SBCS)

• They are all limited to 256 characters

• Due to that, none of them can even cover theletters for the W estern European languages



3/ 31/ 2004 10

From ASCII to Unicode

• Consequence: many different 8-bitencodings were created to fulfill the needsof different user communities

• Solution for data interchange in globalnetworked information society andcollaborative business world:single character set for all languages in use

• Unicode can encode 4.294.967.296different characters, symbols and controlcharacters



3/ 31/ 2004 11

W hat is Unicode exactly?

• Unicode = universally encoded character set tostore information from any language

• Unicode defines• properties for each character

• standardizes script behavior• provides a standard algorithm for bi directional text• defines cross-mappings for other standards

• Unicode defines a unique code value for every

character, regardless of platform, program orprogramming language used





3/ 31/ 2004 13

W hat is Unicode exactly?

• Additionally it also includes punctuationmarks, diacritics, mathematical symbols,technical symbols, musical symbols, arrows,dingbats etc.

• In all, the Unicode Standard comprises>95.000 characters, ideograph sets,

symbols (version 4.0)



3/ 31/ 2004 14

The Unicode Standard

• The Unicode Standard is a character codingsystem designed to support the worldwide• interchange,• processing,

• and displayof written text of the diverse languages andtechnical disciplines of the modern world

• In addition, it supports classical andhistorical texts of many written languages



3/ 31/ 2004 15

W here is Unicode used?

• The Unicode standards has been adopted bymany software and hardware vendors

• Mosts OSs support Unicode

• Unicode is required for international document

and data interchange, the Internet and theW W W , and therefore by modern standards suchas:• Java, C#, Perl, Python• Markup languages such as XML, HTML, XHTML,

MathML, W ML etc.• JavaScript• LDAP• CORBA etc.



3/ 31/ 2004 16

The Unicode Consortium

• The Unicode Consortium is a non-profitorganization originally founded to• develop,• extend,

• and promotethe use of the Unicode Standard

• Members of the Consortium include major

computer corporations, software producers,database vendors, research institutions,international agencies, various user groups,and interested individuals



3/ 31/ 2004 17

The Unicode Consortium

• The Consortium cooperates with• W 3C and• ISO• and has liaison status "C" with ISO/ IEC/

JTC 1/ SC2/ W G2, which is responsible for inrefining the specification and expanding thecharacter set of ISO/ IEC 10646



3/ 31/ 2004 18

Unicode Encodings

• UTF = Unicode Transformation Format• UCS = Universal Character Set

• CESU = Compatibility Encoding Scheme

• Conversion between different encodings is asimple, bit-wise operation (defined instandard)

• No performance excessive conversion tablenecessary!



3/ 31/ 2004 19

Unicode Encodings

• UTF-8: Unicode Transformation based on 8-bit representation

• CESU-8: Compatibility Encoding Scheme ofUTF-16 on an 8-bit base

• UTF-16: Unicode Transformation based on

16-bit representation



3/ 31/ 2004 20

Unicode Encodings

• UCS-2: Universal Character Set 2 bytevariation (16-bit)

• UTF-32: Unicode Transformation based on32-bit representation

• UCS-4: Universal Character Set 4 byte

variation (32 bit)



3/ 31/ 2004 21

Unicode Encodings

• Not all Unicode characters are 2 bytes long’ no doubling of hw requirements in thefirst place

• Unicode encoding determines the length ofa character

• Character in one Unicode encoding can belonger than 1 byte; therefore Unicode

characters can be longer than charactersdefined in a standard code page





3/ 31/ 2004 23

UTF-8

• UTF-8 used for UNIX-platforms, HTML and mostInternet Browsers

• Main benefits of UTF-8:• compact storage requirements for European

scripts• in general European scripts will occupy less

storage on disk and memory

• ease of migration –> since 7-bit ASCII dataremains the same in UTF-8, data conversioneffort between ASCII based character sets andUTF-8 is reduced significantly



3/ 31/ 2004 24

UTF-8 / CESU-8 (8-bit encodings)

• 8-bit encodings are well-suited for datatransfer since all 7-bit ASCII and 8-bit ISOcharacters retain the same code points

• Easier communication with legacy and non-Unicode systems

• Downside: variable character length



3/ 31/ 2004 25

UCS-2

• UCS-2 has a fixed width of 16 bit (2 bytes)• UCS-2 is the Unicode encoding for Java & W in NT 4.0

• Main benefits of UCS-2:• More compact storage requirements for Asian scripts

(each character represented with 2 bytes only)• String processing will be faster because all charactersare of the same width

• Good compatibility with Java and Microsoft clients

• Downside:

• UCS-2 can support Unicode characters defined up toUnicode 3.0 only (max. 65.536)



3/ 31/ 2004 26

UTF-16

• UTF-16 is the 16-bit encoding of Unicode• Basically an extension of UCS-2

• One Unicode character can be 2 or 4 bytes inthe encoding

• Characters from European and most Asianscripts are represented in 2 bytes

• Supplementary characters are represented in 4

bytes• UTF-16 is the main Unicode encoding from

Windows 2K



3/ 31/ 2004 27

UTF-16

• Main benefits of UTF-16:• More compact storage requirements for Asian

scripts (2 bytes for commonly used characters)• Ideal if European and Asian scripts are used

together--> UTF-16 will occupy less storage on disk andmemory than with UTF-8 (3 bytes for Asian part)

• Balance of efficient access to characters andeconomical use of storage

• Above mentioned points reason for use of UTF-16in SAP W eb Ap p licat ion Serv er



3/ 31/ 2004 28

UCS-2 / UTF-16 (16-bit encodings)

• 16-bit encodings offer a compromisebetween the pros and cons of the 8-bit andthe 32-bit encodings, respectively

• They do not need as much memory as 32-bitencodings, but offer quasi fixed characterlength

• UCS-2 has a fixed character length, but itcannot define more than 2^16 (65.636)characters



3/ 31/ 2004 29

UTF-32

• 32-Bit encoding

• Popular when memory space is no concern

• Fixed width (4Byte)



3/ 31/ 2004 30

UCS-4 / UTF-32 (32-bit encodings)

• All 32-bit encodings have a fixed length

• This advantage is outweighed by the

extensive memory & storage requirements



3/ 31/ 2004 31

Example #1

D834 DD1EN/AF0 9D 84 9E•

98759875E4 BA 75

06640664DA 64

00F600F6C3 B6Ö

00C600C6C3 86Æ

0063006363c

0041004141A

UTF-16UCS-2UTF-8Character



3/ 31/ 2004 32

Example #2 – character “ • ” U+AC00

00CAHEX

0000000011001010BINUTF-

16

00001100 00001010Regroup bits

0000000000111010

000010000000101110101110Remove lead

bytes

000010000000101110101110BIN

080BAEHEXUTF-

8

Lead By te Indicator Tra il ing By te Indicato r



3/ 31/ 2004 33

Unicode & SAP inGeneral



3/ 31/ 2004 34

2. Unicode & SAP in General

• Languages and characters• Characters on Disk/ Memory

• Code Pages

• SAP & Code Pages• Language Combinations before Unicode

• Recommendations from SAP (w/ o Unicode)

• Unicode-compliant SAP products

• W hen/ why do customers need Unicode?



3/ 31/ 2004 35

Language and characters

• Languages are written in fonts• Only a few languages use the same fonts

• A font is a group of characters



3/ 31/ 2004 36

Characters on Disk/ Memory

• A character is stored as a byte sequence on disk• a code page defines the mapping between the

byte sequence and a character




3/ 31/ 2004 37

Code Pages

• The code page determine what character you cansee and enter




3/ 31/ 2004 38

Code Pages

• different code pages map different characters tothe same byte sequence


Sing le By te Double Byte



3/ 31/ 2004 39

SAP & Code Pages

Language Combinations before



3/ 31/ 2004 40

Language Combinations beforeUnicode

• Single Standard Code Pages• supports specific sets of languages• the number and combination of languages that are supported

cannot be altered

• Standard code pages and R/ 3 languages (w/ o EBCDIC)

Double-Byte Code Pages




3/ 31/ 2004 41


• It is also possible to specify a customer-specific language; this language must useone of the code pages that SAP supports;see Note 0112065




3/ 31/ 2004 42


• Blend ed Cod e Pages (≥ Rel. 3 .1 D)• SAP proprietary code pages that contain characters

from one or more standard code pages

• increases the combinations of languages that can be

used

• functionally, a Blended Code Page system uses asingle code page

• a Blended Code Page is a single code page system• users can see and enter all characters contained in the

code page, regardless of their log-in language




3/ 31/ 2004 43


SAP Code Page Suppor ted Languages




3/ 31/ 2004 44


• the availability of SAP blended code pages isplatform dependent, because SAP blendedlocales need to be created for each platform

• Blended Locale Status (x = available −− = not available)






3/ 31/ 2004 46

Example

• Each user can only access one code page at atime: a user who logs in as a Japanese usercannot enter German characters, and all Germancharacters in the database will not be correctlydisplayed

1100 – ISO-1

8 0 00 - SJ I S


DBAppl ica t ion

Server

Front End

Japan

Germany




3/ 31/ 2004 47


Example

JapaneseUser

GermanUser




3/ 31/ 2004 48


Please Note:• It is possible for a user to log on with German and

then manipulate the character set and font settingsso that he can enter what appear to be Japanesecharacters; these characters will not be correctlystored in the database and th is da ta w i ll becorrupt

• If a user wants to enter f.i. Japanese, he/ she mustlog on in Japanese




3/ 31/ 2004 49


Please Note:• To insure that no data corruption occurs, the

following restrictions must be followed:

• Global data must contain only 7-bit ASCIIcharacters, which are in all code pages

• Users may use only the characters of their log-inlanguage or 7-bit ASCII

• Batch processes must be assigned with the

correct user ID and language• EBCDIC code pages are not supported

Recommendations from SAP



3/ 31/ 2004 50

Recommendations from SAP(w/ o Unicode)

• In general, using a single standard code page fornew installations and upgrades is the optimaldecision

• If additional languages or language combinationsare needed, SAP recommends UnambiguousBlended Code Pages for new installations andMDMP for existing installations

• Unambiguous Blended Code Pages only supportcertain language combinations and therefore an

MDMP setup may be the only possibility for newinstallations as well



3/ 31/ 2004 51

Unicode-compliant SAP products

• All Unicode installations are currently planned onlywith written permission of SAP carried out ascustomer projects together with SAP, except ofnew installations of R/ 3 Enterprise Extension Set2.0

Unicode compliant SAP products (SAP



3/ 31/ 2004 52

Unicode-compliant SAP products (SAPNote 79991)

ü

SAP Web A ppl icat ion Server (≥

6.20)

üm y SAP Custom er Relationship M anag ement(CRM)• The Unicode version of mySAP CRM 4.0 is available via Ramp-Up

üm y SAP Sup p ly Cha in M ana gement (SCM )• The Unicode version of mySAP SCM 4.0 is available via Ramp-Up

üm y SAP Sup p lier Relationship M ana gem ent (SRM )• The Unicode version of mySAP SRM 4.0 is available via Ramp-Up• conversions (with or without MDMP) of existing SRM installations

Unicode compliant SAP products (SAP



3/ 31/ 2004 53

Unicode-compliant SAP products (SAPNote 79991)

ü

mySAP Business Intelligence (BW)• The Unicode version of mySAP BW 3.5 is available via Ramp-Up• the conversion of existing BW installations as customer project• SAP Note 643813 has a collection of all relevant SAP notes

concerning Unicode-based SAP BW installations

üm y SAP Prod uct Lifecy cle Mana gement (PLM )• The Unicode version of mySAP PLM 4.0 is available via Ramp-Up

üSAP R/ 3 Enterpr ise (Ex t. 1.1 0 & hig her)

üSAP Exchange Infrastructure

W hen/ why do customers need



3/ 31/ 2004 54

yUnciode?

• Global businesses that require IT systems tosupport multilingual data without any restrictions

’ f.i. customers with one WW central SAP system

• W eb interfaces open the door to a globalcustomer base, and IT systems must consequentlybe able to support multiple local languagessimultaneously

W hen/ why do customers need



3/ 31/ 2004 55

yUnciode?

• W ith J2EE integration, mySAP components fullysupport web standards, and with Unicode, it nowcan take full advantage of XML and Java

• Only Unicode makes it possible to seamlesslyintegrate inhomogeneous SAP and non-SAP systemlandscapes’ NetWeaver



3/ 31/ 2004 56

Technology in Depth



3/ 31/ 2004 57

3. Technology in Depth

• Unicode & Operating Systems• Unicode & Databases

• SAP Unicode-based Code Pages

• How to Unicode-enable a program

• Unicode-enabled ABAP

• Migrating to Unicode enabled ABAP

• Unicode Conversion, IMIG Lab Test

• SAP System-to-System communication

• Printing & Output Management

Unicode & Operating Systems –



3/ 31/ 2004 58

HP-UX

• HP-UX is Unicode-enabled since version 10.x• All Unicode locales in the HP-UX operating

environment are based on the UTF-8 format

• Each locale includes a base language in the UTF-8

code set and the regional data related to this baselanguage

• This includes local formatting rules, text messages,help messages, and other related files

• Each locale also supports several other scripts forinput, display, code conversion, and printing

Unicode & Operating Systems -



3/ 31/ 2004 59

Windows

• Some Unicode support has been included inMicrosoft Win d o w s since Windows 95, andWindows NT 4

• W indows 2000 and W indows XP/ 2003 arebased on Unicode instead of the ANSI or W GL4character sets

• Before W in2K, your version of W i n d o w s mayhave used a different character set if you live in acountry such as Egypt, Greece, Israel, Russia or

Thailand that uses a non-Latin alphabet

Unicode & Operating Systems –



3/ 31/ 2004 60

Windows

• The first 128 characters were the same as inANSI, but many of the places in the second set of128 were taken by characters from the Arabic,Greek, Hebrew, Cyrillic or Thai alphabets

• This caused and still causes problems whenmoving documents between operating systemssuch as DOS, W indows, Mac OS and UNIX orexchanging documents electronically that were

created on computers using different character sets

Unicode & Operating Systems –Li



3/ 31/ 2004 61

Linux

• Before UTF-8 emerged, Linux users all over theworld had to use various different language-specific extensions of ASCII

• Most popular were ISO 8859-1 and ISO 8859-2in Europe, ISO 8859-7 in Greece, KOI-8 / ISO8859-5 / CP1251 in Russia, EUC and Shift-JIS inJapan, BIG5 in Taiwan, etc.

• This made the exchange of files difficult andapplication software had to worry about varioussmall differences between these encodings




3/ 31/ 2004 62

Linux

• Because of these difficulties, major Linuxdistributors and application developers have nowstarted to phase out these older legacy encodingsin favor of UTF-8

• UTF-8 support has improved dramatically over thelast few years and ever more people now use UTF-8 on a daily basis in• text files (source code, HTML files, email messages, etc.)

• file names• standard input and standard output, pipes• …




3/ 31/ 2004 63

Linux

• In UTF-8 mode, terminal emulators (such as xterm)transform every keystroke into the correspondingUTF-8 sequence and send it to the stdin of theforeground process

• Similarly, any output of a process on stdout is sentto the terminal emulator, where it is processed witha UTF-8 decoder and then displayed using a 16-

bit font




3/ 31/ 2004 64

Linux

• Before you start experimenting with UTF-8 underLinux, update your installation to a recentdistribution with up-to-date UTF-8 support

• This is particular the case if you use an installationolder than SuSE 8.1 or Red Hat 8.0

• Before these, UTF-8 support was far too limited

and experimental to be recommendable for dailyuse

Littl Bi E di



3/ 31/ 2004 65

Little vs. Big Endian

• UCS and Unicode are first of all just codetables that assign integer numbers tocharacters

• There exist several alternatives for how asequence of such characters or their respectiveinteger values can be represented as asequence of bytes

• The two most obvious encodings store Unicodetext as sequences of either 2 or 4 bytessequences

Little vs Big Endian



3/ 31/ 2004 66


• The official terms for these encodings are UCS-2 and UCS-4, respectively

• Unless otherwise specified, the most significantbyte comes first in these (Big Endian

convention)• An ASCII or Latin-1 file can be transformed into

a UCS-2 file by simply inserting a 0x00 byte infront of every ASCII byte

• If we want to have a UCS-4 file, we have toinsert three 0x00 bytes instead before everyASCII byte

Little vs Big Endian



3/ 31/ 2004 67


66 5353 66E6 99 93U+6653

05 D0D0 05D7 90U+05D0

03 B1B1 03CE B1U+03B1

00 C4C4 00C3 84U+00C4Ä

00 4141 0041U+0041A

[BigEndian]

[LittleEndian]

UTF-16UTF-16UTF-8 / CESU-8Unicode Scalar ValueCharacter

Unicode & Databases



3/ 31/ 2004 68

Unicode & Databases

P----PPPPSAP DB

P?PPPPPDB2

P----PPPPOracle

------------PSQL ServerLinuxOS/ 390OS/ 400AIXSolarisHP-UXWin2K

P Avai lable ? Currently not av ai lab le -- Unsupp orted in general

Supported Databases by SAP (WAS 6.20)

Unicode & Databases



3/ 31/ 2004 69

Unicode & Databases

UTF-88.0

UTF-167.0SAP DBUTF-16AS400

CESU-8AIXDB2

UTF-8 / UTF-1610g

UTF-8 / UTF-169i

UTF-88

UTF-87.2Oracle

UTF-162000SQL Server

EncodingsVersionManufacturer

SAP Unicode based Code Pages



3/ 31/ 2004 70

SAP Unicode-based Code Pages

• W ith the Unicode enablement of mySAP.comcomponents (check chapter #1), the old codepage management had to be changed

• Instead of using SAP character numbers allcode pages are now based on Unicode

cha racter Ids

• ’ 5 digit SAP Character numbers no longeradequate

This cha ng e is va lid for both Unicod eand N on -Unicod e Sy stems!




3/ 31/ 2004 71





3/ 31/ 2004 72

SAP Unicode-based Code Pages• Connection between SAP

character number &Unicode character ID isfound in table TCP01

• You can see the connectionin the SPAD charactersection

• NOTE: not every characterhas a correspondingUnicode character ID!

f.i.




3/ 31/ 2004 73

SAP Unicode based Code Pages

•

The migration of all SAP code pages from the oldto the new format was done using reportRSCP0126

• The definition of code pages is still in TCP00

Custom ers m ust m igr ate their ow n codepages (9x x x ) using RSCP0126

themselves!

How to Unicode-enable a program



3/ 31/ 2004 74

How to Unicode enable a program• Separate Unicode and Non-Unicode version of

R/ 3ABAP

source Non-Unicode

R/ 3

Unicod e R/ 3

• 1 char acter = 1 by te(types C, N, D, T, STRING)

• N on-Unicode k ernel

• N on-Unicode d atab ase

• 1 char acter = 2 by tes’ UTF-16

(types C, N, D, T, STRING)

• Unicode kernel

• Unicode da taba se

• No explicit Unicode data type in ABAP

• Single ABAP source for Unicode and non-Unicode systems




3/ 31/ 2004 75

How to Unicode enable a program

• Major part of ABAP coding is ready forUnicode without any changes

• Minor part of ABAP coding has to beadapted to comply with Unicode restrictions

(f.i. syntactical restrictions)




3/ 31/ 2004 76

How to Unicode enable a program• Program attribute

„Unicode checksactive“

Unicode Enabled ABAP



3/ 31/ 2004 77


Design Goals

• Platform independenceØIdentical behavior on Unicode and non-Unicode systems

• Highest level of compatibility to the pre-Unicode

worldØMinimize costs for Unicode enabling of ABAP Programs

Main Features

• Clear distinction between character and byte

processing1 Character <> 1 By te




3/ 31/ 2004 78


ABAP lists: Difference between memory and displaylength

Migrating to Unicode enabled ABAP



3/ 31/ 2004 79

g g

Step 1

• In non-Unicode system

• Adapt all ABAP programs to Unicode syntax and

runtime restrictions

• Set attribute "Unicode enabled" for all programs




3/ 31/ 2004 80

g g

Step 2

• Set up a Unicode system• Unicode kernel + Unicode database• Only ABAP programs with the Unicode attribute are executable

•

Do runtime tests in Unicode system

• Check for runtime errors

• Look for semantic errors

• Check ABAP list layout with former double byte characters




3/ 31/ 2004 81

g g

Use UCCHECK to analyze your applications:

• Remove errors

• Inspect statically not analyzable places (optional)• Untyped field symbols

• Offset with variable length• Generic access to database tables

• Set Unicode program attribute using UCCHECK orSE38 / SE24 / ...

• Do additional checks with SLIN (e.g. matching ofactual and formal parameters in function modules)




3/ 31/ 2004 82

g g




3/ 31/ 2004 83

g g

Upgrade to



pUnicode

Upgrade to Unicode



3/ 31/ 2004 85

• W ith Unicode, there are no limitations on users,and all languages in the ISO639 standard can beused

• Unicode is technically supported as of Basis

Release 6.20, see Note 0379940 for moreinformation

• A single code page system (standard or

Unambiguous Blended Code Page) can beupgraded to Unicode using the normal upgrademethod

Unicode Conversion Roadmap



3/ 31/ 2004 86

Preparat ion

• During preparation, topics such as

• additional hardware requirements,• downtime issues,

• Unicode-enabling of customer developments,• and the special treatment of MDMP systems

have to be taken into consideration




3/ 31/ 2004 87

Conversion

• The Unicode conversion process is based on asystem copy, and during this process, thedatabase conversion and system shutdown/ restartare as automated as possible

• For small to mid-size databases (< 1 TB), this isbased on an SAP Unload/ Reload of the completedatabase; minimum downtime tools will be usedfor larger databases.




3/ 31/ 2004 88

Post-Conversion

• Once the Unicode system is up and running, you need to• verify data consistency on a scenario basis,• as well as carry out general integration testing

• For systems that support multiple languages, specialemphasis needs to be placed on cross-language handlingduring the test phase.

• Correction tools are provided by SAP, which can be usedin the case that conversion did not run properly.




3/ 31/ 2004 89

Post-Conversion

• Additional Tool: SAP Data Management - reducing thedatabase size and growth

• To keep your database costs in check, the SAP DataManagement service frees up valuable database resourcesby showing you how to reduce the size and growth of yourdatabase by typically 25 % (see details).

Unicode Conversion at a Glance



3/ 31/ 2004 90

Preparation

Conversion

Post-Conversion

Set up the Unicode

Conversion Project

Check Prerequisites

Data Analysis fordowntime minimization –special MDMP treatment

Enabling of CustomerDevelopments

Highly automated

System will be downduring database

conversion

Unload /reload processfor small databases

Minimum downtime toolfor large databases

Unicode system is upand running

Verification ofData Consistency

Integration Testingfocused on

language handling

Upgrade Paths to Unicode (R/ 3Enterprise)S



3/ 31/ 2004 91

R/3 4.6c

Source system Target system

R/3 Enterprise

non-Unicode

R/3 Enterprise

Unicode

R/3 4.5b

R/3 3.1i

l First upgrade, then conversion to Unicode

l R/3 Enterprise Ramp-Up started 2002-07

l Unicode availability follows a phase of

restricted shipment with pilot customers

R/3 4.6b

R/3 4.0b Conversion

Directupgrade

Upgrade Paths to Unicode (BW 3.1)S t T



3/ 31/ 2004 92

BW 3.0


BW 3.1

non-Unicode

BW 3.1

Unicode

BW 2.1C

BW 2.0B

l Interfacing R/3 MDMP on a project base only

l Unicode BEXGUI restrictions apply


l BW 3.1 Ramp-Up starting 2002-12

l Unicode availability follows a phase ofrestricted shipment with pilot customers

Conversion

Upgrade Paths to Unicode (CRM3.1)

S t T



3/ 31/ 2004 93

CRM 3.0


CRM 3.1

non-Unicode

CRM 3.1

Unicode

l Selected scenarios onlyçècooperation with SAP GBU CRM required


l CRM 3.1 Ramp-Up starting 2002-12

l Unicode availability follows a phase ofrestricted shipment with pilot customers

CRM 2.0B

CRM 2.0C

Conversion




3/ 31/ 2004 94

Preparation

Conversion

Post-Conversion

Set up the Unicode

Conversion Project

Check Prerequisites



Highly automated


conversion






language handling

Prerequisites, special MDMPtreatment



3/ 31/ 2004 95

• OSS Note 548016Conversion from Unicode to non-Unicode is not possible

The Unicode Conversion of MDMP AND also AmbiguousCode page systems ( Code Page numbers 6100, 6200 and6500 ) is only supported on project basis with SAP

involvement

• OSS Note 543715The Unicode Conversion of a BW 3.1 system requiresadditional steps regarding the system copy

• OSS Note 573044If you are using HR functionality within R/ 3 Enterprise , alsoadditional steps are mandatory

6.30 Unicode & MCOD



3/ 31/ 2004 96

ABAP Stack (non Unicode/Unicode)

ABAP Stack (non Unicode/Unicode)

Java Stack (Unicode)

Java Stack (Unicode)

System QA1

System TC2

SAPQA1

SAPQA1DB

SAPTC2

SAPTC2DB

• W ith SAP W ebAS 6.30 a database abstraction layer for

the Java stack was introduced – OpenSQL for Java• Tables of the Java stack are stored in the same database

instance like the tables of the ABAP stack in two differentschema (except Informix)

• The concept of MCOD installations is fully supported by

the combined stack of ABAP and Java




3/ 31/ 2004 97

Preparation

Conversion

Post-Conversion

Set up the Unicode

Conversion Project

Check Prerequisites



Highly automated


conversion






language handling





3/ 31/ 2004 99

SAP System-to-SystemCommunication

SAP System-to-System communication



3/ 31/ 2004 100

• SAP W eb Application Server (≥ 6.20)

• Only one source code exists for Unicode-based and non-Unicode-based systems,’ new developments can be smoothly exchanged

• The interfaces (e.g. RFC) have been extended, so thatcommunication between other Unicode-based systems ornon-Unicode-based systems is possible. Furthermore,SAP provides standard tools for the installation of (andconversion to) Unicode-based systems that can also be

used for checking and Unicode-enabling of customerdevelopments

SAP System-to-System communicationl id li

Latin-1 SJIS



3/ 31/ 2004 101

• solid l ines:

receiver can receiveall characters

• do tted l ines:receiver cannotreceive characters,which are not in itsown code page. Butas long as you restrictthe character set, datacan be sent fromeverywhere toeverywhere.

Unicode R/ 3

W W W

http/ RFC

http/ RFC

SJIS

Latin-1

Non-UnicodeR/ 3SJIS

MDMP R/ 3


RFC



3/ 31/ 2004 102

RFC

• Unicod e <-> Unicod e• no problem

• non Unicod e <-> non Unicod e

• old stuff, receiver converts code page if possible

• Unicod e <-> non Unicod e• the Unicode side converts from/ to the code page of the

non Unicode side• MDMP is converted with a languages key• System settings allow the configuration of error handling




RFC (SM 59) U i d U i d



3/ 31/ 2004 104

RFC (SM 59) – Unicod e <–> no n Unicode

Printing & Output Management

W ha t is a SAP device type?



3/ 31/ 2004 105

W ha t is a SAP device type?• configuration file for the SAP printer driver that ensures

proper functionality between the SAP data stream and theprinter or output device where the data is sent

Prin ter d riv ers & dev ice ty pes• In R/ 3, a distinction is made between "printer driver" and

"device type“

• A device type consists of a variety of attributes defined foran output device

• One of these attributes is the printer driver to be used bySAPscript (R/ 3 forms processor) for this particular printer


device types cover aspects such as control commands



3/ 31/ 2004 106

• device types cover aspects such as control commands

for font selection, page size, character set selection,character set used and so on

• a device type must be specified to enable direct-printing from the SAP applications for every newprinter defined in SAP environment

• device types are created by SAP for the entire HPLaserJet printer family on the basis of PCL5, PCL6 andPostScript

• SAP develops, tests and supports device types for HP

products that can be found here:http:/ / h40045.www4.hp.com/ printing_solutions/ Device_Types.html


at present there are five SAPscript printer drivers



3/ 31/ 2004 107

• at present, there are five SAPscript printer drivers

They include:• HP-PCL5 (for example, HP Laserjet 3,4,5,6 series)• PostScript printers (PS level 2)• PRESCRIBE (for example, Kyocera FS-1500)• device types SW IN/ SAPW IN/ xxSW IN/ xxSAPW IN


Unicode Device Types



3/ 31/ 2004 108

Unicode Device Types• LEXMARK is going into HP accounts, claiming that only

LEXMARK could support SAP UNICODE printing.

Background:

• in order to support UNICODE character-sets on an HP

printer, customers need to have a UNICODE compliantprinter and a SAP UNICODE device-type

• UNICODE compliant printer are defined by firmwaresupport for UTF8 and/ or UTF16 and UNICODE fontsloaded on the printer

• today LEXMARK is the preferred vendor for SAP UNICODEprinting


Solution fo r HP



3/ 31/ 2004 109

Solution fo r HP• all OZ based printers (LJ2300 and higher) support by default

UNICODE UTF16 fonts in PCL6

• the LJ2300, CLJ9500 and future products will support UTF8 fontsin PCL5

• firmware role is planned to also support all current OZ based

printers (LJ4200/ 4300, LJ9000, CLJ4600, CLJ5500) to supportUTF-8 in PCL5

• furthermore the UNICODE fonts need to be loaded on the printer(e.g. stored on internal hard-disc)

• today we have a UNICODE-prototype-solution available to print

from an SAP environment• for more information, contact Alan Cooke (U.S.) or Stephen

Westberg (EMEA)



3/ 31/ 2004 110

Sizing Information forUnicode-based SAPSystems

Sizing Info - General

The space requirements for encoding a text



3/ 31/ 2004 111

The space requirements for encoding a text,compared to encodings currently in use (8bit per character for European languages,more for Chinese/ Japanese/ Korean), is asfollows ’ next Slide

This has an influence on disk storage spaceand network download speed (when no

form of compression is used)

Sizing Info - General

UTF-8



3/ 31/ 2004 112

UTF 8

No change for US ASCII, just a few percent more forISO-8859-1, 50% more for Chinese/ Japanese/ Korean,100% more for Greek and Cyrillic

UCS-2 and UTF-16

No change for Chinese/ Japanese/ Korean. 100% morefor US ASCII and ISO-8859-1, Greek and Cyrillic

UCS-4

100% more for Chinese/ Japanese/ Korean. 300% morefor US ASCII and ISO-8859-1, Greek and Cyrillic

Ex pected Hardw areRequirements

• Increase of CPU requirements



3/ 31/ 2004 113

• Increase of CPU requirementsØDepending on existing solution:

ISO-LATIN1 (ASCII) ð Unicode: +30%Double-Byte/ MDMP ð Unicode: + <5%

• Increase of memory requirementsØIncrease of memory requirements depending on

underlying DB (+ ~50%)ØApplication Server internally based on UTF-16; DB either

UTF-8, CESU-8 or UTF-16

Unicode Conversion Demo



3/ 31/ 2004 114

JAVA App let Dem o

Ex pected Hardw areRequirements• Database growth depending on

Ø DB Unicode encoding schema (e g CESU 8 UTF 16)



3/ 31/ 2004 115

Ø DB Unicode encoding schema (e.g. CESU-8, UTF-16)

Ø Languages in use

A

1100 8000 CESU-8 UTF-16

Ä

1100 8000 CESU-8 UTF-16

•

1100 8000 CESU-8 UTF-16

60-70%SQL Server, DB/ 2 (AS400), SAP DB(7.0)

UTF-16

35%Oracle, SAP DB (8.0)

DB/ 2 (AIX)

UTF-8CESU-8

Add it ional Storag eReq‘s

ManufacturersEncoding

1

B y t e

• Network load: (draft results)<7% for Latin-1, about 15% for Japanese, 25% for otherAsian languages


R/3 Release 4 0 4 5 4 6c 4 7 (6 20) non Unicode



3/ 31/ 2004 116

NON-Unicode

R/3 Release 4.0 4.5 4.6c 4.7 (6.20) non-Unicode

CPU 1 +20% +15% +5%

Memory 1 +20% DB: +20%; +5%App:+10%

Disk 1 +10% +10% +10%


R/3 Release 4 7 (6 20) non-Unicode 4 7 with Unicode



3/ 31/ 2004 117

Unicode

R/3 Release 4.7 (6.20) non-Unicode 4.7 with Unicode

CPU 1 +30% to 35%

Memory 1 +50%

Disk 1 +~35% (UTF-8)+60-70% (UTF-16)



Documents

Unicode in Netweaver