7313279 Sap Unicode

7/27/2019 7313279 Sap Unicode

1/82

Unicode Enabling of ABAP

7/27/2019 7313279 Sap Unicode

2/82

Contents

Unicode Overview What is Unicode

Need for Unicode

Unicode Character formats

Unicode @ SAP SAP Pre-Unicode solutions

Why SAP adopted Unicode

Impact of Unicode on ABAP

Overview Concepts and Conventions

Restrictions in Unicode

New ABAP Features

Tools for Unicode enabling

7/27/2019 7313279 Sap Unicode

3/82

What is Unicode?

Unicode Overview

7/27/2019 7313279 Sap Unicode

4/82

What Is Unicode?

Fundamentally, computers store letters and other characters by assigning anumber for each one.

Unicode provides a unique number (Code point) for every character, nomatter what the platform, no matter what the program, no matter what thelanguage.

Notation U+nnnn (where nnnn are hexadecimal digits)

Character encoding schema

for (nearly) all charactersused world wide

7/27/2019 7313279 Sap Unicode

5/82

What is Unicode? (contd.)

Unicode = universally encoded character set to store information from any

language

The Unicode standard primarily encodes scripts rather than languages

Scripts comprise several languages that historically share the same set of

symbols

In many cases a script may serve to write dozens of languages (e.g. the Latin

script)

In other cases one script complies to one language (e.g. Hangul)

Additionally it also includes punctuation marks, diacritics, mathematical

symbols, technical symbols, musical symbols, arrows, etc. In all, the Unicode Standard comprises >95.000 characters, ideograph sets,

symbols (version 4.0)

7/27/2019 7313279 Sap Unicode

6/82


Unicode Standard:

The Unicode Standard is a character coding system designed to support theworldwide interchange, processing and display of written text of the diverselanguages and technical disciplines of the modern world.

In addition, it supports classical and historical texts of many written

languages. The Unicode Consortium:

The Consortium cooperates with

W3C

ISO

and has liaison status "C" with ISO/IEC/

JTC1/SC2/WG2, which is responsible for in refining the specification andexpanding the character set of ISO/IEC 10646

7/27/2019 7313279 Sap Unicode

7/82


Unicode: The last character set?

It is an open character set, which means that it keeps growing and adding

less frequently used characters.

The standard assigns numbers from 0 to 0x10FFFF, which is more than a

million possible numbers for characters.

5% of this space is used, 5% is in preparation, about 13% is reserved

for private use, and about 2% is reserved not for use

The remaining 75% is open for future use but not by any means expected

to be filled up and finally there is a character set with plenty of space!

7/27/2019 7313279 Sap Unicode

8/82

Need for Unicode

Unicode Overview

7/27/2019 7313279 Sap Unicode

9/82

Need For Unicode

Hundreds of encodings have been developed, each for small groups oflanguages and special purposes.

There is no single, authoritative source of precise definitions of many of theencodings and their names.

No single encoding could contain enough characters: for example, theEuropean Union alone requires several different encodings to cover all itslanguages.

Even for a single language like English no single encoding was adequate forall the letters, punctuation, and technical symbols in common use.

Incompatibilities between different code pages

7/27/2019 7313279 Sap Unicode

10/82

Need For Unicode (contd.)

These encoding systems conflict with one another. That is, two encodingscan use the same number for two differentcharacters, or use differentnumbers for thesame character.

Any given computer (especially servers) needs to support many differentencodings; yet whenever data is passed between different encodings orplatforms, that data always runs the risk of corruption.

Programs are written to either handle one single encoding at a time and

switch between them, or to convert between external and internal encodings

7/27/2019 7313279 Sap Unicode

11/82

Need For Unicode (contd.)

Languages & Code Pages

7/27/2019 7313279 Sap Unicode

12/82

Unicode, One Code Page For All Scripts

7/27/2019 7313279 Sap Unicode

13/82

Unicode Character Formats

Unicode Overview

7/27/2019 7313279 Sap Unicode

14/82

Unicode Character Formats

Representation of Unicode characters:

UTF-16: Unicode Transformation Format, 16 bit Encoding- Fixed Length, 1character = 2 bytes

- Platform dependent BYTE order- 2 BYTE alignment restriction

UTF-8: Unicode Transformation Format, 8 bit Encoding

- Variable length, 1character = 1 to 4 BYTES

- Platform Independent

- No alignment restriction

- 7 BIT US ASCII compatible

7/27/2019 7313279 Sap Unicode

15/82

Unicode Character Formats (contd.)

UTF-32: Unicode Transformation Format, 32 bit Encoding

For single characters, 32-bit integer variables are most appropriate for thevalue range of Unicode.

For strings, however, storing 32 bits for each character takes up too muchspace, especially considering that the highest value 0x10FFFF, takes up only21 bits. 11 bits are always unused in a 32-bit word storing a Unicode codepoint.

Therefore, you will find that software generally uses 16-bit or 8-bit units as a

compromise, with a variable number of code units per Unicode code point.

7/27/2019 7313279 Sap Unicode

16/82

Unicode Character Formats (contd.)

7/27/2019 7313279 Sap Unicode

17/82

SAP Pre-Unicode solutions

Unicode @ SAP

7/27/2019 7313279 Sap Unicode

18/82

SAP Pre-Unicode Solutions

Single Code Page System

System using one standard code page which can support a specific setof languages.

Blended Code Page System (Release 3.0D)

Multi byte blended code pages, which contain characters out of severalstandard code pages. Blended code pages are not standard code pages, butSAP-customized code pages that were devised to support an increasednumber of possible language combinations in a single

code page.

a) Ambiguous Blended Code Page System: Two characters can share

the same code point.b) Unambiguous Blended Code Page System: Each code point refersexactly to one character.

7/27/2019 7313279 Sap Unicode

19/82

SAP Pre-Unicode Solutions (contd.)

MDMP System Configuration (Release 3.1I)

Multiple Display/Multiple Processing.

System using more than one system code page on the applicationserver.

Allows languages to be used together in one system although the

characters of those languages are not in the same code page.

7/27/2019 7313279 Sap Unicode

20/82


Language Combinations Before Unicode

Each user can only access one code page at a time: a user who logs in as aJapanese user cannot enter German characters ,and all German characters inthe database will not be correctly displayed.

7/27/2019 7313279 Sap Unicode

21/82


It is possible for a user to log on with German and then manipulate the characterset and font settings so that he can enter what appear to be Japanese characters;these characters will not be correctly stored in the database and this data will becorrupt

If a user wants to enter f.i. Japanese, he/she must log on in Japanese

To insure that no data corruption occurs, the following restrictions must befollowed:

Global data must contain only 7-bit ASCII characters, which are in allcode pages

Users may use only the characters of their log-in language or 7-bit ASCII

Batch processes must be assigned with the correct user ID and languageEBCDIC code pages are not supported

7/27/2019 7313279 Sap Unicode

22/82

View in Different Code Pages

7/27/2019 7313279 Sap Unicode

23/82

Recommendations From SAP (Pre-Unicode)

In general, using a single standard code page for new installations and

upgrades is the optimal decision

If additional languages or language combinations are needed, SAPrecommends Unambiguous Blended Code Pages for new installations and

MDMP for existing installations.

Unambiguous Blended Code Pages only support certain language

combinations and therefore an MDMP setup may be the only possibility for

new installations as well.

7/27/2019 7313279 Sap Unicode

24/82

Why SAP adopted Unicode?

Unicode @ SAP

7/27/2019 7313279 Sap Unicode

25/82

Why SAP adopted Unicode?

Globalization = Internationalization + Localization

The Unicode Standard has already been adopted by industry leaders as

Apple, HP, IBM, JustSystem, Microsoft, Oracle, Sun, Sybase, Unisys and

many others.

Unicode is required by modern standards such as XML, Java, ECMAScript

(JavaScript), LDAP, CORBA 3.0, WML, etc.,

It is the official way to implement ISO/IEC 10646.

7/27/2019 7313279 Sap Unicode

26/82

Why SAP adopted Unicode? (contd.)

Allows text data from different languages to be stored in one repository

Enable a single set of source code to be written to process data in virtually alllanguages

Simplifies addition of new language support to an e-business applicationsince character processing and storage remains unchanged

Lowers cost of implementation

Faster speed to market

Better customer satisfaction

7/27/2019 7313279 Sap Unicode

27/82

Why SAP adopted Unicode? (contd.)

7/27/2019 7313279 Sap Unicode

28/82

View In UNICODE System

7/27/2019 7313279 Sap Unicode

29/82

Unicode - SAP

7/27/2019 7313279 Sap Unicode

30/82

Platforms supported by SAP for Unicode systems

The following indicates the current development status and availability

different OS and Database combinations for Unicode-based mySAP technical

components:

7/27/2019 7313279 Sap Unicode

31/82

Overview

Unicode & ABAP

7/27/2019 7313279 Sap Unicode

32/82

Overview

Non-Unicode Versions of SAP :- Versions prior to 4.7

Unicode Versions of SAP :- Version 4.7 & above

Each character mapped using 16 bits (= 2 bytes) which offers a maximum of 216

bit combinations

Affects any older ABAP program in which an explicit or implicit assumption is

made about the internal length of a character

7/27/2019 7313279 Sap Unicode

33/82

Overview (contd.)

Character Expansion Model

- Separate Unicode & Non-Unicode versions of R/3

- No explicit Unicode Data type in ABAP

- Single ABAP source for Unicode & Non-Unicode systems

7/27/2019 7313279 Sap Unicode

34/82

OVERVIEW (contd.)

Program attributes Unicode checks

active

- Required to run on a Unicode System

If attribute is set additional

restrictions:

- Apply at run time and compile time

- Apply in Unicode as well as Non-

Unicode System.

- Ensure that program will run on both

US & NUS with identical behavior

Non-Unicode

System

Unicode

System

Attribute Set

(Unicodeenabled)

Ok Ok

Attribute not

Set (not

Unicode

enabled)

Ok Not allowed

7/27/2019 7313279 Sap Unicode

35/82

Overview (contd.)

Screen Shot of Program Attribute Screen

7/27/2019 7313279 Sap Unicode

36/82

Concepts & Conventions

Unicode & ABAP

7/27/2019 7313279 Sap Unicode

37/82

Concepts & Conventions

Data Types

Data Layout of Structures

Unicode Fragment Views

7/27/2019 7313279 Sap Unicode

38/82

Data Types

The following data types can be interpreted as Character Type in a

Unicode program

C: Character

N: Numeric character

D: Date

T: Time

STRING: String

Character-type structures:Structures which either directly or in

substructures contain only fields of types C, N, D or T.

Byte Type : X, Xstring (for bit operations)

7/27/2019 7313279 Sap Unicode

39/82

Data Layout of Structures

For several data types (like F and I) the memory address should start with

multiples of 4 or 8 and for Character-type it should start with multiples 2 or 4

depending on the Unicode representation.

Within structures, bytes are inserted before or after components withalignment requirements to achieve the necessary alignment.

Examples:

BEGIN OF struc1,

a(1) TYPE X,

b(1) TYPE X,

c(6) TYPE C,

END OF struc1.

For struc1 there is no alignment gap for NUS or US.

7/27/2019 7313279 Sap Unicode

40/82

Data layout of structures (Contd..)

Examples:

BEGIN OF struc2,

a(1) TYPE X,

BEGIN OF struc3,

b(1) TYPE X,

c(6) TYPE C,

END OF struc3,

d TYPE I,

END OF struc2.

7/27/2019 7313279 Sap Unicode

41/82

Unicode Fragment View

BEGIN OF struc, Unicode Fragment Views:

a(2) TYPE C, F1, F2, F3, F4,F5, F6

b(4) TYPE N,

c TYPE D,d TYPE T,

e TYPE F,f(2) TYPE X,g(4) TYPE X,h(8) TYPE C,i(8) TYPE C,

END OF struc.

7/27/2019 7313279 Sap Unicode

42/82

Restrictions in Unicode

Unicode & ABAP

7/27/2019 7313279 Sap Unicode

43/82

Overview

Access Using Offset & Length Addressing Assignments

Casting Data Objects

Processing Strings in UNICODE Determining Length & Distance

Specifying Key for Table Access

Database Operations

Structure Enhancement Other changes

7/27/2019 7313279 Sap Unicode

44/82

Access Using Offset & Length Addressing

Accessing Single fields:

- Offset or Length based access supported for single fields ofCharacter

Type, strings, X, Xstrings

Accessing Structure:

- This access type results in errors if both character type and non-character type fields are present in the area addressed by the offset and the

length

Passing parameters to subroutines:

- Passing parameters using perform with offset and length specification

beyond field boundaries is not allowed in Unicode programming

7/27/2019 7313279 Sap Unicode

45/82

Access Using Offset & Length Addressing (contd.)

Accessing Field Symbols:

- Offset- or length-based access with ASSIGN is only permitted within a

predefined range

-ASSIGN feld [+off(len)]TO . : Range corresponds to the field

boundaries in case of elementary fields or, in case of flat structures, to the

purely character-type starting fragment

-ASSIGN [+off(len)] TO . : Range of the target FS = Range of the

Source FS

7/27/2019 7313279 Sap Unicode

46/82

Assignments

Conversion between Flat Structures (MOVE):

- The fragments of both structures up to the second-last fragment of the

shorter structure are identical

- The last fragment of the shorter structure must be ofcharacter orbyte-

type group

- The corresponding fragment of the longer structure is a character or

byte type group with a greater length

7/27/2019 7313279 Sap Unicode

47/82

Assignments (contd.)

Conversion between internal tables

- Tables can be converted if theirrow types are convertible. The restrictionsdescribed for structures also apply for the conversion of tables.

Implicit Conversions:

- The same rules also apply to all ABAP statements that use implicit

conversions according to the MOVE semantics.APPEND wa TO itab.

INSERT wa INTO itab.

MODIFY itab FROM wa.

READ TABLE itab ...INTO wa.

LOOP AT itab INTO wa.

7/27/2019 7313279 Sap Unicode

48/82

Processing Strings

Statements for processing strings:CLEAR ... WITHCONCATENATECONDENSECONVERT TEXT ... INTO SORTABLE CODEFINDOVERLAYREPLACE

SEARCHSHIFTSPLITTRANSLATE ... TO UPPER/LOWER CASETRANSLATE ... USING

FROM CODEPAGEandFROM NUMBER FORMATare not allowed with TRANSLATE

The arguments must be single fields of type C, N, D, T, STRING or purelycharacter-type structures

CONCATENATE a x b INTO cstatement is not possible ifa, b, and c are character-type but x has type X.

7/27/2019 7313279 Sap Unicode

49/82

Processing Strings (contd.)

Comparison operators for processing byte and character strings:

- Operators require single fields of type C, N, D, T, STRING or purelycharacter-type structures as arguments

CO BYTE-COCN BYTE-CNCA BYTE-CA

NA BYTE-NA

CS BYTE-CS

NS BYTE-NSCP BYTE-CP

NP BYTE-NP

7/27/2019 7313279 Sap Unicode

50/82


Functions for processing byte and character strings:

STRLENonly for character-type fields and returns the length in characters

XSTRLENfor finding length of byte strings

NUMOFCHAR - returns the number of characters in a character-type field

P i S i ( d )

7/27/2019 7313279 Sap Unicode

51/82


Output in fields and lists:

If the source field is a flat structure in a WRITEstatement, it must have

character type only, in a Unicode system. This affects the following

statements:

WRITEf.

WRITEfTO g[+off][(len)]..

WRITE(name) TO g..

WRITEfTO itab[+off][(len)]INDEXidx..

WRITE(name) TO itab[+off][(len)]INDEXidx..

E t bli hi L th & Di t

7/27/2019 7313279 Sap Unicode

52/82

Establishing Length & Distance

NUSDESCRIBE DISTANCE BETWEEN..US - Addition ofIN BYTE MODE / IN CHARACTER MODE

NUSDESCRIBE FIELD.. LENGTH

US - Addition ofIN BYTE MODE / IN CHARACTER MODE

ABAP Fil I t f

7/27/2019 7313279 Sap Unicode

53/82

ABAP File Interface

Opening Files:

The OPEN DATASETdsn ... statement must include at least one of the

following additions:

...IN TEXT MODE ENCODING ... (UTF-8)

... IN BINARY MODE ... (UTF-8 )

... IN LEGACY TEXT MODE ...(Non Unicode Format)

... IN LEGACY BINARY MODE ... (Non Unicode Format)

ABAP Fil I t f ( td )

7/27/2019 7313279 Sap Unicode

54/82

ABAP File Interface (contd.)

Reading & Writing Files:

READ DATASETdsn INTO f : For Reading File

TRANSFER fTO dsn : For Writing into File

If the file is opened in TEXT MODE, fmust be a character-type field ( C, N, D,

orT), a string, or a purely character-type structure

If the file is opened inLEGACY TEXT MODE, orLEGACY BINARY MODE,

conversion errors may occur

Other Changes relating to Unicode

7/27/2019 7313279 Sap Unicode

55/82

Other Changes relating to Unicode

Bit Statements:

SET BITi OFf [TO g].

GET BITi OFf [INTO g].

fO x, fZx, and fMx .

The field f must be either X or XSTRING in all the above bit operations.

Other Changes relating to Unicode (contd )

7/27/2019 7313279 Sap Unicode

56/82

Other Changes relating to Unicode (contd.)

ADDn1THENn2UNTILnz [ACCORDING TOsel ] GIVING m [RANGEstr].

ADDn1THENn2UNTILnz TOm [RANGEstr].

Operands n1, n2, and nz must be of the same type and length.

The distance between nz and n1 must be an integral multiple of the distancebetween n2 and n1.

The fields n1, n2, and nz must be in one structure. If this is not statically

recognizable, you must use the RANGE str addition to explicitly specify a

structure as a valid area.

If it is discovered that the addressed fields are not within the structure

specified using the RANGE addition, a syntax or runtime error occurs.


7/27/2019 7313279 Sap Unicode

57/82


Loops with VARY & varying Additions:

DO ... VARYINGfFROMf1NEXTf2 [RANGEf3 ].

WHILE... VARYfFROMf1NEXTf2 [RANGEf3 ].

The fields f, f1, and f2 must be type-compatible with one another RANGE for valid accesses must be implicitly or explicitly implemented


7/27/2019 7313279 Sap Unicode

58/82


Generating Subroutine:

GENERATE SUBROUTINE POOL itab NAMEname

The generated program inherits the contents of the Unicode flag of the

generating program.

Saving Programs:

INSERT REPORTprogFROMitab.

This statement now includes a new addition, UNICODE ENABLING uc, with

which the Unicode flag of the inserted report is given the value of uc


7/27/2019 7313279 Sap Unicode

59/82


Types and GET/SET PARAMETER:

GET PARAMETER ID pid FIELD f

SET PARAMETER IDpidFIELD f

The field f must be of character type. For non-character type we useIMPORT

&EXPORTstatements.

Specifying the Key for Tables Accesses

7/27/2019 7313279 Sap Unicode

60/82

A syntax orruntime error occurs when you access the database with a generic keyunless the key is purely character-type. This affects the following commands:

READ TABLEdbtab ...SEARCHGKEQ ...

READ TABLEdbtab ...SEARCHGKGE ...

LOOP ATdbtab ...

REFRESHitabFROM TABLEdbtab.

Please note that this statement is Obsolete and should no longer be used.

Database Operations

7/27/2019 7313279 Sap Unicode

61/82

Database Operations

SELECT* FROMdbtab ... INTO wa/ INTO TABLEitab ...FETCH NEXT CURSOR c ... INTO wa / INTO TABLEitab.

INSERT INTO dbtab ... FROMwa / FROM TABLEitab.

UPDATEdbtab ... FROMwa/ FROM TABLEitab.

MODIFYdbtab ... FROMwa / FROM TABLEitab.

DELETEdbtabFROMwa / FROM TABLEitab.

The fragment views of the work area and the database table must beidentical with regard to the length of the database table.

If the work area is a single field, the field must be character-type and thedatabase table purely character-type

Storing Data Clusters in Database Tables

7/27/2019 7313279 Sap Unicode

62/82

Data clusters are not converted when they are migrated from a non-

Unicode database to a Unicode system. For this reason, there may be

ABAP cluster tables in a Unicode system that contain non-Unicode

characters. These characters are automatically converted during eachimport.

When the data is exported, any Unicode characters that may be contained in

the data objects are stored in a platform-specific format in the Unicode

system.

Structure Enhancements

7/27/2019 7313279 Sap Unicode

63/82

Structure Enhancements

Problems caused by structure enhancements:

Enhancements change the fragment views and hence affect the checks for

assignments and comparisons

Enhancement classification in ABAP Dictionary:

Structure Enhancements (contd.)

7/27/2019 7313279 Sap Unicode

64/82

( )

Enhancement handling in program check:

Unicode & RFC

7/27/2019 7313279 Sap Unicode

65/82

Unicode & ABAP

7/27/2019 7313279 Sap Unicode

66/82

New ABAP Features

Assignments to Field Symbols

7/27/2019 7313279 Sap Unicode

67/82

g y

Range:

ASSIGN feld1 TO RANGEfeld2.

Sets the range limits, making it possible to define addresses past field limits

Increment:

ASSIGNfield INCREMENTn TO First the range for the access is defined from the length of field and the

INCREMENTdefinition of the range for the access as defined byASSIGN

fld+n*sizeof[field] (sizeof[fld]) TO .

If sy-subrc > 0 then no increment is made to the field symbol

Assignments to Field Symbols (contd.)

7/27/2019 7313279 Sap Unicode

68/82

Casting:

ASSIGNfield TO CASTING.

ASSIGNfield TO CASTING TYPEtype.

ASSIGNfieldTO CASTING TYPE (typename).

ASSIGNfieldTO CASTING LIKEfld.

ASSIGNfieldTO CASTING DECIMALSdec.Provides different views on a structure with casts on different types.

Treats the contents of a field as a value of another type using a field symbol

field must be at least as long as the type that was assigned to the field

symbol,

If the field symbol type is a deep structure the system checks for the offsetand reference component match of the area covered by

Includes with Group Names

7/27/2019 7313279 Sap Unicode

69/82

Creating Data Objects Dynamically

7/27/2019 7313279 Sap Unicode

70/82

Data Objects:

Creating and Accessing data objects on the heap

Creating Data Objects Dynamically (contd.)

7/27/2019 7313279 Sap Unicode

72/82

Using fields of type XSTRING as data containers

Writing data to an XSTRING (EXPORT):

Storing Data Clusters (contd.)

7/27/2019 7313279 Sap Unicode

73/82

Reading Data from an XSTRING (IMPORT):

Automatic Conversion of data during IMPORT.

Generic Types for Field Symbols & Parameters

7/27/2019 7313279 Sap Unicode

74/82

New Classes For UNICODE

7/27/2019 7313279 Sap Unicode

75/82

Character Utilities:

Class CL_ABAP_CHAR_UTILITIES

New Classes For UNICODE (CONTD.)

7/27/2019 7313279 Sap Unicode

76/82

Conversion Classes:

Unicode & ABAP

7/27/2019 7313279 Sap Unicode

77/82

Tools for Unicode Enabling

ABAP Unicode Scan Tool UCCHECK

7/27/2019 7313279 Sap Unicode

78/82

Transaction UCCHECK is used to examine a Unicode program set for

syntax errors without having to set the program attribute "Unicode checks

active" for every individual program.

From the list of Unicode syntax errors, one can go directly to the affected

programs and remove the errors.

It is also possible to automatically create transport requests and set the

Unicode program attribute for a program set.

Transaction UCCHECK

7/27/2019 7313279 Sap Unicode

79/82

ABAP Coverage Analyzer

7/27/2019 7313279 Sap Unicode

80/82

Coverage Analyzer (transaction SCOV):

Persistently traces the execution of all program objects within one system.

Traces all processing blocks i.e. forms, methods, modules and ABAP events.

Collects information

number of calls

number of runtime errors

number of program changes

Transaction SCOV

7/27/2019 7313279 Sap Unicode

81/82

7/27/2019 7313279 Sap Unicode

82/82

THANK YOU

Documents

7313279 Sap Unicode