Xi31211 Ds Migration Considerations En

8/2/2019 Xi31211 Ds Migration Considerations En

1/226

Data Services Migration

Considerations Guide

BusinessObjects Data Services XI 3.1 (12.1.1)


2/226

2008 Business Objects, an SAP company. All rights reserved. Business Objectsowns the following U.S. patents, which may cover products that are offered and

Copyright

licensed by Business Objects: 5,295,243; 5,339,390; 5,555,403; 5,590,250;5,619,632; 5,632,009; 5,857,205; 5,880,742; 5,883,635; 6,085,202; 6,108,698;6,247,008; 6,289,352; 6,300,957; 6,377,259; 6,490,593; 6,578,027; 6,581,068;

6,628,312; 6,654,761; 6,768,986; 6,772,409; 6,831,668; 6,882,998; 6,892,189;6,901,555; 7,089,238; 7,107,266; 7,139,766; 7,178,099; 7,181,435; 7,181,440;7,194,465; 7,222,130; 7,299,419; 7,320,122 and 7,356,779. Business Objects andits logos, BusinessObjects, Business Objects Crystal Vision, Business ProcessOn Demand, BusinessQuery, Cartesis, Crystal Analysis, Crystal Applications,Crystal Decisions, Crystal Enterprise, Crystal Insider, Crystal Reports, CrystalVision, Desktop Intelligence, Inxight and its logos , LinguistX, Star Tree, TableLens, ThingFinder, Timewall, Let There Be Light, Metify, NSite, Rapid Marts,RapidMarts, the Spectrum Design, Web Intelligence, Workmail and Xcelsius aretrademarks or registered trademarks in the United States and/or other countriesof Business Objects and/or affiliated companies. SAP is the trademark or registeredtrademark of SAP AG in Germany and in several other countries. All other names

mentioned herein may be trademarks of their respective owners.

Business Objects products in this release may contain redistributions of softwarelicensed from third-party contributors. Some of these individual components may

Third-party

Contributors

also be available under alternative licenses. A partial listing of third-partycontributors that have requested or permitted acknowledgments, as well as requirednotices, can be found at: http://www.businessobjects.com/thirdparty

2008-11-28


3/226

ContentsIntroduction 9Chapter 1

Welcome to Data Services........................................................................10

Welcome..............................................................................................10

Documentation set for Data Services...................................................10

Accessing documentation....................................................................13

Business Objects information resources..............................................14

Data Services Migration Considerations 17Chapter 2

Behavior changes in version 12.1.1..........................................................18

Blob data type enhancements..............................................................19

Neoview bulk loading...........................................................................20


Cleansing package changes................................................................21

DTD-to-XSD conversion.......................................................................21

Minimum requirements for international addressing directories...........22Try/catch exception groups..................................................................22

Upgrading from version 12.0.0 to version 12.1.0.................................25


Case transform enhancement..............................................................26

Data Quality projects in Data Integrator jobs ......................................27

Data Services web address..................................................................27

Large object data type enhancements.................................................28

License keycodes.................................................................................31

Locale selection....................................................................................31

ODBC bigint data type..........................................................................33

Persistent and pageable cache enhancements...................................33

Data Services Migration Considerations Guide 3


4/226

Row delimiter for flat files.....................................................................33

Behavior changes in version 11.7.3...........................................................34

Data flow cache type............................................................................35

Job Server enhancement.....................................................................35

Logs in the Designer............................................................................35

Pageable cache for memory-intensive data flows................................35

Adapter SDK........................................................................................36

PeopleSoft 8.........................................................................................36


Embedded data flows...........................................................................37

Oracle Repository upgrade..................................................................37Solaris and AIX platforms.....................................................................39


Data Quality..........................................................................................40

Distributed data flows...........................................................................42

JMS Adapter interface..........................................................................43

XML Schema enhancement.................................................................43

Password management........................................................................43

Repository size.....................................................................................44

Web applications..................................................................................44

Web services........................................................................................44


Netezza bulk loading............................................................................45

Conversion between different data types.............................................46

Behavior changes in version 11.5.1.5........................................................46



Web Services Adapter..........................................................................47

Varchar behavior..................................................................................47

Central Repository................................................................................48


4 Data Services Migration Considerations Guide

Contents


5/226

Teradata named pipe support...............................................................48



Statistics repository tables....................................................................49


Crystal Enterprise adapters..................................................................50


Changes to code page names.............................................................51

Data Cleansing.....................................................................................52

License files and remote access software...........................................53

Behavior changes in version 6.5.1............................................................53Behavior changes in version 6.5.0.1.........................................................54

Web services support...........................................................................54

Sybase bulk loader library on UNIX.....................................................55

Behavior changes in version 6.5.0.0.........................................................55

Browsers must support applets and have Java enabled......................55

Execution of to_date and to_char functions.........................................56

Changes to Designer licensing............................................................57

License files and remote access software...........................................57

Administrator Repository Login............................................................58

Administrator Users..............................................................................59

Data Quality to Data Services Migration 61Chapter 3

Overview of migration................................................................................62

Who should migrate?...........................................................................62

Why migrate?.......................................................................................62

Introduction to the interface..................................................................64

Downloading blueprints and other content objects..............................66Introduction to the migration utility........................................................67

Terminology in Data Quality and Data Services...................................67

Naming conventions.............................................................................69


Contents


6/226

Deprecated objects..............................................................................70

Premigration checklist..........................................................................72

Using the migration tool.............................................................................73

Overview of the migration utility...........................................................73

Migration checklist................................................................................74

Connection information........................................................................75

Running the dqmigration utility ............................................................76

dqmigration utility syntax and options..................................................79

Migration report ...................................................................................83

How Data Quality repository contents migrate..........................................85

How projects and folders migrate.........................................................85How connections migrate.....................................................................91

How substitution files and variables migrate........................................97

How data types migrate......................................................................102

How Data Quality attributes migrate..................................................103

How transforms migrate...........................................................................103

Overview of migrated transforms.......................................................103

Address cleansing transforms............................................................111

Reader and Writer transforms............................................................121

How Data Quality integrated batch Readers and Writers migrate.....159

How Data Quality transactional Readers and Writers migrate...........165

Matching transforms...........................................................................169

UDT-based transforms.......................................................................178

Other transforms................................................................................188

Suggestion Lists options....................................................................202

Post-migration tasks................................................................................203

Further cleanup .................................................................................203

Improving performance .....................................................................210

Troubleshooting..................................................................................215


Contents


7/226

Index 223


Contents


8/226


Contents


9/226

Introduction

1


10/226

This document contains the following migration topics:

Migration considerations for behavior changes associated with eachversion of the Data Integrator and Data Services products.

Migration of your Data Quality Projects into Data Services.

Welcome to Data Services

Welcome

Data Services XI Release 3 provides data integration and data qualityprocesses in one runtime environment, delivering enterprise performance

and scalability.

The data integration processes of Data Services allow organizations to easilyexplore, extract, transform, and deliver any type of data anywhere acrossthe enterprise.

The data quality processes of Data Services allow organizations to easilystandardize, cleanse, andconsolidatedata anywhere, ensuring that end-usersare always working with information that's readily available, accurate, andtrusted.

Documentation set for Data Services

You should become familiar with all the pieces of documentation that relateto your Data Services product.

What this document providesDocument

Information about available Data Services books,languages, and locations

Documentation Map

Highlights of key features in this Data Services re-lease

Release Summary

Important information you need before installing anddeploying this version of Data Services

Release Notes


Introduction1 Welcome to Data Services


11/226


An introduction to Data ServicesGetting Started Guide

Information about and procedures for installing DataServices in a Windows environment.

Installation Guide for Windows

Information about and procedures for installing DataServices in a UNIX environment.

Installation Guide for UNIX

Guidelines and options for migrating applications in-cluding information on multi-user functionality andthe use of the central repository for version control

Advanced Development Guide

Information about how to use Data Services DesignerDesigner Guide

Information for third-party developers to access DataServices functionality. Also provides information abouthow to install, configure, and use the Data Services

Adapter for JMS.

Integrator's Guide

Information about how to use Data Services Adminis-trator

Management Console: Administrator

Guide

Information about how to use Data Services MetadataReports

Management Console: Metadata Re-

ports Guide

Information about: Release-specific product behavior changes from

earlier versions of Data Services to the latest re-lease

How to migrate from Data Quality to Data Services

Migration Considerations Guide

Information about how to improve the performanceof Data Services

Performance Optimization Guide

Detailed reference material for Data Services Design-erReference Guide


1Introduction



12/226


A compiled master PDF of core Data Services books

containing a searchable master table of contents andindex: Getting Started Guide

Installation Guide for Windows

Installation Guide for UNIX

Designer Guide

Reference Guide

Management Console: Metadata Reports Guide

Management Console: Administrator Guide

Performance Optimization Guide

Advanced Development Guide

Supplement for J.D. Edwards

Supplement for Oracle Applications

Supplement for PeopleSoft

Supplement for Siebel

Supplement for SAP

Technical Manuals

A step-by-step introduction to using Data ServicesTutorial

In addition, you may need to refer to several Adapter Guides andSupplemental Guides.


Information about how to install, configure, and use the DataServices Salesforce.com Adapter Interface

Salesforce.com Adapter

Interface

Information about license-controlled interfaces between DataServices and J.D. Edwards World and J.D. Edwards OneWorld

Supplement for J.D. Ed-

wards

Information about the license-controlled interface between DataServices and Oracle Applications

Supplement for Oracle Ap-plications

Information about license-controlled interfaces between DataServices and PeopleSoft

Supplement for PeopleSoft




13/226


Information about license-controlled interfaces between Data

Services, SAP ERP, and SAP BI/BW

Supplement for SAP

Information about the license-controlled interface between DataServices and Siebel

Supplement for Siebel

Accessing documentation

You can access the complete documentation set for Data Services in severalplaces.

Accessing documentation on Windows

After you install Data Services, you can access the documentation from theStart menu.

1. Choose Start > Programs > BusinessObjects XI 3.1 >BusinessObjects Data Services > Data Services Documentation.

Note:Only a subset of the documentation is available from the Start menu. Thedocumentation set for this release is available in LINK_DIR\Doc\Books\en.

2. Click the appropriate shortcut for the document that you want to view.

Accessing documentation on UNIX

After you install Data Services, you can access the online documentation bygoing to the directory where the printable PDF files were installed.

1. Go to LINK_DIR/doc/book/en/.

2. Using Adobe Reader, open the PDF file of the document that you wantto view.


1Introduction



14/226

Accessing documentation from the Web

You can access the complete documentation set for Data Services from theBusiness Objects Customer Support site.

1. Go to http://help.sap.com.

2. Cick Business Objects at the top of the page.

You can view the PDFs online or save them to your computer.

Business Objects information resources

A global network of Business Objects technology experts provides customersupport, education, and consulting to ensure maximum business intelligencebenefit to your business.

Useful addresses at a glance:

ContentAddress




15/226

ContentAddress

Information about Customer Support programs,as well as links to technical articles, downloads,

and online forums. Consulting services canprovide you with information about how Busi-ness Objects can help maximize your businessintelligence investment. Education services canprovide information about training options andmodules. From traditional classroom learningto targeted e-learning seminars, Business Ob-

jects can offer a training package to suit yourlearning needs and preferred learning style.

Customer Support, Consulting, and Educationservices

http://service.sap.com/

Get online and timely information about DataServices, including tips and tricks, additionaldownloads, samples, and much more. All con-tent is to and from the community, so feel freeto join in and contact us if you have a submis-sion.

Data Services Community

https://www.sdn.sap.com/irj/sdn/businessobjects-ds

Search the Business Objects forums on theSAP Community Network to learn from otherData Services users and start posting questionsor share your knowledge with the community.

Forums on SCN (SAP Community Network)

https://www.sdn.sap.com/irj/sdn/businessob

jects-forums

Blueprints for you to download and modify to fit

your needs. Each blueprint contains the neces-sary Data Services project, jobs, data flows, fileformats, sample data, template tables, andcustom functions to run the data flows in yourenvironment with only a few modifications.

Blueprints

http://www.sdn.sap.com/irj/boc/blueprints


1Introduction

http://www.service.sap.com/https://www.sdn.sap.com/irj/sdn/businessobjects-dshttps://www.sdn.sap.com/irj/sdn/businessobjects-dshttps://www.sdn.sap.com/irj/sdn/businessobjects-forumshttps://www.sdn.sap.com/irj/sdn/businessobjects-forumshttp://www.sdn.sap.com/irj/boc/blueprintshttp://www.sdn.sap.com/irj/boc/blueprintshttps://www.sdn.sap.com/irj/sdn/businessobjects-forumshttps://www.sdn.sap.com/irj/sdn/businessobjects-forumshttps://www.sdn.sap.com/irj/sdn/businessobjects-dshttps://www.sdn.sap.com/irj/sdn/businessobjects-dshttp://www.service.sap.com/


16/226


17/226

Data Services MigrationConsiderations

2


18/226

This chapter describes behavior changes associated with the Data Integratorproduct since version 6.5 and in Data Services since 12.0.0 including DataQuality functionality. Each behavior change is listed under the version number

in which the behavior originated.For information about how to migrate your Data Quality Projects into DataServices, see Data Quality to Data Services Migration on page 61.

For the latest Data Services technical documentation, consult the DataServices Technical Manuals included with your product.

This Migration Considerations document contains the following sections:

Behavior changes in version 12.1.1 on page 18

Behavior changes in version 12.1.0on page 20

Behavior changes in version 12.0.0on page 26 Behavior changes in version 11.7.3 on page 34





Behavior changes in version 11.5.0.0on page 47



Behavior changes in version 11.0.1.1 on page 49




Behavior changes in version 6.5.0.1 on page 54


To read or download Data Integrator and Data Services documentation forprevious releases (including Release Summaries and Release Notes), seethe SAP Business Objects Help Portal at http://help.sap.com/.

Behavior changes in version 12.1.1

The following sections describe changes in the behavior of Data Services12.1.1 from previous releases of Data Services and Data Integrator. In mostcases, the new version avoids changes that would cause existing applications


Data Services Migration Considerations2 Behavior changes in version 12.1.1
http://help.sap.com/http://help.sap.com/


19/226

to modify their results. However, under some circumstances a change hasbeen deemed worthwhile or unavoidable.

If you are migrating from Data Quality to Data Services, see Data Quality to

Data Services Migration on page 61.This section includes migration-specific information associated with thefollowing features:

Blob data type enhancements on page 19

Neoview bulk loadingon page 20

Blob data type enhancements

Data Services 12.1.1 provides the following enhancements for binary largeobject (blob) data types:

You can now define blob data type columns in a fixed-width file format,and you can read from and load to blob columns in fixed-width files

The dqmigration utility now migrates Data Quality binary data types infixed-width flat files to Data Services blob (instead of varchar) data typesin fixed-width file formats. You no longer need to change the data typefrom varchar to blob after migration.

In a fixed-width file, the blob data is always inline with the rest of the data in

the file. The term "inline" means the data itself appears at the location wherea specific column is expected.

The 12.1.0 release of Data Services introduced support for blob data typesin a delimited file. In a delimited file, the blob data always references anexternal file at the location where the column is expected. Data Servicesautomatically generates the file name.

The following table summarizes the capabilities that each release providesfor blob data types:

InlineFile Type

12.1.112.1.012.1.112.1.0

YesYesNoNoblob in delimited file

NoNoYesNoblob in fixed-width file


2Data Services Migration Considerations



20/226

These capabilities help customers migrate their existing Data Quality projectsthat handle binary data in flat files to Data Services fixed-width file formats.The Data Services blob data type now supports blob data types from DataQuality XI R2 and legacy Firstlogic products.

Related Topics

Reference Guide: Data Types, blobs

Neoview bulk loading

If you plan to bulk load data to a Neoview database, we recommend thatyou set Timeout to 1000 in your Neoview target table.

If you create a new repository in version 12.1.1, you do not need to setTimeout because its default value is 1000.

If you use a 12.1.0 repository when you install version 12.1.1, the defaultvalue forTimeout is 60. Therefore, increase Timeout to 1000 for newdata flows that bulk load into a Neoview database.

Related Topics

Reference Guide: Data Services Objects, HP Neoview target table options

Behavior changes in version 12.1.0The following sections describe changes in the behavior of Data Services12.1.0 from previous releases of Data Services and Data Integrator. In mostcases, the new version avoids changes that would cause existing applicationsto modify their results. However, under some circumstances a change hasbeen deemed worthwhile or unavoidable.

If you are migrating from Data Quality to Data Services, see the Data Qualityto Data Services Migration Guide.

This section includes migration-specific information associated with thefollowing features:

Cleansing package changes on page 21

DTD-to-XSD conversion on page 21

Minimum requirements for international addressing directories on page 22

Try/catch exception groups on page 22




21/226

Upgrading from version 12.0.0 to version 12.1.0on page 25

Cleansing package changes

Global Parsing Options have been renamed cleansing packages.

You can no longer use the Global Parsing Options installer to install cleansingpackages directly into the repository. You must now use a combination ofthe cleansing package installer and the Repository Manager instead.

If you have made any changes to your existing cleansing packagedictionaries, you must do the following:

1. Export the changes using Export Dictionary Changes in the Dictionary

menu of the Data Services Designer.2. Install the latest cleansing package.

3. Use the Repository Manager to load the cleansing package into the datacleanse repository.

4. Import the changes into the new cleansing package using Bulk Load inthe Dictionary menu in the Designer.

Related Topics

Designer Guide: Data Quality, To export dictionary changes

Installation Guide for Windows: To create or upgrade repositories

Designer Guide: Data Quality, To import dictionary changes

DTD-to-XSD conversion

Data Services no longer supports publishing a DTD-based real-time job asa Web service if the job uses a DTD to define the input and output messages.

If you migrate from Data Services 12.0.0 to version 12.1.0, you do not needto do anything unless you change the DTD. If you change the DTD, reimport

it to the repository and publish the Web service as in the following procedure.

If you migrate from Data Integrator 11.7 or earlier versions to Data Services12.1.0 and publish a DTD-based real-time job as a Web service, you mustreimport the Web service adapter function because the Web address changedfor the Management Console in version 12.0.0. Therefore, you must do thefollowing after you upgrade your repository to version 12.1.0:





22/226

1. Use any DTD-to-XSD conversion tool to convert the DTD to XSD.

2. Use the Designer to import the XSD to the Data Services repository.

3. Open the original data flow that is using the DTD and replace it with XSD.

4. Publish the real-time job as Web service.5. Reimport the service as a function in the Web Service datastore.

Related Topics

Data Services web address on page 27

Minimum requirements for international addressingdirectories

Due to additional country support and modified database structures (forperformance tuning), theminimum disk space requirement for the internationaladdressing directories (All World) has increased as follows:

For the Global Address Cleanse transform (ga_country.dir,ga_loc12_gen.dir, ga_loc12_gen_nogit.dir, ga_loc34_gen.dir,ga_region_gen.dir), the minimum requirement has increased from 647MB to 2.71 GB.

If you purchase all countries, the the disk space requirement has increasedfrom 6.1 GB to 9.34 GB.

Try/catch exception groups

This version of Data Services provides better defined exception groups oferrors, new exception groups, and an enhanced catch editor that allows youto select multiple exception groups in one catch to consolidate actions.

After you upgrade your repository to version 12.1, your try/catch blockscreated in prior versions contain the 12.1 exception group names and

numbers. Be aware of the following situations and additional actions thatyou might need to take after you upgrade to version 12.1:

The repository upgrade will map Parser errors (1) and Resolve errors (2)to Execution errors (1000) and will map email errors(16) to SystemResource errors (1008). You need to re-evaluate all the actions that arealready defined in all the catch blocks and modify them as appropriate,




23/226

based on the new catch exception group definitions. See the tables belowfor the mapping of exception groups from version 12.0 to version 12.1and for the definitions of new exception groups.

All recoverable jobs in a pre-12.1 system lose their recoverable statewhen you upgrade. After you upgrade to version 12.1, you need to runthe job from the beginning.

If you upgrade a central repository, only the latest version of a work flow,data flow audit script, and user function contain the 12.1 exception groupnames. Older versions of these objects contain the pre-12.1 exceptiongroup names.

In version 12.1, if you have a sequence of catch blocks in a workflow andone catch block catches an exception, the subsequent catch blocks willnot be executed. For example, if your work flow has the followingsequence and Catch1 catches an exception, then Catch2 and CatchAll

will not execute. In prior versions, both Catch1 and CatchAll will execute.

Try > DataFlow1 > Catch1 > Catch2 > CatchAll

Note:

If you import pre-12.1 ATL files, any catch objects will not contain the newexception group names and numbers. Only a repository upgrade convertsthe pre-12.1 exception groups to the 12.1 exception group names andnumbers.

The following table shows how the exception groups in version 12.0 map tothe exception groups in version 12.1:

12.1 Description

12.1 Exception

group (group

number)

12.0 Description

12.0 Exception

group (group

number)

All errorsAll exceptionsAll errorsCatch All Excep-tions

Parser errors are not

caught because parsingoccurs prior to execution.

Pre-execution er-

rors (1000)

Errors encountered while

parsing the languageParser Errors (1)





24/226

12.1 Description

12.1 Exception

group (group

number)

12.0 Description

12.0 Exception

group (group

number)

Resolver errors are notcaught because parsingoccurs prior to execution.

Pre-execution er-rors (1000)

Errors encountered whilevalidating the semanticsof Data Services objectswhich have recommend-ed resolutions

ResolverErrors (2)

Errors from the Data Ser-vices job server or trans-forms

Execution errors(1001)

Internal errors that occurduring the execution of adata movement specifica-tion

Execution Errors(5)

Errors from the databaseserver while reading data,writing data, or bulk load-ing to tables

Database AccessErrors (1002)

GenericDatabase AccessErrors

Database AccessErrors (7)

Errors processing flat filesFlat file processingerrors (1004)

Errors accessing filesthrough file formats

File Access Errors(8)

Errors accessing localand FTP files

File Access Errors(1005)

Errors accessing the DataServices repository

Repository accesserrors (1006)

Errors accessing theDataServices repository

Repository AccessErrors (10)

Errors connecting todatabase servers

Database Connec-tion errors (1003)

Errors connecting todatabase servers andbulk loading to tables onthem

Connection andbulk loader errors(12)

Errors while generatingABAP programs, during

ABAP generated usertransforms, or while ac-cessing R/3 systemusingits API

R/3 system errors(1007)

Predefined transform er-rors

Predefined Trans-forms Errors (13)

ABAP generation errorsABAP GenerationErrors (14)

R/3 execution errorsR/3 Execution Er-rors (15)




25/226

12.1 Description

12.1 Exception

group (group

number)

12.0 Description

12.0 Exception

group (group

number)

Errors while accessing orusing operating systemresources, or while send-ing emails

System Resourceerrors (1008)

Email errorsEmail Errors (16)

System exception errorsSystem ExceptionErrors (17)

Errors from the Data Ser-vices job server or trans-forms

Execution errors(1001)

Engine abort errorsEngine Abort Er-rors (20)

The following table shows the new exception groups in version 12.1:

DescriptionNew 12.1 Exception group (group

number)

Errors from the SAP BW system.SAP BW execution errors (1009)

Errors processing XML files andmessages

XML processing errors (1010)

Errors processing COBOL copybook

files

COBOL copybook errors (1011)

Errors processing Excel booksExcel book errors (1012)

Errors processing Data Qualitytransforms

Data Quality transform errors (1013)

Upgrading from version 12.0.0 to version 12.1.0

If you are installing version 12.1.0 and the installer detects a previousinstallation of version 12.0, you will be prompted to first uninstall version12.0. The installer will maintain your configuration settings if you install inthe same directory.





26/226

If you are installing version 12.1.0 on top of version 11.x, you do not needto uninstall the previous version.

Behavior changes in version 12.0.0The following sections describe changes in the behavior of Data Services12.0.0 from previous releases of Data Integrator. In most cases, the newversion avoids changes that would cause existing applications to modify theirresults. However, under some circumstances a change has been deemedworthwhile or unavoidable.

If you are migrating from Data Quality to Data Services, see the Data Qualityto Data Services Migration Guide.


Case transform enhancementon page 26

Data Quality projects in Data Integrator jobs on page 27

Data Services web address on page 27

Large object data type enhancements on page 28

License keycodes on page 31

Locale selection on page 31

ODBC bigint data type on page 33

Persistent and pageable cache enhancements on page 33 Row delimiter for flat files on page 33

Case transform enhancement

In this version, you can choose the order of Case expression processing toimprove performance by processing the less CPU-intensive expressionsfirst. When the Preserve case expression orderoption is not selected inthe Case transform, Data Services determines the order to process the case

expressions. The Preserve case expression orderoption is available onlywhen the Row can be TRUE for one case only option is selected.

By default, the Row can be TRUE for one case only option is selected andthe Preserve case expression orderoption is not selected. Therefore,when you migrate to this version, Data Services will choose the order toprocess your Case expressions by default.




27/226

However, the reordering of expressions can change your results becausethere is no way to guarantee which expression will evaluate to TRUE first.If your results changed in this version and you want to obtain the same resultsas prior versions, select the Preserve case expression orderoption.

Data Quality projects in Data Integrator jobs

To do data cleansing in version Data Integrator 11.7, you created a DataQualilty datastore and imported integrated batch projects as Data Qualitytransforms. When these imported Data Quality transforms were used in an11.7 job, the data was passed to Data Quality for cleansing, and then passedback to the Data Integrator job.

In Data Services 12, the Data Quality transforms are built in. Therefore, ifyou used imported Data Quality transforms in Data Integrator 11.7, you mustreplace them in Data Services with the new built-in Data Quality transforms.

Related Topics

Modifying Data Integrator 11.7 Data Quality projects on page 161

Migrating Data Quality integrated batch projects on page 159

How integrated batch projects migrate on page 89

Data Services web address

In this release, Data Integrator has become part of Data Services. Therefore,the Web address has changed for the Management Console. In previousreleases, the Web address used "diAdmin" as the following format shows:

http://computername:port/diAdmin

In Data Services, the Web address uses DataServices:

http://computername:port/DataServices

Therefore, when you migrate to Data Services you must make changes inthe following situations:





28/226

If you created a bookmark that points to the Management Console in aprevious release, you must update the bookmark to the changed Webaddress.

If you generated a Web Service Definition Language (WSDL) file in aprevious version of Data Integrator, you must regenerate it to use thechanged Web address of the Administrator.

Large object data type enhancements

Data Services 12.0 extends the support of large objects as follows:

Adds support for binary large object (blob) data types from the currentlysupported database systems (Oracle, DB2, Microsoft SQL Server, and

so on). Extends support for character large object (clob) and national character

object (nclob) data types to other databases.

Prior versions treat the clob and nclob data types as long data types, andthis version continues to treat them as long data types.

The following table shows the large data types that version 11.7 supportsas long data types and the additional large data types that version 12 nowsupports. If your pre-version 12 jobs have sources that contain thesepreviously unsupported large data types and you now want to use them in

version 12, you must re-import the source tables and modify your existingjobs to select these newly supported data types.




29/226

Table 2-4: Database large object data types supported

Version

12.0 data

type

Version

12.0 sup-

ports

Version

11.7 sup-

ports

CategoryDatabase da-

ta type

Database

LONGYesYesCLOBLONG VAR-CHAR

DB2

LONGYesYesCLOBCLOB

LONGYesNoNCLOBLONG VAR-GRAPHIC

LONGYesNoNCLOBDBCLOB

BLOBYesNoBLOBBLOB

VARCHARYesYesVARCHAR

LVARCHAR

Informix

LONGYesYesCLOBTEXT

BLOBYesNoBLOBBYTE

LONGYesYesCLOBCLOB

BLOBYesNoBLOBBLOB

LONGYesYesCLOBTEXT

MicrosoftSQL Server

LONGYesNoNCLOBNTEXT

LONGYesNoCLOBVARCHAR(max)

LONGYesNoNCLOBNVARCHAR

(max)

BLOBYesNoBLOBIMAGE

BLOBYesNoBLOBVARBINARY(max)





30/226

Version

12.0 data

type

Version

12.0 sup-

ports

Version

11.7 sup-

ports

CategoryDatabase da-

ta typeDatabase

LONGYesYesCLOBTEXTMySQL

BLOBYesNoBLOBBLOB

LONGYesYesCLOBSQL_LONGVARCHAR

ODBC LONGYesNoNCLOBSQL_WLONGVARCHAR

BLOBYesNoBLOBSQL_LONGVARBINARY

LONGYesYesCLOBLONG

Oracle

BLOBYesNoBLOBLONGRAW

LONGYesYesCLOBCLOB

LONGYesYesNCLOBNCLOB

BLOBYesNoBLOBBLOB

LONGYesNoCLOBTEXTSybaseASE

BLOBYesNoBLOBIMAGE


Sybase IQ12.6 or later

BLOBYesNoBLOBLONG BINA-RY


TeradataLONGYesYesCLOBCLOB

BLOBYesNoBLOBBLOB




31/226

License keycodes

In this version, Data Services incorporates the BusinessObjects Enterpriseinstallation technology and uses keycodes to manage the licenses for thedifferent features. Therefore, Data Services does not use .lic license filesanymore but manages keycodes in the License Manager.

Locale selection

In this version, you no longer set the locale of the Job Server when you install

Data Services. After installation, the locale of the Job Server is set to which enables Data Services to automatically set the locale forthe repository connection (for the Designer) and to process job data (for theJob Server) according to the locale of the datastore or operating system.This capability enables Data Services to automatically change the locale forbetter performance (for example, set the locale to non-UTF-8 if the datastoreis non-Unicode data).

The following table shows different datastores and Job Server locale settings,the resulting locale that prior versions set, and the new locale that version12.0 sets for the data flow. In this table, the Job Server locale is set to

and derives its value from the operating system.

Data flow lo-

cale in ver-

sion 12.0

Data flow lo-

cale in prior

version

Job Server

locale

Datastore 2

locale

Datastore 1

locale

UnicodeSame localeas Job Server

Single-bytecode page orMulti-bytecode page

Multi-bytecode page

Single-bytecode page

UnicodeSingle-bytecode page

Single-bytecode page

Multi-bytecode page

Multi-bytecode page

UnicodeUnicodeMulti-bytecode page

Multi-bytecode page

Multi-bytecode page





32/226

Data flow lo-

cale in ver-

sion 12.0

Data flow lo-

cale in prior

version

Job Server

locale

Datastore 2

locale

Datastore 1

locale

UnicodeSingle-bytecode page 3

Single-bytecode page 3














The following table summarizes the locale that Data Services now sets foreach data flow when the locale of the Job Server is set to . Differentdata flows in the same job can run in either single-byte or Unicode.

Locale that Data

Services sets

Job Server lo-

caleLocale of datastores in data flow

UnicodeSingle-byte orMulti-byteOne datastore has multi-byte locale

UnicodeSingle-byte orMulti-byte

Different single-byte locales

Single-byteSingle-byteSame single-byte locale

UnicodeMulti-byteSame single-byte locale

You can override the default locale for the Job Server by using the DataServices Locale Selector utility. From the Windows Start menu, selectPrograms > BusinessObjects XI 3.1 > BusinessObjects Data Services> Data Services Locale Selector.




33/226

ODBC bigint data type

For an ODBC datastore, Data Services now imports a bigint data type asdecimal. In prior releases of Data Integrator, the bigint data type was importedas a double data type. If your pre-version 12 jobs have sources that containbigint data types, you must re-import the source tables and modify yourexisting jobs to handle them as decimal data types.

Persistent and pageable cache enhancements

This release of Data Services provides performance enhancements for the

persistent and pageable caches. Decimal data types now use only half thememory used in prior versions.

However, persistent cache tables created in prior versions are not compatiblewith Data Services. You must recreate them by rerunning the jobs thatoriginally created and loaded the target persistent cache tables.

Row delimiter for flat files

In Data Services 12, you can now specify the following values as rowdelimiters for flat files:

{new line}

If you specify this value for the row delimiter, Data Services writes theappropriate characters for the operating system on which the Job Serveris running:

CRLF (\r\n) in Windows

LF (\n) in UNIX

any character sequenceIn this case, Data Services writes the characters you entered.

{UNIX new line}

In this case, Data Services writes the characters LF (\n) regardless of theoperating system.





34/226

{Windows new line}

In this case, Data Services writes the characters CRLF (\r\n) regardlessof the operating system.

In previous releases, you could only specify the following values as rowdelimiters for flat files, and the behavior is the same as in the new release:

{new line}

any character sequence

If your target appends to an existing file that was generated in a prior release,Data Services is not backward compatible for the following situations:

Your Job Server runs on a Windows platform and you choose {UNIX newline} for the row delimiter.

Your Job Server runs on a UNIX system and you choose {Windows newline} for the row delimiter.

In these situations, you must define a new file format, load data from theexisting file into the new file specifying the new row delimiter, and then appendnew data to the new file with the new row delimiter.


The following sections describe changes in the behavior of Data Services

12.0 from previous releases of Data Integrator. In most cases, the new versionavoids changes that would cause existing applications to modify their results.However, under some circumstances a change has been deemed worthwhileor unavoidable.


Data flow cache type on page 35

Job Server enhancementon page 35

Logs in the Designeron page 35

Pageable cache for memory-intensive data flows on page 35




35/226


36/226

Specify a pageable cache directory that:

Contains enough disk space for your data. To estimate the amount ofspace required for pageable cache, consider factors such as thenumber of concurrently running jobs or data flows and the amount ofpageable cache required for each concurrent data flow

Exists on a separate disk or file system from the Data Integrator systemand operating system (such as the C: drive on Windows or the rootfile system on UNIX).

Adapter SDK

The Adapter SDK no longer supports native SQL or partial SQL.

PeopleSoft 8

PeopleSoft 8 support is implemented for Oracle only.

Data Integrator jobs that ran against previous versions of PeopleSoft are notguaranteed to work with PeopleSoft 8. You must update the jobs to reflectmetadata or schemadifferences between PeopleSoft 8 andprevious versions.


The following sections describe changes in the behavior of Data Integrator11.7.2 from previous releases. In most cases, the new version avoids changesthat would cause existing applications to modify their results. However, undersome circumstances a change has been deemed worthwhile or unavoidable.


Embedded data flows on page 37

Oracle Repository upgrade on page 37

Solaris and AIX platforms on page 39




37/226

Embedded data flows

In this version of Data Integrator, you cannot create embedded data flowswhich have both an input port and an output port. You can create a newembedded data flow only at the beginning or at the end of a data flow withat most one port, which can be either an input or an output port.

However, after upgrading to Data Integrator version 11.7.2, embedded dataflows created in previous versions will continue to run.

Oracle Repository upgrade

If you previously upgraded your repository to Data Integrator 11.7.0 and openthe "Object State Report" on the Central repository from the Web

Administrator, you may see the error "ORA04063 view ALVW_OBJ_CINOUThas errors". This occurs if you had a pre-11.7.0. Oracle central repositoryand upgraded the central repository to 11.7.0.

Note:

If you upgraded from a pre-11.7.0.0 version of Data Integrator to version11.7.0.0 and you are now upgrading to version 11.7.2.0, this issue may occur,and you must follow the instructions below. Alternatively, if you upgraded

from a pre-11.7.0.0 version of Data Integrator to 11.7.2.0 without upgradingto version 11.7.0.0, this issue will not occur and has been fixed in 11.7.2.0.

To fix this error, manually drop and recreate the view ALVW_OBJ_CINOUTusing an Oracle SQL editor, such as SQLPlus.

Use the following SQL statements to perform the upgrade:

DROP VIEW ALVW_OBJ_CINOUT;

CREATE VIEW ALVW_OBJ_CINOUT (OBJECT_TYPE, NAME, TYPE, NORMNAME,

VERSION, DATASTORE, OWNER,STATE, CHECKOUT_DT, CHECKOUT_REPO,

CHECKIN_DT,

CHECKIN_REPO, LABEL, LABEL_DT,COMMENTS,SEC_USER,SEC_USER_COUT)

AS

(





38/226

select OBJECT_TYPE*1000+TYPE,NAME, N'AL_LANG' , NORMNAME,VER

SION,DATASTORE, OWNER, STATE, CHECKOUT_DT, CHECKOUT_REPO,

CHECKIN_DT,

CHECKIN_REPO, LABEL, LABEL_DT,COMMENTS,SEC_USER ,SEC_USER_COUT

from AL_LANG L1 where NORMNAME NOT IN ( N'CD_DS_D0CAFAE2' ,

N'XML_TEMPLATE_FORMAT' , N'CD_JOB_D0CAFAE2' , N'CD_DF_D0CAFAE2'

, N'DI_JOB_AL_MACH_INFO' , N'DI_DF_AL_MACH_INFO' ,

N'DI_FF_AL_MACH_INFO' )

union

select 20001, NAME,FUNC_TYPE ,NORMNAME, VERSION, DATASTORE,

OWNER, STATE, CHECKOUT_DT, CHECKOUT_REPO, CHECKIN_DT,


from AL_FUNCINFO F1 where FUNC_TYPE = N'User_Script_Function'

OR OWNER N'acta_owner'

union

select 30001, NAME, N'PROJECT' , NORMNAME, VERSION, N'' , N''

, STATE, CHECKOUT_DT, CHECKOUT_REPO, CHECKIN_DT,


from AL_PROJECTS P1

union

select 40001, NAME,TABLE_TYPE, NORMNAME, VERSION, DATASTORE,

OWNER, STATE, CHECKOUT_DT, CHECKOUT_REPO, CHECKIN_DT,


from AL_SCHEMA DS1 where DATASTORE N'CD_DS_d0cafae2'

union

select 50001, NAME, N'DOMAIN' , NORMNAME, VERSION, DATASTORE,

N'' , STATE, CHECKOUT_DT, CHECKOUT_REPO, CHECKIN_DT,

CHECKIN_REPO, N'' ,to_date( N'01/01/1970' , N'MM/DD/YYYY' ),

N'' ,SEC_USER ,SEC_USER_COUT

from AL_DOMAIN_INFO D1




39/226

);

Solaris and AIX platforms

Data Integrator 11.7.2 on Solaris and AIX platforms is a 64-bit applicationand requires 64-bit versions of the middleware client software (such as Oracleand SAP) for effective connectivity. If you are upgrading to Data Integrator11.7.2 from a previous version, you must also upgrade all associatedmiddleware client software to the 64-bit version of that client. You must alsoupdate all library paths to ensure that Data Integrator uses the correct 64-bitlibrary paths.

Behavior changes in version 11.7.0The following sections describe changes in the behavior of Data Integrator11.7.0. from previous releases. In most cases, the new version avoidschanges that would cause existing applications to modify their results.However, under some circumstances a change has been deemed worthwhileor unavoidable.


Data Qualityon page 40

Distributed data flows on page 42

JMS Adapter interface on page 43

XML Schema enhancementon page 43

Password managementon page 43

Repository size on page 44

Web applications on page 44

Web services on page 44





40/226

Data Quality

Data Integrator 11.7.0 integrates the BusinessObjects Data Quality XIapplication for your data quality (formerly known as Data Cleansing) needs,which replaces Firstlogic's RAPID technology.

Note:

The following changes are obsolete with Data Services version 12.0 becausethe Data Quality transforms are built into Data Services, and you can usethem just like the regular Data Integrator transforms in a data flow.

The following changes to data cleansing occurred in Data Integrator 11.7.0:

Depending on the Firstlogic products you owned, you previously had upto three separate transforms that represented data quality functionality:Address_Enhancement, Match_Merge, and Name_Parsing.

Now, the data quality process takes place through a Data Quality Project.To upgrade existing data cleansing data flows in Data Integrator, replaceeach of the cleansing transforms with an imported Data Quality Projectusing the Designer.

You must identify all of the data flows that contain any data cleansingtransforms and replace them with a new Data Quality Project that connects

to a Data Quality blueprint or custom project. Data Quality includes many example blueprints - sample projects that

are ready to run or can serve as a starting point when creating your owncustomized projects. If the existing blueprints do not completely suit yourneeds, just save any blueprint as a project and edit it. You can also createa project from scratch.

You must use the Project Architect (Data Quality's graphical user interface)to edit projects or create new ones. Business Objects stronglyrecommends that you do not attempt to manually edit the XML of a projector blueprint.

Each imported Data Quality project in Data Integrator represents areference to a project or blueprint on the data quality server. The DataIntegrator Data Quality projects allow field mapping.




41/226

To migrate your data flow to use the new Data Quality transforms

Note:The following procedure is now obsolete with Data Services version 12.0because the Data Quality transforms are now built into Data Services andyou can use them just like the regular Data Integrator transforms in a dataflow. If you performed this procedure in Data Integrator version 11.7, youwill need to migrate these data flows to Data Services. See Data Quality

projects in Data Integrator jobs on page 27.

1. Install Data Quality XI, configure and start the server. For installationinstructions, see your Data Quality XI documentation.

Note:

You must start the server before using Data Quality XI with Data Integrator.

2. In the Data Integrator Designer, create a new Business Objects DataQuality datastore and connect to your Data Quality server.

3. Import the Data Quality projects that represent the data qualitytransformations you want to use. Each project appears as a Data Qualityproject in your datastore. For the most common data qualitytransformations, you can use existing blueprints (sample projects) in theData Quality repository

4. Replace each occurrence of the old data cleansing transforms in your

data flows with one of the imported Data Quality transforms. Reconnectthe input and output schemas with the sources and targets used in thedata flow.

Note:

If you open a data flow containing old data cleansing transforms(address_enhancement, name_parsing, match_merge), Data Integratordisplays the old transforms (even though they no longer appear in the objectlibrary). You can even open the properties and see the details for each oldtransform.

If you attempt to validate a data flow that contains an old data cleansingtransform, Data Integrator throws an error. For example:

[Custom Transform:Address_Enhancement] BODI-1116074: First Logicsupport is obsolete. Please use the new Data Quality feature.

If you attempt to validate a data flow that contains an old data cleansingtransform, Data Integrator throws an error. For example:





42/226

If you attempt to execute a job that contains data flows using the old datacleansing transforms Data Integrator throws the same type of error.

If you need help migrating your data cleansing data flows to the new Data

Quality transforms, contact the SAP Business Objects Help Portal athttp://help.sap.com/.

Distributed data flows

After upgrading to this version of Data Integrator, existing jobs have thefollowing default values and behaviors:

Job distribution level: Job.

All data flows within a job will be run on the same job server.

The cache type for all data flows: In-memory type

Uses STL map and applies to all join caches, table comparison cachesand lookup caches, and so forth.

Default forCollect statistics for optimization and Collect statistics formonitoring: deselected.

Default forUse collected statistics: selected.

Since no statistics are initially collected, Data Integrator will not initiallyuse statistics.

Every data flow is run as a process (not as a sub data flow process).

New jobs and data flows you create using this version of Data Integratorhave the following default values and behaviors:

Job distribution level: Job.

The cache type for all data flows:Pageable.

Collect statistics for optimization and Collect statistics formonitoring: deselected.

Use collected statistics: selected.

If you want Data Integrator to use statistics, you must collect statistics foroptimization first.


http://help.sap.com/http://help.sap.com/


43/226

Every data flow is run as a single process. To run a data flow as multiplesub data flow processes, you must use the Data_Transfer transform orselect the Run as a separate process option in transforms or functions.

All temporary cache files are created under the LINK_DIR\Log\PCachedirectory. You can change this option from the Server Manager.

JMS Adapter interface

A new license key may be required to install the JMS Adapter interface. Ifyou have a license key issued prior to Data Integrator XI R2 version 11.5.1,send a request to [email protected] with "Data IntegratorLicense Keys" as the subject line.

XML Schema enhancement

Data Integrator 11.7 adds the new Include schema location option for XMLtarget objects. This option is selected by default.

Data Integrator 11.5.2 provided the keyXML_Namespace_No_SchemaLocation for section AL_Engine in theDesigner option Tools > Options > Job Server > General, and the default

value, FALSE, indicates that the schema location is included. If you upgradefrom 11.5.2 and had set XML_Namespace_No_SchemaLocation to TRUE(indicates that the schema location is NOT included), you must open theXML target in all data flows and clear the Include schema location optionto keep the old behavior for your XML target objects.

Password management

Data Integrator now encrypts all password fields using two-fish algorithm.

To simplify updating new passwords for the repository database, DataIntegrator includes a password file feature. If you do not have a requirementto change the password to the database that hosts the repository, you maynot need to use this optional feature.



http://[email protected]/http://[email protected]/


44/226

However, if you must change the password (for example, securityrequirements stipulate that you must change your password every 90 days),then Business Objects recommends that you migrate your scheduled orexternal job command files to use this feature.

Migration requires that every job command file be regenerated to use thepassword file. After migration, when you update the repository password,you need only regenerate the password file. If you do not migrate using thepassword file feature, then you must regenerate every job command fileevery time you change the associated password.

Repository size

Due to the multi-byte metadata support, the size of the Data Integratorrepository is about two times larger for all database types except Sybase.

Web applications

The Data Integrator Administrator (formerly called the Web Administrator)and Metadata Reports interfaces have been combined into the newManagement Console in Data Integrator 11.7. Now, you can start anyData Integrator Web application from the Management Console launch

pad (home page). If you have created a bookmark or favorite that pointsto the previous Administrator URL, you must update the bookmark topoint to http://computername:port/diAdmin.

If in a previous version of Data Integrator you generated WSDL for Webservice calls, you must regenerate the WSDL because the URL to the

Administrator has been changed in Data Integrator 11.7.

Web services

Data Integrator is now using Xerces2 library. When upgrading to 11.7 orabove and configuring the Web Services adapter to use the xsdPathparameter in the Web Service configuration file, delete the old Web Servicesadapter and create a new one. It is no longer necessary to configure thexsdPath parameter.




45/226


Data Integrator version 11.6.0.0 will only support automated upgrade fromActaWorks and Data Integrator versions 5.2.0 and above to version 11.5.For customers running versions prior to ActaWorks 5.2.0 the recommendedmigration path is to first upgrade to Data Integrator version 6.5.1.

If you are upgrading to 11.6.0.0, to take advantage of the new Netezzabulk-loading functionality, you must upgrade your repository.

If you are upgrading from Data Integrator versions 11.0.0 or older to DataIntegrator 11.6.0.0, you must upgrade your repository. If upgrading from11.5.0 to 11.6.0.0, you must upgrade your repository to take advantage ofthe new:

Data Quality Dashboards feature

Preserving database case feature

Salesforce.comAdapter Interface (delivered with version 11.5SP1AdapterInterfaces)

The following sections describe changes in the behavior of Data Integratorfrom previous releases. In most cases, the new version avoids changes thatwould cause existing applications to modify their results. However, undersome circumstances a change has been deemed worthwhile or unavoidable.

This section includes:

Netezza bulk loadingon page 45

Conversion between different data types on page 46

Netezza bulk loading

If you are upgrading to 11.6.0.0, to take advantage of the new Netezza

bulk-loading functionality, follow this procedure:





46/226

Conversion between different data types

For this release, there is a change in behavior for data conversion betweendifferent data types. Previously, if an error occurred during data conversion(for example when converting a varchar string to an integer, the varcharstring contains non-digits), the result was random. Now, the return value willbe NULL for any unsuccessful conversion.

Previously Data Integrator returned random data for the result of a varcharto datetime conversion when the varchar string contains an illegal date format.Now, the return value will be NULL.

Behavior changes in version 11.5.1.5Data Integrator version 11.5.1.5 supports only automated upgrade from

ActaWorks and Data Integrator versions 5.2.0 and above to version 11.5.For customers running versions prior to ActaWorks 5.2.0 the recommendedmigration path is to first upgrade to Data Integrator version 6.5.1.

If you are upgrading to 11.5.1.5, to take advantage of the new Netezzabulk-loading functionality, you must upgrade your repository.

If you are upgrading from Data Integrator versions 11.0.0 or older to DataIntegrator 11.5.1.5, you must upgrade your repository. If upgrading from11.5.0 to 11.5.1.5, you must upgrade your repository to take advantage ofthese features:

Data Quality Dashboards

Preserving database case

Salesforce.comAdapter Interface (delivered with version 11.5SP1AdapterInterfaces)


No new behavior changes in this release.


Data Services Migration Considerations2 Behavior changes in version 11.5.1.5


47/226

Behavior changes in version 11.5.0.0

Data Integrator version 11.5 will only support automated upgrade fromActaWorks and Data Integrator versions 5.2.0 and above to version 11.5.For customers running versions prior to ActaWorks 5.2.0 the recommendedmigration path is to first upgrade to Data Integrator version 6.5.1.

If you are upgrading from Data Integrator version 11.0.0 to Data Integratorversion 11.5.0.0, you must upgrade your repository.

The following sections describe changes in the behavior of Data Integratorfrom previous releases. In most cases, the new version avoids changes thatwould cause existing applications to modify their results. However, undersome circumstances a change has been deemed worthwhile or unavoidable.

Web Services Adapter

Newer versions of Data Integrator will overwrite the XSD that defines theinput and output message files. Data Integrator stores those XSDs in aTomcat directory and during Metadata import passes to the engine a URLto the XSDs. The engine expects the XSDs to be available at run time.

To avoid this issue, prior to installing Data Integrator version 11.5, save a

copy of your LINKDIR\ext\webservice contents. After installing the newversion of Data Integrator, copy the directory contents back in place. If youdo not follow this procedure, the web service operations will require re-importfrom the web service datastore after the installation completes.

Varchar behavior

Data Integrator 11.5.0 and newer (including Data Services) conforms to ANSISQL-92 varchar behavior. When you upgradeand save your previous

configurations, thesoftware uses the previous varchar behavior as thedefault.However, Business Objects recommends that you use the ANSI varcharbehavior because the previous varchar behavior will not be supported infuture versions.





48/226

Central Repository

Business Objects recommends the following guidelines to upgrade yourcentral repository when installing a new version of Data Integrator.

Maintain a separate central repository for each Data Integrator versionto preserve object history.

An upgrade of the central repository might override the versions of anyobject whose internal representation (ATL) has changed in the newrelease.

Behavior changes in version 11.0.2.5The following sections describe changes in the behavior of Data Integratorfrom previous releases. In most cases, the new version avoids changes thatwould cause existing applications to change their results. However, undersome circumstances a change has been deemed worthwhile or unavoidable.

Teradata named pipe support

Beginning with Data Integrator version 11.0.2.5, you can use named pipesto bulk load data into Teradata databases. When Data Integrator tries toconnect to the pipes, Teradata Warehouse Builder might not yet have createdthe pipes. Data Integrator waits one second and retries to connect, up to 30seconds. If you need to change the 30-second wait time, go to the Designerselect Tools > Options > Job Server > General window, enteral_enginefor Section, enterNamedPipeWaitTime for Key, and enter a different value.

If you use Warehouse Builder and Teradata Tools and Utilities version 7.0or 7.1, go to the Data Integrator DesignerTools > Options > Job Server >General window, enteral_engine for Section, enterPreTWB5_Syntax forKey, and set the value to TRUE.


There are no new behavior changes for this release.




49/226


The following sections describe changes in the behavior of Data Integratorfrom previous releases. In most cases, the new version avoids changes thatwould cause existing applications to change their results. However, undersome circumstances a change has been deemed worthwhile or unavoidable.

Statistics repository tables

The repository tables AL_FLOW_STAT and AL_JOB_STAT are no longerpopulated for job, work flow, or data flow statistics. These two tables nolonger exist in the newly created Data Integrator version 11.0.1 repositories.They may still exist in the repositories that are upgraded to version 11.0.1.

Beginning with Data Integrator version 11.0.1, all job, work flow, and dataflow statistics are stored in the AL_HISTORY, AL_HISTORY_INFO, and

AL_STATISTICS tables. In particular, the job statistics, previously stored inAL_JOB_STAT, are nowstored in theAL_HISTORY andAL_HISTORY_INFOtables. The work flow and data flow statistics, previously stored in

AL_FLOW_STAT are now stored in the AL_STATISTICS table. Use themetadata reporting tool to retrieve all job, work flow and data flow statistics.



If you are upgrading from Data Integrator version 11.0.0 to Data Integratorversion 11.0.1, you must upgrade your repository to use the followingfeatures:

Sybase IQ as a source or target

New Data Integrator functions:

replace_substr_ext

long_to_varchar





50/226

varchar_to_long

load_to_xml

extract_from_xml

match_pattern

match_regex

literal

Generate delete row types in table comparison transform

COBOL data file reader

Impact Analysis in Metadata Reports for BusinessObjects XI

Auditing (some additional functionality was introduced in version 11.0.1)


Crystal Enterprise adapters

After installing a new version of Business Objects, you might receive a classnot found error when trying to start Crystal Enterprise adapter. Update theCrystal Enterprise adapter configuration by updating theBusinessObjects directory name in the classpath for boththe Crystal Enterprise adapter and the DMAC command file.







51/226

Changes to code page names

Starting with Version 11.0, Data Integrator displays some code pages withdifferent names. For example, code page MS1252 displays as CP1252.

Additionally, two code pages are no longer supported. These changes arevisible only in the locale screens (in the installer, datastore, and file formats).The Data Integrator engine does not have any upgrade issues and caninterpret the old code page names. The following new code page namesapply to Data Integrator version 11.0 and later.

New nameOld code page name

Not supportediso885910

CP949MS949

CP950MS950

CP1250MS1250

CP1251MS1251

CP1253MS1253

CP1254MS1254

CP1255MS1255

CP1257MS1257





52/226

New nameOld code page name

CP874MS874

CP1258MS1258

Not SupportedMS1361

US-ASCIIUSASCII

Data Cleansing

The new Data Integrator installation will overwrite all template job files shippedwith Data Integrator with the upgraded job files compatible with Firstlogic7.50. If the user had manually modified the template job files with DataIntegrator version 6.5 and wants to preserve the changes, Business Objectsrecommends that a copy is made of the job file, or that the job file is renamed,prior to installing Data Integrator version 11.0.

After installing Data Integrator version 11.0 and the Firstlogic 7.5 suite, youshould do the following:

From the Data Integrator CD, copy pwdiqjob.upd from the \Utilitiesdirectory to the dtr_iq directory. Click Yes to overwrite existing file.

For all your job templates, run Firstlogic's edjob utility to upgrade yourFirstlogic job files from the previous version to 7.50. Find the edjob utilityunder the installation of Firstlogic. Run the utility to upgrade all jobs ofthe same extension by specifying *. in the utility. Extensionnames are "diq", "iac", and "mpg". For instructions on how to use theedjob utility, refer to Firstlogic documentation.

Note:

If the jobs have already been updated to 7.50c (using the .upd file sent withthe master install with edjob), you will receive the following error when trying




53/226

to open a 7.50 job: Error: The "Report Defaults" block, occurrence 1, has anincorrect number of lines and the job cannot be loaded.

These jobs will need to be manually edited to change the version string in

the General block from 7.50c to 7.10c. Then edjob can be rerun with thecorrect .upd file.

For all job templates created under the address enhancement application,manually edit the job templates using an editor such as Notepad, andcomment out the "Create Output File" block by adding the '*' character at thebeginning of the line. For example:

*BEGIN Create Output File

License files and remote access software

In release 6.0 and higher, Data Integrator includes a specific change tolicense support when activating Data Integrator components acrossa terminalserver, such as PC Anywhere or Terminal Services Client. You must requestnew license files to use release 6.x or above when using a terminal server.Specifically, you require:

New Designer licenses to use a terminal server to initiate a Designersession across a terminal server connection (when using evaluation oremergency licenses, or node-locked licenses)

New Job Server and Access Server licenses to open the Service ManagerUtility across a terminal server connection (when using any type of license)

If you do not make the license change, you may see the error "TerminalServer remote client not allowed." If you require terminal services [email protected].







54/226



Web services support

When using Data Integrator with Web services, the following limitations exist.

SAP R/3 IDoc message sources in real-time jobs are not supported.

The Data Integrator Web services server uses the first Job Server in the listof those available for a batch job. The first Job Server might be a servergroup.

If you are using Web services and upgrading to version 6.5.0.1, Web servicesserver (call-in) functionality that you created with previous versions of DataIntegrator is not supported. Perform the following changes to manuallyupgrade the Web services server:

Regenerate a WSDL file

Prior to release 6.5.0.1, every batch job and real-time service configuredin Data Integrator was published in the WSDL file. In release 6.5.0.1,Data Integrator only publishes batch jobs and real-time services that youselect. Clients that call Data Integrator using Web services will fail untilyou authorize the jobs and services being invoked. Consult Data Integratortechnical manuals for instructions about how to select jobs, enablesecurity, and generate a WSDL file for a Web services server.

Change SOAP calls to match those in the 6.5.0.1 version.

A session ID is not included in SOAP messages unless you enable

security for the Web services server. Prior to release 6.5.0.1, the sessionIDmessage part was published in every SOAP message.

To fix this problem, change SOAP calls to match those in the 6.5.0.1version of the WSDL file.

If security is off, remove the sessionID line from the SOAP inputmessage.




55/226

If security is on, copy the sessionID from the SOAP body to the SOAPheader as a session ID now appears in the SOAP header instead ofthe body.

Note:When you install 6.5.0.1, the installer automatically upgrades the WebServices Adapter and your existing Web services clients will functionproperly.

Sybase bulk loader library on UNIX

The installed Sybase client on UNIX platforms does not include the Sybase

bulk loader dynamic shared library that Data Integrator requires. Please referto the READMEineither the CDROM Location/unix/PLATFORM/AWSybas

Documents

Xi31211 Ds Migration Considerations En