Ds 42 Workbench En

SAP Data ServicesDocument Version: 4.2 Support Package 1 (14.2.1.0) - 2013-11-14

Workbench Guide

Table of Contents1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.1 Welcome to SAP Data Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

1.1.1 Welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.1.2 Documentation set for SAP Data Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.1.3 Accessing documentation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.1.4 SAP information resources. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7

1.2 Overview of this guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.1 About this guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.2.2 Who should read this guide. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

1.3 Naming Conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Workbench User Interface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.1 Logging into the Workbench. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.2 Workbench window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.3 Menu bar. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12

2.3.1 File menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3.2 Edit menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.3.3 Tools menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.3.4 Validate menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3.5 Window menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3.6 Help menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .152.3.7 Replication job editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.4 Toolbar. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.5 Project explorer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.6 Data Flow editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.6.1 Additional Data Flow options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.7 Datastore editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.8 File Format editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.9 Target file properties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 272.10 Monitoring editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.11 Preferences. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.11.1 Data Services. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.11.2 General. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.11.3 Languages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3 Database Migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .403.1 Migrating data and schema information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.1.1 Creating a Workbench project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403.1.2 Using the quick replication wizard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2 © 2013 SAP AG or an SAP affiliate company. All rights reserved.Workbench Guide

Table of Contents

3.1.3 Using the Datastore editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.1.4 Using the replication job editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

3.1.5 Using the Data Flow editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.1.6 Converting replication tables to data flows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50

3.1.7 Expression macros . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

3.1.8 Using the File Format editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

3.1.9 Deploying and executing the replication job. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

3.2 Delta load migration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53

3.2.1 Delta load jobs in the Workbench. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3.2.2 Delta load options for tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

3.2.3 Delta load options for SAP applications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

4 Using an SAP Information Steward Data Cleansing Solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . .56

4.1 Viewing data cleansing solutions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56

4.1.1 Deleting a data cleansing solution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

4.2 Configuring a data flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57

4.2.1 Using the Basic Cleanse transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

4.3 Modifying the reference data location. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

4.4 Modifying the reports substitution variables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59

4.5 Validating a data flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60

4.6 Deploying a data flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60

4.7 Executing a data flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

4.8 Viewing data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.9 Modifying a Workbench job in Data Services Designer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5 Additional Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

5.1 Supported data migration sources and targets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .62

5.2 Using the expression editor. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5.2.1 Supported operators. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

5.2.2 Including functions in expressions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .64

5.3 Managing Workbench project information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

5.3.1 Creating a Workbench project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .72

5.3.2 Exporting resources and preferences. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

5.3.3 Importing resources and preferences. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

Workbench GuideTable of Contents © 2013 SAP AG or an SAP affiliate company. All rights reserved. 3

1 Introduction

1.1 Welcome to SAP Data Services

1.1.1 Welcome

SAP Data Services delivers a single enterprise-class solution for data integration, data quality, data profiling, and text data processing that allows you to integrate, transform, improve, and deliver trusted data to critical business processes. It provides one development UI, metadata repository, data connectivity layer, run-time environment, and management console—enabling IT organizations to lower total cost of ownership and accelerate time to value. With SAP Data Services, IT organizations can maximize operational efficiency with a single solution to improve data quality and gain access to heterogeneous sources and applications.

1.1.2 Documentation set for SAP Data Services

You should become familiar with all the pieces of documentation that relate to your SAP Data Services product. The latest Data Services documentation can be found on the SAP Help Portal.

Document What this document provides

Adapter SDK Guide Information about installing, configuring, and running the Data Services Adapter SDK

Administrator Guide Information about administrative tasks such as monitoring, lifecycle management, security, and so on.

Customer Issues Fixed Information about customer issues fixed in this release.

NoteIn some releases, this information is displayed the Release Notes.

Designer Guide Information about how to use Data Services Designer.

Documentation Map Information about available Data Services books, languages, and locations.

Installation Guide for Windows

Information about and procedures for installing Data Services in a Windows environment.

Installation Guide for UNIX

Information about and procedures for installing Data Services in a UNIX environment.

Integrator Guide Information for third-party developers to access Data Services functionality using web services and APIs.

Master Guide Information about the application, its components and scenarios for planning and designing your system landscape. Information about SAP Information Steward is also provided in this guide.


Introduction

http://help.sap.com/bods


Management Console Guide

Information about how to use Data Services Administrator and Data Services Metadata Reports.

Performance Optimization Guide

Information about how to improve the performance of Data Services.

Reference Guide Detailed reference material for Data Services Designer.

Release Notes Important information you need before installing and deploying this version of Data Services.

Technical Manuals A compiled, searchable, “master” PDF of core Data Services books:

● Administrator Guide● Designer Guide ● Reference Guide● Management Console Guide● Performance Optimization Guide● Integrator Guide● Supplement for J.D. Edwards● Supplement for Oracle Applications● Supplement for PeopleSoft ● Supplement for Salesforce.com● Supplement for Siebel● Supplement for SAP ● Workbench Guide

Text Data Processing Extraction Customization Guide

Information about building dictionaries and extraction rules to create your own extraction patterns to use with Text Data Processing transforms.

Text Data Processing Language Reference Guide

Information about the linguistic analysis and extraction processing features that the Text Data Processing component provides, as well as a reference section for each language supported.

Tutorial A step-by-step introduction to using Data Services.

Upgrade Guide Information to help you upgrade from previous releases of Data Services and release-specific product behavior changes from earlier versions of Data Services to the latest release.

What's New Highlights of new key features in this SAP Data Services release. This document is not updated for support package or patch releases.

Workbench Guide Provides users with information about how to use the Workbench to migrate data and database schema information between different database systems.

In addition, you may need to refer to several Supplemental Guides.


Supplement for SAP Information about interfaces between Data Services, SAP Applications, SAP Master Data Services, SAP NetWeaver BW, and SAP Master Data Services.

Workbench GuideIntroduction © 2013 SAP AG or an SAP affiliate company. All rights reserved. 5


Supplement for SuccessFactors Information about interfaces between Data Services and SuccessFactors.

Supplement for Salesforce.com Information about how to install, configure, and use the SAP Data Services Salesforce.com Adapter Interface.

Supplement for J.D. Edwards Information about interfaces between Data Services and J.D. Edwards World and J.D. Edwards OneWorld.

Supplement for Oracle Applications

Information about the interface between Data Services and Oracle Applications.

Supplement for PeopleSoft Information about interfaces between Data Servicesand PeopleSoft.

Supplement for Siebel Information about the interface between Data Services and Siebel.

We also include these manuals for information about SAP BusinessObjects Information platform services.


Information platform services Administrator Guide Information for administrators who are responsible for configuring, managing, and maintaining an Information platform services installation.

Information platform services Installation Guide for UNIX

Installation procedures for SAP BusinessObjects Information platform services on a UNIX environment.

Information platform services Installation Guide for Windows

Installation procedures for SAP BusinessObjects Information platform services on a Windows environment.

1.1.3 Accessing documentation

You can access the complete documentation set for SAP Data Services in several places.

1.1.3.1 Accessing documentation on Windows

After you install SAP Data Services, you can access the documentation from the Start menu.

1. Choose Start Programs SAP Data Services 4.2 Data Services Documentation All Guides .2. Click the appropriate shortcut for the document that you want to view.

1.1.3.2 Accessing documentation on UNIX

After you install SAP Data Services, you can access the documentation by going to the directory where the printable PDF files were installed.


Introduction

1. Go to <LINK_DIR>/doc/book/en/.

2. Using Adobe Reader, open the PDF file of the document that you want to view.

1.1.3.3 Accessing documentation from the Web

You can access the complete documentation set for SAP Data Services from the SAP Business Users Support site.

To do this, go to http://help.sap.com/bods.

You can view the PDFs online or save them to your computer.

1.1.4 SAP information resources

A global network of SAP technology experts provides customer support, education, and consulting to ensure maximum information management benefit to your business.

Useful addresses at a glance:

Address Content

Customer Support, Consulting, and Education services

http://service.sap.com/

Information about SAP support programs, as well as links to technical articles, downloads, and online forums. Consulting services can provide you with information about how SAP can help maximize your information management investment. Education services can provide information about training options and modules. From traditional classroom learning to targeted e-learning seminars, SAP can offer a training package to suit your learning needs and preferred learning style.

Product documentation

http://help.sap.com/bods/

SAP product documentation.

Supported Platforms (Product Availability Matrix)

https://service.sap.com/PAM

Get information about supported platforms for SAP Data Services.

Use the search function to search for Data Services. Click the link for the version of Data Services you are searching for.

SAP Data Services Community Network

http://scn.sap.com/community/data-services

Get online and timely information about SAP Data Services, including forums, tips and tricks, additional downloads, samples, and much more. All content is to and from the community, so feel free to join in and contact us if you have a submission.

Blueprints

http://scn.sap.com/docs/DOC-8820

Blueprints for you to download and modify to fit your needs. Each blueprint contains the necessary SAP Data Services project, jobs, data flows, file formats, sample data, template



http://help.sap.com/disclaimer?site=http://service.sap.com/


http://help.sap.com/disclaimer?site=https://service.sap.com/PAM

http://help.sap.com/disclaimer?site=http://scn.sap.com/community/data-services

http://help.sap.com/disclaimer?site=http://scn.sap.com/docs/DOC-8820

Address Content

tables, and custom functions to run the data flows in your environment with only a few modifications.

SAPTerm

https://portal.wdf.sap.corp/go/sapterm

SAP’s terminology database, the central repository for defining and standardizing the use of specialist terms.

1.2 Overview of this guide

Welcome to the Workbench Guide. The Data Services Workbench provides a graphical user interface (GUI) development environment in which you define data application logic to migrate data and database schema information between different databases in a data warehousing environment.

Key features of the Workbench include:

● Browsing table metadata and data.● Selecting individual source tables for migration.● Specifying the order in which the source tables should be migrated.● Adjusting the table schema in detail. For example, adding or removing columns, defining constraints,

partitions, indexes, and so on.● Specifying filters and simple projection expressions.● Specifying source and target table options such as array fetch size and bulk-loading options.● Executing a replication job as an initial load or delta load.

● Modeling and consuming a data quality job.

1.2.1 About this guide

This guide contains two kinds of information:

● Conceptual information that helps you understand the Data Services Workbench and how it works.● Procedural information that explains in a step-by-step manner how to accomplish a task.

You will find this guide most useful:

● While you are learning about the product.● While you are performing tasks in the design and early testing phases of your data-movement projects.● As a general source of information during any phase of your projects.

1.2.2 Who should read this guide

This and other Data Services product documentation assumes the following:


Introduction

http://help.sap.com/disclaimer?site=https://portal.wdf.sap.corp/go/sapterm

● You are an application developer, consultant, or database administrator working on data warehousing.● You understand your source data systems and RDMBS.● You understand your organization's data needs.● You are familiar with SQL (Structured Query Language).● You are familiar with Data Services installation environments — Microsoft Windows or UNIX.

1.3 Naming Conventions

In this documentation, the following naming conventions apply:

Terminology

● “Data Services system” refers to “SAP Data Services”.● “BI platform” refers to “SAP BusinessObjects BI platform”.

NoteThe BI platform components required by Data Services may also be provided by SAP BusinessObjects Information platform services (IPS).

● “CMC” refers to the Central Management Console provided by the BI or IPS platform.● “CMS” refers to the Central Management Server provided by BI or IPS platform.

Variables

Variables Description

<INSTALL_DIR> The installation directory for the SAP software.

Default location:

● Windows platformsC:\Program Files (x86)\SAP BusinessObjects

● UNIX platforms$HOME/sap businessobjects

<BIP_INSTALL_DIR> The root directory of the BI or IPS platform.

Default location:

● Windows platforms<INSTALL_DIR>/SAP BusinessObjects Enterprise XI 4.0

● UNIX platforms


Variables Description

<INSTALL_DIR>/enterprise_xi40

NoteThese paths are the same for both the SAP BusinessObjects BI platform and SAP BusinessObjects Information platform services.

<LINK_DIR> The root directory of the Data Services system.

Default location:

● All platforms<INSTALL_DIR>/Data Services

This system environment variable is created automatically during installation.

<DS_COMMON_DIR> The common configuration directory for the Data Services system.

Default location:

● Windows (Vista and newer)ALLUSERSPROFILE\SAP BusinessObjects\Data Services

● Windows (Older versions)ALLUSERSPROFILE\Application Data\SAP BusinessObjects\Data Services

● UNIX systems (for compatibility)<LINK_DIR>

This system environment variable is created automatically during installation.

<DS_USER_DIR> The user-specific configuration directory for the Data Services system.

Default location:

● Windows (Vista and newer)USERPROFILE\AppData\Local\SAP BusinessObjects\Data Services

● Windows (Older versions)USERPROFILE\Local Settings\Application Data\SAP BusinessObjects\Data Services

This user environment variable is created automatically during installation.

NoteThis variable is used only for Data Services client applications on Windows, such as the Designer. <DS_USER_DIR> is not used on UNIX platforms.


Introduction

2 Workbench User InterfaceThis section provides basic information about the Workbench user interface.

2.1 Logging into the Workbench

Before you can use the Workbench to create, edit, and execute replication jobs, you must log in as a user defined in the Central Management Server (CMS).

1. Launch the Workbench.

The Workbench can be accessed from the Windows Start Menu: All Programs SAP Data Services <x.x>Data Services Workbench .

2. Enter your user credentials for the CMS.Option Description

System The server name and optionally the port number for the CMS.

User name The user name to use to log into the CMS.

Password The password to use to log into the CMS.

Authentication mode The authentication type used by the CMS.

3. Click Connect.The software attempts to connect to the CMS using the specified information. If the authentication is successful, the Workbench window is displayed.

2.2 Workbench window

By default, the Workbench user interface consists of several primary elements:

● Menu bar● Toolbar● Project explorer

The Workbench project explorer contains the current project, and all jobs and datastores that are a part of it.● Workspace

The workspace is the area of the Workbench window where you define, display, and modify objects. When you create a new object, the Workbench automatically opens a new editor tab in the workspace. The workspace is also used to display the data of a source table and to display the job status and execution logs.

Workbench GuideWorkbench User Interface © 2013 SAP AG or an SAP affiliate company. All rights reserved. 11

Workspace editors

The workspace area of the Workbench contains tabs for the editors you currently have open. The editors that may appear in the workspace include the following:

● Replication job editorThe replication job editor allows you to display and edit replication job options. For example, the target table schema, projections expressions and filters, and replication groups. The name of the open replication job is indicated in the tab's title. If the replication job contains unsaved data, it is indicated with an asterisk (*).

● Data Flow editorThe data flow editor allows you to create data flows and include them in the replication job. The supported tranforms are Query transform, Case transform, Merge transform, Row Generation transform, MAP Operation transform, and SQL transform. You can reference existing data flows from the Replication Job by dragging and dropping the data flows in from the Explorer. You can verify all data flows and specify whether the data flow is for initial load, delta load, or both.

● Datastore editorThe datastore editor allows you to display and edit datastore properties, browse and import metadata, and display and edit table properties. The name of the open datastore is indicated in the tab's title. If the datastore contains unsaved data, it is indicated with an asterisk (*).

● File Format editorThe file format editor allows you to set properties for file format templates and source and target file formats.

● Monitoring editorThe monitoring editor displays your replication jobs' status, execution progress, and any applicable logs. Trace, monitor, and error logs are available.

Related InformationReplication job editor [page 15]Datastore editor [page 20]Data Flow editor [page 17]Monitoring editor [page 31]File Format editor [page 20]

2.3 Menu bar

This section contains a brief description of the Workbench menus.

2.3.1 File menu

The File menu contains standard Windows as well as software-specific options.


Workbench User Interface

Option Description

New Define a new folder, project, datastore, data flow, or replication job.

Open File Open an existing Workbench file.

Close Close the active Workbench editor.

Close All Close all open editors.

Save Save the object in the active editor.

Save All Save all objects in open editors.

Exit Exit the Workbench.

2.3.2 Edit menu

The Edit menu contains standard Windows options.

Option Description

Undo Undo the last operation.

Redo Redo the last undone operation.

Cut Cut the selected objects or text and place it on the clipboard.

Copy Copy the selected objects or text to the clipboard.

Paste Paste the contents of the clipboard into the active editor or text box.

Delete Delete the selected objects or text.

Select All Select all objects in the active editor or all text in the active text box.

2.3.3 Tools menu

The Tools menu contains replication job editor and datastore editor options.

Option Description

Open monitoring editor Open the job monitoring editor.

Quick replication wizard Start the quick replication wizard.

Export to ATL Export a selected replication job to the local file system.

Deploy Deploy a replication job or datastore definition to a Data Services repository.

Execute Deploy a replication job to a Data Services repository and execute the job.


Option Description

NoteThis option is available only for the replication job editor.

2.3.4 Validate menu

The Validate menu contains replication job editor and datastore editor validation options.

Option Description

Validate Validate the active Workbench replication job for errors.

NoteThe Validate option is available only for the replication job editor.

Show DDL Display the data definition language (DDL) for the active Workbench object.

NoteThe Show DDL option is available only for the replication job editor and only applies when the replication job editor is active.

Show ATL Display the ATL for the active Workbench object.

NoteDo not use this option to create a file for import into Data

Services. Use the Tools Export Replication Job

option to export a replication job.

2.3.5 Window menu

The Window menu provides software-specific window options.

Option Description

Data Cleansing Solutions Display the Data Cleansing Solutions tab in the workspace area.

Error Log Display the Error Log tab in the workspace area.

Problems Display the Problems tab in the workspace area.



Option Description

Project Explorer Display the project explorer.

Reference Analysis Display a reference impact analysis for the active Workbench object in the workspace area.

Other Display other window panes.

Reset Perspective Reset the Workbench interface perspective to its default settings.

Preferences Display the Preferences window.

2.3.6 Help menu

The Help menu provides standard help options.

Option Description

Welcome Display the Workbench welcome screen.

Help Display the SAP Data Services Workbench Guide.

About Display information about the software including the version of the Workbench and copyright information.

Key Assist Display information about keyboard shortcuts available in the Workbench.

2.3.7 Replication job editor

The Workbench replication job editor opens in the workspace when you create a new replication job or open an existing replication job from the project explorer.

The replication job editor consists of several areas:

● Project explorerDisplays the name of the source datastore and all imported tables.

● Target datastore paneDisplays the name of the target datastore and all mapped tables and columns. In this pane, mapped objects may be grouped into replication groups.

● Replication job configuration paneWhen you select an object such as a datastore table or column in the replication job editor, its configuration options are displayed in this pane.The option groups that are available depend on the type of object that is selected:

Object Available options

Project explorer Properties



Replication job Properties

Variables

Scripts

Replication group Properties

Target table Properties

Columns

Foreign Keys

Indexes

Options

Filter data

DDL Options

Delta load

Column Properties

You can adjust the size of each pane by dragging the borders between them. In addition, you can hide or reveal the replication job configuration pane by clicking the arrows in the center of its border.

Related InformationUsing the replication job editor [page 44]

2.4 Toolbar

In addition to many of the standard Windows tools, the software provides application specific tools, including:

Icon Tool Description

New Define a new folder, project, datastore, or replication job.

Save Save the object in the active editor.

Cut Cut the selected objects or text and place it on the clipboard.

Copy Copy the selected objects or text to the clipboard.

Paste Paste the contents of the clipboard into the active editor or text box.

Delete Delete the selected objects or text.

Open monitoring editor Open the job monitoring editor.



Icon Tool Description

Validate Validate the active Workbench resource for errors.

Show DDL Display the data definition language (DDL) for the active Workbench resource.

Deploy ATL Deploy the ATL for the active replication job to the Data Services repository.

Execute Deploy the ATL for the active replication job to the Data Services repository, and execute the replication job.

Quick replication wizard Start the quick replication wizard to easily define source and target datastores and create a basic replication job.

2.5 Project explorer

The project explorer contains the projects currently defined in the user workspace. Each Workbench project is represented as a folder containing all datastores and replication jobs associated with that project.

When you create new projects, datastores, data flows, file formats, and replication jobs, they are displayed in the project explorer. You can also use the project explorer to import and export Workbench project information.

2.6 Data Flow editor

Workbench Data Flow editor enables you to create data flows and include them in the replication job. The supported tranforms are the Query transform, Case transform, Merge transform, Row Generation transform, MAP Operation transform, and SQL transform. You can reference existing data flows from the Replication Job by dragging and dropping the data flows in from the Explorer. You can verify all data flows and specify whether the data flow is for initial load, delta load, or both.

You also have the ability to convert replication tables to data flows. Highlight multiple replication tables, right click the selection, and select convert data. You have the option to generate a single data flow or one data flow per replicated table.

Data Flow editor options

The Workbench Data Flow editor includes the Data Flow diagram, a list of input schemas for the selected transform, a list of available Transforms, and a tabbed area containing the properties of the object currently selected in the Data Flow diagram.



Data Flow diagram The Data Flow diagram is composed of a graphical area and a palette of elements you can drop on the graphical area. The Data Flow diagram toolbar includes options, such as Zoom slider, Snap to Grid, Highlight Related Element, and so on.

Data Flow diagram toolbar The Data Flow diagram toolbar includes the following:

● Zoom slider● Snap to grid● Enable highlight related elements● Enable descriptions in the diagram● Summary mode● Save a screenshot of the diagram

Transforms This panel displays all available transforms. Drag and drop the transforms onto the Data Flow diagram.

Input Schemas panel This panel displays all the available input schemas / fields that the user can drag-and-drop to the properties area. Every field that is being used in the current property tab is marked with a black triangle. Some fields are marked with a key icon indicating they are part of the primary key.

Object properties Displays the properties of the object that is currently selected in the Data Flow diagram.

The options for Table Reader properties, File Reader properties, Table Loader properties, Merge properties, Case properties, SQL properties, Row Generation properties, Map Operation properties, Date Generation properties, and the Effective Date properties are the same as the Data Services Designer.

The Template Table Loader uses the same database specific loader options as Data Services Designer as well as some extra options:

● Use NVARCHAR for VARCHAR● Drop and re-create table● Double quote names● Table Type (for HANA only)

Use data file as source or target

You can select and drop data from the Project Explorer onto the Data Flow diagram and choose to use it as a source or target.

Validation in the Data Flow editor

Objects containing errors will be underlined in red and a tooltip will show the list of errors/warnings for an element.



2.6.1 Additional Data Flow options

The Workbench Data flow Editor includes several options that are not available in Designer Data Flow editor.

Template file loader

Located in the Transforms panel, the Template File loader enables you to add data to a file without having to create a file format. Drop the Template file loader onto the Data Flow diagram and set the template file properties. You must enter the fully qualified file name(s).

Detect Mappings

Located on the Mappings tab, the Detect mappings option assists you during target to source mapping by detecting source fields with similar names. The dialog provides a list of proposed candidate mappings for you to view and confirm. Every proposed candidate field has a matching score (from 0 to 100), and proposals are displayed from the highest score to the lowest to allow you to see the most relevant proposals first. Only the proposals that are confirmed are applied.

Detect Joins

Join detection combines both foreign key relationships analysis (if the source has foreign keys) and name, data type matching. Every join that is detected is assigned a detection score and shown in a confirmation dialog. A join found using foreign key detection will always have a score of 100, and a detection using name will have a score between 0 and 99 depending on the similarity of the fields that are proposed as a join.

Create Query transform

The Data Flow feature allows you to right-click and select to add a new query transform to the data flow. Workbench will create the query transform object, connect it to both selected sources, connect it to the loader, run the detect joins, and detect mapping.

The Data Flow detections page under Preference gives you the flexibility to be more restrictive for the minimum score and allows you to choose whether to always confirm the detections or not.

Related InformationData Flow Detections [page 32]


2.7 Datastore editor

The Workbench datastore editor opens in the workspace when you create a new datastore or open an existing datastore from the project explorer or replication job editor.

The datastore editor consists of these areas:

● Navigation paneThe navigation pane displays the configuration and imported tables and views of the datastore. For SAP datastores, the navigation pane also displays the imported SAP extractors.When the datastore configuration is active, you can edit the datastore properties and settings. When the datastore tables, extractors, or views are active, you can import or re-import tables, extractors, and views, view data, and compare imported metadata with the metadata in the database.

● Object properties paneDisplays tabs for the properties and options for the object currently selected in the navigation pane. The option tabs that are available depend on the type of object that is selected:

Object Available option

Table Properties

Columns

Foreign Keys

Indexes

Partitions

Attributes

Column Properties

Attributes

You can adjust the size of each pane by dragging the borders between them. In addition, you can hide or reveal the navigation pane by clicking the arrow in the center of its border.

Related InformationUsing the Datastore editor [page 42]

2.8 File Format editor

A file format describes the structure of an ASCII file. A file format consists of multiple properties. You set the properties in the File Format editor.

The work area on the left, in the File Format editor, lists file format properties that are not field specific. The following table lists all of the options. These options are filtered by the mode you are using.

Option Possible values Description Mode

General




File format Delimited The format of the data in the text file. Available properties change based on the selected file format type.

New, Edit

Adaptable file reading

Yes, No Indicates whether the schema of a delimited file format is adaptable or fixed.

● Yes indicates that the schema is adaptable. The actual file can contain fewer or more columns than indicated by the file format. If a row contains fewer columns than expected, the software loads null values into the columns missing data. If a row contains more columns than expected, the software ignores the additional data.

● No indicates that the schema is fixed. The software requires the number of columns in each row to match the number of columns specified in the file format.The default is No. If you select Yes, you must ensure that the selected column and row delimiters do not appear inside the actual data.

New, Edit, Source

Custom transfer Yes, No Enables the software to use a third-party file transfer program. Displays additional Custom transfer program options in the File Format Editor below the Input/Output properties.

All

Skip error handling

Yes, No Selecting Yes disables the Error Handling section in the Format editor. The default is No.

New, Edit, Source

Number of processing threads

Integer greater than 0, {none}, {default}

Specifies the number of threads for parallel processing. Parallel processing can improve performance by maximizing CPU usage on the Job Server computer. For example, if you have four CPUs, enter 4 for this option. For more information, see “File multi-threading” in the Performance Optimization Guide.

For jobs that process USPS certification tests, the value should be set to {none}.

All

Data File(s)

File location Local, Job Server During design, indicates the files are located on the local machine or on the machine that runs the Job Server. If you select Job Server, you must enter the absolute path to files. Remember that UNIX systems are case-sensitive.

During execution, all files must be located on the Job Server machine that executes the job. If you use different files to design your job, change the file

All



specified (through the Root directory and File properties) before you execute the job.

Root folder Path name for the file or blank

The directory where the file is located.

For added flexibility, you can enter a variable for this option.

If you enter a directory name, then enter only the file name for the File property.

If you leave the root directory blank, then enter a file name that includes the full path name in the File property.

New, Edit

File name(s) File name(s), file name(s) including full path name, or blank

In new and edit modes, specifies an existing file on which you base the file format description. Data from this file appears in the Column Attributes area. In these modes, you can leave this property blank.

In source and target modes, specify the location of the actual file for this source or target. In these modes, you cannot leave this property blank. For added flexibility, you can enter:

● A variable that is set to a particular file with full path name. Use variables to specify file names that you cannot otherwise enter, such as file names that contain multi-byte characters.

● A list of files, separated by commas, or a file name containing a wildcard. In this case, the software reads all these files as a single source.See “Reading multiple files at one time” in the Designer Guide.

All

Delimiters

Column delimiter Tab, Semicolon, Comma, Space, or any character sequence

For delimited file formats, the character sequence that indicates the end of one column and the beginning of the next.

New, Edit

Row delimiter {new line}, {Windows new line}, {Unix new line}, or any character sequence

A character sequence that indicates where one row of data ends and the next begins.

New, Edit

Row delimiter within text string

Character, Row delimiter

Defines how the row delimiter is interpreted within a text string.

New, Edit




● Character: The specified row delimiter is treated as a character within the text string.

● Row delimiter: The specified row delimiter is interpreted and defines rows within the text string.

Text delimiter Single quotation marks ('), double quotation marks ("), or {none}

Denotes the start and end of a text string. All characters (including those specified as column delimiters) between the first and second occurrence of this character is a single text string. The treatment of the row characters is defined by the Row within text string setting.

New, Edit

NoteData in columns cannot include the column or row delimiter, unless you also specify a text delimiter. For example, if you specify a comma as your column delimiter, none of the data in the file can contain commas. However, if you specify a comma as the column delimiter and a single quote as the text delimiter, commas are allowed in strings in the data.

You can use any ASCII characters (including non-printing characters) for column and row delimiters.

You can specify an ASCII character by entering a forward slash (/) followed by the decimal representation of the character. For example, to use Y umlaut (Ÿ) as a delimiter, enter /255 in the delimiter property box.

Default Format

Escape character

Any character sequence, or {none}

A special character sequence that causes the software to ignore the normal column delimiter. Characters following the escape character sequence are never used as column delimiters.

For example, suppose you specify a forward slash as the escape character and a comma as the column delimiter. Then, you must have a forward slash to have a comma appear inside a field.

New, Edit

NULL indicator {none} or any other character sequence

A special character sequence that the software interprets as NULL data.

The software ignores any NULL indicator specified in the file format for blob columns.

New, Edit

Ignore row marker(s)

{none} or any other character sequence

A character sequence, which when found at the beginning of rows, causes the software to ignore the row when reading the file or automatically creating metadata. To enter multiple character sequences, separate each with a semi-colon. To include a semi-colon or backslash as a marking character, precede with a backslash.

New, Edit, Source

Date format yyyy.mm.dd The date format for reading or writing date values to and from the file.

New, Edit



or other combinations

Time format hh24:mi:ss


The time format for reading or writing time values to and from the file.

New, Edit

Date-Time format

yyyy.mm.dd hh24:mi:ss


The datetime format for reading or writing datetime values to and from the file.

New, Edit

Input/Output

Style Headers or BOF/EOF

The format of the start and end of the file. Available properties in the Input/Output property group may change, based on this selection.

New, Edit

Skipped rows Integer For file formats using Headers style, the number of rows skipped when reading the file. Specify a non-zero value when the file includes comments or other non-data information.

New, Edit

Skip row header Yes, No For file formats using the Headers style, indicates whether the first row of data in the file contains the column names and should be skipped when reading the file. The software uses this property in addition to the Skipped rows property.

When you select Yes, the software does not read data from the first row, and uses data in the first row to determine the file's column names.

New, Edit

Write row header Yes, No For file formats using Headers style, indicates whether to write column names in the first row of the output file.

New, Edit

Write BOM Yes, No For file formats using UTF-8 and UTF-16, determines the writing of BOM characters into the file. Choose Yes if you want to include BOM characters into a UTF-8 or UTF-16 file in which byte order is not otherwise defined. For a UTF-16 file, the software assumes the file to be UTF-16be, unless BOM characters are added by this property.

New, Edit

Beginning string in a file

Any character sequence, including a blank space, an empty string, or {none}

For file formats using BOF/EOF style, the string that marks the start of data in the file.

New, Edit

Ending string in a file

Any character sequence,

For file formats using BOF/EOF style, the string that marks the end of data in the file.

New, Edit




including a blank space, an empty string, or {none}

Custom Transfer options

Program executable

File name (Required) The name of the custom transfer program or its initialization script. For example: MyProgram.exe or MyProgram.cmd.

New, Edit

User name Any character sequence, including a blank space, an empty string, or {none}

(Optional) Log in ID for the server to which the custom transfer program connects. You may want to allow a custom program user to enter their user name when they enter their password in the software.

New, Edit

Password Any character sequence, including a blank space, an empty string, or {none}

(optional) Password for the server to which the custom transfer program connects. Passwords entered into this option are encrypted.

New, Edit

Arguments Any character sequence, including a blank space, an empty string, or {none)

(Optional) You can create arguments in your custom transfer program and then specifically flag them from within the software using this box. For example, you might have security or compression mechanisms in your program. You can also link connection data to your transfer program's flags.

New, Edit

Locale

Language The three-letter language abbreviations specified in the ISO 639-2/T standard

Specifies the human language (for example, Korean, Japanese, or English) in which data is stored or processed. Select from the displayed list.

New, Edit

Code Page The list of supported code pages. See Supported Locales and Encodings.

Specifies the sequence of bits that defines a character. For example, the Japanese code page contains ASCII, Greek, Cyrillic, and Japanese characters, thereby supporting the English, Greek, Russian, and Japanese languages.

New, Edit

Error Handling

Log data conversion warnings

Yes, No Determines whether to include data-type conversion warnings in the error log. The default is Yes.

New, Edit, Source

Log row format warnings

Yes, No Determines whether to include row-format warnings in the error log. The default is Yes.

New, Edit, Source



Maximum warnings to log

Integer greater than 0 or {no limit}

If Log data conversion warnings or Log row format warnings is enabled, you can limit how many warnings the software logs. The default is {no limit}.

New, Edit, Source

Capture data conversion errors

Yes, No Determines whether to capture data-type conversion errors when processing a flat-file source. The default is No.

New, Edit, Source

Capture row format errors

Yes, No Determines whether to capture row-format errors when processing a flat-file source. The default is Yes.

New, Edit, Source

Accepted bad records in file

Integer greater than 0 or {no limit}

Specify the number of bad records in a file that will be accepted.

Write error rows to file

Yes, No Determines whether to write invalid rows to an error file. The default is No.

New, Edit, Source

Error file root directory

Directory path or blank

If Write error rows to file is enabled, type the root directory in which to save the error file. For added flexibility, you can enter a variable for this option.

If you type a directory path here, then only enter the file name in the Error file name property.

If you leave Error file root directory blank, then type a full path and file name in the Error file name property.

New, Edit, Source

Error file name File name, file name including full path name, or blank

If Write error rows to file is enabled, type the name of the file in which to record the invalid rows.

For added flexibility, you can enter a variable that is set to a particular file with full path name. Use variables to specify file names that you cannot otherwise enter such as file names that contain multi-byte characters.

New, Edit, Source

The Fields work area in the File Format editor contains properties about the fields in the file format.

Property Possible values Description

Field name Any sequence of letters or numbers, not including blank spaces

A name that identifies data in this column. If your file format uses the Headers style and you select Yes for the Write row header property, the software writes the field names in the target file.

Data type blob, date, datetime, decimal, double, integer, interval, long, numeric, real, time, varchar

The data type of values in this column. The long data type is not available in fixed-width formats.

Format {none} For all data types other than varchar, specifies the format for this particular field. You can use this property to overwrite the default format. For example, if one date field is different than others, you can specify the different format here.



Property Possible values Description

Nullable Yes, No Indicates whether the the field will accept NULL values.

Description Any sequence of letters or numbers, not including blank spaces

Text that that specifies the type of data in a column. Typically use the field name, or a name similar to the field name. For example, if your field name is LastName, you may want to name the content type Family_Name.

If you delete a file format template from the object library, you must also delete all file sources and targets that are based on that file format template.

Related InformationUsing the File Format editor [page 51]

2.9 Target file properties

The following properties are available on the Target file properties tab in the Replication Job editor.

Option Possible values Description

General

File format Delimited The format of the data in the text file. Available properties change based on the selected file format type.

Adaptable Schema Yes, No Indicates whether the schema of a delimited file format is adaptable or fixed.

● Yes indicates that the schema is adaptable. The actual file can contain fewer or more columns than indicated by the file format. If a row contains fewer columns than expected, the software loads null values into the columns missing data. If a row contains more columns than expected, the software ignores the additional data.

● No indicates that the schema is fixed. The software requires the number of columns in each row to match the number of columns specified in the file format.The default is No. If you select Yes, you must ensure that the selected column and row delimiters do not appear inside the actual data.

Custom transfer program

Yes, No Enables the software to use a third-party file transfer program. Displays additional Custom transfer program options in the File Format Editor below the Input/Output properties.

Parallel process threads

Integer greater than 0, {none}, {default}

Specifies the number of threads for parallel processing. Parallel processing can improve performance by maximizing CPU usage on the Job Server computer. For example, if you have four CPUs,



enter 4 for this option. For more information, see “File multi-threading” in the Performance Optimization Guide.

For jobs that process USPS certification tests, the value should be set to {none}.

Data File

Location Local, Job Server During design, indicates the files are located on the local machine or on the machine that runs the Job Server. If you select Job Server, you must enter the absolute path to files. Remember that UNIX systems are case-sensitive.

During execution, all files must be located on the Job Server machine that executes the job. If you use different files to design your job, change the file specified (through the Root directory and File properties) before you execute the job.

Root directory Path name for the file

The parent directory for the specified file name. If the directory name is entered, the file will be generated there and the file name format will be: <jobname_tablename>.<extension>. If the directory is blank, the file will be generated under <DS_COMMON_DIR>/workspace/.

File Extension File extension The file extension to be applied to all loader files in the replication job.

Delimiters

Column Tab, Semicolon, Comma, Space, or any character sequence

For delimited file formats, the character sequence that indicates the end of one column and the beginning of the next.

Row {new line}, {Windows new line}, {Unix new line}, or any character sequence

A character sequence that indicates where one row of data ends and the next begins.

Row within text string

Character, Row delimiter

Defines how the row delimiter is interpreted within a text string.

● Character: The specified row delimiter is treated as a character within the text string.

● Row delimiter: The specified row delimiter is interpreted and defines rows within the text string.

Text Single quotation marks ('), double quotation marks ("), or {none}

Denotes the start and end of a text string. All characters (including those specified as column delimiters) between the first and second occurrence of this character is a single text string. The treatment of the row characters is defined by the Row within text string setting.




NoteData in columns cannot include the column or row delimiter, unless you also specify a text delimiter. For example, if you specify a comma as your column delimiter, none of the data in the file can contain commas. However, if you specify a comma as the column delimiter and a single quote as the text delimiter, commas are allowed in strings in the data.

You can use any ASCII characters (including non-printing characters) for column and row delimiters.

You can specify an ASCII character by entering a forward slash (/) followed by the decimal representation of the character. For example, to use Y umlaut (Ÿ) as a delimiter, enter /255 in the delimiter property box.

Default Format

Escape character Any character sequence, or {none}

A special character sequence that causes the software to ignore the normal column delimiter. Characters following the escape character sequence are never used as column delimiters.

For example, suppose you specify a forward slash as the escape character and a comma as the column delimiter. Then, you must have a forward slash to have a comma appear inside a field.

NULL indicator {none} or any other character sequence

A special character sequence that the software interprets as NULL data.

The software ignores any NULL indicator specified in the file format for blob columns.

Ignore row marker(s)

{none} or any other character sequence

A character sequence, which when found at the beginning of rows, causes the software to ignore the row when reading the file or automatically creating metadata. To enter multiple character sequences, separate each with a semi-colon. To include a semi-colon or backslash as a marking character, precede with a backslash.

Date format yyyy.mm.dd


The date format for reading or writing date values to and from the file.

Time format hh24:mi:ss


The time format for reading or writing time values to and from the file.

Date-Time format yyyy.mm.dd hh24:mi:ss


The datetime format for reading or writing datetime values to and from the file.

Validate decimal data

Yes, No Perform decimal validation and formatting on output.



Input/Output

Style Headers or BOF/EOF

The format of the start and end of the file. Available properties in the Input/Output property group may change, based on this selection.

Skipped rows Integer For file formats using Headers style, the number of rows skipped when reading the file. Specify a non-zero value when the file includes comments or other non-data information.

Write row header Yes, No For file formats using Headers style, indicates whether to write column names in the first row of the output file.

Write BOM Yes, No For file formats using UTF-8 and UTF-16, determines the writing of BOM characters into the file. Choose Yes if you want to include BOM characters into a UTF-8 or UTF-16 file in which byte order is not otherwise defined. For a UTF-16 file, the software assumes the file to be UTF-16be, unless BOM characters are added by this property.

BOF marker Any character sequence, including a blank space, an empty string, or {none}

For file formats using BOF/EOF style, the string that marks the start of data in the file.

EOF marker Any character sequence, including a blank space, an empty string, or {none}

For file formats using BOF/EOF style, the string that marks the end of data in the file.

Custom Transfer options

Custom transfer program executable

File name (Required) The name of the custom transfer program or its initialization script. For example: MyProgram.exe or MyProgram.cmd.

Custom transfer user name


(Optional) Log in ID for the server to which the custom transfer program connects. You may want to allow a custom program user to enter their user name when they enter their password in the software.

Custom transfer password


(optional) Password for the server to which the custom transfer program connects. Passwords entered into this option are encrypted.

Custom transfer arguments

Any character sequence, including a blank space, an

(Optional) You can create arguments in your custom transfer program and then specifically flag them from within the software using this box. For example, you might have security or




empty string, or {none)

compression mechanisms in your program. You can also link connection data to your transfer program's flags.

Locale

Language The three-letter language abbreviations specified in the ISO 639-2/T standard

Specifies the human language (for example, Korean, Japanese, or English) in which data is stored or processed. Select from the displayed list.

Code Page The list of supported code pages. See Supported Locales and Encodings.

Specifies the sequence of bits that defines a character. For example, the Japanese code page contains ASCII, Greek, Cyrillic, and Japanese characters, thereby supporting the English, Greek, Russian, and Japanese languages.

2.10 Monitoring editor

The Workbench Monitoring editor opens in the workspace when you click the icon in the toolbar or select it from the Tools menu.

The monitoring editor consists of several areas:

Area Description

Job selection toolbar Displays the Data Services jobs available in the repository to which the Workbench is connected and allows you to execute them.

NoteThe monitoring editor displays all standard Data Services jobs in the repository, not Workbench-specific replication jobs.

Execution history pane Displays the execution history for the selected job. Each line indicates the current status, execution date, and duration of a single job execution instance.

Execution history dashboard pane

Displays a graphical representation of the history for the selected job. You can change the type of chart by choosing from the drop-down box in the upper-right corner of the pane.

Execution details pane Displays the details for the individual job execution instance selected in the execution history pane. You can view the trace, monitor, and error logs for the execution instance, as well as the execution statistics.

You can adjust the size of each pane by dragging the borders between them. In addition, you can hide or reveal the dashboard and details panes by clicking the arrows in the center of their borders.


2.11 Preferences

The Preferences options allow you to Customize several areas of Data Services Workbench.

2.11.1 Data Services

From the main Data Services page, specify the Maximum number of errors to show in tooltips.

2.11.1.1 Data Flow Detections

The Data Flow Detection options allow you to customize the way detection algorithms work, for example, setting the minimum scores to display in the Detect Mapping and Detect Join dialogs.

Mappings detection

Option Description

Minimum score threshold Select the minimum mapping score to display. Scores at or above this setting are displayed in the Mapping detection dialog.

Enable mapping detection in create Query context menu

Check to enable mapping detection when creating a query transform from the context menu of a multiple selection of source elements of the data flow and of one Loader.

Minimum score threshold Select the minimum mapping score to display. Scores at or above this setting are displayed in the Mapping detection dialog. This option only applies to the Enable Mapping Detection option in the create query context menu.

Always confirm mappings Select to always confirm mappings when creating a new Query transform. This option applies to the Enable Mapping Detection option in the create query context menu.



Joins detection

Option Description

Minimum score threshold Select the minimum join score to display. Scores at or above this setting are displayed in the join detection dialog.

Enable join detection in create Query context menu Check to enable mapping detection when creating a query transform from the context menu of a multiple selection of source elements of the data flow and of one loader.

Minimum score threshold Select the minimum mapping score to display. Scores at or above this setting are displayed in the Mapping detection dialog.

Always confirm joins Select to always confirm joins when creating a new query transform. This option applies to the Enable Join detection option in the create query context menu.

2.11.1.2 Data Flow editor

Data Flow preferences allow you to customize various settings of the Data Flow editor, including the way that the Auto-arrange option arranges the elements in the Data Flow diagram (for example, minimizing spacing between elements).

Option Description

Top Margin Specify the top margin spacing.

Left Margin Specify the left margin spacing.

Horizontal Margin Specify the horizontal margin spacing.

Vertical Margin Specify the vertical margin spacing.

Horizontal Alignment Specify the horizontal alignment:

● Left Aligned● Center Aligned● Right Aligned

Automatically put names in edit mode when an element is dropped in the diagram

Check this option place the field names in edit mode when they are dropped into the diagram.

2.11.1.3 Data preview

Data preview preferences allow you to customize the way Data preview opens and to change the maximum number of rows and kilobytes to retrieve.


Option Description

Max Rows to retrieve from Tables Specify the maximum number of rows to retrieve from the database.

Max Size (in KB) to retrieve from Files Specify the maximum number of kilobytes to retrieve from the files.

Data Preview opening mode Specify the viewing method:

● In an Editor● In a View● In a Dialog

2.11.1.4 Datastore editor

The Datastore editor preferences page allows you to specify whether to import the row count of the table when importing the metadata and select the table and column information you want to display.

Option Description

Automatically detect row count when importing tables (might be slower)

Specify whether to import the row count of the table when importing the metadata. If selected, the row count will appear in the table properties and the project explorer.

Tables:

Prefix tables with owner Check to prefix tables with owner.

Show column count Check to show column count.

Show row count Check to show row count.

Show row size Check to show row size.

Show description Check to show the table description.

Columns:

Show datatype Check to show the column datatype.

Show description Check to show the column description.

2.11.1.5 Expression editor

Customize the Expression editor by setting the Content assist and Syntax coloring options.



Content assist options

Option Description

Insert single proposals automatically Select this option to automatically insert single proposals.

Insert schema qualified column names Select this option to schema qualified column names.

Enable auto activation Select this option to enable automatic activation.

Activation delay (ms) Enter the activation delay in milliseconds. Default value is 200.

Activation triggers Select the activation triggers you want to include:

'.' to propose qualified names

'$' to propose variables

'[' to propose substitution parameters

Syntax coloring

Use the syntax coloring options to set the color and fonts of the expression editor syntax.

2.11.1.6 Expression macros

These options allow you to manage the expression macros (creation, edition, and deletion) and import and export macros from this screen.

Expression macro listing

Button/option Description

New folder Create a new folder.

New macro Create a new expression macro.

Delete Delete selected expression macro(s).

Move up Move selected macro up.

Move down Move selected macro down.


General tab

Option Description

Description Describes the use for selected function.

Expression Enter the expression macro you want to add.

Conditions tab

Option Description

Restrict to following datatypes Check to only apply the expression macro to select datatypes. Select the datatypes from the dropdown list.

Restrict to following mapping types Check to apply the expression macro to select mapping types. Select mapping types from the dropdown list.

Restrict to the following content types Check to only apply expression macro to select content types. Select the content types from the dropdown list.

Restrict to the following field name pattern Check to restrict a specific field name pattern. Select the field name pattern from the dropdown list.

Other options/buttons

Option/button Description

Import Import the expression macros from file.

Export Export the expression macros to a file.

Restore defaults Restores the expression macros back to the defaults included with Data Services Workbench.

2.11.1.7 File Format editor

The File Format editor options allow you to specify the number of kilobytes to scan during Data Preview and the detection algorithms of files.



Option Description

Maximum file size to use for the detect fields Specify the number of kilobytes to scan during the detection algorithms of files.

Maximum file size to use for Data Preview Specify the number of kilobytes to scan during data preview.

2.11.1.7.1 Detection settings

The Data Type detection options affect the data types detected when creating a file format from an existing file and detecting data types. If a field's values match one of the listed formats, Workbench will assign the corresponding data type to the field: Date, Time or DateTime. Workbench attempts to detect the formats in the order they appear in the list for each format.

Option Descriptioin

DateTime Formats Add, edit, delete, or reorder DateTime formats.

Date Formats Add, edit, delete, or reorder Date formats.

Time Formats Add, edit, delete, or reorder Time formats.

Locales Add, edit, delete, or reorder Locales.

The configured locales can affect the detection of decimal data types if thousands-separators or decimal-separators are used. For example, 10.237.329,95 will not be recognized as a decimal (10,2) unless a locale that is configured with appropriate decimal characters (German) is included in the list.

Null Values Add, edit, delete, or reorder null values.

2.11.1.8 Project explorer

These options allow you to fine-tune the way Data Services Workbench project files are displayed in the project explorer. For example, you can specify how to group the tables of the Datastore file.

Table 1: GeneralOption Description

Show file extensions Check to show file extensions.

Table 2: Data storesOption Description

Show table count Check to show the table count.


Option Description

Show database count Check to show the database type.

Show description Check to show the datastore description.

Table 3: TablesOption Description

Show columns count Check to show the number of columns.

Show table type Check to show the table type.

Show row count Check to show the table row count.

Show description Check to show the description.

show row size Check to show the row size.

Table 4: Tree structureOption Description

Group table by type Check to group tables by type.

Group table by owner Check to group tables by owner.

Enable table chunking Check to enable table chunking.

Chunk size If Enable table chunking is selected, enter the number of tables.

Table 5: ColumnsOption Description

Show data type Check to show the data type.

Show description Check to show the description.

2.11.2 General

The Preferences section allows you to set your preferences for the following:

● General● Appearance● Content Types● Editors● Keys● Perspectives● Workspace



2.11.3 Languages

Enter the product languages you want to use in Workbench. To apply language changes you must restart Workbench.


3 Database MigrationMigrating data and schema information between different database systems is a complex task that can take days or even months depending on the type of database systems, the complexity of the database schema, and the size of the data set being migrated. In addition, incompatibilities between the source and target database types could require manual schema and data corrections.

The Data Services Workbench simplifies this database migration process considerably. In the Workbench, you provide connection information for the source and target databases and select the tables that you want to migrate. The Workbench automatically creates Data Services jobs, work flows, and data flows and imports them into a Data Services repository.

After you have created the replication jobs, you can use the Workbench to execute the jobs and monitor their progress. If required, you can also edit the generated data flows in the Designer, and use other Data Services tools to schedule and execute the jobs.

3.1 Migrating data and schema information

The process of migrating data and schema information from one database to another with the Workbench generally consists of a few basic steps:

1. Create a project in the project explorer.2. Create a source datastore definition.3. Create a target datastore definition.4. Create a replication job using your source and target datastores.

a. Select the source objects that you want to migrate.b. Group the target objects and specify the order of migration.c. Adjust the target schema as required.d. Specify the replication behavior for the job.

5. Deploy and execute the replication job.

You can choose to perform the datastore and replication job definition steps manually with the standard individual datastore and replication job editors, or you can complete the process using the quick replication wizard. If you use the replication wizard, you can modify your replication job by using the standard editors after the wizard completes.

Related InformationProject explorer [page 17]Datastore editor [page 20]Replication job editor [page 15]

3.1.1 Creating a Workbench project

If you want to create a new project, start the new project wizard in one of the following ways:


Database Migration

● Right-click inside the project explorer and choose New Project .

● Choose New Project from the File menu.

● Click the New toolbar icon. Choose General Project and click Next.

1. Specify the name for the new project and click Next.

NoteThe project name can contain only alphanumeric characters and the underscore character.

2. If you want to save the files for the project to a different location, uncheck Use default location and specify a location for the project files.

3. Select the Data Services repository.4. Click Finish to create the project.

The new project is displayed in the project explorer.

You can now add Workbench objects such as datastores and replication jobs to your project.

3.1.2 Using the quick replication wizard

You can use the quick replication wizard to easily migrate source data to one of the supported target databases or files.

1. Click the Quick replication wizard icon in the toolbar.The quick replication wizard is displayed.

2. Specify the name for the new replication project and click Next.


The source datastore configuration screen is displayed.3. Specify the settings for the source datastore and click Next.

a) Select the datastore type.The available configuration settings for the selected database or application type are displayed.

b) Set the datastore configuration settings as required for your database or application.

NoteRequired settings are marked with an asterisk (*), and required settings that are missing a value are highlighted in red in the Property name column. Settings that have changed from the default value are listed in bold.

c) Click Test connection to verify that the connection settings are valid.If the connection test fails, correct the datastore connection settings and re-test the connection.

Workbench GuideDatabase Migration © 2013 SAP AG or an SAP affiliate company. All rights reserved. 41

NoteThe connection test is between the database or application and the Central Management Console (CMC) host, not the Workbench host. Ensure that the connection settings between the CMC and datastore database or application are valid.

The source object selection screen is displayed.4. Select the objects that you want to replicate to the target datastore or files and click Next.

The configure target settings screen is displayed.5. Specify the settings for the target datastore or files and click Next.

a) Select a target type of Files or Datastoreb) If the target type is Datastore, select the datastore type.

The available configuration settings for the selected database or application type are displayed.c) Set the configuration settings as required for your database, files, or application.


d) When you select datastore as the target type, cick Test connection to verify that the connection settings are valid.If the connection test fails, correct the datastore connection settings and re-test the connection.

The repository and job server selection screen is displayed.6. If you want to execute the replication job without making any further changes, select Execute replication job

now.a) In the left column, select the Data Services repository to use.

The jobs generated by the Workbench will be stored in this repository. By default, the first repository in the list is selected.

b) In the right column, select the Data Services Job Server to use to execute the replication job.By default, the Job Server associated with the selected repository is used.

7. Click Finish to save the replication job and and execute it with the selected settings, if applicable.If you chose to execute the replication job, the monitoring editor is displayed.

The Workbench saves the replication job and datastore definitions to the local file system. If you chose to execute the replication job now, it also creates Data Services jobs and datastores for the replication job, imports them to the selected repository, and executes them using the selected Job Server.

3.1.3 Using the Datastore editor

The datastore editor provides a powerful interface that allows you to fine-tune your datastore definition, including importing database tables, inserting or modifying columns, keys, indexes, and partitions, modifying attributes, and so on.


Database Migration

TipThe datastores defined and used in the Workbench are standard Data Services datastores. For more information about Data Services datastores, see the Reference Guide.

If you have already created a datastore, open it in the datastore editor by double-clicking the datastore name in the project explorer.

If you want to create a new datastore definition, start the new datastore wizard in one of the following ways:

● Right-click your project in the project explorer and choose New Datastore .

● Select your project in the project explorer and choose New Datastore from the File menu.

● Select your project in the project explorer and click the New toolbar icon. Choose Data ServicesDatastore and click Next.

1. Specify the name for the new datastore and click Next.

NoteThe datastore name can contain only alphanumeric characters and the underscore character.

2. Specify the settings for the datastore and click Next.a) Select the datastore type.

The available configuration settings for the selected database or application type are displayed.b) Set the datastore configuration settings as required for your database or application.


c) Click Test connection to verify that the connection settings are valid.If the connection test fails, correct the datastore connection settings and re-test the connection.

The object selection screen is displayed.3. Select the objects that you want to import to the datastore definition and click Finish.

The new datastore is opened in the datastore editor.

You can now use the datastore editor to further customize your datastore definition. For example, you can select a table or view and modify its properties, columns, keys, and so on, or use the configurations section to add additional configurations to your datastore.Related InformationDatastore editor [page 20]Supported data migration sources and targets [page 62]Reference Guide: Objects, Descriptions of objects, Datastore Datastore editor [page 34]


3.1.4 Using the replication job editor

The replication job editor provides a powerful interface that allows you to fine-tune your replication job, including filtering datastore tables, customizing target replication groups, modifying table and column mappings, and so on.

NoteThe replication jobs defined and used in the Workbench are defined in Workbench-specific metadata. When you deploy the replication job to a Data Services repository, the Workbench uses this metadata to create standard Data Services objects such as job, work flows, and data flows. For more information about Data Services objects, see the Reference Guide.

If you have already created a replication job, open it in the replication job editor by double-clicking the replication job name in the project explorer.

If you want to create a new replication job, start the new replication job wizard in one of the following ways:

● Right-click your project in the project explorer and choose New Replication Job .

● Select your project in the project explorer and choose New Replication Job from the File menu.

● Select your project in the project explorer and click the New toolbar icon. Choose Data ServicesReplication job and click Next.

1. Specify the name and description for the new replication job and click Finish.

NoteThe replication job name can contain only alphanumeric characters and the underscore character.

The new replication job is opened in the replication job editor.2. Select a Replication Target Type:

○ Datastore○ Files

3. Do one of the following:

○ If you selected Datastore, Click ... to specify the source and target datastores.

A list of available datastores is displayed. If you have not already created the datastores, you can define the datastores later.

○ If you selected Files, set the target file properties. You must enter the root directory.

You can now use the replication job editor to further customize your replication job definition. For example, you can change the replication behavior, modify the target schema, customize column mappings, and so on.Related InformationReplication job editor [page 15]Reference Guide: Objects


Database Migration

3.1.4.1 Modifying replication job properties

When you open a replication job or select the root replication job node in the target datastore pane, the configuration area displays several tabs that you can use to configure the way that the Workbench processes the replication job:

Tab Description

Properties Replication target type of Datastore: Displays the replication target type, name, owner, replication behavior and description for the replication job.

Replication target type of Files: Displays the replication target type, name, description and target file properties.

Variables Displays information about the variables available for use in the replication job.

You can add or delete variables, as well as alter variable properties such as data type, length, precision, scale, and default value.

Scripts Displays the pre-load and post-load scripts for the replication job.

You can modify the scripts directly in the text box or by clicking Edit to launch the Workbench expression editor. You can also validate the scripts.

Related InformationUsing the expression editor [page 63]

3.1.4.1.1 Replication behavior

When you deploy and execute a Workbench replication job, the Workbench automatically generates the ETL metadata for the job and deploys it to a Data Services repository. The metadata that is generated depends on the replication behavior setting for the replication job.

The following replication behavior options are available:

● Schema and dataIn the schema and data scenario, the Workbench assumes that the tables do not exist in the target database and need to be created. As a result, the ETL job metadata that is generated includes the following objects:

○ Source and target datastores○ Source and target tables○ A Data Services job that contains both the script to create the target tables, as well as the work flows and

data flows that move data from the source tables to the target tables.● Data only

In the data-only scenario, the Workbench assumes that the tables already exist in the target database. As a result, the ETL job metadata that is generated includes the following objects:

○ Source and target datastores


○ Source and target tables○ A Data Services job that contains work flows and data flows that move data from the source tables to the

target tables.

NoteSchema and data is the default replication behavior.

3.1.4.2 Adjusting the target schema

3.1.4.2.1 Modifying target table properties

When you select a table in the target datastore pane, the configuration area displays several tabs that you can use to configure the way that the Workbench processes that table:

Tab Description

Properties Displays the table name, owner, and description and provides the ability to view the data in the source table.

Columns Displays information about the target table columns.

After the table has been imported, you can insert and delete columns from the target table. You can also change column names, data types, descriptions, and so on, as well as alter the column mappings with the expression editor.

Foreign Keys Displays information about the table's primary and foreign keys, if any.

After the keys have been imported, you can add, delete, and modify them.

Indexes Displays information about the table's indexes. The primary index is listed first and followed by any secondary indexes.

You can add or remove indexes, as well as add or remove columns from existing indexes.

Options Displays options that Data Services uses to read the source table and load the target table.

For more information about the available source and target options, see the Reference Guide.

NoteOnly options used by the Workbench are displayed.

Filter data Allows you to filter the rows of the source table.

You can use basic filter editor or the expression editor to modify the filter.


Database Migration

Tab Description

DDL Options Allows you to specify the behavior of DDL generation.

For example, whether to create foreign keys or indexes.

Delta load Displays options that modify the delta load behavior for a table.

For more information about the available delta load options, see Delta load migration [page 53].

Related InformationReference Guide: Objects, Descriptions of objects, Source Reference Guide: Objects, Descriptions of objects, Target

3.1.4.2.2 Grouping target tables

Many data warehouse tables have referential integrity constraints such as primary key and foreign key relationships. The primary key table needs to be loaded before the foreign key table if there is a relationship between two tables.

You can avoid referential integrity issues by using the replication job editor to group tables. For example, you can put the primary key tables in the first group and foreign key tables in the second group. In another scenario, you could put dimension tables in the first group and fact tables in the second group.

By default, tables are added to the Default_Group that is created in a new replication job. However, there is no limit to the number of replication groups supported in a Workbench replication job.

To group tables in the replication job editor:

1. Create a new replication group in the target schema.a) Right-click on the replication job root node in the target schema pane and choose New replication group.

A new replication group is added to the target schema.b) Assign a meaningful name to the replication group by highlighting the new replication group and clicking it

again.2. Assign tables to the new group by dragging them from the source pane to the replication group.

NoteA table may belong to only one group in the target schema. You can create multiple target tables from a single source table, but each target table must be named uniquely.

3. Use the up and down arrows in the target schema to order the replication groups as needed for your requirements. When the replication job is executed, the groups will be migrated starting from the top of the list.You can also rearrange replication groups by dragging them within the target schema.

TipTo automatically order the tables in the target schema by their dependencies, click Auto order.


NoteReplication groups can be sequential or parallel. In a sequential group, the data flows generated by the Workbench will be executed in sequential order. In a parallel group, the data flows are executed in parallel. To change the group type, right-click on the group and choose Parallel or Sequential from the Change replication group type menu.

3.1.4.2.3 Changing column data types

Data Services supports conversion to and from its internal data types and performs data type conversions when it imports metadata from external sources or targets into the repository, as well as when it loads data into an external table. When the Workbench creates target tables, the software converts from internal data types to the data types used by the target database.

For more information about internal data type conversions in Data Services, see the Reference Guide.

The Workbench allows you to overwrite the default data conversion rules. To change the data type for a column:

1. Expand the table containing the column in the target schema pane.2. Select the target column.

The column details are displayed in the configuration pane.3. Select the new data type for the column from the Data type drop-down box.

If required for the new data type, set the Length, Precision, and Scale for the column.

Related InformationReference Guide: Data Types, Data type conversion, Conversion to or from internal data types

3.1.4.3 Filtering datastore tables

In a replication job, you might want to migrate only a subset of the rows from a particular source table to your target datastore.

To migrate a subset of table rows, use a filter on the datastore table in the replication job editor:

1. Select the table that you want to filter in the target schema pane.2. Click the Filter data tab in the configuration pane.3. For basic filtering, use the table filters configuration window.

a) Select Basic and click Edit filter.The table filters configuration window is displayed.

b) Drag the columns you want to use to filter from the columns pane to the filters pane.The columns are displayed in the filters panel and grouped together with a logical AND or OR grouping operator.

c) For each column, define the filter logic by specifying the comparison operator and value.


Database Migration

TipYou can create more complex, nested filter expressions by dragging the columns on top of each other. The nesting layout is displayed in a graphical preview.

d) Change the grouping logic as required by double-clicking the grouping operator.e) To remove a filter, click the column or grouping operator to highlight the filter and click the Remove

selected filter button.f) When you are satisfied with the filter, click OK.The filter is displayed in the Filter data tab.

4. For more advanced filtering, use the Workbench expression editor to define the filter.a) Select Advanced and click Edit filter.

The expression editor is displayed.b) Complete your filtering expression and click OK.

The filter expression is displayed in the Filter data tab.c) To validate your filter expression, click Validate.

Related InformationUsing the expression editor [page 63]

3.1.5 Using the Data Flow editor

The Data Flow editor enables you to create data flows and include them in the replication job. You also have the ability to convert replication tables to data flows. Highlight multiple replication tables, right-click the selection, and select convert data. You have the option to generate a single data flow or one data flow per replicated table.

If you have already created a data flow, open it in the Data Flow editor by clicking the data flow name in the project explorer.

To create a new data flow, do one of the following:

● Select your project in the project explorer and right-click your data flow in the project explorer and choose New Data flow .

● Select your project in the project explorer and choose New Data Flow from the File menu.

● Select your project in the project explorer and click the New toolbar icon. Choose Data Services Data Flow and click Next.

1. Specify the name for the new data flow and click Next.

NoteThe data flow name can contain only alphanumeric characters and the underscore character.

2. Enter the description for the new data flow and click Finish.

The new data flow is opened in the Data Flow editor.Related InformationData Flow editor [page 17]Expression macros [page 50]


Reference Guide: Objects, Descriptions of objects, Data Flow Data Flow editor [page 33]

3.1.6 Converting replication tables to data flows

To convert a replication table to a data flow:

1. Highlight the replication tables you want to convert.2. Right-click the selection.3. Select one of the following:

○ Generate a single data flow○ One data flow per replicated table

3.1.7 Expression macros

You have the option to apply expression macros to multiple fields at one time when working in the Query transform of a data flow. You can access the Workbench Preferences options from the Query transform. This enables you to create new expression macros and apply them to the fields as needed.

3.1.7.1 Creating an expression macro

To create a new expression macro, access the Preferences Expression macros options one of the following ways:

● From the Data Flow Mappings tab, right-click a field and choose Apply expression macro Manage expression macros .

● Choose Preferences from the Window menu and select the Expression Macros option.

1. Click the New macro button and enter the macro name.2. From the General tab, enter a Description for the macro.3. From the General tab, enter the Expression.4. From the Conditions tab, select any conditions you want to apply to the expression macro.

Related InformationExpression macros [page 35]

3.1.7.2 Applying an expression macro to multiple fields

To apply an expression macro to multiple fields at one time:

1. Select the fields you want to include.


Database Migration

2. Right-click the highlighted fields and select Apply expression macros.3. Select one of the expressions from the list or select Manage expression macros to add a new expression

macro.

3.1.8 Using the File Format editor

The File Format editor allows you to set properties for file format templates and source and target file formats.

If you have already created a file format, open it in the File Format editor by clicking the file format name in the project explorer.

If you want to create a new file format, start the File Format wizard in one of the following ways:

● Select your project in the project explorer, right-click and choose New Flat File Format .

● Select your project in the project explorer and click the New toolbar icon. Choose Data Services Flat File Format and click Next.

3.1.8.1 Manually create a new file format

To manually create a file format with default properties:

1. Select Manually create a new File Format and click Next:2. Specify the name and description for the new file format and click Finish.

NoteThe name can contain only alphanumeric and underscore characters.

The new file format is opened in the File Format editor.

Now you can further define the properties for the file format templates using the the File Format editor.

3.1.8.2 Automatically import file formats

This option enables you to import multiple files with different formats at the same time and generate multiple file formats. You have the option to auto-detect or specify the file formats.

To automatically import file formats from existing files:

1. Select Automatically import File Formats from existing files and click Next:2. Under File location, select Local or Job Server and click Add files.3. Select the file(s) you want to add and click Next.4. Do one of the following:


○ Click Next to set the detection options.○ Click Finish to run auto-detection.

5. If you chose to set detection options, set the detection options and click Finish.

The new file format is opened in the File Format editor.

Now you can use the File Format editor to further define the properties for the file format templates.

3.1.9 Deploying and executing the replication job

When you are satisfied with your replication job configuration, you can deploy it to a Data Services repository and execute it.

1. Double-click the replication job in the project explorer to open it in the replication job editor.2. Click Validate to verify that the replication job is free of errors.3. If you want to deploy the job but not execute it, click Deploy.

a) Select the repository where you want to deploy the replication job and click OK.A progress bar indicating the deployment progress is displayed.

A message indicating the success or failure of the deployment is displayed. If any errors occur during deployment, the message includes any details available.

4. If you want to deploy and execute the job immediately, click Execute.a) Select the repository where you want to deploy the replication job and click Next.

The job execution parameters screen is displayed.b) Specify whether to execute the job as an initial load or delta load.c) Specify the job server and other execution options to use when executing the job in the Execution options

tab.d) If you want to include additional information in the trace log, change the values in the Trace tab to Yes as

required.e) Click Finish to deploy and execute the job.

A progress bar indicating the deployment progress is displayed. If you want to halt the deployment, click the Cancel operation button next to the progress bar.

The monitoring editor for the job is displayed.

Related InformationMonitoring editor [page 31]

3.1.9.1 Error recovery

If an error occurs while your replication job is being executed, only some rows may be inserted into a target table.

There are several methods that you can use to prevent duplicate rows from being inserted into the target table, including:

● Data flow designYou can design the replication job to completely replace the target table each time the job is executed. You might choose to use this method if the number of rows changed in the target table is a significant percentage


Database Migration

of the target table. You can also use techniques such as bulk loading options to improve the overall performance of the replication job.

● Auto-correct load target table optionYou can use the auto-correct load option to check the target table for existing rows before new rows are added to the table. However, this option can needlessly slow jobs that are not executed in recovery mode. You might choose to use this method if the number of rows changed in the target table is a relatively small percentage of the target table.

● Pre-load SQL commandsYou can use pre-load SQL commands to remove partial database updates that have occurred during an incomplete job execution. Typically, the pre-load SQL command deletes rows based on a variable set before the partial insertion began.

3.2 Delta load migration

Delta load jobs move rows that have been added or modified since the last time the job was executed. There are many reasons to implement a delta load job, but the most common is to reduce the time the loading process takes. Instead of loading millions of rows each time the job is run, you can process only the few that have changed. Another reason could be to maintain historical data; you might want to keep the old data in your data warehouse and add the current state so that you can see the changes over time.

3.2.1 Delta load jobs in the Workbench

When you create a replication job that has a table configured with a delta load method, the Workbench automatically creates two sets of metadata: a first load job and a delta load job. The first load job performs a full replication, with any user-defined filters applied, from the source datastore to the target datastore. The delta load job can be used to capture only changed data after the first load.

The Workbench creates two scripts within the delta load job: the pre-load script and the post-load script. These scripts keep track of the time each job is executed so that the next execution can continue from where the last one ended. You can customize the pre-load and post-load scripts in the Scripts property tab for the replication job.

To help keep track of the delta job executions, the Workbench maintains an internal table in the target datastore that is created after the successful execution of the first load job. Because of this, the full replication job must be successfully executed at least once before the delta load job can be executed.

NoteBy default, all target tables within the Workbench delta load job are created with the Auto correct load option enabled. For more information about this option, see the “Target tables” section in the Reference Guide.

NoteThe delta load mode is disabled in the replication job execution if the selected target type is Files.


Delta job execution

During execution, the delta job first determines the last time that it was executed, if ever. It uses the last execution time in a filter within the data flows to select only those records that are new. After the data flows have completed successfully, the delta job updates the internal metadata with the current time, so that any future delta execution starts from that time.

Related InformationReference Guide: Objects, Descriptions of objects, Target tables

3.2.2 Delta load options for tables

For each table, the Workbench supports several delta load options:

NoteThe delta load option will be disabled when a target type of Files is selected. In this case, only the No delta load option will be enabled.

● No delta loadThe Workbench skips this table in the delta load.

● Reload the full tableThe Workbench performs a complete refresh of the table.

● Use timestamp columnThe Workbench uses a timestamp column or columns from the table to perform the delta load. Data Services uses the timestamp from each load to extract only new data from after the last execution.If a table has more than one datetime or timestamp column, you must choose the column to use for the delta load process. If a table does not contain a datetime or timestamp column, this option is disabled. If you use a column that contains only the date, you can also choose to use an additional column that specifies the time.

● Use CDCThe Workbench uses the changed data capture (CDC) functionality provided by a supported SAP application to extract only the data that has changed.If the datastore does not support CDC functionality, this option is disabled.

● Use custom filterThe Workbench uses a user-defined filter to determine the start time used to perform the delta load.

TipIn the Workbench, the delta load job works best with tables that use a timestamp column as the primary key.

To change the delta load method for a table, right-click the table in the target schema pane and choose the delta load method under Change delta load support. You can also select the table in the target schema pane and choose the delta load method in the Delta load tab of the replication job editor.


Database Migration

3.2.3 Delta load options for SAP applications

For applications that support changed data capture (CDC) functionality, the Workbench automatically extracts changed data for the delta load.

You can use the Workbench-generated variable $END_TIME to specify the time frame for which to extract changed data:

● If the value of $END_TIME is not specified, the Workbench uses the current system time for its value. Each delta load will extract new data from the last execution to the current date-time.

● If you want to continue from a previous execution, retrieve the timestamp of the execution you want to recover from your application and assign it to $END_TIME in the replication job pre-load script.

NoteThe Workbench treats applications that do not support CDC as tables for the delta load.


4 Using an SAP Information Steward Data Cleansing SolutionThe Workbench Data Cleansing Solution feature allows a technical ETL developer with no data quality expertise to create a data flow containing a data cleansing solution and validate the results in a production environment.

First, a data steward uses the Data Cleansing Advisor feature of SAP Information Steward to create and publish a data cleansing solution. The data cleansing solution is stored in the SAP BusinessObjects Business Intelligence platform's Central Management Server (CMS). In the Workbench, the ETL developer then models the data cleansing solution as a Data Cleansing Solution transform within the context of a data flow, and maps the transform's input and output schema to production sources and targets, and validates the results in the production environment. Finally, the ETL developer generates and deploys the data flow as ATL to the Data Services repository for execution in the Data Services engine.

For more information about data modeling in the Workbench, see the SAP Data Services Workbench Guide. For more information about input and output fields and options, see the SAP Data Services Reference Guide. For more information about Data Cleansing Advisor, see the SAP Information Steward User Guide.

4.1 Viewing data cleansing solutions

You can create multiple data flows that use a data cleansing solution. Each data flow can contain one or more unique instances of the Data Cleansing Solution transform that references the data cleansing solution.

The Data Cleansing Solution transform references the latest published version of the data cleansing solution from Data Cleansing Advisor. If a data cleansing solution is updated and republished in Data Cleansing Advisor, Workbench automatically uses the updated solution, and you must redeploy the data flow to the Data Services repository.

1. Select Window Data Cleansing Solutions . The Data Cleansing Solutions tab opens in the workspace and displays the data cleansing solutions that have been published in the Data Quality Advisor.The Published column contains the date and time that the solution was published. Workbench displays only the latest active version of a data cleansing solution published in the Data Cleansing Advisor. Icons display the status of the data cleansing solution.

Icon Description

Indicates a new data cleansing solution that has not been used.

Indicates a current data cleansing solution that has been used in one or more data flows.

Indicates an updated data cleansing solution that has been republished since it was used.

2. To refresh the list of data cleansing solution, click the refresh icon ( ).3. Click a column header to sort the data cleansing solutions by the content in that column.4. To view more details about and identify the appropriate data cleansing solution, double-click a solution.


Using an SAP Information Steward Data Cleansing Solution

The Data Cleansing Solution Details window opens and displays information about the solution such as the publication date and time, cleansing and matching settings, and the connection information and input schema that were used to create the data cleansing solution in Information Steward.

5. To open all of the data flows that contain a data cleansing solution, right-click the solution and select Open Containing Data Flows from the menu.

4.1.1 Deleting a data cleansing solution

You can delete a data cleansing solution in Workbench only if its corresponding project has first been deleted in the Data Cleansing Advisor in Information Steward.

To delete a data cleansing solution, right-click the solution and select Delete from the menu.

4.2 Configuring a data flow

Before you start modeling, a data cleansing solution must have already been created in SAP Information Steward and published to the CMS, and you should create a project, datastore, and data flow in the Workbench.

1. Open or create a data flow. The palette of elements on the right side of the window contains readers, loaders, and Platform and Data Integrator transforms.

2. Add a reader and loader to the data flow, as well as any other transforms required to manipulate the data.

3. Select Window Data Cleansing Solutions .The Data Cleansing Solutions tab opens in the workspace and displays the data cleansing solutions that have been published in the Data Cleansing Advisor.

4. Select a data cleansing solution and drag it into the graphical area of the Data Flow editor. A Data Cleansing Solution transform is placed in the data flow that represents the data quality transforms, such as Data Cleanse, Global Address Cleanse, and Match. The underlying transforms included in the Data Cleansing Solution transform depend on the data cleansing solution recommendations that were chosen in Information Steward.The object properties panel opens below the data flow graphical area with three tabs.

Property tab Description

Input Displays a list of all input fields provided by the data cleansing solution that must be mapped in order for the data cleansing solution to be executed in Data Services. The specific input fields vary depending on the configuration of the data cleansing solution. The Description column displays a text description of the input field based on the content type that was identified by Data Cleansing Advisor.

Properties Displays the Data Services options that you can set for the Data Cleansing Solution transform. Data quality configuration options (the business rules of cleansing and

Workbench GuideUsing an SAP Information Steward Data Cleansing Solution © 2013 SAP AG or an SAP affiliate company. All rights reserved. 57

Property tab Description

matching) are not editable within the Workbench. If an option setting must be modified, this is done in the Data Cleansing Advisor feature in SAP Information Steward, and the data cleansing solution must be republished.

Output Displays a list of all available output fields from the data cleansing solution. The available output fields may vary depending on the configuration of the data cleansing solution.

5. To perform basic cleansing on your data to prepare it for more accurate cleansing and matching results, add a Basic Cleanse transform to the data flow and select the appropriate options. Typically, this basic cleansing is performed in Information Steward rather than Data Services Workbench.

6. Connect the Data Cleansing Solution transform to the reader and loader or other transforms in the data flow.7. To automatically map fields from the reader to the Data Cleansing Solution transform, select the Input tab

and click the Detect Mappings button.The Detect Mapping window opens and displays the field names candidates for mapping.

8. To confirm all of the suggested field mappings, click the Confirm All button; otherwise, select individual field names and click Confirm or Reject as appropriate. Click OK.

9. To manually map fields, select a field in the Input Schema and drag it to the appropriate row in the Input tab.10. In the Output tab, select the field names that you want to output from the Available Output Fields and drag

them into the Selected Fields list. If you want to output passthrough fields containing the original data before it was processed, select them in the Input Schema and drag them into the Selected Fields list. Use the Up and Down buttons to reorder the fields as you want to output them.

For information about transform options and input and output fields, see the SAP Data Services Reference Guide.

Related InformationUsing the Basic Cleanse transform [page 58]

4.2.1 Using the Basic Cleanse transform

In Information Steward, the Data Cleansing Advisor lets the data steward set up textual changes (transformations) to apply to specified columns in the data. Transformations set up in the wizard can help prepare the data for more accurate cleansing and matching results. When the data cleansing solution is published and then used in a data flow in Workbench, the transformations are represented in the Data Cleansing Solution transform, and the options are not editable.

Although basic cleansing is typically set up by the data steward in Information Steward, in Workbench, you can also add a Basic Cleanse transform, a specialized type of Query transform, to your data flow to perform the same cleansing on your data.

When the data flow is converted to ATL and deployed to the Data Services repository, the Basic Cleanse transform is converted to a Query transform.

Option Description

Remove control characters

Removes control characters from the selected column.



Option Description

Remove double quotes Removes double quotation marks (") that appear anywhere in the selected column.

Remove leading spaces Removes spaces before the text in the selected column.

Remove single quotes Removes single quotation marks (') that appear anywhere in the selected column.

Remove trailing spaces Remove spaces after the text in the selected column.

Replace null values with <default>

Enter the text that you want to take the place of empty fields in the selected column.

Search and replace In the Replace box, enter the text that you want to delete from the selected column. In the with box, enter the text that you want to take the place of the deleted text. Click Add.

Repeat these steps for each string of text that you want to replace.

Use the up and down arrow buttons if you need to reorder search-and-replace values.

4.3 Modifying the reference data location

The location of the reference data is always set to the substitution parameter, $$RefFilesAddressCleanse, and cannot be modified within Workbench. The substitution parameter is set to the default location of the reference data during the Data Services installation, and Workbench automatically finds the reference data if you haven't modified the default.

To use a different reference data location to run a Data Cleansing Solution transform within Workbench, use the Data Services Designer to log into the repository that you want to modify, and edit the substitution parameter, $$RefFilesAddressCleanse, to point to the desired reference file directory. All Data Cleansing Solution transforms in Workbench will use this new location when executing a data flow for the repository.

4.4 Modifying the reports substitution variables

Enabling and disabling of data quality reports is set by the substitution parameters, $$ReportsAddressCleanse and $$ReportsMatch, and cannot be modified within Workbench. This allows reporting to be controlled at runtime within Data Services rather than being set within Workbench.

To enable or disable data quality reports for Workbench, use the Data Services Designer to log into the repository that you want to modify, and edit the substitution parameters, $$ReportsAddressCleanse and $$ReportsMatch.


4.5 Validating a data flow

To validate a data flow, click the Validate button. The status bar displays the validation status and the number of errors and warnings, if any.

Workbench performs the following validation on the Data Cleansing Solution transform:

● The Data Cleansing Solution can be retrieved from the SAP BusinessObjects Business Intelligence platform's Central Management Server (CMS).

● All input fields specified by the Data Cleansing Solution are mapped.● All output fields in the Data Cleansing Solution transform output schema have a unique field name.● At least one output field is specified in the Data Cleansing Solution transform.

If a validation error is found, the error is displayed in the Problems tab. Double-click an error to open the relevant Property tab in the Data Flow editor.

4.6 Deploying a data flow

Deploying a data flow publishes it to the Data Services repository.

The Data Cleansing Solution transform references the latest published version of the data cleansing solution from Data Cleansing Advisor. If a data cleansing solution is updated and republished in Data Cleansing Advisor, Workbench automatically uses the updated solution, and you must redeploy the data flow to the Data Services repository.

1. With the data flow open in the Data Flow editor, click the Deploy button. If you haven't saved the data flow, a prompt opens asking if you want to save the changes. Click Yes to save the changes.The Choose Repository window opens.

2. Select the Data Services repository that you want to deploy to, and click OK. When deployment is complete, an information window opens stating that the data flow was successfully deployed on the repository.

3. Click Close.

Deploying a data flow generates ATL that represents the elements in the data flow and can be executed by Data Services. After you deploy the job to the Data Services repository, you can open, modify, and execute it as a new, separate job instance in the Data Services Designer. However, any changes that you make in Data Services Designer are not saved to the Workbench instance of the job.

4.7 Executing a data flow

The execution process validates a data flow, generates ATL, deploys it to a Data Services repository, and executes the data flow.

1. With the data flow open in the Data Flow editor, click the Execute button.If you haven't saved the data flow, a prompt opens asking if you want to save the changes.



2. Click Yes to save the changes.The Execute job wizard opens.

3. In Step 1 of the Execute Job wizard, select the Data Services repository that you want to use and click Next.The wizard proceeds to the next step.

4. In Step 2 of the Execute Job wizard, select the appropriate execution parameters and click Finish.The job executes, and the monitor editor opens and displays the job status, execution progress, and any applicable logs. Trace, monitor, and error logs are available. When the job is completed, the job status is displayed at the top.

4.8 Viewing data

To view the data before and after it is processed, you can view data in both the reader and loader transforms.

1. In the graphical area of the Data Flow editor, select the reader or loader.2. In the Properties tab for the selected loader, click the View Data button. A data grid opens and displays the

first 200 records.3. To view more data rows, edit the Max Rows field and click the refresh button.

4.9 Modifying a Workbench job in Data Services Designer

Workbench allows limited changes to transform options and fields in the data flow. In Data Services Designer, you can modify a data flow that was created in Workbench more extensively; however, any changes that you make in the Designer cannot be exported to or viewed in the Workbench. In the Designer, you see the individual transforms (Data Cleanse, Global Address Cleanse, and Match) that were represented by the Data Cleansing Solution transform in Workbench. You can change options, input fields, and output fields for Data Cleanse and Global Address Cleanse. You cannot modify Match settings. You can edit connection between all of the transforms on the canvas as necessary.

1. In the Local Object Library of the Data Services Designer, click the Jobs tab, and expand Batch Jobs to display the Workbench data cleansing solution job.

2. Double-click the job. The job opens on the canvas and contains a work flow.3. Double-click the work flow. The work flow opens and contains a data flow.4. Double-click the data flow. The data flow opens. Rather than displaying the single Data Cleansing Solution

transform, the data flow contains the data quality transforms that the Data Cleansing Solution transform represents, such as Data Cleanse, Global Address Cleanse, and Match. The data quality transforms used depends on the configuration of the data cleansing solution created in Information Steward.

5. Modify and execute the job as necessary.

Any changes that you make in Data Services Designer are not saved to the Workbench instance of the job.


5 Additional Information

5.1 Supported data migration sources and targets

Supported sources

The Workbench supports the following database and application types as migration sources:

● Data Services-supported databases

○ Attunity Connector○ HP Neoview○ DB2○ SAP HANA○ Informix○ Microsoft SQL Server○ MySQL○ Netezza○ ODBC○ Oracle○ SQL Anywhere○ Sybase ASE○ Sybase IQ○ Teradata

● SAP Applications● SAP BW Source

NoteFor more information about the available options for each supported datastore source type, see the Reference Guide.

Supported targets

The Workbench supports the following database types as migration targets:

● SAP HANA● DB2● Netezza● Oracle● Sybase IQ● Teradata● Sybase ASE


Additional Information

● SQL Anywhere● Microsoft SQL Server

NoteFor more information about the available options for each supported datastore target type, see the Reference Guide.

Related InformationReference Guide: Objects, Descriptions of objects, Datastore

5.2 Using the expression editor

The expression editor allows you to use variables, functions, and basic logic structures in many areas of a replication job. For example, you can use the expression editor to customize the mapping of a target column or the replication job's pre-load or post-load scripts.

5.2.1 Supported operators

This section describes the operators that can be used in Workbench expressions and filters.

Operator Description

+ Addition

- Subtraction

* Multiplication

/ Division

= Comparison, equal to

< Comparison, less than

<= Comparison, less than or equal to

> Comparison, greater than

>= Comparison, greater than or equal to

!= Comparison, not equal to

|| Concatenate

% Modulus

Returns the remainder when one number is divided by another.

AND Logical AND

Workbench GuideAdditional Information © 2013 SAP AG or an SAP affiliate company. All rights reserved. 63

Operator Description

OR Logical OR

NOT Logical NOT

IS NULL Comparison, is a NULL value

IS NOT NULL Comparison, is not a NULL value

5.2.2 Including functions in expressions

In Data Services, functions take input values and produce a return value. Input values can be parameters passed into a data flow, values from a column of data, or variables defined inside a script. This section discusses functions and how to include them in a Workbench replication job.

5.2.2.1 Built-in functions

This section describes each built-in function available in Data Services.

The following tables list the names and descriptions of functions, grouped by each function's category in the Workbench expression editor.

For a complete listing of all built-in functions and their syntax, parameters, and return values, see “Functions and Procedures, Descriptions of built-in functions” in the Reference Guide.

Aggregate functions

Function Description

avg Calculates the average of a given set of values.

count Counts the number of values in a table column.

count_distinct Count the number of distinct non-null values in a table column.

max Returns the maximum value from a list.

min Returns the minimum value from a list.

sum Calculates the sum of a given set of values.



Conversion functions


cast Returns a value in the cast data type.

extract_from_xml Extracts XML directly from single column in a database table, and converts it into its internal nested relational data model (NRDM).

interval_to_char Returns a string representation of the interval.

julian_to_date Converts a Julian value to a date.

load_to_xml Generates XML text from NRDM and loads it into a single database column (Assumes the database supports XML text in its columns).

long_to_varchar Converts a data type from long to varchar.

num_to_interval Converts a numeric value to an interval.

to_char Converts a date or numeric type to a string.

to_date Converts a string to a date.

to_decimal Converts a varchar to a decimal.

varchar_to_long Converts a data type from varchar to long.

Cryptographic functions


decrypt_aes Decrypts the input string using the user-specified passphrase and key length using the AES algorithm.

encrypt_aes Encrypts the input string using the user-specified passphrase and key length using the AES algorithm.

Database functions


key_generation Generates keys for the specified table, after determining the appropriate starting value.

sql Runs a SQL operation in the specified database.

total_rows Returns the number of rows in a particular table in a datastore.


Date functions


add_months Adds a given number of months to a date.

concat_date_time Returns a datetime from separate date and time inputs.

date_diff Returns the difference between two dates or times.

date_part Extracts a component of a given date.

day_in_month Determines the day in the month on which the given date falls.

day_in_week Determines the day in the week on which the given date falls.

day_in_year Determines the day in the year on which the given date falls.

fiscal_day Converts a given date into an integer value representing a day in a fiscal year.

isweekend Indicates that a date corresponds to Saturday or Sunday.

julian Converts a date to its integer Julian value, the number of days between the start of the Julian calendar and the date.

last_date Returns the last date of the month for a given date.

month Determines the month in which the given date falls.

quarter Determines the quarter in which the given date falls.

sysdate Returns the current date as listed by the Job Server's operating system.

systime Returns the current time as listed by the Job Server's operating system.

week_in_month Determines the week in the month in which the given date falls.

week_in_year Determines the week in the year in which the given date falls.

year Determines the year in which the given date falls.

Environment functions


get_env Returns a value for the specified environmental variable.

get_error_filename Returns the full path and file name for the error log.

get_monitor_filename Returns the full path and file name for the monitor log.

get_trace_filename Returns the full path and file name for the trace log.

is_set_env Verifies if the specified environment variable is set.




set_env Sets an environmental variable temporarily to a specified value.

Lookup functions


lookup Finds a value in one table or file based on values in a second table or file.

lookup_ext Finds data from a database table, flat file, or memory datastore table.

Math functions


abs Returns the absolute value of an input number.

ceil Returns the smallest integer value greater than or equal to an input number.

floor Returns the largest integer value less than or equal to an input number.

ln Returns the natural logarithm of the given numeric expression.

log Returns the base-10 logarithm of the given numeric expression.

mod Returns the remainder when one number is divided by another.

power Returns the value of the give expression to the specified power.

rand Returns a random number between 0 and 1.

rand_ext Returns a random number between 0 and 1.

round Rounds a given number to the specified precision.

sqrt Returns the square root of the given expression.

trunc Truncates a given number to the specified precision.


Miscellaneous functions


base64_decode Returns the source data after decoding the base64-encoded input.

base64_encode Returns the base64-encoded data in the engine locale character set.

current_configuration Returns the name of the datastore configuration in use at runtime.

current_system_configuration Returns the name of the system configuration in use at runtime. If no system configuration is defined, returns a NULL value.

dataflow_name Returns the data flow name in which this call exists. If the call is not in a data flow, returns NULL.

datastore_field_value Retrieves the value of a specified datastore field.

db_database_name Returns the database name of the datastore configuration in use at runtime.

db_owner Returns the real owner name for the datastore configuration that is in use at runtime.

db_type Returns the database type of the datastore configuration in use at runtime.

db_version Returns the database version of the datastore configuration in use at runtime.

decode Returns an expression based on the first condition in the specified list that evaluates to TRUE.

file_exists Checks to see if a given file or directory exists.

gen_row_num Returns an integer value beginning with 1 then incremented sequentially by 1 for each additional call. This function can be used to generate a column of row IDs.

gen_row_num_by_group Returns group row number of the record.

gen_uuid Returns a unique varchar string identifier.

get_domain_description Returns the description of a value when given the domain name and the value.

get_file_attribute Returns date created, date modified, or size (in bytes) of a physical file.

greatest Returns greatest of the list of one or more expressions.

host_name Returns the name of the computer on which the job is executing.

ifthenelse Allows conditional logic in mapping and selection operations.

is_group_changed Returns 1 if the group is changed, 0 otherwise.

isempty Indicates if a nested table contains data.




job_name Returns the name of the job in which the call to this function exists.

job_run_id Retrieves the job run ID for the current job execution.

least Returns the least in a list of one ormore expressions.

nvl Replaces NULL values.

previous_row_value Returns the column value of previous row.

pushdown_sql Allows you to create dynamic WHERE clauses.

raise_exception Calling this function causes an exception to be generated.

raise_exception_ext Same as raise_exception, but takes a second parameter for an exit code.

repository_name Returns a database connection string and owner name. For example: beq-local.DBUser. This is the ID for the repository from which the job is run.

sleep Suspends the execution of the data flow or work flow from where it is called.

system_user_name Returns the user name used to log into the Job Server's operating system.

table_attribute Retrieves the value of a specified table attribute.

truncate_table Allows you to explicitly expunge data from a memory table.

wait_for_file Returns the existing files that match the input file pattern.

workflow_name Returns the work flow in which this call exists. Returns the name of the inner most work flow in cases where several work flows enclose this function call. If no work flow is found, returns job name.

SAP functions


sap_openhub_processchain_execute Starts the process chain that extracts data from an SAP NetWeaver Business Warehouse(BW) and loads the extracted data into an Open Hub Destination table.

sap_openhub_set_read_status Sends the read status for the Open Hub table to SAP NetWeaver BW.


String functions


ascii Returns the decimal value of the first character for the given string using ASCII character set. If the character passed is not a valid ASCII character, -1 is returned.

chr Get character representation of provided ASCII value.

double_metaphone Encodes the input string usin the Double Metaphone algorithm and returns a string.

index Returns the index of a given word in a string.

init_cap Changes the characters in a string to title case.

length Returns the number of characters in a given string.

literal Returns an input constant expression without interpolation. Allows you to assign a pattern to a variable without interpolation.

lower Changes the characters in a string to lowercase.

lpad Pads a string with characters froma specified pattern.

lpad_ext Pads a string with logical characters from a specified pattern.

ltrim Removes specified characters from the start of a string.

ltrim_blanks Removes blank characters from the start of a string.

ltrim_blanks_ext Removes blank and control characters from the start of a string.

match_pattern Matches whole input strings to simple patterns supported by Data Services. This function does not match substrings.

match_regex Matches whole input strings to the pattern that you specify with regular expressions (regular expressions based on the POSIX standard) and flags. This function does not match substrings.

match_simple

print Prints the given string to the trace log.

replace_substr Returns a string where every occurrence of a given search string in the input is substituted by the given replacement string.

replace_substr_ext Takes an input string, replaces specified occurrences of a specified sub-string with a specified replacement and returns the result. You can also use this function to search for hexadecimal or reference characters.

rpad Pads a string with characters from a given pattern.

rpad_ext Pads a string with logical characters from a given pattern.

rtrim Removes given characters from the end of a string.

rtrim_blanks Removes blank characters from the end of a string.




rtrim_blanks_ext Removes blank and control characters from the end of a string.

search_replace Searches input parameters and replaces by matching criteria and values specified by search table.

soundex Encodes the input string using the Soundex algorithm and returns a string. Use when you want to push down the function to the database level.

substr Returns a specific portion of a string starting at a given point in the string.

upper Changes the characters in a string to uppercase.

word Returns one word out of a string.

word_ext Returns the word identified by its position in a delimited string.

System functions


exec Sends a command to the operating system for execution.

mail_to Sends the specified e-mail message.

smtp_to Sends the specified e-mail message using the SMTP protocol.

User functions

You can create your own functions by writing script functions in SAP Data Services scripting language using the Designer smart editor.

For more information about user-defined custom functions, see “Functions and Procedures, Custom functions” in the Reference Guide.

Validation functions


is_valid_date Indicates if an expression can be converted into a valid date value.



is_valid_datetime Indicates if an expression can be converted into a valid datetime value.

is_valid_decimal Indicates if an expression can be converted into a valid decimal value.

is_valid_double Indicates if an expression can be converted into a valid double value.

is_valid_int Indicates if an expression can be converted into a valid integer value.

is_valid_real Indicates if an expression can be converted into a valid real value.

is_valid_time Indicates if an expression can be converted into a valid time value.

Related InformationReference Guide: Functions and Procedures, Descriptions of built-in functions Reference Guide: Functions and Procedures, Custom functions

5.3 Managing Workbench project information

5.3.1 Creating a Workbench project

If you want to create a new project, start the new project wizard in one of the following ways:

● Right-click inside the project explorer and choose New Project .

● Choose New Project from the File menu.

● Click the New toolbar icon. Choose General Project and click Next.

1. Specify the name for the new project and click Next.


2. If you want to save the files for the project to a different location, uncheck Use default location and specify a location for the project files.

3. Select the Data Services repository.4. Click Finish to create the project.

The new project is displayed in the project explorer.

You can now add Workbench objects such as datastores and replication jobs to your project.



5.3.2 Exporting resources and preferences

Use the Export wizard to export Workbench folders, resources, and user preferences.

1. Right-click within the project explorer pane and choose Export.The Export wizard appears.

2. Select the type of export that you want to perform and click Next.

○ Archive FileExports selected Workbench resources to a .zip or .tar archive on the local file system.

○ File SystemExports selected Workbench resources directly to a location on the local file system.

○ PreferencesExports Workbench configuration preferences to a location on the local file system.

3. If you are exporting to an archive file or a location on the file system, select the resources to export and set the export options as desired.a) Select the folders and resources that you want to export.

Folders appear in the left pane and resources appear in the right pane.

TipFolders with a grayed-out selection box are not explicitly selected, but contain objects that will be exported.

b) Click Browse to specify the location where the archive file or resource files will be created.c) If you are exporting to an archive file, select the type of archive file to create.d) Select the directory structure mode.

Option Description

Create directory structure for files Creates the directory structure for all selected resources, even if no folders are explicitly selected.

Create only selected directories Creates the directory structure only for folders that are explicitly selected.

4. If you are exporting preferences, select the preferences to export and set the export options as desired.a) Select the preferences to export, or select Export all to export all Workbench preferences.b) Click Browse to specify the location where the preference file will be exported.

5. Click Finish.The selected resources or preferences are exported to the specified archive file or file system location.

5.3.3 Importing resources and preferences

Use the Import wizard to import Workbench folders, resources, and user preferences.

1. Right-click within the project explorer pane and choose Import.The Import wizard appears.

2. Select the type of export that you want to perform and click Next.


○ Archive FileImports selected Workbench resources from a .zip or .tar archive on the local file system.

○ File SystemImports selected Workbench resource files from a location on the local file system.

○ PreferencesImports Workbench configuration preferences from a file on the local file system.

3. If you are importing from an archive file or a location on the file system, select the resources to import and set the import options as desired.a) Click Browse to specify the location of the archive file or resource files.b) Select the folders and resources that you want to import.

Folders appear in the left pane and resources appear in the right pane.

TipFolders with a grayed-out selection box are not explicitly selected, but contain objects that will be imported.

c) Click Browse to specify the folder where the imported resources will be stored.4. If you are importing preferences, select the preferences to import.

a) Click Browse to specify the preference file to import.b) Select the preferences to import, or select Import all to import all Workbench preferences.

5. Click Finish.The selected resources or preferences are imported to the Workbench.



www.sap.com/contactsap

© 2013 SAP AG or an SAP affiliate company. All rights reserved.No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. The information contained herein may be changed without prior notice.Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. National product specifications may vary.These materials are provided by SAP AG and its affiliated companies ("SAP Group") for informational purposes only, without representation or warranty of any kind, and SAP Group shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP Group products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries.Please see http://www.sap.com/corporate-en/legal/copyright/index.epx for additional trademark information and notices.

http://www.sap.com/contactsap

http://www.sap.com/corporate-en/legal/copyright/index.epx

http://www.sap.com/corporate-en/legal/copyright/index.epx

Documents

Ds 42 Workbench En