Upload
konduru25
View
24
Download
4
Tags:
Embed Size (px)
DESCRIPTION
DMM105
Citation preview
Public
Anthony (Tony) Antonello / Global HANA Centre of Excellence
DMM105 A Synopsis of SAP Data Movement Strategies for HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 2 Public
Disclaimer
This presentation outlines our general product direction and should not be relied on in making a
purchase decision. This presentation is not subject to your license agreement or any other agreement
with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to
develop or release any functionality mentioned in this presentation. This presentation and SAP's
strategy and possible future developments are subject to change and may be changed by SAP at any
time for any reason without notice. This document is provided without a warranty of any kind, either
express or implied, including but not limited to, the implied warranties of merchantability, fitness for a
particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this
document, except if such damages were caused by SAP intentionally or grossly negligent.
2014 SAP SE or an SAP affiliate company. All rights reserved. 3 Public
Agenda
1. Overview of SAP Data Movement & Transformation Strategies for HANA
2. Real Time Data Movement Strategies
1. SAP ESP (Event Stream Processor) Into HANA
o DEMO - ESP (Streaming) into HANA demo
2. SAP LT Replication Server (SLT) Into HANA
o DEMO SLT (Replicaton Server) into HANA demo
3. SAP SRS Into HANA
o DEMO - SRS (Replication) into HANA demo
3. Batch Data Movement Strategies
1. SAP DXC (Direct eXtractor Connection) into HANA
2. SAP HANA Studio Bulk Loader
o DEMO HS Bulk Loader
4. Data Synchronization into HANA
1. SAP Mobilink into HANA
o DEMO - Mobilink (Synchronization) into HANA
5. Data Movement with Transformation
1. SAP Data Services into HANA
o DEMO Data Services and SRS (CDC) into HANA
Overview of SAP Data Movement &
Transformation Strategies for HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 5 Public
An Enterprise Worth of Data
SAP HANA In-memory
Platform
2014 SAP SE or an SAP affiliate company. All rights reserved. 6 Public
SAP Data Movement and Transformation for HANA
SAP Event Stream Processing (ESP) Analyze and act on fast moving data
SAP Landscape Transformation (SLT) Designed to work with SAP Netweaver
SAP Replication Server (SRS) Can create complex replication landscapes
SAP Mobilink Designed for occasionally connected replication
Data Services (DS) Data cleansing and transformation prior to loading
Direct eXtractor Connection (DXC) Loading from SAP sources using DataSource extractors
Hana Studio Bulk Loader Load external data in flat file format into an SAP HANA database
Integration Feature Data Sources
Streaming Data Data Feeds, SCADA
Replication (Triggers) RDBMS
Replication (T-Log) RDBMS
Synchronization Ultralite, SQL
Anywhere
Transformation and
Loading
Structured and Un-
structured data
Application Data
Extractors
SAP Applications
Bulk Loaders Flat Files
SAP Data Movement Strategies
REAL TIME DATA MOVEMENT STRATEGIES
SAP ESP (Event Stream Processor) Into HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 8 Public
Our World is Full of Simple EventsAnything Can Happen
Plane Landing Lightning Strike
Barcode Scanned
Temperature Reading
Transaction Created
Stock Price Change
2014 SAP SE or an SAP affiliate company. All rights reserved. 9 Public
Insight from EventsAggregation in Time
Time Window
Raw Events
We look at the stream of events
watching for pre-defined patterns or
trends over a period of time.
We generate an alert (complex event) if the
required pattern is detected.
Temperature is dropping for 5 mins
More than 100 parcels are delayed for 10 mins
Time
2014 SAP SE or an SAP affiliate company. All rights reserved. 10 Public
ESP - SAP Event Stream Processor
Event Stream
Processor
Unlimited number of input streams
Incoming data is processed as it arrives, according
to the business logic defined using Data Flow
Authoring tools
Send output to apps, dashboards
Query on demand
Capture data in SAP HANA or Sybase IQ for
historical analysis
?
INPUT
STREAMS
Market Data
Transactions
Sensors
Alerts
Applications
Studio
(Authoring)
Reference
Data
SAP HANA,
Sybase IQ
Dashboard
Message
Bus
2014 SAP SE or an SAP affiliate company. All rights reserved. 11 Public
ESP Key Terms and Concepts Basics of data-flow programming in Event Stream Processor
Data-Flow Programming
ESP uses data-flow programming for processing event streams.
Events
A business event is a message that contains information about an actual business event that occurred. Many business systems produce streams of such events as things
happen.
ESP Projects: Streams, Windows, Adapters, and Continuous Queries
Is like an application, consisting of a set of event streams, any other required data sources, and business logic applied to incoming event data to produce results.
Streams and Windows
Both streams and windows process events. The difference is that windows have state, meaning they can retain and store data, while streams are stateless and cannot.
2014 SAP SE or an SAP affiliate company. All rights reserved. 12 Public
ESP Key Terms and Concepts Basics of data-flow programming in Event Stream Processor
Data-Flow Programming
ESP uses data-flow programming for processing event streams. Define a set of event streams and the connections between them, and apply
operations to the data as it flows from sources to outputs
Breaks a potentially complex computation into a sequence of operations with data flowing from one operation to the next
2014 SAP SE or an SAP affiliate company. All rights reserved. 13 Public
Data Flow Authoring
2014 SAP SE or an SAP affiliate company. All rights reserved. 14 Public
How ESP Can Be Used With HANA
ESP SAP HANA 1
2
1
1
2
Event
Streams
High speed smart capture (via SAP HANA Output Adapter): ESP can apply logic
to transform, filter, aggregate, enrich incoming event streams and capture data
in HANA over a million events per second
ESP can query HANA to provide context for processing incoming events
ESP publishes alerts or triggers response process
3
3
Alerts
Dashboard
Applications
2014 SAP SE or an SAP affiliate company. All rights reserved. 15 Public
Further ESP Information
Watch SAP d-code Online
www.sapcode.com/online
Related ESP d-code Sessions
DMM110 SAP HANA Streaming Introduction to Continuous Queries
DMM119 Installing and Configuring an SAP HANA Streaming Server in Your Landscape
DMM121 Integration Scenarios for Stream Processing with the SAP HANA Platform
DMM165 Incorporating Real-Time Event Stream Processing into SAP HANA Applications
DMM268 SAP Event Stream Processor and SAP Replication Server
Demo
ESP Into HANA License Plate Scanning
2014 SAP SE or an SAP affiliate company. All rights reserved. 17 Public
Roadside License Plate Reader Streaming into HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 18 Public
Roadside License Plate Reader Streaming into HANA
DMV
Plate scans
and speed
SAP Sybase ESP
Tickets
SAP HANA In-memory
Platform
REAL TIME DATA MOVEMENT
STRATEGIES
SAP LT Replication Server (SLT) Into HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 20 Public
Enable real-time replication within your entire landscape
SAP LT Replication Server
Read Engine
Mapping & Transformation Engine
Write Engine
SAP HANA Appliance
SAP HANA
DB
In-Memory S
AP
HA
NA
Stu
dio
BI Client
non-ABAP (SAP supported DBs)
SAP
Business
Suite
SAP
Business
Suite
SAP
Business
Suite SAP
Business
Suite (>=4.6c)
SAP
Business
Suite
SAP
Business
Suite
SAP
Business
Warehouse
SAP HANA
DB
In-Memory
Source Systems SAP LT Replication Server Target Systems
real time real time
non-ABAP (SAP supported DBs)
Data Services
4.2 SP1
2014 SAP SE or an SAP affiliate company. All rights reserved. 21 Public
SAP HANA Application Tables
Data replication via DB trigger based on
change capturing concept
Unlimited release coverage (from SAP R/3 4.6C onwards)
Automatically non-Unicode to Unicode conversion
during load/replication
Handling of cluster and pool tables
Table settings and transformation capabilities (e.g. data
filtering, enrich table structure, scramble data, etc.)
Fast data replication via DB connection,
integration into SAP HANA Studio
SAP HANA Studio
ABAP Source System SAP LT Replication Server Target Systems
SAP Business Suite
Application table Logging table
DB trigger
Read Engine
Read Engine
Mapping & Transformation Engine
Write Engine
SAP LT Replication Server
RFC
Connection
DB
Connection
Architectural Concept 1/2 Replication from ABAP source system
2014 SAP SE or an SAP affiliate company. All rights reserved. 22 Public
SAP LT Replication Server transfers all metadata table definitions from the non-ABAP source system to the HANA system. From the HANA Studio perspective, non-SAP
source replication works the same as for SAP sources. When a table replication is started, SAP LT Replication Server creates logging tables in the source system. The
read engine is created in the SAP LT Replication Server. The connection the non-SAP source system is established as a database connection.
SAP HANA Application Tables
Non-ABAP System
Application table Logging table
DB trigger
Non-ABAP Source System SAP LT Replication Server Target Systems
SAP LT Replication Server
Read Engine
Mapping & Transformation Engine
Write Engine DB
Connection
DB
Connection
Architectural Concept 2/2 Replication from non-ABAP source system
SAP HANA Studio
2014 SAP SE or an SAP affiliate company. All rights reserved. 23 Public
Application Tables
Efficient implementation of data
replication via DB trigger
based on change capturing concept
Highly scalable and reliable replication
process, including comprehensive data
transformation capabilities on the fly
Fast data replication via DB connection.
ABAP Source System SAP LT Replication Server non-ABAP Target System
SAP Business Suite
Application table Logging table
DB trigger
Read Engine
Read Engine
Mapping & Transformation Engine
Write Engine
SAP LT Replication Server
RFC
Connection
DB
Connection
ABAP to non-ABAP Replication Architectural Concept
Non-ABAP
Start of
replication has to
be executed via
LTRC
2014 SAP SE or an SAP affiliate company. All rights reserved. 24 Public
Setup, Execution & Monitoring of a Replication
Setup Replication Execute Replication Monitor Replication
New guided procedure to create a
configuration between source and
target system.
SAP LT Replication Server
SAP HANA
DB
In-Memory
SAP HANA
Studio
Start the replication for the relevant
tables via SAP HANA Studio.
2014 SAP SE or an SAP affiliate company. All rights reserved. 25 Public
SAP HANA system Source systems
System A
System B
Schema 1 Schema 2
Source systems are connected to
separate HANA schema on the
same HANA system
SAP HANA system Source systems
System A
System B Schema 1
Source systems are connected to
same HANA system and also the
same schema
SAP HANA systems SAP Source system
Schema 1
Schema 2 System A
SAP source system is connected
to separate HANA systems or to
the same system with different
schema name.
Multi System Support
SAP HANA systems Source systems
System A
System B
Schema 1
Schema 2
Source systems are connected to
separate HANA systems. Schema
name can be equal or different
2014 SAP SE or an SAP affiliate company. All rights reserved. 26 Public
DB
Connection RFC/DB
Connection
Data and Structure Transformation Capabilities
Target System SAP LT Replication Server Source system
Reduce Number
of Records
by Filter
Target System SAP LT Replication Server Source system
Conversion
of Data
Target System SAP LT Replication Server Source system
Adjustment of
Target Table Structures (i.e. extension/reduction/change of table
structure and/or adjustment of technical
table Setting
Replicating certain data only Only data of specific years, departments, clients, etc. should be used in HANA.
To make certain fields anonymous i.e. HR reporting To fill initial fields To convert units or currency, and recalculate amounts and values using coded rules
Remove fields that are required in the HANA system Additional fields are required (for example can be calculated/filled during load/replication) Merging the same table (i.e. MARA) from different systems (and create an additional field, for example a client field in order to avoid duplicate entries)
Transformation
of Data
DB
Connection RFC/DB
Connection
DB
Connection RFC/DB
Connection
2014 SAP SE or an SAP affiliate company. All rights reserved. 27 Public
Further LT Replication Information
Watch SAP d-code Online
www.sapcode.com/online
Related ESP d-code Sessions
DMM108 Real-Time Data Replication with SAP Landscape Transformation Replication Server
DMM263 SAP Landscape Transformation Replication Server Real-Time Data Integration
DMM844 Road Map Q&A: SAP LT Replication Server (SLT) realtime replication and CDC
CR701 SAP Landscape Transformation Replication Server Performance Boost
CR702 SAP Landscape Transformation Replication Server Replacing SAP BW Extractors
DMM300 SAP Business Warehouse Data Provisioning from SAP and Non-SAP Sources
DMM360 SAP BW 7.40 Data Provisioning from SAP and Non-SAP Sources
Demo
LT Replication Into HANA ERP
2014 SAP SE or an SAP affiliate company. All rights reserved. 29 Public
LT from ERP into HANA
REAL TIME DATA MOVEMENT
STRATEGIES
SAP SRS Into HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 31 Public
SAP Replication Server Real-time data, delivered reliably, enterprise-wide
SAP Replication Server is a real-time, low-impact, replication solution with enterprise-level features for both SAP and non-
SAP databases and applications. It securely, reliably, and rapidly moves data across multiple data sources in real time to
deliver:
2014 SAP SE or an SAP affiliate company. All rights reserved. 32 Public
Product Architecture
Change Data Capture Replication Engine Routing & Distribution
Primary
Sources*:
SAP HANA SAP ASE Oracle MS SQL Server IBM DB2/UDB IBM DB2/OS390
Targets*:
SAP HANA SAP ASE SAP IQ SAP ESP SAP Data Services Oracle Microsoft SQL Server IBM DB2/UDB IBM DB2/OS390 Message Bus
Transformation
Message Repository
Rules Engine
Administration &
Monitoring
In-b
ou
nd
Co
nn
ectivity(L
TL
) T
arg
et D
B C
on
ne
ctiv
ity
Catalog DB
Replication
Agent
Secondary
Replication
Server
Replication
Server
* Please check user documentation for end-to-end source and target database certification
2014 SAP SE or an SAP affiliate company. All rights reserved. 33 Public
SRS Components
Replicate
Queues
Rep Agent
Primary
Rules Engine
Message Repository
Change Detection
Catalog/Control DB
RSSD
Stable Devices
ECDA Gateway
Sybase ASE
Oracle
MS SQL
IBM DB2/OS390
IBM DB2/UDB
RepConnector
Sybase ASE
HANA
Sybase IQ
Oracle
MS SQL
IBM DB2
Any ODBC
MQ, JDBC,
MSMQ, etc.
Sybase Control Center Monitoring
Route
Replication Server
Replication Server
2014 SAP SE or an SAP affiliate company. All rights reserved. 34 Public
SRS Replication into HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 35 Public
RCL Used to Define Source and Target Mappings
From source
To target
2014 SAP SE or an SAP affiliate company. All rights reserved. 36 Public
Further Replication Server Information
Watch SAP d-code Online
www.sapcode.com/online
Related ESP d-code Sessions
DMM113 SAP Replication Server Usability Enhancements
DMM209 Real-Time Heterogeneous Replication with SAP Replication Server
DMM211 HADR for SAP Business Suite on SAP ASE with SAP Replication Server
DMM212 Real-Time Transactional Replication for SAP Business Suite into SAP HANA
DMM213 SAP ASE and SAP Repllication Server with SAP Business Suite
DMM217 SAP Replication Server for Hadoop
DMM222 SAP Replication Server Performance Tuning and Troubleshooting
DMM262 SAP HANA Replication Using SAP Replication Server
2014 SAP SE or an SAP affiliate company. All rights reserved. 37 Public
Further Replication Server Information - continued
Watch SAP d-code Online
www.sapcode.com/online
Related ESP d-code Sessions
DMM268 SAP Event Stream Processor and SAP Replication Server
DMM269 Real-Time Change Data Capture with SAP Data Services and SAP Replication Server
DMM807 Road Map Q&A: SAP Replication Server Technical
Demo
SRS Into HANA Book Store Analytics
2014 SAP SE or an SAP affiliate company. All rights reserved. 39 Public
ASE
Primary DB
ASE
Replicate DB
HANA
Replicate DB
Transformation
Message Repository
Rules Engine
Administration & Monitoring
LT
L C
onnection
EC
H
Catalog DB
ECH accesses HANA is via
HANA odbc driver
SAP HANA In-memory
Platform
Topology
2014 SAP SE or an SAP affiliate company. All rights reserved. 40 Public
Replication Test using rs_ticket
40
BATCH DATA MOVEMENT STRATEGIES
SAP DXC (Direct eXtractor Connection)
into HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 42 Public
SAP HANA Direct Extractor Connection (DXC)
DXC provides low TCO data acquisition for SAP HANA leveraging existing
delivered foundations for data models oCustomer projects may face significant complexity in modeling entities in SAP Business Suite
systems
oData from some areas in SAP Business Suite systems needs application logic to appropriately
represent business documents
oSAP Business Content DataSource Extractors have been available for many years as a basis for data
modeling and data acquisition for SAP BW; customers want a simple, low-TCO data acquisition
mechanism to use SAP Business Content DataSource Extractors in SAP HANA data mart scenarios
o In many use cases, batch-driven data acquisition is sufficient (e.g. every 10 min)
oDXC became generally available with SAP HANA SP4
2014 SAP SE or an SAP affiliate company. All rights reserved. 43 Public
SAP Delivered Business Content DataSource Extractors
SAP Business Suite
Proprietary Extraction Technology
Application based change data capture (aka delta capabilities)
Extractors are application based and take data from the context of the application itself
Extraction is event/process driven and (in some cases) is accomplished with publishing new or changed data in ERP based Delta Queues for receiving systems (e.g. BW or HANA)
Extract structures can easily be enhanced using append structures
Transformations can be implemented at the time of extraction using Business Add Ins (BADIs)
Extract Structures based on entities in the Business Suite
Asynchronous, mass data capable extraction
2014 SAP SE or an SAP affiliate company. All rights reserved. 44 Public
SAP HANA DXC Overview
SAP ERP
Extractor
DataSource (flat structure)
ERP data
Embedded BW
DataSource
PSA
generic data transfer
InfoCubes
Data Store Objects
InfoObjects
generic data transfer
SAP HANA
Data load into HANA activation queue
Separate activation step
Scheduled batch jobs
Activation
Queue
Active version
In memory DSO
Status
Activation
Processing
Data flow redirected from embedded
BW: transferred over http connection
Data models in HANA are built
using active data table of In-
Memory DataStore object
2014 SAP SE or an SAP affiliate company. All rights reserved. 45 Public
business documents master data
transactional data master data attributes master data text master data hierarchy
Transactional data Master data Texts Hierarchies
ERP & R/3 900 700 2300 100 4000
CRM 160 280 700 40 1180
SRM 30 30 60 10 130
Others 360 210 330 30 930
GRC 20 30 120 10 180
1470 1250 3510 190 6420
Thousands of SAP Business Content DataSources Exist
Totals
2014 SAP SE or an SAP affiliate company. All rights reserved. 46 Public
SAP Business Suite DataSource Extractors Example from Sales Order Item Content
HANA Data Models - virtual
SAP HANA
SAP ERP
Application Tables
Sales Order
Communication Structure
Setup Table
Update Methods
Update Mode
DataSource for transactional data
2LIS_11_VAITM
In-Memory Data Store Object Tables:
Active table: /BIC/A 2LIS_11_VAITM00
Activation Queue table: /BIC/A 2LIS_11_VAITM40
Transfer Structure
Extract Strukture MC11VA0ITM
Application logic
Activation Processing
Data flow redirected from embedded BW
BATCH DATA MOVEMENT
STRATEGIES
SAP HANA STUDIO BULK LOADER
2014 SAP SE or an SAP affiliate company. All rights reserved. 48 Public
1. The IMPORT FROM statement imports
external data from a file into an existing
table
2. Can append contol features on the Import
statement or use a control file to specify
file characteristics
3. Can be run from any HANA editor (HANA
Studio SQL Editor, hdbsql)
4. Data files must reside on the HANA Server
Control File
HANA Table
Flat File HANA SQL
Import cmd
Import into HANA using SQL Import Command
2014 SAP SE or an SAP affiliate company. All rights reserved. 49 Public
1. Uses a rich data mapping toolset within
HANA Studio
2. Data files can reside on the client host or
the HANA server
3. The target table can already exist, or be
created by the import
4. Broad array of schema objects beyond flat
files
Import into HANA using SQL Import Command
2014 SAP SE or an SAP affiliate company. All rights reserved. 50 Public
Further HANA Studio Data Provisioning
Watch SAP d-code Online
www.sapcode.com/online
Related ESP d-code Sessions
DMM271 Developing Custom Adapters Using the SAP HANA Data Provisioning Adapter SDK
Demo
Import using HANA Studio and hdbsql
2014 SAP SE or an SAP affiliate company. All rights reserved. 52 Public
Importing into HANA with hdbsql and HANA Studio
DATA SYNCHRONIZATION
SAP MOBILINK INTO HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 54 Public
What is Mobilink?
1. A bi-directional synchronization
technology for large scale mobile (IoT)
database deployment o Remote database (mobile, embedded, or workgroup
database server)
o Consolidated database (enterprise, workgroup, or
desktop database)
2. A server that processes synchronization
requests from SAP mobile databases
(SQLAnywhere and Ultralite)
3. Maintains the integrity of transactions
across the entire synchronization process.
HANA, DB2
ASE, IQ, MSSQL
Oracle, MySQL,
SQLAnywhere
Consolidated
Database
Communication Infrastructure
(Internet / Wireless)
2014 SAP SE or an SAP affiliate company. All rights reserved. 55 Public
Mobilink Synchronization
1. MobiLink uses system tables in the Consolidated
database to store
o Synchronization scripts
o Synchronization progress & status information
o Synchronization User information
2. A Database Object in the Remote Database data to be
synchronized is Published
3. During synchronization MobiLink progresses through
a series of events synchronization scripts that direct
the data flow during the process
4. Written in the native SQL of the Consolidated
Database
5. MobiLink uses ODBC to connect to the Consolidated
DB
2014 SAP SE or an SAP affiliate company. All rights reserved. 56 Public
MobiLink SQLAnywhere Synchronization
HANA
MobiLink
Ultralite or
SQLAnywhere
Remote
ODBC
dbmlsync
1. Upload
2. Download
3. Ack Download
1. Upload
SQLAny or UltraLite keep track of all changes since previous synchronization
All changes are sent in a single upload stream
MobiLink applies changes in a single transaction
2. Download
Scripts in the consolidated database determine changes that need to be downloaded
All changes are sent in a single download stream
3. Acknowledge download
Optional
2014 SAP SE or an SAP affiliate company. All rights reserved. 57 Public
DBMLSYNC
Synchronization is initiated by a client application
via the dbmlsync utility from SQL Anywhere
Remote databases
TCP/IP, SSL HTTP/HTTPS ActiveSync
Reads the Database engine transaction log
Builds an upstream with the a summary of the
changes to the specific publication since the last
synchronization
Applies the downstream from the MobiLink server
to the Remote Database
2014 SAP SE or an SAP affiliate company. All rights reserved. 58 Public
SQL Anywhere Architecture Scenario Remote Sync
Remote Sync
Synchronize HANA database with SQL Anywhere/Ultralite databases
running on smart systems, kiosks,
intelligent appliances, and mobile
devices
MobiLink Server
ODBC Driver
ODBC/Network interface
Database Server
HANA
MobiLink Server
Machine
HANA
Server Machine
SQL Anywhere Server or UltraLite Server
MobiLink Client
Remote
Machines
Database
File
Database API
Application
SQL Anywhere/UltraLite Interface API
HANA
Network
DMZ
Machine
Firewall
Firewall
Web Server Relay Server
2014 SAP SE or an SAP affiliate company. All rights reserved. 59 Public
Further SQL Anywhere and Mobilink Information
Watch SAP d-code Online
www.sapcode.com/online
Related ESP d-code Sessions
DMM106 Introduction to SAP SQL Anywhere
DMM114 Synchronizing Data in SAP HANA Using SAP SQL Anywhere
DMM116 From Raspberry Pi to SAP HANA SAP SQL Anywhere and the Internet of Things
Demo
Telematics Demo
2014 SAP SE or an SAP affiliate company. All rights reserved. 61 Public
Architecture
On board
SAP HANA Cloud
Platform
2014 SAP SE or an SAP affiliate company. All rights reserved. 62 Public
Vehicle Fleet Management with SAP HANA and SAP SQL Anywhere
DATA MOVEMENT WITH
TRANSFORMATION
SAP DATA SERVICES INTO HANA
2014 SAP SE or an SAP affiliate company. All rights reserved. 64 Public
SAP Data Services and HANA RTDP
SAP Data Services provides batch Data Integration. It offers HANA optimized capabilities for Transforming,
Cleansing* and Integrating (bulk or delta) structured and unstructured* data from many different Sources
(SAP and non-SAP) to the Target (SAP HANA)
2014 SAP SE or an SAP affiliate company. All rights reserved. 65 Public
SAP Data Services Source Connectivity Broadens Enterprise Data Access
HANA
Tables
views
Oracle
DB2
Sybase ASE
Sybase IQ
SQL Server
Informix
MySQL
Teradata
ODBC
Netezza
SAP Bus Suite Extractors
ABAP
BAPI
Idoc
SAP NW BW
JD Edwards
Oracle Apps
PeopleSoft
Siebel
Salesforce.com
SuccessFactors
Text delimited
Text fixed width
EBCDIC
XML
Cobol
Excel
HTTP
JMS
Web Services
HADOOP
ADABAS
ISAM
VSAM
Enscribe
IMS/DB
RMS
Both direct and changed data
Extracts context from any text or binary file types (docs, pdfs, ppts)
Understands context in 31 languages
Databases Applications Files/Transport Mainframe (with partner) Unstructured
Content
SAP Data Services has a long history of providing broad connectivity to databases,
applications, legacy systems, file formats
PAM info posted at:
https://service.sap.com/sap/support/pam?hash=s%3Ddata%2520services%26o%3Dmost_viewed%257Cdesc%26st%3Dl%26rpp%3D20%26page%3D1%
26pvnr%3D01200314690900003943%26pt%3Dg%257Cd
2014 SAP SE or an SAP affiliate company. All rights reserved. 66 Public
For typical database
targets, Data
Services transformations are
done in the DS engine and then
loaded into the target
database
Source Target
Extract Transform Load
E-T-L moves to E-L-T How Data Services Leverages HANA for Transformations
2014 SAP SE or an SAP affiliate company. All rights reserved. 67 Public
For HANA, DS
stages data and then
pushes the execution of any
transformation to
HANA and creates
Calculation Views for
execution and
consumption HANA
HANA as Target Source
Extract Load Transform
E-T-L moves to E-L-T How Data Services Leverages HANA for Transformations
2014 SAP SE or an SAP affiliate company. All rights reserved. 68 Public
Data Services for HANA and RTDP Enhancements with Data Services 4.1+
Additional Sources Support with Data Services
HANA stored procedures, calculation, attribute, and analytical views as a Source
Optimized Hadoop extraction and loading
SuccessFactors source and target
Binary files (PDF, Doc) and 31 additional languages supported for Text data processing
Loading Performance
50% improvement in loading flat file and database to HANA (via DS-HANA pipe)
Increased E-L-T and push down optimizations
Sybase IQ support for bulk updates and binary format data transfer
10x improvement in BW extract and load to HANA, Sybase
Parallelized reading from Business Content Extractors
Modeling, Metadata, and Repository
New Eclipse DS Workbench for very fast, simplified data movement into HANA
Full support for HANA views for metadata impact and lineage analysis
HANA as a repository for Data Services and Information Steward
2014 SAP SE or an SAP affiliate company. All rights reserved. 69 Public
Integrated Data Movement Strategy
SAP SLT / Sybase Rep Server + Data Services
Common runtime environment provides
enriched CDC data provisioning across SAP
ERP and non-SAP data replication
Enables changed data sharing between SAP Sybase Rep Server, SAP SLT, and Data Services
Leverage native CDC reading from Rep Server and SAP SLT
With the
Complex data transformation and data quality management of Data Services
Complex Operational Data
Note: SAP SLT & DS integration is targeted for 4.2 SP1 release, ~Q4 2013
New
with 4.2
2014 SAP SE or an SAP affiliate company. All rights reserved. 70 Public
Data Services Designer
drag-n-drop tables,
files, transforms
from Local Object
Library onto Work
Canvas
Work Canvas
click-n-drop
objects from
Palette onto
Work Canvas
click-n-drag from
source point to target
point to create link
Double-click to drill
into object/transform.
(All objects/transforms
are movable)
Normal/Best practice for
flows to be Left-Right
2014 SAP SE or an SAP affiliate company. All rights reserved. 71 Public
Further Data Services Information
Watch SAP d-code Online
www.sapcode.com/online
Related ESP d-code Sessions
DMM269 Real-Time Change Data Capture with SAP Data Services and SAP Replication Server
DMM805 Road Map Q&A: SAP Data Services
DMM104 Whats New in SAP Data Services
Demo
Bookstore BI into HANA Demo
2014 SAP SE or an SAP affiliate company. All rights reserved. 73 Public
Bookstore BI into HANA demo
Question ?
2014 SAP SE or an SAP affiliate company. All rights reserved. 75 Public
SAP d-code Virtual Hands-on Workshops and SAP d-code Online Continue your SAP d-code education after the event!
SAP d-code Online
Access replays of keynotes, Demo Jam, SAP d-code live interviews, select lecture sessions, and more!
Hands-on replays
http://sapdcode.com/online
SAP d-code Virtual Hands-on Workshops
Access hands-on workshops post-event
Starting January 2015
Complementary with your SAP d-code registration
http://sapdcodehandson.sap.com
2014 SAP SE or an SAP affiliate company. All rights reserved. 76 Public
Further Information
SAP Education and Certification Opportunities
www.sap.com/education
Watch SAP d-code Online
www.sapcode.com/online
SAP Public Web
scn.sap.com
www.sap.com
77 2014 SAP SE or an SAP affiliate company. All rights reserved.
Feedback Please complete your session evaluation for
DMM105.
Thanks for attending this SAP TechEd && d-code session.
2014 SAP SE or an SAP affiliate company. All rights reserved. 77 Public
2014 SAP SE or an SAP affiliate company. All rights reserved. 78 Public
2014 SAP SE or an SAP affiliate company. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an
SAP affiliate company.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE
(or an SAP affiliate company) in Germany and other countries. Please see http://global12.sap.com/corporate-en/legal/copyright/index.epx for additional trademark
information and notices.
Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors.
National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP SE or its
affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP SE or
SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing
herein should be construed as constituting an additional warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or
release any functionality mentioned therein. This document, or any related presentation, and SAP SEs or its affiliated companies strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for
any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. All forward-
looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place
undue reliance on these forward-looking statements, which speak only as of their dates, and they should not be relied upon in making purchasing decisions.