INNOV-4: Breaking Down Enterprise Application Data Silos with Data Services
Ken RuggVice President, Data Services
© 2006 Progress Software Corporation2INNOV-4 Breaking Down Data Silos with Data Services
Agenda
What are Data Services?…and why would I want them?
How does it work? The DataXtendTM Product Line
Breaking Down Enterprise Data Silos with Data Services
© 2006 Progress Software Corporation3INNOV-4 Breaking Down Data Silos with Data Services
The Silos run the asylum
ERP
CRM SRMPLM
BI/DW
B2B
/ B
2C
B2B
CPM
Warranty
Asset Tracking
Vendor Compliance
BillingReservations
© 2006 Progress Software Corporation4INNOV-4 Breaking Down Data Silos with Data Services
ETL Tools
Integration Brokers
Database APIs/Gateways
File Transfer/FTP
Virtual Data Federation ("EII")
Complexity and Breadth
of Applicability
Data Integration Landscape
Data Latency RangeNarrow Wide
DataTransformation
Support
Simplistic
Rich
Data Integration Forms the Technology Foundation of EIM, January 25th 2005
DataXtend Data Services
How is data delivered to applications today?
Replication
DataXtendTM
Data Services
© 2006 Progress Software Corporation5INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM provides
Data integration for distributed, service-oriented applications.
Delivering:transactional views of shared data in a form that applications need in real timein real world environmentswith no change to your applications or business logic
Delivering Data to Applications
© 2006 Progress Software Corporation6INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
.NETJ2EEOpenEdge®
SQLServerOracleOpenEdge®OpenEdge®
OpenEdge®
Delivering Data to Applications
© 2006 Progress Software Corporation7INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
Sonic Enterprise Service Bus
.NETJ2EEOpenEdge®
SQLServerOracleOpenEdge®OpenEdge®
OpenEdge®
Delivering Data to Applications
© 2006 Progress Software Corporation8INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM Data Services
Sonic Enterprise Service Bus
.NETJ2EEOpenEdge®
SQLServerOracleOpenEdge®OpenEdge®
OpenEdge®
Delivering Data to Applications
© 2006 Progress Software Corporation10INNOV-4 Breaking Down Data Silos with Data Services
Data Services and SOA
“…we believe this idea of data services is a very important part of an overall approach to SOA.
Enterprises need to use data integration technology to create data services which
provide data to all these new and different consumers in a consistent, timely, high-quality,
and reliable manner.”
Ted Friedman, Gartner
© 2006 Progress Software Corporation11INNOV-4 Breaking Down Data Silos with Data Services
Agenda
What are Data Services?…and why would I want them?
How does it work? The DataXtendTM Product Line
Breaking Down Enterprise Data Silos with Data Services
© 2006 Progress Software Corporation12INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: Addressing Business Challenges
Application versioning
Data integration
Master Data Management
Disconnected Applications
Service Oriented Architectures (SOA)
Optimized data access
DataXtendTM Data Services
ESBDelivering Data to Applications
© 2006 Progress Software Corporation13INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM: Application Versioning
Description Side-by-side
deployment of old and new application
Database schemas differ
Benefits No change to apps Application & data
correctness Transparent data
conversion
App*App
DB DB*
Delivering Data To Applications
© 2006 Progress Software Corporation14INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM: Data Integration
Description Integrate applications
by integrating their data Database schemas are
known
Benefits No impact to existing apps Application & data
correctness Transparent data
conversion
App 2App 1
DB 1 DB 2
Delivering Data To Applications
© 2006 Progress Software Corporation15INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM: Master Data Management
Description Multiple data sources Standard schema for
integrated data Complex process for
transforming data
Benefits Incremental change
capture Map data sources to
standard schema No impact to existing apps
App 3App 2App 1
DB 1 DB 3DB 2 Master DB
New app
Delivering Data To Applications
© 2006 Progress Software Corporation16INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: Disconnected Operations
Description Enable mobile use of
database application Mostly connected or
mostly disconnected
Benefits No change to apps Application & data
correctness Transparent replication Efficient operation
DataXtend DataXtend Data ServicesDataXtend
MobileApp
Local DB
App
DB
Delivering Data To Applications
© 2006 Progress Software Corporation17INNOV-4 Breaking Down Data Silos with Data Services
Sonic Enterprise Service Bus
.NETJ2EEOpenEdge®
SQLServerOracleOpenEdge®OpenEdge®
OpenEdge®
DataXtendTM: Service Oriented Architectures
“Passing all the data that is required to complete a business process around as XML documents is like a snowplow pushing a snow bank down the street” – Carl Olofson, IDC
Sonic Enterprise Service Bus
Each Service adds more data increasing the size of the “snow bank”
© 2006 Progress Software Corporation19INNOV-4 Breaking Down Data Silos with Data Services
Sonic Enterprise Service Bus
.NETJ2EEOpenEdge®
SQLServerOracleOpenEdge®OpenEdge®
OpenEdge®
DataXtendTM: Service Oriented Architectures
“Passing all the data that is required to complete a business process around as XML documents is like a snowplow pushing a snow bank down the street” – Carl Olofson, IDC
Each Service adds more data increasing the size of the “snow bank” By the end of the process the “snow can be very dirty” as the data becomes out of date
© 2006 Progress Software Corporation20INNOV-4 Breaking Down Data Silos with Data Services
SQLServerOracleOpenEdge®OpenEdge®
Sonic Enterprise Service Bus
DataXtendTM: Service Oriented Architectures
“Passing all the data that is required to complete a business process around as XML documents is like a snowplow pushing a snow bank down the street” – Carl Olofson, IDC
…with Data Services, information can be “passed by reference” to accelerate processing
DataXtendTM Data Services
.NETJ2EEOpenEdge®OpenEdge®
SQL ServerOracleOpenEdge®OpenEdge®
© 2006 Progress Software Corporation21INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM
DataXtendTM: Optimized Data Access
Description Customized object view
of relational data Relational database is
system of record
Benefits Java object binding No change to legacy apps Transactional consistency In-memory performance
New appLegacy
app
DB of record
Legacyapp
DataCache
Delivering Data To Applications
© 2006 Progress Software Corporation22INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: Optimized Data Access
Sonic Enterprise Service Bus
.NET
SQLServer
J2EEOpenEdge®
OracleOpenEdge®OpenEdge®
OpenEdge®
…in the context of SOA
SOA enables reuse of critical business functions as services
© 2006 Progress Software Corporation23INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: Optimized Data Access
Sonic Enterprise Service Bus
.NET
SQLServer
J2EE
Oracle
SOA enables reuse of critical business functions as services
OpenEdge®
OpenEdge®OpenEdge®
OpenEdge® …But these reusable components can become
bottlenecks and single points of failure
…in the context of SOA
© 2006 Progress Software Corporation24INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM: Optimized Data Access
Sonic Enterprise Service Bus
.NET
SQLServer
J2EE
Oracle
SOA enables reuse of critical business functions as services
Data Services enables “Optimized Services” with:• Optimized data model for specialized processing• Increased throughput and availability through
redundant services
OpenEdge®
OpenEdge®OpenEdge®
OpenEdge®
…in the context of SOA
© 2006 Progress Software Corporation25INNOV-4 Breaking Down Data Silos with Data Services
Agenda
What are Data Services?…and why would I want them?
How does it work? The DataXtendTM Product Line
Breaking Down Enterprise Data Silos with Data Services
© 2006 Progress Software Corporation26INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: How does it work?
Distributed transactional synchronization
Distributed Transaction Management• Similar to database clustering,
(e.g. Oracle RAC) What is a transaction
• An atomic unit of work with respect to– Consistency– Recovery
• Transactions provide– Atomicity– Correctness– Isolation– Durability
Before Execution
During Execution
After Execution
Consistent State
Potentially Inconsistent
State
Consistent State
© 2006 Progress Software Corporation27INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: How does it work?
Distributed transactional synchronization
What is a transaction (continued)• Serialization of updates
– If each transaction is correct, serial execution of these transactions is also correct
• Avoid conflicts with lock management
TXN 1
TXN 2
TXN 3
TXN 4
TXN 5
TXN 6
© 2006 Progress Software Corporation29INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: How does it work?
Distributed transactional synchronization Distributed Serialization
• Each replicated copy applies transactions in a common order• Avoid conflicts with distributed lock management
Similar to XA…• Distributed transactions• Strong transactional guarantees
…but different• All data goes every place
– each transaction can execute locally• Highly scalable, performant and available
TXN 1
TXN 2TXN 3
TXN 4TXN 5
TXN 6
TXN 1
TXN 2TXN 3
TXN 4TXN 5
TXN 6
TXN 1
TXN 2TXN 3
TXN 4TXN 5
TXN 6
TXN 1
TXN 2TXN 3
TXN 4TXN 5
TXN 6
TXN 1
TXN 2TXN 3
TXN 4TXN 5
TXN 6
Server 1 Server 3Server 2 Server 4 Server 5
© 2006 Progress Software Corporation30INNOV-4 Breaking Down Data Silos with Data Services
Response Time: DataXtendTM vs. 2PC
0
5
10
15
20
25
30
1 Server 2 Servers 3 Servers 4 Servers 5 Servers
2 Phase Commit DataXtend Protocol
Response time in seconds of DataXtend Protocol vs. two phase commit across distributed servers
© 2006 Progress Software Corporation31INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: How does it work?
File System
Application
DatabaseTriggers
C. TablesS. Tables
Replication Engine
File Replicator
Distribution Manager
DataXtend™ RE DataXtend™ RE
DataXtend™ RE
DataXtend™ RE
~10-15% overhead for change detection
Transparent Change Detection
© 2006 Progress Software Corporation32INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM: How does it work?
TABLE: CUSTOMER
ID NAME CITY STATE
114602992
100114105183108244200162200711
Joe A. SmithMary JonesFrank Lee AndersonMadeline JohnsonHenry OswaldRavi KumarJon FranklinSid Bush
BostonSan JoseWichitaSt. LouisChicagoDallasButteErie
MACAKSMOILTXMTPA
TABLE: PROSPECTS
ID NAME CITY STATE
11451838244
Madeline JohnsonHenry OswaldRavi Kumar
St. LouisChicagoDallas
MOILTX
TABLE: CONTACTS
ID FIRST MIDDLE LAST ADDRESS
114602992
JoeMaryFrank
A.
Lee
SmithJonesAnderson
Boston, MASan Jose, CAWichita, KS
TABLE: CUST
ID NAME ZIP CITY
162711
Franklin, JonBush, Sid
5970116501
Erie, PAButte, MT
TABLE: ZIPCODE
ZIP
1650159701
CRM System
Customer Self-Service
Data Warehouse
Order Entry
Transactional Data Mapping
© 2006 Progress Software Corporation33INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM: How does it work?
TABLE: CUSTOMER
ID NAME CITY STATE
114602992
100114105183108244200162200711
Joe A. SmithMary JonesFrank Lee AndersonMadeline JohnsonHenry OswaldRavi KumarJon FranklinSid Bush
BostonSan JoseWichitaSt. LouisChicagoDallasButteErie
MACAKSMOILTXMTPA
TABLE: PROSPECTS
ID NAME CITY STATE
11451838244
Madeline JohnsonHenry OswaldRavi Kumar
St. LouisChicagoDallas
MOILTX
TABLE: CONTACTS
ID FIRST MIDDLE LAST ADDRESS
114602992
JoeMaryFrank
A.
Lee
SmithJonesAnderson
Boston, MASan Jose, CAWichita, KS
TABLE: CUST
ID NAME ZIP CITY
162711
Franklin, JonBush, Sid
5970116501
Erie, PAButte, MT
TABLE: ZIPCODE
ZIP
1650159701
CRM System
Customer Self-Service
Data Warehouse
Order Entry
Data Type Translation
Transactional Data Mapping
© 2006 Progress Software Corporation34INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM: How does it work?
TABLE: CUSTOMER
ID NAME CITY STATE
114602992
100114105183108244200162200711
Joe A. SmithMary JonesFrank Lee AndersonMadeline JohnsonHenry OswaldRavi KumarJon FranklinSid Bush
BostonSan JoseWichitaSt. LouisChicagoDallasButteErie
MACAKSMOILTXMTPA
TABLE: PROSPECTS
ID NAME CITY STATE
11451838244
Madeline JohnsonHenry OswaldRavi Kumar
St. LouisChicagoDallas
MOILTX
TABLE: CONTACTS
ID FIRST MIDDLE LAST ADDRESS
114602992
JoeMaryFrank
A.
Lee
SmithJonesAnderson
Boston, MASan Jose, CAWichita, KS
TABLE: CUST
ID NAME ZIP CITY
162711
Franklin, JonBush, Sid
5970116501
Erie, PAButte, MT
TABLE: ZIPCODE
ZIP
1650159701
CRM System
Customer Self-Service
Data Warehouse
Order Entry
Format Translation
Transactional Data Mapping
© 2006 Progress Software Corporation35INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM: How does it work?
TABLE: CUSTOMER
ID NAME CITY STATE
114602992
100114105183108244200162200711
Joe A. SmithMary JonesFrank Lee AndersonMadeline JohnsonHenry OswaldRavi KumarJon FranklinSid Bush
BostonSan JoseWichitaSt. LouisChicagoDallasButteErie
MACAKSMOILTXMTPA
TABLE: PROSPECTS
ID NAME CITY STATE
11451838244
Madeline JohnsonHenry OswaldRavi Kumar
St. LouisChicagoDallas
MOILTX
TABLE: CONTACTS
ID FIRST MIDDLE LAST ADDRESS
114602992
JoeMaryFrank
A.
Lee
SmithJonesAnderson
Boston, MASan Jose, CAWichita, KS
TABLE: CUST
ID NAME ZIP CITY
162711
Franklin, JonBush, Sid
5970116501
Erie, PAButte, MT
TABLE: ZIPCODE
ZIP
1650159701
CRM System
Customer Self-Service
Data Warehouse
Order Entry
Identity Translation
Transactional Data Mapping
© 2006 Progress Software Corporation36INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM: How does it work?
DataXtendTM Data Services
TABLE: CUSTOMER
ID NAME CITY STATE
114602992
100114105183108244200162200711
Joe A. SmithMary JonesFrank Lee AndersonMadeline JohnsonHenry OswaldRavi KumarJon FranklinSid Bush
BostonSan JoseWichitaSt. LouisChicagoDallasButteErie
MACAKSMOILTXMTPA
TABLE: PROSPECTS
ID NAME CITY STATE
11451838244
Madeline JohnsonHenry OswaldRavi Kumar
St. LouisChicagoDallas
MOILTX
TABLE: CONTACTS
ID FIRST MIDDLE LAST ADDRESS
114602992
JoeMaryFrank
A.
Lee
SmithJonesAnderson
Boston, MASan Jose, CAWichita, KS
TABLE: CUST
ID NAME ZIP CITY
162711
Franklin, JonBush, Sid
5970116501
Erie, PAButte, MT
TABLE: ZIPCODE
ZIP
1650159701
CRM System
Customer Self-Service
Data Warehouse
Order Entry
Transactional Data Mapping
© 2006 Progress Software Corporation37INNOV-4 Breaking Down Data Silos with Data Services
Agenda
What are Data Services?…and why would I want them?
How does it work? The DataXtendTM Product Line
Breaking Down Enterprise Data Silos with Data Services
© 2006 Progress Software Corporation38INNOV-4 Breaking Down Data Silos with Data Services
DataXtendDataXtend Data Services
DataXtendTM Data Services
Sonic Enterprise Service Bus
Enables Data Integration as SOA enables Process Integration
OpenEdge®
OpenEdge®
SQLServer
.NET
Oracle
J2EE
OpenEdge®
OpenEdge®
© 2006 Progress Software Corporation39INNOV-4 Breaking Down Data Silos with Data Services
DataXtend DataXtend Data Services
DataXtendTM Data Services
Sonic Enterprise Service Bus
Enables Data Integration as SOA enables Process Integration
OpenEdge®
OpenEdge®
SQLServer
.NET
Oracle
J2EE
OpenEdge®
OpenEdge®
DataXtend
© 2006 Progress Software Corporation40INNOV-4 Breaking Down Data Silos with Data Services
DataXtend DataXtend Data Services
DataXtendTM Data Services
Sonic Enterprise Service Bus
Enables Data Integration as SOA enables Process Integration
OpenEdge®
OpenEdge®
SQLServer
.NET
Oracle
J2EE
OpenEdge®
OpenEdge®
DataXtend
DataCache
© 2006 Progress Software Corporation41INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Enterprise
Sonic Enterprise Service Bus
.NETJ2EEOpenEdge®
SQL ServerOracleOpenEdge®OpenEdge®
OpenEdge®
Enables Data Integration as SOA enables Process Integration
© 2006 Progress Software Corporation42INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM Enterprise
Sonic Enterprise Service Bus
.NETJ2EEOpenEdge®
SQL ServerOracleOpenEdge®OpenEdge®
OpenEdge®
Enables Data Integration as SOA enables Process Integration
© 2006 Progress Software Corporation43INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
DataXtendTM Enterprise
Sonic Enterprise Service Bus
.NETJ2EEOpenEdge®
SQL ServerOracleOpenEdge®OpenEdge®
OpenEdge®
Enables Data Integration as SOA enables Process Integration
© 2006 Progress Software Corporation45INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Remote Edition
Remote Office Operations•Support geographically dispersed operations of an enterprise
•Allows local office to continue operation when corporate network connection drops
Field Force Automation•Enterprise applications for users usually not connected to the corporate network
•User can continue to do business with no change in the quality of service of the application
Disconnected Operations
© 2006 Progress Software Corporation46INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Remote Edition
US Army Troop Deployment & Readiness SystemIssues: Information regarding troops did not give an
accurate picture of current units’ state of readiness Material asset readiness was old and outdated Battlefield conditions were not being captured and
delivered to where they were needed
Solution: Embed two-way Hub & Spoke replication in
FusionNet Data can be sliced for each field commander and his
specific troops
Results: All personal records, material assets, and troop
readiness can be viewed from the unit level
“DataXtendTM has been a great help in achieving our disconnected client and server replication goals”
Major Kurt WarnerCorps Knowledge
Management OfficerMulti-National Corps
Iraq, Office of the ACoS C6
© 2006 Progress Software Corporation48INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Caching Edition
DBDB DBDB DBDB DBDB DBDB
DBDB DBDB DBDB DBDB DBDB
Equities Trading
ServiceService ServiceService ServiceService ServiceService ServiceService
Enterprise Service Bus
40 global trading applications, $7B trades/day
Each application “re-invents” the data access layer:• Reduces developer productivity• Increases maintenance costs• Raises operating risks, system failures, downtime
DataXtendTM Caching EditionIntegrated Data Access And Caching
© 2006 Progress Software Corporation49INNOV-4 Breaking Down Data Silos with Data Services
DataXtendTM Data Services
Delivering Data to Applications
DataXtendTM Data Services
ESBDataXtendTM provides
Data integration for distributed,
service-oriented applications.
Delivering:transactional views of shared data in a form that applications need in real timein real world environmentswith no change to your applications or business logic
© 2006 Progress Software Corporation50INNOV-4 Breaking Down Data Silos with Data Services
Questions?
© 2006 Progress Software Corporation51INNOV-4 Breaking Down Data Silos with Data Services
Thank you foryour time
© 2006 Progress Software Corporation52INNOV-4 Breaking Down Data Silos with Data Services