SAP R/3 Data Acquisition in Near Real Time
Naghman Waheed
Information Architecture Lead, Monsanto
Who Are We Monsanto is 100% focused on agriculture
Our Mission
We work to deliver agricultural products and solutions to:
• Meet the world’s growing food needs
• Conserve natural resources
• Protect the environment
“We succeed when farmers succeed.”
-Hugh Grant, Monsanto CEO
Monsanto Company is a leading global provider of technology-based tools and
agricultural products that improve farm productivity and food quality.
Who We Are Today We are committed to providing producers with agronomic tools that make them more efficient
and maintain their profitability in their farming operation. By helping producers be more
productive, with fewer resources and with less overall effect on the environment, we believe we
are making the world a better place.
•Headquartered in St. Louis, Missouri
•CEO Hugh Grant
• Approximately 22,000 employees
• 500 locations in 5 regions.
• $15.42 billion in annual sales
Produce with more
judicious use
of limited natural
resources.
While
improving the
lives of the
world’s
farmers.
Increase
production
to meet
needs of
a growing
population.
Meet the needs
of the present
while improving the
ability of future
generations to meet
their own needs.
• Treat data as an asset
• Eliminate data duplication
• Improve data quality
• Improve data accessibility
• Turn data into information and knowledge
Opportunities
• Develop key data models using enterprise-wide Information architecture techniques
• Key analytical data (OLAP) will be extracted and separately managed from transactional sources (OLTP)
• Key analytical and BI systems will utilize data models to support business operations
• Keep data as close to the source as possible
Objectives
Implementation
Create a centralized data
modeling repository
Establish an
integrated data
platform
Develop data models
for key functional
areas
Monsanto Information Models
Conceptual Architecture
x
x
x
Information Consumer
Transactional Systems
Web, Audio/Video, Text, Image
Data Scientists
Aggregated Information
“Big Data”
Core Data
External Partners
General Ledger
Current Assets
Expenses
Liabilities Profit /
Cost Centers
Sales Orders
Fixed Assets
Balance Sheet
Broad freedom to join all data types based on
iterative model development and the data-discovery process
Information Pro-Consumer
Monsanto Teradata Architecture
Data Staging
SAP
Salesforce
SAP Source Tables
Non-SAP Source tables
Core Data Models
Application Data Views
Reporting / Analytics Tools
Business Objects
Universes
Other Toolsets LAS
Vegetables Supply Inventory
Analyzer
Delivery Demand Planning
Material
(Cost
Simplification)
Unit costing model
Teradata
Production Planning
Process Orders
Delivery
Purchase Orders
Sales Orders
Goods Movement
Inventory
Other
2 3 BI
1
Excel
Informatica IDR
IBM Cast Iron
Teradata Labs
Sales
Billing
Grower Inventory Other
Informatica PC
Data Availability, Integration, Abstraction, Accessibility,
Uniqueness
• Timestamps do not exist on source tables in SAP systems
• Intermediate Document ( IDoc )
• Business Objects Data Services ( BODS )
• SAP Business Warehouse ( BW) – OpenHub
• ABAP Extracts
• SAP Query & SE16
• Database snapshots
• Third party tools e.g. Informatica’s Netweaver Connector
SAP data extraction - challenges
Near real time data replication architecture
System 1
Oracle
Stand Alone Oracle instance
No IDR binaries installed
System n
Initial Load
Archive
Logs
Archive
Logs
Initial Load
NFS share
Archive
Logs
NFS share
Archive
Logs
Teradata
Optimized Merge Apply Processing
IDR Binaries installed on Teradata
Server Only
Logs written
every 30 minutes
Informatica IDR
Server
• Data accessibility
> Simplified data extraction architecture
> Single data distribution architecture
• Reduced time to delivery
• 15 – 60 minute data updates
• Near real time and “snapshot” reporting
• Multipurpose platform
> Integration
> Business intelligence
Benefits
• Pooled and clustered tables – TAS 2.2
• Shift from batch processing to near real time
• New technology challenges
• Must have table recovery strategy
• Data validation strategy
• Data refresh policies
• Designing “configs” within IDR
• Preparation work with the BASIS team
Lessons Learned
PARTNERS Mobile App
InfoHub Kiosks
teradata-partners.com
Twitter: @
Email:
nnwahe
Follow Teradata
Twitter.com/teradatanews
Linkedin.com/company/teradata