Embed Size (px)
Citation preview
SAP AG 1999 VLDB ‘99 / 1
Rudolf Munz
SAP AG
Usage Scenarios of DBMS
SAP AG 1999 VLDB ‘99 / 2
Agenda
Who are we?11
Where do we come from? 22
Where do we want to go?33
SAP AG 1999 VLDB ‘99 / 3
SAP Today
l SAP AG in 1998 revenues: 8.47 Bill. DEM (5.05 Bill. USD)n 4th largest independent software vendor in the world
n Market Leader in Enterprise Applications Software Licenses(36% market share amongst TOP TEN; IDC, 1999)
l 22,000+ customers in 100+ countries team with us ton Extend their competitive capabilitiesn Integrate their business processes
n Get a better return on information at a lower total cost of ownership
l Focused on users in all enterprises regardless of sizen Increased customer satisfaction and strong customer loyalty
n Heavy investment into SAP’s worldwide business communityn 21,000+ SAP employees
SAP AG 1999 VLDB ‘99 / 4
Why Do Customers Buy SAP Software?
l Outsourcing of enterprise application development including its maintenance and enhancement
l SAP is integrating software
l Outsourcing of system technology/platform decisions
SAP AG 1999 VLDB ‘99 / 5
Agenda
Who are we?11
Where do we come from? 22
Where do we want to go?33
SAP AG 1999 VLDB ‘99 / 6
R/3Applicatio
Logic
R/3Presentati
Logic
R/3Applicatio
Logic
R/3Presentati
Logic
ApplicatioLogic
PresentatiLogic
ApplicatioLogic
PresentatiLogic
PresentatiLogicPresentati
Logic
1992: SAP Introduces the 3-Tier Architecture
Databaselayer
Applicationlayer
Databaselayer
Databaselayer
R/3Application
layer
R/3Presentation
layer
Applicationlayer
Presentationlayer
Presentationlayer
LAN required,many roundtripsdata volumeabout 20 KB
WAN-enabled,few roundtripsdata volume< 10 KB
SW deployment
SAP AG 1999 VLDB ‘99 / 7
R/3 Ideology
DB server
Appserver
Frontend
Databasetier
Appservertier
Frontendtier
Appserver
Frontend
Enterprise integration via one single database (no borders between enterprise units)
Frontend
SAP AG 1999 VLDB ‘99 / 8
R/3 Statistics (as of Rel. 4.5)
l 15 000+ tables
l 200 000+ columns
l Growth rate per major release: + 30%
l 35 000 000 lines of application coding
l 8 GB footprint on disk
l more than 20 languages supported
l 650+ software developers in SAP‘s system technology
l 2500+ software developers in SAP‘s applications
SAP AG 1999 VLDB ‘99 / 9
R/3 Architecture
Operating System
DBMS
R/3 Application ServerABAP Interpreter
TP MonitorDBMS InterfaceTable Caching
R/3 Applications
Coded in C/C++
Coded in ABAP
Abstractionlayer
SAP AG 1999 VLDB ‘99 / 10
R/3 Platforms
DB2 forOS/390DB2 forOS/390
Windows 95, Windows 98, Windows NT,Java Desktops and Browsers
HTML Browsers
OS/390OS/390AIX
Digital UNIXHP-UX
AIXDigital UNIX
HP-UX
Reliant
SOLARIS
ReliantLinux
SOLARISWindows NTWindows NT OS/400OS/400
HardwareIntelIntel Systems
DigitalHPDigitalHP
IBMSNISUN
IBMSNISUN
UNIX SystemsUNIX SystemsIBM
AS/400IBM
AS/400
Operatingsystems
FrontendSAPGUI
Databases
IBMS/390IBM
S/390
INFORMIXORACLE
DB2 UDB, SAP DBINFORMIX
ORACLE
DB2 INFORMIX
ORACLEMS SQL Server
DB2 UDB INFORMIX
ORACLEMS SQL Server
DB2 for AS/400DB2 for AS/400
Intel
SAP AG 1999 VLDB ‘99 / 11
FrontendsFrontends
Application serverApplication server
Database serverDatabase server
Scalability . . . . . . . . . . . .
R/3 Scalability
SAP AG 1999 VLDB ‘99 / 12
SAP’s Scalability Benchmarks
l Presentation Layern More than 14,000 very active users
connected to one database
l Application Layern Up to 143 application servers
n The highest number of application servers at customer sitesis less than 30
l Database Layern Scalability through SMP architecture of the database server
n More than 60 CPUs
n More than 700 GB database size
SAP AG 1999 VLDB ‘99 / 13
Published Results for SD Benchmarks
Highest Number of Sales & Distribution (SD) Benchmark Users
1993 1994 1995 1996 1997 1998
800
1600
4000
6000
8000
12000
18000
2400
3200
0189
14001100
900800
600300
1700 18002000
2900
3700
67506030
5320
14400
SAP AG 1999 VLDB ‘99 / 14
Key figures for the 14,400 SD User Benchmark
l 593 DB transactions / second
l 65,200 DB calls / second
l 14,900 DB changes / second
l 167 Mbit / second network traffic to the database server
l 1.9 MB average disk read / second
l 17 MB average disk write / second (peak: 50 MB/sec)
l 1.1 TB total disk space
SAP AG 1999 VLDB ‘99 / 15
Key figures for the 14,400 SD User Benchmark
l Database was running on a 64-processor SMP server
l R/3 application servers used 391 processors
l 10.4 GB of dirty data pages written in 25 minutes
l 1.21 GB of data pages read in 25 minutes
SAP AG 1999 VLDB ‘99 / 16
R/3’s Scalability on Different Platforms
3000
9000
6000
12000
4512
8000 Paral.
6651
S/390AS/400
SD Bench-markUsers
15000 14400
UNIX NT
SAP AG 1999 VLDB ‘99 / 17
R/3 Standard SD Benchmark Figures
July ’94
March ’95
Jan. ’96Oct. ’95
300 SD User
800 SD User
1400 SD User
1700 SD User
189 SD User April ’93
May ’95900 SD User
Sep. ’951110 SD User
Feb. ’95600 SD User
1800 SD User Aug. ’96March ’97
May ’97Aug. ’97
2900 SD User2000 SD User
3600 SD UserNov. ’97
Nov. ’973700 SD User
5320 SD UserDec. ’976030 SD User
14400 SD User Sept. ’98
SAP AG 1999 VLDB ‘99 / 18
Reasons for this Progress
l More RAM in application and database servers
l Increased CPU power, especially in SMP configurations
l Table caching in the R/3 application server
l Asynchronous database update (via update task)
l Application level locking
SAP AG 1999 VLDB ‘99 / 19
Presentation
Applicationserver
DB InterfaceTable Cache
Databaseserver
SAPGUISAPGUI SAPGUISAPGUI
Dispatcher
Workprocess
Workprocess
Workprocess
DBDBMS Processes
R/3 Architecture
TP MonitorABAP VM
SAP AG 1999 VLDB ‘99 / 20
Firstdialog step
PBO
Seconddialog step
Thirddialog step
Time
SAP transaction
PAI PBO PAI PBO PAIScreen 2 Screen 3Screen 1
Processingstep
Processingstep
SAP Transactions and Dialog Steps
User commit
SAP AG 1999 VLDB ‘99 / 21
ABAPABAPInterpreterInterpreter
DBDBInterfaceInterface
EXEC SQL.SELECT ...END EXEC.
DatabaseDatabaseLocalLocaltabletablecachecache
SELECT *FROM ... Native SQLOpen SQL
SQL data
SQL data
SQL data
SQL data
Native SQL
Application Server Database Server
DB
Table Caching in R/3 App Server
ABAP Bytecode
SAP AG 1999 VLDB ‘99 / 22
Lock requestfor application object
Enqueue Server
. . .
Lock table inmain memory
. . .
Application Server
Application Level Locking
U WPU WP
Dispatcher
E WPE WP
Dispatcher
WPWP
MessageMessageServerServer
SAP AG 1999 VLDB ‘99 / 23
Application log of alldatabase updates
Asynchronous Database Update: Phase 1
DBVBLOG
Enqueue Server
. . . U WP
Dispatcher
E WP. . .
Application Server
Dispatcher
WP
MessageMessageServerServer
SAP AG 1999 VLDB ‘99 / 24
VBLOG DB
DatabaseUpdate
COMMIT Request
Asynchronous Database Update: Phase 2
. . .
Application Server
Dispatcher
WP
Enqueue Server
. . . U WP
Dispatcher
E WP
Message-Message-ServerServer
SAP AG 1999 VLDB ‘99 / 25
Asynchronous Database Update: Phase 3
Lock table inmain memory
. . .
Application Server
Dispatcher
WP
Enqueue Server
. . . U WP
Dispatcher
E WP
MessageServer
SAP AG 1999 VLDB ‘99 / 26
Start of SAP transaction Start of
update taskEnd ofupdate task
SAP transaction 1
Database transaction 1
SAP transaction 2
Lock of anapplication object
Log of database updates
Release of thelocked application object
End ofSAP transaction
Asynchronous Update Protocol
Work process
Update process
SAP AG 1999 VLDB ‘99 / 27
Agenda
Who are we?11
Where do we come from? 22
Where do we want to go? 33
SAP AG 1999 VLDB ‘99 / 28
DBMS Application Areas
l Laptop / Palmtop / Handheld (e. g. SAP Customer Relationship Management)
l OLTP (Online Transaction Programming) (e. g. SAP R/3)
l Data Warehouse (e. g. SAP Business Information Warehouse)
l Document Server (e. g. SAP Content Server)
l Message Server (e. g. mySAP.com)
l Object-Oriented Database Management (e. g. SAP liveCache, persistent ABAP Objects)
SAP AG 1999 VLDB ‘99 / 29
DBMS Requirements
l Low footprint to (nearly) no footprint (ultra-lite DBMS)
l Fast start-up/shut-down
l Simple or no administration
l Unattended installation and upgrade
Laptop / Palmtop / Handheld
SAP AG 1999 VLDB ‘99 / 30
Application Profile
l Many users
l Short transactions
l High transaction frequencies
l High number of updates
l Simple SQL commands
l Known transaction profile
l Medium to large databases (10 - 500 GB)
l Snapshot of the organization
OLTP
SAP AG 1999 VLDB ‘99 / 31
Database Requirements
l Performance
l Performance
l Performance
l Availability
l Ease of use
OLTP
SAP AG 1999 VLDB ‘99 / 32
Application Profile
l Many users (report producers vs. report consumers)
l Queries (read-only)
l No transactions, no updates, no locking
l Complex, long-running SELECT commands
l Unknown workload
l High command frequency
l Very large databases (500 GB - 2 TB)
l History of the organization
Data Warehouse
SAP AG 1999 VLDB ‘99 / 33
Benefits
l Consistent data across the enterprise
l Better performance for ad-hoc queries, decoupling of OLTP workload from queries/reports
l Easier data access for end-users
Data Warehouse
SAP AG 1999 VLDB ‘99 / 34
DBMS Requirements
l For the OLTP database:
n Mass-data extraction
n Parallel extracts
n Consistent extracts without locking
Data Warehouse
SAP AG 1999 VLDB ‘99 / 35
DBMS Requirements
l For the data warehouse database:
n Fast loading of mass-data
n Fast incremental indexing
n Fast deletion of mass-data (partitions)
n Very large database support (parallel backup/restore, no reorgs, no down-times)
Data Warehouse
SAP AG 1999 VLDB ‘99 / 36
DBMS Requirements
l For the data warehouse database:
n Transactions are irrelevant
n Updates/logging/locking are irrelevant
n But: staging area for data translation/cleaning
n But: forcasting applications need updateable versions
n Checkpoint recovery (= previous state) is sufficient
n Star join support
n Compact indexing
n Aggregate support (maintenance, navigation, statistics/wizard)
Data Warehouse
SAP AG 1999 VLDB ‘99 / 37
Application Profile
l DBMS-based storing of documents (text, image, audio, video)
l Documents are stored in BLOB container fields
l Documents are produced by editors and presented by viewers
l Internal document format and coding is unknown to the DBMS
l Document attributes are stored in descriptive fields
l Documents are organized via nested folders (similar to hierarchical file systems)
l Content is static, life span varies
l Read-only scenarios (user documentation, training)
l Read/write scenarios with shared folders
Document Server (1)
SAP AG 1999 VLDB ‘99 / 38
Application Profile
l The browser is the general document viewer
l Many (remote) users
l Workload is dominated by read accesses
l Client-side caching by proxy servers required to saveroundtrips and bandwidth
l Very large databases (500 GB to 2 TB)
l Knowledge of an organization
Document Server (2)
SAP AG 1999 VLDB ‘99 / 39
Document Server Proxies
BrowserBrowser
Browser
URL -> HTMLURL -> HTML
URL -> HTML
Virtualdocumentserver
Proxyserver
Proxyserver
Proxyserver
Documentserver
SAP AG 1999 VLDB ‘99 / 40
Database Requirements
l Good BLOB implementation (storage overhead, read/write performance, logging overhead)
l Good support for complex selects, especially recursive selectsfor implementing folder hierarchies
l Support of browser-based accesses (URL -> SQL -> HTML)including session pooling
l Search engine (full-text retrieval) integration, but search engine should run on separate server
l Good handling of extremely large databases (parallel backup/restore, no reorgs, no down-times)
Document Server
SAP AG 1999 VLDB ‘99 / 41
Application Collaboration in the Internet
App server App serverHTTP + XML messages
l Asynchronous flow of XML messages (documents)based on HTTP
l Collaboration of loosely coupled applications
SAP AG 1999 VLDB ‘99 / 42
Message Server
App server App server
App server
Messageserver
Collaboration of different applications
HTTP + XMLHTTP + XML
HTTP + XML
Enterprise boundary
SAP AG 1999 VLDB ‘99 / 43
Message Server Concepts (1)
l Availability
l Queuing
l Routing & delivery
l Mapping & translation
SAP AG 1999 VLDB ‘99 / 44
Message Server Concepts (2)
l Store and forward of XML messages (documents) via HTTP
l Service requestors do not need to know service providers, they just address services
l Message servers are message communication hubs
l High availability independent of the service provider availability
l Message translation from XML format A to XML format B needed,due to lack of XML message standardization
SAP AG 1999 VLDB ‘99 / 45
XML Messages
l Messages are short-lived pieces of information
l Message delivery (one-to-one, one-to-many) follows thesend/receive paradigm (send/receive/delete, messages aredelivered exactly once)
l Message translation is based on XML content, otherwise themessage content is not interpretable
l XML messages are typically data used for server/serverinteraction
l XML messages can be forms (data + presentation) suitablefor user/server or user/user interaction
l XML messages can be mail as a special type of a form
SAP AG 1999 VLDB ‘99 / 46
Application Profile
l DBMS-based storing of XML messages
l Messages are stored in BLOB container fields
l Message translation is programmed as stored proceduresoperating on XML data
l Message attributes are stored in descriptive fields
l Messages are kept in a single message table
l Content is very dynamic and short lived
l Reliable message delivery required
l High transaction workload (Insert, Select, Delete)
l Small (active) database size but message tracking will benecessary (message history)
Message Server
SAP AG 1999 VLDB ‘99 / 47
Database Requirements
l Good BLOB implementation (storage overhead, read/write performance, logging overhead)
l Support for HTTP-based services in the database kernel
l Stored procedure concept with XML parser and renderingsupport, programming environment
Message Server
SAP AG 1999 VLDB ‘99 / 48
Application Profile
l Object relationships expressed by OID references
l Explicit navigation and access paths modeled via OIDs
l Uni-directional access paths mirror intended usage
l Low abstraction, programmer-based optimization
l Main-memory biased representation of object relationships
l Fast object navigation (in main-memory)
l No performance degeneration for complex objects
l No query support
l Persistence does not blend well with object-orientation
l Object-oriented concepts are main-memory minded
Object-oriented Applications
SAP AG 1999 VLDB ‘99 / 49
Application Profile
Relational Applications l Object relations expressed by data (foreign keys)
l Implicit and bi-directional access paths via data
l Access paths determined by SQL optimizer
l High abstraction, system-based optimization
l Disk biased represention of object relationships
l Object navigation via key accesses (slow compared to main-memory references)
l Performance degeneration for complex objects(too many table accesses)
l Good query support
l Relational concepts are disk minded
SAP AG 1999 VLDB ‘99 / 50
Object-oriented DBMS Trends
Object-relational DBMS l Extend SQL with object-orientation
Hybrid DBMS l Put SQL and object-oriented technology side-by-side
OO-DBMS l Disk-based implementations of OIDs and object navigation
SAP AG 1999 VLDB ‘99 / 51
DBMS Trends
ORDBMS Approach
SQL layer
Basis layer
Data BladesData CartridgesType Extenders
(third party)
SQL wrapping of object-oriented extensions
Interaction with:* Indexing* Optimizer & execution* Administration
SAP AG 1999 VLDB ‘99 / 52
DBMS Trends
Hybrid Approach: liveCache
Two separate worlds of SQL data on disk and object instances in main-memory,OIDs supported as SQL attributes (anchors), SQL UDFs allow access to object instances
DBMS basis layer
SQL basis layer
SQL layer
Object basis layer
Object layer
Application layer
SAP AG 1999 VLDB ‘99 / 53
liveCache
liveCache
DBMS basis
SQL basis
SQL
OMS basis
OMS
ApplicationStored procedures
(C++)SQL
methodsOMS
methods
Application code is executed inside the DBMS address space
DLL
SQL packet
Application serverDatabase Interface
ABAP Application
Architecture
SAP AG 1999 VLDB ‘99 / 54
liveCache Perfomance
0 50 100 150 200 250 300 350 400
C++
OMS Cache
OMS
SQL SP
SQL
Navigational Access Time (µs)
SAP AG 1999 VLDB ‘99 / 55
Object Persistence
l liveCache supports only transient shared objects
l Support for persistent shared objects is needed
l Simple persistent objects are directly mapped to rows of acorresponding SQL table (class table)
l Persistence concepts for complex objects?
SAP AG 1999 VLDB ‘99 / 56
Complex Objects (1)
Complex objects = hierarchically nested objects
Object type A
Object type B
Object type C Object type D
Nested object example (type level):
SAP AG 1999 VLDB ‘99 / 57
Complex Objects (2)
Instance Level
Object A1
Object B2
Object C3Object D1
Object B1 Object B3 Object B4
Object C4Object C2
Object C1
Object D2
Object D3
Object D4
SAP AG 1999 VLDB ‘99 / 58
Persistence Support for Complex Objects
l Mapping of complex objects to nested tables (multi-level master-detail structures):
n Relational modelling with OO extensions
n Too many tables, too many rows involved
n Too many DML operations needed to assemble the object
l Sequentialize complex object using XML
n Document-type modelling
n Store XML presentation in CLOB column
n Storing and retrieving is very fast
n Queries on complex objects not (directly) supported
SAP AG 1999 VLDB ‘99 / 59
Querying XML Documents (= Complex Objects)
l Search engines for full-text retrieval
n Too general approach to be efficient
n XML documents are structured, not unstructured
n XQL?
l Index XML documents using SQL tables
n Define a mapping from XML components (tags) to a set ofSQL tables and SQL columns
n These tables and their columns act as indexing structure forcomplex objects
n SELECTs an these tables provide object filters which can beused in iterators
n XML documents are the original data, SQL-based indexing structure can be redefined and rebuilt
SAP AG 1999 VLDB ‘99 / 60
Persistence Model for Complex Objects
Object A1
Object B2
Object C3Object D1
Object B1 Object B3 Object B4
Object C4Object C2
Object C1
Object D2
Object D3
Object D4
Main memory
Database
Object indexes (as SQL tables)
MappingObject table
<a A1> <b B1> <c C1> <c C2> </c> </b> <b B2> </b> <b B3> <d D1> <d D2> </d> </b> <b B4> <c C3> <c C4> </c> <d D3> <d D4> </d> </b> </a>
XML representationof complex object as CLOB
SAP AG 1999 VLDB ‘99 / 61
Lessons Learned
l DBMS instances should know about their usage (config param)
l There is an application layer on top of a DBMS
l Do not try to push all data semantics down to the DBMS, a scalable system architecture has a well-defined split of workbetween the application layer and the database layer
l SAP‘s systems technology is driven by application needs not vice versa
SAP AG 1999 VLDB ‘99 / 62
Usage Scenarios for DBMS
