of DBS 1. Architecture of Database · PDF fileof DBS 1. Architecture of Database Systems ... DBMS Architecture ... Layered DBMS Architecture – System Evolution!

Realizationof DBS

1. Architecture of Database Systems1. Architecture of Database Systems

Theo Härderwww.haerder.de

Towards a relational DBS architecture

Evolution of the layer model

Optimization issues

Provisions for ACID properties

Architectural extensions for DBS

© 2011 AG DBIS

Realization of Database Systems – SS 2011

DBS use in CA* environments

Main reference:Theo Härder:DBMS Architecture – The Layer Model and its Evolution, in: Datenbank-Spektrum, dpunkt-Verlag, Heft 13, May 2005, pp. 45-57.

Realizationof DBS

Layer modelevolution


Towards anarchitectureTowards anarchitecture

The “Religious War” – Which Abstraction Level Wins?The “Religious War” – Which Abstraction Level Wins?

Network model• The more complex the data structure, the better• But: very simple operations

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS

DB caching forWeb applicationsDB caching for

Web applications

y p p

Relational model• “Data structure of Spartan simplicity” (E. F. Codd)• Operations with closure property

use of pointers, navigation, record orientation

© 2011 AG DBIS

DBS use in CA*environments


1-2

• Operations with closure property• Each embellishment needs additional operations• Therefore: simplicity is the secret for data independence

Use of values, declarative queries, set orientation

Realizationof DBS




E. W. Dijkstra‘s ideas• “Notes on Structured Programming” (EWD249, 1969)• “The Goto Statement Considered Harmful” (Comm. ACM,

1968)

Structured Programming and Data IndependenceStructured Programming and Data Independence

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

.

Since 1971: D. L. Parnas publishes his ideas concerning • “Information Hiding” at the IFIP conference • Hierarchical structuring of software systems, …

Battle cry translated into the world of data bases:

pointers are the evil!

© 2011 AG DBIS



1-3

The concept of “information hiding” as a software design principle is widely accepted in academic circles. Many successful designs can be seen as successful applications of abstraction and information hiding. On the other hand, most industrial software developers do not apply the idea and many consider it unrealistic …

Realizationof DBS




30 Years Ago: Towards a Layered DBMS Architecture30 Years Ago: Towards a Layered DBMS Architecture

Relational model• Declarative set-oriented access and data independence• How can the postulated independence at the language and

implementation levels be transformed into a system?

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Monolitical approach?• Q1: Select B.Title, P.Year, A.Name

From Books B, Authors A Where B.Author = A.Author

And A.Name = „S*“ And B.Topic = „DBMS“

DB-Server

© 2011 AG DBIS



1-4

DB

• Interface to external memory: read and write of pages (DB is a very long bit string!)

Realizationof DBS




Layered DBMS Architecture – System Evolution!Layered DBMS Architecture – System Evolution!

Permanent evolution expected• Long lifetime of a DBMS > 30 years• Growing information needs:

object types, integrity constraints, ...

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

• New storage structures and access methods, ...• Rapid changes in technology, storage media, ...

But no revolution!• Space and time, data streams?• Unstructured and semi-structured documents? …

© 2011 AG DBIS



1-5

Important challenges• Logical and physical data independence

- Separation of applic. programs and data as strong as possible- Isolation of the interface from all changes in the DBMS

• Encapsulation and hierarchical structuring in the system

Realizationof DBS




Abstraction is Geared to the Mapping HierarchyAbstraction is Geared to the Mapping Hierarchy

Tables/views with set-oriented operations

T2T1

data system

Variety of record types

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Variety of record types and access paths with record-orientedoperations

R1 …R2 Rn

access system

S1 S2 Sm

DB buffer in memoryproviding page access

R1S2

S1

P1

P2

© 2011 AG DBIS



1-6

External storagewith filesof different types

P1

P2 P3

P17

providing page access

storage system

R2

P17

P3

Realizationof DBS




Layer Building in the Layer ModelLayer Building in the Layer Model

Goal: Architecture of a data-independent DBS How many layers are needed?

There is no architectural theory for the construction of large SW systems

Recommended concepts

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

p• Information hiding• Hierarchical structuring

Construction principleoperators: Oi+1, 1, ...

data objects: ti+1, 1, ...

l i„realizes“

layer i + 1 „uses“

Oi+1, p (ti+1, q)

+ implementationlanguage

© 2011 AG DBIS



1-7

“Uses” relation• A uses B, if A calls B and the correct execution of B

is necessary for the complete execution of A

layer i

operators: Oi, 1, ...

data objects: ti, 1, ...

g g

Oi, r(ti, 1, ..., ti, k), ... , Oi, s (ti, 1, ..., ti, n)

Realizationof DBS




5-Layer Model (1)5-Layer Model (1)

transaction programs

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

storage structures

logical access paths

logical data structures

p g

DB b ff i t f

internal record interfaceStore record (in B*-tree), ...

record-oriented DB interfaceFIND NEXT/STORE ...

set-oriented DB interfaceSQL, QBE ...

© 2011 AG DBIS



1-8

external storage

file services

propagation control

device interfacechannel programs

file interfaceread/write block

DB buffer interfacefix page / unfix page

Realizationof DBS






Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

storage structures



p g

addressing units:blocks, files

auxiliary structures: file catalogues, free-placement info, ...

addressing units:tracks, cylinders, channels

© 2011 AG DBIS



1-9

external storage

file services

propagation control

device interfacechannel programs

file interfaceread/write block

Realizationof DBS






Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

storage structures



p g

DB b ff i t f

addressing units:pages, segments

auxiliary structures: pages tables, block tables, ...

addressing units:blocks, files

© 2011 AG DBIS



1-10

external storage

file services

propagation controlDB buffer interface

fix page / unfix page

Realizationof DBS






Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

storage structures



p g

internal record interfaceStore record (in B*-tree), ...

© 2011 AG DBIS



1-11

external storage

file services

propagation controladdressing units:

int. records, B*-trees, hash tables, …

auxiliary structures: page indexes, address tables, ...

addressing units:pages, segments

Realizationof DBS






Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

addressing units:external records, sets, keys, access paths, ...

storage structures



record-oriented DB interfaceFIND NEXT/STORE ...

© 2011 AG DBIS



1-12

auxiliary structures:access path data

addressing units:internal records, B*-trees, hash tables, ...

external storage

file services

propagation control

Realizationof DBS






Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

storage structures



p g

auxiliary structures:

addressing units:tables, views, tuples, ...

set-oriented DB interfaceSQL, QBE ...

© 2011 AG DBIS



1-13

external storage

file services

propagation controladdressing units:

external records, sets, keys, access paths, ...

auxiliary structures:external schema description

Realizationof DBS






tables,views

externalrecords

• • • ab 2 xyz bb 3 xyz

2 xyz 2 ab 3 bb 3 xyz rec. 2rec. 1 rec. 3

1 aa

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

i l

storage structures



internalrecords

DB buffer

aa 1123 ab 2456 bb 3789

record1 record2 record3

record 3

record1

record2record 2

A

B

C

© 2011 AG DBIS



1-14

external storage

file services

propagation controlsegment

mapping

file

externalstorage

A

A A’ B

B C

C’C

1

2

3

4

5

1 2 3 4 5

Realizationof DBS




The Magic TriangleThe Magic Triangle

“Classical” key requirements for a DBMS

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

performance

© 2011 AG DBIS



1-15

reliability functionality

Realizationof DBS




Layer Model – Role of n?Layer Model – Role of n?• For DBMSs, it is especially true:

- “Performance is not everything, but without performance everything is worth nothing!”

- What is the appropriate number of layers?

• Complexity / reliability of the mapping layer vs

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

• Complexity / reliability of the mapping layer vs. optimization potential of the layer

• Optimized layer model at run time

Open Scan(IB(Topic),Topic=’DBMS’,Topic>’DBMS’)/*SCB1*/Sort Access (SCB1) ASC AuthId Into T1 (AuthID, …)Close Scan (SCB1)Open Scan (IA(Name), Name>=’S’, Name>’S’) /*SCB2*/S t A (SCB2) ASC A thID I t T2 (A thID )

SQL query Q1

L4

© 2011 AG DBIS



1-16

Sort Access (SCB2) ASC AuthID Into T2 (AuthID, …)Close Scan (SCB2)Open Scan (T1, BOF, EOF) /*SCB3*/Open Scan (T2, BOF, EOF) /*SCB4*/While Not FinishedDo

Fetch Tuple (SCB3, Next, None)Fetch Tuple (SCB4, Next, None). . .

End

L4

L3

L2large

DB buffer

Realizationof DBS




DBMS – Paradigm of a Generic SystemDBMS – Paradigm of a Generic System

Pre-compilation and early binding are good ideas• What – table-, attribute-descriptions, indexes,

access rights, assertions, …• When – spectrum of points in time

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

When spectrum of points in time

translation time AP run time (RT)

invalidation by schema modifications

accesses (at RT):

efficient

accesses (at RT):

expensivebalancerequired!

© 2011 AG DBIS



1-17

Layer-crossing optimization• Avoidance of thrashing: L1 – L4/L5• Reservation of buffer space (Hot Set Model)• Optimization of lock management (Lock Contention)

data dependent! data independent!

Realizationof DBS




What can go wrong, will go wrong …What can go wrong, will go wrong …

Goal of Development Build a system used by millions of people that is always available –out less than 1 second per 100 years = 8 9’s of availability! (J. Gray: 1998 Turing Lecture)

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

1998 Turing Lecture)

Availability today (optimistically):• For Web sites: 99%• For well-administrated systems: 99,99%• zSeries OS only: 99,9999%• For an entire information system –

realistic goal today: 99,999%

© 2011 AG DBIS



1-18

There are some 9‘ missing (to be achieved until ~2050?)

ACID transactions are introduced to guarantee far-reaching assertions concerning the quality of data

Jim Gray: We have added three 9s in 45 years (starting with 90%), or about 15 years per order-of-magnitude improvement in availability. We should aim for five more 9s: an expectation of one second outage in a century. This is an extreme goal, but it seems achievable if hardware is very cheap and bandwidth is very high. One can replicate the services in many places, use transactions to manage the data consistency, use design diversity to avoid common mode failures, and quickly repair nodes when they fail. Again, this is not something you will be able to test: so achieving this goal will require careful analysis and proof.

Realizationof DBS




Transaction as Dynamic Control StructureTransaction as Dynamic Control Structure

Atomicity“All or nothing” property of any DBMS action

C i t d ti i t it

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Consistency and semantic integrity of the DB is endangered by erroneous data and operations of an application program

Isolated executionmeans “logical single-user mode”

© 2011 AG DBIS



1-19

Durability requires that modified data of successful transactions must survive any type of failure

Realizationof DBS




Transactions are abstractions• Building blocks: atomic actions (AAs)• AAs are abstractions, too• Even if AAs would be implementable in an atomic way,

Atomicity is not a Natural Property of ComputersAtomicity is not a Natural Property of Computers

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

p y,hierarchy of AAs would not!

SQL-Op

© 2011 AG DBIS



1-20

Realizationof DBS




SQL guarantees “statement atomicity”• “Do it twice” or “Do it once”

And, of course, transaction atomicity Key mechanisms:

Isolation Requirements of a TransactionIsolation Requirements of a Transaction

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Key mechanisms: • Concurrency control and logging & recovery

SQL-Op1 SQL-Opn. . .

© 2011 AG DBIS



1-21

Realizationof DBS




Modeling for Concurrency ControlModeling for Concurrency Control Correctness criterion: conflict serializability History writer

• Read/write model (Page Model)• Record operations

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

• Record operations• Logical predicates

U I RUpdate t1 From PersInsert 4711 into IPers(Pnr)

data system

S S DSelect * From Pers Where PDelete From Pers Where Q

© 2011 AG DBIS



1-22

storage system

r1(x) w2(y) w1(y)Read PageWrite Page

access systemInsert 4711 into IPers(Pnr)

Realizationof DBS




Aspects of DB-RecoveryAspects of DB-Recovery

Consistency of a layer requires layer-specific consistency of all layers below

system hierarchy + DB-consistency

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

system hierarchy + DB-consistency

transaction consistency

TAP IncreaseSalary

action consistency Update IPers(Pnr) Using 4711

tables,views, …

API consistency Update Pers Set … Where Q

records

© 2011 AG DBIS



1-23

records, fields, … consistency of

elementary opsInsert 4711 Into Page 0815

pages

tracks, cylinders, …,

device-/file-consistency

Read Page / Write Page

Realizationof DBS




DB Consistency and LoggingDB Consistency and Logging

SQL-operation hierarchy

TA

System hierarchy +

TA consistency

DB consistency in case of crash

TA-programparameter

Log granule

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

SQL-Op1

Opn

SQL-Opm

Op1 . . .

. . .

. . .. . .tables,views, ...

records, fields, ...

API consistency

actionconsistency

DML operation

action

parameter

action(elementary) l O

© 2011 AG DBIS



1-24

w2

,

pages, ...

destroyeddevice

deviceconsistencypage

archive file/archive log

(elementary)

w1

el. Op

Realizationof DBS




Non-Atomic Propagation MethodsNon-Atomic Propagation Methods

DB is “chaos consistent” after a crash If blocks are destroyed: device recovery Otherwise: device consistency as minimal condition, using

• Page logging: entire pages for Redo/Undo (extremely expensive!)

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Page logging: entire pages for Redo/Undo (extremely expensive!)• Trick: physiological logging using LSNs

tables,views, ...

SQL-Op1

Opn

Op1 . . .

© 2011 AG DBIS



1-25


pages, ...

destroyed device

action consistency

device consistency

elem. op.

Opn

Op1

w2

w1

el. Op

Realizationof DBS




Atomic Propagation MethodsAtomic Propagation Methods

Operations of higher layers span several pages• Log unit must perform Undo/Redo in several pages

• Pages must be completely or not at all in the DB

Role of checkpoints (limitation of Redo recovery)

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Role of checkpoints (limitation of Redo recovery)

TA

API consistency

tables,views, ...

TA consistency

action consistency

SQL-Op1

Opn

SQL-Opm

Op1 . . .

. . .

. . .. . .

© 2011 AG DBIS



1-26


pages, ...

y Opn

Op1 . . .

w2w1

el. Op

Realizationof DBS




How far does this Architecture Model Reach?How far does this Architecture Model Reach?

The invariants of DBMS processing determine the mapping steps in the DBMS architecture• Layer model aims at the navigational or set-oriented

processing of record structured data

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

communication /adaptation / mediation

processing of record-structured data• Horizontal distribution of DBMS processing in

- Distributed DBMS (SN/SD)- Federated DBMS, Multi-DBS- …

© 2011 AG DBIS



1-27

data system

access system

storage system

DB

data system

access system

storage system

DB

Realizationof DBS




Classification of Distributed DBS Classification of Distributed DBS Approach to distribution

• Partitioning of data (possibly including replication)• Communication among processes to access distant data

Desired visibility “local”i

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

user view

global system view: distribution

localsystem view

localsystem view

Homogeneous system model

© 2011 AG DBIS



1-28

g y

data system

access system

storage system

1

2

3

DBSlayers

DBMS

data system

access system

storage system

Realizationof DBS




Classification of Distributed DBS (2)Classification of Distributed DBS (2)

Homogeneous system model

data system1 D data system

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Several (homogeneous) realization alternatives• Global system view is obtained by a (additional) DBS layer

data system

access system

storage system

1

2

3

DBSlayers

DBMS

data system

access system

storage system

© 2011 AG DBIS



1-29

• Global system view is obtained by a (additional) DBS layer • Distribution of entire DML operations resp. of suboperations

(layer 1)• Distribution of internal record operations (layer 2)• Distribution of page accesses (layer 3)

Realizationof DBS




Multi-Computer DBSMulti-Computer DBS

DBMS

AP

DBMS DBMSDBMS

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Distribution under control of the DBS• Cooperation among (homogeneous) DBS

DB-P1

DBMS

DB-P2

DB-P1

DBMS

DB-P3

DBMSDBMS

© 2011 AG DBIS



1-30

p g ( g )• Location transparency for AP (a single DB schema):

Single system image• Flexibility for the distribution of data and load

Realizationof DBS




Multi-Computer DBS (2)Multi-Computer DBS (2)

AP

DBMSDBMS DBMSDBMS

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Architecture classes1. DB partitioning (Shared Nothing, DDBS) • Each node owns full functionality and manages one DB partition

• Data replication increases availability and reliabilityf ff f “ ” f l

DB-P2

DB-P1DB-P1

DB-P3

© 2011 AG DBIS



1-31

Minimization of effects of “remote” failures, Failure masking by redundancy

• Processing principle: The load follows the data

2. DB sharing (Shared Disk)• Local execution of DML operations• Processing principle: Data transport to the computer processing the TA• Local reachability of disks (external storage)

Realizationof DBS




Processing Overhead in DDBSProcessing Overhead in DDBSQ1: SELECT ...

FROM EMP EWHERE E.DNO = ‘K55’

AND E.SAL < E.PROV

Function Request Shipping

Q2: SELECT ...FROM EMP E, DEPT DWHERE E.DNO = D.DNO

AND E.DNO = ‘K55’

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

data system1

access system1

DB buffer1

data system2

access system2

DB buffer2

DBS1 DBS2. . .

to DBS2

entire Q1Q1 result

Scan for all recordswith E.DNO = ‘K55’request next recordtransmit record

for all pages of EMP file:request next page

to DBS1

D.DNO = ‘K55’ to DBS1result 1 ‘K55’ record

Scan for all recordswith D.DNO = ‚K55‘request next recordtransmit record

for all pages of DEPT file:request next page

© 2011 AG DBIS



1-32

Assumptions:- EMP file has 105 pages - DEPT file has 103 pages- E.DNO = ‘K55’: 100 records in EMP - E.DNO = ‘K55’ AND E.SAL < E.PROV:

5 records in EMP

1

storage system1

DEPT

2

storage system2

EMP

q p gtransmit page

q p gtransmit page

Realizationof DBS




DB CachingDB Caching Caching of DB data in Web applications

• Caching is a proven technique to improve scalability and performance in large, distributed systems !

• Pre-generation of documents having dynamic contents• Acceptance of stale information (inconsistencies)• Information for the user by adding the generation time

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

• Caching methods- Passive or exact-match caching of query results (QR caching) - Static caching (replication) of full tables, materialized

views or sub-tables (specified via simple predicates)- Declarative caching requires BE metadata to dynamically load the

cache (caching of several QRs in shared tables)- On-demand caching applies BE metadata and “hints” of the TA to

dynamically fill or replace data in the cache

CustomersW HERE R i ‘W ’

© 2011 AG DBIS



1-33

applicationserver

DBS cache

DBS cache

Web server

applicationlogic

Front-End (FE)DB server

Back-End (BE)DB server

browser requirementsof clients

DB server

W HERE Region=‘West’

CustomersW HERE Region=‘East’

applicationserver

Realizationof DBS




DB Caching (2)DB Caching (2)

Web

• DB caching “close” to the application server- Acceleration of the DB read access- (So far still) transfer of update statements to BE- Consistency problems

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

application

server

data system

access system

storage system

data system

access system

storage system

applicationserver

applicationserverlogic

FE-DB server

© 2011 AG DBIS



1-34

data system

access system

storage system

BE-DB server

CustomersWHERE Region = ‘East’

CustomersWHERE Region = ‘West’

Realizationof DBS




DB Caching (3)DB Caching (3)

Caching of DB data (cont.)• Desirable: DB caching with

evaluation of queries having simple predicates and n joins• Columns of tables can be specified in the DB cache with property

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Columns of tables can be specified in the DB cache with property “column complete” (domain complete) i.e., if a column value is found in the cache, the cache mgr guarantees that all records (rows) having this value are in the cache;hence, UNIQUE columns are always column complete

• “Linkage” of tables in the cache by Referential Cache Constraints- Most important case: primary-/foreign-key relationships

(Owner-Member)

© 2011 AG DBIS



1-35

( )- If a PK value is found in the cache, the cache mgr guarantees that all

records having the same FK value are in the cache- In this way, so-called Cache Groups can be formed, which support join

operations in the cache

Realizationof DBS




Layer Models for Client/Server DBSLayer Models for Client/Server DBS

Critical for the DB-related performance capabilities• Kind and complexity of DB operations (set-oriented, navigational)• Exploitation of reference locality when accessing data

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Processing concept so far• In all architectural proposals, “shipping of the queries”

to the (server) DBS was assumed (query shipping approach):• DML statement is sent to the server • After its processing, the result is made available to the application

(client) • Important property: there is no replication of DB data

© 2011 AG DBIS



1-36

p p p y p

No exploitation of inherent reference locality in the client

Realizationof DBS




Layer Models for Client/Server DBS (2)Layer Models for Client/Server DBS (2)

Further processing concept:

The following approach is applied by most of the object-oriented DBS (OODBS) (data shipping approach):

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

oriented DBS (OODBS) (data shipping approach):• All data needed for the evaluation of a query,

are transported to the client and are buffered there • Then, the query (or a sequence of queries) is evaluated in the

client buffer • Note: The query complexity is severely restricted

- Typically only navigation operations- Queries with simple search arguments (SSA) are sometimes supported,

b t j i ti ti

© 2011 AG DBIS



1-37

but no join operations, aggregations, ...

This concept is primarily advantageous in case of high reference locality. It is utilized by the Checkout/Checkin concept

Realizationof DBS




Layer Models for Client/Server DBS (3)Layer Models for Client/Server DBS (3) Advantages of data shipping

• Local data buffering reduces interactions between client and server(reduction of network load; avoidance of repeated server queries)

• Aggregated computer power and totally available storage capacity of the system are increased

• Server load is reduced

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Server load is reduced • System scalability is improved

Client/Server architecture: principle

WDBMS

Work-stations Tools

WDBMS

Tools

WDBMS

Tools

privat DB

© 2011 AG DBIS



1-38

SDBMSserver

server DB

privat DB

• Local disks for private DBand log data

• Enhancement of performance: distributed server system, parallel DB processing

Realizationof DBS




Layer Models for Client/Server DBS (4)Layer Models for Client/Server DBS (4)

Client/Server architectures for object-oriented DBS

query managerobject buffer

application

communicationClie

nt

application

object manager

access system

page buffer

applicationquery manager

query result buffer

communication

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

object manager

access system

erve

rpage buffer

communication

page buffer

query managertransfer buffer

data system

access system

© 2011 AG DBIS



1-39 In object buffers, typically navigational processing is performed!

storage system

DBObject Server

Se storage system

DBPage Server

storage system

DBQuery Server

Vertical distribution of layers: DB functionality in Server and Client Seamless connection with the application program, but many clients

Realizationof DBS




Processing in DB-based Design Systems — ExampleProcessing in DB-based Design Systems — Example

Construction of composed solids

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

- *

U *

boundary representation (simplified)construction tree

© 2011 AG DBIS



1-40

View of the designer at the dialog interface(with operations of the application model)

Realizationof DBS




Processing in DB-based Design Systems — Example (2)Processing in DB-based Design Systems — Example (2)

ER representation of the product model for composite parts

ProductModel

1 1

n n

mm

Physical Technological

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

. . . . . .n

n

BREP PART

PhysicalModel

TechnologicalModel

1

111

n

nn

m

nn

m

EDGE

PARAMETER ENVIRONMT

nn

mn1 1

1

Structure ModelTRACK

n

11

n

FACE

MPEMV

SPEMV

nm

n

1

© 2011 AG DBIS



1-41

Abstract view of the data model interface

mm

n POINT

Geometry Model

n 1

cornerinformation

non-redundantBREP information

track information

neighborhood

Realizationof DBS





Mapping of BREP information into Relational Model (fragment)

BREP (BID, . . . )

FACE (FID, . . . , BID )

SELECT PID, X-COORD, Y-COORD, Z-COORDFROM POINTWHERE PID IS IN

SELECT PID

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

FA-ED (FID, EID, . . . )

EDGE (EID, ELENGTH, . . . )

ED-PO (EID, PID, . . . )

POINT (PID, . . . )

SELECT PIDFROM ED-POWHERE EID IS IN

SELECT EIDFROM EDGEWHERE ELENGTH > ’10’AND EID IS INSELECT EIDFROM FA-EDWHERE FID IS IN

© 2011 AG DBIS



1-42

Relational model is not well suited! (object-oriented or object-relational models enable more “natural” modeling and processing

WHERE FID IS INSELECT FID

FROM FACEWHERE BID = ’4711’

a) DB schema b) query example

Realizationof DBS





Evaluation of object creation and manipulation

POLYCYL(n) 5 25 50

Dialog interface 1 1 1

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Dialog interface

application model int.

data model interface.

1

64

1958

1

304

15078

1

604

41828

a) Generation of a cylinder approximated by a polyhedra

operation #DMLs

UNION(CYL(3), CYL(3))



16051

20658

31432

© 2011 AG DBIS



1-43

Exploitation of inherent reference localityrequired in the client

b) UNION of two parameterized polyhedras

Realizationof DBS




DBS Use in Design Environments DBS Use in Design Environments Engineering applications (CA*) need local data management

• Exploitation of reference locality during processing• Long-term storage (object buffer) of large data sets in main memory• Local representation of objects at the client window (screen)• Fast transformation and processing of complex objects

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Fast transformation and processing of complex objects

DBS which should support design applications, need appropriate functionality for data management / transaction support at client and server side

New requirements• Design tasks (CA*), knowledge-based applications, ...• DB-supported processing of large data sets in client• Long transactions

© 2011 AG DBIS



1-44

• Long transactions• Large data sets and reference locality

Consequences: • Data management in client and server• Replication of data in DB buffer and object buffer• Increased overhead for the integrity control

Realizationof DBS




DBS Use in Design Environments (2)DBS Use in Design Environments (2)

DB kernel architecture3D-CADCAM

dialog interface

Workstation-oriented or client/server DB architectures• They should support so-called Non-Standard applications• Mapping of the DB processing onto client/server architectures

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

data system

access system

model mappingfor 3D-CAD

3D-CADapplication

model mappingfor CAM

CAM

privatDB

privatDB data model

interface

application model interface

• • •

DBS

Ker

application

Separation of

© 2011 AG DBIS



1-45

storage system

shared DB

S n• Application-independentfunctions in the server and

• Application-related functions in the client

Key concepts• Data model for complex objects at the server interface• Tailor-made data model at the application interface• Object buffer to exploit reference locality and to save communication requests

Realizationof DBS




DBS Use in Design Environments (5)DBS Use in Design Environments (5)

Processing support by• Set-oriented data supply (generic query language)• Application-related representation of objects (views)• Flexible integrity control (immediate, deferred)

(assertions, referential integrity, ECA rules, ...)• Set oriented propagation

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

• Set-oriented propagation Minimization of dependencies between client and server

(communication, failure isolation)

High performance requirements• For navigating tool operations: >105 ref./sec for CAD• Imply exploitation of locality close to the application

(all objects with re-reference in the object buffer)• Require memory-based addressing of the DB objects

(when >3 references : Pointer Swizzling)

© 2011 AG DBIS



1-46

Further aspects• Long duration of design work (weeks/months)• How can synchronization be done? Typically, conflicting accesses to design data are rare• Support of versions (special lock modes)• Controlled cooperation among several designers

(special synchronization measures or exchange operations)

Realizationof DBS




Processing Principle: Load, Operate, IntegrateProcessing Principle: Load, Operate, Integrate

Data requirements for a design transaction• Data loading by one or several Checkout requests• Design objects are stored into the object buffer• Use of special storage structures and access paths

Data manipulation

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

Data manipulation• In case of complex-structured objects: use of hierarchical cursors• Entire object processing is local: accumulation of updates• Application model: set of ADTs

Processing control and recovery measures• Client DBS propagates persistent intermediate state in a private DB• Local rollback measures by Savepoints (SAVE, RESTORE)• Interruption of processing (SUSPEND, RESUME)

© 2011 AG DBIS



1-47

What about the ACID properties in case of design transactions?

Data propagation• By Checkin at the end of the design transaction,• Then, (deferred) checking of integrity conditions is performed

in the server DB

What happens when a violation of integrity is detected?

Realizationof DBS




Load, Operate, IntegrateLoad, Operate, Integrate

t o o ls

o b je c tb u f fe r

C l ie n t S e r v e rS E L E C T . ..F R O M .. .W H E R E . ..

1

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

C D B M S

t o o ls

o b je c tf f

t r a n s fe rb u f f e r

S D B M S

M o d i fi c a t i o n

c h e c k o u t

23

4

© 2011 AG DBIS



1-48

C D B M S

b u f fe r

t r a n s f e r b u f f e r

S D B M S

c h e c k i n

5 6

Realizationof DBS




Model of a Design TransactionModel of a Design Transaction

design transaction

START END

RESUMESUSPENDworkstation

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

local processing local processing local processingSAVE RESTORE

Checkout1 Checkout3Checkout2

isolation of design object 1isolation of design object 2

isolation of design object 3

server

© 2011 AG DBIS



1-49

Characteristics: 0 .. n Checkout0 .. 1 Checkinlong duration

Storage of intermediate states of a design transaction to:• To interrupt processing (SUSPEND, RESUME)• Rollback to earlier processing states (SAVE, RESTORE)

Realizationof DBS




SummarySummary

General aspects of the layer model• Layer model is general explanation model for the

DBS implementation • Layer mapping can be refined/coarsened when needed:

applicability for distributed DBS, client/server DBS, ...

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

• Design of suitable interfaces requires great implementation experience

• Concrete implementations sometimes violate the layer isolationfor performance reasons

( critical functions have “reach-through”)

Implementation concepts• The fundamental implementation concepts of centralized DBS can

b l f d i h d f di ib d DBS

© 2011 AG DBIS



1-50

be also found in each node of a distributed DBS• Knowledge of implementation concepts enables the objective

evaluation and comparison of existing DBS • Their accurate knowledge is a prerequisite for performance

analyses resp. estimation of the system behavior• By identifying few, fundamental concepts, one obtains a deepened

understanding of what is meant by “data independence”

Realizationof DBS




Summary (2)Summary (2)

DB caching• Caching vs. replication for the support of Web-based DB-AW• Caching is the proven technique to improve scalability and

performance in large, distributed systems! • Columns of tables can be attributed in a DB cache with the

t “ l l t ”

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

property “column complete” • Cache Groups support join operations in case of DB caching

Classification of distributed DBS• Single System Image• Flexibility for data- and load distribution• Multi-computer DBS as Shared Nothing or Shared Disk Systems

Client/server systems

© 2011 AG DBIS



1-51

Client/server systems• Data Shipping vs. Query Shipping• Data Shipping is primarily advantageous in case of high reference

locality. It is employed by the Checkout/Checkin concept• Object-oriented DBS as Page, Object, or Query server• Typical form of processing in the client is navigational

(for embedded systems and design systems)• Updates of DB data required in client and server

Realizationof DBS




Further ReferencesFurther References

• Chamberlin, D. D.; Astrahan, M. M.; Blasgen, M. W., Gray, J. et al.:A History and Evaluation of System R. Commun. ACM 24(10): 632-646 (1981)

• Härder, T.; Reuter, A.: Principles of Transaction-Oriented Database Recovery. ACM Comput. Surv. 15(4): 287-317 (1983)

Client/serverDBS

Client/serverDBS

ACIDprovisions

ACIDprovisions

DistributedDBS

DistributedDBS


Web applications

y p ( ) ( )• Härder, T.; Reuter, A.: Concepts for Implementing a Centralized

Database Management System. Proc. Int. Computing Symp. on Application Systems Development, 1983, Nürnberg, B.G. Teubner-Verlag, 28-60

• Härder, T.; Reuter, A.: Database Systems for Non-Standard Applications. Proc. Int. Computing Symp. on Application Systems Development, 1983, Nürnberg, B.G. Teubner-Verlag, 452-466

• Härder, T.; Reuter, A.: Architektur von Datenbanksystemen für Non Standard Anwendungen BTW 1985: 253 286

© 2011 AG DBIS



1-52

Non-Standard-Anwendungen. BTW 1985: 253-286• Härder, T.: DBMS Architecture – New Challenges Ahead.

Datenbank-Spektrum, dpunkt-Verlag, Heft 14, Aug. 2005, pp. 38–48.

• Härder, T., Rahm, E.: Datenbanksysteme: Konzepte und Techniken der Implementierung, 2nd edition, Springer 2001, Kap. 1

Documents

of DBS 1. Architecture of Database · PDF fileof DBS 1. Architecture of Database Systems ... DBMS Architecture ... Layered DBMS Architecture – System Evolution!