Distributed DBMSs Advanced Concepts

8/3/2019 Distributed DBMSs Advanced Concepts

1/97

Chapter 23

Distributed DBMSs - Advanced

Concepts

Transparencies

Pearson Education Limited 1995, 2005


2/97

2

Chapter 23 - Objectives

Distributed transaction management.

Distributed concurrency control.

Distributed deadlock detection.

Distributed recovery control.

Distributed integrity control.

X/OPEN DTP standard.

Distributed query optimization.

Oracles DDBMS functionality.



3/97

3

Distributed Transaction Management

Distributed transaction accesses data stored atmore than one location.

Divided into a number ofsub-transactions, one

for each site that has to be accessed, representedby an agent.

Indivisibility of distributed transaction is stillfundamental to transaction concept.

DDBMS must also ensure indivisibility of eachsub-transaction.



4/97

4

Distributed Transaction Management

Thus, DDBMS must ensure:

synchronization of subtransactions with otherlocal transactions executing concurrently at a

site;synchronization of subtransactions withglobal transactions running simultaneously atsame or different sites.

Global transaction manager (transactioncoordinator) at each site, to coordinate globaland local transactions initiated at that site.



5/97

5

Coordination of Distributed Transaction



6/97

6

Distributed Locking

Look at four schemes:

Centralized Locking.

Primary Copy 2PL.

Distributed 2PL.

Majority Locking.



7/97

7

Centralized Locking

Single site that maintains all locking information.

One lock manager for whole of DDBMS.

Local transaction managers involved in global

transaction request and release locks from lockmanager.

Or transaction coordinator can make all locking

requests on behalf of local transaction managers.

Advantage - easy to implement.

Disadvantages - bottlenecks and lower reliability.



8/97

8

Primary Copy 2PL

Lock managers distributed to a number of sites.

Each lock manager responsible for managing locks

for set of data items.

For replicated data item, one copy is chosen as

primary copy, others areslave copies

Only need to write-lock primary copy of data item

that is to be updated. Once primary copy has been updated, change can

be propagated to slaves.



9/97

9

Primary Copy 2PL

Disadvantages - deadlock handling is more

complex; still a degree of centralization in

system.

Advantages - lower communication costs andbetter performance than centralized 2PL.



10/97

10

Distributed 2PL

Lock managers distributed to every site.

Each lock manager responsible for locks for

data at that site.

If data not replicated, equivalent to primarycopy 2PL.

Otherwise, implements a Read-One-Write-All

(ROWA) replica control protocol.



11/97

11

Distributed 2PL

Using ROWA protocol:

Any copy of replicated item can be used for

read.

All copies must be write-locked before itemcan be updated.

Disadvantages - deadlock handling more

complex; communication costs higher thanprimary copy 2PL.



12/97

12

Majority Locking

Extension of distributed 2PL.

To read or write data item replicated at n sites,

sends a lock request to more than half the n sites

where item is stored. Transaction cannot proceed until majority of

locks obtained.

Overly strong in case of read locks.



13/97

13

Distributed Timestamping

Objective is to order transactions globally so

older transactions (smaller timestamps) get

priority in event of conflict.

In distributed environment, need to generateunique timestamps both locally and globally.

System clock or incremental event counter at

each site is unsuitable.

Concatenate local timestamp with a unique site

identifier: .



14/97

14

Distributed Timestamping

Site identifier placed in least significant position

to ensure events ordered according to their

occurrence as opposed to their location.

To prevent a busy site generating largertimestamps than slower sites:

Each site includes their timestamps in messages.

Site compares its timestamp with timestamp in

message and, if its timestamp is smaller, sets it

to some value greater than message timestamp.



15/97

15

Distributed Deadlock

More complicated if lock management is notcentralized.

Local Wait-for-Graph (LWFG) may not showexistence of deadlock.

May need to create GWFG, union of all LWFGs.

Look at three schemes:

Centralized Deadlock Detection.

Hierarchical Deadlock Detection.Distributed Deadlock Detection.



16/97

16

Example - Distributed Deadlock

T1 initiated at site S1 and creating agent at S2,

T2 initiated at site S2 and creating agent at S3,

T3

initiated at site S3

and creating agent at S1

.

Time S1 S2 S3

t1 read_lock(T1, x1) write_lock(T2, y2) read_lock(T3, z3)

t2 write_lock(T1, y1) write_lock(T2, z2)t3 write_lock(T3, x1) write_lock(T1, y2) write_lock(T2, z3)



17/97

17

Example - Distributed Deadlock



18/97

18

Centralized Deadlock Detection

Single site appointed deadlock detection

coordinator (DDC).

DDC has responsibility for constructing and

maintaining GWFG. If one or more cycles exist, DDC must break

each cycle by selecting transactions to be rolled

back and restarted.



19/97

19

Hierarchical Deadlock Detection

Sites are organized into a hierarchy.

Each site sends its LWFG to detection site above

it in hierarchy.

Reduces dependence on centralized detectionsite.



20/97

20

Hierarchical Deadlock Detection



21/97

21

Distributed Deadlock Detection

Most well-known method developed by

Obermarck (1982).

An external node, Text, is added to LWFG to

indicate remote agent. If a LWFG contains a cycle that does not involve

Text, then site and DDBMS are in deadlock.



22/97

22


Global deadlock may exist if LWFG contains a

cycle involving Text.

To determine if there is deadlock, the graphs

have to be merged. Potentially more robust than other methods.



23/97

23




24/97

24


S1: Text T3 T1 TextS2: Text T1 T2 TextS3: Text T2 T3 Text

Transmit LWFG for S1 to the site for which

transaction T1 is waiting, site S2.

LWFG at S2 is extended and becomes:

S2: Text T3 T1 T2 Text



25/97

25


Still contains potential deadlock, so transmit

this WFG to S3:

S3: Text T3 T1 T2 T3 Text GWFG contains cycle not involving Text, so

deadlock exists.



26/97

26


Four types of failure particular to distributedsystems:

Loss of a message.

Failure of a communication link.Failure of a site.

Network partitioning.

Assume first are handled transparently by DCcomponent.



27/97

27

Distributed Recovery Control

DDBMS is highly dependent on ability of all

sites to be able to communicate reliably with

one another.

Communication failures can result in networkbecoming split into two or more partitions.

May be difficult to distinguish whether

communication link or site has failed.



28/97

28

Partitioning of a network



29/97

29

Two-Phase Commit (2PC)

Two phases: a voting phase and a decision phase.

Coordinator asks all participants whether they

are prepared to commit transaction.

If one participant votes abort, or fails torespond within a timeout period, coordinator

instructs all participants to abort transaction.

If all vote commit, coordinator instructs all

participants to commit.

All participants must adopt global decision.



30/97

30

Two-Phase Commit (2PC)

If participant votes abort, free to abort

transaction immediately

If participant votes commit, must wait for

coordinator to broadcast global-commit orglobal-abort message.

Protocol assumes each site has its own local log

and can rollback or commit transaction reliably.

If participant fails to vote, abort is assumed.

If participant gets no vote instruction from

coordinator, can abort. Pearson Education Limited 1995, 2005


31/97

31

2PC Protocol for Participant Voting Commit



32/97

32

2PC Protocol for Participant Voting Abort



33/97

33

2PC Termination Protocols

Invoked whenever a coordinator or participant fails to

receive an expected message and times out.

Coordinator

Timeout in WAITING state

Globally abort transaction.

Timeout in DECIDED state

Send global decision again to sites that have not

acknowledged.



34/97


35/97

35

State Transition Diagram for 2PC

(a) coordinator; (b) participant



36/97

36

2PC Recovery Protocols

Action to be taken by operational site in event of

failure. Depends on what stage coordinator or

participant had reached.

Coordinator Failure

Failure in INITIAL state

Recovery starts commit procedure.

Failure in WAITING state

Recovery restarts commit procedure.



37/97

37

2PC Recovery Protocols (Coordinator Failure)

Failure in DECIDED state

On restart, if coordinator has received all

acknowledgements, it can complete

successfully. Otherwise, has to initiatetermination protocol discussed above.



38/97

38

2PC Recovery Protocols (Participant Failure)

Objective to ensure that participant on restartperforms same action as all other participants andthat this restart can be performed independently.

Failure in INITIAL state Unilaterally abort transaction.

Failure in PREPARED state

Recovery via termination protocol above.

Failure in ABORTED/COMMITTED states

On restart, no further action is necessary.



39/97

39

2PC Topologies



40/97

40

Three-Phase Commit (3PC)

2PC is nota non-blocking protocol.

For example, a process that times out after

voting commit, but before receiving global

instruction, is blocked if it can communicate onlywith sites that do not know global decision.

Probability of blocking occurring in practice is

sufficiently rare that most existing systems use

2PC.



41/97

41


Alternative non-blocking protocol, called three-phase commit (3PC) protocol.

Non-blocking for site failures, except in event offailure of all sites.

Communication failures can result in differentsites reaching different decisions, therebyviolating atomicity of global transactions.

3PC removes uncertainty period for participantswho have voted commit and await globaldecision.



42/97

42


Introduces third phase, called pre-commit,

between voting and global decision.

On receiving all votes from participants,

coordinator sends global pre-commit message. Participant who receives global pre-commit,

knows all other participants have voted commit

and that, in time, participant itself will definitely

commit.



43/97

43

State Transition Diagram for 3PC

(a) coordinator; (b) participant



44/97

44

3PC Protocol for Participant Voting Commit



45/97

45

3PC Termination Protocols (Coordinator)

Timeout in WAITING state

Same as 2PC. Globally abort transaction.

Timeout in PRE-COMMITTED state Write commit record to log and send GLOBAL-

COMMIT message.

Timeout in DECIDED state

Same as 2PC. Send global decision again to sites

that have not acknowledged.



46/97

46

3PC Termination Protocols (Participant)

Timeout in INITIAL state

Same as 2PC. Unilaterally abort transaction.

Timeout in the PREPARED state

Follow election protocol to elect new coordinator.

Timeout in the PRE-COMMITTED state

Follow election protocol to elect new coordinator.



47/97

47

3PC Recovery Protocols (Coordinator Failure)


Recovery starts commit procedure.

Failure in WAITING state

Contact other sites to determine fate of transaction. Failure in PRE-COMMITTED state

Contact other sites to determine fate of transaction.

Failure in DECIDED state If all acknowledgements in, complete transaction;otherwise initiate termination protocol above.



48/97

48

3PC Recovery Protocols (Participant Failure)


Unilaterally abort transaction.

Failure in PREPARED state

Contact other sites to determine fate oftransaction.

Failure in PRE-COMMITTED state

Contact other sites to determine fate oftransaction.

Failure in ABORTED/COMMITTED states

On restart, no further action is necessary.



49/97

49

3PC Termination Protocol After New Coordinator

Newly elected coordinator will send STATE-REQ message to all participants involved inelection to determine how best to continue.

1. If some participant has aborted, then abort.

2. If some participant has committed, thencommit.

3. If all participants are uncertain, then abort.

4. If some participant is in PRE-COMMIT, thencommit. To prevent blocking, send PRE-COMMIT and after acknowledgements, sendGLOBAL-COMMIT.



50/97

50

Network Partitioning

If data is not replicated, can allow transaction to

proceed if it does not require any data from site

outside partition in which it is initiated.

Otherwise, transaction must wait until sites itneeds access to are available.

If data is replicated, procedure is much more

complicated.



51/97

51

Identifying Updates



52/97


53/97

53

Maintaining Integrity



54/97

54

Maintaining Integrity

Successfully completed update operations by users

in different partitions can violate constraints.

Have constraint that account cannot go below 0.

In P1, withdrawn 60 from account and in P2,withdrawn 50.

At start, both partitions have 100 in balx, then on

completion one has 40 in balx and other has 50.

Importantly, neither has violated constraint.

On recovery, balx is 10, and constraint violated.



55/97

55

Network Partitioning

Processing in partitioned network involvestrade-off in availability and correctness.

Correctness easiest to provide if no processing ofreplicated data allowed during partitioning.

Availability maximized if no restrictions placedon processing of replicated data.

In general, not possible to design non-blocking

commit protocol for arbitrarily partitionednetworks.



56/97

56

X/OPEN DTP Model

Open Group is vendor-neutral consortium whosemission is to cause creation of viable, globalinformation infrastructure.

Formed by merge of X/Open and Open SoftwareFoundation.

X/Open established DTP Working Group withobjective of specifying and fostering appropriateAPIs for TP.

Group concentrated on elements of TP systemthat provided the ACID properties.



57/97

57

X/OPEN DTP Model

X/Open DTP standard that emerged specified

three interacting components:

an application,a transaction manager (TM),

a resource manager (RM).



58/97


59/97

59

X/OPEN DTP Model - Interfaces

Application may use TX interface tocommunicate with a TM.

TX provides calls that define transaction scope,and whether to commit/abort transaction.

TM communicates transactional informationwith RMs through XA interface.

Finally, application can communicate directly

with RMs through a native API, such as SQL orISAM.



60/97

60

X/OPEN DTP Model Interfaces



61/97

61

X/OPEN Interfaces in Distributed Environment



62/97

62

Distributed Query Optimization



63/97

63


Query decomposition: takes query expressed onglobal relations and performs partial

optimization using centralized QO techniques.

Output is some form of RAT based on global

relations.

Data localization: takes into account how data

has been distributed. Replace global relations at

leaves of RAT with their reconstructionalgorithms.



64/97

64


Global optimization: uses statistical informationto find a near-optimal execution plan. Output is

execution strategy based on fragments with

communication primitives added.

Local optimization: Each local DBMS performs

its own local optimization using centralized QO

techniques.



65/97

65

Data Localization

In QP, represent query as R.A.T. and, usingtransformation rules, restructure tree intoequivalent form that improves processing.

In DQP, need to consider data distribution.

Replace global relations at leaves of tree withtheir reconstruction algorithms - RA operationsthat reconstruct global relations from fragments: For horizontal fragmentation, reconstruction

algorithm is Union;

For vertical fragmentation, it is Join.



66/97

66

Data Localization

Then use reduction techniques to generatesimpler and optimized query.

Consider reduction techniques for following

types of fragmentation:Primary horizontal fragmentation.

Vertical fragmentation.

Derived fragmentation.



67/97

67

Reduction for Primary Horizontal Fragmentation

If selection predicate contradicts definition offragment, this produces empty intermediaterelation and operations can be eliminated.

For join, commute join with union.

Then examine each individual join to determinewhether there are any useless joins that can beeliminated from result.

A useless join exists if fragment predicates do notoverlap.



68/97

68

Example 23.2 Reduction for PHF

SELECT *

FROM Branch b, PropertyForRent p

WHERE b.branchNo = p.branchNo AND p.type = Flat;

P1:

branchNo=B003 type=House(PropertyForRent)

P2: branchNo=B003 type=Flat (PropertyForRent)P3: branchNo!=B003 (PropertyForRent)

B1:

branchNo=B003 (Branch)B2: branchNo!=B003 (Branch)



69/97

69




70/97

70




71/97

71




72/97

72

Reduction for Vertical Fragmentation

Reduction for vertical fragmentation involvesremoving those vertical fragments that have no

attributes in common with projection

attributes, except the key of the relation.



73/97

73

Example 23.3 Reduction for Vertical Fragmentation

SELECT fName, lName

FROM Staff;

S1: staffNo,position, sex, DOB, salary(Staff)S2: staffNo,fName, lName, branchNo (Staff)



74/97

74

Example 23.3 Reduction for Vertical Fragmentation



75/97

75

Reduction for Derived Fragmentation

Use transformation rule that allows join andunion to be commuted.

Using knowledge that fragmentation for one

relation is based on the other and, incommuting, some of the partial joins should be

redundant.



76/97

76

Example 23.4 Reduction for Derived Fragmentation

SELECT *

FROM Branch b, Client c

WHERE b.branchNo = c.branchNo AND

b.branchNo = B003;

B1 = branchNo=B003 (Branch)

B2 =

branchNo!=B003 (Branch)Ci = Client branchNo Bi i = 1, 2


E l 23 4 R d ti f D i d


77/97

77

Example 23.4 Reduction for Derived

Fragmentation



78/97

78

Global Optimization

Objective of this layer is to take the reducedquery plan for the data localization layer and

find a near-optimal execution strategy.

In distributed environment, speed of network hasto be considered when comparing strategies.

If know topology is that of WAN, could ignore all

costs other than network costs.

LAN typically much faster than WAN, but still

slower than disk access.



79/97

79

Global Optimization

Cost model could be based on total cost (time),as in centralized DBMS, or response time.

Latter uses parallelism inherent in DDBMS.



80/97

80

Global Optimization R*

R* uses a cost model based on total cost and staticquery optimization.

Like centralized System R optimizer, algorithm is

based on an exhaustive search of all join orderings,

join methods (nested loop or sort-merge join), and

various access paths for each relation.

When Join is required involving relations at

different sites, R* selects the sites to perform Joinand method of transferring data between sites.



81/97

81


For a Join of R and S with R at site 1 and S at site2, there are three candidate sites:

site 1, where R is located;

site 2, where S is located;some other site (e.g., site of relation T, which is

to be joined with join of R and S).



82/97

82


In R*, there are 2 methods for transferring data:1. Ship whole relation

2. Fetch tuples as needed.

First method incurs a larger data transfer but

fewer message then second. R* considers only the following methods:

1. Nested loop, ship whole outer relation to site of inner.

2. Sort-merge, ship whole inner relation to site of outer.

3. Nested loop, fetch tuples of inner relation as neededfor each tuple of outer relation.

4. Sort-merge, fetch tuples of inner relation as neededfor each tuple of outer relation.

5. Ship both relations to third site.



83/97

83

Global Optimization SDD-1

Based on an earlier method known as hillclimbing, a greedy algorithm that starts with aninitial feasible solution which is then iterativelyimproved.

Modified to make use of Semijoin to reducecardinality of join operands.

Like R*, SDD-1 optimizer minimizes total cost,although unlike R* it ignores local processing costs

and concentrates on communication message size. Like R*, query processing timing used is static.



84/97

84


Based on concept of beneficial Semijoins. Communication cost of Semijoin is simply cost of

transferring join attribute of first operand to site

of second operand. Benefit of Semijoin is taken as cost of

transferring irrelevant tuples of first operand,

which Semijoin avoids.



85/97

85


Phase 1 Initialization: Perform all localreductions using Selection and Projection.Execute Semijoins within same site to reducesizes of relations. Generate set of all beneficial

Semijoins across sites (Semijoin is beneficial if itscost is less than its benefit).

Phase 2 Selection of beneficial Semijoins:Iteratively select most beneficial Semijoin from

set generated and add it to execution strategy.After each iteration, update database statistics toreflect incorporation of the Semijoin and updatethe set with new beneficial Semijoins.



86/97

86


Phase 3 Assembly site selection: Select, among allsites, site to which transmission of all relations incurs

a minimum cost. Choose site containing largest

amount of data after reduction phase so that sum of

the amount of data transferred from other sites willbe minimum.

Phase 4 Postoptimization: Discard useless

Semijoins; e.g. if R resides in assembly site and R is

due to be reduced by Semijoin, but is not used toreduce other relations after Semijoin, then since R

need not be moved to another site during assembly

phase, Semijoin on R is useless and can be discarded.



87/97

87

Oracles DDBMS Functionality

Oracle does not support type of fragmentationdiscussed previously, although DBA can distributedata to achieve similar effect.

Thus, fragmentation transparency is not supportedalthough location transparency is.

Discuss: connectivity

global database names and database links

transactions

referential integrity

heterogeneous distributed databases

Distributed QO.



88/97

88

Connectivity Oracle Net Services

Oracle Net Services supports communicationbetween clients and servers.

Enables both client-server and server-servercommunication across any network, supporting

both distributed processing and distributedDBMS capability. Also responsible for translating any differences in

character sets or data representation that mayexist at operating system level.



89/97

89

Global Database Names

Unique name given to each distributed database. Formed by prefixing the databases network

domain name with the local database name. Domain name follows standard Internet

conventions, with levels separated by dotsordered from leaf to root, left to right.



90/97

90

Database Links

Used to build distributed databases. Defines a communication path from one Oracle

database to another (possibly non-Oracle)database.

Acts as a type of remote login to remote database.

CREATE PUBLIC DATABASE LINK

RENTALS.GLASGOW.NORTH.COM;

SELECT *FROM [email protected];

UPDATE [email protected]

SET salary = salary*1.05;



91/97

91

Types of Transactions

Remote SQL statements: Remote query selectsdata from one or more remote tables, all of whichreside at same remote node. Remote updatemodifies data in one or more tables, all of whichare located at same remote node.

Distributed SQL statements: Distributed queryretrieves data from two or more nodes.

Distributed update modifies data on two or morenodes.

Remote transactions: Contains one or moreremote statements, all of which reference a singleremote node.



92/97

92

Types of Transactions

Distributed transactions: Includes one or morestatements that, individually or as a group,update data on two or more distinct nodes of adistributed database. Oracle ensures integrity ofdistributed transactions using 2PC.



93/97

93

Referential Integrity

Oracle does not permit declarative referentialintegrity constraints to be defined acrossdatabases.

However, parent-child table relationships across

databases can be maintained using triggers.



94/97

94

Heterogeneous Distributed Databases

Here one of the local DBMSs is not Oracle. Oracle Heterogeneous Services and a non-Oracle

system-specific agent can hide distribution andheterogeneity.

Can be accessed through:transparent gatewaysgeneric connectivity.



95/97

95

Transparent Gateways



96/97

96

Generic Connectivity

Pearson Education Limited 1995,2005


97/97

Oracle Distributed Query Optimization

A distributed query is decomposed by the localOracle DBMS into a number of remote queries,which are sent to remote DBMS for execution.

Remote DBMSs execute queries and send results

back to local node. Local node then performs any necessarypostprocessing and returns results to user.

Only necessary data from remote tables areextracted, thereby reducing amount of data thatneeds to be transferred.

Documents

Distributed DBMSs Advanced Concepts