Chaps26.1

CSE 4701

Chapter 26, 6e - 24, 5e

Distributed Databases

Prof. Steven A. Demurjian, Sr. Computer Science & Engineering Department

The University of Connecticut191 Auditorium Road, Box U-155

Storrs, CT 06269-3155 steve@engr.uconn.edu

http://www.engr.uconn.edu/~steve(860) 486 - 4818

A portion of these slides are being used with the permission of Dr. Ling Lui, Associate Professor, College of Computing, Georgia Tech.

Remaining slides represent new material.

Chaps26.2

CSE 4701

Classical and Distributed Architectures Classic/Centralized DBMS Dominated the

Commercial Market from 1970s Forward Problems of this Approach

Difficult to Scale w.r.t. Performance Gains If DB Overloaded, replace with a Faster Computer this can Only Go So Far - Disk Bottlenecks

Distributed DBMS have Evolved to Address a Number of Issues Improved Performance Putting Data “Near” Location where it is Needed Replication of Data for Fault Tolerance Vertical and Horizontal Partitioning of DB Tuples

Chaps26.3

CSE 4701

Common Features of Centralized DBMS Data Independence

High-Level Representation via Conceptual and External Schemas

Physical Representation (Internal Schema) Hidden Program Independence

Multiple Applications can Share Data Views/External Schema Support this Capability

Reduction of Program/Data Redundancy Single, Unique, Conceptual Schema Shared Database

Almost No Data Redundancy Controlled Data Access Reduces Inconsistencies Programs Execute with Consistent Results

Chaps26.4

CSE 4701

Common Features of Centralized DBMS Promote Sharing: Automatically Provided via CC

No Longer Programmatic Issue Most DBMS Offer Locking for Key Shared Data

Oracle Allows Locks on Data Item (Attributes) For Example, Controlling Access to Shared Identifier

Coherent and Central DB Administration Semantic DB Integrity via the Automatic Enforcement

of Data Consistency via Integrity Constraints/Rules Data Resiliency

Physical Integrity of Data in the Presence of Faults and Errors

Supported by DB Recovery Data Security: Control Access for Authorized Users

Against Sensitive Data

Chaps26.5

CSE 4701

Shared Nothing Architecture In this Architecture, Each DBMS

Operates Autonomously There is No Sharing

Three Separate DBMSs on Three Different Computers

Applications/Clients Must Know About the External Schemas of all Three DBMSs for Database Retrieval Client Processing

Complicates Client Different DBMS

Platforms(Oracle, Sybase, Informix, ..)

Different Access Modes(Query, Embedded, ODBC)

Difficult for SWE to Code

Chaps26.6

CSE 4701

Difficulty in Access – Manage Multiple APIs Each Platform has a Different API

API1 , API3 , …. , APIn An App Programmer Must Utilize All three APIs

which could differ by PL – C++, C, Java, REST, etc.

Any interactions Across 3 DBs – must be programmatically handled without DB Capabilities

API1 API2APIn

Chaps26.7

CSE 4701

NW Architecture with Centralized DB High-Speed NWs/WANs Spawned Centralized DB

Accessible Worldwide Clients at Any Site can Access Repository Data May be “Far” Away - Increased Access Time In Practice, Each Remote Site Needs only Portion

of the Data in DB1 and/or DB2 Inefficient, no Replication w.r.t. Failure

Chaps26.8

CSE 4701

Fully Distributed Architecture The Five Sites (Chicago, SF, LA, NY, Atlanta) each

have a “Portion” of the Database - its Distributed Replication is Possible for Fault Tolerance Queries at one Site May Need to Access Data at

Another Site (e.g., for a Join) Increased Transaction Processing Complexity

Chaps26.9

CSE 4701

Distributed Database Concepts A transaction can be executed by multiple networked

computers in a unified manner. A distributed database (DDB) processes a Unit of

execution (a transaction) in a distributed manner. A distributed database (DDB) can be defined as

Collection of multiple logically related database distributed over a computer network

Distributed database management system as a software system that manages a distributed database while making the distribution transparent to the user.

Chaps26.10

CSE 4701

Goals of DDBMS Support User Distribution Across Multiple Sites

Remote Access by Users Regardless of Location Distribution and Replication of Database Content

Provide Location Transparency Users Manipulate their Own Data Non-Local Sites “Appear” Local to Any User

Provide Transaction Control Akin to Centralized Case Transaction Control Hides Distribution CC and Serializability - Must be Extended

Minimize Communications Cost Optimize Use of Network - a Critical Issue Distribute DB Design Supported by Partitioning

(Fragmentation) and Replication

Chaps26.11

CSE 4701

Goals of DDBMS Improve Response Time for DB Access

Use a More Sophisticated Load Control for Transaction Processing

However, Synchronization Across Sites May Introduce Additional Overhead

System Availability Site Independence in the Presence of Site Failure Subset of Database is Always Available Replication can Keep All Data Available, Even

When Multiple Sites Fail Modularity

Incremental Growth with the Addition of Sites Dedicate Sites to Specific Tasks

Chaps26.12

CSE 4701

Advantages of DDBMS There are Four Major Advantages1. Transparency

Distribution/NW Transparency User Doesn’t Know about NW Configuration (Location

Transparency) User can Find Object at any Site (Naming

Transparency) Replication Transparency (see next PPT)

User Doesn’t Know Location of Data Replicas are Transparently Accessible

Fragmentation Transparency Horizontal Fragmentation (Distribute by Row) Vertical Fragmentation (Distribute by Column)

Chaps26.13

CSE 4701

Data Distribution and Replication

Chaps26.14

CSE 4701

Other Advantages of DDBMS

2. Increased Reliability and Availability Reliability - System Always Running Availability - Data Always Present Achieved via Replication and Distribution Ability to Make Single Query for Entire DDBMS

3. Improved Performance Sites Able to Utilize Data that is Local for Majority

of Queries4. Easier Expansion

Improve Performance of Site by Upgrading Processor of Computer Adding Additional Disks Splitting a Site into Two or More Sites

Expansion over Time as Business Grows

Chaps26.15

CSE 4701

Challenges of DDBMS Tracking Data - Meta Data More Complex

Must Track Distribution (where is the Data) V & H Fragmentation (How is Data Split) Replication (Multiple Copies for Consistency)

Distributed Query Processing Optimization, Accessibility, etc., More Complex Block Analysis of Data Size Must also Now

Consider the NW Transmitting Time Distributed Transaction Processing

TP Potentially Spans Multiple Sites Submit Query to Multiple Sites Collect and Collate Results

Distributed Concurrency Control Across Nodes

Chaps26.16

CSE 4701

Challenges of DDBMS Replicated Data Management

TP Must Choose the Replica to Access Updates Must Modify All Replica Copies

Distributed Database Recovery Recovery of Individual Sites Recovery Across DDBMS

Security Local and Remote Authorization During TP, be Able to Verify Remote Privileges

Distributed Directory Management Meta-Data on Database - Local and Remote Must maintain Replicas of this - Every Site Tracks

the Meta-Data for All Sites

Chaps26.17

CSE 4701

A Complete Schema with Keys ...

Keys Allow us to Establish Links Between Relations

Chaps26.18

CSE 4701

…and Corresponding DB Tables

which Represent Tuples/Instances of Each Relation

1455

ASCnullWBnullnull

Chaps26.19

CSE 4701

…with Remaining DB Tables

Chaps26.20

CSE 4701

What is Fragmentation? Fragmentation Divides a DB Across Multiple Sites Two Types of Fragmentation

Horizontal Fragmentation Given a Relation R with n Total Tuples, Spread Entire

Tuples Across Multiple Sites Each Site has a Subset of the n Tuples Essentially Fragmentation is a Selection

Vertical Fragmentation Given a Relation R with m Attributes and n Total

Tuples, Spread the Columns Across Multiple Sites Essentially Fragmentation is a Projection Not Generally Utilized in Practice

In Both Cases, Sites can Overlap for Replication

Chaps26.21

CSE 4701

Horizontal Fragmentation A horizontal subset of a relation which contain those

of tuples which satisfy selection conditions. Consider Employee relation with condition DNO = 5 All tuples satisfying this create a subset which will be

a horizontal fragment of Employee relation. A selection condition may be composed of several

conditions connected by AND or OR. Derived horizontal fragmentation:

Partitioning of a primary relation to other secondary relations which are related with Foreign keys.

Chaps26.22

CSE 4701

Horizontal Fragmentation Site 2 Tracks All Information Related to Dept. 5

Chaps26.23

CSE 4701

Horizontal Fragmentation Site 3 Tracks All Information Related to Dept. 4 Note that an Employee Could be Listed in Both Cases,

if s/he Works on a Project for Both Departments

Chaps26.24

CSE 4701

Refined Horizontal Fragmentation Further Fragment from Site

2 based on Dept. that Employee Works in

Notice that G1 + G2 + G3 is the Same as WORKS_ON5

there is no Overlap

Chaps26.25

CSE 4701

Refined Horizontal Fragmentation Further Fragment from Site

3 based on Dept. that Employee Works in

Notice that G4 + G5 + G6 is the Same as WORKS_ON4

Note Some Fragments can be Empty

Chaps26.26

CSE 4701

Vertical Fragmentation Subset of a relation created via a subset of columns.

A vertical fragment of a relation will contain values of selected columns.

There is no selection condition used in vertical fragmentation.

A strict vertical slice/partition Consider the Employee relation.

A vertical fragment of can be created by keeping the values of Name, Bdate, Sex, and Address.

Since no condition for creating a vertical fragment Each fragment must include the primary key

attribute of the parent relation Employee. All vertical fragments of a relation are connected.

Chaps26.27

CSE 4701

Vertical Fragmentation Example Partition the Employee Table as Below Notice Each Vertical Fragment Needs Key Column

EmpDemo EmpSupvrDept

Chaps26.28

CSE 4701

Homogeneous DDBMS Homogeneous

Identical Software (w.r.t. Database) One DB Product (e.g., Oracle, DB2, Sybase) is

Distributed and Available at All Sites Uniformity w.r.t. Administration, Maintenance,

Client Access, Users, Security, etc. Interaction by Programmatic Clients is Consistent

(e.g., JDBC or ODBC or REST API …)

Chaps26.29

CSE 4701

Non-Federated Heterogeneous DDBMS Non-Federated Heterogeneous

Different Software (w.r.t. Database) Multiple DB Products (e.g., Oracle at One Site,

MySQL at another, Sybase, Informix, etc.) Replicated Administration (e.g., Users Needs

Accounts on Multiple Systems) Varied Programmatic Access - SWEs Must Know

All Platforms/Client Software Complicated Very Close to Shared Nothing Architecture

Chaps26.30

CSE 4701

Federated DDBMS Federated

Multiple DBMS Platforms Overlaid with a Global Schema View

Single External Schema Combines Schemas from all Sites

Multiple Data Models Relational in one

Component DBS Object-Oriented in

another DBS Hierarchical in a

3rd DBS

Chaps26.31

CSE 4701

Federated DBMS Issues Differences in Data Models

Reconcile Relational vs. Object-Oriented Models Each Different Model has Different Capabilities These Differences Must be Addressed in Order to

Present a Federated Schema Differences in Constraints

Referential Integrity Constraints in Different DBSs Different Constraints on “Similar” Data Federated Schema Must Deal with these Conflicts

Differences in Query Languages SQL-89, SQL-92, SQL2, SQL3 Specific Types in Different DBMS (Oracle Blobs )

Differences in Key Processing & Timestamping

Chaps26.32

CSE 4701

Heterogeneous Distributed Database Systems Federated: Each site may run different database system but the

data access is managed through a single conceptual schema. The degree of local autonomy is minimum. Each site must adhere to a centralized access policy There may be a global schema.

Multi-database: There is no one conceptual global schema For data access a schema is constructed dynamically as

needed by the application software.

Communications network

Site 5Site 1

Site 2Site 3

NetworkDBMS

Relational

Site 4

ObjectOriented

LinuxLinux

Unix

Hierarchical

ObjectOriented

RelationalUnix

Window

Chaps26.33

CSE 4701

Query Processing in Distributed Databases

Issues Cost of transferring data (files and results) over the network.

This cost is usually high so some optimization is necessary. Example relations: Employee at site 1 and Department at Site 2

– Employee at site 1. 10,000 rows. Row size = 100 bytes. Table size = 106 bytes.

– Department at Site 2. 100 rows. Row size = 35 bytes. Table size = 3,500 bytes.

Q: For each employee, retrieve employee name and department name Where the employee works.

Q: Fname,Lname,Dname (Employee Dno = Dnumber Department)

Fname Minit Lname SSN Bdate Address Sex Salary Superssn Dno

Dname Dnumber Mgrssn Mgrstartdate

Chaps26.34

CSE 4701

Query Processing in Distributed Databases Result

The result of this query will have 10,000 tuples, assuming that every employee is related to a department.

Suppose each result tuple is 40 bytes long. The query is submitted at site 3 and the result is

sent to this site. Problem: Employee and Department relations are

not present at site 3.

Chaps26.35

CSE 4701

Query Processing in Distributed Databases Strategies:

1. Transfer Employee and Department to site 3. Total transfer bytes = 1,000,000 + 3500 = 1,003,500

bytes.2. Transfer Employee to site 2, execute join at site 2 and send

the result to site 3. Query result size = 40 * 10,000 = 400,000 bytes. Total

transfer size = 400,000 + 1,000,000 = 1,400,000 bytes.3. Transfer Department relation to site 1, execute the join at site

1, and send the result to site 3. Total bytes transferred = 400,000 + 3500 = 403,500 bytes.

Optimization criteria: minimizing data transfer. Preferred approach: strategy 3.

Chaps26.36

CSE 4701

Query Processing in Distributed Databases Consider the query

Q’: For each department, retrieve the department name and the name of the department manager

Relational Algebra expression: Fname,Lname,Dname (Employee Mgrssn = SSN Department)

Chaps26.37

CSE 4701

Query Processing in Distributed Databases Result of query has 100 tuples, assuming that every

department has a manager, the execution strategies are:1. Transfer Employee and Department to the result site and

perform the join at site 3. Total bytes transferred = 1,000,000 + 3500 = 1,003,500

bytes.2. Transfer Employee to site 2, execute join at site 2 and send

the result to site 3. Query result size = 40 * 100 = 4000 bytes. Total transfer size = 4000 + 1,000,000 = 1,004,000 bytes.

3. Transfer Department relation to site 1, execute join at site 1 and send the result to site 3. Total transfer size = 4000 + 3500 = 7500 bytes.

Preferred strategy: Choose strategy 3.

Chaps26.38

CSE 4701

Query Processing in Distributed Databases Now suppose the result site is 2. Possible strategies :

1. Transfer Employee relation to site 2, execute the query and present the result to the user at site 2. Total transfer size = 1,000,000 bytes for both queries Q

and Q’.

2. Transfer Department relation to site 1, execute join at site 1 and send the result back to site 2. Total transfer size for Q = 400,000 + 3500 = 403,500

bytes and for Q’ = 4000 + 3500 = 7500 bytes.

Chaps26.39

CSE 4701

DDBS Concurrency Control and Recovery Distributed Databases encounter a number of

concurrency control and recovery problems which are not present in centralized databases, including: Dealing with multiple copies of data items

How are they All Updated if Needed? Failure of individual sites

How are Queries Restarted or Rerouted? Communication link failure

Network Failure Distributed commit

How to Know All Updates Done at all Sites? Distributed deadlock

How to Detect and Recover?

Chaps26.40

CSE 4701

The Next Big Challenge Interoperability

Heterogeneous Distributed Databases Heterogeneous Distributed Systems Autonomous Applications

Scalability Rapid and Continuous Growth

Amount of Data Variety of Data Types Dealing with personally identifiable information (PII)

and personal health information (PHI) Emergence of Fitness and Health Monitoring Apps Google Fit and Apple HealthKit New Apple ResearchKit for Medical Research

Chaps26.41

CSE 4701

Interoperability: A Classic View

FDB Global Schema

FederatedIntegration

Local Schema

Local Schema

Local Schema

FDB Global Schema 4

FederatedIntegration

FDB 1Local

SchemaFDB3

Federation Federation

Simple Federation Multiple Nested Federation

Chaps26.42

CSE 4701

LegacyApplication

Network

Java Client

Java Application Code

WRAPPER

Mapping Classes

JAVA LAYER

NATIVE LAYER

Native Functions (C++)RPC Client Stubs (C)

Interactions Between Java Clientand Legacy Appl. via C and RPC

C is the Medium of Info. Exchange

Java Client with C++/C Wrapper

Java Client with Wrapper to Legacy Application

Chaps26.43

CSE 4701

Network

Java Application Code

JAVA NETWORK WRAPPER

Mapping Classes

NATIVE LAYER

JAVA LAYER

Native Functions that Map to COTS Appl

Java Client Java Client

Java Application Code

JAVA NETWORK WRAPPER

Mapping Classes

NATIVE LAYER

JAVA LAYER

Native Functions that Map to Legacy Appl

COTS Application Legacy Application

Java is Medium of Info. Exchange - C/C++ Appls with Java Wrappers

COTS and Legacy Appls. to Java Clients

Chaps26.44

CSE 4701

Java Client

LegacyApplication

Relational Database

System(RDS)

Transformed Legacy Data

Updated Data

Extract and Generate Data

Transform andStore Data

Java Client to Legacy App via RDBS

Chaps26.45

CSE 4701

Information Broker

•Mediator-Based Systems•Agent-Based Systems

Database Interoperability in the Internet Technology

Web/HTTP, JDBC/ODBC, CORBA (ORBs + IIOP), XML, SOAP, REST API, WSDL

Architecture

Chaps26.46

CSE 4701

Driver Driver

Java Application

JDBC API

Driver Manager

Oracle SybaseAccess

Driver

JDBC

Driver Driver

JDBC API Provides DB Access Protocols for Open, Query, Close, etc.

Different Drivers for Different DB Platforms

Chaps26.47

CSE 4701

Connecting a DB to the Web

Web Server are Stateless

DB Interactions Tend to be Stateful

Invoking a CGI Script on Each DB Interaction is Very Expensive, Mainly Due to the Cost of DB Open

DBMS

Web Server

Browser

Internet

CGI Script Invocationor JDBC Invocation

Chaps26.48

CSE 4701

Connecting More Efficiently

To Avoid Cost of Opening Database, One can Use Helper Processes that Always Keep Database Open and Outlive Web Connection

Newly Invoked CGI Scripts Connect to a Preexisting Helper Process

System is Still Stateless

DBMS

Web Server

Browser

Internet

CGI Scriptor JDBC Invocation

Helper Processes

Chaps26.49

CSE 4701

DB-Internet Architecture

WWW Client(Netscape)

WWW Client(HotJava)

WWW client(Info. Explore)

Internet

HTTP Server

DBWeb Gateway

DBWeb Gateway

DBWeb Gateway

DBWeb Gateway

DBWeb Dispatcher

Chaps26.50

CSE 4701

EJB Architecture

Chaps26.51

CSE 4701

The focus is to make information available to users, in the right form, at the right time, in the appropriate place.

Technology Push Computer/Communication Technology (Almost Free)

Plenty of Affordable CPU, Memory, Disk, Network Bandwidth

Next Generation Internet: Gigabit Now Wireless: Ubiquitous, High Bandwidth

Information Growth Massively Parallel Generation of Information on

the Internet and from New Generation of Sensors Disk Capacity on the Order of Peta-bytes

Small, Handy Devices to Access Information

Chaps26.52

CSE 4701 Ubiquitous/Pervasive

Many computers and information appliances everywhere,

networked together

Research Challenges Inherent Complexity:

Coping with Latency (Sometimes Unpredictable)

Failure Detection and Recovery (Partial Failure)

Concurrency, Load Balancing, Availability, Scale

Service Partitioning Ordering of Distributed Events

“Accidental” Complexity: Heterogeneity: Beyond the Local

Case: Platform, Protocol, Plus All Local Heterogeneity in Spades.

Autonomy: Change and Evolve Autonomously

Tool Deficiencies: Language Support (Sockets,rpc), Debugging, Etc.

Chaps26.53

CSE 4701

Problem: too many sources,too much information

Internet:Information Jungle

Clean, Reliable,Timely Information,Anywhere

DigitalEarth

Sensors

PersonalizedFiltering &Info. Delivery

Infopipes

Resou

rce A

dapta

tion Property Mgmt

Information QualityContinual Queries

Mic

rofe

edba

ck

specializationInfosphere

Chaps26.54

CSE 4701

ThinClient

WebServer

MainframeDatabaseServer

Current State-of-Art – Has Mobile Changed This?

Chaps26.55

CSE 4701 Infotaps &

Fat Clients

Varietyof Servers

Sensors

DatabaseServer

Many sources

Infosphere Scenario – Where Does Mobile Fit?

Chaps26.56

CSE 4701

Heterogeneity and Autonomy Heterogeneity:

How Much can we Really Integrate? Syntactic Integration

Different Formats and Models XML/JSON/RDF/OWL/SQL Query Languages

Semantic Interoperability Basic Research on Ontology, Etc. DoD Maps (Grid, True, and Magnetic North)

Autonomy No Central DBA on the Net Independent Evolution of Schema and Content Interoperation is Voluntary Interface Technology DCOM: Microsoft Standard

CORBA, Etc...

Chaps26.57

CSE 4701

Security and Data Quality Security

System Security in the Broad Sense Attacks: Penetrations, Denial of Service System (and Information) Survivability

Security Fault Tolerance Replication for Performance, Availability, and

Survivability Data Quality

Web Data Quality Problems Local Updates with Global Effects Unchecked Redundancy (Mutual Copying) Registration of Unchecked Information Spam on the Rise

Chaps26.58

CSE 4701

Legacy Data Challenge Legacy Applications and Data

Definition: Important and Difficult to Replace Typically, Mainframe Mission Critical Code Most are OLTP and Database Applications

Evolution of Legacy Databases Client-server Architectures Wrappers Expensive and Gradual in Any Case

Chaps26.59

CSE 4701

Potential Value Added/Jumping on Bandwagon

Sophisticated Query Capability Combining SQL with Keyword Queries

Consistent Updates Atomic Transactions and Beyond

But Everything has to be in a Database! Only If we Stick with Classic DB Assumptions

Relaxing DB Assumptions Interoperable Query Processing Extended Transaction Updates

Commodities DB Software A Little Help is Still Good If it is Cheap Internet Facilitates Software Distribution Databases as Middleware

Chaps26.60

CSE 4701

Concluding Remarks Goals of Distributed DBS

Support User Distribution Across Multiple Sites Provide Location Transparency Provide Transaction Control Akin to Centralized

Case Minimize Communications Cost

Advantages of Distributed DBS Transparency Increased Reliability and Availability Improved Performance Easier Expansion