SQL and SQAPL. Data Models A Database models some portion of the real world. Data Model is link between user’s view of the world and bits stored in computer

SQL and SQAPL

Data Models

• A Database models some portion of the real world.

• Data Model is link between user’s view of the world and bits stored in computer.

• We will concentrate on the Relational Model

Data Models• A data model is a collection of concepts

for describing data.• A database schema is a description of a

particular collection of data, using a given data model.

• The relational model of data is the most widely used model today.– Main concept: relation, basically a

table with rows and columns.– Every relation has a schema, which

describes the columns, or fields.

Levels of Abstraction• Views describe how

users see the data.

• Conceptual schema defines logical structure

• Physical schema describes the files and indexes used.

• (sometimes called the ANSI/SPARC model)

Physical Schema

Conceptual Schema

View 1 View 2 View 3

DB

Users

Data Independence• A Simple Idea:

Applications should be insulated from how data is structured and stored.

• Logical data independence: Protection from changes in logical structure of data.

• Physical data independence: Protection from changes in physical structure of data.

Physical Schema

Conceptual Schema

View 1 View 2 View 3

DB

SQL

SQL consists of the following parts:• Data Definition Language (DDL)• Interactive Data Manipulation Language (Interactive DML)• Embedded Data Manipulation Language (Embedded DML)• Views• Integrity• Transaction Control• Authorization• Catalog and Dictionary Facilities

FLT-INSTANCE

FLT-WEEKDAY

AIRPLANE

CUSTOMER

flt# date plane#

RESERVATION

flt# airline dtime from-airportcode atime to-airportcode miles price

flt# weekday

plane# plane-type total-#seats

cust# first middle last phone# street city state zip

flt# date cust# seat# check-in-status ticket#

AIRPORT

airportcode name city state

#avail-seats

FLT-SCHEDULE

DDL - Overview

• primitive types• domains• schema• tables

DDL - Primitive Types• numeric

– INTEGER (or INT), SMALLINT are subsets of the integers (machine dependent)

– REAL, DOUBLE PRECISION are floating-point and double-precision floating-point (machine dependent)

– FLOAT(N) is floating-point with at least N digits– DECIMAL(P,D) (or DEC(P,D), or NUMERIC(P,D)), with P digits

of which D are to the right of the decimal point

DDL - Primitive Types (cont.)• character-string

– CHAR(N) (or CHARACTER(N)) is a fixed-length character string

– VARCHAR(N) (or CHAR VARYING(N), or CHARACTER

VARYING(N)) is a variable-length character string with at most N characters

• bit-strings– BIT(N) is a fixed-length bit string– VARBIT(N) (or BIT VARYING(N)) is a bit string

with at most N bits

DDL - Primitive Types (cont.)• time

– DATE is a date: YYYY-MM-DD

– TIME, a time of day: HH-MM-SS

– TIME(I), a time of day with I decimal fractions of a second: HH-MM-SS-F....F

– TIME WITH TIME ZONE, a time with a time zone added: HH-MM-SS-HH-MM

DDL - Primitive Types (cont.)

– TIME-STAMP, date, time, fractions of a second and an optional WITH TIME ZONE

qualifier: YYYY-MM-DD-HH-MM-SS-F...F{-HH-MM}

– INTERVAL, relative value used to increment or decrement DATE, TIME, or

TIMESTAMP: YEAR/MONTH or DAY/TIME

DDL - Domains• a domain can be defined as follows:

CREATE DOMAIN AIRPORT-CODE CHAR(3);CREATE DOMAIN FLIGHTNUMBER CHAR(5);

• using domain definitions makes it easier to see which columns are related

• changing a domain definition one place changes it consistently everywhere it is used

• default values can be defined for domains• constraints can be defined for domains

(later)

DDL - Domains (cont.)• all domains contain the value, NULL.

• to define a different default value:CREATE DOMAIN AIRPORT-CODE CHAR(3) DEFAULT ‘<literal>’;CREATE DOMAIN AIRPORT-CODE CHAR(3) DEFAULT ‘niladic function’;

• literal, such as ‘???’, ‘NO-VALUE’,...• niladic function, such as USER, CURRENT-USER,

SESSION-USER, SYSTEM-USER, CURRENT-DATE, CURRENT-TIME, CURRENT-TIMESTAMP

• defaults defined in a column takes precedence over the above

DDL - Domains (cont.)• a domain is dropped as follows:

DROP DOMAIN AIRPORT-CODE RESTRICT;

DROP DOMAIN AIRPORT-CODE CASCADE;

• restrict: drop operation fails if the domain is used in column definitions

• cascade: drop operation causes columns to be defined directly on the underlying data type

DDL - Schema

• create a schema:CREATE SCHEMA AIRLINE AUTHORIZATION LEO;

• the schema AIRLINE has now been created and is owner by the user “LEO”

• tables can now be created and added to the schema

DDL - Schema

• to drop a schema:DROP SCHEMA AIRLINE RESTRICT;

DROP SCHEMA AIRLINE CASCADE;

• restrict: drop operation fails if schema is not empty

• cascade: drop operation removes everything in the schema

DDL - Tables• to create a table in the AIRLINE schema:CREATE TABLE AIRLINE.FLT-SCHEDULE

(FLT# FLIGHTNUMBER NOT NULL,

AIRLINE VARCHAR(25),

FROM-AIRPORTCODE AIRPORT-CODE,

DTIME TIME,

TO-AIRPORTCODE AIRPORT-CODE,

ATIME TIME,

PRIMARY KEY (FLT#),

FOREIGN KEY (FROM-AIRPORTCODE) REFERENCES AIRPORT(AIRPORTCODE),

FOREIGN KEY (TO-AIRPORTCODE)

REFERENCES AIRPORT(AIRPORTCODE));

DDL - Tables (cont.)CREATE TABLE AIRLINE.FLT-WEEKDAY


WEEKDAY CHAR(2),

UNIQUE(FLT#, WEEKDAY),

FOREIGN KEY (FLT#)

REFERENCES FLTT-SCHEDULE(FLT#));

CREATE TABLE AIRLINE.FLT-INSTANCE


DATE DATE NOT NULL,

#AVAIL-SEATS SMALLINT,

PRIMARY KEY(FLT#, DATE),

FOREIGN KEY FLT#

REFERENCES FLT-SCHEDULE(FLT#));

DDL - Tables (cont.)CREATE TABLE AIRLINE.RESERVATION


DATE DATE NOT NULL,

CUST# INTEGER NOT NULL,

SEAT# CHAR(4),

CHECK-IN-STATUS CHAR,UNIQUE(FLT#, DATE, CUST#),FOREIGN KEY (FLT#)

REFERENCES FLT-INSTANCE(FLT#), FOREIGN KEY (DATE) REFERENCES FLT-INSTANCE(DATE),

FOREIGN KEY (CUST#) REFERENCES CUSTOMER(CUST#));

DDL - Tables (cont.)

• to drop a table:DROP TABLE RESERVATION RESTRICT;

DROP TABLE RESERVATION CASCADE;

• restrict: drop operation fails if the table is referenced by view/constraint definitions

• cascade: drop operation removes referencing view/constraint definitions


• to add a column to a table: ALTER TABLE AIRLINE.FLT-SCHEDULE

ADD PRICE DECIMAL(7,2);

• if no DEFAULT is specified, the new column will have NULL values for all tuples already in the database


• to drop a column from a table ALTER TABLE AIRLINE.FLT-SCHEDULE

DROP PRICE RESTRICT (or CASCADE);

• restrict: drop operation fails if the column is referenced

• cascade: drop operation removes referencing view/constraint definitions

Case

• A dataset consisting of 5 tables are available as .csv files or as an excel file

• Let us get this data into APL• Getdata• Next let us create the database using

SQAPL• First you must create a database in Access• SQAPL

Interactive DML - Overview• select-from-where

• select clause• where clause• from clause• tuple variables• string matching• ordering of rows• set operations• built-in functions• nested subqueries• joins• recursive queries• insert, delete, update

Interactive DML - select-from-where

• the SELECT clause specifies the columns of the result

• the FROM clause specifies the tables to be scanned in the query

• the WHERE clause specifies the condition on the columns of the tables in the FROM clause

SELECT A1, A2, ... An

FROM R1 , R2 , ... Rm

WHERE P

Interactive DML - select clause• “Find the airlines in FLT-SCHEDULE” SELECT AIRLINE

FROM FLT-SCHEDULE; SELECT ALL AIRLINE FROM FLT-SCHEDULE;

• “Find the airlines in FLT-SCHEDULE with duplicates removed”

SELECT DISTINCT AIRLINE

FROM FLT-SCHEDULE;

Interactive DML - select clause• “Find all columns in FLT-SCHEDULE” SELECT * FROM FLT-SCHEDULE;

• “Find FLT# and price raised by 10%” SELECT FLT#, PRICE*1.1 FROM FLT-SCHEDULE;

Interactive DML - where clause

• “Find FLT# and price in FLT-SCHEDULE for flights out of Atlanta”

SELECT FLT#, PRICE

FROM FLT-SCHEDULE

WHERE FROM-AIRPORTCODE=“ATL”;

Interactive DML - from clause• “Find FLT#, WEEKDAY, and FROM-AIRPORTCODE in

FLT-WEEKDAY and FLT-SCHEDULE” SELECT FLT-SCHEDULE.FLT#,

WEEKDAY, FROM-AIRPORTCODE

FROM FLT-WEEKDAY, FLT-SCHEDULE

WHERE FLT-WEEKDAY.FLT# = FLT-SCHEDULE.FLT#;

• dot-notation disambiguates FLT# in FLT-WEEKDAY and FLT-SCHEDULE

Interactive DML - tuple variables

• alias definition: SELECT S.FLT#, WEEKDAY, T.FROM-AIRPORTCODE

FROM FLT-WEEKDAY S, FLT-SCHEDULE T

WHERE S.FLT# = T.FLT#;

• S and T are tuple variables

Interactive DML - tuple variables

• SQL’s heritage as a tuple calculus language shows• tuple variables are useful when one relation is

used “twice” in a query:

SELECT S.FLT#

FROM FLT-SCHEDULE S, FLT-SCHEDULE T

WHERE S.PRICE > T.PRICE

AND T.FLT# = “DL212”;

Interactive DML - string matching• wildcard searches use:

%: matches any substring_: matches any character

SELECT S.FLT#, WEEKDAY FROM FLT-WEEKDAY S, FLT-SCHEDULE T WHERE S.FLT# = T.FLT# AND T.AIRLINE LIKE “%an%”;

• “%an%” matches American, Airtran, Scandinavian, Lufthansa, PanAm...

• “A%” matches American, Airtran, ...• “ %” matches any string with at least three

characters

Interactive DML - ordering of rows• the order by clause orders the rows in a query

result in ascending (asc) or descending (desc) order• “Find FLT#, airline, and price from FLT-SCHEDULE for

flights out of Atlanta ordered by ascending airline and descending price:”

SELECT FLT#, AIRLINE, PRICE

FROM FLT-SCHEDULE

WHERE FROM-AIRPORTCODE=“ATL”

ORDER BY AIRLINE ASC, PRICE DESC;

Interactive DML - set operations

• “Find FLT# for flights on Tuesdays in FLT-WEEKDAY and FLT# with more than 100 seats in FLT-INSTANCE ”

SELECT FLT# FROM FLT-WEEKDAY WHERE WEEKDAY = “TU” UNION SELECT FLT# FROM FLT-INSTANCE WHERE #AVAIL-SEATS > 100;

• UNION ALL preserves duplicates

TSS T

S union T

Interactive DML - set operation

• “Find FLT# for flights on Tuesdays in FLT-WEEKDAY with more than 100 seats in FLT-INSTANCE”

SELECT FLT# FROM FLT-WEEKDAY WHERE WEEKDAY = “TU” INTERSECT SELECT FLT# FROM FLT-INSTANCE WHERE #AVAIL-SEATS > 100;

• INTERSECT ALL preserves duplicates

TSS T

S intersect T

Interactive DML - set operation

• “Find FLT# for flights on Tuesdays in FLT-WEEKDAY except FLT# with more than 100 seats in FLT-INSTANCE”

SELECT FLT# FROM FLT-WEEKDAY WHERE WEEKDAY = “TU” EXCEPT SELECT FLT# FROM FLT-INSTANCE WHERE #AVAIL-SEATS > 100;

• EXCEPT ALL preserves duplicates

TSS \ T

S minus T

Interactive DML - built-in functions• count (COUNT), sum (SUM), average (AVG), minimum (MIN),

maximum (MAX)• “Count flights scheduled for Tuesdays from FLT-WEEKDAY” SELECT COUNT( *) FROM FLT-WEEKDAY WHERE WEEKDAY = “TU”;

• “Find the average ticket price by airline from FLT-

SCHEDULE” SELECT AIRLINE, AVG(PRICE) FROM FLT-SCHEDULE GROUP BY AIRLINE;

Interactive DML - built-in functions• “Find the average ticket price by airline for scheduled

flights out of Atlanta for airlines with more than 5 scheduled flights out of Atlanta from FLT-SCHEDULE”

SELECT AIRLINE, AVG(PRICE) FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE = “ATL” GROUP BY AIRLINE HAVING COUNT (FLT#) >= 5;• “Find the highest priced flight(s) out of Atlanta from

FLT-SCHEDULE” SELECT FLT#, MAX(PRICE) FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE = “ATL”;

Interactive DML - nested subqueries

• Set membership: IN, NOT IN• “Find airlines from FLT-SCHEDULE where FLT#

is in the set of FLT#’s for flights on Tuesdays from FLT-WEEKDAY”

SELECT DISTINCT AIRLINE FROM FLT-SCHEDULE WHERE FLT# IN (SELECT FLT# FROM FLT-WEEKDAY WHERE WEEKDAY = “TU”);

Interactive DML - nested subqueries• “Find FLT#’s for flights on Tuesdays or

Thursdays from FLT-WEEKDAY” SELECT DISTINCT FLT# FROM FLT-WEEKDAY WHERE WEEKDAY IN (“TU”, “TH”);

Interactive DML - nested subqueries• “Find FLT# for flights from Atlanta to Chicago with a

price that is lower than all flights from Birmingham to Chicago”

SELECT FLT# FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE=“ATL” AND TO-AIRPORTCODE=“CHI” AND PRICE < ALL (SELECT PRICE FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE=“BIR” AND TO-AIRPORTCODE=“CHI”);

Interactive DML - joins• cross join: Cartesian product• [inner] join: only keeps rows that satisfy the join

condition• left outer join: keeps all rows from left table; fills in

nulls as needed• right outer join: keeps all rows from right table; fills in

nulls as needed• full outer join: keeps all rows from both tables; fills in

nulls as needed• natural or on-condition must be specified for all inner

and outer joins• natural: equi-join on columns with same name; one

column preserved

Interactive DML - joins• “Find all two-leg, one-day trips out of Atlanta;

show also a leg-one even if there is no connecting leg-two the same day”

SELECT X.FLT# LEG-ONE, Y.FLT# LEG-TWOFROM ((FLT-SCHEDULE NATURAL JOIN FLT-INSTANCE) X LEFT OUTER JOIN (FLT-SCHEDULE NATURAL JOIN FLT-INSTANCE) Y ON (X.TO-AIRPORTCODE=Y.FROM-AIRPORTCODE AND X.DATE=Y.DATE AND X.ATIME<Y.DTIME))WHERE X.FROM-AIRPORTCODE=“ATL”;

Interactive DML- recursive queries

• not in SQL2; maybe in SQL3...(?)• “Find all reachable airports for multi-leg trips

out of Atlanta”WITH PAIRS AS SELECT FROM-AIRPORTCODE D, TO-AIRPORTCODE

A FROM FLT-SCHEDULE, RECURSIVE REACHES(D, A) AS /*initially empty*/ PAIRS UNION (SELECT PAIRS.D, REACHES.A FROM PAIRS, REACHES WHERE PAIRS.A=REACHES.D)SELECT A FROM REACHES WHERE D=“ATL”;

Interactive DML - insert, delete, updateINSERT INTO FLT-SCHEDULE

VALUES (“DL212”, “DELTA”, 11-15-00, “ATL”, 13-05-00, ”CHI”, 650, 00351.00);

INSERT INTO FLT-SCHEDULE(FLT#,AIRLINE)

VALUES (“DL212”, “DELTA”); /*default nulls added*/

• “Insert into FLT-INSTANCE all flights scheduled for Thursday, 9/10/98”INSERT INTO FLT-INSTANCE(FLT#, DATE)

(SELECT S.FLT#, 1998-09-10

FROM FLT-SCHEDULE S, FLT-WEEKDAY D

WHERE S.FLT#=D.FLT#

AND D.WEEKDAY=“TH”);

Interactive DML - insert, delete, update

“Cancel all flight instances for Delta on 9/10/98”

DELETE FROM FLT-INSTANCE

WHERE DATE=1998-09-10

AND FLT# IN

(SELECT FLT#

FROM FLT-SCHEDULE

WHERE AIRLINE=“DELTA”);

Interactive DML- insert, delete, update“Update all reservations for customers

on DL212 on 9/10/98 to reservations on AA121 on 9/10/98”

UPDATE RESERVATION

SET FLT#=“AA121”

WHERE DATE=1998-09-10

AND FLT#=“DL212”;

Embedded DML - Overview• host languages

• precompilation• impedance mismatch• database access• cursor types• fetch orientation• exception handling

Embedded DML - host languages

• SQL doesn’t do iteration, recursion, report printing, user interaction, and SQL doesn’t do Windows

• SQL may be embedded in host languages, like COBOL, FORTRAN, MUMPS, PL/I, PASCAL, ADA, C, C++, JAVA

• Or used from languages like APL

Embedded DML- impedance mismatch

• SQL is a powerful, set-oriented, declarative language

• SQL queries return sets of rows• host languages cannot handle large sets of

structured data• cursors resolve the mismatch:• Demo

Views - definition, use, update

• a view is a virtual table • how a view is defined: CREATE VIEW ATL-FLT AS SELECT FLT#, AIRLINE, PRICE FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE = “ATL”;

• how a query on a view is written: SELECT * FROM ATL-FLT WHERE PRICE <= 00200.00;


• how a query on a view is computed: SELECT FLT#, AIRLINE, PRICE FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE=“ATL” AND PRICE<00200.00;

• how a view definition is dropped: DROP VIEW ATL-FLT [RESTRICT|CASCADE];

Views - definition, use, update• views inherit column names of the base

tables they are defined from• columns may be explicitly named in the

view definition• column names must be named if inheriting

them causes ambiguity• views may have computed columns, e.g.

from applying built-in-functions; these must be named in the view definition

Views - definition, use, updatethese views are not updatable

CREATE VIEW ATL-PRICES AS SELECT AIRLINE, PRICE FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE=“ATL”;

CREATE VIEW AVG-ATL-PRICES AS SELECT AIRLINE, AVG(PRICE) FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE=“ATL” GROUP BY AIRLINE;

this view is theoretically updatable, but cannot be updated in SQL CREATE VIEW FLT-SCHED-AND-DAY AS SELECT S.*, D.WEEKDAY FROM FLT-SCHEDULE S, FLT-WEEKDAY D WHERE D.FLT# = S.FLT#;

not uniquecom

puted

attribute

join of two

tables


a view is updatable if and only if:• it does not contain any of the keywords JOIN, UNION,

INTERSECT, EXCEPT • it does not contain the keyword DISTINCT• every column in the view corresponds to a uniquely

identifiable base table column• the FROM clause references exactly one table which must

be a base table or an updatable view• the table referenced in the FROM clause cannot be

referenced in the FROM clause of a nested WHERE clause• it does not have a GROUP BY clause• it does not have a HAVING clauseupdatable means insert, delete, update all ok

Views - definition, use, update CREATE VIEW LOW-ATL-FARES /*updatable view*/ AS SELECT * FROM FLT-SCHEDULE WHERE FROM-AIRPORTCODE=“ATL” AND PRICE<00200.00;

UPDATE LOW-ATL-FARES /*moves row */ SET PRICE = 00250.00 /* outside the view*/ WHERE TO-AIRPORTCODE = “BOS”;

INSERT INTO LOW-ATL-FARES /*creates row */ VALUES (“DL222”, ”DELTA”, /*outside the view*/ ”BIR”, 11-15-00, ”CHI”, 13-05-00, 00180.00);

Integrity - constraints

• constraint: a conditional expression required not to evaluate to false

• a constraint cannot be created if it is already violated

• a constraint is enforced from the point of creation forward

• a constraint has a unique name• if a constraint is violated its name is made

available to the user• constraints cannot reference parameters or host

variables; they are application independent• data type checking is a primitive form of constraint

Integrity - domain constraints

• associated with a domain; applies to all columns defined on the domain

CREATE DOMAIN WEEKDAY CHAR(2) CONSTRAINT IC-WEEKDAY CHECK (VALUE IN ( “MO”, “TU”, “WE”, “TH”, “FR”, “SA”, “SU”));

CREATE DOMAIN PRICE DECIMAL(7,2) CONSTRAINT IC-PRICE CHECK (VALUE > 00000.00 );

CREATE DOMAIN FLT# CHAR(5) CONSTRAINT IC-FLT# CHECK (VALUE NOT NULL);

Integrity - base table, column constraints

• associated with a specific base tableCREATE TABLE AIRLINE.FLT-SCHEDULE

(FLT# FLIGHTNUMBER NOT NULL,AIRLINE VARCHAR(25),FROM-AIRPORTCODE AIRPORT-CODE,DTIME TIME,TO-AIRPORTCODE AIRPORT-CODE,ATIME TIME,CONSTRAINT FLTPK PRIMARY KEY (FLT#),CONSTRAINT FROM-AIRPORTCODE-FKFOREIGN KEY (FROM-AIRPORTCODE)

REFERENCES AIRPORT(AIRPORTCODE) ON DELETE SET NULL ON UPDATE CASCADE,FOREIGN KEY (FROM-AIRPORTCODE)

REFERENCES AIRPORT(AIRPORTCODE) ON DELETE SET NULL ON UPDATE CASCADE,CONSTRAINT IC-DTIME-ATIMECHECK DTIME < ATIME);

Integrity - general constraints

• applies to an arbitrary combination of columns and tables

• connecting RESERVATIONS for a customer must make sense:

CREATE ASSERTION IC-CONNECTING-FLIGHTS CHECK (NOT EXISTS (SELECT * FROM FLT-SCHEDULE FS1 FS2, RESERVATION R1 R2 WHERE FS1.FLT#=R1.FLT# AND FS2.FLT#=R2.FLT# AND R1.DATE=R2.DATE AND FS1.TO-AIRPORTCODE=FS2.FROM-AIRPORTCODE AND FS1.ATIME+ INTERVAL “30” MINUTE > FS2.DTIME));

Integrity - (not so) general constraints

• not all constraints can be specifiedCREATE TABLE AIRLINE.FLT-WEEKDAY

(FLT# FLIGHTNUMBER NOT NULL,WEEKDAY CHAR(2),.... ));

CREATE TABLE AIRLINE.FLT-INSTANCE(FLT# FLIGHTNUMBER NOT NULL,DATE DATENOT NULL,.... ));

CREATE ASSERTION DATE-WEEKDAY-CHECK (NOT EXISTS (SELECT * FROM FLT-INSTANCE FI, FLT-WEEKDAY FSD WHERE FI.FLT#=FSD.FLT# AND weekday-of(FI.DATE) <> FSD.WEEKDAY));

• weekday-of: DATE WEEKDAY

Transaction Control• atomic, consistent, isolated, durable (ACID)

transactions are supported by:– COMMIT and – ROLLBACK

EXEC SQL OPEN FLT;

WHILE TRUE DO

EXEC SQL FETCH FLT

INTO :FLT#, :AIRLINE, :PRICE;

DO YOUR THING WITH THE DATA;

END-WHILE;

EXEC SQL CLOSE FLT;

QUIT: IF SQLCODE < 0 THEN EXEC SQL ROLLBACK

ELSE EXEC SQL COMMIT;

Authorization• Discretionary Access Control (DAC) is supported by

GRANT and REVOKE:GRANT <privileges> ON <table> TO <users>[WITH GRANT OPTION];

REVOKE [GRANT OPTION FOR] <privileges>ON <table> FROM <users> {RESTRICT | CASCADE};

<privileges>: SELECT, INSERT(X), INSERT,

UPDATE(X), UPDATE, DELETE

CASCADE: revoke cascades through its subtree

RESTRICT: revoke succeeds only if there is no subtree

AuthorizationGRANT INSERT, DELETE ON FLT-SCHEDULE TO U1, U2WITH GRANT OPTION;

GRANT UPDATE(PRICE) ON FLT-SCHEDULETO U3;

REVOKE GRANT OPTION FOR DELETEON FLT-SCHEDULEFROM U2 CASCADE;

REVOKE DELETEON FLT-SCHEDULEFROM U2 CASCADE;

Catalog and Dictionary Facilities

• an INFORMATION_SCHEMA contains the following tables (or rather views) for the CURRENT_USER:– INFORMATION-_SCHEMA_CATALOG_NAME: single-row, single-column table

with the name of the catalog in which the INFORMATION_SCHEMA resides– SCHEMATA created by CURRENT_USER– DOMAINS accessible to CURRENT_USER– TABLES accessible to CURRENT_USER– VIEWS accessible to CURRENT_USER– COLUMNS of tables accessible to CURRENT_USER– TABLE_PRIVILEGES granted by or to CURRENT_USER– COLUMN_PRIVILEGES granted by or to CURRENT_USER– USAGE_PRIVILEGES granted by or to CURRENT_USER– DOMAIN_CONSTRAINTS – TABLE_CONSTRAINTS– REFERENTIAL_CONSTRAINTS– CHECK_CONSTRAINTS– and 18 others ...

Structure of a DBMS• A typical DBMS has a layered

architecture.• The figure does not show the

concurrency control and recovery components.

• Each system has its own variations.• The book shows a somewhat more

detailed version.• You will see the “real deal” in

PostgreSQL.– It’s a pretty full-featured example

• Next class: we will start on this stack, bottom up.

Query Optimizationand Execution

Relational Operators

Files and Access Methods

Buffer Management

Disk Space Management

DB

These layersmust considerconcurrencycontrol andrecovery

TRANSACTION CONTROL

Transaction control• Transaction Concept• Concurrent Execution• Conflict Serializability• Locking• SQL 92 Consistency Levels• Oracle multi version concurrency control• Transaction Log• Crash Recovery

Transaction concept• A sequence of SQL statements that form a

logical unit of work.• Changes the database from one logically

consistent state to another. E.g delete order– DB consistent

• delete from order_header where ono = 123;

– DB inconsistent RI failure now• delete from order_line where ono = 123;

– DB consistent No unmatched FK’s

Transaction properties

• Atomicity– All SQL complete or none are completed.

• Consistency– Programmer responsibility.

• Isolation– Multiple transactions execute in isolation.

• Durability– Successful commits survive system

failures.

Transaction State Diagram

Active

PartiallyCommitted Committed

Failed Rollback

User Action System Action

Transaction states

• Active - state whilst executing

• Partially Committed– SQL commit statement execution started.

• Failed– Normal execution cannot proceed

• Committed - changes guaranteed to stay

• Rollback - changes guaranteed undone

SQL Transaction statements• The first SQL statement starts a

transaction that is terminated by– commit;

• makes all changes permanent

– rollback; • removes all changes

– A DDL command– A system generated rollback– Quitting the session

Commit at regular intervals • This will keep your transactions short

– avoids wasting system resources– prevents loss of work if system

generated rollback occurs– allows use of rollback to undo user

mistakes

Benefits from concurrent execution of transactions• Interleaving of CPU and I/O

– Allows different transactions to execute in parallel.

– Increases system utilisation and throughput.• Allows long and short running transactions to

make progress if not accessing the same data. – Alternative is to serialise and then the short

transactions have long waits.

Database Reads and Writes• Each transaction can be regarded as a

sequence of database reads and writes.• Concurrent transactions only conflict

when accessing the same data.• Local variables are in separate address

space so no conflict.• Computation on local variables only

explains how the write value is formed.

Lost Update ( SQL version )

U1 Time U2

select val = current from grant; t1

val = val + 500;

t2 select val = current from grant;

val = val * 1.1;

update grant set current = val; t3

t4 update grant set current = val;

Database Table grant(current)

@t1 2000 @t3 2500 @t4 2200

Lost Update ( Read/Write)U1 Time U2

Read(current) t1

val = val + 500;

t2 Read(current)

val = val * 1.1;

Write(current) t3

t4 Write(current)


@t1 2000 @t3 2500 @t4 2200

Lost Update Serial Schedule 1U1 Time U2

Read(current) t1

val = val + 500;

Write(current) t2

t3 Read(current)

val = val * 1.1;

t4 Write(current)


@t1 2000 @t2 2500 @t4 2750

Lost Update Serial Schedule 2U1 Time U2


Definitions

• A Schedule is a sequence of reads/writes of a set of concurrent users that preserves the order of the reads/writes of the individual users. E.g. Lost Update(Read/Write) slide

• A serial schedule is a schedule where the reads/writes of each users are executed consecutively without any interleaving of different users reads/writes E.g. Lost Update Serial Schedule 1 slide

Database schedules

U1 U2 U1 U2 U1 U2 U1 U2

read A read A write A write Aread A write A read A write

A

Consider the effect of reversing the interleaved sequence U1 U2.

Only the two read A may be interchanged without effect - rest are conflicting actions.

I.e. the new sequence changes the value read or the value after the write.

Read A Write B are not conflicting as on different values.

Conflict Serialisability

• Schedule S is conflict serialisable if it is conflict equivalent to a serial schedule.

• Two schedules are conflict equivalent if one can be transformed into the other by a series of swaps of non conflicting actions.

Is the Lost Update schedule Conflict Serialisable ?

Conflict Schedule Serial Schedule

U1 U2 U1 U2

read A read A

read A write A

write A read A

write A write A

?

.

A Conflict Serialisable schedule? U1 U2 U1 U2

read A read A

write A write A

read A read B

write A write B

read B read A

write B write A

read B read B

write B write B

?

Conflict Serialisable schedules via locking

• Each transaction must obtain a lock before accessing a data value.– Shared lock granted allows read access.– eXclusive lock granted allows write access.

• Locks are requested from and granted by the Lock Manager process.

• Ensures that any interleaving of concurrent transactions produces a conflict serialisable schedule.

Lock Compatibility MatrixThe lock manager grants lock requests by reference to:

Lock already granted on data A.

Shared eXclusive

Lock requested Shared Yes No

on data A eXclusive No No

If table entry Yes then lock types are compatible and requested lock is granted.

If table entry No then requesting transaction is placed into a wait state until the lock holding transaction(s) have completed and returned the lock.

Rigorous Two Phase Locking• Growing Phase = Active state.

– Locks may be requested but not dropped.• Shrinking Phase = Partially committed or

failed– Locks are dropped but not requested.

• Thus all locks held until end of transaction signalled by rollback or commit

Lost Update Locking solution

T1 Lock Manager T2

request S (A)

read A

request S (A)

read A

request X (A)

request X (A)Waiting for lock

Waiting for lock

granted

granted

Lost Update Locking solution(continued from previous slide)

T1 Lock Manager T2

request X (A)

request X (A)

write A

commit locks dropped

Restarted by programmer.

Transactions have been serialised.

Deadlock - System aborted

Drops S (A) lock

granted

Summary of Lock Requests

T1 T2

S(A)

S(A)

W(A)

W(A)

Wait for grant of lock held by T pointed to

Why use W locks on reads for values about to be updated ?T1 T2


Why use W locks on reads for values about to be updated ?T1 T2

W(A)

W(A)

waiting for W lock

continues avoids deadlocking


Effect of locking

• Lock manager prevents writers and readers accessing the same data item concurrently.

• Thus a schedule containing conflicting actions cannot occur.

• Thus all allowed schedules are conflict serialisable.

Lock starvation

• Writers may not make progress when many Readers are accessing the same data item.

• Prevent by not granting further Shared lock requests when an eXclusive request is waiting.

• Then process waiting requests in FIFO order.

Readers Wait for Writers

T1 T2 T3 T4 Time

read A

read A

write A start wait

read A start wait

commit

commit end wait

commit end wait

Deadlock Detection

• Lock Manager builds wait-for graph for all the transactions in process.

• Deadlock exists if the graph contains a cycle. Detection algorithm run frequently.

• Lost Update Wait-for graph showing cycle.

T1 T2

Effect of locking on database performance• Readers and Writers concurrently

accessing the same data item under Two Phase locking will be forced to serialise I.e wait until exclusive access is possible.

• Users will experience delayed responses.• To improve performance reduced levels

of locking (or consistency of final results) are available. Use with caution.

SQL 92 Consistency Levels- From highest to lowest level• Serializable i.e Two Phase Locking• Repeatable Read.

– phantom reads possible• Read Committed

– read locks dropped after data has been read.– Non repeatable & phantom reads possible.

• Read Uncommitted (Lowest Level)– reads data that still have write locks in place.– dirty read possible

Transaction interactions between T1 and T2• dirty read

– T1 can read uncommitted data from T2

• non repeatable read– T1 re-reads data committed by T2 and

now sees the new data value

• phantom read– T1 re-executes a query and discovers new

data inserted or changed by committed T2

Non Repeatable Read emp id dept_id dept dept_id name

d1 Sales

T1 T2

insert into emp select e1, dept_id from dept

where name = ‘Sales’

update dept set dept_id = ‘d2’

where dept_id = ‘d1’;commit;

insert into emp select e2, dept_id from dept

where name = ‘Sales’; commit;

Non Repeatable Read - locking at Read Committed level

T1 T2

acquire locks X(emp) S(dept)insert ……...drop lock S(dept) * acquire lock X(dept) update ……..; commit; drop lock X(dept)acquire lock S(dept)insert …...; commit;drop locks X(emp) S(dept)

Is this problem possible under Rigorous Two Phase Locking?T1 T2


Inconsistent Analysis / Phantom Read

T1 Stock pno qty T2

select sum(qty) from stock;

update stock set qty = 2 1 2 2

where pno = 2; 2 4 2 4

3 1 1

update stock set qty = 3 4 5 5

where pno = 5; 5 1 3 3

commit; Actual 13 Reported 15

Inconsistent Analysis / Phantom Read locking analysis• T2 For each row acquire lock S(stock) ,

read value, drop lock S(stock)• T1 Allows lock X(stock) to be acquired

on previously read row Completes updates. Drops X(stock) lock.

• T2 acquires S(stock) and so reads both old and new values causing the problem.



Uncommitted / Dirty Readaccount acc_id balance

1 -2000

T1 T2

update account set balance = 10000

where acc_id = 1;select balance from account

where acc_id = 1;

passes credit check rollback;

Uncommitted / Dirty Read locking analysis• Locking at Read Uncommitted Level

which effectively means Lock Compatability Matrix is changed.

• Now Shared and eXclusive locks changes from No to Yes.

• Note Concurrent Writes are still not allowed.



Lock granularity

• Table locking provides least concurrency• Disk Page locking

– Most common level. The number of rows locked depends on row size/page size ratio.

• Row locking provides maximum concurrency– Commonly used for on line transactions – Highest system cost as requires most locks

Transaction Locking and Chrash recovery

Transaction Log - Immediate Update of row data on DiskTransaction Write Ahead Log on Disk

T0 start

update stock set qty = 100 where p# = 1; T0, 2.0, qty, 200, 100

delete from stock where p# = 2; T0, 2.1, p#, 2,, qty, 100,

insert into stock values( 3, 600); T0, 2.2, p#,, 3, qty,, 600

commit; T0 commit

Disk Page 2 offset 0 1 100 200

holds stock values offset 1 2 100

offset 2 3 600

Transaction Log sequence• Write Ahead Protocol means BI,AI and

other log entries are written before the data row on disk is changed.

• If transaction ends with a commit no further action is required.

• If ended with a rollback then system must apply BI to row data to restore the original values on disk.

Transaction Log entriesT0 T1 Database Values

read a read c a b c

a = a - 50 c = c - 100 1000 2000 700

write a write c

read b

b = b + 50

write bWrite out log assuming execute in order T0 T1 and show DatabaseValues assuming Immediate Update of database values.

Transaction log entries

Log Database Values

a b c

1000 2000 700

Transaction log entries

Log Database Values

T0 start a b c

T0 a 1000 950 1000 950 2000 700

T0 b 2000 2050 2050

T0 commit

T1 start

T1 c 700 600 600

T1 commit

Immediate Update of row data in memory update stock set qty = qty * 2Row Data Buffer Page Memory Log Buffer Page

p# qty T1, 9.1, qty,10,20

1 10 20 T1, 9.2, qty, 40, 80

2 40 80 flush on commit/page full before row data buffer flushed

Stock Database Page Disk T1, 9.1, qty,10,20

1 10 T1, 9.2, qty, 40, 80

2 40 Log File Page

Log Based Recovery• System crash loses memory buffer pages.• Committed changes may not be on disk.

– Row data Buffer pages not yet flushed out.

• Non committed changes may be on disk.– Row data Buffer pages already flushed out.

• WAL ensures that Log pages are already on disk ( a different disk from the table data!) before row data buffer pages flushed to disk

Recovery Processing• Read log forwards from the start

putting all transactions found onto one of the lists:– Undo transactions with no commit

terminator– Redo transaction with a commit terminator

• Go backwards from crash point ‘undoing’– Using Before Images

• Go forwards from start ‘redoing’– Using After Images

Database recoveryUsing the data from the Transaction Log slide show the recovered database values. Assume failure occurred

i after write b

ii after write c

Database recovery after write b

Log Database Values

T0 start a b c

T0 a 1000 950 1000 950 2000 700

T0 b 2000 2050 2050

recovery starts by reading from start Undo list Redo list

Database recovery after write b

Log Database Values

T0 start a b c

T0 a 1000 950 1000 950 2000 700

T0 b 2000 2050 2050


T0 No entries as no ends

Undo T0 b 2000 2050 by applying BI 2050 2000

Undo T0 a 1000 950 950 1000

Database values as at start

Database recovery after write c


T0 start

T0 a 1000 950

T0 b 2000 2050 Database Values

T0 commit a b c

T1 start 950 2050 600

T1 c 700 600

Database recovery after write c

recovery starts by reading from start T Undo list T Redo list

T0 start T0

T0 a 1000 950 T1

T0 b 2000 2050 Database Values

T0 end a b c

T1 start ? ? ?

T1 c 700 600 redo 950 redo 2050 undo 700

Why undo backwards & redo forwards through log?Log BI AI Database Values

T0 a 10 20 a 10 20

T0 a 20 10 10

if undo backwards using BI ? 20 10

if redo forwards using AI ? 20 10

Checkpoint

• Flushes all log buffer pages to disk.• Flushes all modified data buffer

pages to disk.• Writes a log record with list of open

transactions.• Recovery takes place from the latest

checkpoint record in the log.– Reduces recovery time

Disk Crash Recovery

• Assumes database dumps are taken periodically.– Flushes all buffers; copies database files;

writes out a dump record.

• Load latest database dump onto disk.• Using log started immediately after

the dump which has been loaded rollforward ‘redoing’ all committed transactions.

The End

Documents

SQL and SQAPL. Data Models A Database models some portion of the real world. Data Model is link between user’s view of the world and bits stored in computer