Sekolah Tinggi Ilmu Statistik (STIS). Lecture 7 Dr. Said Mirza Pahlevi, M.Eng. 2

Sekolah Tinggi Ilmu Statistik (STIS)

Advanced Database Systems

Dr. Said Mirza Pahlevi, M.Eng. 2

Query Processing 1Lecture 7

Today's LectureIntroduction to query processingPhases of Query ProcessingQuery decompositionHeuristical Approach to Query Optimization

IntroductionFirst Topic

Sekolah Tinggi Ilmu Statistik (STIS) 5

IntroductionIn network and hierarchical DBMSs, low-level procedural

query language is generally embedded in high-level programming language.

Programmer’s responsibility to select most appropriate execution strategy.

With declarative languages such as SQL, user specifies what data is required rather than how it is to be retrieved.

Relieves user of knowing what constitutes good execution strategy.

Dr. Said Mirza Pahlevi, M.Eng.

IntroductionAlso gives DBMS more control over system

performance.

Two main techniques for query optimization: Comparing different strategies based on relative costs, and

selecting one that minimizes resource usage. Heuristic rules that order operations in a query;

Disk access tends to be dominant cost in query processing for centralized DBMS.

Query Processing Activities involved in: parsing, validating,

optimizing, and executing a query

Aims of QP:

Transform query written in high-level language (e.g. SQL), into correct and efficient execution strategy expressed in low-level language (implementing relational algebra/RA);

Execute strategy to retrieve required data.

Query OptimizationActivity of choosing an efficient execution strategy for

processing query.

As there are many equivalent transformations of same high-level query, aim of QO is to choose one that minimizes resource usage.

Generally, reduce total execution time of query.

May also reduce response time of query.

Problem computationally intractable with large number of relations, so strategy adopted is reduced to finding near optimum solution.

Example: Different StrategiesFind all Managers who work at a London branch.

SELECT *FROM Staff s, Branch bWHERE s.branchNo = b.branchNo AND (s.position = ‘Manager’ AND b.city = ‘London’);

Example: Different StrategiesThree equivalent RA queries are:

1. (position='Manager') (city='London') (Staff.branchNo=Branch.branchNo) (Staff X Branch)

2. (position='Manager') (city='London')(Staff Staff.branchNo=Branch.branchNo Branch)

3. (position='Manager'(Staff)) Staff.branchNo=Branch.branchNo (city='London' (Branch))

SELECT * FROM Staff s, Branch bWHERE s.branchNo = b.branchNo AND (s.position = ‘Manager’ AND b.city = ‘London’);

Example: Different StrategiesAssume:

1000 tuples in Staff; 50 tuples in Branch; 50 Managers; 5 London branches; no indexes or sort keys; results of any intermediate operations stored on disk; cost of the final write is ignored; tuples are accessed one at a time.

Disk Access Cost(position='Manager') (city='London') (Staff.branchNo=Branch.branchNo) (Staff X

Branch) Cost: (1000 + 50) + 2*(1000 * 50) = 101050

(position='Manager') (city='London')(Staff Staff.branchNo=Branch.branchNo Branch) Cost: 2*1000 + (1000 + 50) = 3050

(position='Manager'(Staff)) Staff.branchNo=Branch.branchNo (city='London' (Branch)) Cost: 1000 + 2*50 + 50 + 2*5 = 1160

Cartesian product & join operations are much more expensive than selection, and third option significantly reduces size of relations being joined together.

1000 tuples in Staff; 50 tuples in Branch;50 Managers; 5 London branches;

Phases of Query Processing

Second Topic

Phases of Query ProcessingQP has four main phases:

Decomposition (consisting of parsing and validation); Optimization; Code generation; Execution.

Phases of Query Processing

Dynamic versus Static OptimizationThe first three phases of QP can be carried out:

Dynamically every time query is run; Statically when query is first submitted.

Advantages of dynamic QO arise from fact that information is up to date.

Disadvantages are that performance of query is affected, time may limit finding optimum strategy.

Dynamic versus Static OptimizationAdvantages of static QO are removal of runtime

overhead, and more time to find optimum strategy.

Disadvantages arise from fact that chosen execution strategy may no longer be optimal when query is run.

Could use a hybrid approach to overcome this. If system detects that the database statistics have changed

significantly then use the dynamic QO

Query DecompositionThird Topic

Query DecompositionAims are to transform high-level query into RA

query and check that query is syntactically and semantically correct.

Typical stages are:1. analysis, 2. normalization, 3. semantic analysis, 4. simplification, 5. query restructuring.

1. AnalysisAnalyze query lexically and syntactically using

compiler techniques.

Verify relations and attributes exist.

Verify operations are appropriate for object type.

Analysis - ExampleSELECT staff_noFROM StaffWHERE position > 10;

This query would be rejected on two grounds:

staff_no is not defined for Staff relation (should be staffNo).

Comparison ‘>10’ is incompatible with type position, which is variable character string.

AnalysisFinally, query transformed into some internal

representation more suitable for processing.

Some kind of query tree is typically chosen, constructed as follows:

Leaf node created for each base relation.

Non-leaf node created for each intermediate relation produced by RA operation.

Root of tree represents query result.

Sequence is directed from leaves to root.Dr. Said Mirza Pahlevi, M.Eng.

Example: R.A.T.

(s.position=‘Manager’(Staff)) s.branchNo= b.branchNo(b.city=‘London’(Branch))

2. NormalizationConverts query into a normalized form for easier

manipulation.

Predicate can be converted into one of two forms:

Conjunctive normal form: (position = 'Manager' salary > 20000) (branchNo = 'B003')

Disjunctive normal form: (position = 'Manager' branchNo = 'B003' ) (salary > 20000

branchNo = 'B003')

3. Semantic AnalysisRejects normalized queries that are incorrectly

formulated or contradictory.

Query is incorrectly formulated if components do not contribute to generation of result. If some join specifications are missing

Query is contradictory if its predicate cannot be satisfied by any tuple. (position = 'Manager' position = Assistant' )

Algorithms to determine correctness exist only for queries that do not contain disjunction and negation.

Semantic AnalysisFor queries we can construct:

A relation connection graph. Normalized attribute connection graph.

Relation connection graph Create node for each relation and node for result. Create edges between two nodes that represent a join, and edges between nodes that represent projection.

If not connected, query is incorrectly formulated.

Example: Checking Semantic CorrectnessSELECT p.propertyNo, p.streetFROM Client c, Viewing v, PropertyForRent pWHERE c.clientNo = v.clientNo AND

c.maxRent >= 500 AND c.prefType = ‘Flat’ AND p.ownerNo = ‘CO93’;

Relation connection graph not fully connected, so query is not correctly formulated.

Have omitted the join condition (v.propertyNo = p.propertyNo) .

c.maxRent >= 500c.prefType = ‘Flat’ p.ownerNo = ‘CO93’

c.clientNo = v.clientNo

4. SimplificationObjectives of this stage

Detects redundant qualifications, Eliminates common sub-expressions, Transforms query to semantically equivalent but more easily

and efficiently computed form.

Typically, access restrictions, view definitions, and integrity constraints are considered.

Assuming user has appropriate access privileges, first apply well-known idempotency rules of boolean algebra.

Idempotency Rules

p (p) p p (p) pp false false p false pp true p p true truep ( ~ p) false p (~p) truep (p q) p p (p q) p

Example: Simplification

no result

Heuristical Approach to Query Optimization

Fourth Topic

Transformation Rules: Rule 1 Conjunctive Selection operations can cascade

into individual Selection operations (and vice versa).

pqr(R) = p(q(r(R)))

Sometimes referred to as cascade of Selection.

branchNo='B003' salary>15000(Staff) = branchNo='B003'(salary>15000(Staff))

Transformation Rules: Rule 2 Commutativity of Selection.

p(q(R)) = q(p(R))

branchNo='B003'(salary>15000(Staff)) = salary>15000(branchNo='B003'(Staff))

Transformation Rules: Rule 3 In a sequence of Projection operations, only the

last in the sequence is required.

LM … N(R) = L(R)

lNamebranchNo, lName(Staff) = lName(Staff)

Transformation Rules: Rule 4 Commutativity of Selection and Projection.

If predicate p involves only attributes in projection list, Selection and Projection operations commute:

A1, …, Am(p(R)) = p(A1, …, Am(R))

where p {A1, A2, …, Am}

fName, lName(lName='Beech'(Staff)) = lName='Beech'(fName,lName(Staff))

Transformation Rules: Rule 5 Commutativity of Theta join (and Cartesian

product).

R p S = S p R R X S = S X R

Rule also applies to Equijoin and Natural join. For example:

Staff staff.branchNo=branch.branchNo Branch =

Branch staff.branchNo=branch.branchNo Staff

Transformation Rules: Rule 6 Commutativity of Selection and Theta join (or

Cartesian product).

If selection predicate involves only attributes of one of join relations, Selection and Join (or Cartesian product) operations commute:

p(R r S) = (p(R)) r S

p(R X S) = (p(R)) X S

where p {A1, A2, …, An}

Rule 6 (cont’) If selection predicate is conjunctive predicate having form (p

q), where p only involves attributes of R, and q only attributes of S, Selection and Theta join operations commute as:

p q(R r S) = (p(R)) r(q(S))

p q(R X S) = (p(R)) X (q(S))

Rule 6 (cont’) For example:

position='Manager' city='London'(Staff Staff.branchNo=Branch.branchNo Branch) =

(position='Manager'(Staff)) Staff.branchNo=Branch.branchNo (city='London'

(Branch))

Transformation Rules: Rule 7Commutativity of Projection and Theta join (or

Cartesian product).

If projection list is of form L = L1 L2, where L1 only has attributes of R, and L2 only has attributes of S, provided join condition only contains attributes of L, Projection and Theta join commute:

L1L2(R r S) = (L1(R)) r(L2(S))

Rule 7 (cont’)If join condition contains additional attributes not

in L (M = M1 M2 where M1 only has attributes of R, and M2 only has attributes of S), a final projection operation is required:

L1L2(R r S) = L1L2( (L1M1(R)) r (L2M2(S)))

Rule 7 (cont’)For example:

position,city,branchNo(Staff Staff.branchNo=Branch.branchNo Branch) =

(position, branchNo(Staff)) Staff.branchNo=Branch.branchNo (city, branchNo (Branch))

and using the latter rule:

position, city(Staff Staff.branchNo=Branch.branchNo Branch) =

position, city ((position, branchNo(Staff)) Staff.branchNo=Branch.branchNo

( city, branchNo (Branch)))

Transformation Rules: Rule 8Commutativity of Union and Intersection (but not

set difference).

R S = S R R S = S R

Transformation Rules: Rule 9Commutativity of Selection and set operations

(Union, Intersection, and Set difference).

p(R S) = p(S) p(R) p(R S) = p(S) p(R) p(R – S) = p(S) – p(R)

Transformation Rules: Rule 10Commutativity of Projection and Union.

L(R S) = L(S) L(R)

Transformation Rules: Rule 11Associativity of Theta join (and Cartesian

product).

Cartesian product and Natural join are always associative: (R S) T = R (S T) (R X S) X T = R X (S X T)

If join condition q involves attributes only from S and T, then Theta join is associative: (R p S) q r T = R p r (S q T)

Rule 11 (cont’)For example:

(Staff Staff.staffNo=PropertyForRent.staffNo PropertyForRent)

ownerNo=Owner.ownerNo staff.lName=Owner.lName Owner =

Staff staff.staffNo=PropertyForRent.staffNo staff.lName=lName (PropertyForRent

ownerNo Owner)

Cannot simply move the bracket!

Transformation Rules: Rule 12Associativity of Union and Intersection (but not

Set difference).

(R S) T = S (R T) (R S) T = S (R T)

Example: Use of Transformation RulesFor prospective renters of flats, find properties that

match requirements and owned by CO93 Assumption: there are fewer properties owned by owner CO93

than prospective renters

SELECT p.propertyNo, p.streetFROM Client c, Viewing v, PropertyForRent pWHERE c.prefType = ‘Flat’ AND

c.clientNo = v.clientNo AND v.propertyNo = p.propertyNo ANDc.maxRent >= p.rent AND c.prefType = p.type AND p.ownerNo = ‘CO93’;

ExampleSELECT p.propertyNo, p.streetFROM Client c, Viewing v, PropertyForRent pWHERE

c.prefType = ‘Flat’ AND c.clientNo = v.clientNo AND v.propertyNo = p.propertyNo ANDc.maxRent >= p.rent AND c.prefType = p.type AND p.ownerNo = ‘CO93’;

Example

Rule 1, 2, 6

1. Split conjuction of selections into individual selection (rule 1)

2. Reorder selection and commute the selection and cartesian products (rule 2 and 6)

Example

Cartesian Product +

Selection = Join

Example

Rule 11: reorder the equijoin

10050 1004

Assumption: there are fewer properties owned by owner CO93 than prospective renters

Example

Rule 4 & 7

Move projections down to past the Equijoints and create new Projection as required

Example

p.type=‘flat’

Heuristical Processing Strategies Perform Selection operations as early as possible.

Keep predicates on same relation together.

Combine Cartesian product with subsequent Selection whose predicate represents join condition into a Join operation.

Use associativity of binary operations to rearrange leaf nodes so leaf nodes with most restrictive Selection operations executed first.

Heuristical Processing StrategiesPerform Projection as early as possible.

Keep projection attributes on same relation together.

Compute common expressions once. If common expression appears more than once, and result not

too large, store result and reuse it when required. Useful when querying views, as same expression is used to

construct view each time.

Quiz 1Buatlah R.A.T dari query berikut dan kemudian

lakukan optimisasi dengan menggunakan pendekatan Heuristic! Tuliskan: SQL query Relational algebra Initial R.A.T Optimized R.A.T

Tampilkan firstname dan nomor branch dari staff yang bekerja di branch London dimana salary mereka lebih besar dari 10000!

AnswerSQL Query

SELECT fName, branchNo FROM Branch, Staff WHERE Branch.branchNo=Staff.branchNo AND

salary>10000 AND city='London'

Relational Algebra

fName, branchNo (city=‘London’ salary>10000

(Staff Staff.branchNo=Branch.branchNo Branch))

Answer

staff branch

S.branchNo=B.branchNo

city=‘London’ salary>10000

fName, branchNo

staff branch

S.branchNo=B.branchNo

salary>10000

fName, branchNo

city=‘London’

fName, branchNo branchNo

Tampilkan propertyNo beserta fname dari ownernya, dimana property tersebut berada pada kota Glasgow, memiliki kamar lebih dari 3 dan nomor telpon ownernya berawalan 0141!

AnswerSQL Query

SELECT propertyNo, fnameFROM PropertyForRent p, PrivateOwner oWHERE p.ownerNo=o.ownerNo AND city='Glasgow' AND

rooms > 3 AND telNo LIKE '0141%'

Relational Algebra

propertyNo, fName(city=‘Glasgow’ rooms>3 telNo like ‘0141%’

(PropertyForRent p.ownerNo=o.ownerNo PrivateOwner))

Answer

PropertyForRent PrivateOwner

p.ownerNo=o.ownerNo

city=‘Glasgow’ rooms > 3

telNo like ‘0141%’

propertyNo, fName

PropertyForRent PrivateOwner

p.ownerNo=o.ownerNo

city=‘Glasgow’ rooms > 3

propertyNo, fName

telNo like ‘0141%’

propertyNo, ownerNo fName, ownerNo

Tampilkan nomor branch dan nomor property yang dikelola oleh branch tersebut dengan persyaratan bahwa jenis property adalah Flat dan jumlah kamar 4.

Tampilkan nomor branch dan nomor property yang dikelola oleh branch tersebut dengan persyaratan bahwa jenis property adalah Flat dan jumlah kamarnya 4!

AnswerSQL Query

SELECT branchNo, propertyNo FROM Branch as b, PropertyForRent as pWHERE b.branchNo=p.branchNo AND

p.type=‘Flat’ AND p.rooms=4

Relational Algebra

branchNo, propertyNo (p.type=‘Flat’ p.rooms=4

(Branch b.branchNo=p.branchNo PropertyForRent))

Answer

Branch PropertyForRent

s.branchNo=b.branchNo

type=‘Flat’ rooms=4

branchNo, propertyNo

Branch PropertyForRent

s.branchNo=b.branchNo

propertyNo, s.branchNo

type=‘Flat’ rooms=4

propertyNo, branchNo branchNo

branchNo, propertyNo (p.type=‘Flat’ p.rooms=4

(Branch b.branchNo=p.branchNo PropertyForRent))

Quiz 4Buatlah R.A.T dari query berikut dan kemudian lakukan

optimisasi dengan menggunakan pendekatan Heuristic! Tuliskan: SQL query Relational algebra Initial R.A.T Optimized R.A.T

Tampilkan firstname dan lastname dari staff beserta nomor property dan nomor owner dari property yang dikelola oleh staff tersebut, dengan persyaratan bahwa posisi staff adalah Assistent dan jenis Propertynya adalah Flat!

AnswerSQL Query

SELECT fname, lname, propertyNo, ownerNoFROM Staff s, PropertyForRent pWHERE s.staffNo=p.staffNo AND s.position=‘Assistent' AND

p.type = ‘Flat’

Relational Algebra

fname, lname, propertyNo, ownerNo(s.position=‘Assistent’ p.type=‘Flat’

(Staff s.staffNo=p.staffNo PropertyForRent))

Tampilkan firstname dan lastname dari staff beserta nomor property dan nomor owner dari property yang dikelola oleh staff tersebut, dengan persyaratan bahwa posisi staff adalah Assistent dan jenis Propertynya adalah Flat!

Answer

PropertyForRent Staff

s.staffNo=p.staffNo

s.position=‘Assistent’

p.type=‘Flat’

fname,lname, propertyNo, ownerNo

PropertyForRent Staff

p.staffNo=s.staffNo

p.type=‘Flat’

fname,lname, propertyNo, ownerNo

s.position=‘Assistant’

propertyNo, staffNo, ownerNo fName, lname, staffNo

fname, lname, propertyNo, ownerNo(s.position=‘Assistent’ p.type=‘Flat’

(Staff s.staffNo=p.staffNo PropertyForRent))

Sekolah Tinggi Ilmu Statistik (STIS). Lecture 7 Dr. Said Mirza Pahlevi, M.Eng. 2

Documents

STScI STIS Coron - NASA...STIS Coronagraphic Imaging ! Main function of STIS occulting wedge structure is to prevent bleeding over region of interest in image ! Mainly used to image

STIS Time-Tag Analysis Guide · 2020. 5. 12. · Instrument Science Report STIS 2000-02 STIS Time-Tag Analysis Guide Ilana Dashevsky, Kailash Sahu, Ed Smith September 6, 2000 ABSTRACT

Background information for users of STIS

M.Eng. (Metallurgical Engineering).pdf

Viral STIs

Road networks (m.eng civil)

Overview of STIS Repair Plans

Reducing Risks for Pregnancy and STIs

12 Jurnal STIS, Desember 2014

Yasser Mahgoub, B.Eng., M.Eng

Common STIs

CAPI-STIS: Integrated Digitalized Data Collection …...CAPI-STIS: Integrated Digitalized Data Collection Software System for Official Statistics Survey Takdir STIS Polytechnic of

The Secret Epidemic: STIs on Campus

Spectral Characterization of calibration lamps: STIS

STIS Status

VIVA M.Eng. 2006 UTM

STIs, bacterial vaginosis & HIV in pregnancy

Finite Element Analysis of Radiofrequency Ablation Abirvab Deb- BME M.Eng ‘14 Brice Lekavich- BME M.Eng ‘14 Cristian Vilorio- BME M.Eng ‘14

M.eng Dissertation- Okolo GN

M.Eng Final Report(7) - Cornell University