EDBT 2009 - Provenance for Nested Subqueries

Provenance for Nested Subqueries

Boris Glavic

Database Technology Group

Department of Informatics University of Zurich

glavic@ifi.uzh.ch

Zur Anzeige wird der QuickTime™ Dekompressor „“

benötigt.

Gustavo Alonso

Systems GroupDepartment of Computer

Science ETH Zurich

alonso@inf.ethz.ch

benötigt.

Overview

1. Introduction2. The Provenance of Subqueries3. Experimental Results4. Conclusion

benötigt.

1. Introduction

Which input data item(s) influenced which output data item(s)? Granularity

Tuple Attribute Value ...

Contribution semantics Influence (Lineage / Why) Copy (Where) ...

benötigt.

1. Introduction

Most application domains that benefit from provenance use complex queries Subqueries

Correlated Nested

Not supported by existing systems Semantics not clear Complex computation

benötigt.

1. Introduction

Steps to solve this problem1. Establish sound semantics for

provenance of subqueries2. Algorithms for subquery provenance

computation3. Integrate algorithms into a Provenance

Management system (Perm)

benötigt.

1. Introduction

Steps to solve this problem1. Establish sound semantics for

provenance of subqueries2. Algorithms for subquery provenance

computation3. Integrate algorithms into a Provenance

Management system (Perm)

benötigt.

1. Introduction Definition of contribution semantics

Why/Influence-provenance Introduced in [Cui, Widom ICDE ‘00] Provenance represented as list of

subsets of the input relations Defined for a single algebra operator

and a single result tuple

benötigt.

1. Introduction Definition 1: For a single algebra

operator op with input relations T1, ... , Tn a list (T1*, ... ,Tn*) of maximal subsets of the input relation is the provenance of a tuple t from the result of op iff:

u op(T1*, ..., Tn*) = t

u For all i and t* with t* in Ti*:op(T1*, ... Ti-1*, t* , Ti+1*, ... ,Tn*) !=

benötigt.

1. Introduction Perm

Provenance Extension of the Relational Model

Provenance Management System (PMS) “Pure” Relational representation of

provenance Provenance computation trough

algebraic query rewrite Implemented as extension of

PostgreSQL

benötigt.

1. Introduction

Provenance representation

OriginalAttributes

Relation 1 Attributes

Relation n Attributes

OriginalResult

benötigt.

1. Introduction

Provenance representation

OriginalAttributes

Relation R Attributes

Relation S Attributes

OriginalResult

t 1 r1

t 1 r2

benötigt.

1. Introduction Provenance Computation though

query rewrite: Given query q generate query q+ that

computes the provenance of q Representation as defined before

Rewrites operate on the algebraic representation of a query Rewrite rules for each operator op that

transform op into a algebra statement that propagates the provenance

benötigt.

1. Introduction

Rewrite rules example:SELECT agg, GFROM TGROUP BY G

SELECT agg, G, prov(T)FROM

(SELECT agg, G FROM T GROUP BY G) AS agg,LEFT OUTER JOIN(SELECT G AS G’, prov(T) FROM T+) AS provON G = G’

benötigt.

1. Introduction

Rewrite rules example:SELECT sum(revenue) AS sum, shopFROM salesGROUP BY shop

shop month revenue

Migros Jan 100

Migros Feb 10

Migros Mar 10

Coop Jan 25

Coop Feb 25

salessum shop

120 Migros

50 Coop

result

benötigt.

1. Introduction

SELECT sum, shop, pShop, pMonth, pRevenueFROM

(SELECT sum(revenue) AS sum, shop FROM sales GROUP BY shop) AS aggLEFT OUTER JOIN(SELECT shop AS shop’, pShop, pMonth, pRevenue FROM sales ) AS provON shop = shop’

sum shop pShop pMonth pRevenue

120 Migros Migros Jan 100

120 Migros Migros Feb 10

120 Migros Migros Mar 10

50 Coop Coop Jan 25

50 Coop Coop Feb 25

benötigt.

Overview

benötigt.

2. The Provenance of Subqueries

Sublinks Subqueries in e.g. SELECT-clause

Correlated References outside attributes

Nested Sublink that contains sublinks

σ a IN σ (b=3) (S)(R)

σ a IN σ (b=a ) (S)(R)

σ a IN σ (b = ANY (T )) (S)(R)

benötigt.

What is the provenance of a sublink according to Definition 1? Sublinks can be used in different

contexts Selection Projection ...

Sublink either Produces exactly one value Or produces a boolean value

benötigt.

Single uncorrelated ANY-sublinks in selection conditions

For other Types of sublinks Correlated sublinks Nested sublinks

benötigt.

For other Types of sublinks Correlated sublinks Nested sublinks

READ THE PAPER!

benötigt.

Single uncorrelated ANY-sublinks in selection conditions The result of the sublink query is fixed For a given input tuple t the sublink

condition is either true or false

σ a =ANY σ (b=3) (S)(R)

benötigt.

Some terminology The query of a sublink

The conditional expression of a sublink

q =σ a =ANY Πb (S)(R)

Πb(S)

a = ANY Πb (S)

Tsub€

benötigt.

Sublink condition can play different roles in a condition C of a selection (for one input tuple t): Reqtrue: the selection condition is true, iff is true Reqfalse: the selection condition is true,

iff is false

Ind: the selection condition is true indepedent of the result of €

benötigt.

Some more terminology All tuples from the sublink query that

fulfill the “unquantified” sublink condition

All tuples from the sublink query that do not fulfill the “unquantified” sublink condition€

Tsubtrue(t)

Tsubfalse(t)

Csub = (a = ANY σ b=3(S))

Csub° = (a = b)

benötigt.

Back to ANY-sublinks in selections Proposition:

Tsub*(t) =

Tsubtrue(t) reqtrue

Tsub reqfalse, ind

⎧ ⎨ ⎩

benötigt.

Result

Compute provenance for

t = (1)

Example:

benötigt.

Tsub = Πb (S)

Tsubtrue(t) = {(1)}

is reqtrue

Tsub* =Tsub

Csub° = (a = b)

benötigt.

Tsubtrue(t) = {(1)}

Csub° = (a = b)

t = (1)

benötigt.

R* Tsub*b

Result

t = (1)

benötigt.

Definition 1 is ambiguous for queries with more than one sublink!

q =σ C1∨C2(U )

C1 = (a =ANY R)

C2 = (a > ALL S)

t = (5)

Resulta

benötigt.

Definition 1 is ambiguous for queries with more than one sublink!

q =σ C1∨C2(U )

C1 = (a =ANY R)

C2 = (a > ALL S)

t = (5)

Resulta

benötigt.

S*R*€

q =σ C1∨C2(U )

C1 = (a =ANY R)

C2 = (a > ALL S)

t = (5)

U*Solution 1 Solution 2

benötigt.

S*R*€

q =σ C1∨C2(U )

C1 = (a =ANY R)

C2 = (a > ALL S)

t = (5)

benötigt.

S*R*€

q =σ C1∨C2(U )

C1 = (a =ANY R)

C2 = (a > ALL S)

t = (5)

benötigt.

Reasons for this ambiguity: The definition requires the provenance

to produce the same result But not to produce the same results for

the sublinks

-> Definition 1 produces false positives

benötigt.

Solution: Extend definition 1 Add a third condition: For each sublink:

If computed for one result tuple t one tuple from the provenance of the sublink

Produces same sublink result as in the original query

benötigt.

S*R*€

q =σ C1∨C2(U )

C1 = (a =ANY R)

C2 = (a > ALL S)

t = (5)

benötigt.

How to compute the provenance according to the extended definition?

Use query rewrite Generic strategy (Gen) Specialized strategies

Use un-nesting Check: does not change the provenance

benötigt.

Gen-strategy For queries we cannot un-nest

1. Join original query with all possible provenance tuples (base relations)

2. Rewrite the sublink query3. Introduce additional correlation to

simulate a join between 1) and 2)

benötigt.

Overview

benötigt.

3. Experimental Results TPC-H benchmark (10 MB size)

benötigt.

3. Experimental Results TPC-H benchmark (1 GB size)

benötigt.

Overview

benötigt.

4. Conclusion

Definition 1 fails in the presence of sublinks Can be extended to deal with sublinks

Provenance computation for sublinks By using query rewrites Implemented in the Perm

Future Work Physical provenance-aware operators

benötigt.

Questions

EDBT 2009 - Provenance for Nested Subqueries

Science

Joins And Subqueries

SQL - Subqueries and Schema · 2011. 7. 22. · Subqueries • Subquery – one SELECT statement inside another • Used in the WHERE clause • Subqueries can return many rows. •

Subqueries and Joins

Personal Information Management Systems - EDBT/ICDT'15 Tutorial

Chapter 7 Subqueries

Subqueries For Superheroes

Shared slides-edbt-keynote-03-19-13

SUBQUERIES IN SQL - web.cecs.pdx.eduweb.cecs.pdx.edu/~lmd/cs386/subqueries-in-sql.pdf · SUBQUERIES IN SQL Week 4 January 30, 2013 CS 386/586 Winter 2013 Lois Delcambre CS386/586

SAFE EDBT 2011

How to code subqueries · 1.Code SELECT statements that require subqueries. 2.Code SELECT statements that use common table expressions (CTEs) to define the subqueries. Knowledge 1.Describe

Database · B.E., MCA, M.Phil., Head of Computer Science and Information Technology Department, ... Subqueries – Subqueries with IN, EXISTS, Subqueries Restrictions, Nested Subqueries,

Subqueries, Backups, Users and Privileges

Apollo: Learning Query Correlations for Predictive Caching ...openproceedings.org/2018/conf/edbt/paper-100.pdf · International Conference on Extending Database Technology (EDBT),

SQL Unit 9 Correlated Subqueries

Execution Strategies for SQL Subqueries

Similarity in Wikipedia Articles (EDBT Summer School)

Scalar subqueries

Subquery I. - Eötvös Loránd University · 2017-02-28 · Guidelines for Using Subqueries –Enclose subqueries in parentheses. –Place subqueries on the right side of the comparison

Chapter Subqueries important

EDBT 2015: Summer School Overview